Fri Apr 13 14:50:51 2007 Google Inc. <opensource@google.com>

	* google-perftools: version 0.90 release
	* (As the version-number jump hints, this is a major new release:
	  almost every piece of functionality was rewritten.  I can't do
	  justice to all the changes, but will concentrate on highlights.)
	*** USER-VISIBLE CHANGES:
	* Ability to "release" unused memory added to tcmalloc
	* Exposed more tweaking knobs via environment variables (see docs)
	* pprof tries harder to map addresses to functions
	* tcmalloc_minimal compiles and runs on FreeBSD 6.0 and Solaris 10
	*** INTERNAL CHANGES:
	* Much better 64-bit support
	* Better multiple-processor support (e.g. multicore contention tweaks)
	* Support for recent kernel ABI changes (e.g. new arg to mremap)
	* Addition of spinlocks to tcmalloc to reduce contention cost
	* Speed up tcmalloc by using __thread on systems that support TLS
	* Total redesign of heap-checker to improve liveness checking
	* More portable stack-frame analysis -- no more hard-coded constants!
	* Disentangled heap-profiler code and heap-checker code
	* Several new unittests to test, e.g., thread-contention costs
	* Lots of small (but important!) bug fixes: e.g., fixing GetPC on amd64
	*** KNOWN PROBLEMS:
	* CPU-profiling may crash on x86_64 (64-bit) systems.  See the README
	* Profiling/heap-checking may deadlock on x86_64 systems.  See README


git-svn-id: http://gperftools.googlecode.com/svn/trunk@28 6b5cf1ce-ec42-a296-1ba9-69fdba395a50

6 files changed, 241 insertions, 380 deletions

diff --git a/src/google/heap-checker.h b/src/google/heap-checker.h
index f888ae0..9abe233 100644
--- a/src/google/heap-checker.h
+++ b/src/google/heap-checker.h
@@ -30,244 +30,68 @@
 // ---
 // Author: Maxim Lifantsev (with design ideas by Sanjay Ghemawat)
 //
-// Heap memory leak checker (utilizes heap-profiler and pprof).
 //
-// TODO(jandrews): rewrite this documentation
-// HeapLeakChecker, a heap memory leak checking class.
+// Module for detecing heap (memory) leaks.
 //
-// Verifies that there are no memory leaks between its
-// construction and call to its *NoLeaks() or *SameHeap() member.
+// For full(er) information, see doc/heap-checker.html
 //
-// It will dump two profiles at these two events
-// (named <prefix>.<name>-beg.heap and <prefix>.<name>-end.heap
-//  where <prefix> is determined automatically to some temporary location
-//  and <name> is given in the HeapLeakChecker's constructor)
-// and will return false in case the amount of in-use memory
-// is more at the time of *NoLeaks() call than
-// (or respectively differs at the time of *SameHeap())
-// from what it was at the time of our construction.
-// It will also in this case print a message on how to process the dumped
-// profiles to locate leaks.
+// This module can be linked into programs with
+// no slowdown caused by this unless you activate the leak-checker:
 //
-// GUIDELINE: In addition to the local heap leak checking between two arbitrary
-// points in program's execution via an explicit HeapLeakChecker object,
-// we provide a way for overall whole-program heap leak checking,
-// which is WHAT ONE SHOULD NORMALLY USE.
+//    1. Set the environment variable HEAPCHEK to _type_ before
+//       running the program.
 //
-// Currently supported heap-check types, from less strict to more
-// strict, are:
-//     "minimal", "normal", "strict", "draconian"
+// _type_ is usually "normal" but can also be "minimal", "strict", or
+// "draconian".  (See the html file for other options, like 'local'.)
 //
-// There are also two more types: "as-is" and "local".
-//
-// GUIDELINE CONT.: Depending on the value of the HEAPCHECK variable
-// -- as well as other flags of this module -- different modifications
-// of leak checking between different points in program's execution
-// take place.  The "as-is" value leaves control to the other flags of
-// this module.  The "local" value does not start whole-program heap
-// leak checking but activates all the machinery needed for local heap
-// leak checking via explicitly created HeapLeakChecker objects.
-//
-// For the case of "normal" everything from before execution of all
-// global variable constructors to normal program exit (namely after
-// main() returns and after all REGISTER_HEAPCHECK_CLEANUP's are
-// executed, but before any global variable destructors are executed)
-// is checked for the absence of heap memory leaks.
-//
-// NOTE: For all but "draconian" whole-program leak check we also
-// ignore all heap objects reachable (at the time of the check)
-// from any global variable or any live thread stack variable
-// or from any object identified by a HeapLeakChecker::IgnoreObject() call.
-//
-// CAVEAT: We do a liveness flood by traversing pointers to heap objects
-// starting from some initial memory regions we know to potentially
-// contain live pointer data.
-// -- It might potentially not find some (global)
-//    live data region to start the flood from,
-//    but we consider such cases to be our bugs to fix.
-// The liveness flood approach although not being very portable
-// and 100% exact works in most cases (see below)
-// and saves us from writing a lot of explicit clean up code
-// and other hassles when dealing with thread data.
-//
-// The liveness flood simply attempts to treat any properly aligned
-// byte sequences as pointers to heap objects and thinks that
-// it found a good pointer simply when the current heap memory map
-// contains an object with the address whose byte representation we found.
-// As a result of this simple approach, it's unlikely but very possible
-// for the flood to be inexact and occasionally result in leaked objects
-// being erroneously determined to be live.
-// Numerous reasons can lead to this, e.g.:
-// - Random bit patters can happen to look
-//   like pointers to leaked heap objects.
-// - Stale pointer data not corresponding to any live program variables
-//   can be still present in memory regions (e.g. thread stacks --see below)
-//   that we consider to be live.
-// - Stale pointer data that we did not clear can point
-//   to a now leaked heap object simply because the heap object
-//   address got reused by the memory allocator, e.g.
-//     char* p = new char[1];
-//     delete p;
-//     new char[1];  // this is leaked but p might be pointing to it
-//
-// The implications of these imprecisions of the liveness flood
-// are as follows:
-// - For any heap leak check we might miss some memory leaks.
-// - For a whole-program leak check, a leak report *does* always
-//   correspond to a real leak (unless of course the heap-checker has a bug).
-//   This is because in this case we start with an empty heap profile,
-//   so there's never an issue of it saying that some heap objects
-//   are live when they are not.
-// - For local leak checks, a leak report can be a partial false positive
-//   in the sense that the reported leaks might have actually occurred
-//   before this local leak check was started.
-//   Here's an example scenario: When we start a local check
-//   heap profile snapshot mistakenly says that some previously
-//   leaked objects are live.
-//   When we end the local check the heap profile snapshot now correctly
-//   determines that those objects are unreachable and reports them as leaks
-//   for this leak check, whereas they had been already leaked before
-//   this leak check has started.
-//
-// THREADS and heap leak checking: At the time of HeapLeakChecker's
-// construction and during *NoLeaks()/*SameHeap() calls we grab a lock so that
-// heap activity in other threads is paused for the time
-// we are recording or analyzing the state of the heap.
-// For any heap leak check it is possible to have
-// other threads active and working with the heap
-// when we make the HeapLeakChecker object or do its leak checking
-// provided all these threads are discoverable with the implemetation
-// of thread_lister.h (e.g. are linux pthreads).
-// In this case leak checking should deterministically work fine.
-//
-// CAVEAT: Thread stack data ignoring (via thread_lister.h)
-// does not work if the program is running under gdb, probably becauce the
-// ptrace functionality needed for thread_lister is already hooked to by gdb.
-//
-// As mentioned above thread stack liveness determination
-// might miss-classify objects that very recently became unreachable (leaked)
-// as reachable in the cases when the values of the pointers
-// to the now unreachable objects are still present in the active stack frames,
-// while the pointers actually no longer correspond to any live program
-// variables.
-// For this reason trivial code like the following
-// might not produce the expected leak checking outcome
-// depending on how the compiled code works with the stack:
-//
-//   int* foo = new int [20];
-//   HeapLeakChecker check("a_check");
-//   foo = NULL;
-//   CHECK(check.NoLeaks());  // this might succeed
-//
-// HINT: If you are debugging detected leaks, you can try different
-// (e.g. less strict) values for HEAPCHECK to determine the cause of
-// the reported leaks (see the code of
-// HeapLeakChecker::InternalInitStart for details).
-//
-// GUIDELINE: Below are the preferred ways of making your (test)
-// binary pass the above recommended overall heap leak check in the
-// order of decreasing preference:
-//
-// 1. Fix the leaks if they are real leaks.
-//
-// 2. If you are sure that the reported leaks are not dangerous
-//    and there is no good way to fix them, then you can use
-//    HeapLeakChecker::DisableChecks(Up|In|At) calls (see below) in
-//    the relevant modules to disable certain stack traces for the
-//    purpose of leak checking.  You can also use
-//    HeapLeakChecker::IgnoreObject() call to ignore certain leaked
-//    heap objects and everythign reachable from them.
-//
-// 3. If the leaks are due to some initialization in a third-party
-//    package, you might be able to force that initialization before
-//    the heap checking starts.
-//
-//    I.e. if HEAPCHECK == "minimal" or less strict, if you put the
-//    initialization in a global constructor the heap-checker will
-//    ignore it.  If HEAPCHECK == "normal" or stricter, only
-//    HeapLeakChecker::LibCPreallocate() happens before heap checking
-//    starts.
-//
-// Making a binary pass at "strict" or "draconian" level is not
-// necessary or even desirable in the numerous cases when it requires
-// adding a lot of (otherwise unused) heap cleanup code to various
-// core libraries.
-//
-// NOTE: the following should apply only if
-//       HEAPCHECK == "strict" or stricter
-//
-// 4. If the found leaks are due to incomplete cleanup in destructors
-//    of global variables, extend or add those destructors or use a
-//    REGISTER_HEAPCHECK_CLEANUP to do the deallocations instead to
-//    avoid cleanup overhead during normal execution.  This type of
-//    leaks get reported when one goes from "normal" to "strict"
-//    checking.
-//
-// NOTE: the following should apply only if
-//       HEAPCHECK == "draconian" or stricter
-//
-// 5. If the found leaks are for global static pointers whose values are
-//    allocated/grown (e.g on-demand) and never deallocated,
-//    then you should be able to add REGISTER_HEAPCHECK_CLEANUP's
-//    or appropriate destructors into these modules
-//    to free those objects.
-//
-// Example of local usage (anywhere in the program):
-//
-//   HeapLeakChecker heap_checker("test_foo");
-//
-//   { <code that exercises some foo functionality
-//      that should preserve memory allocation state> }
-//
-//   CHECK(heap_checker.SameHeap());
-//
-// NOTE: One should set HEAPCHECK to a non-empty value e.g. "local"
-// to help suppress some false leaks for these local checks.
+// After that, just run your binary.  If the heap-checker detects
+// a memory leak at program-exit, it will print instructions on how
+// to track down the leak.
 
 
 #ifndef BASE_HEAP_CHECKER_H__
 #define BASE_HEAP_CHECKER_H__
 
-#include <sys/types.h>    // for size_t
+#include <sys/types.h>  // for size_t
+#include <stdint.h>     // for uintptr_t
 #include <vector>
 
-// A macro to declare module heap check cleanup tasks
-// (they run only if we are doing heap leak checking.)
-// Use
-//  public:
-//   void Class::HeapCleanup();
-// if you need to do heap check cleanup on private members of a class.
-#define REGISTER_HEAPCHECK_CLEANUP(name, body)  \
-  namespace { \
-  void heapcheck_cleanup_##name() { body; } \
-  static HeapCleaner heapcheck_cleaner_##name(&heapcheck_cleanup_##name); \
-  }
-
-// A class that exists solely to run its destructor.  This class should not be
-// used directly, but instead by the REGISTER_HEAPCHECK_CLEANUP macro above.
-class HeapCleaner {
- public:
-  typedef void (*void_function)(void);
-  HeapCleaner(void_function f);
-  static void RunHeapCleanups();
- private:
-  static std::vector<void_function>* heap_cleanups_;
-};
 
 class HeapLeakChecker {
  public:  // Static functions for working with (whole-program) leak checking.
- 
+
   // If heap leak checking is currently active in some mode
   // e.g. if leak checking was started (and is still active now)
   // due to any valid non-empty --heap_check flag value
   // (including "local") on the command-line
   // or via a dependency on //base:heapcheck.
-  // The return value reflects iff HeapLeakChecker objects manually 
+  // The return value reflects iff HeapLeakChecker objects manually
   // constructed right now will be doing leak checking or nothing.
   // Note that we can go from active to inactive state during InitGoogle()
   // if FLAGS_heap_check gets set to "" by some code before/during InitGoogle().
   static bool IsActive();
 
+  // Return pointer to the whole-program checker if (still) active
+  // and NULL otherwise.
+  // This is mainly to access BytesLeaked() and ObjectsLeaked() (see below)
+  // for the whole-program checker after one calls NoGlobalLeaks()
+  // or similar and gets false.
+  static HeapLeakChecker* GlobalChecker();
+
+  // Do whole-program leak check now (if it was activated for this binary);
+  // return false only if it was activated and has failed.
+  // The mode of the check is controlled by the command-line flags.
+  // This method can be called repeatedly.
+  // Things like GlobalChecker()->SameHeap() can also be called explicitly
+  // to do the desired flavor of the check.
+  static bool NoGlobalLeaks();
+
+  // If whole-program checker if active,
+  // cancel its automatic execution after main() exits.
+  // This requires that some leak check (e.g. NoGlobalLeaks())
+  // has been called at least once on the whole-program checker.
+  static void CancelGlobalCheck();
+
  public:  // Non-static functions for starting and doing leak checking.
 
   // Start checking and name the leak check performed.
@@ -288,27 +112,30 @@ class HeapLeakChecker {
   //  and thus is not detected by BriefNoLeaks.)
   // CAVEAT: pprof will do no checking over stripped binaries
   // (our automatic test binaries are stripped)
-  bool NoLeaks() { return DoNoLeaks(false, true, true); }
+  // NOTE: All *NoLeaks() and *SameHeap() methods can be called many times
+  // to check for leaks at different end-points in program's execution.
+  bool NoLeaks() { return DoNoLeaks(NO_LEAKS, USE_PPROF, PPROF_REPORT); }
 
   // Return true iff the heap does not seem to have more objects allocated
   // w.r.t. its state at the time of our construction
   // by looking at the number of objects & bytes allocated.
   // This also tries to do pprof reporting of detected leaks.
-  bool QuickNoLeaks() { return DoNoLeaks(false, false, true); }
+  bool QuickNoLeaks() { return DoNoLeaks(NO_LEAKS, USE_COUNTS, PPROF_REPORT); }
 
   // Return true iff the heap does not seem to have more objects allocated
   // w.r.t. its state at the time of our construction
   // by looking at the number of objects & bytes allocated.
   // This does not try to use pprof at all.
-  bool BriefNoLeaks() { return DoNoLeaks(false, false, false); }
+  bool BriefNoLeaks() { return DoNoLeaks(NO_LEAKS, USE_COUNTS, NO_REPORT); }
 
   // These are similar to their *NoLeaks counterparts,
   // but they in addition require no negative leaks,
   // i.e. the state of the heap must be exactly the same
   // as at the time of our construction.
-  bool SameHeap() { return DoNoLeaks(true, true, true); }
-  bool QuickSameHeap() { return DoNoLeaks(true, false, true); }
-  bool BriefSameHeap() { return DoNoLeaks(true, false, false); }
+  bool SameHeap() { return DoNoLeaks(SAME_HEAP, USE_PPROF, PPROF_REPORT); }
+  bool QuickSameHeap()
+    { return DoNoLeaks(SAME_HEAP, USE_COUNTS, PPROF_REPORT); }
+  bool BriefSameHeap() { return DoNoLeaks(SAME_HEAP, USE_COUNTS, NO_REPORT); }
 
   // Detailed information about the number of leaked bytes and objects
   // (both of these can be negative as well).
@@ -321,24 +148,10 @@ class HeapLeakChecker {
   ssize_t BytesLeaked() const;
   ssize_t ObjectsLeaked() const;
 
-  // Destructor
-  // (verifies that some *SameHeap or *NoLeaks method has been called).
+  // Destructor (verifies that some *NoLeaks or *SameHeap method
+  // has been called at least once).
   ~HeapLeakChecker();
 
-  // Accessors to determine various internal parameters.  These should
-  // be set as early as possible.
-
-  // If overall heap check reports found leaks via pprof.  Default: true
-  static void set_heap_check_report(bool);
-  // Location of pprof script.  Default: $prefix/bin/pprof
-  static void set_pprof_path(const char*);
-  // Location to write profile dumps.  Default: /tmp
-  static void set_dump_directory(const char*);
-
-  static bool heap_check_report();
-  static const char* pprof_path();
-  static const char* dump_directory();
-
  private:  // data
 
   char* name_;  // our remembered name
@@ -348,9 +161,6 @@ class HeapLeakChecker {
   ssize_t inuse_bytes_increase_;  // bytes-in-use increase for this checker
   ssize_t inuse_allocs_increase_;  // allocations-in-use increase for this checker
 
-  static pid_t main_thread_pid_;  // For naming output files
-  static std::string* dump_directory_; // Location to write profile dumps
-
  public:  // Static helpers to make us ignore certain leaks.
 
   // NOTE: All calls to DisableChecks* affect all later heap profile generation
@@ -358,7 +168,7 @@ class HeapLeakChecker {
   // They do nothing when heap leak checking is turned off.
 
   // CAVEAT: Disabling via all the DisableChecks* functions happens only
-  // up to kMaxStackTrace stack frames (see heap-profiler.cc)
+  // up to kMaxStackTrace stack frames (see heap-profile-table.h)
   // down from the stack frame identified by the function.
   // Hence, this disabling will stop working for very deep call stacks
   // and you might see quite wierd leak profile dumps in such cases.
@@ -455,28 +265,35 @@ class HeapLeakChecker {
 
  public:  // Initializations; to be called from main() only.
 
-  // Full starting of recommended whole-program checking.  This runs after
-  // HeapChecker::BeforeConstructors and can do initializations which may
-  // depend on configuration parameters set by initialization code.
-  // Valid values of heap_check type are:
-  //  - "minimal"
-  //  - "normal"
-  //  - "strict"
-  //  - "draconian"
-  //  - "local"
-  static void InternalInitStart(const std::string& heap_check_type);
+  // Full starting of recommended whole-program checking.
+  static void InternalInitStart();
+
+ public:  // Internal types defined in .cc
 
+  struct Allocator;
   struct RangeValue;
-  struct StackExtent;
 
  private:  // Various helpers
 
-  // Helper for dumping start/end heap leak checking profiles.
-  void DumpProfileLocked(bool start, const StackExtent& self_stack);
+  // Type for DumpProfileLocked
+  enum ProfileType { START_PROFILE, END_PROFILE };
+  // Helper for dumping start/end heap leak checking profiles
+  // and getting the byte/object counts.
+  void DumpProfileLocked(ProfileType profile_type, void* self_stack_top,
+                         size_t* alloc_bytes, size_t* alloc_objects);
   // Helper for constructors
   void Create(const char *name);
-  // Helper for *NoLeaks and *SameHeap
-  bool DoNoLeaks(bool same_heap, bool do_full, bool do_report);
+  // Types for DoNoLeaks and its helpers.
+  enum CheckType { SAME_HEAP, NO_LEAKS };
+  enum CheckFullness { USE_PPROF, USE_COUNTS };
+  enum ReportMode { PPROF_REPORT, NO_REPORT };
+  // Helpers for *NoLeaks and *SameHeap
+  bool DoNoLeaks(CheckType check_type,
+                 CheckFullness fullness,
+                 ReportMode report_mode);
+  bool DoNoLeaksOnce(CheckType check_type,
+                     CheckFullness fullness,
+                     ReportMode report_mode);
   // Helper for IgnoreObject
   static void IgnoreObjectLocked(void* ptr);
   // Helper for DisableChecksAt
@@ -484,11 +301,11 @@ class HeapLeakChecker {
   // Helper for DisableChecksIn
   static void DisableChecksInLocked(const char* pattern);
   // Helper for DisableChecksToHereFrom
-  static void DisableChecksFromTo(void* start_address,
-                                  void* end_address,
-                                  int max_depth);
+  static void DisableChecksFromToLocked(void* start_address,
+                                        void* end_address,
+                                        int max_depth);
   // Helper for DoNoLeaks to ignore all objects reachable from all live data
-  static void IgnoreAllLiveObjectsLocked(const StackExtent& self_stack);
+  static void IgnoreAllLiveObjectsLocked(void* self_stack_top);
   // Callback we pass to ListAllProcessThreads (see thread_lister.h)
   // that is invoked when all threads of our process are found and stopped.
   // The call back does the things needed to ignore live data reachable from
@@ -519,44 +336,89 @@ class HeapLeakChecker {
   // in a debug message to describe what kind of live object sources
   // are being used.
   static void IgnoreLiveObjectsLocked(const char* name, const char* name2);
+  // Heap profile object filtering callback to filter out live objects.
+  static bool HeapProfileFilter(void* ptr, size_t size);
   // Runs REGISTER_HEAPCHECK_CLEANUP cleanups and potentially
   // calls DoMainHeapCheck
   static void RunHeapCleanups();
   // Do the overall whole-program heap leak check
   static void DoMainHeapCheck();
 
-  // Type of task for UseProcMaps
-  enum ProcMapsTask { 
-    RECORD_GLOBAL_DATA_LOCKED, 
-    DISABLE_LIBRARY_ALLOCS 
+  // Type of task for UseProcMapsLocked
+  enum ProcMapsTask {
+    RECORD_GLOBAL_DATA,
+    DISABLE_LIBRARY_ALLOCS
+  };
+  // Success/Error Return codes for UseProcMapsLocked.
+  enum ProcMapsResult {
+    PROC_MAPS_USED,
+    CANT_OPEN_PROC_MAPS,
+    NO_SHARED_LIBS_IN_PROC_MAPS
   };
-  // Success/Error Return codes for UseProcMaps.
-  enum ProcMapsResult { PROC_MAPS_USED, CANT_OPEN_PROC_MAPS,
-                        NO_SHARED_LIBS_IN_PROC_MAPS };
   // Read /proc/self/maps, parse it, and do the 'proc_maps_task' for each line.
-  static ProcMapsResult UseProcMaps(ProcMapsTask proc_maps_task);
+  static ProcMapsResult UseProcMapsLocked(ProcMapsTask proc_maps_task);
   // A ProcMapsTask to disable allocations from 'library'
   // that is mapped to [start_address..end_address)
   // (only if library is a certain system library).
-  static void DisableLibraryAllocs(const char* library,
-                                   void* start_address,
-                                   void* end_address);
+  static void DisableLibraryAllocsLocked(const char* library,
+                                         uintptr_t start_address,
+                                         uintptr_t end_address);
+
+  // Return true iff "*ptr" points to a heap object;
+  // we also fill *object_size for this object then.
+  // If yes, we might move "*ptr" to point to the very start of the object
+  // (this needs to happen for C++ class array allocations
+  // and for basic_string-s of C++ library that comes with gcc 3.4).
+  static bool HaveOnHeapLocked(void** ptr, size_t* object_size);
 
  private:
 
+  // Helper for VerifyHeapProfileTableStackGet in the unittest
+  // to get the recorded allocation caller for ptr,
+  // which must be a heap object.
+  static void* GetAllocCaller(void* ptr);
+  friend void VerifyHeapProfileTableStackGet();
+
   // This gets to execute before constructors for all global objects
   static void BeforeConstructors();
   friend void HeapLeakChecker_BeforeConstructors();
   // This gets to execute after destructors for all global objects
   friend void HeapLeakChecker_AfterDestructors();
+  // Helper to shutdown heap leak checker when it's not needed
+  // or can't function properly.
+  static void TurnItselfOff();
 
  private:
+
   // Start whole-executable checking.
   HeapLeakChecker();
 
-  // Don't allow copy constructors -- these are declared but not defined
+ private:
+  // Disallow "evil" constructors.
   HeapLeakChecker(const HeapLeakChecker&);
   void operator=(const HeapLeakChecker&);
 };
 
+
+// A class that exists solely to run its destructor.  This class should not be
+// used directly, but instead by the REGISTER_HEAPCHECK_CLEANUP macro below.
+class HeapCleaner {
+ public:
+  typedef void (*void_function)(void);
+  HeapCleaner(void_function f);
+  static void RunHeapCleanups();
+ private:
+  static std::vector<void_function>* heap_cleanups_;
+};
+
+// A macro to declare module heap check cleanup tasks
+// (they run only if we are doing heap leak checking.)
+// 'body' should be the cleanup code to run.  'name' doesn't matter,
+// but must be unique amongst all REGISTER_HEAPCHECK_CLEANUP calls.
+#define REGISTER_HEAPCHECK_CLEANUP(name, body)  \
+  namespace { \
+  void heapcheck_cleanup_##name() { body; } \
+  static HeapCleaner heapcheck_cleaner_##name(&heapcheck_cleanup_##name); \
+  }
+
 #endif  // BASE_HEAP_CHECKER_H__
diff --git a/src/google/heap-profiler.h b/src/google/heap-profiler.h
index b62056e..b603ca7 100644
--- a/src/google/heap-profiler.h
+++ b/src/google/heap-profiler.h
@@ -32,55 +32,25 @@
 //
 // Module for heap-profiling.
 //
-// This module is safe to link into any program you may wish to
-// profile at some point.  It will not cause any noticeable slowdowns
-// unless you activate by setting the environment variable
-// HEAPPROFILE, e.g.:
-// $ export HEAPPROFILE=/tmp/my_program_profile ; ./my_program
-// $ ls /tmp/my_program_profile.*
-//    /tmp/my_program_profile.0000.heap
-//    /tmp/my_program_profile.0001.heap
-//    /tmp/my_program_profile.0002.heap
-//    ...
+// For full(er) information, see doc/heapprofile.html
 //
-// This allows you to easily profile your program at any time without
-// having to recompile, and doesn't slow things down if HEAPPROFILE is
-// unset.  We refuse to do profiling if uid != euid, to avoid
-// environment-based security issues if your program is accidentally
-// setuid.  Note that this library should generally not be linked into
-// setuid programs.  It has not been reviewed or tested for security
-// under setuid conditions.
+// This module can be linked into your program with
+// no slowdown caused by this unless you activate the profiler
+// using one of the following methods:
 //
-// If heap-profiling is turned on, a profile file is dumped every GB
-// of allocated data.  You can override this behavior by calling
-// HeapProfilerSetAllocationInterval() to a number of bytes N.  If
-// you do that, a profile file will be dumped after every N bytes of
-// allocations.
+//    1. Before starting the program, set the environment variable
+//       "HEAPPROFILE" to be the name of the file to which the profile
+//       data should be written.
 //
-// If heap profiling is on, we also dump a profile when the
-// in-use-bytes reach a new high-water-mark.  Only increases of at
-// least 100MB are considered significant changes in the
-// high-water-mark.  This number can be changed by calling
-// HeapProfilerSetInuseInterval() with a different byte-value.
+//    2. Programmatically, start and stop the profiler using the
+//       routines "HeapProfilerStart(filename)" and "HeapProfilerStop()".
 //
-// STL WARNING: The HeapProfiler does not accurately track allocations
-// in many STL implementations.  This is because it is common for the
-// default STL allocator to keep an internal pool of memory and nevery
-// return it to the system.  This means that large allocations may be
-// attributed to an object that you know was destroyed.  For a simple
-// example, see TestHeapLeakCheckerSTL in
-// src/tests/heap-checker_unittest.cc.
-//
-// This issue is resolved for GCC 3.3 and 3.4 by setting the
-// environment variable GLIBCXX_FORCE_NEW, which forces the STL
-// allocator to call `new' and `delete' explicitly for every
-// allocation and deallocation.  For GCC 3.2 and previous you will
-// need to compile your source with -D__USE_MALLOC.  For other
-// compilers / STL libraries, there may be a similar solution; See
-// your implementation's documentation for information.
+// Use pprof to view the resulting profile output.
+//    % google3/perftools/pprof <path_to_executable> <profile_file_name>
+//    % google3/perftools/pprof --gv  <path_to_executable> <profile_file_name>
 
-#ifndef _HEAP_PROFILER_H
-#define _HEAP_PROFILER_H
+#ifndef BASE_HEAP_PROFILER_H__
+#define BASE_HEAP_PROFILER_H__
 
 #include <stddef.h>
 
@@ -102,19 +72,4 @@ extern void HeapProfilerDump(const char *reason);
 // free()-ed as soon as the caller does not need it anymore.
 extern char* GetHeapProfile();
 
-// ---- Configuration accessors ----
-
-// Level of logging used by the heap profiler and heap checker (if applicable)
-// Default: 0
-extern void HeapProfilerSetLogLevel(int level);
-
-// Dump heap profiling information once every specified number of bytes
-// allocated by the program.  Default: 1GB
-extern void HeapProfilerSetAllocationInterval(size_t interval);
-
-// Dump heap profiling information whenever the high-water 
-// memory usage mark increases by the specified number of
-// bytes.  Default: 100MB
-extern void HeapProfilerSetInuseInterval(size_t interval);
-
-#endif /* _HEAP_PROFILER_H */
+#endif /* BASE_HEAP_PROFILER_H__ */
diff --git a/src/google/malloc_extension.h b/src/google/malloc_extension.h
index e088154..5244aa1 100644
--- a/src/google/malloc_extension.h
+++ b/src/google/malloc_extension.h
@@ -30,13 +30,13 @@
 // ---
 // Author: Sanjay Ghemawat <opensource@google.com>
 //
-// Extra interfaces exported by some malloc implementations.  These
-// interfaces are accessed through a virtual base class so an
+// Extra extensions exported by some malloc implementations.  These
+// extensions are accessed through a virtual base class so an
 // application can link against a malloc that does not implement these
-// interfaces, and it will get default versions that do nothing.
+// extensions, and it will get default versions that do nothing.
 
-#ifndef _GOOGLE_MALLOC_EXTENSION_H__
-#define _GOOGLE_MALLOC_EXTENSION_H__
+#ifndef BASE_MALLOC_EXTENSION_H__
+#define BASE_MALLOC_EXTENSION_H__
 
 #include <stddef.h>
 #include <string>
@@ -137,6 +137,26 @@ class MallocExtension {
   // REQUIRES: property != NULL
   virtual bool SetNumericProperty(const char* property, size_t value);
 
+  // Mark the current thread as "idle".  This routine may optionally
+  // be called by threads as a hint to the malloc implementation that
+  // any thread-specific resources should be released.  Note: this may
+  // be an expensive routine, so it should not be called too often.
+  //
+  // Also, if the code that calls this routine will go to sleep for
+  // a while, it should take care to not allocate anything between
+  // the call to this routine and the beginning of the sleep.
+  //
+  // Most malloc implementations ignore this routine.
+  virtual void MarkThreadIdle();
+
+  // Try to free memory back to the operating system for reuse.  Only
+  // use this extension if the application has recently freed a lot of
+  // memory, and does not anticipate using it again for a long time --
+  // to get this memory back may require faulting pages back in by the
+  // OS, and that may be slow.  (Currently only implemented in
+  // tcmalloc.)
+  virtual void ReleaseFreeMemory();
+
   // The current malloc implementation.  Always non-NULL.
   static MallocExtension* instance();
 
@@ -170,4 +190,4 @@ class MallocExtension {
   virtual void** ReadHeapGrowthStackTraces();
 };
 
-#endif  // _GOOGLE_MALLOC_EXTENSION_H__
+#endif  // BASE_MALLOC_EXTENSION_H__
diff --git a/src/google/malloc_hook.h b/src/google/malloc_hook.h
index 799658e..aadb9dc 100644
--- a/src/google/malloc_hook.h
+++ b/src/google/malloc_hook.h
@@ -35,9 +35,14 @@
 // NULL, they are not invoked.
 //
 // One important user of these hooks is the heap profiler.
+//
+// CAVEAT: If you add new MallocHook::Invoke* calls (not for chaining hooks),
+// then those calls must be directly in the code of the (de)allocation
+// function that is provided to the user and that function must have
+// an ATTRIBUTE_SECTION(malloc_hook_callers) attribute.
 
-#ifndef _GOOGLE_MALLOC_HOOK_H
-#define _GOOGLE_MALLOC_HOOK_H
+#ifndef _MALLOC_HOOK_H
+#define _MALLOC_HOOK_H
 
 #include <stddef.h>
 #include <sys/types.h>
@@ -72,7 +77,7 @@ class MallocHook {
 
   // The MmapHook is invoked whenever a region of memory is mapped.
   // It may be passed MAP_FAILED if the mmap failed.
-  typedef void (*MmapHook)(void* result, 
+  typedef void (*MmapHook)(void* result,
                            void* start,
                            size_t size,
                            int protection,
@@ -98,7 +103,7 @@ class MallocHook {
                                           fd, offset);
   }
 
-  // The MunmapHook is invoked whenever an object is deallocated.
+  // The MunmapHook is invoked whenever a region of memory is unmapped.
   typedef void (*MunmapHook)(void* ptr, size_t size);
   inline static MunmapHook GetMunmapHook() { return munmap_hook_; }
   inline static MunmapHook SetMunmapHook(MunmapHook hook) {
@@ -110,12 +115,57 @@ class MallocHook {
     if (munmap_hook_ != NULL) (*munmap_hook_)(p, size);
   }
 
+  // The MremapHook is invoked whenever a region of memory is remapped.
+  typedef void (*MremapHook)(void* result,
+                             void* old_addr,
+                             size_t old_size,
+                             size_t new_size,
+                             int flags,
+                             void* new_addr);
+  inline static MremapHook GetMremapHook() { return mremap_hook_; }
+  inline static MremapHook SetMremapHook(MremapHook hook) {
+    MremapHook result = mremap_hook_;
+    mremap_hook_ = hook;
+    return result;
+  }
+  inline static void InvokeMremapHook(void* result,
+                                      void* old_addr,
+                                      size_t old_size,
+                                      size_t new_size,
+                                      int flags,
+                                      void* new_addr) {
+    if (mremap_hook_ != NULL) (*mremap_hook_)(result,
+                                              old_addr, old_size,
+                                              new_size, flags, new_addr);
+  }
+
+  // The SbrkHook is invoked whenever sbrk is called.
+  typedef void (*SbrkHook)(void* result, ptrdiff_t increment);
+  inline static SbrkHook GetSbrkHook() { return sbrk_hook_; }
+  inline static SbrkHook SetSbrkHook(SbrkHook hook) {
+    SbrkHook result = sbrk_hook_;
+    sbrk_hook_ = hook;
+    return result;
+  }
+  inline static void InvokeSbrkHook(void* result, ptrdiff_t increment) {
+    if (sbrk_hook_ != NULL) (*sbrk_hook_)(result, increment);
+  }
+
+  // Get the current stack trace.  Try to skip all routines up to and
+  // and including the caller of MallocHook::Invoke*.
+  // Use "skip_count" (similarly to GetStackTrace from stacktrace.h)
+  // as a hint about how many routines to skip if better information
+  // is not available.
+  static int GetCallerStackTrace(void** result, int max_depth, int skip_count);
+
  private:
+
   static NewHook     new_hook_;
   static DeleteHook  delete_hook_;
   static MmapHook    mmap_hook_;
   static MunmapHook  munmap_hook_;
-
+  static MremapHook  mremap_hook_;
+  static SbrkHook    sbrk_hook_;
 };
 
-#endif /* _GOOGLE_MALLOC_HOOK_H */
+#endif /* _MALLOC_HOOK_H */
diff --git a/src/google/profiler.h b/src/google/profiler.h
index 5eea0de..29404fa 100644
--- a/src/google/profiler.h
+++ b/src/google/profiler.h
@@ -32,37 +32,29 @@
 //
 // Module for CPU profiling based on periodic pc-sampling.
 //
-// To use this module, link it into your program.  There should
-// be no slowdown caused by this unless you activate the profiler
-// using one of the steps given below.
+// For full(er) information, see doc/cpuprofile.html
 //
-// To activate the profiler, do one of the following:
+// This module is linked into your program with
+// no slowdown caused by this unless you activate the profiler
+// using one of the following methods:
 //
 //    1. Before starting the program, set the environment variable
-//       "CPUPROFILE" to be the name of the file to which the profile
+//       "PROFILE" to be the name of the file to which the profile
 //       data should be written.
 //
-//    2. Programmatically, start and stop the profiler using
-//       the routines "ProfilerStart(filename)" and "ProfilerStop()".
+//    2. Programmatically, start and stop the profiler using the
+//       routines "ProfilerStart(filename)" and "ProfilerStop()".
 //
 // All threads in the program are profiled whenever profiling is on.
-// There used to be a mechanism where a subset of the threads could be
-// profiled, but that functionality no longer exists (it would not
-// work correctly in new systems since the interval timer used by the
-// profiler is a per-address-space setting in new systems instead of
-// being a per-thread setting in 2.4 and earlier systems).
+// (Note: if using linux 2.4 or earlier, only the main thread may be
+// profiled.)
 //
-// Limitation: on 2.4 and earlier kernels, just the main thread will
-// be profiled.
-//
-// Use pprof to view the resulting profile output.  If you have dot and
-// gv installed, you can also get a graphical representation of CPU usage.
+// Use pprof to view the resulting profile output.
 //    % pprof <path_to_executable> <profile_file_name>
-//    % pprof --dot <path_to_executable> <profile_file_name>
 //    % pprof --gv  <path_to_executable> <profile_file_name>
 
-#ifndef _GOOGLE_PROFILER_H
-#define _GOOGLE_PROFILER_H
+#ifndef BASE_PROFILER_H__
+#define BASE_PROFILER_H__
 
 #include <time.h>       // For time_t
 
@@ -122,4 +114,4 @@ class ProfilerThreadState {
   void ThreadCheck() { }
 };
 
-#endif /* _GOOGLE_PROFILER_H */
+#endif /* BASE_PROFILER_H__ */
diff --git a/src/google/stacktrace.h b/src/google/stacktrace.h
index a70ade2..c0259f8 100644
--- a/src/google/stacktrace.h
+++ b/src/google/stacktrace.h
@@ -60,22 +60,4 @@
 // Linux/x86 machines.
 extern int GetStackTrace(void** result, int max_depth, int skip_count);
 
-// Compute the extent of the function call stack by traversing it up.
-// Input: "sp" is either NULL, or is a stack pointer
-// (e.g., a value of the ebp register for x86).
-// If "sp == NULL", the stack pointer for the current thread is implied.
-//
-// Stores the range of addresses covered by the specified stack
-// in *stack_top and *stack_bottom.  Returns true if successful,
-// false on failure (e.g., an inability to walk the stack).
-//
-// If it returns true, *stack_top and *stack_bottom respectively correspond
-// to the most-recetly pushed frame of the call stack
-// and the intial frame that started the call stack.
-// Their relative ordering as integers though
-// depends on the underlying machine's architecture.
-extern bool GetStackExtent(void* sp,
-                           void** stack_top,
-                           void** stack_bottom);
-
 #endif /* _GOOGLE_STACKTRACE_H */