Merge branch 'kb/perf-trace'

* kb/perf-trace:
  api-trace.txt: add trace API documentation
  progress: simplify performance measurement by using getnanotime()
  wt-status: simplify performance measurement by using getnanotime()
  git: add performance tracing for git's main() function to debug scripts
  trace: add trace_performance facility to debug performance issues
  trace: add high resolution timer function to debug performance issues
  trace: add 'file:line' to all trace output
  trace: move code around, in preparation to file:line output
  trace: add current timestamp to all trace output
  trace: disable additional trace output for unit tests
  trace: add infrastructure to augment trace output with additional info
  sha1_file: change GIT_TRACE_PACK_ACCESS logging to use trace API
  Documentation/git.txt: improve documentation of 'GIT_TRACE*' variables
  trace: improve trace performance
  trace: remove redundant printf format attribute
  trace: consistently name the format parameter
  trace: move trace declarations from cache.h to new trace.h

2 files changed, 138 insertions, 18 deletions

diff --git a/Documentation/git.txt b/Documentation/git.txt
index d0ddfcb0aa..05857c96ff 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -906,31 +906,54 @@ for further details.
 	based on whether stdout appears to be redirected to a file or not.
 
 'GIT_TRACE'::
-	If this variable is set to "1", "2" or "true" (comparison
-	is case insensitive), Git will print `trace:` messages on
-	stderr telling about alias expansion, built-in command
-	execution and external command execution.
-	If this variable is set to an integer value greater than 1
-	and lower than 10 (strictly) then Git will interpret this
-	value as an open file descriptor and will try to write the
-	trace messages into this file descriptor.
-	Alternatively, if this variable is set to an absolute path
-	(starting with a '/' character), Git will interpret this
-	as a file path and will try to write the trace messages
-	into it.
+	Enables general trace messages, e.g. alias expansion, built-in
+	command execution and external command execution.
++
+If this variable is set to "1", "2" or "true" (comparison
+is case insensitive), trace messages will be printed to
+stderr.
++
+If the variable is set to an integer value greater than 2
+and lower than 10 (strictly) then Git will interpret this
+value as an open file descriptor and will try to write the
+trace messages into this file descriptor.
++
+Alternatively, if the variable is set to an absolute path
+(starting with a '/' character), Git will interpret this
+as a file path and will try to write the trace messages
+into it.
++
+Unsetting the variable, or setting it to empty, "0" or
+"false" (case insensitive) disables trace messages.
 
 'GIT_TRACE_PACK_ACCESS'::
-	If this variable is set to a path, a file will be created at
-	the given path logging all accesses to any packs. For each
+	Enables trace messages for all accesses to any packs. For each
 	access, the pack file name and an offset in the pack is
 	recorded. This may be helpful for troubleshooting some
 	pack-related performance problems.
+	See 'GIT_TRACE' for available trace output options.
 
 'GIT_TRACE_PACKET'::
-	If this variable is set, it shows a trace of all packets
-	coming in or out of a given program. This can help with
-	debugging object negotiation or other protocol issues. Tracing
-	is turned off at a packet starting with "PACK".
+	Enables trace messages for all packets coming in or out of a
+	given program. This can help with debugging object negotiation
+	or other protocol issues. Tracing is turned off at a packet
+	starting with "PACK".
+	See 'GIT_TRACE' for available trace output options.
+
+'GIT_TRACE_PERFORMANCE'::
+	Enables performance related trace messages, e.g. total execution
+	time of each Git command.
+	See 'GIT_TRACE' for available trace output options.
+
+'GIT_TRACE_SETUP'::
+	Enables trace messages printing the .git, working tree and current
+	working directory after Git has completed its setup phase.
+	See 'GIT_TRACE' for available trace output options.
+
+'GIT_TRACE_SHALLOW'::
+	Enables trace messages that can help debugging fetching /
+	cloning of shallow repositories.
+	See 'GIT_TRACE' for available trace output options.
 
 GIT_LITERAL_PATHSPECS::
 	Setting this variable to `1` will cause Git to treat all
diff --git a/Documentation/technical/api-trace.txt b/Documentation/technical/api-trace.txt
new file mode 100644
index 0000000000..097a651d96
--- /dev/null
+++ b/Documentation/technical/api-trace.txt
@@ -0,0 +1,97 @@
+trace API
+=========
+
+The trace API can be used to print debug messages to stderr or a file. Trace
+code is inactive unless explicitly enabled by setting `GIT_TRACE*` environment
+variables.
+
+The trace implementation automatically adds `timestamp file:line ... \n` to
+all trace messages. E.g.:
+
+------------
+23:59:59.123456 git.c:312               trace: built-in: git 'foo'
+00:00:00.000001 builtin/foo.c:99        foo: some message
+------------
+
+Data Structures
+---------------
+
+`struct trace_key`::
+
+	Defines a trace key (or category). The default (for API functions that
+	don't take a key) is `GIT_TRACE`.
++
+E.g. to define a trace key controlled by environment variable `GIT_TRACE_FOO`:
++
+------------
+static struct trace_key trace_foo = TRACE_KEY_INIT(FOO);
+
+static void trace_print_foo(const char *message)
+{
+	trace_print_key(&trace_foo, message);
+}
+------------
++
+Note: don't use `const` as the trace implementation stores internal state in
+the `trace_key` structure.
+
+Functions
+---------
+
+`int trace_want(struct trace_key *key)`::
+
+	Checks whether the trace key is enabled. Used to prevent expensive
+	string formatting before calling one of the printing APIs.
+
+`void trace_disable(struct trace_key *key)`::
+
+	Disables tracing for the specified key, even if the environment
+	variable was set.
+
+`void trace_printf(const char *format, ...)`::
+`void trace_printf_key(struct trace_key *key, const char *format, ...)`::
+
+	Prints a formatted message, similar to printf.
+
+`void trace_argv_printf(const char **argv, const char *format, ...)``::
+
+	Prints a formatted message, followed by a quoted list of arguments.
+
+`void trace_strbuf(struct trace_key *key, const struct strbuf *data)`::
+
+	Prints the strbuf, without additional formatting (i.e. doesn't
+	choke on `%` or even `\0`).
+
+`uint64_t getnanotime(void)`::
+
+	Returns nanoseconds since the epoch (01/01/1970), typically used
+	for performance measurements.
++
+Currently there are high precision timer implementations for Linux (using
+`clock_gettime(CLOCK_MONOTONIC)`) and Windows (`QueryPerformanceCounter`).
+Other platforms use `gettimeofday` as time source.
+
+`void trace_performance(uint64_t nanos, const char *format, ...)`::
+`void trace_performance_since(uint64_t start, const char *format, ...)`::
+
+	Prints the elapsed time (in nanoseconds), or elapsed time since
+	`start`, followed by a formatted message. Enabled via environment
+	variable `GIT_TRACE_PERFORMANCE`. Used for manual profiling, e.g.:
++
+------------
+uint64_t start = getnanotime();
+/* code section to measure */
+trace_performance_since(start, "foobar");
+------------
++
+------------
+uint64_t t = 0;
+for (;;) {
+	/* ignore */
+	t -= getnanotime();
+	/* code section to measure */
+	t += getnanotime();
+	/* ignore */
+}
+trace_performance(t, "frotz");
+------------