delta/git.git/lockfile.h, branch ws/curl-http-proxy-over-https

lockfile: update lifetime requirements in documentation

2017-09-06T08:19:54+00:00

Now that the tempfile system we rely on has loosened the
lifetime requirements for storage, we can adjust our
documentation to match.

Signed-off-by: Jeff King 
Signed-off-by: Junio C Hamano

tempfile: auto-allocate tempfiles on heap

2017-09-06T08:19:54+00:00

The previous commit taught the tempfile code to give up
ownership over tempfiles that have been renamed or deleted.
That makes it possible to use a stack variable like this:

  struct tempfile t;

  create_tempfile(&t, ...);
  ...
  if (!err)
          rename_tempfile(&t, ...);
  else
          delete_tempfile(&t);

But doing it this way has a high potential for creating
memory errors. The tempfile we pass to create_tempfile()
ends up on a global linked list, and it's not safe for it to
go out of scope until we've called one of those two
deactivation functions.

Imagine that we add an early return from the function that
forgets to call delete_tempfile(). With a static or heap
tempfile variable, the worst case is that the tempfile hangs
around until the program exits (and some functions like
setup_shallow_temporary rely on this intentionally, creating
a tempfile and then leaving it for later cleanup).

But with a stack variable as above, this is a serious memory
error: the variable goes out of scope and may be filled with
garbage by the time the tempfile code looks at it.  Let's
see if we can make it harder to get this wrong.

Since many callers need to allocate arbitrary numbers of
tempfiles, we can't rely on static storage as a general
solution. So we need to turn to the heap. We could just ask
all callers to pass us a heap variable, but that puts the
burden on them to call free() at the right time.

Instead, let's have the tempfile code handle the heap
allocation _and_ the deallocation (when the tempfile is
deactivated and removed from the list).

This changes the return value of all of the creation
functions. For the cleanup functions (delete and rename),
we'll add one extra bit of safety: instead of taking a
tempfile pointer, we'll take a pointer-to-pointer and set it
to NULL after freeing the object. This makes it safe to
double-call functions like delete_tempfile(), as the second
call treats the NULL input as a noop. Several callsites
follow this pattern.

The resulting patch does have a fair bit of noise, as each
caller needs to be converted to handle:

  1. Storing a pointer instead of the struct itself.

  2. Passing the pointer instead of taking the struct
     address.

  3. Handling a "struct tempfile *" return instead of a file
     descriptor.

We could play games to make this less noisy. For example, by
defining the tempfile like this:

  struct tempfile {
	struct heap_allocated_part_of_tempfile {
                int fd;
                ...etc
        } *actual_data;
  }

Callers would continue to have a "struct tempfile", and it
would be "active" only when the inner pointer was non-NULL.
But that just makes things more awkward in the long run.
There aren't that many callers, so we can simply bite
the bullet and adjust all of them. And the compiler makes it
easy for us to find them all.

Signed-off-by: Jeff King 
Signed-off-by: Junio C Hamano

lockfile: do not rollback lock on failed close

2017-09-06T08:19:53+00:00

Since the lockfile code is based on the tempfile code, it
has some of the same problems, including that close_lock_file()
erases the tempfile's filename buf, making it hard for the
caller to write a good error message.

In practice this comes up less for lockfiles than for
straight tempfiles, since we usually just report the
refname. But there is at least one buggy case in
write_ref_to_lockfile(). Besides, given the coupling between
the lockfile and tempfile modules, it's less confusing if
their close() functions have the same semantics.

Just as the previous commit did for close_tempfile(), let's
teach close_lock_file() and its wrapper close_ref() not to
rollback on error. And just as before, we'll give them new
"gently" names to catch any new callers that are added.

Signed-off-by: Jeff King 
Signed-off-by: Junio C Hamano

tempfile: do not delete tempfile on failed close

2017-09-06T08:19:53+00:00

When close_tempfile() fails, we delete the tempfile and
reset the fields of the tempfile struct. This makes it
easier for callers to return without cleaning up, but it
also makes this common pattern:

  if (close_tempfile(tempfile))
	return error_errno("error closing %s", tempfile->filename.buf);

wrong, because the "filename" field has been reset after the
failed close. And it's not easy to fix, as in many cases we
don't have another copy of the filename (e.g., if it was
created via one of the mks_tempfile functions, and we just
have the original template string).

Let's drop the feature that a failed close automatically
deletes the file. This puts the burden on the caller to do
the deletion themselves, but this isn't that big a deal.
Callers which do:

  if (write(...) || close_tempfile(...)) {
	delete_tempfile(...);
	return -1;
  }

already had to call delete when the write() failed, and so
aren't affected. Likewise, any caller which just calls die()
in the error path is OK; we'll delete the tempfile during
the atexit handler.

Because this patch changes the semantics of close_tempfile()
without changing its signature, all callers need to be
manually checked and converted to the new scheme. This patch
covers all in-tree callers, but there may be others for
not-yet-merged topics. To catch these, we rename the
function to close_tempfile_gently(), which will attract
compile-time attention to new callers. (Technically the
original could be considered "gentle" already in that it
didn't die() on errors, but this one is even more so).

Signed-off-by: Jeff King 
Signed-off-by: Junio C Hamano

lockfile: add a new method, is_lock_file_locked()

2017-05-23T05:29:54+00:00

It will soon prove useful.

Signed-off-by: Michael Haggerty 
Signed-off-by: Junio C Hamano

lockfile: move REPORT_ON_ERROR bit elsewhere

2016-12-27T17:12:09+00:00

There was LOCK_NO_DEREF defined as 2 = 1<<1 with the same value,
which was missed due to a huge comment block.  Deconflict by moving
the new one to 4 = 1<<2 for now.

Signed-off-by: Junio C Hamano

lockfile: LOCK_REPORT_ON_ERROR

2016-12-07T19:31:59+00:00

The "libify sequencer" topic stopped passing the die_on_error option
to hold_locked_index(), and this lost an error message from "git
merge --ff-only $commit" when there are competing updates in
progress.

The command still exits with a non-zero status, but that is not of
much help for an interactive user.  The last thing the command says
is "Updating $from..$to".  We used to follow it with a big error
message that makes it clear that "merge --ff-only" did not succeed.

What is sad is that we should have noticed this regression while
reviewing the change.  It was clear that the update to the
checkout_fast_forward() function made a failing hold_locked_index()
silent, but the only caller of the checkout_fast_forward() function
had this comment:

	    if (checkout_fast_forward(from, to, 1))
    -               exit(128); /* the callee should have complained already */
    +               return -1; /* the callee should have complained already */

which clearly contradicted the assumption X-<.

Add a new option LOCK_REPORT_ON_ERROR that can be passed instead of
LOCK_DIE_ON_ERROR to the hold_lock*() family of functions and teach
checkout_fast_forward() to use it to fix this regression.

After going thourgh all calls to hold_lock*() family of functions
that used to pass LOCK_DIE_ON_ERROR but were modified to pass 0 in
the "libify sequencer" topic "git show --first-parent 2a4062a4a8",
it appears that this is the only one that has become silent.  Many
others used to give detailed report that talked about "there may be
competing Git process running" but with the series merged they now
only give a single liner "Unable to lock ...", some of which may
have to be tweaked further, but at least they say something, unlike
the one this patch fixes.

Reported-by: Robbie Iannucci 
Signed-off-by: Junio C Hamano

mingw: ensure temporary file handles are not inherited by child processes

2016-08-23T16:09:55+00:00

When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent to O_CLOEXEC (which does not exist on
Windows), so let's just open temporary files with the O_CLOEXEC flag and
map that flag to O_NOINHERIT on Windows.

As Eric Wong pointed out, we need to be careful to handle the case where
the Linux headers used to compile Git support O_CLOEXEC but the Linux
kernel used to run Git does not: it returns an EINVAL.

This fixes the test that we just introduced to demonstrate the problem.

Signed-off-by: Ben Wijen 
Signed-off-by: Johannes Schindelin 
Signed-off-by: Junio C Hamano

lockfile: remove function "hold_lock_file_for_append"

2015-08-28T18:32:01+00:00

With 77b9b1d (add_to_alternates_file: don't add duplicate entries,
2015-08-10) the last caller of function "hold_lock_file_for_append"
has been removed, so we can remove the function as well.

Signed-off-by: Ralf Thielow 
Acked-by: Jeff King 
Signed-off-by: Junio C Hamano

tempfile: a new module for handling temporary files

2015-08-10T19:57:14+00:00

A lot of work went into defining the state diagram for lockfiles and
ensuring correct, race-resistant cleanup in all circumstances.

Most of that infrastructure can be applied directly to *any* temporary
file. So extract a new "tempfile" module from the "lockfile" module.
Reimplement lockfile on top of tempfile.

Subsequent commits will add more users of the new module.

Signed-off-by: Michael Haggerty 
Signed-off-by: Junio C Hamano