gspawn: Optimize with posix_spawn codepath

When the amount of free memory on the system is somewhat low, gnome-shell
will sometimes fail to launch apps, reporting the error:
  fork(): Cannot allocate memory

fork() is failing here because while cloning the process virtual address
space, Linux worries that the thread being forked may end up COWing the
entire address space of the parent process (gnome-shell, which is
memory-hungry), and there is not enough free memory to permit that to
happen.

In this case we are simply calling fork() in order to quickly call exec(),
which will throw away the entirity of the duplicated VM, so we should
look for ways to avoid the overcommit check.

The well known solution to this is to use clone(CLONE_VM) or vfork(), which
completely avoids creating a new memory address space for the child.
However, that comes with a bunch of caveats and complications:

  https://gist.github.com/nicowilliams/a8a07b0fc75df05f684c23c18d7db234
  https://ewontfix.com/7/

In 2016, glibc's posix_spawn() was rewritten to use this approach
while also resolving the concerns.
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9ff72da471a509a8c19791efe469f47fa6977410

I experimented with a similar approach in glib, but it was not practical
because glibc has several items of important internal knowledge (such as
knowing which signals should be given special treatment because they are
NPTL implementation details) that are not cleanly exposed elsewhere.

Instead, this patch adapts the gspawn code to use posix_spawn() where
possible, which will reap the benefits of that implementation.
The posix_spawn API is more limited than the gspawn API though,
partly due to natural limitations of using CLONE_VM, so the posix_spawn
path is added as a separate codepath which is only executed when the
conditions are right. Callers such as gnome-shell will have to be modified
to meet these conditions, such as not having a child_setup function.

In addition to allowing for the gnome-shell "Cannot allocate memory"
failure to be avoided, this should result in a general speedup in this
area, because fork()'s behaviour of cloning the entire VM space
has a cost which is now avoided. posix_spawn() has also recently
been optimized on OpenSolaris as the most performant way to spawn
a child process.

1 files changed, 5 insertions, 0 deletions

diff --git a/meson.build b/meson.build
index 6abc2e6e2..9e6aa4707 100644
--- a/meson.build
+++ b/meson.build
@@ -481,6 +481,11 @@ if cc.has_function('posix_memalign', prefix : '#include <stdlib.h>')
   glib_conf.set('HAVE_POSIX_MEMALIGN', 1)
 endif
 
+# Check that posix_spawn() is usable; must use header
+if cc.has_function('posix_spawn', prefix : '#include <spawn.h>')
+  glib_conf.set('HAVE_POSIX_SPAWN', 1)
+endif
+
 # Check whether strerror_r returns char *
 if have_func_strerror_r
   if cc.compiles('''#define _GNU_SOURCE