AC_FUNC_SETPGRP: Don’t depend on the return type of setpgrp.

AC_FUNC_SETPGRP determines whether you have the historic BSD setpgrp,
which takes two arguments and returns int, or the historic POSIX
setpgrp, which takes no arguments and returns int.  Solaris has yet a
third variant, which takes no arguments and returns a pid_t (the new
process group ID).  This difference causes AC_FUNC_SETPGRP’s test
program to fail to compile under AC_LANG([C++]), which in turn causes
the macro to report that setpgrp does take arguments, which is wrong.

It is not worth adding a new result #define for this variant,
since *all* forms of setpgrp are deprecated in favor of setpgid, which
is old enough that it can be used unconditionally.  However, it is
worth documenting that this variant exists, and fixing AC_FUNC_SETPGRP
to produce the right value for its existing result #define on Solaris
with C++.

* lib/autoconf/functions.m4 (AC_FUNC_SETPGRP): Redesign test program to
  not depend on the return type of setpgrp.
* doc/autoconf.texi (AC_FUNC_SETPGRP): Mention that the macro does not
  check for the Solaris variant of setpgrp that returns pid_t.  Change
  programming advice to recommend use of setpgid.

2 files changed, 15 insertions, 12 deletions

diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index e8abc9ff..9ba182af 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -5599,13 +5599,17 @@ If @code{setpgrp} takes no argument (the Posix version), define
 @code{SETPGRP_VOID}.  Otherwise, it is the BSD version, which takes
 two process IDs as arguments.  This macro does not check whether
 @code{setpgrp} exists at all; if you need to work in that situation,
-first call @code{AC_CHECK_FUNC} for @code{setpgrp}.
+first call @code{AC_CHECK_FUNC} for @code{setpgrp}.  This macro also
+does not check for the Solaris variant of @code{setpgrp}, which returns
+a @code{pid_t} instead of an @code{int}; portable code should only use
+the return value by comparing it against @code{-1} to check for errors.
 
 The result of this macro is cached in the @code{ac_cv_func_setpgrp_void}
 variable.
 
-This macro is obsolescent, as current systems have a @code{setpgrp}
-whose signature conforms to Posix.  New programs need not use this macro.
+This macro is obsolescent, as all forms of @code{setpgrp} are also
+obsolescent.  New programs should use the Posix function @code{setpgid},
+which takes two process IDs as arguments (like the BSD @code{setpgrp}).
 @end defmac
 
 @defmac AC_FUNC_STAT
diff --git a/lib/autoconf/functions.m4 b/lib/autoconf/functions.m4
index a91eeebc..49aa5c58 100644
--- a/lib/autoconf/functions.m4
+++ b/lib/autoconf/functions.m4
@@ -1541,17 +1541,16 @@ rm -rf conftest*
 # AC_FUNC_SETPGRP
 # ---------------
 AC_DEFUN([AC_FUNC_SETPGRP],
-[AC_CACHE_CHECK(whether setpgrp takes no argument, ac_cv_func_setpgrp_void,
-   [AC_COMPILE_IFELSE(
-      [AC_LANG_PROGRAM(
-	 [[#include <unistd.h>
-	   static int (*p) (void) = setpgrp;]],
-	 [[return setpgrp ();]])],
-      [ac_cv_func_setpgrp_void=yes],
-      [ac_cv_func_setpgrp_void=no])])
+[AC_CACHE_CHECK(whether setpgrp requires zero arguments,
+ ac_cv_func_setpgrp_void,
+[# Call it with two arguments.
+AC_COMPILE_IFELSE([AC_LANG_PROGRAM([AC_INCLUDES_DEFAULT], [setpgrp(0, 0);])],
+                  [ac_cv_func_setpgrp_void=no],
+                  [ac_cv_func_setpgrp_void=yes])
+])
 if test $ac_cv_func_setpgrp_void = yes; then
   AC_DEFINE(SETPGRP_VOID, 1,
-	    [Define to 1 if the `setpgrp' function takes no argument.])
+	    [Define to 1 if the `setpgrp' function requires zero arguments.])
 fi
 ])# AC_FUNC_SETPGRP