@node Invoking gnulib-tool @chapter Invoking gnulib-tool @c Copyright (C) 2005, 2006, 2007 Free Software Foundation, Inc. @c Permission is granted to copy, distribute and/or modify this document @c under the terms of the GNU Free Documentation License, Version 1.2 or @c any later version published by the Free Software Foundation; with no @c Invariant Sections, with no Front-Cover Texts, and with no Back-Cover @c Texts. A copy of the license is included in the ``GNU Free @c Documentation License'' file as part of this distribution. @pindex gnulib-tool @cindex invoking @command{gnulib-tool} The @command{gnulib-tool} command is the recommended way to import Gnulib modules. It is possible to borrow Gnulib modules in a package without using @command{gnulib-tool}, relying only on the meta-information stored in the @file{modules/*} files, but with a growing number of modules this becomes tedious. @command{gnulib-tool} simplifies the management of source files, @file{Makefile.am}s and @file{configure.ac} in packages incorporating Gnulib modules. Run @samp{gnulib-tool --help} for information. To get familiar with @command{gnulib-tool} without affecting your sources, you can also try some commands with the option @samp{--dry-run}; then @code{gnulib-tool} will only report which actions it would perform in a real run without changing anything. @menu * Initial import:: First import of Gnulib modules. * Modified imports:: Changing the import specification. * Simple update:: Tracking Gnulib development. * CVS Issues:: Integration with CVS. @end menu @node Initial import @section Initial import @cindex initial import Gnulib assumes your project uses Autoconf and Automake. Invoking @samp{gnulib-tool --import} will copy source files, create a @file{Makefile.am} to build them, generate a file @file{gnulib-comp.m4} with Autoconf M4 macro declarations used by @file{configure.ac}, and generate a file @file{gnulib-cache.m4} containing the cached specification of how Gnulib is used. Our example will be a library that uses Autoconf, Automake and Libtool. It calls @code{strdup}, and you wish to use gnulib to make the package portable to C89 and C99 (which don't have @code{strdup}). @example ~/src/libfoo$ gnulib-tool --import strdup Module list with included dependencies: absolute-header extensions strdup string File list: lib/dummy.c lib/strdup.c lib/string_.h m4/absolute-header.m4 m4/extensions.m4 m4/gnulib-common.m4 m4/strdup.m4 m4/string_h.m4 Creating directory ./lib Creating directory ./m4 Copying file lib/dummy.c Copying file lib/strdup.c Copying file lib/string_.h Copying file m4/absolute-header.m4 Copying file m4/extensions.m4 Copying file m4/gnulib-common.m4 Copying file m4/gnulib-tool.m4 Copying file m4/strdup.m4 Copying file m4/string_h.m4 Creating lib/Makefile.am Creating m4/gnulib-cache.m4 Creating m4/gnulib-comp.m4 Finished. You may need to add #include directives for the following .h files. #include Don't forget to - add "lib/Makefile" to AC_CONFIG_FILES in ./configure.ac, - mention "lib" in SUBDIRS in Makefile.am, - mention "-I m4" in ACLOCAL_AMFLAGS in Makefile.am, - invoke gl_EARLY in ./configure.ac, right after AC_PROG_CC, - invoke gl_INIT in ./configure.ac. ~/src/libfoo$ @end example By default, the source code is copied into @file{lib/} and the M4 macros in @file{m4/}. You can override these paths by using @code{--source-base=DIRECTORY} and @code{--m4-base=DIRECTORY}. Some modules also provide other files necessary for building. These files are copied into the directory specified by @samp{AC_CONFIG_AUX_DIR} in @file{configure.ac} or by the @code{--aux-dir=DIRECTORY} option. If neither is specified, the current directory is assumed. @code{gnulib-tool} can make symbolic links instead of copying the source files. The option to specify for this is @samp{--symlink}, or @samp{-s} for short. This can be useful to save a few kilobytes of disk space. But it is likely to introduce bugs when @code{gnulib} is updated; it is more reliable to use @samp{gnulib-tool --update} (see below) to update to newer versions of @code{gnulib}. Furthermore it requires extra effort to create self-contained tarballs, and it may disturb some mechanism the maintainer applies to the sources. For these reasons, this option is generally discouraged. @code{gnulib-tool} will overwrite any pre-existing files, in particular @file{Makefile.am}. Unfortunately, separating the generated @file{Makefile.am} content (for building the gnulib library) into a separate file, say @file{gnulib.mk}, that could be included by your handwritten @file{Makefile.am} is not possible, due to how variable assignments are handled by Automake. Consequently, it is a good idea to choose directories that are not already used by your projects, to separate gnulib imported files from your own files. This approach is also useful if you want to avoid conflicts between other tools (e.g., @code{gettextize} that also copy M4 files into your package. Simon Josefsson successfully uses a source base of @file{gl/}, and a M4 base of @file{gl/m4/}, in several packages. After the @samp{--import} option on the command line comes the list of Gnulib modules that you want to incorporate in your package. The names of the modules coincide with the filenames in Gnulib's @file{modules/} directory. Some Gnulib modules depend on other Gnulib modules. @code{gnulib-tool} will automatically add the needed modules as well; you need not list them explicitly. @code{gnulib-tool} will also memorize which dependent modules it has added, so that when someday a dependency is dropped, the implicitly added module is dropped as well (unless you have explicitly requested that module). If you want to cut a dependency, i.e., not add a module although one of your requested modules depends on it, you may use the option @samp{--avoid=@var{module}} to do so. Multiple uses of this option are possible. Of course, you will then need to implement the same interface as the removed module. A few manual steps are required to finish the initial import. @code{gnulib-tool} printed a summary of these steps. First, you must ensure Autoconf can find the macro definitions in @file{gnulib-comp.m4}. Use the @code{ACLOCAL_AMFLAGS} specifier in your top-level @file{Makefile.am} file, as in: @example ACLOCAL_AMFLAGS = -I m4 @end example You are now ready to call the M4 macros in @code{gnulib-comp.m4} from @file{configure.ac}. The macro @code{gl_EARLY} must be called as soon as possible after verifying that the C compiler is working. Typically, this is immediately after @code{AC_PROG_CC}, as in: @example ... AC_PROG_CC gl_EARLY ... @end example The core part of the gnulib checks are done by the macro @code{gl_INIT}. Place it further down in the file, typically where you normally check for header files or functions. It must come after other checks which may affect the compiler invocation, such as @code{AC_MINIX}. For example: @example ... # For gnulib. gl_INIT ... @end example @code{gl_INIT} will in turn call the macros related with the gnulib functions, be it specific gnulib macros, like @code{gl_FUNC_ALLOCA} or autoconf or automake macros like @code{AC_FUNC_ALLOCA} or @code{AM_FUNC_GETLINE}. So there is no need to call those macros yourself when you use the corresponding gnulib modules. You must also make sure that the gnulib library is built. Add the @code{Makefile} in the gnulib source base directory to @code{AC_CONFIG_FILES}, as in: @example AC_CONFIG_FILES(... lib/Makefile ...) @end example You must also make sure that @code{make} will recurse into the gnulib directory. To achieve this, add the gnulib source base directory to a @code{SUBDIRS} Makefile.am statement, as in: @example SUBDIRS = lib @end example or if you, more likely, already have a few entries in @code{SUBDIRS}, you can add something like: @example SUBDIRS += lib @end example Finally, you have to add compiler and linker flags in the appropriate source directories, so that you can make use of the gnulib library. Since some modules (@samp{getopt}, for example) may copy files into the build directory, @file{top_builddir/lib} is needed as well as @file{top_srcdir/lib}. For example: @example ... AM_CPPFLAGS = -I$(top_srcdir)/lib -I$(top_builddir)/lib ... LDADD = lib/libgnu.a ... @end example Don't forget to @code{#include} the various header files. In this example, you would need to make sure that @samp{#include } is evaluated when compiling all source code files, that want to make use of @code{strdup}. In the usual case where Autoconf is creating a @file{config.h} file, you should include @file{config.h} first, before any other include file. That way, for example, if @file{config.h} defines @samp{restrict} to be the empty string on a pre-C99 host, or a macro like @samp{_FILE_OFFSET_BITS} that affects the layout of data structures, the definition is consistent for all include files. Also, on some platforms macros like @samp{_FILE_OFFSET_BITS} and @samp{_GNU_SOURCE} may be ineffective, or may have only a limited effect, if defined after the first system header file is included. A final word of warning: Gnulib currently assumes it will be responsible for @emph{all} functions that end up in the Autoconf @code{@@LIBOBJS@@} variables (and/or @code{@@LTLIBOBJS@@} if using Libtool), e.g., those specified in @code{AC_REPLACE_FUNCS} in your @file{configure.ac}. Therefore, if you have any functions which are not covered by Gnulib which need that treatment, you have to essentially reimplement AC_REPLACE_FUNCS using different names; for an example, see the Findutils sources. Perhaps this will be improved in the future. @node Modified imports @section Modified imports You can at any moment decide to use Gnulib differently than the last time. If you only want to use more Gnulib modules, simply invoke @command{gnulib-tool --import @var{new-modules}}. @code{gnulib-tool} remembers which modules were used last time. The list of modules that you pass after @samp{--import} is @emph{added} to the previous list of modules. For most changes, such as added or removed modules, or even different choices of @samp{--lib}, @samp{--source-base} or @samp{--aux-dir}, there are two ways to perform the change. The standard way is to modify manually the file @file{gnulib-cache.m4} in the M4 macros directory, then launch @samp{gnulib-tool --import}. The other way is to call @command{gnulib-tool} again, with the changed command-line options. Note that this doesn't let you remove modules, because as you just learned, the list of modules is always cumulated. Also this way is often impractical, because you don't remember the way you invoked @code{gnulib-tool} last time. The only change for which this doesn't work is a change of the @samp{--m4-base} directory. Because, when you pass a different value of @samp{--m4-base}, @code{gnulib-tool} will not find the previous @file{gnulib-cache.m4} file any more... A possible solution is to manually copy the @file{gnulib-cache.m4} into the new M4 macro directory. In the @file{gnulib-cache.m4}, the macros have the following meaning: @table @code @item gl_MODULES The argument is a space separated list of the requested modules, not including dependencies. @item gl_AVOID The argument is a space separated list of modules that should not be used, even if they occur as dependencies. Corresponds to the @samp{--avoid} command line argument. @item gl_SOURCE_BASE The argument is the relative file name of the directory containing the gnulib source files (mostly *.c and *.h files). Corresponds to the @samp{--source-base} command line argument. @item gl_M4_BASE The argument is the relative file name of the directory containing the gnulib M4 macros (*.m4 files). Corresponds to the @samp{--m4-base} command line argument. @item gl_TESTS_BASE The argument is the relative file name of the directory containing the gnulib unit test files. Corresponds to the @samp{--tests-base} command line argument. @item gl_LIB The argument is the name of the library to be created. Corresponds to the @samp{--lib} command line argument. @item gl_LGPL The presence of this macro corresponds to the @samp{--lgpl} command line argument. It takes no arguments. @item gl_LIBTOOL The presence of this macro corresponds to the @samp{--libtool} command line argument and to the absence of the @samp{--no-libtool} command line argument. It takes no arguments. @item gl_MACRO_PREFIX The argument is the prefix to use for macros in the @file{gnulib-comp.m4} file. Corresponds to the @samp{--macro-prefix} command line argument. @end table @node Simple update @section Simple update When you want to update to a more recent version of Gnulib, without changing the list of modules or other parameters, a simple call does it: @smallexample $ gnulib-tool --import @end smallexample This will create, update or remove files, as needed. @node CVS Issues @section CVS Issues All files created by @code{gnulib-tool}, except @file{gnulib-cache.m4}, should be treated like generated source files, like for example a @file{parser.c} file is generated from @file{parser.y}. @itemize @item In projects which commit all source files, whether generated or not, into CVS, the @code{gnulib-tool} generated files should all be committed. Gnulib also contains files generated by @command{make} (and removed by @code{make clean}, using information determined by @command{configure} They should not be checked into CVS, but instead added to @file{.cvsignore}. When you have a Gnulib source file of the form @file{lib/foo_.h}, the corresponding @file{lib/foo.h} is such a file. @item In projects which customarily omit from the CVS all files that generated from other source files, all these files and directories would not be added into CVS. The only file that must be added to CVS is @file{gnulib-cache.m4} in the M4 macros directory. Also, the script for restoring files not in CVS, customarily called @file{autogen.sh} or @file{bootstrap.sh}, will typically contain the statement for restoring the omitted files: @smallexample $ gnulib-tool --update @end smallexample The @samp{--update} option operates much like the @samp{--import} option, but it does not offer the possibility to change the way Gnulib is used. Also it does not report in the ChangeLogs the files that it had to add because they were missing. @end itemize The same holds for other version control systems than CVS, such as @samp{git} or @samp{svn}.