path: root/libc/EGLIBC.cross-building

                                                        -*- mode: text -*-

                        Cross-Compiling EGLIBC
                  Jim Blandy <jimb@codesourcery.com>


Introduction

Most GNU tools have a simple build procedure: you run their
'configure' script, and then you run 'make'.  Unfortunately, the
process of cross-compiling the GNU C library is quite a bit more
involved:

1) Build a cross-compiler, with certain facilities disabled.

2) Configure the C library using the compiler you built in step 1).
   Build a few of the C run-time object files, but not the rest of the
   library.  Install the library's header files and the run-time
   object files, and create a dummy libc.so.

3) Build a second cross-compiler, using the header files and object
   files you installed in step 2.

4) Configure, build, and install a fresh C library, using the compiler
   built in step 3.

5) Build a third cross-compiler, based on the C library built in step 4.

The reason for this complexity is that, although GCC and the GNU C
library are distributed separately, they are not actually independent
of each other: GCC requires the C library's headers and some object
files to compile its own libraries, while the C library depends on
GCC's libraries.  EGLIBC includes features and bug fixes to the stock
GNU C library that simplify this process, but the fundamental
interdependency stands.

In this document, we explain how to cross-compile an EGLIBC/GCC pair
from source.  Our intended audience is developers who are already
familiar with the GNU toolchain and comfortable working with
cross-development tools.  While we do present a worked example to
accompany the explanation, for clarity's sake we do not cover many of
the options available to cross-toolchain users.


Preparation

EGLIBC requires recent versions of the GNU binutils, GCC, and the
Linux kernel.  The web page <http://www.eglibc.org/prerequisites>
documents the current requirements, and lists patches needed for
certain target architectures.  As of this writing, EGLIBC required
binutils 2.17, GCC 4.1, and Linux 2.6.19.1.

First, let's set some variables, to simplify later commands.  We'll
build EGLIBC and GCC for a PowerPC target, known to the Linux kernel
as 'powerpc', and we'll do the build on an Intel Linux box:

    $ build=i686-pc-linux-gnu
    $ host=$build
    $ target=powerpc-none-linux-gnu
    $ linux_arch=powerpc

We're using the aforementioned versions of Binutils, GCC, and Linux:

    $ binutilsv=binutils-2.17
    $ gccv=gcc-4.1.1
    $ linuxv=linux-2.6.20

We're carrying out the entire process under '~/cross-build', which
contains unpacked source trees:

    $ top=$HOME/cross-build/ppc
    $ src=$HOME/cross-build/src
    $ ls $src
    binutils-2.17  gcc-4.1.1  libc  linux-2.6.20

We're going to place our build directories in a subdirectory 'obj',
we'll install the cross-development toolchain in 'tools', and we'll
place our sysroot (containing files to be installed on the target
system) in 'sysroot':

    $ obj=$top/obj
    $ tools=$top/tools
    $ sysroot=$top/sysroot


Binutils

Configuring and building binutils for the target is straightforward:

    $ mkdir -p $obj/binutils
    $ cd $obj/binutils
    $ $src/$binutilsv/configure \
    >     --target=$target \
    >     --prefix=$tools \
    >     --with-sysroot=$sysroot 
    $ make
    $ make install


The First GCC

For our work, we need a cross-compiler targeting a PowerPC Linux
system.  However, that configuration includes the shared library
'libgcc_s.so', which is compiled against the EGLIBC headers (which we
haven't installed yet) and linked against 'libc.so' (which we haven't
built yet).

Fortunately, there are configuration options for GCC which tell it not
to build 'libgcc_s.so'.  The '--without-headers' option is supposed to
take care of this, but its implementation is incomplete, so you must
also configure with the '--with-newlib' option.  While '--with-newlib'
appears to mean "Use the Newlib C library", its effect is to tell the
GCC build machinery, "Don't assume there is a C library available."

We also need to disable some of the libraries that would normally be
built along with GCC, and specify that only the compiler for the C
language is needed.

So, we create a build directory, configure, make, and install.

    $ mkdir -p $obj/gcc1
    $ cd $obj/gcc1
    $ $src/$gccv/configure \
    >     --target=$target \
    >     --prefix=$tools \
    >     --without-headers --with-newlib \
    >     --disable-shared --disable-threads --disable-libssp \
    >     --disable-libgomp --disable-libmudflap \
    >     --enable-languages=c
    $ PATH=$tools/bin:$PATH make
    $ PATH=$tools/bin:$PATH make install


Linux Kernel Headers

To configure EGLIBC, we also need Linux kernel headers in place.
Fortunately, the Linux makefiles have a target that installs them for
us.  Since the process does modify the source tree a bit, we make a
copy first:

    $ cp -r $src/$linuxv $obj/linux
    $ cd $obj/linux

Now we're ready to install the headers into the sysroot:

    $ PATH=$tools/bin:$PATH \
    > make headers_install \
    >      ARCH=$linux_arch CROSS_COMPILE=$target- \
    >      INSTALL_HDR_PATH=$sysroot/usr


EGLIBC Headers and Preliminary Objects

Using the cross-compiler we've just built, we can now configure EGLIBC
well enough to install the headers and build the object files that the
full cross-compiler will need:

    $ mkdir -p $obj/eglibc-headers
    $ cd $obj/eglibc-headers
    $ BUILD_CC=gcc \
    > CC=$tools/bin/$target-gcc \
    > CXX=$tools/bin/$target-g++ \
    > AR=$tools/bin/$target-ar \
    > RANLIB=$tools/bin/$target-ranlib \
    > $src/libc/configure \
    >     --prefix=/usr \
    >     --with-headers=$sysroot/usr/include \
    >     --build=$build \
    >     --host=$target \
    >     --disable-profile --without-gd --without-cvs --enable-add-ons

The option '--prefix=/usr' may look strange, but you should never
configure EGLIBC with a prefix other than '/usr': in various places,
EGLIBC's build system checks whether the prefix is '/usr', and does
special handling only if that is the case.  Unless you use this
prefix, you will get a sysroot that does not use the standard Linux
directory layouts and cannot be used as a basis for the root
filesystem on your target system compatibly with normal GLIBC
installations.

The '--with-headers' option tells EGLIBC where the Linux headers have
been installed.  

We can now use the 'install-headers' makefile target to install the
headers:

    $ make install-headers install_root=$sysroot \
    >                      install-bootstrap-headers=yes

The 'install_root' variable indicates where the files should actually
be installed; its value is treated as the parent of the '--prefix'
directory we passed to the configure script, so the headers will go in
'$sysroot/usr/include'.  The 'install-bootstrap-headers' variable
requests special handling for certain tricky header files.

Next, there are a few object files needed to link shared libraries,
which we build and install by hand:

    $ mkdir -p $sysroot/usr/lib
    $ make csu/subdir_lib
    $ cp csu/crt1.o csu/crti.o csu/crtn.o $sysroot/usr/lib

Finally, 'libgcc_s.so' requires a 'libc.so' to link against.  However,
since we will never actually execute its code, it doesn't matter what
it contains.  So, treating '/dev/null' as a C source file, we produce
a dummy 'libc.so' in one step:

    $ $tools/bin/$target-gcc -nostdlib -nostartfiles -shared -x c /dev/null \
    >                        -o $sysroot/usr/lib/libc.so


The Second GCC

With the EGLIBC headers and selected object files installed, we can
now build a GCC that is capable of compiling EGLIBC.  We configure,
build, and install the second GCC, again building only the C compiler,
and avoiding libraries we won't use:

    $ mkdir -p $obj/gcc2
    $ cd $obj/gcc2
    $ $src/$gccv/configure \
    >     --target=$target \
    >     --prefix=$tools \
    >     --with-sysroot=$sysroot \
    >     --disable-libssp --disable-libgomp --disable-libmudflap \
    >     --enable-languages=c
    $ PATH=$tools/bin:$PATH make
    $ PATH=$tools/bin:$PATH make install


EGLIBC, Complete

With the second compiler built and installed, we're now ready for the
full EGLIBC build:

    $ mkdir -p $obj/eglibc
    $ cd $obj/eglibc
    $ BUILD_CC=gcc \
    > CC=$tools/bin/$target-gcc \
    > CXX=$tools/bin/$target-g++ \
    > AR=$tools/bin/$target-ar \
    > RANLIB=$tools/bin/$target-ranlib \
    > $src/libc/configure \
    >     --prefix=/usr \
    >     --with-headers=$sysroot/usr/include \
    >     --build=$build \
    >     --host=$target \
    >     --disable-profile --without-gd --without-cvs --enable-add-ons
    $ PATH=$tools/bin:$PATH make
    $ PATH=$tools/bin:$PATH make install install_root=$sysroot

At this point, we have a complete EGLIBC installation in '$sysroot',
with header files, library files, and most of the C runtime startup
files in place.


The Third GCC

Finally, we recompile GCC against this full installation, enabling
whatever languages and libraries we would like to use:

    $ mkdir -p $obj/gcc3
    $ cd $obj/gcc3
    $ $src/$gccv/configure \
    >     --target=$target \
    >     --prefix=$tools \
    >     --with-sysroot=$sysroot \
    >     --enable-__cxa_atexit \
    >     --disable-libssp --disable-libgomp --disable-libmudflap \
    >     --enable-languages=c,c++
    $ PATH=$tools/bin:$PATH make
    $ PATH=$tools/bin:$PATH make install

The '--enable-__cxa_atexit' option tells GCC what sort of C++
destructor support to expect from the C library; it's required with
EGLIBC.

And since GCC's installation process isn't designed to help construct
sysroot trees, we must manually copy certain libraries into place in
the sysroot.

    $ cp -d $tools/$target/lib/libgcc_s.so* $sysroot/lib
    $ cp -d $tools/$target/lib/libstdc++.so* $sysroot/usr/lib


Trying Things Out

At this point, '$tools' contains a cross toolchain ready to use
the EGLIBC installation in '$sysroot':

    $ cat > hello.c <<EOF
    > #include <stdio.h>
    > int
    > main (int argc, char **argv)
    > {
    >   puts ("Hello, world!");
    >   return 0;
    > }
    > EOF
    $ $tools/bin/$target-gcc -Wall hello.c -o hello
    $ cat > c++-hello.cc <<EOF
    > #include <iostream>
    > int
    > main (int argc, char **argv)
    > {
    >   std::cout << "Hello, C++ world!" << std::endl;
    >   return 0;
    > }
    > EOF
    $ $tools/bin/$target-g++ -Wall c++-hello.cc -o c++-hello


We can use 'readelf' to verify that these are indeed executables for
our target, using our dynamic linker:

    $ $tools/bin/$target-readelf -hl hello
    ELF Header:
    ...
      Type:                              EXEC (Executable file)
      Machine:                           PowerPC

    ...
    Program Headers:
      Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
      PHDR           0x000034 0x10000034 0x10000034 0x00100 0x00100 R E 0x4
      INTERP         0x000134 0x10000134 0x10000134 0x0000d 0x0000d R   0x1
          [Requesting program interpreter: /lib/ld.so.1]
      LOAD           0x000000 0x10000000 0x10000000 0x008f0 0x008f0 R E 0x10000
    ...

Looking at the dynamic section of the installed 'libgcc_s.so', we see
that the 'NEEDED' entry for the C library does include the '.6'
suffix, indicating that was linked against our fully build EGLIBC, and
not our dummy 'libc.so':

    $ $tools/bin/$target-readelf -d $sysroot/lib/libgcc_s.so.1
    Dynamic section at offset 0x1083c contains 24 entries:
      Tag        Type                         Name/Value
     0x00000001 (NEEDED)                     Shared library: [libc.so.6]
     0x00000001 (NEEDED)                     Shared library: [ld.so.1]
     0x0000000e (SONAME)                     Library soname: [libgcc_s.so.1]
    ...


And on the target machine, we can run our programs:

    $ $sysroot/lib/ld.so.1 --library-path $sysroot/lib:$sysroot/usr/lib \
    > ./hello
    Hello, world!
    $ $sysroot/lib/ld.so.1 --library-path $sysroot/lib:$sysroot/usr/lib \
    > ./c++-hello
    Hello, C++ world!