Gnu Toolchain/GLIBC/Building GLIBC

From Devpit
Jump to: navigation, search



This page was originally created by Ryan S. Arnold aka RandomTask.


Due to issues obvious to anyone who has ever tried replacing glibc on a running system ( *KABOOM* ) it is suggested that building and installing GLIBC take place in a chroot environment. The focus of this section is on building with and installing over GLIBC in an '/opt/biarch/' toolchain within a chroot environment. A bootstrap install over the base C library install in a chroot env and over the actual base C library on the filesystem will be discussed later in this document. Additionally, patching and installing distro GLIBC source RPM's will be discussed.


The GLIBC build framework depends upon kernel headers of the operating system the built GLIBC is to execute on. In order to create headers that GLIBC can use from a particular kernel download you kernel sources and then do the following:

make headers_install ARCH=powerpc INSTALL_HDR_PATH=$your_header_dir

Then when you configure GLIBC you can pass --with-headers=$your_header_dir.

Build and Install GLIBC in a chroot env

Building and installing GLIBC into a chroot environment has a number of advantages. The main one being not having to worry about blowing away a running machine by replacing the C libraries out from underneath it. The second benefit is that if you screwed up something in GLIBC you haven't damaged a real GLIBC install.

My personal preference for doing GLIBC work is to build and install in a chroot environment. I use a copy of the biarch toolchain in /opt/biarch for building the libraries and I install over the libraries in that location (from within a safely jailed [read: copied] chroot environment).

Early caveats

  • One should build and install a full 32-bit and 64-bit toolchain in /opt/biarch to provide a basis for the chroot environment.
  • Then one can simply enter the chroot environment and replace GLIBC in the /opt/biarch toolchain.
  • All absolute paths in this tutorial that begin with '/' are predicated on the chroot environment root. In our later examples the file system image that we'll be using to chroot into will be:

So, when refering to /opt/biarch/example-20060301 we'll actually be refering to the file system image that we'll be chrooting into located at


And not the real biarch toolchain that was originally installed in the system's root '/opt/biarch'


I like to: locate the source, configure, and build everything required for the toolchain in one structured directory. This tutorial, therefore, assumes that almost everything is done in the following directory:


This is where the entire toolchain can be built if needed, including 'gcc', 'binutils', 'gdb', 'glibc', etc.

«user@host»:~/toolchain§ ls
binutils  builds  gcc  gdb  glibc  kernelheaders

The package names: 'binutils', 'gcc', 'gdb', and 'glibc' are parent directories for several package source checkpoints. The 'build' directory is where I store various builds of these 'glibc' source checkpoints.

  • Warning: Installation is not done into this directory. The targets for installation are discussed in the following sections.

Getting the Source

Install the GLIBC source in /home/user/toolchain/glibc/ by doing the cvs checkout in that directory.

You can find GLIBC source information on the GLIBC sources webpage.


"anoncvs" is the password
cvs -z 9 -d login


cvs -z 9 -d co libc

Checkout by date

cvs -z 9 -d co -D "2006/02/26 00:00:00 GMT" -P libc

To get the ports

cd libc
cvs -z 9 -d co ports

The previous commands check out the GLIBC source and put it into a directory named 'libc'. I like to rename this to a dated glibc directory e.g.

«user@host»:~/toolchain/glibc/§ mv libc glibc-20060317

patching 32-bit

The following patch is required for successfully building a 32-bit library due to a disagreement between GCC and GLIBC:


diff -urN libc24-cvstip-20050613/elf/dynamic-link.h libc24/elf/dynamic-link.h
--- libc24-cvstip-20050613/elf/dynamic-link.h   2005-03-15 16:57:25.000000000 -0600
+++ libc24/elf/dynamic-link.h   2005-06-14 13:54:24.349406848 -0500
@@ -173,8 +173,8 @@
   assert (info[DT_FLAGS] == NULL
          || info[DT_FLAGS]->d_un.d_val == DF_BIND_NOW);
   /* Flags must not be set for  */
-  assert (info[DT_RUNPATH] == NULL);
-  assert (info[DT_RPATH] == NULL);
+  info[DT_RUNPATH] = NULL;
+  info[DT_RPATH] = NULL;
   if (info[DT_FLAGS] != NULL)

Attempt dry-run patch:

«user@host»:~/toolchain/glibc/glibc-tree§ patch --dry-run -p1 < ppc32-dynamic-link-20050613.patch

Actually patch sources:

«user@host»:~/toolchain/glibc/glibc-tree§ patch -p1 < ppc32-dynamic-link-20050613.patch

create 32-bit and 64-bit build directories

GLIBC requires that it be built in a directory other than the source directory. Create the following directories:


'32' and '64' refer to 32-bit and 64-bit builds, respectively. '24' refers to GLIBC '2.4'.

kernel headers

Building GLIBC requires that you have access to the system kernel headers. Referencing the source in "/usr/src/linux" is not adequate. GLIBC requires that the headers be organized in a particular fashion.

  • Firstly, you need a place where different kernel headers will be stored. Create the following directory:
  • Put the following script in your user's bin directory to add it to your execution path, or elsewhere in your execution path.


if [ "$kernel_src" == "" ]; then
     echo "Missing first parameter (kernel source directory)."
elif [ ! -d $kernel_src ]; then
     echo "$kernel_src is not a directory"
elif [ ! -e "$kernel_src/Makefile" ]; then
     echo "No Makefile found at $kernel_src.  Can't determine kernel version."

if [ "$2" == "" ]; then
     echo "Missing second parameter (kernel header destination directory)."

# Strip a trailing '/'
kernel_headers=`echo "$2" | sed 's/\/$//'`

kernel_version=`sed -n 's/^VERSION \= \(\S*\)$/\1/p' < $1/Makefile`
kernel_patchlevel=`sed -n 's/^PATCHLEVEL \= \(\S*\)$/\1/p' < $1/Makefile`
kernel_sublevel=`sed -n 's/^SUBLEVEL \= \(\S*\)$/\1/p' < $1/Makefile`

#echo "Headerizing a ${kernel_version}.${kernel_patchlevel}.${kernel_sublevel} kernel source."
#echo "and placing in $kernel_headers."

#save what is presently in the target directory to a backup directory
if [ -d $kernel_headers ]; then
     backdate=`date +"%Y%m%d.%H%M%S"`
     mv "${kernel_headers}" "${kernel_headers}-${backdate}.back"

mkdir -p "${kernel_headers}"/include/
cp -a "${kernel_src}"/include/linux "${kernel_headers}"/include/linux
cp -a "${kernel_src}"/include/asm-generic "${kernel_headers}"/include/asm-generic

#2.6.15+ kernels use asm-powerpc
if [ $kernel_sublevel -ge 15 ]; then
     cp -a "${kernel_src}"/include/asm-powerpc "${kernel_headers}"/include/asm-powerpc
     ln -s "${kernel_headers}"/include/asm-powerpc "${kernel_headers}"/include/asm
else # less-than 2.6.15 kernels use /asm-ppc64
     mkdir -p "${kernel_headers}"/include/asm
     cp -a "${kernel_src}"/include/asm-ppc "${kernel_headers}"/include/asm-ppc
     cp -a "${kernel_src}"/include/asm-ppc64 "${kernel_headers}"/include/asm-ppc64
          cd "${kernel_headers}"/include/asm-ppc64
          header_list="$(/bin/ls *.h)"
          cd ../asm
          for header in ${header_list}; do
               rm -f ${header}
               macro=`echo ${header} | sed 'y/.abcdefghijklmnopqrstuvwzyz/_ABCDEFGHIJKLMNOPQRSTUVWXYZ/'`
               cat >> ${header} <<EOF
#ifndef __ASM_STUB_${macro}__
# define __ASM_STUB_${macro}__
# if defined __powerpc64__
#  include <asm-ppc64/${header}>
# elif defined __powerpc__
#  include <asm-ppc/${header}>
# endif

You can invoke the previous script against the system kernel sources in /usr/src/linux and it will create a kernel headers directory that will work for GLIBC builds. This is what we'll do for the purpose of our how-to: /usr/src/linux /home/user/toolchain/kernelheaders/kernel-265

Additionally you can direct it to the patch of a linux kernel tree not held in "/usr/src/". e.g. /home/user/kernels/2.6.15.git /home/user/toolchain/kernelheaders/kernel-2.6.15.git
  • Warning: after 2.6.15 the ppc64 and ppc architecture branches of the Linux kernel tree were merged into 'powerpc'. If you have a system with a kernel older than 2.6.15 make sure to use older kernel headers that still include the 'ppc64' and 'ppc' architecture branches. While not imperative, it may be wise to use kernel headers for your toolchain that match the kernel version you have booted on your system. The script is smart enough to create the proper headers

Configuring GLIBC

biarch toolchain paths

Generally I install full 32-bit and 64-bit biarch toolchains into "/opt/biarch/example-20060301". I then invoke the toolchain binaries in the following manner:

/opt/biarch/example-20060301/bin/gcc -m64 -o someexec somesource.c -L/opt/biarch/example-20060301/lib64 -lm

A directory listing of a biarch toolchain in '/opt/biarch/example-20060301' follows:

«user@host»:~§ ls /opt/biarch/example-20060301/
bin  bin64  etc  include  info  lib  lib64  libexec  libexec64  man  powerpc-linux  sbin  sbin64  share

When we talk about building with a biarch toolchain and using a chroot environment to install over a biarch toolchain we're talking about the toolchain we've already built and installed into '/opt/biarch/example-20060301'.

32-bit config

The following script should be stored in /home/user/toolchain/build32-24/

# glibc sources
echo "source=$source"
echo "targetdir=$targetdir"
echo "kernelheaders=$kernelheaders"
echo "kernelversion=$kernelversion"
CC="${targetdir}/bin/gcc -m32"
export CC
echo "CC=$CC"
CXX="${targetdir}/bin/g++ -m32"
export CXX
echo "CXX=$CXX"
BUILD_CC="${targetdir}/bin/gcc m32"
export BUILD_CC

$source/configure powerpc--linux \
     --build=powerpc-linux \
     --host=powerpc-linux \
     --target=powerpc-linux \
     --srcdir=${source} \
        --with-tls \
        --with-__thread \
     --enable-add-ons=nptl \
     --enable-shared \
     --without-cvs \
     --enable-all-warnings \
     --with-headers=${kernelheaders} \
        --enable-kernel=${kernelversion} \
     --prefix=${targetdir} \
     --exec-prefix=${targetdir} \
     --libdir=${targetdir}/lib | tee configure32.txt

64-bit config

The following script should be stored in /home/user/toolchain/build64-24/

  • Notice the comment in the script below regarding the 'configparms' file. TAKE HEED lest this become a source of hours of frustration for you like it was for me.

# glibc sources
echo "source=$source"
echo "targetdir=$targetdir"
echo "kernelheaders=$kernelheaders"
echo "kernelversion=$kernelversion"
CC="${targetdir}/bin/gcc -m64"
export CC
echo "CC=$CC"
CXX="${targetdir}/bin/g++ -m64"
export CXX
echo "CXX=$CXX"
BUILD_CC="${targetdir}/bin/gcc -m64"
export BUILD_CC

# farking configparms file is a hidden/secret config file and is the only
# place slibdir can be set.  If you don't set slibdir to lib64/ it'll put
# in lib/ which'll make gcc, objdump, and friends puke.
echo slibdir="${targetdir}"/lib64 > configparms
echo bindir="${targetdir}"/bin64 >> configparms
echo sbindir="${targetdir}"/sbin64 >> configparms
echo rootsbindir="${targetdir}"/sbin64 >> configparms
echo cross-compiling=no >> configparms

$source/configure powerpc64--linux\
     --build=powerpc64-linux \
     --host=powerpc64-linux \
     --target=powerpc64-linux \
     --srcdir=${source} \
        --with-tls \
        --with-__thread \
     --enable-add-ons=nptl \
     --enable-shared \
     --without-cvs \
     --enable-all-warnings \
     --with-headers=${kernelheaders} \
        --enable-kernel=${kernelversion} \
     --prefix=${targetdir} \
     --libdir="${targetdir}"/lib64 \
     --libexecdir="${targetdir}"/libexec64  | tee configure64.txt

Creating and initializing the chroot environment

create the file system image

WARNING Carefully chose the libraries you want from /lib, /lib64, and /usr/lib to include in your file system image. During the GLIBC install script it'll check every script in /lib and /lib64 for compatibility. This means that if you've not included a library that one library depends on your installation will not succeed. This seems to happen with the graphics related libraries. They're safe to remove from the file system entirely.

mount -bind home directory

invoke chroot environment

Once the chroot environment is installed and the mount --bind is properly executed you can enter the chroot env with the following commands:

«user@host»:~/jail§ sudo /usr/bin/chroot /home/user/jail/chroot /home/user/jail/chroot/bin/bash

which will log the user in as the 'root' user with the chroot directory as the root directory:


Build GLIBC from within chroot env

  • You should build from within the chroot environment because there are install steps later that depend on what libraries were present during the build step. If you built outside of the chroot environment but had fewer libraries inside of the chroot environment the install stage could fail when it looks for libraries that aren't in the chroot env but were in the base lib dirs.
  • Caveat: If you munge with your GLIBC install in the chrooted /opt/biarch too much you won't be able to rebuild the next GLIBC revision with it.

Invoke 'make' from the GLIBC build directory for each biarch (32 & 64):

«root@host»:/home/user/toolchain/builds/build32-24-glibc¤ make 2>&1 | tee __makeout

«root@host»:/home/user/toolchain/builds/build64-24-glibc¤ make 2>&1 | tee __makeout

Installing GLIBC

Install over a chroot /opt/biarch toolchain

Invoke 'make install' from the GLIBC build directory for each biarch (32 & 64):

«root@host»:/home/user/toolchain/builds/build32-24-glibc¤ make install 2>&1 | tee __makeinstall

«root@host»:/home/user/toolchain/builds/build64-24-glibc¤ make install 2>&1 | tee __makeinstall

After glibc is installed you'll be able to execute and test applications from within the chroot environment using that new toolchain in /opt/biarch.

Bootstrap install over a chroot /lib and /lib64 GLIBC install

  • Install into an alternate root
  • Once install is done copy files into chroot /lib, /lib64, /bin, /bin64, etc.

Build and bootstrap install GLIBC to system /lib and /lib64

I'm afraid to even address this. The GLIBC community recommends investigating how the distributions do this. I'll leave this out for now until I actually do this.

Building and installing using a distribution's GLIBC source rpm

Installing the source RPM and building outside of the /usr/src/packages/

It is recommended that you use the following steps to build the RPM as a non-root user.

To do so you'll have to process the src rpm and build the rpms in a hierarchy other than /usr/src/packages.

  • Add the following to the /home/$USER/.rpmmacros file:
«user@host»: vim ~/.rpmmacros
%_topdir /home/$USER/packages

This will cause the source rpms to be unpacked into /home/$USER/packages/SOURCES and /home/$USER/packages/SPECS and the sources will be patched and built in /home/$USER/packages/BUILD. The resultant rpms will be located in /home/$USER/packages/RPMS.

Using an alternate %_topdir allows you to build the rpms as a non-root user.

  • You'll want to create the necessary directories or your rpmbuild will fail, e.g.
mkdir /home/$USER/packages/BUILD
mkdir /home/$USER/packages/SRPMS
mkdir /home/$USER/packages/RPMS
mkdir /home/$USER/packages/SOURCES
mkdir /home/$USER/packages/SPECS

With an alternate %_topdir set running rpm -i on a source rpm as $USER will unpack it into /home/$USER/packages/SOURCES and /home/$USER/packages/SPECS.

Source RPM Build and Install on SUSE

32-bit vs. 64-bit builds

SuSE's rpm spec files have a few intricacies which reveal that they build 32-bit applications on a system with a 32-bit userspace and they build 64-bit applications on a system with a 64-bit userspace. This causes problems when you want to build their src rpm on a biarch system (both 32-bit and 64-bit userspaces).

64-bit builds

  • You'll invoke rpmbuild with --target ppc64.
64-bit rpm builds of SLES10 SP2 GLIBC

The SLES10 SP2 glibc src rpm has a bug in it which prevents successful building of a 64-bit GLIBC on a biarch-system. They have a bare gcc invocation in the spec file. On a 64-bit system this will result in errors like the following:

/usr/bin/ld: warning: powerpc:common64 architecture of input file `cc-nptl/csu/crt1.o' is incompatible with powerpc:common output

Notice how the 32-bit ld is being used for the 64-bit build.

In order to fix this, apply the following patch which replaces the bare gcc invocation with '$BuildCC'. For ppc64 $BuildCC is defined as gcc -m64.

--- SPECS/glibc-2.4-31.54.spec	2008-07-29 12:04:45.000000000 -0500
+++ SPECS/	2008-07-30 10:28:02.000000000 -0500
@@ -573,7 +573,7 @@
 # Build glibc_post_upgrade binary
-gcc -static -Os -g $RPM_SOURCE_DIR/glibc_post_upgrade.c -o glibc_post_upgrade \
+$BuildCC -static -Os -g $RPM_SOURCE_DIR/glibc_post_upgrade.c -o glibc_post_upgrade \
      -Lcc-nptl -Bcc-nptl/csu \
     '-DLIBDIR="/%{_lib}"' '-DGCONV_MODULES_DIR="%{_prefix}/%{_lib}/gconv"'

32-bit builds

  • You'll invoke rpmbuild with --target ppc.

Some version of the rpm spec files do not account for the fact that the rpms may be built on a biarch system and you'll have problems getting the library paths to be correct for 32-bit builds. If this is the case then use the following suggestion:

  • In order to build 32 bit libs add the following information to your ~/.rpmmacros file:
«user@host»: vim ~/.rpmmacros
%_target_cpu ppc
%_libs lib

Find the sources

  • Find the GLIBC src rpm, e.g.:

Test and install the source package

  • Test the source package install to see if anything dies:
rpm -i --test glibc-2.3.3-98.61.src.rpm
  • Install the source package:
rpm -i glibc-2.3.3-98.61.src.rpm
  • If you're installing and building as root then rpm's %_topdir variable is set to /usr/src/packages.
  • If you're installing and building as $USER then rpm's %_topdir variable is set to /home/$USER/packages provided that you followed the instructions for setting %_topdir given earlier.
  • This will unpack the src rpm contents into %_topdir/SOURCES.
  • It will place the src rpm glibc.spec file in %_topdir/SPECS.
  • The source data in %_topdir/SOURCES isn't really usable at this point because none of the patches in the directory have been applied. This is simply a location used by the rpm build system to store sources and patches. The following section will show how to apply a patch into this source and how to build an rpm out of the sources in %_topdir/SOURCES.

Test assemble the source from the src rpm

  • Test assemble the sources from the rpms by executing the rpmbuild prep stage. This will unpack the files and test apply the patches. Look for any errors. We'll do this again later after we've applied our own patch.
cd %_topdir
«root@host»:%_topdir§ rpmbuild -bp SPECS/glibc.spec

Source RPM Patch

If you want to apply a secondary patch to the src rpm source code that will be applied when the src rpm is built into an rpm do the following:

  • Copy the source RPM patch to the SOURCES directory. In our example below glibc-2.3.4-64k.patch is our secondary patch that we wish to apply to the src rpm sources.
«root@host»:%_topdir§ cp <path-to-patch>/glibc-2.3.4-64k.patch %_topdir/SOURCES/

Patching src rpm spec file to remove make check

We're going to make a copy of the glibc.spec file which will hold modifications that coincide with our own particular needs. This is done to preserve the original spec file. From now on we'll use the new spec file and will direct the rpm build applications to use our new spec file as well.

«root@host»:%_topdir§ cp -i SPECS/glibc.spec SPECS/glibc-2.3.3-98.61.spec


--- SPECS/glibc-2.3.3-98.61.spec                      2005-12-11 16:35:07.000000000 -0600
+++ SPECS/glibc-2.3.3-98.61.removemakecheck.spec      2006-02-08 16:58:14.775075736 -0600
@@ -471,22 +471,6 @@
 cd ..
-%ifarch alpha mips armv4l sparc sparc64 hppa s390 s390x
-   make -C cc -k check || echo make check failed
-   make -C cc check
-   cat /usr/src/packages/BUILD/glibc-2.3/cc/posix/tst-nanosleep.out ||:
-%ifarch i686
-   make -C cc-i686 check
-%ifarch %{nptlarches}
-%ifarch ppc64 alpha ia64 s390 s390x
-   make -C cc-nptl -k check || echo make check failed
-   make -C cc-nptl check
  • Apply the preceding patch to the %_topdir/SPECS/glibc-2.3.3-98.61.spec rpm specification file to remove the make check portions.
«root@host»:%_topdir§ patch -p0 < glibc-removemakecheck.patch

Update the release number in the .spec file

Modify the rpm release number in the .spec file SPECS/glibc-2.3.3-98.61.spec so that the resultant rpm doesn't have the same release or the packages will collide when you try to install it with rpm.

  • Create the following patch and increment the Release as appropriate e.g. 98.61.1:


--- SPECS/glibc-2.3.3-98.61.spec     2005-12-11 16:35:07.000000000 -0600
+++ SPECS/ 2005-12-11 17:49:10.689115880 -0600
@@ -26,7 +26,7 @@
 Obsoletes:    shlibs ngpt ngpt-devel
 Autoreqprov:  on
 Version:      2.3.3
-Release:      98.61
+Release:      98.61.1
 PreReq:       filesystem
 BuildRoot:    %{_tmppath}/%{name}-%{version}-build
  • Apply the patch:
«root@host»:%_topdir§ patch -p0 < glibc-updaterelease.patch

Note: Your glibc-2.3.3-98.61.spec file may differ greatly from this example. In such a case simply modify Release: by hand.

Dry run patch source with patch

  • Make sure you have the code patch, glibc-2.3.4-64k.patch, in %_topdir/SOURCES/
  • Test and see if the patch applies to the source in %_topdir/BUILD/glibc/ that was installed during the first rpmbuild -bp command executed earlier. Don't remove the --dry-run though. This is just an investigative step. We'll let the rpmbuild command do the real patching later.
«root@host»:%_topdir§ cd /BUILD/glibc-2.3
«root@host»:%_topdir/BUILD/glibc-2.3§ patch --dry-run -p1 < ../../SOURCES/glibc-2.3.4-64k.patch

If there are any errors with the patching make sure to fix them before continuing.

Update the spec file to include instructions to apply custom patch

  • Create the following patch which will add instructions to the spec file to direct rpm to apply our custom patch before it builds the rpm.


--- SPECS/glibc-2.3.3-98.61.spec               2005-12-11 16:35:07.000000000 -0600
+++ SPECS/glibc-2.3.3-98.61-addpatch.spec      2006-02-08 16:58:14.775075736 -0600
> Patch100:     glibc-2.3.4-64k.patch 
> %patch100 -p1

Apply the patch:

«root@host»:%_topdir§ patch -p0 < add-patch-to-spec.patch

Remember, if your .spec file differs from this you'll have to patch it by hand and add your patch to the list of patches to be applied.

Test build the GLIBC rpm with the custom source patch

rpmbuild -bp SPECS/glibc-2.3.3-98.61.spec > glibc-2.3.3-98.61.log 2>&1
  • look in that log file to see if the build was successful (actually just the source reconstruction; -bp = "build prep"), if that part is good then continue to fully build the rpm.

Build the rpm

We will use the rpmbuild switch --buildroot to direct rpmbuild to test the install capabilities of the rpm. The --buildroot switch is necessary in order to override the %install stage's default installation location.

WARNING WARNING WARNING: --buildroot is destructive to the target directory. DO NOT specify a root level directory as the target, i.e. specifying /home/$USER would be VERY BAD and will result in the removal of that directory. Always specify an empty non-root level directory as the target.

rpmbuild -ba GLIBC/glibc-2.3.3-98.61.spec --define='jobs 3' --buildroot /home/$USER/packages/buildroot/ > glibc-2.3.3-98.61.log 2>&1

The resultant rpms should be stored in /%_topdir/RPMS/<arch>/<rpmname>.rpm.

  • Note: --nodeps can be passed to rpmbuild to circumvent dependency requirements, for instance, when you want to cross build a SLES glibc on a RHEL system.

You can install the rpm with rpm -i or use rpm2cpio in conjunction with cpio to unpack the rpm into a local directory, namely ./:

rpm2cpio %_topdir/RPMS/<rpmname>.rpm | cpio -ivd

Source RPM Build and Install on Redhat

Source DEB Build and Install on Debian