NAME

pakfire-bootstrap - Building the first packages of a new architecture

DESCRIPTION

Pakfire builds every package in an environment that it installs the build dependencies of the package into. That cannot work for the first packages of an architecture: there is no package that could be installed, and the packages that are being built first (the C library, the compiler, …) are the ones that the environment is made of.

pakfire-builder build --bootstrap PATH builds in an environment that has been made some other way. This page describes what it does, what it leaves out, and how to get from nothing to a distribution that builds itself.

WHAT --BOOTSTRAP DOES

The directory is the build environment. It is mounted as it is, below a temporary filesystem that takes everything the build writes, so that

  • the directory is never changed, no matter what is built in it,

  • every build starts from the same state, and

  • any number of builds can use the same directory at the same time.

The source package is extracted into that environment, and nothing else is installed: not the packages that the distribution wants in every build, and not the dependencies of the package. What these would bring could replace the files that the environment is made of.

More packages can be added with --package, which can be given as often as needed. Their files are extracted into the environment, in the same way, and nothing they depend on is checked or installed. The runtime linker cache is updated if they contain libraries, but no scriptlets are run and no users or groups are created. This is how the packages that have been built so far are made available to the next build, and how anything that the environment lacks is added to it, which includes what the build itself needs to find at a fixed place. The arguments are paths of package files.

Without --bootstrap, --package adds the packages that the argument names to the build environment, and installs them with what they need.

Everything else is as in any other build. The stages are run in the jail (see pakfire-jail(7)) by the programs inside the environment, the packages are put into the local repository, and --target copies them out.

The macros and the scripts that Pakfire runs are those of the machine that builds. They are not part of the environment and need not be in it.

What is not checked

A build that starts from an environment does not

  • check that the dependencies of the source package can be installed,

  • install the build dependencies, the packages that every build gets, or update what is installed, and

  • try to install the packages that it has built (the install test), because what they depend on is not available as a package yet.

The build warns about that when it starts.

Everything that judges what has been built is still being done: unpackaged files, broken symbolic links, paths to the build root in files, the linter, the dependencies that are found in the files, and the tests (unless --disable-tests is given).

Limits

The option is for builds on a machine of your own. It cannot be used together with snapshots (a snapshot is what the directory takes the place of), and the build service never builds that way.

Pakfire does not know what the directory contains. If it is missing something that a package needs, the build fails, like it would if a program was missing on any other machine.

WHAT THE ENVIRONMENT NEEDS

The environment is an ordinary root filesystem for the architecture that is being built for:

  • A shell (/bin/bash; the build scripts are not written for any other), and the tools that makefiles and build systems expect: coreutils, findutils, sed, grep, gawk, tar, gzip and other compressors, make and file.

  • A compiler and linker for that architecture, and its C library with the headers. Compilers never cross-compile in Pakfire, so they have to run natively in the environment (see pakfire-architectures(7)).

  • The directories /proc, /sys and /dev, which the jail mounts onto.

Only what a package needs is required: the first packages need much less than a complete system. What is missing is found out by building and reading what fails.

Another architecture

An architecture that this machine does not have is emulated, which needs an emulator that is registered with the kernel (qemu-user with binfmt_misc). See pakfire-architectures(7). Choose the architecture with --arch. The environment has to be of the same architecture, and the build is a lot slower than on the real thing.

HOW TO BOOTSTRAP

1. Add the architecture

The architecture has to be known to Pakfire before anything can be built for it. See ADDING AN ARCHITECTURE in pakfire-architectures(7).

2. Make a build environment

Whichever tool is used, the result is a directory with a root filesystem that has the programs listed above in it, and that a program for the architecture can be run in (with the emulator, if the machine is of another architecture).

debootstrap can make one of Debian for any architecture that Debian has:

debootstrap --foreign --arch=riscv64 --variant=minbase \
        --include=build-essential,file,gawk trixie /srv/bootstrap/riscv64

--foreign stops after the packages have been unpacked, because they cannot be configured on a machine of another architecture. Finish it with the emulator that is running in the new directory (copy it there first):

cp /usr/bin/qemu-riscv64-static /srv/bootstrap/riscv64/usr/bin/
chroot /srv/bootstrap/riscv64 /debootstrap/debootstrap --second-stage

The environment is only used to build the first packages with. Debian changes the defaults of its compiler and of many build systems (multiarch library directories, the places where Python modules go, …), and a package that is built with them can take those over. Build as little as possible with the environment, and check the packages that come out of it.

Other ways to get a directory like this are building a toolchain with crosstool-NG, or any other distribution’s tool to make a root filesystem. Buildroot makes small systems with a cross compiler. As far as we know, it has not supported a compiler on the target since 2012.11 (its FAQ says to use another tool if the target needs one; check its current manual), so it can be the cross compiler that a native compiler is built with, but probably not the environment itself.

3. Build the first packages

pakfire-builder --arch=riscv64 build --bootstrap /srv/bootstrap/riscv64 \
        --target /srv/packages/riscv64 binutils.nm gcc.nm glibc.nm ...

The directory is mounted and the packages are built one after the other, each in a new environment. Build in the order that the packages need each other: the first ones with what the environment has, the later ones after the packages they need have been added (see below).

Read the log of a build that fails like the log of any other. A shell for debugging is opened on failure unless --non-interactive is given.

4. Use what has been built

The packages that come out are those of the distribution, built with the programs of the environment. The next package can be built with them by adding them with --package:

pakfire-builder --arch=riscv64 build --bootstrap /srv/bootstrap/riscv64 \
        --package /srv/packages/riscv64/glibc-2.42-1.ipfire3.riscv64.pfm \
        --target /srv/packages/riscv64 zlib.nm

To remove the dependency on the environment:

  • Make a repository of the packages in the target directory.

  • Install them into a new, empty root (pakfire --root PATH install ...), starting with the ones that have no dependencies and working up to the toolchain.

  • Build the same packages once more without --bootstrap, in an environment that Pakfire installs from that repository.

The packages of the second round are built by the distribution’s own tools and are the ones to publish. If you want to be sure that nothing of the first environment is left, build everything one more time and compare the results.

SEE ALSO

pakfire(8), pakfire-architectures(7), pakfire-jail(7)

AUTHORS

Michael Tremer