NAME

pakfire-jail - What the build environment protects against, and what it does not

DESCRIPTION

Pakfire builds packages from makefiles and sources that somebody else has written. Building a package means running whatever its makefile, its build system and its test suite want to run, as root. The build service does this for makefiles that anybody could have sent.

Everything that a build runs is being run inside the jail. This page describes what the jail is made of, what a build can and cannot do in it, and what whoever runs builds for others has to take care of themselves.

There are no virtual machines. The jail is made of what the Linux kernel offers to separate processes from each other, and it is as strong as the kernel is. A build that is able to exploit a bug in the kernel is able to leave the jail.

DESIGN GOALS

  • A build must not be able to read or change anything on the host, or in another build.

  • A build must not be able to use more resources than it has been given.

  • A build should be able to run as much of what it brings as possible, which includes test suites that create containers of their own. Nothing is being forbidden only because it is unusual.

WHAT THE JAIL IS MADE OF

Namespaces

Every command runs in new user, mount, PID, IPC, UTS, cgroup and time namespaces, and in a new network namespace unless it has been given access to the network (see below).

Users

Root inside the jail is not root on the host. The users inside are being mapped to the IDs that /etc/subuid and /etc/subgid give to whoever runs Pakfire, and have no privileges over anything on the host that does not belong to them.
Pakfire itself does not have to be root either: pakfire-builder and pakfire-worker then become root in a user namespace of their own first (which needs newuidmap and newgidmap), and the jail is another user namespace inside of that one.
If Pakfire runs as root, root inside the jail is root on the host. It has no capabilities there, but files that belong to root belong to it, which includes some through which the kernel can be told what to do. What is known of those is read-only inside the jail, but somebody who has all capabilities inside the jail should not be expected to stay behind that. Builds of makefiles that are not being trusted must be run by a user who is not root.
This is why pakfire-worker, which builds what anybody has sent to the build service, does not start as root. pakfire-builder does: whoever builds their own packages on their own system (to debug a firewall, for example) knows what they are building.

Processes

The first process of a command is PID 1 of its namespace. When it exits, the kernel kills everything else that is left. Everything is also being killed if Pakfire itself dies.

Network

Builds have no network. There is nothing but a loopback interface.
The only exception is an interactive shell (pakfire-builder shell) which shares the network of the host so that packages can be installed and things can be tried out.

File system

A build sees its own root file system and nothing of the host. The root is either a directory, or it is being kept in memory (see disk_limit below). The following is being mounted into it:

/proc

A new instance which only shows the processes of the jail. /proc/sys (apart from /proc/sys/net), /proc/sysrq-trigger, /proc/irq and /proc/bus cannot be written to.

/sys

A new instance which only has the network interfaces of the jail and cannot be written to. What tells too much about the host or reaches into the kernel is hidden: firmware and DMI tables, power capping, and the bpf, pstore, debug, security and tracing file systems.

/dev

A new and empty file system with null, zero, full, random, urandom, tty and pseudo terminals. There are no disks, no kernel log (/dev/kmsg), no clock (/dev/rtc0) and no memory devices.

/tmp, /dev/shm, /run

Kept in memory, with 4 GiB, 1 GiB and 256 MiB.

The BUILDROOT

Where a build installs what is going to be packaged.

The compiler cache

See COMPILER CACHE below.

Anything that Pakfire does with the root from the outside (installing packages, reading what has been built, packaging it) is being done in a way that cannot leave the root: a build may create any symbolic link that it likes, but paths are always being resolved like a process inside the jail would resolve them.

System calls

A process inside the jail is allowed to create namespaces of its own, in which it has all capabilities again. That is needed to run test suites of software that deals with containers, and means that dropping capabilities would not achieve much. What still applies in there is:

  • Nothing can gain privileges by executing a program (no_new_privs): the setuid and setgid bits and file capabilities are being ignored.

  • A filter denies a list of system calls. This is a list of what is forbidden and not of what is allowed, because nobody can tell what the next compiler or test suite is going to need.

The filter denies:

  • The keyring of the kernel (keyctl, add_key, request_key), which is not being separated by any namespace

  • bpf, perf_event_open and userfaultfd

  • io_uring, which is being reported as not existing so that programs fall back

  • open_by_handle_at

  • Reading the log of the kernel (syslog)

  • Sockets to the hypervisor and to other virtual machines (AF_VSOCK)

  • What only the host should do, and what the kernel denies already: loading kernel modules, kexec, swap, setting the clock, process accounting, disk quota and access to I/O ports

Programs that use the system calls of another ABI (for example 32 bit programs on a 64 bit system) are being killed, because the filter does not know those.

Still allowed on purpose are creating namespaces and mounting in them, ptrace, and all other kinds of sockets. That includes netlink and with it netfilter, which is being separated by network namespaces: Pakfire builds a firewall distribution whose packages have to be able to test themselves.

Resources

Every build runs in a control group of its own. The following can be set in the [build] section of the configuration of the distribution:

memory_limit

How much memory a build may use. There is no limit by default.

memory_guaranteed

How much memory is being kept for a build (2 GiB by default).

pid_limit

How many processes and threads a build may have (2048 by default).

disk_limit

How much a build may write. With a limit, the root and the BUILDROOT are both being kept in memory and each of them can become this large. What a build writes then counts as its memory, so that there has to be enough memory or swap, and memory_limit has to leave room for it.
Without a limit (which is the default), builds that the worker runs write to /var/tmp on the disk of the host without any limit.

ccache_max_size

How large the compiler cache of a package may become (5 GiB by default).

A build that is not root on the host can only have these limits if a control group has been delegated to whoever runs it (Delegate=yes in the service file of the worker). The worker does not start without one. A build that somebody runs for themselves with pakfire-builder runs without any limits in that case, and says so.

Builds run with a lower priority (nice level 5).

There is no limit of how long a build may take. The build service has to abort builds that take too long.

COMPILER CACHE

Builds can use ccache to not compile the same code again. Whatever a build puts into its cache is being used by the next build that has the same cache, which means that a build can change what comes out of another one. So a cache is never being shared by builds that should not be able to do that:

  • If somebody builds for themselves, every package has its own cache.

  • The build service tells the worker which cache to use. Every user has their own caches, one for each package, and test builds have theirs.

  • Builds that are being released do not use a cache at all, so that nothing that has been built before can end up in them.

After a build that the worker has run, a cache that has become more than twice as large as ccache_max_size is being removed. Caches that nobody has used for 30 days are being removed by the worker once a day (ccache_max_age in the [worker] section of its configuration, zero turns this off), and by pakfire-builder clean --ccache-max-age=30d.

BEFORE THE BUILD

A makefile is being read, and its sources are being downloaded, before anything runs inside the jail. On the build service, this happens in the build service itself.

  • Makefiles are being parsed by Pakfire and nothing in them is being executed at that time. A value cannot become longer than 256 KiB when variables are being expanded.

  • Sources are only being downloaded with HTTP and HTTPS, and their names have to be plain file names. A source may be 2 GiB at most (max_source_size in the [dist] section of the configuration), and a download that does not make any progress for two minutes is being given up.

  • Pakfire does not check which hosts sources are being downloaded from. Whoever runs this for others has to use a proxy (proxy in the [general] section of the configuration) which keeps makefiles from reaching hosts on the internal network.

  • Nothing checks that a source is what its makefile says. A makefile is being trusted to know where its sources come from.

WHAT THE JAIL DOES NOT PROTECT AGAINST

  • Bugs in the kernel. A build can call most of what the kernel offers, and can create namespaces in which it has all capabilities.

  • What comes out of a build. A package contains whatever its makefile has put into it, and runs as root on the system that it is being installed on. The jail keeps a build from changing anything but its own packages; it does not make those packages trustworthy.

  • The CPU. Builds on the same machine share it, and nothing is being done about what they could learn about each other through that.

  • Running out of disk space if disk_limit is not set, and builds that never finish.

RECOMMENDATIONS FOR WORKERS

  • Run the worker as a user of its own, on machines that do nothing else. It does not start as root (see Users above). The service file runs it as the user pakfire, who needs IDs in /etc/subuid and /etc/subgid.

  • Set memory_limit, pid_limit and disk_limit.

  • Keep the kernel up to date.

  • Let the machine reach the build service and the mirrors, and nothing else that is worth protecting.

SEE ALSO

pakfire(8)

AUTHORS

Michael Tremer