(draft) The future of the boot system in Debian

[Petter Reinholdtsen]

This is the current verison of the draft available from
<URL:http://wiki.debian.org/LSBInitScripts/DependencyBasedBoot>.  I
suspect the text is almost ready to be shipped done, but would welcome
more improvements.  I suspect we should add more links to relevant
documents.

Who is interested in being listed as a signer of this announcement?
Please add your name on the wiki page.


The future of the boot system in Debian
=======================================

Over the last few years, the boot system in Debian has progressively
deteriorated due to changes in the Linux kernel which make the kernel
more and more event based. For example, the kernel and its drivers no
longer block all processing while detecting disks, network interfaces
and other hardware, making the once trusty old boot system in Debian
increasingly fragile. During the current boot sequence device files in
/dev/ are often missing when fsck or mount are looking for them, or
the network is not available when the boot system tries to mount NFS
entries because network interfaces were slow to initialise, or audio
devices are missing when audio settings should be set. The problem is
fundamental to the way we boot Debian today - sequentially, and a
solution needs to address this fundamental problem. We believe the
solution is to migrate to an event based boot system.

In addition to this, there are long lasting problems with the boot
sequence of the existing init.d scripts, for some combination of
packages. The boot sequence is wrong in these cases, and to solve it
one needs to change the sequence numbers of all the immediate forward
and reverse dependencies of the init.d script in question - and their
forward and reverse dependencies and so forth until the boot sequence
is correct. In some cases the change needs to happen to several
scripts in different packages at the same time, which is impossible
with the old way of ordering init.d scripts. Previously the ordering
was done by asking the package maintainers to guess on and update
sequence numbers, a process that tended to introduce new problems and
took a long time to be solved properly. The solution to this problem
is to change how we order boot scripts. Change it from static sequence
numbers to calculate the boot sequence using dependency information
provided in the init.d scripts themselves. Since 2009-07-27 this is
the default in Debian unstable, and it will be the way init.d scripts
are ordered in Squeeze.  Switching to a dependency based boot
sequencing allows us to ensure its correctness, detect and fix
dependency loops, and in general fix a set of bugs in the distribution
that have been very hard to fix before.  Other solved problems with
the new system are incorrect stop sequences (the default should have
been S20/K80, not S20/K20), and the misleading runlevels 0 and 6,
where start symlinks are called with 'stop' as the argument to the
scripts. All of these problems are solved when Debian now moves to
dependency based boot sequencing.

It will take longer to solve the fundamental problem. It requires a
rewrite of how we handle the boot, and can not be done by just
modifying the boot sequence framework.

Before explaining the current plan, some background information. The
current boot system can be seen as consisting of three different parts:
 1. The implementation of /sbin/init, reading /etc/inittab and starting
    the script implementing part 2 (/etc/init.d/rc). This is normally
    done using the sysvinit package, but other replacements are
    available, like initng and upstart.
 2. The implementation of /etc/init.d/rc, which is responsible for
    calling the init.d scripts in the correct sequence. This is
    normally done using the sysv-rc package. An alternative is the
    file-rc package, which uses a file /etc/runlevel.conf instead of
    symlinks in /etc/rc?.d/ to decide what to execute and in which
    order.
 3. The individual init.d scripts, taking care of the tasks that need
    to be done during boot. The basic framework is provided by the
    initscripts package, and the rest is handled by individual packages
    like udev, netbase, ifupdown, apache, etc. :) There are
    approximately 850 packages with init.d scripts in Debian unstable.

According to the Linux Software Base specification, all LSB[1]
compliant distributions must handle packages with init.d scripts. As
Debian plans to continue to follow the LSB, this mean the boot system
needs to continue to handle init.d scripts. Because of this, we need a
boot system in debian that is both event based for the early boot, and
which also calls init.d scripts at the appropriate time.

  1. http://refspecs.freestandards.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/initsrcinstrm.html

Part 2 (sysv-rc) and 3 have been changed to use dependency based boot
sequencing. file-rc needs to be changed to also order using dependency
information (See #539591) for those who use it, to allow Debian
maintainers to stop guessing sequence numbers.

To solve the fundamental problem, the plan is to replace /sbin/init
with an implementation that is able to handle kernel events. It will
allow us to modify the boot system for the early boot to become event
based, while keeping the existing boot stuff working. We could rewrite
sysvinit to become event based, or have a look at the existing boot
systems that handle kernel events. After checking the options and the
systems used in other distributions, upstart seems like the most
promising candidate. It is used by Ubuntu and Fedora at the moment, and
solves the problem in a backwards compatible way. The plan is to change
upstart to actually use /etc/inittab, to ease the switch between
sysvinit and upstart. We will also change the init.d script handling to
treat upstart jobs as init.d scripts, to provide an alternative for
architectures lacking upstart support. These changes should make it
transparent for the users which package provides /sbin/init, and thus
make it easier to migrate from sysvinit to upstart.

When /sbin/init is changed to an event based framework, the next step
is to rewrite the early boot system to use these events when available,
and behave the traditional way when there are no events. When this step
is finished the fundamental problem will be solved, and the boot will
be robust and should work correctly even in edge cases with slow device
buses.

The planned time frame for this is to replace /sbin/init with upstart
for Squeeze, and see if we manage to change the very early boot to
become event based in time for Squeeze too, fixing the most pressing of
the current boot problems (failing fsck and mount with USB disks). For
Squeeze+1, more of the early boot system will be converted, to handle
more of the existing problems.

References:
  * http://wiki.debian.org/LSBInitScripts/DependencyBasedBoot

Petter Reinholdtsen, Kel Modderman, Armin Berres

Happy hacking,
-- 
Petter Reinholdtsen