Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

(Newly cc'd elogind maintainers: Please see #945269 for context)

On Sun, 04 Jun 2023 at 12:15:41 +0100, Luca Boccassi wrote:
> On Sun, 4 Jun 2023 at 12:02, Sean Whitton <spwhitton at spwhitton.name> wrote:
> > On Tue 09 May 2023 at 01:44AM +01, Luca Boccassi wrote:
> > > For now I've kept only a mention of the 'systemd-tmpfiles' virtual
> > > package. As maintainers we would really prefer if the 'main'
> > > implementation is pulled in whenever possible. When a minimal
> > > installation is desired (ie, a minbase), it is possible to manually
> > > specify the -standalone variant.
> > >
> > > This was a controversial point last year, see:
> > >
> > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1017441
> >
> > Hmm.  I don't have personal experience with this sort of thing, but
> > based on some of the examples in that bug, it seems like doing this
> > could cause apt to change people's systems around in ways they strongly
> > disprefer.  What you propose seems like it could cause unpleasant
> > surprises.
> 
> Only due to bugs in said other packages, or if the wrong commands are
> passed to apt/aptitude/etc.

I think one way or another, if anyone is going to set a package-level
dependency on systemd-tmpfiles, the first (preferred) dependency needs to
be on either a concrete provider (systemd or systemd-tmpfiles-standalone
in this case), or a default-systemd-tmpfiles virtual package
that only has one provider per architecture (which is the way
{default-,}dbus-{system,session}-bus are handled). Otherwise, you
can get a non-deterministic choice of default implementation, which
seems strictly worse than either depending on systemd or depending on
systemd-tmpfiles-standalone - if you're unlucky, it can have all the
disadvantages of either one of those.

So I think the only realistic options for packages that hard-require
this functionality (not all do) are:

1. Depends: systemd | systemd-tmpfiles
2. Depends: systemd-tmpfiles-standalone | systemd-tmpfiles
3. Depends: default-systemd-tmpfiles | systemd-tmpfiles

(where the third one is equivalent to one of the first two, depending on
how default-systemd-tmpfiles was implemented), possibly with some more
less-preferred options between the first "|" and the virtual-package
fallback.

I also think that Policy shouldn't be recommending this interface without
also being able to give guidance on what an appropriate dependency would
look like, because if some packages choose (1.) and some choose (2.),
again we'll get a non-deterministic result, depending on which packages
you happen to install first and what their maintainers think; and I don't
want individual services' maintainers to be required to constantly argue
whether (1.) or (2.) is better.

In #1017441 the debhelper maintainers declined to generate such a
dependency, but even if they had wanted to generate it centrally, we'd
have the same distro-integration considerations about saying what the
right dependency would look like.

Before describing pros and cons of those options in different scenarios,
I want to reiterate that installing the systemd package (which contains
the default/recommended implementation of the systemd-tmpfiles interface)
does not imply using systemd as an init system, and everywhere in this
message that I have said "systemd", I specifically mean the systemd
package and not systemd-as-pid-1.

>From the point of view of init systems, I think the interesting scenarios
are:

A. systemd package installed (typically via systemd-sysv)
B. non-systemd init system (typically sysvinit) and no systemd package
C. no init system at all (typically in Docker or a chroot)

For (A.), there is no ambiguity: systemd is installed and provides the
tmpfiles interface, and everything is fine. Any dependency sequence ((1.),
(2.) or (3.) above) should give us the result we want. The only problem
scenario I can think of for (A.) with (2.) is:

- a package (let's say foo-service) has chosen (2.) and so depends on
  systemd-tmpfiles-standalone | systemd-tmpfiles
- we install a base system with neither foo-service nor systemd
  - this must be a minbase debootstrap or similar, because a default
    debootstrap already includes systemd-sysv
- we install foo-service first, without systemd
  - systemd-tmpfiles-standalone is pulled in
- afterwards, we install systemd (via the init metapackage and
  systemd-sysv, or directly)
  - desired result: systemd-tmpfiles-standalone is removed and replaced by
    systemd
  - actual result: apt's heuristic might have difficulty realising that
    it needs to do that

I would have expected that anyone wanting an environment with an init
system is more likely to include it in the bootstrap or in the first
batch of post-bootstrap additions than to install it later, but maybe
that's wrong? Or are there other problem scenarios here?

I'll come back to (B.), non-default init, in a moment.

No init system at all, (C.), can only happen when starting with a
minbase debootstrap or equivalent (because a default debootstrap
includes the init metapackage due to its Priority: required). In
this scenario it *usually* doesn't really matter whether we
install systemd or systemd-tmpfiles-standalone. systemd is somewhat
larger, so container/chroot maintainers might prefer to install
systemd-tmpfiles-standalone for size optimization, but either one should
give us correct results. What I'm hearing from Luca and other systemd
maintainers is that the systemd team would prefer this scenario to
get the (much more widely-tested) full-sized systemd package, unless
the container/chroot maintainer has explicitly taken steps to get a
non-default implementation for reduced on-disk size. Is that correct?
As far as I can see, either (1.) or (2.) would achieve that in practice,
again because a default debootstrap includes systemd, and I think asking
for minbase should be counted as an example of explicitly taking steps
to get a reduced on-disk size.

One possible problem scenario for (C.) with (2.) is that if we start from
minbase but then some other dependency pulls in the full systemd package
anyway (perhaps as a way to get systemd.pc into a buildd chroot, or as
a dependency of systemd-container) then we will have the same problem
that I outlined for (A.): apt's heuristic needs to be able to realise
that removing systemd-tmpfiles-standalone to be replaced by systemd
is a valid solution, and because problem resolution is a heuristic,
there's a chance that that won't happen.

Now the elephant in the room, (B.): non-default (non-systemd) init. The
main problem with (1.), having systemd as the first (preferred)
dependency, is that it is mutually exclusive with elogind (because the
systemd maintainers are not willing to support systemd tools being
co-installed with a non-systemd reimplementation of logind, which
is an entirely reasonable choice IMO), but users of non-systemd init
systems rely on elogind as a dependency of desktop environments and
other higher-level functionality. This gives us #1014805, #1016006,
#1017441 and similar issues: if a user of non-systemd init has not
yet installed any implementation of the tmpfiles interface, when they
upgrade a package to a version that depends on systemd-tmpfiles, it is
entirely possible that apt's heuristic will propose installing systemd and
therefore removing elogind and the desktop environment, instead of the
result the user would presumably have preferred, which is installation
of systemd-tmpfiles-standalone. Obviously that's not what we want to
happen. The situation is rare and only arises with a non-default init,
but the failure mode is very bad.

It is technically correct to say (as was said on #1014805, etc.) that
installing the systemd package does not preclude experiments with
non-default init systems, but in practice that reasoning only holds for
relatively small software stacks: if you want a desktop environment, in
practice you'll likely need a logind implementation, which means either
the full systemd stack with systemd-logind (which requires systemd
as pid 1), or elogind (which is mutually exclusive with systemd). I
am personally happy with systemd as pid 1 and so I see no reason to
install elogind myself, but it is available as a non-default option,
and we should be honest about the impact of related decisions on the
ability to make use of that option.

A couple of mitigations for the problem case I described above were
suggested on #1014805 and #1016006:

- On #1014805, Michael Biebl suggested having sysvinit-core pull in
  systemd-tmpfiles-standalone via Depends (or maybe Recommends).
  Adam Borowski objected to this for two reasons:
  - sysvinit-core does not actually require any tmpfiles implementation,
    and it is only the individual services that might require one, so a
    dependency there (for A.) doesn't have much theoretical justification;
  - he felt that adding ~ 450K to minimal chroots (for C.) is too much.

- On #1016006, Mark Hindley suggested making systemd-standalone-tmpfiles
  the first (preferred) alternative in the or-group, which is what I
  called (2.) above. I haven't seen a rebuttal for that; did the systemd
  maintainers reject this because it would have the side-effect of always
  installing systemd-standalone-tmpfiles in the init-less scenario (C.),
  or were there other reasons?

Another possible mitigation which I haven't previously seen proposed
is giving *elogind* a Depends or Recommends on systemd-*-standalone.
I think that would work to mitigate the failure mode with (1.) and (B.),
and the installed-size argument seems less interesting here because the
sort of systems that require elogind are already much larger anyway.
Would the elogind maintainers be willing to consider this? Does anyone
see a reason why it wouldn't work?

Taking a step back from the specifics of -tmpfiles for a moment:

As a maintainer of system services, I would not be at all happy with
expecting the maintainer of every system service that requires tmpfiles
to have this conversation again and again. Obviously as a technical
committee member and occasional Policy contributor, I've chosen to take
on some of the burden of decisions like this one; but when I'm working
on dbus or polkitd or even openarena-server, I should be able to follow
a project decision, and be reasonably confident that I won't get angry
RC bug reports about it (and if I do, I should be able to refer the bug
reporter to a project decision instead of having to re-litigate it).
Putting this sort of thing on individual package maintainers seems like
a recipe for making it no longer fun to maintain a package.

I would like to be able to rely on sysusers.d for declarative creation
of system users in future, and that will make correct dependency handling
more important than it is now for tmpfiles.d, because unlike tmpfiles.d,
packages that own a system user often *do* have a hard dependency
on that system user being created in the postinst (and as a result,
dh_installsysusers in compat level >= 14 generates a dependency, unlike
dh_installtmpfiles).

At the moment, mostly as an experiment, polkitd depends on
adduser | systemd-sysusers (and it can't rely on dh_installsysusers
anyway, as far as I can see, as a result of some tricky sequencing
considerations in the postinst), but that's a workaround and I would
prefer to be using sysusers unconditionally.

If we can't get a consensus here, one option would be to refer the
question to the technical committee.

> > > We could even decide that no dependency is added at all by dh

For the record, that is the decision that the debhelper maintainers made
for tmpfiles.d in #1017441, on the basis that processing tmpfiles.d is
often a nice-to-have rather than a mandatory dependency: for example,
dbus has a tmpfiles.d snippet but will work correctly without it, with
only slightly reduced functionality.

However, debhelper does add a dependency for sysusers.d snippets, which
are out-of-scope for this particular Policy bug, but are "the same shape"
as tmpfiles.d in terms of how they're handled by systemd and how they
will need to be handled by non-default init systems.

> at some point it's either one or the other: set up the
> dependencies so that the default case (which in debian is systemd)
> works out of the box and the right thing happens magically, and people
> who want to use non-default init systems (which are supported for the
> purpose of "experimenting and exploring alternatives") will need to
> fix their packages and provide the right instructions to apt, or no
> dependency is set at all and bootstrapping/image building/etc tools
> need to adapt and manually pull the right tools at the right time

Luca, I see you're effectively ruling out what I described in
(2.) above. Is that because of the problem scenario I outlined above with
late installation of systemd potentially having difficulty with replacing
systemd-tmpfiles-standalone, or are there other problem scenarios that
you have in mind?

Thanks,
    smcv