[Pkg-rust-maintainers] Bug#1034939: librust-<crate>-<semver>-dev: missing Breaks+Replaces for librust-mio-dev when upgrading from bullseye

[Fabian Grünbichler]

as reference, the (simplified) problematic combination:

rust-foobar in version X.Y.Z-A
ships librust-foobar-dev which provides librust-foobar-X-dev,
librust-foobar-X.Y-dev and librust-foobar-X.Y.Z-dev (all in version
X.Y.Z-A)

is what I call the "unversioned" package in the rest of this mail (it
doesn't have any part of the upstream version encoded in either the
source package name or its binary package names. like all rust crates,
it does have the prefixes of the upstream version encoded in virtual
package names).

rust-foobar-X in version X.Y.Z-B
ships librust-foobar-X-dev which provides librust-foobar-dev,
librust-foobar-X.Y-dev and librust-foobar-X.Y.Z-dev (all in version
X.Y.Z-B)

is the "semver suffix" package (because its source package name and the
binary package name(s) contain a suffix with the first part semver
component of the upstream version) - it provides both the unversioned
package name and the prefixes that are no already in its actual package
name.

note that A and B don't have to be identical at any point of these
packages' lifecycle, although they might be. so exact matching based on
the full package version is not possible.

there's the special case of X being 0 that change some details, but not
in a way that matter. there might also be more librust-foobar[-X]+F-dev
binary packages with corresponding provides, or additional provides
without their own binary package, where F is some feature of the crate.
AFAICT this also shouldn't change anything about the analysis below,
provided the set of features is identical across the two source
packages, so I am going to ignore this as well :)

On Fri, Apr 28, 2023 at 04:21:08PM +0100, Peter Green wrote:
> On 28/04/2023 06:05, Helmut Grohne wrote:
> > Can you go into more detail as to what you mean with "don't support
> > version ranges"?
> 
> You can place a lower bound on the version, place an upper bound
> on the version or constrain to a precise version. But you can't
> constrain to a range of versions.

exactly, and since the conflict ist actually on an upstream-version
specific path, and the debian revisions of the two involved source
packages might not be related/identical, we can't do an exact match.

so the only solution IMHO is to have an upper bound on the next upstream
version, i.e. if the semver "fork point" is X.Y.Z, << X.Y.(Z+1)~ ? an
unversioned librust-foobar-dev that contains crate version > X.Y.Z, even
with the first two components identical, is okay, since the path in
/usr/share/cargo/registry is different then.

> >   In principle, you could declare the Replacs to be
> > slightly broader than necessary (i.e. instead of declaring against the
> > specific range, you can replace all old packages). Do you agree that
> > this is feasible?
> It would indeed be feasible to continue declaring the breaks
> against the virtual package, while declaring the replaces
> against the physical package.
> 
> My only concern is this may bring in bug reports complaining
> about a replaces without a matching breaks. Tehcnically I
> don't see anything in policy actually forbidding such but
> it goes against the norm.

the question is whether switching just the Replaces as opposed to both
B+R to the unversioned package has any advantage in the way apt and co
handle it? if not, then switching the B+R to have the same dependency
might be a good idea if just for consistency's sake.

> >   5. In a different bug, Samuel Thibault observed that the target package
> >      was not part of bullseye. As such, an upgrade scenario involving it
> >      was unlikely. All of the 6 affected librus-*-dev packages are not
> >      part of bullseye. We could argue that the risk of these effects
> >      showing up in practice is not big enough to warrant an invasive fix.
> >      Rather, we'd downgrade them to important (and upgrade to serious
> >      when we see them in the wild) and close them after bookworm (since
> >      we don't support skip upgrades).
> 
> While the target packages aren't part of bullseye, they could well end up pulled
> in as part of an upgrade, since the purpose of these packages is to continue
> providing older versions of a crate when the main package of the crate is
> upgraded.

semver suffix packages are problematic at two points:

1) when they are initially created in sid, where for a time (until the
unversioned package is updated) both source packages and their binary
packages might co-exist in sid and/or testing

this could be avoided by always updating the unversioned first, but one
common reason is avoiding breaking half of the archive when doing such a
transition, so..

2) if the "fork point" corresponds to the version in the soon-to-be-old
stable release, and the semver suffix package is still in testing when
that becomes the stable release (as then the unversioned package in
(old)stable and the semver suffix package in (new)stable ship the same
path)

is 2) actually true for any of the filed bugs at the moment? if not,
then IMHO we can ignore this until bookworm is out the door, decide on
and ship the fix in debcargo, and ease out the problematic packages by
updating them (basically option 5). like Peter noted below, these are
packages almost certainly only installed on developer machines (or the
odd programmer's who doesn't use the stock, upstream way of doing rust
developement) and in 'unclean' build environments, not on regular
servers or end user machines.

> It then comes down to unpack order, if the package manager unpacks the upgrade
> before the new package then all is cool. OTOH if the new package is unpacked
> first you would have a conflict.
> 
> What I don't know is how real-world package managers handle installation
> order and thus how likely seeing the "bad" ordering is in practice.
> 
> The issue won't go away after bookworm. These particular packages won't be
> affected for a bookworm->trixie upgrade but future packages almost certainly
> will be.
> 
> On the other hand these are packages that are mostly used as
> build-dependencies and are rarely installed on end-user systems.
> 
> > > Do any other members of the rust team have an opinion on this? I'm
> > > personally inclined towards option 1 and intend to implement it if
> > > noone objects in the next couple of days.
> > Let me know if you see 4 as a viable option.
> I would say it's viable.
> > > Do the release team have an opinion on this? It looks like only one of
> > > the packages involved (rust-env-logger-0.7) is a key package.
> > I filed these as serious, because these are policy violations with a
> > trivial fix. You made it quite clear that the premise of the fix being
> > trivial is false here. If the cure is worse than the symptoms
> 
> I don't think the cure is worse than the symptoms.

I am going to circle back here, and try to summarize the proposed
options. besides option five noted above (which still needs to select
either of these options here as future fix!), we have

option "1", instead of B+R, do a "whole package replacement" with
Replaces+Conflicts on the virtual package

option "2", B+R, but with << next upstream version, and on the "real"
binary package

option "4", B like now, R like in option 2

1 would support virtual packages as target (so we could use
librust-foobar-X-Y-Z-dev, which is provided by all rust-foobar and
rust-foobar-X packages in version X-Y-Z, which is exactly the set of
conflicting packages shipping the problematic path). I am not sure if
it's actually what we want in the "problematic point 2" above, since if
I am on bullseye and have librust-foobar-dev installed, I likely want
that to be upgraded to the newer librust-foobar-dev, and I think the
Conflict of bookworm's librust-foobar-X-dev would remove it instead?

2 and 4 are similar, 4 might(?) make lintian unhappy and is a bit
strange, the real question is whether it's better w.r.t. ordering when
upgrading and/or resolving.

with 2, we would switch from (no version constraints on the dpkg/APT
level!):

librust-foobar-X-dev
  Breaks: librust-foobar-X-Y-Z-dev
  Replaces: librust-foobar-X-Y-Z-dev

where librust-foobar-X-Y-Z-dev only exists as virtual package, provided
by both librust-foobar-dev and librust-foobar-X-dev, to

librust-foobar-X-dev
  Breaks: librust-foobar-dev (<< X.Y.(Z+1)~)
  Replaces: librust-foobar-dev (<< X.Y.(Z+1)~)

where librust-foobar-dev is both a real package (that might or might not
be around with a matching version) *and* a virtual package provided by
librust-foobar-X-dev itself, in a version matching the constraint. the
latter only "counts" for Breaks, since Replaces without Conflicts
ignores virtual packages.. I am not sure whether the addition of a
version constraint for the Breaks (on a self-provided virtual package)
changes anything here either for upgrading, or for (build) dependency
resolution in general..

with 4, only the previously ignored Replaces changes, and without a
Conflicts, it would only apply to the unversioned package.

I see no practical issue with 2 meaning we can't have multiple semver
suffix packages variants of a single crate installed - having the
unversioned and one semver suffix package in one suite at any given time
should already be the exception, having more than that should be even
more rare, and there should be no need to have them *installed* at the
same time since these packages are only used as build deps.

I can volunteer replacing (heh) the currently broken implement in
debcargo with whatever we settle on.