[buildd-tools-devel] testers wanted: sbuild and build-dependencies

Roger Leigh rleigh at codelibre.net
Wed Nov 10 16:39:56 UTC 2010


On Wed, Nov 10, 2010 at 01:00:29PM +0100, Goswin von Brederlow wrote:
> Roger Leigh <rleigh at debian.org> writes:
> 
> > A new "aptitude" resolver has been written by Marc 'HE' Brockschmidt.
> > This creates a dummy "dependency package" which is installed and
> > contains Depends and Conflicts for all the appropriate Build-Depends
> > and Build-Conflicts, and uses aptitude to install and remove the
> > appropriate packages to satisfy them.  This has been in use on our
> > experimental buildds for about a year, and we now want to look towards
> > making it the default.  However, we need some more widespread testing
> > to make sure the dependency resolving doesn't result in inconsistent
> > installation of packages and hence inconsistent library dependencies
> > or break building of any packages etc.
> 
> Yet another or did he take the one from pbuilder?

It's new, but based upon the ideas in pbuilder.

Just to mention in passing, I also wrote an additional "apt"
resolver last night, based on the aptitude resolver, which works
in exactly the same way.  Not usable under all circumstances yet,
but also available (in git only at present) for testing.  The
main issue with apt is getting "apt-get -yf install" to not choose
to remove the dependency package as a solution!  Currently looking
at the possibilities here--any thoughts welcome!

> This destroys the determinicity of build dependencies. So if I say
> 
>   Build-Depends: lib-new-name | lib-old-name
> 
> so the package builds (for users) in both stable/testing and unstable it
> is no longer given which library is used by the unstable buildds. Some
> will pick lib-new-name and some, where the new lib isn't compiled yet,
> lib-old-name. And so on.
> 
> I always heard determinicity was a wanted feature for the buildds.

It is, and this is something to look at carefully.  In the example
above, this is a very real problem (which could be rectified by
having stricter build-deps).

The root problem here is that at any given moment in time, unstable
is not in a consistent state--packages may be outdated on several
architectures, and so a package build may use (and depend on)
different versions in different architectures.  There are several
places this could be fixed:

• we could keep packages out of the archive until built on all
  architectures.
• we could dep-wait a package until all its deps are the same
  across all architectures
• we could keep the current approach of banning alternatives
  entirely (but note that this does not solve the entire problem,
  it's still easy to get inconsistency anyway)

The existing approach to determinism is not to support alternatives
at all, which is clearly not ideal.  While I don't think we should
be encouraging the use of alternative build-deps, this is one of the
most commonly reported bugs in sbuild--there are valid use cases for
them.

All the modern buildds are based around snapshotting the build chroot,
which gives us a clean slate at the start of each build.  Providing we
have a common base environment across all our buildds, this should
provide a measure of stability--I don't think any of the existing
resolvers choose packages randomly--so the resolver should pick the
same package sets across architectures so long as the available
packages are the same.  Now, this might not always be true, but this
also affects the existing resolver as well.  Not letting binary
packages enter unstable until built on all architectures would be a
solution to this.  I'd have to say, solving the problem at its root,
and giving our tools more flexibility would be the ideal outcome.

I think it's also useful to examine exactly what we mean by
determinism.  I don't think being deterministic over time is useful--
the available packages can change, which is the primary reason for
binNMUs.  Being deterministic across architectures at a single
point in time is probably the main goal.  So long as the buildds
for each arch do the same thing when a package is queued for build,
that's probably sufficient for our needs.

If apt-get and/or aptitude provide any means of tweaking how they
consider alternatives, then that would also be something to look at.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20101110/a42ebda7/attachment.pgp>


More information about the Buildd-tools-devel mailing list