[Reproducible-builds] [Reproducible] On making Stretch self-contained IRT to reproducibility

[Holger Levsen]

Hi,

On Mittwoch, 24. Februar 2016, Niels Thykier wrote:
> The topic of rebuilding all of Stretch to make it self-contained (IRT to
> reproducibility) was brought up on the release IRC meeting today (topic
> originally proposed in [1]). The highlights:
>  * To my knowledge, only people from the release team were present.
>    - The meeting minutes are available at [2].
> 

thanks for having this discussion and forwarding it to us! Much appreciated.

>  * We had at least 3 concerns brought up:
>    - Possible lack of buildd resources to do the rebuild.  Notably, due
>      to Multi-Arch:same we would generally need to do the rebuild on all
>      architectures.
>    - In our current infrastructure, arch:all packages cannot trivially
>      be rebuilt.
>    - Build-dependency loops would need to be handled somehow.

I think you've been discussing topics which are not that relevant yet…:
 
>  * At this point, we do *not* think it is feasible to do a full rebuild
>    of the archive for Stretch to make it self-contained (IRT to
>    reproducibility).

and this is mostly because we are not there yet: "85%" might sound impressive, 
but have a look at 
https://tests.reproducible-builds.org/unstable/amd64/pkg_set_key_packages.html
and you will see that we still have >500 source packages to fix for the key 
packages alone.

We won't reach this for Stretch and so Stretch *cannot be* self contained 
reproducible.

Even "just" achieving this for 
https://tests.reproducible-builds.org/unstable/amd64/pkg_set_build-essential-
depends.html
seems unlikely to me.

>  * We are currently (also) awaiting better dak support so we know what
>    the support of .buildinfo etc. will be like.

Yup, that's really the big thing we are waiting for (tracked as #763822 titled 
"ftp.debian.org: please include .buildinfo file in the archive"), besides a 
dpkg which can produce reproducible binary packages and which generates 
.buildinfo files.

And once we have that, we'll have 0% (zero) reproducible packages in Debian.

Then we will need rebuilds *of everything*, so that we'll get packages with 
.buildinfo files into the archive.

And then we'll the problems described at the beginning (eg "In our current 
infrastructure, arch:all packages cannot trivially be rebuilt.") - and I agree 
it's good to start discussing this now.

But this is not for making "Stretch self-contained IRT to reproducibility" but 
just for "making Stretch partly reproducible somehow" ;-)


Obviously I might miss something as well, but this is my understanding of 
where we're at regarding reproducible Stretch.

> Please note that we may /not/ have fleshed out all problems during the
> meeting.
> 
> 
> On the arch:all rebuild-ability
> ===============================
> 
> Rebuilding arch:all packages currently requires manual uploads of all
> packages.  While we have "arch:all" buildds, we do *not* have support
> for binNMUing arch:all packages in general.  A very limiting factor is
> the substvar handling, which assumes that the version of the arch:all
> package does not change during binNMUs (e.g. by using ${source:Version}).

ouch. Big ouch. Does that affect all "arch:all" packages or only some? If the 
latter, how many approximatly?

I dont think doing thousands of sourceful uploads is realistic nor useful, but 
*if* we have to do that, we should think about also including fixes which will 
allow arch:all binNMUs in future.



cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20160309/46885cbc/attachment.sig>