[Reproducible-builds] automated diffoscope for parallel build bugs

[Holger Levsen]

Hi,

On Fri, Aug 19, 2016 at 12:58:24AM +0100, Chris Lamb wrote:
> > I understand that building with parallel disabled takes much longer
> > for many packages so I don't know if this is just a lack of resources
> It's more that we have two builds and I think (?) we would prefer to do
> different values of N > 1. So a different kind of resource problem.

this and there are more issues:

- quite a lot of packages force parallel builds in some way or another
  in their build systems, so these wouldnt be tested in this regard
- this aint a real world scenario for our use case, which is testing and
  working on reproducible builds. IOW: this is just another area of QA
  work, which I agree should probably be done, but it's out of scope for
  our project and doing it would draw "human ressources".
- actual tests would run a *a lot* slower, thus we would see the results
  were are interested in, at a later time, even more disconnected from
  the actual upload
- the overall results would become older

> It wouldn't be too much work to run such an test yourself or to use
> our infrastructure.

it's actually rather easy to build all packages in a loop… seriously, I
mean:

cd $path_to_svn
for i in $(find * -type d) ; do 
	cd $i && debuild && cd ..
done

now you will have a lot of debs :) copy them away, apply whatever means
to enable or disable parallelism and run that loop again.

now do another loop, diff each pair of debs and if they are the same,
delete them. investigate the rest using diffoscope :)
 
> The problem is all about how you use the results; in the case where a
> package differed between builds you would not know whether this was
> due to the parallelism that was introduced or whether the package
> is inherently non-deterministic.

yup, but 
https://tests.reproducible-builds.org/debian/unstable/amd64/pkg_set_gnome.html
suggests most of the core gnome packages are actually reproducible
today…

> > Building everything in pkg-gnome svn individually with and without
> > parallel and then running diffoscope by hand to compare every binary
> > package is not much fun.
> 
> Right, so make all the pkg-gnome packages build reproducibly without
> paralellism enabled (which you should do anyway) and then simply
> build with parallel enabled.

yes, I cannot imagine there's something which is reproducible if build
in parallel but not if not.

So indeed, making packages build reproducible when build in parallel
will fix your original issue as well! :)


-- 
cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 811 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20160819/2a9a38a2/attachment.sig>