Bug#784845: libdevel-gdb-perl: autopkgtest failure in t/expect.t

[Santiago Vila]

On Tue, Aug 16, 2016 at 10:04:01PM +0300, Niko Tyni wrote:
> On Tue, Aug 16, 2016 at 07:18:39PM +0200, Santiago Vila wrote:
> > severity 784845 serious
> > retitle 784845 libdevel-gdb-perl: FTBFS: t/expect.t #8 sometimes fails
> > thanks
> 
> > Bugs about packages not building from source are RC because there is a
> > release goal saying "packages must autobuild".
> 
> It did autobuild fwiw.
> 
>   https://buildd.debian.org/status/logs.php?pkg=libdevel-gdb-perl&ver=2.02-3&arch=all

A successful build log just means it did autobuild the last time it
was tried, not that the package "usually autobuilds".

The last time could have been a lot of time ago, maybe with a
different compiler or a different perl interpreter.

It does not mean the source package is currently ok. You will notice
that FTBFS bugs are RC even if the last official build was ok. It's
not just the last tried build what counts but our expectation about
the *next* build.

> > A build which is only successful "most of the time" should not count
> > as a package which builds from source, so I'm raising severity accordingly.
> > 
> > [ If my interpretation of release policy is incorrect, someone please
> >   tell me the amount of probability of failure which is acceptable, because
> >   release policy does not talk about probabilities anywhere ].
> 
> Quoting myself in a similar case in #799718:
> 
>   While I'm not going to start a severity war (and agree that the package
>   should be fixed), this is not the traditional interpretation. FTBFS bugs
>   have been routinely downgraded to 'important' in the past when the build
>   failures were nondeterministic and the build succeeded part of the time.
> 
>   My understanding is that the baseline requirement from the release team
>   is that 'buildds need to be able to build the packages'.

Hmm, but what's the meaning of "need to be able"?

IMHO, buildds need to be able to build the packages at any given time,
which means they should always succeed.

Otherwise, if we are just happy with "buildds need to be able to build
the packages", and a build which fails randomly is not considered RC,
because then you retry the build until it builds ok, then *any*
probability of success > 0, not matter how small, will make the
buildds to be "able" to build the packages: you just need to retry
enough times.

By this interpretation of policy, a package which builds ok with
probability 1/2 would be ok to be released.

But of course I think our standards should be higher.

BTW: In this particular package, I decided to raise the severity after
finding the failure myself while checking for bugs regarding "dpkg-buildpackage -A"
*and* also getting unsuccessful build logs from reproducible builds as well.

So in some sense, I'm already applying not the policy which I believe
is the correct one (packages should build with 100% probability), but
the unwritten policy that packages should autobuild with "very high
probability", it's just that the probability of this package in
particular is certainly too low for what I would consider reasonable
(as I managed to get failed logs in reproducible builds and my own
autobuilder).

Please note that I'm not even asking that the .deb package is always
the same (that would be the goal of the reproducible builds project) I'm
just saying that package building should not be a lottery.

Thanks.