Bug#785659: libdist-zilla-perl: FTBFS: t/plugins/testrelease.t failure

[Niko Tyni]

On Tue, May 19, 2015 at 10:00:59AM +0200, gregor herrmann wrote:
> On Mon, 18 May 2015 23:37:26 +0300, Niko Tyni wrote:
> 
> > Package: libdist-zilla-perl
> > Version: 5.020-1
> > Severity: serious
> > Tags: stretch sid
> > 
> > This package fails to build from source for me on current sid, but not
> > on jessie.  No idea yet which package upgrade broke it.
> 
> This looks like #723930 reappearing.
> 
> What has changed since jessie is that debhelper now indeed runs tests
> verbosely.
> 
> Maybe the "test-in-the-test" (if I'm reading the output correctly) is
> the problem here?

Ah, it took some time but I finally understand the issue.

The test-in-the-test is indeed the problem: the whole output of building
a test "DZT" module and running its tests is part of the TAP output for
the main tests in t/tester.t and t/plugins/testrelease.t. When the DZT
module tests are run in verbose mode, they get mixed up with the main
TAP. If the DZT module tests just output 
  t/basic.t .. ok
that's all fine and doesn't mess up the main TAP.

Just about the only way to make the sub-build tests run in verbose mode
seems to be invoking make with TEST_VERBOSE=1, because that gets passed
to the sub-make via MAKEFLAGS. This happens to be exactly what the newer
debhelper does.

So it can be triggered with
 make test TEST_VERBOSE=1
which can be reduced to for instance
 MAKEFLAGS="TEST_VERBOSE=1" prove t/tester.t

Still not sure if we need any other fix than forcing TEST_VERBOSE=0.
-- 
Niko Tyni   ntyni at debian.org