Bug#799444: pkg-perl-autopkgtest: prove --merge breaks some test suites
Niko Tyni
ntyni at debian.org
Sat Sep 19 09:07:14 UTC 2015
Package: pkg-perl-autopkgtest
Version: 0.25
In the 'smoke' tests, which run the upstream test suite against an
installed module, we currently use the 'prove' '--merge' option.
This interleaves STDERR and STDOUT, trying to preserve their relative
ordering. While wonderful when it works, this apparently breaks some
test suites: at least libterm-progressbar-perl (see #785045) and possibly
libcapture-tiny-perl (#785022).
Even prove(1) says '--merge' is risky:
Caveat: this is a bit of a kludge. In particular note that if
anything that appears on STDERR looks like a test result the test
harness will get confused. Use this option only if you understand
the consequences and can live with the risk.
I'm inclined to drop the option rather than make it configurable.
This will make the logs a bit harder to read, but should make the
tests more robust.
I note that we still need to redirect STDERR, otherwise output on that
will be considered a test failure. Alternatively we could change the test
definitions in the autodep8 package to add the restriction 'allow-stderr'.
Adding a 2>&1 to the 'prove' call in the 'smoke' script seems easier.
Thoughts welcome, I'll wait a bit before implementing this.
--
Niko Tyni ntyni at debian.org
More information about the pkg-perl-maintainers
mailing list