[Debian-med-packaging] Bug#738962: python-csb: More issues with the tests

[Dmitry Shachnev]

Hi Tomas,

On Fri, 14 Feb 2014 15:03:28 +0100, Tomas Di Domenico wrote:
> I'm a bit confused about the dependencies and referencing to the
> source or binary packages, and whether they're missing from only the
> tests or also from the main package. Where should the references to
> the binary package python3-numpy and -scipy be added?

I am not very familiar with this package, but searching for numpy and
scipy in the code returns a lot of results, thus I believe these
dependencies should be added to the binary packages.

Alternatively, you can make binary packages suggest/recommend them,
but then you will need to add those packages as dependencies in
debian/tests/control.

>> I would write the command like this:
>> 
>> for py in $(pyversions -i) $(py3versions -i); do PYTHONWARNINGS=d
>> $py test/app.py --generated-resources /tmp --verbosity 2 done
>> 
>> Note that I also increased verbosity to get the more understandable
>> output in case of failures.
>
> Thank you very much for your suggestion. I've updated the code
> accordingly.

The Git looks out of date (it does not even include the latest release),
where did you update the code?

>> 3) One of the tests fails (at least with Python 2.7): 4) The tests
>> failures are ignored. When the test fails for python2.7, the next 
>> tests are run.
>
> After I figure out the correct setup for the tests I'll try to
> reproduce 3) and  4), and will probably get back to you with more
> questions ;)

Note: this failure seems to only happen on i386 and not amd64.

For running the tests, I personally use just adt-virt-null runner
in a clean environment, but probably using something like
adt-virt-schroot will be an easier solution. See this message
for some instructions:

  https://lists.debian.org/debian-devel/2014/01/msg00507.html

And yes, it is very sad that we have no documentation on that.

>> Restrictions: allow-stderr
>
> Is the addition of this restriction a good idea, or should I rather
> try to get upstream to change the behaviour?

Printing to stderr is default behaviour of unittest module, so it is
not specific to python-csb. Most Python packages with autopkgtests
either use that restriction or manually redirect output using 2>&1.

I am attaching a patch that addresses 1), 2) and 5), and also adds
a dependency on python3-all to ensure that tests are run for all
supported Python 3 versions. That patch will make the tests green
on ci.debian.net, but will not fix the real failure (3).

--
Dmitry Shachnev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: autopkgtest_fixes.diff
Type: text/x-diff
Size: 1487 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/debian-med-packaging/attachments/20140215/b2642cdf/attachment-0001.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/debian-med-packaging/attachments/20140215/b2642cdf/attachment-0001.sig>