RFC: test helper for executables
Antonio Terceiro
terceiro at debian.org
Thu Mar 19 16:41:56 UTC 2015
On Wed, Mar 18, 2015 at 09:42:10PM +0100, Christian Kastner wrote:
> On 2015-03-16 15:38, Antonio Terceiro wrote:
> > On Sun, Mar 15, 2015 at 04:06:33PM +0100, Christian Kastner wrote:
> >> The tests I want to add are just of a very basic nature; I merely want
> >> to increase the _breadth_ of testing. Increasing the _depth_ will be up
> >> to the individual maintainers as they have the experience with the
> >> package necessary to do this.
> >
> > If you end up looking for a place to consolidate shared code that will
> > be used in multiple packages, we can think about adding it to autodep8.
>
> I've started to work on some packages, by descending popcon, and already
> encountered a few without any sort of tests.
>
> For packages providing executables, I chose the common pattern of
> creating a test runner [1] that will compare the output of a test with
> an expected output, and fail if they do not match:
>
> debian/tests
> ├ [control]
> │ > Tests: mytests
> │
> ├ [mytests]
> │ > adt-helper-executables mytests.d [2], [3]
you can use
Test-Command: adt-helper-executables mytests.d
directly in debian/tests/control and skip the extra script.
> │
> └ mytests.d/
> ├ [testname.1.in]
> │ > #!/bin/sh
> │ > echo "Foo!"
> │
> ├ [testname.1.expected]
> │ > Foo!
> │
> ├ [testname.2.in]
> | > #!/bin/sh
> │ > factor 1334
> │
> ├ [testname.2.expected]
> │ > 1334: 2 23 29
> │
> ....
>
> This makes writing tests for executables quite trivial. The boilerplate
> of running an autopkgtest test is consolidated in the helper, and actual test
> creation is reduced to a few lines of code (3 in the best case).
>
> I plan to test the efficacy of this approach against src:shadow, which
> produces 36 executables -- some of them essential -- yet has no testing
> whatsoever. I'll report back with my findings (that will take a while as
> I will be busy with something else over the next few days).
>
> You can find a working prototype, with more documentation, in this
> version of debianutils I created [5].
>
> This is purely exploratory at this point; I won't submit any tests to the
> maintainers until I'm more confident that it's going in the right direction
> (or can easily be adapter later on). Feedback would therefore by highly
> welcome!
>
> Regards,
> Christian
>
> [1] Don't mind the name, it is purely arbitrary at this point.
>
> [2] I'm assuming for now that the helper is somewhere in PATH.
>
> [3] As you can see, the "mytests" autopkgtest is just a trampoline.
>
> [4] An interesting possibility would be to allow regexs in the .expected
> files, but at believe it would premature to consider this at this point.
>
> [5] http://www.kvr.at/debian/pool/main/d/debianutils/debianutils_4.4+nmu1.dsc
This looks nice so far.
--
Antonio Terceiro <terceiro at debian.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20150319/cefcbece/attachment.sig>
More information about the autopkgtest-devel
mailing list