Bug#764868: IO::Socket::SSL - external tests in test suite can be disabled but are enabled by default

[Niko Tyni]

Hi Steffen, thanks for looking at this and for your work on free software!

On Sat, Oct 11, 2014 at 11:12:52PM +0200, Steffen Ullrich wrote:

> I can understand that you are not comfortable to let arbitrary tests connect
> to external sites in all situations. But, in my opinion, anybody who does not
> like this should run any builds and tests within restricted environments
> without or with only limited network access anyway.

Yeah, there are conflicting requirements here and the same setup can't
cater for everybody. The tests are certainly useful, and I very much
appreciate you having separated them under t/external/ and having
Makefile.PL prompt about them. This bug is mostly about the Debian
packaging not using that prompt (although an environment variable would
be more convenient, see below.)

Your point about the privacy conscious being able to firewall their
build environment is a good one (and has a basic requirement that the
test suite passes without network access, which yours does.)

This is basically opt-in privacy; I'm a bit uncomfortable with that and
would like to have privacy be the default in a Debian package build.
I don't have a stance about the upstream default for CPAN builds.

The current Makefile.PL prompt() solution is not the easiest thing for
noninteractive builds, which end up always taking the default choice.
An environment variable like NETWORK_TESTING or NO_NETWORK_TESTING that
we could set for Debian builds would solve this fine.  It's unfortunate
that such a thing isn't standardized yet. I found this related thread:

 http://www.nntp.perl.org/group/perl.cpan.workers/2014/06/msg1117.html

which understandably seems to lean a bit on the opt out side, i.e.
skipping the tests if the user has set NO_NETWORK_TESTING=1.

> Apart from that I'm happy to check any kind of environment variables which
> would indicate that the user is not willing to run external tests. Just let
> me know the details.

Thanks! I think the least intrusive thing for now would be to change the
default of the Makefile.PL prompt if NO_NETWORK_TESTING is set. Would
that be OK with you?

We can also patch that in just for the Debian package for now if you'd
prefer to wait and see if a consensus emerges on the CPAN side somehow.

I'm not sure anymore if we have a hard policy in the Debian project about
cases like this, though I (perhaps optimistically?) think we do have a
consensus that network tests should generally be disabled by default.

I think I'll take the general issue up on the broader debian-devel
mailing list to gather some opinions.

Thanks again,
-- 
Niko Tyni   ntyni at debian.org