Bug#886494: polymake: Can't locate loadable object for module Polymake::Ext in @INC

[Niko Tyni]

Thanks for your reply, Benjamin! I'm copying perl at packages.debian.org
again and looking for input from my comaintainer Dominic. See below.

On Sat, Jan 06, 2018 at 10:07:47PM +0100, Benjamin Lorenz wrote:
> On 01/06/2018 08:08 PM, Niko Tyni wrote:
> > Running polymake currently fails with
> >
> >   $ polymake
> >   Can't locate loadable object for module Polymake::Ext in @INC
> >
> > Apparently this is because it was built with Perl 5.26.0, but
> > we've since moved to 5.26.1.
> >
> > I see two better alternatives:
> >
> > 1) [preferrable] fix polymake not to install its extensions in a
> >   versioned directory (currently /usr/lib/polymake/perlx/5.26.0).
> 
> This would work for minor updates (5.26.0 -> 5.26.1 seems to work) but
> the extension does depend heavily on the major version, so I dont think
> this is a good idea.

I was looking at the Debian packaging point of view, where it would be
fine, as there's no expectation of multiple versions being installed at
the same time, and package dependencies ensure binary compatibility with
/usr/bin/perl. But yeah, I can see that it's problematic upstream. It
seems that it would be possible to make that configurable, though.

> > I note that the confusing @INC list above (it has
> > /usr/lib/polymake/perlx/5.26.0 so it looks like it should work) is
> > because "use lib" will add versioned subdirectories to @INC if it
> > finds any, but will apparently *not* add arch-specific subdirectories
> > (like /usr/lib/polymake/perlx/5.26.0/x86_64-linux-gnu-thread-multi)
> > under those, and that's what would be needed here. I'm not quite sure
> > if this is a glitch in lib.pm's handling of binary-compatible versions
> > ($Config{inc_version_list}).
> 
> This is a good point, it looks like the Config.pm contains:
>     inc_version_list => '5.26.0',
> 
> but the perl INSTALL file states:
> 
> > If you do want to use modules from some previous perl versions, the
> > variable must contain a space separated list of directories under the
> > site_perl directory, and has to include architecture-dependent
> > directories separately, eg.
> >
> >       sh Configure -Dinc_version_list="5.16.0/x86_64-linux 5.16.0" ...

Thanks, I had not noticed this! So it's not a glitch in lib.pm
but expected behaviour.

> It seems the perl Configure script tries to generate this list
> automatically (probably from $sitelib=/usr/local/share/perl/5.26.1) but
> failed to pick up the arch-specific directory because it might not exist?

We already disable the autogeneration and pass inc_version_list to
Configure manually when building the perl package.

> I dont have such an installation at hand but this could be tested by
> adding the arch-specific path for inc_version_list in
> '/usr/lib/x86_64-linux-gnu/perl/5.26.1/Config.pm' manually first. If
> this helps, the Configure command for perl should probably get an extra
> -Dinc_version_list=.

I tried it out locally with
 inc_version_list='5.26.0 5.26.0/x86_64-linux-gnu-thread-multi'
and it indeed fixes this issue, at the cost of three useless stat()
calls on every perl invocation.

 stat("/usr/local/lib/site_perl/x86_64-linux-gnu-thread-multi", ) = -1 ENOENT (No such file or directory)
 stat("/usr/local/lib/x86_64-linux-gnu/perl/5.26.0", ) = -1 ENOENT (No such file or directory)
 stat("/usr/local/share/perl/5.26.0", ) = -1 ENOENT (No such file or directory)
+stat("/usr/local/lib/x86_64-linux-gnu/perl/5.26.0/x86_64-linux-gnu-thread-multi", ) = -1 ENOENT (No such file or directory)
+stat("/usr/local/share/perl/5.26.0/x86_64-linux-gnu-thread-multi", ) = -1 ENOENT (No such file or directory)
 stat("/usr/lib/x86_64-linux-gnu/perl-base/5.26.0", ) = -1 ENOENT (No such file or directory)
+stat("/usr/lib/x86_64-linux-gnu/perl-base/5.26.0/x86_64-linux-gnu-thread-multi", ) = -1 ENOENT (No such file or directory)
 stat("/usr/lib/x86_64-linux-gnu/perl-base/x86_64-linux-gnu-thread-multi", ) = -1 ENOENT (No such file or directory)

Unlike the others, these new directories are never going to exist on
Debian systems, because our sitelib and sitearch are separated between
/usr/local/share and /usr/local/lib, as opposed to the upstream structure
of sitearch being a subdirectory of sitelib.

The wasted stats could certainly be fixed by modifying our relevant
changes to perl.c
 https://sources.debian.org/src/perl/5.26.1-3/debian/patches/debian/mod_paths.diff/
but I haven't looked into that properly yet.

I'm rather undecided here. This is the first time that not having the
arch-specific subdirectories on inc_version_list has been a problem.
It seems to require a specific set of conditions to become one:

- the software has their own versioned installation directory
  with arch-specific code
- the software is not rebuilt when the system perl changes to a newer
  minor (binary compatible) release
- the software is originally relying on lib.pm's looking in $dir/$archname,
  and needs inc_version_list to produce matching behaviour

which explains why we've been fine with the current situation for 15+ years.

On the other hand, the cost (three more wasted stats at the worst)
is not very great, and following the INSTALL recommendations does seem
like the right thing to do.

Dominic, any opinion?
-- 
Niko Tyni   ntyni at debian.org