where is Pd's "-stdlib"? (was Re: request sponsor/upload for pd-pdstring)

Tue Oct 4 09:19:52 UTC 2011

On Mon, 2011-10-03 at 13:06 -0400, Hans-Christoph Steiner wrote:
> On Oct 3, 2011, at 3:54 AM, IOhannes m zmoelnig wrote:
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > @pd-dev: in the course of making packages for debian, we discovered
> > another slight problem with the "-stdpath" and "-stdlib" flags for
> > declare (and probably this also expands to the "-nostdpath" startup
> > flag). following is an excerpt of the discussion in the debian  
> > packaging
> > team:
> >
> >
> > On 2011-10-01 14:11, Roman Haefeli wrote:
> >>
> >> On Fri, 2011-09-30 at 17:02 +0200, IOhannes m zmölnig wrote:
> >>>
> >>> i'm not entirely sure though (given the nastiness of [declare])
> >>> if you think that it is a bug in "puredata-core", please file a  
> >>> bugreport.
> >>
> >> Yeah, that is indeed the case. Before filing a bug report, I'd like  
> >> to
> >> clear up the meanings of the different paths.
> >>
> >> /usr/lib/pd/extra
> >>  Am I right in assuming that this path is supposed to be searched by
> >>  all flavors of Pd (all packages that provide the virtual package  
> >> pd)?
> >>  This also the path where usually external libraries are installed to
> >>  because from there they can be loaded from any flavor of pd?
> >>
> >> /usr/lib/puredata/extra
> >>  is only searched by puredata / pd from the puredata package?
> >>  This is where libraries are installed that only are suitable for
> >>  the pd provided by the puredata package?
> >>
> >> /usr/lib/pd-extended/extra
> >>  is only searched by pdextended / pd from the pd-extended package?
> >>  Libs that are only useful with pdextended go there?
> >>
> >> If that is the case, then there is definitely a bug in the puredata- 
> >> core
> >> package as it is ignoring /usr/lib/pd/extra.
> >
> > it might as well be a bug in puredata upstream (that's why i want to
> > discuss it; probably a more appropriate place for discussion is the
> > pd-dev mailinglist which i include in the recipients)
> >
> > imho, the issue boils down to the question "what are stdpaths?" (and i
> > assume that "stdlibs" are std because they live in "stdpaths").
> >
> > for the sake of simplicity, i will only talk about the "linux" version
> > of Pd (and with Pd i mean Pd-vanilla)
> >
> > before Pd-0.43 (vanilla!), there was only a single "stdpath", which  
> > was
> > the path were the Pd binaries lived in. this usually was
> > /usr/local/lib/pd/ or /usr/lib/pd/
> >
> > since 0.43, a few more paths have been added, namely:
> > /usr/local/lib/pd-externals and ~/pd-externals
> > on Debian and derivatives, yet another path is added: since Pd is
> > installed into /usr/lib/puredata/ (in order to allow pd-extended live
> > side by side with puredata), the path "/usr/lib/pd" is also added as a
> > "common system-managed search path".
> >
> > now all these paths are handled separately from the user defined
> > search-paths; e.g. they do not show up in the path dialog, and they  
> > can
> > be disabled with the "-nostdpaths" flag.
> >
> > otoh, [declare] has not adapted to these changes.
> > if you add "-stdpath extra/foo", it will only search in
> > /usr/local/lib/pd/extra/foo (given that pd is installed in
> > /usr/local/lib/pd), but it will not search in
> > /usr/local/lib/pd-externals/extra/foo nor in ~/pd-externals/extra/foo.
> > (the same applies for the additional Debian-specific search path
> > /usr/lib/pd/extra/foo).
> > hence i do think that the problem is general problem with Pd-vanilla
> > (and not specific to Debian; it only shows here)
> 
> My guess is that [declare] should be adapted to consider as stdpath  
> the same stuff that -nostdpaths does.

I agree. I think that the various standard paths shouldn't be treated
differently. As a patch creator I don't care whether I installed the foo
library in ~/pd-externals or /usr/local/lib/pd/extra, in either case I
want to load a library the same way. 

I believe it makes sense to distinguish only two different scopes here:

* system environment
    _all_ standard paths belong to it and the '-stdpath' and '-stdlib' 
    flags of [declare] should expand all those paths, respectively try 
    to load libraries from those paths.

* patch / local environment
    paths relative to the patch's location. The '-lib' and '-path'
    flags of [declare] cover those. 

I can't come up with another meaningful scope. But if there is any,
probably new flags for [declare] would be necessary.

> >> This also means, that currently all Pd libraries in unstable that
> >> install to /usr/lib/pd/extra (most of them do) are currently  
> >> broken, as
> >> there is no proper way to actually load them in pd (you still can
> >> specify the absolute path to the library, which renders your patch
> >> unportable to other OS').
> >
> > you could also simply start Pd with "-lib foo" and it will just work.
> > so i wouldn't consider "all broken".
> > obviously, we lack the possibility to express a library dependency
> > within the patch, which is a shame (but which is also due to the  
> > current
> > broken implementation of [declare] in general).
> >
> >
> > anyhow, what i'm mainly asking is, whether "std" prefixed declare
> > options and the "std" prefixed cmdline flags are supposed to work on  
> > the
> > same "standard". if so, does this mean to be exclusive (e.g. only have
> > the Pd install path) or inclusive (additionally have
> > /usr/local/lib/pd-externals/ and ~/pd-externals/ (and on debian the
> > additional /usr/lib/pd/extra/)
> >
> > i generally tend towards an inclusive solution, though i'm not 100%  
> > sure
> > whether this is the user expectancy with regards to ~/pd-externals/
> 
> 
> [import foo] will load libraries from all paths, I think.  Or at least  
> it should.  I never understood [declare]'s -stdpath and -stdlib, that  
> lead to me writing [import].

I'm all for [import] and I also think it is much nicer to use, but
unfortunately it is not (yet) part of vanilla. And yes, I think also
[declare] should work in an 'inclusive' manner. 

And no, [declare]'s '-stdlib' and '-stdpath' flags are not that
complicated as you make the appear.

[declare -stdpath extra/foo -stdlib extra/foo] 

The above [declare] covers all possible kinds of libraries:
  * multi-class-one-file libraries (foo.pd_linux or foo/foo.pd_linux)
  * foo/*.pd libraries
  * foo/*.pd_linux libraries
  * a mix of the two above

So I think that [import foo] is equivalent to the [declare] above.

I never understood why [declare]'s '-std*' flags work relative to
<pd-install> and  not <pd-install>/extra. Having to specify 'extra/foo'
each time instead of simply 'foo' doesn't make sense to me. And it's
making us trouble now since not all standard paths end with '/extra'.

Since <pd-install> is not even a standard path (only <pd-install>/extra
is), I think we could leave it in [declare] for backwards compatibility
reasons, but otherwise make it work on _all_ true standard paths. So the
future usage would be:

[declare -stdpath foo -stdlib foo]

and it would load all library variants from any of the standard paths.

As for the concern with the inclusion of ~/pd-externals, I don't see why
people wouldn't it consider it a standard path. But I think the order in
which the various standard paths are checked is important. Isn't it
common practice that a local-first rule is used? Like
in: /usr/local/bin/pd is tried before /usr/bin/pd.

Applied to standard paths this would lead to the following search order:

1) ~/pd-externals
2) /usr/local/lib/puredata/extra
3) /usr/local/lib/pd/extra
4) /usr/lib/puredata/extra
5) /usr/lib/pd/extra

(please add some more, if I forgot some)

I am not sure at this point whether 3) and 4) should be swapped. I put
5) last because it is the most 'general' where all the libraries are
installed that are not flavor specific. If someone has both installed
'pd-zexy' and (not yet existing) 'pdextended-zexy', I think it is sane
to assume that /usr/lib/pdextended/extra should be search first, so that
the patch gets the same 'zexy' as it would get in some other Pd-extended
installation on some other OS.

my 2¢

Roman

P.S.: Sorry, if this isn't pkg-multimedia related anymore. Should we
move that to pd-dev only?