[Pkg-javascript-devel] Javascript team policy and rejection of node-three binary package [and 1 more messages]

[Simon McVittie]

On Fri, 09 Mar 2018 at 17:50:30 +0530, Pirate Praveen wrote:
> How about the change given below? (is the intent clear at least even if
> disagreement on content remains)
> 
> 5. should add 'Provides: node-foo' in debian/control and install
> package.json in /usr/lib/nodejs/foo, if the script is usable also for
> Nodejs.

I think saying "script" is perhaps unhelpful here, because outside
Javascript, that usually refers to something executable with #! at the
beginning.

It might be clearer to think about this in terms of libraries and
executables: some Javascript packages are libraries (reusable code
intended to be depended on by something else), some Javascript packages
are executables (an executable script in /usr/bin or similar, starting
with #!/usr/bin/nodejs or similar), and some have both.

As far as I know, Javascript has two interesting issues from a packaging
point of view, which make it require some extra thought compared with
Perl or Python:

* Some Javascript modules are very small, resulting in lots of small
  packages

* There are multiple Javascript runtimes/interpreters (nodejs, gjs,
  seed, web browsers) and each Javascript library is usable by one or
  more of those interpreters, but not necessarily by all of them

The second issue applies equally to Python: some Python libraries
only work in CPython 2 (python-foo), some only work in CPython 3
(python3-foo), and some also work in PyPy (pypy-foo). For Python, we've
chosen to distribute libraries for those three runtimes as separate
binary packages, because that doesn't result in an overwhelmingly large
number of binary packages.

For Javascript, because some libraries (particularly in the nodejs
ecosystem) are very small and numerous, following the same convention as
Python would result in a very large number of binary packages, and the ftp
team are uncomfortable about accepting those; so we might want to have
a different policy, where (whenever possible) a single binary package
libjs-foo provides the 'foo' library for all the runtimes it supports.

> Exceptions (cases where a provides may be insufficient):
> 1. should generate binary package 'foo' ('node-foo' in case of a name
> conflict) and declare dependency on nodejs, if script includes a command
> line tool

Perhaps a better way to think about this would be something like:

"""
If a Javascript source package includes reusable library code, then that
part of the package should be packaged as a library:

* the binary package that enables you to require('foo') in nodejs should
  either have Provides: node-foo or be named node-foo
* if the library is designed for use by nodejs only, it should be in a
  package named node-foo
* if the library is usable by multiple Javascript interpreters, it should
  be in a package named libjs-foo that Provides: node-foo
"""

(In case you haven't noticed, I'm a big fan of "names are APIs and
APIs are names": I think the systematic policies that we have for some
languages, where libfoo2 always means you can run binaries with
DT_NEEDED: libfoo.so.2, libfoo-bar-perl always means you can "use
Foo::Bar" in Perl, and python3-foo always means you can "import foo"
in Python 3 are a very good idea.)

And for executables, perhaps something like this:

"""
If a Javascript package includes an executable or executables that are
useful in their own right, then they should be packaged in a binary package
that represents those executables, as opposed to the library:

* must depend on the necessary interpreter, usually nodejs
* must depend on any Javascript libraries that those executables need
* should not be named node-* without a suffix like -bin or -tools (?)
"""

I'm not sure how this would be phrased in a policy, but I think the
goal should perhaps be: if the user wants to run the uglifyjs(1)
command-line utility, they should be able to install a package that
represents the uglifyjs command-line tool, without needing to know
or care whether uglifyjs is itself written in Javascript, or whether
it's been reimplemented in Python or C or Perl. The goal of the user
of uglifyjs(1) is to minify/compress some Javascript, and the fact that
they do that by running a tool that is itself written in Javascript is
just an implementation detail.

This gets annoying if the name of the executable would be too generic
(namespace pollution) without a node- prefix, like node-static (I think
"static" would be a bad name for an executable), but perhaps that could be
packaged as node-static-tools or node-static-bin or something?

> 2. should generate binary package 'node-foo' in addition to 'libjs-foo',
> if the script requires a newer version of nodejs than available in
> testing. This is to facilitate proper testing migration (provides will
> not be sufficient to block migration to testing before required nodejs
> also migrates).

Breaks might be a better mechanism for this, but I'm not sure about the
corner cases. I think Breaks would prevent migration as long as there is
some other package that depends on both nodejs and the library (because
migrating the library would reduce installability), but it would not
prevent migration if there is no such package.

I think I should also point out here that these libraries shouldn't be
in unstable until the corresponding nodejs is ready or almost ready
to migrate: in general, packages in unstable should be "almost ready"
for migration to testing, and they should certainly be *installable*.
If you're uploading libraries or tools that require a version of nodejs
that is not yet available in unstable, or that require a version of
nodejs that is in unstable but is not going to migrate for a while (for
instance if it has RC bugs that are not going to be easy to solve),
then they should probably be going to experimental instead.

If packages that require nodejs 6 don't hit unstable until nodejs 6
is (about to be) in testing, then the corner cases where Breaks would
be a problem are perhaps not something that will occur in practice?

Ian Jackson wrote:
> > Also your comments about namespace make me wonder: do many of these
> > node scripts have poorly chosen command names ?  Can I get a list of
> > them easily somehow ?
> 
> apt-file find /usr/bin |grep ^node-
> 
> will give most of it, but some packages like mocha only provides
> node-mocha so the search should include source package names too.

I think things like mocha are OK: that's a "brand name" that isn't just
a description of what it does or a commonly used jargon term.

If the command-line tool provided by src:node-static was just called
static, I'd be uncomfortable with that. (But it's called node-static,
which seems fine.)

> > I think "handlebars" and "uglify" are OK for this, but I don't know
> > what the whole ecosystem is like.

Those seem fine to me too.

> > In practice, as an easier guideline, maybe it would be better to say
> > that the command and package should *usually* be renamed, unless the
> > script is high propfile and has a good name which is unlikely to
> > conflict.
> 
> I don't agree. I don't think the benefit outweigh the cost. Changing
> node to nodejs costed us so much effort in patching at so many places.
> And as soon as we could use node again, we switched back.

I think "should usually be renamed" might be going too far, for the
reasons Pirate mentioned; but it's certainly something the maintainer
(and the sponsor if any, and the ftp team reviewer) should at least be
thinking about, to the same extent as for any other executable in the
flat global namespace of $PATH.

If the executable is a non-obvious "brand name" like mocha or coveralls
or gyp or uglifyjs, then that seems reasonable to ship as-is without
renaming.

If the executable is something really generic like po2json or static
(I'm looking at the output of `apt-file search /usr/bin/node` for
inspiration here) then I think it would be necessary to rename for
disambiguation. (Note that the real executables in Debian here are
node-po2json and node-static, so they're fine; I don't know whether
that's because they're OK upstream or whether they've been renamed in
Debian.)

I don't think there's any substitute for the maintainer and reviewers
putting case-by-case thought into this, tbh.

    smcv