[debhelper-devel] Bug#827132: debhelper: make-4.2 renamed --jobserver-fds to --jobserver-auth causing parallel build issues

[Martin Dorey]

Package: debhelper
Version: 9.20120909
Severity: normal

Dear Maintainer,

The make-4.2 announcement at https://lists.gnu.org/archive/html/info-gnu/2016-05/msg00013.html includes [

* The interface to GNU make's "jobserver" is stable as documented in the
  manual, for tools which may want to access it.

  WARNING: Backward-incompatibility! The internal-only command line option
  --jobserver-fds has been renamed for publishing, to --jobserver-auth.

* The amount of parallelism can be determined by querying MAKEFLAGS, even when
  the job server is enabled (previously MAKEFLAGS would always contain only
  "-j", with no number, when job server was enabled).

].  Although I'm only testing with a Wheezy debhelper, I have looked at https://anonscm.debian.org/git/debhelper/debhelper.git/tree/Debian/Debhelper/Dh_Lib.pm and its call from https://anonscm.debian.org/git/debhelper/debhelper.git/tree/Debian/Debhelper/Buildsystem/makefile.pm and seen that the former mentions jobserver-fds but not jobserver-auth.  When make-4.2 reaches Debian, my experiments suggest that users, if there are any in this situation, intending to run serial make from debuild from parallel make will see the like of:

make[2]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

If I've understood correctly, then that's an effective regression of the fix for https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=532805 (dh_auto_build: prevents make jobs from being run simultaneously).

I suffered a worse problem than a warning or unwanted serialization - my build broke, sometimes.  I doubt anyone else will ever see my very particular situation, but I'll explain in case.  My build system now uses /usr/local/bin/make-4.2.1 -j8 to invoke debuild, which appears to invoke /usr/bin/make, which, on my system, is Wheezy's make-3.81, to execute a proprietary makefile.  make-4.2.1 left -j8 in MAKEFLAGS and, because it used --jobserver-auth rather than --jobserver-fds, debhelper didn't know to remove the parallelism flags.  This exposed a previously latent race condition in the underlying proprietary makefile.  I've fixed that makefile, so now I happily have a working, if accidentally parallel build.

While it looks straightforward to s/jobserver-fds/jobserver-(?:fds|auth)/g, this line will also need attention to support the second make-4.2 bullet I quoted above:

$ENV{MAKEFLAGS} =~ s/(?:^|\s)-j\b//g;

... as it won't match the like of -j8.  Perhaps you'd be content to replace the \b with \d*\b to cope with both new and old versions of make.  Sorry for not supplying a patch, but it'd either be for the Wheezy debhelper or it'd be (probably) untested.  At least I'm realistic enough not to request a fix in Wheezy or Jessie.  I just want to make the diagnosis easier for the next poor sap.

-- System Information:
Debian Release: 7.8
  APT prefers oldstable-updates
  APT policy: (990, 'oldstable-updates'), (990, 'oldstable'), (500, 'oldoldstable'), (500, 'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.2.0-4-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages debhelper depends on:
ii  binutils    2.22-8+deb7u2
ii  dpkg        1.16.16
ii  dpkg-dev    1.16.16
ii  file        5.11-2+deb7u8
ii  html2text   1.3.2a-15
ii  man-db      2.6.2-1
ii  perl        5.14.2-21+deb7u2
ii  po-debconf  1.0.16+nmu2

debhelper recommends no packages.

Versions of packages debhelper suggests:
pn  dh-make  <none>

-- no debconf information