[Debian-med-packaging] Bug#822701: Bug#822701: Bug#822701: samtools: FTBFS: UNEXPECTED PASS: Task worked when we expected failure;

[John Marshall]

On 1 Jun 2016, at 05:25, Afif Elghraoui <afif at debian.org> wrote:
> I sincerely apologize that this comes across as offensive-- it truly
> wasn't intended that way. I was (and still am) somewhat confused about
> this subject, thoug. My confusion is mostly about the tightness of the
> htslib/samtools/bcftools interdependence and their coordinated releases.
> I have never been worried about your breaking externally developed
> programs.

Thank you; I am glad to see the specific contexts leading to your concerns and sorry to have interpreted the previous comment as FUD.

> bcftools 1.3 build failure after introduction of htslib 1.3.1:
> https://ci.debian.net/data/packages/unstable/amd64/b/bcftools/20160425_215548.autopkgtest.log.gz

This is a bcftools test suite failure rather than an actual build failure.  A bug was fixed that required fixes in both htslib (cad00ea) and bcftools (5d25295).  bcftools-1.3+htslib-1.3.1 has half a fix so is unsurprisingly detectably broken.

This very Debian bug (#822701) is another case of an old release's test suite failing due to a bug fix in a newer htslib.  See https://github.com/samtools/htslib/issues/374 for the explanation.

I am sure you agree that fixing bugs is beneficial.  Unfortunately it is always the case that client software's test suites may be broken by bug fixes in a library used (if only because expected output differs), so it is possible that you may have to tolerate mere test failures when building old-{sam,bcf}tools-with-newer-htslib (or patch the old release's test suite until it succeeds).

But presumably Debian has some policy for this scenario, which is surely not uncommon?

> bcftools 1.2 build failure after introduction of htslib 1.3:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=813465

This one was an actual build failure.

In fact samtools and bcftools are in a *worse* position than other htslib-using software due to being maintained primarily by the same group: from time to time, functionality moves from {sam,bcf}tools to htslib as it is generalised and made available publicly.  In this case, limitations were found in htslib's bcf_hdr_combine() interface and a new htslib function was needed in this area.  A natural name for it was bcf_hdr_merge() which conflicted with a local function in bcftools.  We weighed the temporary bcftools-1.2+htslib-1.3 build failure (which affects only bcftools) against the ongoing desirable API function (which affects everyone), and considered that the build failure of an artificial combination (would you really want such a combination in production?) was an acceptable infelicity.  It's suboptimal and YMMV, but I think it's a defensible choice.

It does not affect binary compatibility: bcftools-1.2 runs fine against htslib.so.1.3 nonetheless.

> My mention of internal interfaces was based on my impression of what's
> described in this thread:
> <https://lists.alioth.debian.org/pipermail/debian-med-packaging/2016-January/038827.html>

What's described in that thread is undocumented internal private htslib functions that should always have been static getting made static, and Debian notices only because of their symbols file not because any actual code actually breaks.  Just move on!  (When your symbols script comes up with MISSING it would be helpful if Debian would start by running "git log -S symbolname" which will usually provide an explanation, rather than assuming that something terrible has happened.)

    John

-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.