Select provider of libav* libraries

[Reinhard Tartler]

On Sun, May 31, 2015 at 7:21 AM, Andreas Cadhalpun
<andreas.cadhalpun at googlemail.com> wrote:

>> and would rather propose to just rename the current "libav" source
>> package to "ffmpeg" and package the latest ffmpeg source tarball.
>
> Then that ffmpeg source package would take over the binary package
> names as well. If you think it's not necessary to preserve the
> development packages from src:libav, my proposal becomes even simpler:
> Just rename lib*-ffmeg-dev to lib*-dev in the current ffmpeg source
> package. That'd work, because src:ffmpeg has a higher epoch.

That's not what I'm proposing.

>
>> I'd rather keep the packaging of the package that is currently called
>> "libav", because on many architectures, it compiles multiple "flavors"
>> with hardware optimized flavors (on i386 for instance, there is a
>> flavor without latest SEE, etc).
>
> I had considered producing optimized flavors, but I came to the conclusion
> that it would complicate debian/rules without much benefit, because
> the flavors are mostly unnecessary:

That's not only because of optimization, but also for licensing
issues: The lib*-extra-* packages are licensed with GPLv3+, which is
not acceptable for the majority of applications in debian.

Note that it does not matter whether or not there are GPLv3+
applications in the archive. These -extra- package do provide
additional functionality that users do appreciate. However, we cannot
provide that in the "standard" libav* packages because we do have
GPLv2-only applications that we must not link against a GPLv3
libavcodec.

>  * On i386 all the SSE etc. optimizations are guarded by runtime
>    CPU detection, so that even when compiled with SSE the libraries work
>    fine on CPUs without SSE.
>    The only optimization were this doesn't apply are the i686 optimizations,
>    which currently amount to a dozen assembler lines. So I don't think
>    this justifies shipping two flavors.

Have a look at #688384 - the conclusion was to compile for i586, which
seems absurd to me for most users, even if most optimizations are
indeed the runtime selected hand-written assembler parts. The libav
packaging solves this with multiple compilation passes.

>  * The situation for arm*, where the neon/vfp optimizations are runtime
>    guarded, is similar.

I don't think so, have a look at #657885, which shows a SIGILL in
cmdutils.c. This can hardly be caught at runtime.

>  * Sparc is on it's way out, possibly going to be replaced by sparc64.
>    So building a sparc64 flavor on sparc isn't really useful.
>  * This leaves powerpc, where configure would enable '-maltivec'
>    together with the hand-written altivec optimizations, which are guarded
>    by runtime CPU detection. This causes gcc to add some altivec
>    instructions, which are not guarded and thus cause SIGILL.
>    Looking into fixing this in the configure script is on my TODO list.

It is impossible to fix (or rather not worth the effort to fix
properly). Either you disable altivec completely, or you need need to
pass -maltivec which allows you to use the altivec optimizations and
break on non-altivec machines. So you either make all multimedia
package unusably slow for altivec machines because altivec is not
enabled, or you exclude all uses of non-altivec machines.

> Are there good reasons to produce these optimized flavors?

Yes, there are, at least on i386, arm and powerpc. I would consider
not having them a serious regression.

>> Andreas, would that approach be acceptable to you?
>
> I'd rather continue with the packaging of the current ffmpeg source
> package, because the one from src:libav uses a pre-dh7 debian/rules,
> and a quite complicated one at that.

I'm happy to simplify or refactor the packaging, but I fear I have to
insist to continue using the current src:libav packaging.

It also seems easier to conduct the resulting transition if the
package names of what application packages depend on don't change.
Please let's not overcomplicate these things.

>
>> If so, then I'd propose to get a test package ready in Git, and I'd
>> then do a mass-rebuild on my amd64 box to see how many packages fail
>> to build.
>
> I already did a mass-rebuild for my proposal and the results can be
> found in my mail [1] mentioned above.
> The blender and paraview FTBFS bugs have been resolved and the new mpd
> version is now in sid, so unless new, unrelated FTBFS bugs were
> introduced, only 7 packages would fail to build.
> Three of those FTBFS already (dvswitch has a patch in #747868,
> gstreamer0.10-ffmpeg should be removed and jitsi is fixed in NEW
> since 8 months).
> Two (gpac and xbmc) would require changes for the next Libav version
> as well, because they use private API, which was changed.
> The hardcoded sonames in taoframework have to be updated, like in every
> Libav transition. (Maybe someone can fix taoframework to use the sonames
> present in the build environment?)
> So only gst-libav1.0 needs a FFmpeg specific change, which is an additional
> build-dependency on libavresample-dev, because it currently relies on
> libavcodec-dev to bring this indirectly, but FFmpeg's libavcodec uses
> libswresample instead of libavresample.

Good. Did you check whether current FFmpeg is ABI compatible to the
current libav packages, or do we need to recompile all packages in any
case?

>
>> If that number seems acceptable, then first upload it to
>> experimental, and then coordinate with the release team to get a
>> transition slot.
>
> That's a sensible plan, similar to what I had in mind.

I'm happy to hear that.

>> Also, I would feel much more comfortable if someone could convince me
>> that FFmpeg is really not a one man show, and is taking maintenance of
>> both internal and external APIs seriously.
>
> Your allegation that FFmpeg was a "one man show" is highly disrespectful
> to all other FFmpeg contributors, including the Libav developers, whose
> improvements are merged by Michael.
>
> Anyway, let me try to convince you again:
> FFmpeg 2.4 was released around the same time as Libav 11 in September 2014.
> Let's look at the number of commits since then:
> Libav:
> $ git shortlog -ns --no-merges v11..HEAD | head -n15
>    294  Vittorio Giovara
>    253  Martin Storsjö
>    206  Anton Khirnov
>    131  Luca Barbato
>     72  Diego Biurrun
>     46  Michael Niedermayer
>     32  Rémi Denis-Courmont
>     21  Andreas Cadhalpun
>     17  Hendrik Leppkes
>     16  Gabriel Dume
>     16  Himangi Saraogi
>     16  wm4
>     14  Federico Tomassetti
>     12  Peter Meerwald
>     11  Janne Grunau
>
> FFmpeg:
> $ git shortlog -ns --no-merges n2.4..HEAD | head -n15
>   1831  Michael Niedermayer
>    294  Vittorio Giovara
>    252  Martin Storsjö
>    197  Anton Khirnov
>    179  Clément Bœsch
>    155  James Almer
>    150  Carl Eugen Hoyos
>    114  Andreas Cadhalpun
>    113  Luca Barbato
>     98  Lukasz Marek
>     93  Paul B Mahol
>     85  Ronald S. Bultje
>     83  wm4
>     66  Christophe Gisquet
>     48  Benoit Fouet
>
> So while Michael is clearly the most active FFmpeg developer, it is not
> at all a "one man show".
> Claiming that is a bit like claiming that Linux development was a
> "one man show", because Linus is the most important developer.

I think the statistics indicate that the situation is much worse than
I expected. Please keep in mind that because of the daily merges, all
commits that are made in Libav also appear in FFmpeg, but not the
other way round. With a finger-in-the-wind estimate of subtracting the
numbers in the Libav table from the FFmpeg table, the distance between
Michael and the other FFmpeg developers because even more extreme.

I'm sorry if you felt that my characterization was disrespectful - it
was certainly not my intention to undermine anyone's contribution. I
was rather trying to make a point with risk assessment of two
competing products: the FFmpeg codebase and the Libav codebase. As
outline above, the product itself is not the only consideration in
this debate - the health of the upstream project is also very
important to consider.

>> Thank you all for bearing with my rather slow responses. I've been
>> traveling for the last four weeks and am finally back home now.
>
> I hope you enjoyed your journey.

I did, thank you very much!

-- 
regards,
    Reinhard