Select provider of libav* libraries

[Reinhard Tartler]

On May 15, 2015 4:56 PM, "Andreas Cadhalpun" <
andreas.cadhalpun at googlemail.com> wrote:
>
> Hi Reinhard,
>
> thanks for explaining your point of view here.
>
> On 15.05.2015 09:23, Reinhard Tartler wrote:
> > Thanks for this insightful post, Dmitry,
> >
> > On Mon, May 11, 2015 at 5:44 AM, Dmitry Smirnov <onlyjob at debian.org
<mailto:onlyjob at debian.org>> wrote:
> >> On Sat, 9 May 2015 16:00:49 Andreas Cadhalpun wrote:
> >>> What would you count as very compelling reasons if more features,
less bugs
     > >>> and better security support are not sufficient?
> >>
> >> More features is not necessary means less maintenance burden;
> >> Less bugs is not always means better software (it is a matter of how
upstream
> >> manages bugs);
> >> Quality of security support is something that remains to be seen.
> >
> > I think security is not a decisive topic where either project cannot
> > clearly claim of having a clearly better handle.
>
> I disagree: FFmpeg is clearly better at fixing security issues.
> To take a random example, an out of bounds read in the bink decoder was
fixed
> in FFmpeg three years ago [1], while Libav git master is still vulnerable
> today.
>
> One can find more such examples, but I have better things to do with my
time.

I think this sea0¨˜j67
>
> > These days, FFmpeg for
> > sure asks for most (if not all) CVE numbers recently assigned, and
claims
> > to provide patches for them.
>
> FFmpeg not only claims to provide patches, but actually does provide them:
> most CVEs link to the corresponding patch.
>
> > What is less visible is what happens behind the scenes:
> >
> > Most of these issues originate from Google Guys that work on security
and
> > conduct fuzzing tests on libavcodec/libavformat. Their main target is of
> > course their own libavcodec/libavformat fork that they ship in Chrome,
but
>
> As far as I'm aware they use a git snapshot of FFmpeg that they update
> from time to time. They only change the build process to produce one
single
> libffmpegsumo library instead of libavcodec/libavformat/libavutil.
>
> > they (of course) also share their samples with both Michael (FFmpeg) and
> > Luca/Anton (Libav). Michael seems to have much more capacity and time,
and
> > thus is usually faster with pushing patches for such crashers.
>
> Yes, Michael does an awesome job at fixing those [2].
>
> > Libav takes
> > the time to investigate, reproduce and understand those patches.
>
> That's a commendable idea, but if the result is that these patches
> don't get applied in a reasonable time frame, it's rather bad.
>
> > Unfortunately, in the majority of cases, this is not trivial at all,
often
> > because of terse (or even wrong) commit messages, or the fact that there
> > are better places to fix a particular issue in the code.
>
> In that case it would probably help to ask the author of the patch.
> Did the Libav developers do that?
>
> > "Better" usually
> > means that more than a single instance of the issue is fixed.
>
> Can you give examples for this?
>
> > Also, given that Libav supports significantly less codecs and formats
(and
> > in some cases specific variants or features of codecs), many security
> > issues simply don't apply.
>
> While it's true that some security issues only affect code not present in
> Libav, the majority of issues affect both projects (until they are fixed
in
> FFmpeg).
> Much of the additional code in FFmpeg poses nearly no security problem.
> For example, even if there was a potentially security relevant bug in one
> of the more than 100 additional filters in FFmpeg, it would be very hard
to
> exploit, since the filters generally have to be invoked manually.
> Only decoders/demuxers can be easily triggered automatically,
> e.g. by visiting a web site or opening a folder with images, for which
> thumbnails are created.
> Additionally it would be much easier to disable some rarely used features
> in Debian's FFmpeg than to enable desired features in Debian's Libav,
> when they are not present upstream.
>
> > Libav could for sure do a much better job here by releasing faster. In
the
> > past, I looked at doing new point releases about every 2-3 months, but
for
> > family reasons, I wasn't able to keep that pace. Release frequency is
> > clearly something that needs improvement. As far as for the release
> > content, I am not aware of critical fixes being missed, so quality wise
I
> > think we are good.
>
> You are wrong about the quality. Have a look at the bink example I gave
above.
>
> >> I have a feeling that Debian already became life support for libav.
> >> Ever since Debian chosen libav, ffmpeg remained alive and apparently
doing
> >> well without our help. I'm not too sure if libav would be able to stay
alive
> >> without Debian.
> >
> > That's an interesting take on the matter. I don't think it is accurate;
for
> > instance OpenEmbedded or Gentoo also ship Libav (in favor of FFmpeg), so
> > Debian&Ubuntu are clearly not alone.
>
> Interestingly Gentoo recently switched to FFmpeg by default [3] after
conducting
> a survey [4]. About 300 people participated in that survey and the outcome
> was rather clear:
> 62%    [ 189 ]    "I prefer ffmpeg, and it should be the default."
>  4%     [ 15 ]    "I prefer libav, and it should be the default."
>
> > It is true that Libav owes the largest
> > portion of its userbase to Debian&Ubuntu, however even if Debian would
stop
> > shipping Libav, then I doubt that this would threat Libav development in
> > any way.
>
> I'm not going to speculate about this.
>
> >> From maintenance prospective libav seems to be a liability. We have to
carry
> >> patches for packages where upstreams are not too concerned about
supporting
> >> libav.
> >
> > This statement is a bit surprising to me.
>
> Have you looked at the xbmc-libav-hacks [5]?
> I can understand that they dropped them when releasing Kodi.
>
> > At the time Debian started to
> > ship Libav, there was absolutely *no* difference to FFmpeg. Since that,
> > many things have changed, and the two code-bases have diverged. While
> > FFmpeg started to merge as much functionality as possible, Libav
focused on
> > maintainability, cleaned and straightened-up APIs and removed fringe
codecs
> > and formats.
>
> You forgot to mention that FFmpeg also fixed many bugs in the existing
code
> base and most of those fixes didn't make it to Libav.
>
> >> Also in the light of past libav transitions and deprecations that
required
> >> multiple changes in Debian and upstream I know no upstream who is
happy to
> >> support libav.
> >
> > In practice, maintainability means something different for an upstream
> > project and a distribution package. While upstream developers extend,
fix
> > and refactor the code-base,as package maintainers we make sure that
> > the libavcodec package works fine with all applications that make use
of it.
> >
> > To be totally blunt: The libavcodec/libavformat API is horrible. This is
> > partly caused because of the aim to have a single library that covers
every
> > thinkable container and codec flavor using a common API. Libav has
> > identified the development process as the culprit, where developers
> > submit unfinished and unreviewed work to the main codebase, which is
then
> > used by applications and can basically never be fixed.
> >
> > In an attempt to provide applications a better interface to
> > libavcodec/libavformat, Libav has in the past aggressively devised new
> > and deprecated old APIs for maintenance reasons, i.e., helping the
> > maintenance of the libav codebase.
> > For Debian with a large two digit number of reverse dependencies, this
> > caused a lot of fall-out that needed to be fixed. Libav developers were
> > instrumental with providing patches to make them build again. FFmpeg
tries
> > hard to keep old APIs, which for sure lowers the effort for Linux
> > distributions, but is it really better to keep applications that use
> > obsolete and broken APIs? I'm not sure. In the long run, having
> > applications use less horrible APIs is for sure a good thing.
>
> FFmpeg merged those new APIs, while sometimes retaining the old APIs
> longer to give applications more time to adapt.
> When not many applications using the old APIs remain, they are dropped.
>
> > I think the debate on the development methodology is the biggest
> > disagreement between the two projects: FFmpeg insists on keeping its
> > development velocity and allowing developers to "get work done",
> > compromising on maintainability and common understanding of the code
base
> > in favor of more features.
>
> I don't think getting work done is bad, because if work (like bug fixes)
> doesn't get done, it is worse for maintainability.
>
> > Libav on the other hand rather focuses on clean
> > implementation and let's say better designed APIs.
>
> Let's say Libav focuses more on cosmetics like consistent indentation,
> use of spaces around brackets, a linear commit history and such.
>
> > This requires more work,
> > which in effect significantly lowers the development velocity. For a
system
> > integration job on the scale of Debian, less velocity seems more
appealing
> > to me.
>
> Less velocity in fixing (security) issues seems everything but appealing.
>
> > I'm having a very hard time to find an answer to the question which
project
> > is better for Debian. One way of looking at it is which causes less
effort
> > for the package maintainers. Andreas' and Dmity's mails make it sound
that
> > Debian could save a lot of effort by switching to FFmpeg. I'm honestly
not
> > sure if that is true. For instance, chromium upstream does not support
> > linking against a system libavcodec library. Andreas had to provide a
> > pretty invasive patch in #763632 for this, and I doubt that chromium
> > upstream is interested in merging it.
>
> The patch [6] is not invasive. The largest part is splitting a list of
FFmpeg
> functions used by chromium in three lists for functions from libavcodec,
> libavformat and libavutil. (Chromium builds those into only one library,
> libffmpegsumo.) Other than that it's about two dozen changed lines in
> chromium upstream code to adapt to using three instead of one library
> and to remove configuration #defines incompatible with the FFmpeg
libraries
> in Debian.
> The reason for this is that chromium upstream basically contains support
> for building against system FFmpeg libraries, it seems just rather
unmaintained
> and thus quite broken.
>
> > My biggest concern with the list that Andreas provided in
> >
http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/043980.html
> > is that the different upstream require specific upstream versions of
> > FFmpeg, and will break with other versions, which was pretty much the
state
> > 2009-2010 when I more or less took over of the package: Many upstream
just
> > bundled some internal copy of libavcodec that they have developed
against
> > and were updated with a very inconsistent pace. I'm happy to see that
> > FFmpeg finally no longer discourages the use of release tarballs in
favor
> > for "svn trunk" or nowadays "git master" on the download page.
>
> I'm only aware of two projects that use sufficiently outdated FFmpeg
versions
> that they probably won't work with current FFmpeg: Avidemux and MythTV
[7].
> Neither of those are in Debian, precisely due to this fact.
>
> > Nevertheless, the release frequency is just insanely fast,
>
> It is about four major releases per year. There's nothing insane about
that.
> Libav now tries to make two major releases per year, because users
requested
> more frequent releases [8].
>
> > and long-term
> > support is practically non-existent: the oldest release branch was cut
in
> > 2014 -- for Debian/Ubuntu, we require significantly longer cycles.
>
> This is plain wrong. FFmpeg generally supports release branches that are
> still widely in use. If they aren't there isn't much point in supporting
them.
> For example the latest release of the FFmpeg 0.5 series was 0.5.15 in
November
> 2014, while the latest release of the Libav 0.5 series was 0.5.10 in
February
> 2013, despite being used in Debian squeeze LTS.
>
> > Long-term support is not the only problem - I remain unconvinced that
> > switching from Libav to FFmpeg will solve the fragmentation problem. I
fear
> > that it would be replaced by fragmentation across different FFmpeg
upstream
> > versions. This may or may not be true in practice, but this is
> > what both my experience and intuition on this topic tells me.
>
> I don't believe there would be more fragmentation than there is currently.
> There could even be the contrary effect: If FFmpeg libraries are available
> in Debian, there is less reason to embed a copy of them.
>
> > I'm sure that this issue could be addressed in FFmpeg, I'm just not sure
> > how insistent (and helpful) FFmpeg is here, because this for sure will
> > compromise development velocity, which as written above, is the main
> > disagreement between FFmpeg and Libav in my eyes. Maybe Andreas could
> > comment on this point?
>
> How do you think FFmpeg could help in this regard?
> After all, FFmpeg developers can't decide which version Avidemux/MythTV
> uses.
>
> >> I am not qualified for technical comparison between ffmpeg and libav.
> >
> > I think nobody really is, because there is no technical metric or
similar
> > criteria that could help us here.
>
> I disagree. For example, one metric would be to look at fixed issues in
> FFmpeg's bug tracker (e.g. [9]) and open issues in Libav's bug tracker
> (e.g. [10]) and compare.
>
> > What are the right questions for assessing FFmpeg vs Libav? Let me try
to
> > formulate questions that focus on maintainability:
> >
> > What project is less effort to package?
> > - I'd think Libav, because of the less frequent release cycle. Both
> > projects have rather accessible upstreams.
>
> If release frequency really would be a problem, one could simply skip
> some releases.
>
> > What project is doing a better job with handling defect reports?
> > - I'd think FFmpeg, because the Libav bug tracking system is currently
> > dysfunctional. AFAIUI this is being worked on. I've worked around this
for
> > quite some time by talking to individuals on IRC directly, but this
clearly
> > doesn't scale.
>
> On the other hand bugs not fixed in Libav, but in FFmpeg, can increase the
> maintenance effort due to more bug reports in the Debian bug tracker.
>
> > What project causes less effort for applications?
> > - A significant number of upstream prefer FFmpeg. OTOH, FFmpeg makes
great
> > effort to provide source-code compatibility, but obviously a number of
> > projects only develop and test against FFmpeg, so this doesn't help.
>
> I'm sure the situation would be far worse if FFmpeg wouldn't make such an
> effort to be compatible.
>
> > What project is less effort for the security team?
> > - I'd say Libav, the security patches have significantly better commit
> > messages and descriptions, and generally make much more sense at least
to
> > me. Maybe Moritz can elaborate on this.
>
> It seems he already did [11]:
> "I think ffmpeg is doing better in terms of handling security issues; when
> I contacted Michael Niedermeyer in private we has always quick to reply,
> while libav-security@ seems understaffed: Several queries in the past
needed
> additional poking, some were left unaddressed until today. Also, the
Google
> fuzzer guys stated that more samples are unfixed in libav compared to
ffmpeg."
>
> > My personal impression is that In FFmpeg, security support is handled
by a
> > single person whose work is not exactly easy to follow.
>
> Michael is not the only one who cares about FFmpeg's security. For
example,
> I do as well and I recently fixed a number of issues that I found by
fuzzing
> myself. Not all of them are fixed in Libav yet, even though I sent my
patches
> also to libav-devel at libav.org.
>
> > For me personally, this looks like a tie.
>
> For me it looks like a clear choice.
>
> > I would like to think that a
> > high-profile, high-performance and universal multimedia codec and format
> > library deserves a clean code-base, clean documentation, excellent test
> > coverage and a steady and thorough release process. Neither Libav nor
> > FFmpeg achieve that fully. Libav had so much potential, but struggles
with
> > achieving it for IMO mostly manpower reasons - working in or with FFmpeg
> > for some reason seems to be more attractive.
>
> My main reason for preferring to work on FFmpeg is that it already works
better.
>
> > Clearly, Libav has lost a lot of its drive when it took off. This drive
has
> > significantly diminished - significant contributors from its inception
are
> > no longer active. I remain convinced that for the Jessie release, Libav
was
> > the better choice.
>
> Unfortunately FFmpeg was stuck in the NEW queue until after the
transition freeze,
> so for Jessie the choice was between Libav and Libav+FFmpeg, which the
security
> team didn't want.
>
> > For Stretch, it is less clear. Libav seems to me like
> > the prudent choice, FFmpeg like the more aggressive one.
>
> For stretch I think FFmpeg would be the sensible choice, while staying
with
> Libav would just be the 'status quo' choice.
>
> > I'd rather
> > encourage people to help with porting missing features and bugs to
Libav,
> > because it is the cleaner and more maintainable codebase, but given
that we
> > are all volunteers, this may or may not happen.
>
> I don't think Libav has a much cleaner or more maintainable codebase, it's
> just smaller. Fixing issues in Libav that are already fixed in FFmpeg is
> just duplicated work, which is not very rewarding to do.
>
> > Wow, that was a much longer email than I hoped. And given that I'm still
> > traveling and doing many family visits, it also took me much longer to
> > write this than I wished. If we really had to vote right now which way
to
> > go, I'd probably abstain, because I also have to face the fact that I no
> > longer have the capacity to support a high-profile package like
libavcodec
> > with the same time and devotion as I did for the last 5 years. How ever
we
> > as a team decide to move forward, I think libavcodec (and related
> > libraries) are better team maintained (pkg-multimedia is just fine for
> > this), and I'd be happy to help out with the packaging either way.
>
> I'm not opposed to pkg-multimedia team maintenance for FFmpeg.
> If anyone wants to help maintaining it, feel free to add yourself as
uploader.
>
> Best regards,
> Andreas
>
>
> 1:
https://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=b3675f890abee0bc446495711223a5c790234672
> 2: http://j00ru.vexillium.org/?p=2211
> 3: http://thread.gmane.org/gmane.linux.gentoo.devel/95339/focus=95585
> 4: https://forums.gentoo.org/viewtopic-t-1010096.html
> 5:
https://sources.debian.net/src/xbmc/2:13.2%2Bdfsg1-4/lib/xbmc-libav-hacks
> 6:
https://bugs.debian.org/cgi-bin/bugreport.cgi?msg=5;filename=chromium-browser_system-ffmpeg.patch;att=1;bug=763632
> 7: https://trac.ffmpeg.org/wiki/Downstreams
> 8:
https://sources.debian.net/src/libav/6:11.3-3/doc/RELEASE_NOTES/?hl=9#L9
> 9: https://trac.ffmpeg.org/ticket/4431
> 10: https://bugzilla.libav.org/show_bug.cgi?id=840
> 11: https://lists.debian.org/debian-devel/2014/08/msg00060.html
>
> _______________________________________________
> pkg-multimedia-maintainers mailing list
> pkg-multimedia-maintainers at lists.alioth.debian.org
>
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/attachments/20150518/46aac899/attachment-0001.html>