Bug#974139: libpango1.0-dev: PangoFcFont, PangoFcFontMap no longer subclassable

[Marc Lehmann]

On Thu, Nov 12, 2020 at 11:27:13AM +0000, Simon McVittie <smcv at debian.org> wrote:
> So, the core issue there is less that PangoFcFontClass,
> PangoFcFontMapClass are no longer available for subclassing, but more
> that they *were* visible in the past, and therefore code outside Pango
> (like yours) might already be relying on them to continue to have the
> same struct layout that they had in buster. Is that correct?

Yes. And in addition, upstream has changed their members and increased
their sizes, so existing binaries might or might not work, and might or
might not cause out of bounds accesses.

> Are you aware of code in Debian that subclasses them, or is this only a
> concern for non-Debian code making use of the fact that subclassing is/was
> allowed?

I don't know of a registry or anything to keep track of that - certainly
most libpango users don't use this functionality (at leats not in this
way), but it is the only way to get your own renderer as far as I can see.

There are two issues: subclassing embeds (by necessity) the structure inside
the client binary. That is probably quite rare, as most programs just use one
of the existing backends. The size change will affect that.

The other issue is changed structure members. This is probably quite a bit
more common (but again, I have no way to find out), because a lot more code
might want to access the previously public members. This will typically not
(immediately) cause out of bounds accesses, but more likely in crashes, but
every case

I also only looked at two of the many classes in pango, namely
pangofc-font and pangofc-fontmap, but the changes are larger and ongoing,
and a few minor changes already have caused issues with existing programs
such as lilypad and have been undone, and I initially hoped that pango
upstream might undo the other breaking changes, but it seems clear to me
that the changes overall are extensive and they are not willing to undo
more than tiny parts of it (although I got very frustrated because I just
couldn't get a clear answer).

So I don't know what else changed in detail, and even less what the impact
is, and even less than that how to find affected programs.

I also don't know if this ABI breakage is ongoing - in the past, pango
develoeprs have broken the AIB before without changing the soname (I
opened a bug report for this in debian in 2009 and the then debian
maintainer got upstream to partially change it), but the conflicting claim
by pango upstream is that essentially nbody is working on pango, but on
the other hand, there is recent widespread breakage introduced by changign
a lot of the API, and and unknown amount of the ABI.

So while I don't know the extent of actual ABI breakage overall, most of the
changes so far seem to have *prepared* libpango for more extensive ABI
breakage.

As a data point, I created a copy of the class structure with extra
padding at the end to aovid out of bounds issues and recompiled one of my
opengl clients with it. It did seem to run with 1.45, but crashed with
1.47, so my idea with just adding the header files back wouldn't work
anyway, as pango not only changed the struct size, it also changed the
meaning of some struct members in incompatible ways.

So debian would have to bump the soname on, possibly, every release. And/or
would have to assess the amount fo breakage and the danger introduced on
every release.

Mabye this explains some of my frustration and the wording I used -
upstream is aware of this, but clearly doesn't care and the result puts
distributions such as debian in a horrible spot. And again, it all makes
very little sense, because of the claim that pango is essentially in
maintenance-only mode, there are no developers left, so who does all this
breakage and why.

That's of course only my opinion based on two of the pango classes that
directly affect me. Since this will force me to dump pango and implement
my own replacement for pango (probably based on raqm, but it's still a lot
of work), these changes in pango will, medium-term, not affect me at all
anymore.

> It might be possible to revert the changes to the layout of those classes
> to how they were in buster, mark them as private/deprecated/something,
> and if necessary move the new functionality that upstream subsequently
> added into an intermediate subclass that is then used by their in-tree
> subclasses.

Yes, making part of the API private but keeping the ABI would have been
what pango clearly should have done, followed by a 2.0 release which then
removes all the deprecated stuff. Upstream has said they won't do this, of
course.

> The good news is that, because the affected versions treat
> those classes as private and non-subclassable, this approach shouldn't
> break anything that has been compiled against the affected versions and
> expects to see the ABI that is currently in bullseye.

Yes, but I don't see how this amounts to anything less then forking pango.

For example, some of the crashes I observed are due to pango not
maintaining the freetype face pointer members anymore, causing null
pointer accesses. To fix this, you'd essentially have to roll back to an
older version of pango, as trhe changes seem to be extensive.

> This sort of thing seems to be a recurring problem in libraries that have
> two APIs: one facing towards "normal" users of the library (in Pango's
> case this means things like a typical GTK app), and one facing towards
> projects that extend the library itself (like Pango engines and
> renderers), with API/ABI breaks in the typical app-facing API surface
> treated as more important than API/ABI breaks in the extension API.
 
Possibly. From my own (maybe limited) experience, I only ever had this
kind of issue with pango, and I use hundreds of libraries in my many
public and proprietary projects. But maybe I am just lucky because I don't
use so amyn gnoem libraries and possibly this is more of a gnome thing.

My experience is that on incompatible ABI changes, soname gets bumped or
the change is considered a bug.

> I am not a primary maintainer of the pango1.0 package in Debian, but the
> listed maintainers have too many responsibilities and not enough time
> (and so do I, but here I am answering bugmail anyway). They're welcome
> to take over, but will probably say the same things I'm saying.

Well, it totally makes sense to me. I'm quite unhappy to have to implement
my own pango library now. I wouldn't be pleased if it was just API
breakage and would require adjustments (such as switching from freetype
to harfbuzz), but the fact that the problem is two levels higher makes it
so frustrating: not only is not not an API change, they simply retracted
the API completely, disallowing people to use pango at all, and on top
of that, existing binaries now crash and might even have security bugs
that might be able to be triggered by outside (e.g. when my game client
displays chat messages and allocates and changes pangofc-fonts... And I
wouldn't even know where to start anylzing that).

> I am unlikely to be able to keep compatibility with the old ABI in
> a long-term-sustainable way without spending a significant amount of
> time doing careful and patient work to persuade upstream that this is
> both necessary and viable.

I would conjecture that you probably won't even be able to keep
compatibility for short term - the changes in 1.47 seem to be quite a bit
larger than just hiding struct members and enlarging the structs.

I only found out the extent of these changes yesterday.

> We do not have authority over the upstream
> maintainer, who is more overworked than I am,

upstream claims no maintainer exists, actually.

> If I do this, I would ask you not to jump in on upstream issues/MRs
> and insist that it happen sooner:

I will not communicate with upstream anymore, I wasn't lying when I told
tham I have to disengange.

It's clear to me that these changes will stay, and I have to rip out pango
out of all my programs. I have to do that with speed, as existing binaries
will start, and debian is not the only target, so even if debian could
magically fix the ABI, it won't help me.

I think what pango upstream does is completely irresponsible.

> > Bumping the soname is not forking pango, but required by debian policy.
> 
> Bumping the SONAME makes us incompatible with upstream, potentially on a
> long-term basis (see libcurl, libpcre3) and should not be done lightly.

It's not clear to me where this incompatibility would come from - existing
code would compile and run just fine. What wouldn't work would be existing
binaries from other distributions, but if the ABI is incompatible, thats not
actually more incompatible than not bumping the soname, it is safer though.

Granted, I am not a distribution maintainer, and don't envy you, so maybe
I am overlooking something in the big picture. But header files and
library names wouldn't change by a soname bump, i.e. there is typically no
reliance on the soname in program sources.

There are of course exceptions when programs dynamically load libraries,
which happens a lot with, say, libGL, maybe with libcurl, but probably not
with pango.

> on a distro-specific basis, but the results of doing so keep coming back
> to haunt us, and I think the project (and in particular the release team)
> is coming round to a consensus that it's usually a bad idea, and should
> be a last resort at best.

Well, I could have told the release team that it is a bad idea all
along. The question is always what is "badder", bumping the soname or
not. When an upstream project builds shared libraries it is always the
best solution if they bump it on ABI breaking changes. Not doing so
causes issues for every downstream, including but not limited to all OS
distributions.

> Pragmatically, I suspect a SONAME transition might also cause us more
> crashes in the short term than the affected ABIs do: while the transition
> is incomplete, apps will end up loading both the "old" and "new" Pango
> into the same address space, which seems likely to be bad news.

Right, but this is the case even when upstream bumps the soname as
normal part of their work, so debian surely must be able to deal with
this. Existing binaries will continue to work as opposed to having unknown
bugs.

Anyway, the TL;DR is (yeah, I put it at the end :) that the changes seem
to be extensive, and not limited to a single pango release. Just restoring
the members/size doesn't seem to be as feasible as I originally thought,
as there have been *actual* semantic changes behind the scenes. And I only
looked at two of the classes.

PS: Thanks for caring - I am relying on the overhelmingly good work of
Debian maintainers ever since I switched to debian some 20 years ago.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\