Bug#953105: gtk-update-icon-cache does not produce reproducible results on 32-bit architectures

[Simon McVittie]

On Wed, 04 Mar 2020 at 10:51:59 -0500, Daniel Kahn Gillmor wrote:
> It seems possible that we could fix this problem in the balsa package by
> not shipping an icon cache at all, of course (and maybe running
> update-icon-caches in the postinst?  though i'd like to avoid
> introducing a postinst script if we don't need to).

It's certainly unusual to want to have your own icon cache. Most packages
that have icons install them into the system-wide hicolor icon theme
directory (for historical reasons "hicolor" is hard-coded to be the
default/fallback icon theme - it was the KDE 3 default). This causes a
dpkg trigger shipped by hicolor-icon-theme to update the icon cache.

According to `apt-file search`, balsa is the only package that
looks like it might possibly be correct to ship this file - but if every
other application manages to get by without having its own private icon
cache, I don't know what's so special about balsa?

medit ships /usr/share/icons/hicolor/icon-theme.cache, which is definitely
a bug: hicolor-icon-theme's trigger will make gtk-update-icon-cache
overwrite it.

numix-icon-theme-circle ships
/usr/share/icons/Numix-Circle/icon-theme.cache, which is definitely a bug:
its own postinst(!) asks update-icon-caches to overwrite that file.

> But it seems odd that the 32-bit architectures would produce
> unreproducible caches when the 64-bit version is reproducible.

I have no idea why this is the case. Having looked at the code that
generates them, I would expect these cache files to be unreproducible
on all architectures. If they're reproducible on 64-bit it's by
luck and coincidence.

I don't know how much sense they make to ship in packages (as I said,
of three packages that do, balsa is possibly wrong and the others are
definitely wrong), but if the cache was reproducible it would reduce
churn when people distribute complete filesystem trees in Flatpak,
ostree, casync, Docker, tarballs and so on.

On Wed, 04 Mar 2020 at 10:22:31 -0800, Chris Lamb wrote:
> I haven't checked the build history but assuming this is not in-of
> itself a nondetermistic distinction (in which I would blame the hash
> table usage) then this is likely due to filesystem ordering. Indeed if
> you look at gtk/updateiconcache.c src:gtk+3.0 then, after a quick
> skim, both of these things are apparent in this file.

Yes, it's both. The order of directory entries is in readdir() order,
the order of insertion of per-file entries into a hash table is also
influenced by readdir() order, and the contents of the hash table
are written into a hash-table-on-disk data structure in arbitrary hash
table iteration order.

Following some late-night hacking, I now have patches that I believe
should fix this, which I need to test (not even compiled yet), and a
Python script to dump the contents of a cache in a readable format,
which I intend to use to check that the result with the patched GTK
is equivalent.

(The file format is more complicated than it probably ought to be, for
compatibility with historical GTK versions.)

On Thu, 05 Mar 2020 at 09:36:25 -0500, Daniel Kahn Gillmor wrote:
> If i'm understanding you correctly, this suggests that the results
> created by gtk-update-icon-cache do not belong in any arch-independent
> package.
> 
> Is this correct?

No, the results of gtk-update-icon-cache are not intentionally
architecture-dependent. It's all done with fixed-size integers and network
endianness, so in principle it ought to be portable and reproducible -
but the implementation doesn't make any attempt to avoid being influenced
by hash and readdir() order.

> Is there some magic that a tool like lintian could use to identify an
> icon-cache file shipped in an arch: all package?

No. The only fixed string is that it starts with 0x00 0x01 0x00 0x00
(major version 1, minor version 0) which probably collides with dozens
of other ad-hoc binary file formats. You'd be better off recognising
the filename.

    smcv