[Reproducible-builds] generating reproducible ISOs with xorriso

[Thomas Schmitt]

Hi,

the current situation with data extents looks not good for
the purpose of reproducability.

The files are grafted into a red-black tree according to
their inode and device numbers on hard disk. This is done
to merge hardlinks.
The tree is then serialized into an array which gets
sorted according to the weight values. The extent addresses
are then assigned according to the sorted sequence.

It is not trivial to deduce the paths in the ISO from
the IsoFileSrc objects in the tree. They know best their origin
data on hard disk but have few clue about their destination.

Brute force would be a giant weight list which gives
the sorting rank of each file. Not very appealing to users.

I will have to think about this.

Maybe determine MD5 of file content and sort according to
it before sorting for weights ?
Time consuming and needs at least 16 extra bytes per file,
but would work automatically.
CRC32 would be smaller but the birthday paradox would hit
us within the expectable number of files.
Half an MD5 would be good for about 4 billion files.
Probably.

Or optionally omit the red-black tree and rather create
directly the array from traversing the tree of upcomming
ISO files ? (That was my idea before Vreixo Formoso implemented
the red-black tree for hardlink detection.)
The ISO might get larger that way. Not every user will like.

I will have to think more ...


> it looks like grub-mkrescue can accept xorriso arguments.

Yes. Beware of intermediate GRUB2 git states where the syntax
was changed. Meanwhile Vladimir added more changes to bring it
back to old syntax but i'm not sure whether it is done yet.

It is about whether "--" is needed between grub-mrescue
options and options for xorriso -as mkisofs.
Old and very new syntax is that "--" is not needed. If it
is given, then it will be forwarded to xorriso and end
its -as mkisofs emulation mode.

Intermediate syntax was that "--" separates options of
grub-mkrescue and of xorriso -as mkisofs. (Actually the
better CLI. But the old one was released with GRUB 2.00.)
If "--" was desired for xorriso, it had to be given twice.


Have a nice day :)

Thomas