[Reproducible-builds] Bug#807997: diffoscope: CbfsFile checks make diffoscope extra slow

[Mike Hommey]

Source: diffoscope
Version: ada1a1dcdc19217fb611e0a1e57bc3744399aefa
Severity: important

Dear Maintainer,

   * What led up to the situation?

Comparing
http://archive.mozilla.org/pub/firefox/try-builds/mh@glandium.org-93eecb52fcadc562b0f52cb22a8971770446a12e/try-linux64/firefox-45.0a1.en-US.linux-x86_64.tar.bz2
with
https://queue.taskcluster.net/v1/task/Y1kcxbpQQcOQhIHDp7oRxA/runs/0/artifacts/public/build/target.tar.bz2

   * What exactly did you do (or not do) that was effective (or
     ineffective)?

Ran `diffoscope --html output.html firefox-45.0a1.en-US.linux-x86_64.tar.bz2 target.tar.bz2`

   * What was the outcome of this action?

It took 9 minutes and 15 seconds on my machine.

   * What outcome did you expect instead?

It taking much less time. In fact, after doing some profiling and seeing that
CbfsFile was a hotspot, applying the following patch made the command take 80
seconds instead, most of which is spent bunzip2'ing the two archives
sequentially, and objdump --disassemble'ing libxul.so in the two archives in
parallel (so there could be a win with bunzip2'ing the two archives in
parallel, too).

diff --git a/diffoscope/comparators/__init__.py b/diffoscope/comparators/__init__.py
index b4615c9..c4cffbb 100644
--- a/diffoscope/comparators/__init__.py
+++ b/diffoscope/comparators/__init__.py
@@ -129,7 +129,6 @@ FILE_CLASSES = (
     DebDataTarFile,
     TextFile,
     Bzip2File,
-    CbfsFile,
     CpioFile,
     DebFile,
     DexFile,



-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.2.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=ja_JP.UTF-8, LC_CTYPE=ja_JP.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)