[Reproducible-builds] concrete steps for improving apt downloading security and privacy

Fri Sep 19 16:39:35 UTC 2014

Hi Hans--

I think we're in agreement here about most things actually, despite our
back-and-forth.  hopefully this is a clarifying response:

On 09/19/2014 12:11 PM, Hans-Christoph Steiner wrote:
> Daniel Kahn Gillmor wrote:
>> If we succeed in creating reproducible builds (we're farther along
>> toward that goal than i had dared hope, it's exciting!) then one of the
>> nice opportunites we have is for other people or organizations to
>> corroborate the build after the package is first distributed.  For
>> example, an upload to sid might have one signature (by the original
>> uploader), but maybe it waits to transition to testing until there are
>> corroborations from multiple builders. (Note: this is not a concrete
>> proposal or an expectation of exactly what will happen, just a thought
>> experiment)
>>
>> In this scenario, how do you propose to add those signatures into the
>> package?  If we bundle them into the .deb, then the size and digest of
>> the .deb itself changes after it is first distributed.  This seems like
>> it would violate all sorts of existing expectations -- i can't imagine
>> how many different tools and pieces of infrastructure expect that
>> foo_1.2-3_mipsel.deb should always have the same size and digest.
> 
> There needs to be a canonical, immutable signature (which could be a set of
> individual signatures) on the final, official package for this to be a
> workable system.  I like the idea of having a threshold of builder signatures
> before a package is accepted into the official repo.  That makes a canonical
> signature easy to: it would just be a whatever signatures that triggered the
> inclusion, then perhaps also a dak signature.

In that case, the .deb that was installed on a sid system *is not* the
.deb that is installed on a testing system.

If i run a mixed unstable/testing system (i do, actually, this is not
hypothetical) should i need to re-install foo_1.2-3_mipsel.deb when that
package transitions from unstable to testing (without any changes made
to it other than new signatures)?  That seems odd, but the .debs are now
no longer bytewise identical.  should archive operators who are doing
rsync mirroring of a number of pools update their .debs as new
signatures are added to them?  can they still use rsync for this cleanly
without a massive increase in bandwidth between mirrors or do we need to
define a new synchronization mechanism?

The question of what is a "canonical, immutable signature" for any given
distribution is also problematic, because it ties the policy of the
distribution (already defined by what that apt repository includes and
references in Release.gpg) to a set of individual package signatures.
But this is exactly the point where we'd like more flexibility.  People
who care about apt repo X and use it online can use Release.gpg, while
people who are *not* using the apt repo might have a different set of
policies.  And some repos might want to share specific packages with
each other -- what if their signing policies about the "canonical"
signature conflict?  should they have to rebuild the package?

> Android's APK signature is a good example of this.  The Android system uses
> TOFU/POP on the set of signatures.  At least one signature is required for an
> APK to be installed, but it can also be many signatures.  Then whenever
> updating, the new APK's set of signatures must match the installed APK's set.
>  This means that it is easy to install test versions of packages as long as it
> is not updating an installed package with a different signature.

I think this is a question about cryptographically-strong package
upgrade paths from a heterogeneous and fluid set of origins, which is a
really neat topic.  But it's maybe orthogonal to the current discussion,
so i'm going to avoid pursuing it here now :)

>> I don't think this is the case.  People who download a .deb and verify
>> it could also download the associated .buildinfo file and whichever
>> signatures they'd like (or all of them) and verify the package that way.
> 
> I strongly disagree.  Adding a separate file for the signature turns it into
> something that only security-conscious techie people will do because it turns
> it into a conscious decision and action.  Only a minority of techies actively
> think about verifying software, it is basically a non-existent practice for
> non-techies.  The signature and it's verification must be transparent to
> actually work for the 99% of users who do not think about this stuff.
>
> Having the signature embedded in the file itself means that .deb files can be
> freely swapped via whatever available means, be it USB thumb drives, direct
> download, LibraryBox, the cloud, SparkleShare, email, IM, whatever.  Then the
> signature can be transparently verified by dpkg on install, and the user can
> be entirely ignorant that the signature even exists unless there is a problem
> with it.

You're entirely right that when fetching files via the web directly
(instead of an apt repository) or sneakernet, people tend to transfer
only the minimal set of possible files, and therefore having detached
signatures is a bad idea for adoption.

But i don't think this addresses the concern raised above that specific
.deb files have constant size and contents, which is an assumption that
permeates the repository, mirror, and distribution mechanisms.
Rejecting that assumption means potentially breaking a lot of
infrastructure that currently works, as well as forcing incompatible
policy changes on archive operators.

So i'd like to have this cake and eat it too, please :)

Here's a proposal for chewing over:

 * define a new package format called .debs

 * foo_1.2-3_mipsel.debs is a tarball that contains at least three files:

foo_1.2-3_mipsel.deb
foo_1.2-3_mipsel.buildinfo
foo_1.2-3_mipsel.buildinfo.0EE5BE979282D80B9F7540F1CCD2ED94D21739E9.asc

  (it can contain more than one .asc if it wants to include multiple
signatures)

 * if you invoke "dpkg -i foo_1.2-3_mipsel.debs", dpkg should unpack and
inspect the .debs, and the signatures, and refuse to install the .deb if
the signatures don't meet local policy.  (i'm hand-waving here about
what "local policy" is, since i think that's a separate discussion)

Now we can leave the current online archive distribution alone -- apt
works (modulo bugs) and archive operators can continue to function as
they currently do.  But we tell users and upstream developers that if
they want to install packages via sneakernet or by downloading them
individually from the web that they really should be passing around
.debs files, and not .deb files.  We could even modify dpkg to reject
installations of plain .deb files unless a package manager (which has
presumably already verified the package by other means) is doing the
installation.

what do you think?

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20140919/901b87ec/attachment.sig>