[Soc-coordination] Bootstrappable Debian - Report 3
P. J. McDermott
pjm at nac.net
Tue Jul 3 01:06:58 UTC 2012
This is report three for the "Bootstrappable Debian" project [1][2]
mentored by Wookey and co-mentored by Jonathan Austin.
Copies of this report are sent to the debian-bootstrap [3] and
debian-embedded lists.
[1]: http://wiki.debian.org/SummerOfCode2012/Projects#Bootstrappable_Debian
[2]: http://wiki.debian.org/SummerOfCode2012/StudentApplications/PJMcDermott
[3]: http://lists.mister-muffin.de/cgi-bin/mailman/listinfo/debian-bootstrap
Work Done
=========
Staged Build Support in dpkg-dev
--------------------------------
Working to support "Build-Depends-StageN" fields for any value of "N" in
dpkg-dev, I added %FIELDS_RE, field_exists(), and field_get() to
Dpkg::Control::Fields and started changing the module's subroutines to
use the new additions [5]. Then I realized that the field names are
used in other modules and in scripts, e.g. to match fields in
dpkg-gencontrol. I took some notes [6] on these uses of %FIELDS but
didn't get farther than dpkg-gencontrol.
[5]: http://bootstrap.pehjota.net/dpkg/dpkg-fields-re.patch
[6]: http://bootstrap.pehjota.net/dpkg/dpkg-fields-usage.txt
Perl Cross Building Support
---------------------------
I've been investigating what needs done to refresh the src:perl cross
building patch that Steve McIntyre proposed in bug #633884 [7] so that
it applies to the newer versions of src:perl in sid and experimental. I
looked at the differences [8] between the 5.12.4-1 package, the
5.14.2-11 package, the 5.16.0-1 package, and the patch that Steve
posted.
I found that refreshing the Debian packaging part of the patch will be
trivial. Refreshing the upstream build system part, however, will be
much more involved. Between versions of Perl there are several hundreds
of lines of changes to the build system (which is generated using the
"metaconfig" tool in the long-unmaintained "dist" package [9], then
changed/fixed by hand because "metaconfig is hard to change" [10]).
[7]: http://bugs.debian.org/633884
[8]: http://bootstrap.pehjota.net/cross/pkgs/perl/diff-notes.txt
[9]: http://anonscm.debian.org/gitweb/?p=users/srivasta/debian/dist.git
[10]:
http://perl5.git.perl.org/perl.git/blob/c3143508:/Porting/pumpkin.pod#l311
New Domain and Package Archive
------------------------------
In an effort to better organize my published work on this project, I set
up a new domain, bootstrap.pehjota.net, and copied many of my files into
the new document root [11]. Old hyperlinks will continue to work, but I
am no longer pushing files into my personal home directory on my server.
I also set up a package archive [12] using reprepro to serve all the
source and binary packages that I produce throughout this project.
Currently there are two components in this archive:
* main: This component provides support packages for things like cross
building. (See below for more information.)
* cross: This component provides base build system and loop-breaking
packages that have been cross built. In cases where a package has
to be modified to successfully cross build, a source package is
provided. (See below for more information.)
[11]: http://bootstrap.pehjota.net/
[12]: http://bootstrap.pehjota.net/archive/
Support Packages for Cross Building
-----------------------------------
I noticed [13] that sbuild (>= 0.63.0) installs packages necessary for
cross building through a package named "crossbuild-essential-<target>".
However, this package does not yet exist in the Debian archive.
So, I forked src:build-essential into a new source package named
"cross-build-essential" [14][15] that builds arch:all packages named
"crossbuild-essential-<target>". Once Thibaut Girka's multiarch cross
toolchains move into the Debian archive, src:build-essential and
src:cross-build-essential can be merged back together [16] and the
Debian archive can provide packages that list and depend upon packages
that are necessary for cross building.
I also noticed that the sbuild source code lists pkg-config-<triplet>
(also not in Debian) as an essential package for cross building. Wookey
explained that this package is supposed to simply provide a link at
/usr/bin/<triplet>-pkg-config to /usr/share/pkg-config-crosswrapper.
After finishing src:cross-build-essential, I made from scratch a
src:pkg-config-cross package [17][18] that builds arch:all packages
named "pkg-config-<triplet>".
Finally, I realized that we'll need gcc-<triplet>, cpp-<triplet>,
g++-<triplet>, etc. binary packages that depend on and provide symbolic
links to default versions of cross tools built from GCC sources. These
packages would be similar to the gcc, cpp, g++, etc. packages built from
src:gcc-defaults. Working around the lack of such packages is easy
enough, so I just noted [19] this as an item to be done eventually. The
discussion continued between the debian-bootstrap and debian-embedded
[20] lists with update-alternatives offered as an alternative to default
version metapackages.
[13]:
http://lists.mister-muffin.de/pipermail/debian-bootstrap/2012-June/000224.html
[14]: http://bootstrap.pehjota.net/build-essential/
[15]:
http://bootstrap.pehjota.net/archive/pool/main/c/cross-build-essential/
[16]:
http://lists.mister-muffin.de/pipermail/debian-bootstrap/2012-June/000231.html
[17]: http://bootstrap.pehjota.net/pkg-config/
[18]: http://bootstrap.pehjota.net/archive/pool/main/p/pkg-config-cross/
[19]:
http://lists.mister-muffin.de/pipermail/debian-bootstrap/2012-July/000248.html
[20]: https://lists.debian.org/debian-embedded/2012/07/msg00000.html
sbuild Chroot for Cross Building
--------------------------------
On my wheezy amd64 system I set up a sid i386 chroot with sbuild/schroot
and installed in it Thibaut's multiarch cross toolchain packages and my
cross building support packages.
I experienced a couple of rather significant setbacks in this vain.
First, when I tried to install a C cross compiler, APT in its default
configuration tried to remove the build-essential, cpp, g++, and gcc
metapackages, all of which are necessary for building anything with
sbuild. I traced the problem to libc6-dev:armhf and its Recommends
relationship with gcc. Configuring APT to not install recommended
packages solved this issue.
Second, Thibaut's g++-4.7-<target> packages can't be installed until bug
#678623 [21] against libstdc++6-4.7-dev is resolved; in the meantime, I
used equivs to generate a fake g++-4.7-arm-linux-gnueabihf package [22]
to satisfy the dependency of my crossbuild-essential-armhf package.
I documented [23][24] the whole setup procedure for my own reference and
in case it is of interest to anyone else. Eventually, this information
should be added to the Debian Wiki.
[21]: http://bugs.debian.org/678623
[22]: http://bootstrap.pehjota.net/cross/g++-4.7-arm-linux-gnueabihf.ctl
[23]: http://bootstrap.pehjota.net/cross/sbuild-setup.txt
[24]: http://bootstrap.pehjota.net/cross/sbuild-setup.html
Base Build System Cross Building
--------------------------------
With cross building support packages built and a cross-building chroot
setup, I've finally begun to attempt to cross build the packages that
comprise a base system that is capable of natively building other
packages.
I started with a fairly simple arch:any package: src:base-files. This
package cross built successfully [25][26] as it was.
Then I moved on to a simple package that used a compiler: src:dash.
This package initially failed [27] to cross build because it executed
"strip" instead of "<triplet>-strip" when cross building. So I patched
[28] the source package and built binaries [29][30] for armhf. Finally,
I found a report of this problem [31] submitted by Marcin Juszkiewicz,
followed up on it, and categorized it with the cross build usertag [32].
See also the armhf Packages file [33] in my archive that describes my
cross-built packages.
Overall, src:base-files and src:dash have been good proof-of-concept
packages to test my cross building chroot and to help me adjust to the
cross building, patching, and bug reporting workflow.
Finally, I've begun development of a (not yet published) shell script in
an attempt to partially automate and speed up my workflow of:
* Downloading a Debian source package,
* Editing it if necessary,
* Building it with sbuild, and
* Pushing my work to my server.
I don't except the script to be very useful beyond this project and my
own cross building workflow, as it's not designed for an automated
bootstrap (rather it's designed for one in which many packages need to
be fixed manually). But its use can be scripted, so it may be of
interest someday for at least semi-automated cross building.
[25]:
http://bootstrap.pehjota.net/cross/builds/base-files/base-files_6.11_armhf-20120701-1535.build
[26]: http://bootstrap.pehjota.net/archive/pool/cross/b/base-files/
[27]:
http://bootstrap.pehjota.net/cross/builds/dash/dash_0.5.7-3_armhf-20120701-2018.build
[28]: http://bootstrap.pehjota.net/cross/pkgs/dash/dash-cross.debdiff
[29]:
http://bootstrap.pehjota.net/cross/builds/dash/dash_0.5.7-3.1_armhf-20120701-2131.build
[30]: http://bootstrap.pehjota.net/archive/pool/cross/d/dash/
[31]: http://bugs.debian.org/665971
[32]:
http://bugs.debian.org/cgi-bin/pkgreport.cgi?users=crossbuild@debian.org&tag=cross
[33]:
http://bootstrap.pehjota.net/archive/dists/sid/cross/binary-armhf/Packages
Next Steps
==========
Perl Cross Building Support
---------------------------
I will continue to investigate Perl's build system in my efforts to
refresh the existing cross building patch to apply to the newer versions
of src:perl.
Base Build System Cross Building
--------------------------------
I will attempt to cross build the rest of the base build system packages
and try to fix packages that don't yet cross build. This is the first
major component of a full architecture bootstrap.
Also, I plan to spend another day or two finishing the aforementioned
workflow automation script. It should make my cross building work a
little more efficient and deterministic. I should have it published
sometime within the next few days; mostly I just need to figure out
where I want to put it – e.g. in a Git repository accessible through
odin1.pehjota.net and/or on Gitorious.
--
P. J. McDermott (_/@\_) ,--.
http://www.pehjota.net/ o < o o > / oo \
http://www.pehjota.net/contact.html o \ `-/ | <> |.
o o o "~v /_\--/_/
More information about the Soc-coordination
mailing list