[Soc-coordination] Assorted bootstrapping and cross-compiling projects

[Wookey]

Hello people.

I'm not quite sure what the right/most-useful thing to do here is so
I'm just explaining what I've done and why in terms of GSOC project
proposals.

As you know I am interested in improving debian with regard to
bootstrapping and crossbuilding and associated infrastructural/tools
changes (which brings in subjects like multiarch and sbuild amongst
others). This is useful for new architectures, 'embedded stuff',
general code and packaging quality, working on slower arches etc.

There are various aspects to this work, two of which I posted on the 
wiki last week:
1) Building cross-toolchains with/for multiarch with a view to archive
inclusion at some point
2) A tool to order bootstrap builds to disentangle cyclic build-deps

I have already had interest from students for both of these. 

But task 2 is actually one half of what is needed to make something
useful. It is just the analysis part. You also need some metadata to
analyse and the ability to cross-build at least core stuff .

So there is a 3rd project: 
3) Put reduced dependency info into necessary packages, fix up tools
to understand it, and make sure that the relevant packages cross-build.

Last year we had a 'bootstrapping' task which was actually both 2 and
3 combined. It quickly because clear that that was waaay too much work
for one GSOC and whilst last year's student make valiant efforts from
an initial position of knowing nothing, he only got a fairly small
part of the work done.

However I am very pleased that this student (Gustavo Alkmim) has stuck
around and is keen to work on this again, now much better placed to
produce practical results because he knows what's what. 

I initially felt a bit foolish putting up a project rather similar to
last years, which is why '2' focussed on the part of it we didn't even
start. But on reflection it's all remains stuff that needs doing, and
is not beyond the capabilities of motivated students, so I've now put
up '3' as well 'Bootstrappable Debian', and Gustavo seems to have
started hacking already! 

2 and 3 remain closely related (as I have explained in the text) as
both use the output of the other. However they can also proceed
independently (in 2's case by faking up some test-cases rather than
using real packages).

In pratice it seems quite likely that only one of the two projects
will be selected depending on number of slots, quality of students,
and whatever else we do for rating. It would be good if we got both,
and they both came to fruition, but nothing breaks if that doesn't
happen. Having thought about it carefully, and in the light of last
year, I don't think combining them would work - that's too big a job
for GSOC.

I'm happy to mentor both and have found co-mentors with relevant
detailed expertise and Linaro are OK with me spending some time on
this. So that should all work OK.

Is everybody OK with this? I don't want people to feel I'm spamming
you all with 3 versions of the same thing :-) 

(Actually the multiarch toolchain project is largely independent, but
is related in that we do need something in order to do M-A
crossbuilding in Debian). Project 3 may well start off using rebuilt
ubuntu toolchains as the easiest way to get stuff done until something
is available from Debian/Emdebian.

I guess a related question is how are we selecting/rating projects and
students this year?

Proposal text from the wiki:

Bootstrappable Debian
---------------------

Debian is currently extremely difficult to bootstrap from scratch,
which makes new ports and optimisations very difficult to do. The
primary issues are breaking cyclic build-dependencies and the need to
cross-build at least the initial set of packages. Work has been
ongoing to first analyse and then design a fix this problem over the
last year or two, but a lot of work remains to turn it into a
practical reality. The design is described in
http://wiki.debian.org/DebianBootstrap. The analysis at
http://wiki.debian.org/CircularBuildDependencies

This project consists of doing the actual work of modifying packages
to make optional the optional parts so that build-dependency cycles
can be broken by building packages in 'minimal' form. These changes
need to be made in an upstreamable way so they don't have to be done
manually by every porter. In order to test the work you need to build
modified build-tools that understand and use the extra fields, and in
order to make it useful you need to make sure that the packages
involved cross-build properly, using the new multiarch mechanisms.

This project is closely associated with the 'Port bootstrap
build-ordering tool' project outlined elsewhere. The output of this
work forms the test-data for that project, and the mechanism of this
project is made automable by the output of that project. The two
projects use each other's results, but both can also do their work
independently. Both are required for a fully automated bootstrap.

    Confirmed Mentor: Wookey

    How to contact the mentor: wookey at wookware.org or IRC #emdebian, #multiarch on OFTC

    Confirmed co-mentors: Jonathan Austin <Jonathan.Austin at arm.com>

    Deliverables of the project:
        Set of patches for control and rules files to provide staged 'bootstrap' builds that break cyclic build-deps
        Multiarch cross-building patches that allow cross-building of the initial core set of packages 

    Desirable skills:
        Knowledge of packaging and package-building
        Knowledge of cross-building methods and issues 

    What the student will learn:
    How Debian is bootstrapped, dependencies fit together, and how
    multiarch cross-building works. You will learn what is core and
    what is optional in a range of pieces of software, as well as
    getting great insight into the myriad forms of packaging of
    different tools, languages and libraries. You will understand what
    a core GNU/linux system is actually made from, the whole set of
    package and build tools, what _really_ happens when you build a
    bit of software, and the coolness of the multiarch mechanism. 


Wookey
-- 
Principal hats:  Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/