Atlas proposal

[Ian Jackson]

I think the key thing to remember here is that both writing fast code,
and its subsequent deployment and performance tuning, are very hard
problems which are fields of research in their own right.  Our job in
Debian is not primarily to try to outdo others' work by producing
programs which are better in all of performance, portability,
convenience, and impact on other users of the system.  

Our job is to package up and present the existing work so that our
users have access to, and their pick of, the best trade-offs currently
available.  So:

Josselin Mouette writes ("Re: Atlas proposal"):
> Just ship two packages: libatlas3gf-base and libatlas3gf-auto, with
> appropriate Conflicts/Provides just as we have now. And have
> libatlas3gf-auto depend on all build-dependencies, ship the source, and
> build itself in its postinst. Gentoo-style.

I think this is exactly the right thing to do.  It also solves the
problem correctly in cluster or hpc farm deployments: each node gets
a version of the library optimised according to its local properties.

Admins who have more complicated setups (eg, multiple users
timesharing on a single node) can either just install the -base
package, or they can use the debconf parameters to detune the library.
The -base package should be optimised for a modest number of CPUs (4,
say) and the smallest possible cache size etc., so that the
performance isn't crippled on even the slowest machines.

> In libatlas3gf-auto you could use debconf for overriding things like the
> number of threads or some optimizations, but I wouldn?t use debconf for
> selecting the implementation.

Right.  The debconf questions should be fairly low priority and should
default to "make it fast on this very computer".  dpkg-reconfigure
should be made to always rebuild the package (eg if the host hardware
has changed).

The only remaining questions are: where in the filesystem should all
of this take place and which files should be deleted or kept ?  I
would say:
  * Source code distributed in the .deb as
     /usr/src/autobuild/libatlas3gf-auto/*
  * README.DO-NOT-EDIT file in the above directory, and all the
     .deb-installed files are mode 600, to try to avoid the user
     editing the source.
  * Built in that directory and when completed, installed in
     /usr/lib, /usr/include.  Some way of tracking the files involved
     would be a good idea; perhaps have the library install in
     a tmp subdirectory just like for a package build and then
     make a list of the files before installing them.
  * Build tree cleaned after installation - have a debconf question
     to avoid this.
  * Package removal (or pre-upgrade) always removes all the installed
     files and removes everything in /usr/src/autobuild.
  * Do some tests to make sure it doesn't leave cruft behind.

It might be worth doing this:

   Recommends: ccache

   PATH=/usr/lib/ccache:$PATH
   CCACHE_DIR=/var/cache/autobuild/ccache

Ian.