Atlas proposal

[Don Armstrong]

On Tue, 17 Aug 2010, Samuel Thibault wrote:
> Roger Leigh, le Tue 17 Aug 2010 22:45:50 +0100, a écrit :
> > Why can't this be fixed the correct way:
> > by building all optimised variants for a given architecture and
> > selecting the appropriate variant at runtime based upon the system's
> > capabilities e.g. from CPUID on i386/amd64?
> 
> Because atlas doesn't optimise only for the instruction set, but
> also the number of available cores, the size of the caches, etc.
> etc.

All of these are things that can be detected at run time and
appropriate libraries dlopened or codepaths diverged, etc. Whether
upstream can be convinced of the wisdom of this doesn't change that
this is the right design from a distribution perspective, and I
submit, for an upstream perspective as well.

> > Disabling threading is also suspect: how can the optimal number of
> > threads possibly be determined at build time?
> 
> Because it changes how Atlas will statically schedule the
> computation kernels.

This answers the wrong half of the question; there's no way to know at
build time what precisely the machine is going to be doing. It's
possible that you'll have 8 different processes running atlas, and so
atlas shouldn't be using as many threads as if you were only running 1
atlas process.
 
> It's not "wrong", it's HPC. And HPC people will happily rebuild the
> package to get an optimized version.

It's wrong even in HPC unless you tweak the settings of atlas
compilation for your particular problem set as well as your hardware
and software architecture.

But all of that is fine; we can't possibly hope to optimize to get the
last iota of performance out of a system. We should attempt to provide
a reasonable set of optimized binaries (whether that means one or ten
is up to the package maintainer), and then provide an easy method to
build packages which can seemlessly be installed and used in a Debian
system.

This doesn't mean using debconf. After all, anyone who is actually
doing HPC is going to build the packages on one machine, upload them
to their local Debian mirror, and deploy them across their cluster.


Don Armstrong

-- 
Where I sleep at night, is this important compared to what I read
during the day? What do you think defines me? Where I slept or what I
did all day?
 -- Thomas Van Orden of Van Orden v. Perry

http://www.donarmstrong.com              http://rzlab.ucr.edu