[pymvpa] how to do multiprocessing in pymvpa?

[Meng Liang]

Hi Nick,

Thanks so much for your reply! Following your suggestion, I downloaded and installed the pprocess on Neurodebian virtual machine on a Windows 7 PC and now the script runs much faster! 

However, it does not work on Mac. I googled but didn't find much relevant info about it - I got the feeling that pprocess is not supported in Mac. Is it true? If so, doesn't that mean running pymvpa on Mac machines cannot utilize multi-core computation?

Finally, an irrelevant question: is the value of 'radius' parameter in the Searchlight command in millimeter? It says so in Pymvpa documentation ( http://www.pymvpa.org/modref/mvpa.measures.searchlight.html ). But in a given example on Pymvpa website, it seems to suggest that the value of 'radius' means the number of voxels ( http://www.pymvpa.org/examples/searchlight_minimal.html ): "# setup searchlight with 2 voxels radius" and "sl = sphere_searchlight(cv, radius=2, space='myspace',...". It's a bit confusing to me...

Also, do you know if multi-core support is also available for other commands other than searchlight? I find that cross-validation with large number of permutations is also quite slow - it would be good if multi-core support is also implemented in the permutation scenario. 

Best,
Meng



> Date: Mon, 4 Feb 2013 14:17:10 +0100
> From: nikolaas.oosterhof at unitn.it
> To: pkg-exppsy-pymvpa at lists.alioth.debian.org
> Subject: Re: [pymvpa] how to do multiprocessing in pymvpa?
> 
> On 4 February 2013 14:09, Meng Liang <meng.liang at hotmail.co.uk> wrote:
> > I'm trying to run a searchlight analysis using the whole brain mask but I
> > found it impossible to finish within a reasonable time - it has been running
> > three days for one two-way classification task in one subject, and it still
> > hasn't finished. To make the computing faster, would multiprocessing be an
> > option? I didn't see any part of the tutorial mentioning multiprocessing -
> > is it because it is already embedded/implemented in pymvpa commands?
> 
> Yes, multi-core support is already part of pyMVPA's searchlight
> implementation. It requires, however, the presence of pprocess [1].
> The Searchlight constructor has an optional parameter 'nproc' that
> controls the number of parallel processes to use. By default
> (nproc=None) it uses the number of cores of the machine - unless
> pprocess is not available, in which case nproc=1.
> 
> As per the example (doc/examples/searchlight.py) it is recommended to
> enable the searchlight debug messages - these include messages related
> to  parallelization.
> 
> In my experience it's easy to use pprocess with neurodebian. On a mac
> (OS X Lion) I did not get it to work, although I have to admit I did
> not try very hard.
> 
> [1] http://pypi.python.org/pypi/pprocess
> 
> 
> 
> >
> > Best,
> > Meng
> >
> > _______________________________________________
> > Pkg-ExpPsy-PyMVPA mailing list
> > Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
> 
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20130204/e9313312/attachment.html>