[pymvpa] Searchlight + cross validation & significance (permutation test)

[Nynke van der Laan]

Thanks for your replies!
Francisco, great suggestion, I will absolutely consider that
classifier for my next study. For the present study/additional
analysis though, I would like to stick with the v4.0 version and the
linear SVM as this showed good results (and I preferably don't switch
to a different version in between analyses/revisions).
Jo, using multiple computers is indeed a good option,

However, in the documentation I read that if there is a strong
assumptiom of the data being normally/gaussian distributed
(http://v04.pymvpa.org/examples/permutation_test.html), 30
permutations would also suffice. As my data indeed approaches a normal
distribution, I would like to try this first. But, unfortunately I did
not manage to implement this into the searchlight. In the example, it
is clearly described how this permutation test would be done in a
'normal' analysis, but how exactly to combine this with the
searchlight is unclear to me. I assume the null_dist = MCNullDist...
should be included in the TransferError in the script below, but the
searchlight now only returns the error scores for each voxel and no
other results. How could I perform the permutation test for each
searchlight and let the searchlight output also the p values for
testing against the null distribution?

cv = CrossValidatedTransferError(
         TransferError(LinearCSVMC()),
         NFoldSplitter())
sl = Searchlight(cv, radius=6)
sl_map = sl(dataset_total)