[pymvpa] Train and test on different classes from a dataset

[Yaroslav Halchenko]

On Sat, 02 Feb 2013, Michael Hanke wrote:
     

>      now (if you are already asleep, I will try doing it later) - just extend
>      it into the power (or ROC detection) analysis -- what we are actually
>      after here.  Add SNR=0.00 -- collect both types of samples and
>      describe each plot with its power to detect the true signal using both
>      kinds of permutation.   Ideally your power should grow with SNR ;)

>    I am not asleep, but I am having a good time doing nothing ;-)

I have tortured your script a bit to bootstrap multiple cases and add
plotting ROCs (even cheated and used scikit-learn for that since we
apparently need overhaul of ROCCurve).  As you see, keeping
testing  portion intact results in lower detection power

-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        
-------------- next part --------------
A non-text attachment was scrubbed...
Name: permutation_test_rocs.png
Type: image/png
Size: 100020 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20130204/a36533de/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: permutation_test_rocs.py
Type: text/x-python
Size: 4574 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20130204/a36533de/attachment-0001.py>