[pymvpa] Within-subject classification, getting each subject's scores averaged over number of runs

[Martin Sjøgård]

Thanks, exactly what I needed!

Martin

________________________________________
From: Pkg-ExpPsy-PyMVPA <pkg-exppsy-pymvpa-bounces+sjogard=stud.ntnu.no at lists.alioth.debian.org> on behalf of pkg-exppsy-pymvpa-request at lists.alioth.debian.org <pkg-exppsy-pymvpa-request at lists.alioth.debian.org>
Sent: Saturday, April 23, 2016 2:01 PM
To: pkg-exppsy-pymvpa at lists.alioth.debian.org
Subject: Pkg-ExpPsy-PyMVPA Digest, Vol 98, Issue 8

Send Pkg-ExpPsy-PyMVPA mailing list submissions to
        pkg-exppsy-pymvpa at lists.alioth.debian.org

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

or, via email, send a message with subject or body 'help' to
        pkg-exppsy-pymvpa-request at lists.alioth.debian.org

You can reach the person managing the list at
        pkg-exppsy-pymvpa-owner at lists.alioth.debian.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Pkg-ExpPsy-PyMVPA digest..."


Today's Topics:

   1. Within-subject classification, getting each subject's scores
      averaged over number of runs (Martin Sj?g?rd)
   2. Re: Within-subject classification, getting each subject's
      scores averaged over number of runs (Yaroslav Halchenko)


----------------------------------------------------------------------

Message: 1
Date: Fri, 22 Apr 2016 12:17:03 +0000
From: Martin Sj?g?rd <sjogard at stud.ntnu.no>
To: "pkg-exppsy-pymvpa at lists.alioth.debian.org"
        <pkg-exppsy-pymvpa at lists.alioth.debian.org>
Subject: [pymvpa] Within-subject classification, getting each
        subject's scores averaged over number of runs
Message-ID:
        <HE1PR05MB106566BB2B8EC0F863C4D9E88A6F0 at HE1PR05MB1065.eurprd05.prod.outlook.com>

Content-Type: text/plain; charset="iso-8859-1"

I'm running some within-subject classifications in pyMVPA (same as the example script on the site/in the manual in the hyperalignment section). Everything works fine, and I get the average within-subject classification scores. However, I also want to the classification scores for each subject.


The results are organized by the lines:

wsc_results = [cv(sd) for sd in sd_all)

wsc_results = vstack(wsc_results)


If I type in:

print wsc_results


I get each subject's scores per run (7 runs), but I of course want to average the runs per subject. I've tried:


wsc_results = [cv(sd)/nruns for sd in sd_all)


But that doesn't work (TypeError: unsupported operand type(s) for /: 'Dataset' and 'int').


Do you know how to get each subject's classification scores, averaged over number of runs?



Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20160422/734b84bd/attachment-0001.html>

------------------------------

Message: 2
Date: Fri, 22 Apr 2016 08:38:43 -0400
From: Yaroslav Halchenko <debian at onerussian.com>
To: Development and support of PyMVPA
        <pkg-exppsy-pymvpa at lists.alioth.debian.org>
Subject: Re: [pymvpa] Within-subject classification, getting each
        subject's scores averaged over number of runs
Message-ID: <20160422123843.GR23764 at onerussian.com>
Content-Type: text/plain; charset=utf-8


On Fri, 22 Apr 2016, Martin Sj?g?rd wrote:

>    I'm running some within-subject classifications in pyMVPA (same as the
>    example script on the site/in the manual in the hyperalignment section).
>    Everything works fine, and I get the average within-subject classification
>    scores. However, I also want to the classification scores for each
>    subject.

>    The results are organized by the lines:

>    wsc_results = [cv(sd) for sd in sd_all)

>    wsc_results = vstack(wsc_results)

>    If I type in:

>    print wsc_results

>    I get each subject's scores per run (7 runs), but I of course want to
>    average the runs per subject. I've tried:

>    wsc_results = [cv(sd)/nruns for sd in sd_all)

>    But that doesn't work (TypeError: unsupported operand type(s) for /:
>    'Dataset' and 'int').

>    Do you know how to get each subject's classification scores, averaged over
>    number of runs?

even this would work

[np.mean(cv(sd)) for sd in sd_all]

but I would recommend to collect not just "mean scores" but all the
confusion matrices "just in case", e.g.

# enable_ca=['stats'] for your cv definion
cms = []
errors = []
for sd in sd_all:
 errors.append(np.mean(cv(sd)))
 cms.append(cv.ca.stats)

then you can get all kinds of stats, e.g.

[cm.stats['ACC%'] for cm in cms]

would be your mean accuracies as well
--
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik



------------------------------

Subject: Digest Footer

_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

------------------------------

End of Pkg-ExpPsy-PyMVPA Digest, Vol 98, Issue 8
************************************************