[pymvpa] Alternatives to Cross-validation

[Jacob Itzhacki]

"e.g.  some super-ordinate category (e.g.  animate-vs-inanimate)  you
would like to cross-validate not across functional runs BUT across
sub-ordinate stimuli categories (e.g. train on
humans/reptiles/shoes/scissors to discriminate animacy and
cross-validate into bugs/houses, then continue with another pair to take
out)."

BTW, this exactly what I would like to do but I still don't figure out how
to leave out the test trials from the training trials, so they don't get
classified into themselves.

On Wed, Oct 24, 2012 at 5:15 PM, Jacob Itzhacki <jitzhacki at gmail.com> wrote:

> Please do!
>
> and thank you for all the responses :D
>
> Don't want to come across as lazy but I'm not a master coder at all so
> sometimes figuring out what one line of code does can be quite the ordeal,
> in my case.
>
> J
>
> On Wed, Oct 24, 2012 at 3:54 PM, Yaroslav Halchenko <debian at onerussian.com
> > wrote:
>
>>
>> On Wed, 24 Oct 2012, MS Al-Rawi wrote:
>> >    Cross-validation is fine even in this case, you'll just need to
>> rearrange
>> >    your data in a way to leave-a-set-of-stimuli out, instead of
>> >    leave-one-run-out. Perhaps PyMVPA has some functionality to do this.
>>
>> now it is getting interesting -- I think you got close to what I thought
>> the question was about:  to investigate the conceptual/true effect of
>> e.g.  some super-ordinate category (e.g.  animate-vs-inanimate)  you
>> would like to cross-validate not across functional runs BUT across
>> sub-ordinate stimuli categories (e.g. train on
>> humans/reptiles/shoes/scissors to discriminate animacy and
>> cross-validate into bugs/houses, then continue with another pair to take
>> out).  And that is what I thought for a moment the question was
>> about ;)
>>
>> This all can be (was) done with PyMVPA although would require 3-4
>> lines of code instead of 1 to accomplish ATM.  If anyone interested I
>> could provide an example ;)... ?
>>
>> --
>> Yaroslav O. Halchenko
>> Postdoctoral Fellow,   Department of Psychological and Brain Sciences
>> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
>> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
>> WWW:   http://www.linkedin.com/in/yarik
>>
>> _______________________________________________
>> Pkg-ExpPsy-PyMVPA mailing list
>> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
>> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20121025/ed742167/attachment.html>