Thanks again Yarik, for helping with RFE! <br>While working on this again, I ran the script for RFE and uncommented line 62:<br><br>zscore(ds, chunks_attr=None)<br><br>using chunks_attr=None as you suggested since I only have two samples per chunk. However, I got much lower accuracies which is odd... <br>
SMLR: 35.42% ACC<br>SVM: 50% ACC<br>RFE (overfit): 43.75% ACC<br>RFE (overfit less?): 43.75% ACC<br>RFE (half/splits+rank order): 50% ACC<br><br>In looking at the documentation for zscoremapper, I wonder if it's possible for zscoremapper to zscore voxel-wise or per target per chunk/run? It seems like it only zscores sample/target wise.<br>
<br>I appreciate your help! <br><br>Thank you,<br><br>Kimberly Zhou<br><br><br><div class="gmail_quote">On Fri, Jul 8, 2011 at 7:00 AM, <span dir="ltr"><<a href="mailto:pkg-exppsy-pymvpa-request@lists.alioth.debian.org">pkg-exppsy-pymvpa-request@lists.alioth.debian.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Send Pkg-ExpPsy-PyMVPA mailing list submissions to<br>
<a href="mailto:pkg-exppsy-pymvpa@lists.alioth.debian.org">pkg-exppsy-pymvpa@lists.alioth.debian.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa" target="_blank">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a><br>
<br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:pkg-exppsy-pymvpa-request@lists.alioth.debian.org">pkg-exppsy-pymvpa-request@lists.alioth.debian.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:pkg-exppsy-pymvpa-owner@lists.alioth.debian.org">pkg-exppsy-pymvpa-owner@lists.alioth.debian.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Pkg-ExpPsy-PyMVPA digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: RFE and dataset splits (Yaroslav Halchenko)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Thu, 7 Jul 2011 11:04:08 -0400<br>
From: Yaroslav Halchenko <<a href="mailto:debian@onerussian.com">debian@onerussian.com</a>><br>
To: <a href="mailto:pkg-exppsy-pymvpa@lists.alioth.debian.org">pkg-exppsy-pymvpa@lists.alioth.debian.org</a><br>
Subject: Re: [pymvpa] RFE and dataset splits<br>
Message-ID: <<a href="mailto:20110707150408.GJ26537@onerussian.com">20110707150408.GJ26537@onerussian.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
just to catch up on things:<br>
<br>
cons: indeed recipe was not correctly adopted from 0.4.x<br>
pros: no changes needed in PyMVPA itself<br>
<br>
see attached a complete script with few ways for RFE, depending on what you<br>
take for the sensitivity and for the error to decide on when to stop. I was<br>
going also implement a "biased" version, close to the original RFE version from<br>
Guyon, where stopping point is chosen by having a sneak view and the<br>
cross-validation held-out set, which would have negative consequences on<br>
significance testing, which is a corner-stone of machine-learning in<br>
neuroimaging... so I haven't done that -- only truly unbiased, although<br>
possibly slightly overfitting versions available ;-)<br>
<br>
we are yet to adjust the webpage describing the RFE in 0.6.x<br>
<br>
on the dataset you have provided, here is what I got with the script (d'oh --<br>
forgotten about zscoring being commented out):<br>
<br>
SMLR: 58.33% ACC<br>
SVM: 64.58% ACC<br>
RFE (overfit): 66.67% ACC<br>
RFE (overfit less?): 66.67% ACC<br>
RFE (half/splits+rank order): 64.58% ACC<br>
<br>
altogether -- results are comparable with SMLR and I bet with a simple nested<br>
cross-validation to choose lm parameter for SMLR you might get better<br>
results for lower computing cost<br>
<br>
<br>
On Wed, 06 Jul 2011, Yaroslav Halchenko wrote:<br>
<br>
> neh -- that was a good call from me but still not it -- just read it<br>
> nevertheless ;)<br>
<br>
--<br>
=------------------------------------------------------------------=<br>
Keep in touch <a href="http://www.onerussian.com" target="_blank">www.onerussian.com</a><br>
Yaroslav Halchenko <a href="http://www.ohloh.net/accounts/yarikoptic" target="_blank">www.ohloh.net/accounts/yarikoptic</a><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: dataset_rfe.py<br>
Type: text/x-python<br>
Size: 12162 bytes<br>
Desc: not available<br>
URL: <<a href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20110707/692e692a/attachment-0001.py" target="_blank">http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20110707/692e692a/attachment-0001.py</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Pkg-ExpPsy-PyMVPA mailing list<br>
<a href="mailto:Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org">Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org</a><br>
<a href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa" target="_blank">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a><br>
<br>
End of Pkg-ExpPsy-PyMVPA Digest, Vol 41, Issue 6<br>
************************************************<br>
</blockquote></div><br>