<div dir="ltr"><div>Thanks so much for responding to my question - I used the CustomPartitioner option and it worked! (I'm so sorry it took so long to reply, wanted to wait til I ran the searchlight analyses, which I just finished, to have a more coherent answer). I had 2 follow up questions regarding interpretation of results... </div><div><br></div><div>1) if my searchlight script doesn't include the "sl_map.samples - 1" line, does that mean that all my values when I visualize them on FSL for example will be error rates? (instead of accuracy values?) Here's my script:</div><div><br></div><div><i>results = sl(dataset)</i></div><div><i>niftiresults = map2nifti(results, imghdr=dataset.a.imghdr)</i></div><div><br></div><div>Do I need to add the following lines to my script to get accuracy rates?</div><div><i>sphere_errors = results.samples[0]</i></div><div><i>map2nifti(1-sphere_errors)</i></div><div><br></div><div>2) I ran a one-sample t-test on SPM using the .nii images I got for each participant from the searchlight analyses (after converting them to standard space) but I'm not sure how to threshold the image to correct for multiple comparisons? Or is this something that the program does? For example, when I open the group contrast (or beta) image after running the one-sample t-test on FSL and specify min=0.5, max=1, I get several regions that show up but when I do min=0.51 there are a lot less regions (regions that I hypothesized would be able to discriminate between trials in my task) but 0.51 doesn't seem that much different from 0.5 - how can one tell what the threshold to be different from chance should be?</div><div>Similarly when I open the contrast (or beta) image on xjview, if I change the intensity value from 0.5 to 0.51 I get similar results.</div><div><br></div><div>Thank you again for any clarification you might provide!</div><div>Lynda</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 6, 2017 at 5:03 PM, Yaroslav Halchenko <span dir="ltr"><<a href="mailto:debian@onerussian.com" target="_blank">debian@onerussian.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
On Thu, 05 Oct 2017, Lynda Lin wrote:<br>
> I've tried it 3 different ways and I'm getting different results for each<br>
> way so just wanted to know if any of these ways is valid:<br>
<br>
</span>sorry... just a quick reply, let's see if it would satisfy ;)<br>
<span class=""><br>
> 1) Using the manual split example from the tutorial and calling the<br>
> "training_stats" conditional attribute in the classifier<br>
> In the tutorial we can get the individual accuracies for each run through<br>
> cv_results.samples but I'm interested in the TPR (True Positive Rate) for<br>
> Ingroup and Outgroup separately so I'm looking to print the confusion<br>
> matrix to calculate these numbers<br>
<br>
</span>you could access .stats dictionary with all those values, e.g.<br>
cve.ca.stats.stats["TPR"]<br>
<span class=""><br>
> ds_split1 = ds[ds.sa.chunks == 1.]<br>
> ds_split2 = ds[ds.sa.chunks == 2.]<br>
> clf = LinearCSVMC(enable_ca=['<wbr>training_stats'])<br>
> clf.set_postproc(BinaryFxNode(<wbr>mean_mismatch_error,'targets')<wbr>)<br>
> clf.train(ds_split1)<br>
> err = clf(ds_split2)<br>
> clf.ca.training_stats.as_<wbr>string(description=True)<br>
<br>
</span>this is training_stats, so how well you fit (you can get it right after<br>
.train), not as you predict<br>
<span class=""><br>
> 2) Using the HalfPartitioner function's "count" argument<br>
<br>
</span>> Â clf = LinearCSVMC(enable_ca=['<wbr>training_stats']) #The training_stats<br>
<span class="">> confusion matrix from this method doesn't match the one above<br>
</span>> Â hpart = HalfPartitioner(count=1, attr='chunks')<br>
> Â cvte = CrossValidation(clf,hpart,<wbr>errorfx=lambda p,t:<br>
> np.mean(p==t),enable_ca=['<wbr>stats'])<br>
> Â cv_results = cvte(ds)<br>
> Â cvte.ca.stats.as_string(<wbr>description=True)<br>
<br>
You could also just use CustomPartitioner([([1], [2])])<br>
which should generate a single split where first partition would have<br>
only the ones with value 1, and the second with the value 2.<br>
ca.stats should be appropriate<br>
<br>
is your dataset balanced in terms of # of samples per label?<br>
output of ds.summary()<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Yaroslav O. Halchenko<br>
Center for Open Neuroscience <a href="http://centerforopenneuroscience.org" rel="noreferrer" target="_blank">http://<wbr>centerforopenneuroscience.org</a><br>
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755<br>
Phone: <a href="tel:%2B1%20%28603%29%20646-9834" value="+16036469834">+1 (603) 646-9834</a> Fax: <a href="tel:%2B1%20%28603%29%20646-1419" value="+16036461419">+1 (603) 646-1419</a><br>
WWW: <a href="http://www.linkedin.com/in/yarik" rel="noreferrer" target="_blank">http://www.linkedin.com/in/<wbr>yarik</a><br>
</font></span></blockquote></div><br></div>