<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Dear all,<div><br></div><div>I am trying to obtain the sensitivity values for all splits of the dataset during leave-one-out cross-validation (classification using SVM). I found in the tutorial "Classification Model Parameters – Sensitivity Analysis" ( <a href="http://www.pymvpa.org/tutorial_sensitivity.html" target="_blank" style="font-size: 12pt;">http://www.pymvpa.org/tutorial_sensitivity.html</a> <span style="font-size: 12pt;">) that </span><span style="font-size: 12pt;">RepeatedMeasure(sensana, NFoldPartitioner()) should give the sensitivity values for each fold</span><span style="font-size: 12pt;">. Here are the code snippet I used in my script slightly adapted from the tutorial:</span></div><div><span style="font-size: 12pt;"><br></span></div><div><div> clf = LinearNuSVMC()</div><div> cv = CrossValidation(clf, NFoldPartitioner(),enable_ca=['stats'])</div><div> sensana = clf.get_sensitivity_analyzer()</div><div> cv_sensana = RepeatedMeasure(sensana, NFoldPartitioner())</div><div> error = cv(ds)</div><div> sensmap_cv = cv_sensana(ds)</div></div><div><br></div><div>'print sensmap_cv.shape' <span style="font-size: 12pt;">gave me: (14L, 87L). </span></div><div><span style="font-size: 12pt;"><br></span></div><div><span style="font-size: 12pt;">I have 14 subjects and I am using leave-one-subject-out cross-validation, and there are 87 features.</span><span style="font-size: 12pt;"> </span><span style="font-size: 12pt;">So the data structure seems correct. However, when I look at the values of this 14x87 array, all the rows in the array contain exactly the same values (i.e., the first row looks the same with all the other rows). </span></div><div><span style="font-size: 12pt;"><br></span></div><div><span style="font-size: 12pt;">Is this the correct way to obtain the sensitivity values for each fold of cross-validation classification? If not, any suggestions how to do?</span></div><div><span style="font-size: 12pt;"><br></span></div><div><span style="font-size: 12pt;">A related question about normalizing the sensitivity values: in the "Closing Words" of the tutorial on the same webpage, it says: "</span>It should also be noted that sensitivities can not be directly compared to each other, even if they stem from the same algorithm and are just computed on different dataset splits. In an analysis one would have to normalize them first.<span style="font-size: 12pt;">" My question is: if we cannot compare the sensitivity values from different data splits without normalizing them first, why can we average them or take the maximum value across data splits without applying any normalization (the example script snippets in the tutorial seem to do so)? I would imagine that the average or the max value would also be affected by the scale of the data. </span></div><div><span style="font-size: 12pt;"><br></span></div><div><span style="font-size: 12pt;">Any help would be very much appreciated!</span></div><div><span style="font-size: 12pt;"><br></span></div><div><span style="font-size: 12pt;">Best,</span></div><div><span style="font-size: 12pt;">Meng</span></div><div><span style="font-size: 12pt;"><br></span></div><div><span style="font-size: 12pt;"><br></span></div> </div></body>
</html>