<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Hello,</div><div><br></div><div>I'd like perform some feature selection (using Recursive Feature Elimination) on a data set I'm analyzing, but I haven't been able to make it work. </div><div><br></div><div>I could not find any full example of how to use (rather than just create and/or train) a FeatureSelectionClassifier; I think a full example would be useful. The one example in the documentation showing how to train a FeatureSelectionClassifier did it by calling </div><div><br></div><div>clf.train(dataset)</div><div><br></div><div>... and then calling dataset.selectFeatures(clf.feature_ids)</div><div><br></div><div>This didn't work for me (see the code and errors below). I was working with a different classifier (linear SVM multi-class instead of kNN), and I was working with a slightly different data set (masked data set loaded from a Matlab matrix), but it seems that the same principles should apply. What am I doing wrong? </div><div><br></div><div>I suspect my problem may have something to do with the (bug?) that I wrote to you about previously (<a href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2009q4/000806.html">http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2009q4/000806.html</a>)</div><div><br></div><div>To review, the function clf.getSensitivityAnalyzer(), rather than combining feature sensitivities across comparisons of the data (this is a multi-class classifier), was combining across features. Thus I got 3 sensitivity values (for the comparisons of 1vs2, 1vs3, and 2vs3) rather than 649 values (1 per feature (voxel) in my data set). I was able to read out the feature sensitivities by calling </div><div><br></div><div>clf.getSensitivityAnalyzer(transformer=None,combiner=None), </div><div><br></div><div>but now it seems like the RFE algorithm needs a correct combiner to work. I could not find any documentation on other arguments to provide besides "None" (combiner=??). </div><div><br></div><div>Help? Any idea what's going on? </div><div><br></div><div>The code I'm using and the error messages I get are provided below. </div><div><br></div><div>Thanks (again) for your time, </div><div><br></div><div>Mark </div><div><br></div><div><br></div><div>~~~~~~~~~~~~~</div><div><br></div><div><br></div><div><div>from scipy.io import loadmat</div><div>from mvpa.suite import *</div><div><br></div><div>DatFile = 'WholeBrainMatFile.mat' # 4-D .mat file of 2x2x2 voxels - 440x80x60x69</div><div>MaskFile = 'ROI_Mask.mat' # Contains a mask for 649 voxels in the Lateral Occipital area</div><div>AttrFile = 'ConditionLabels.txt'</div><div><br></div><div>D = loadmat(DatFile)</div><div>Data = D['Data']</div><div>M = loadmat(MaskFile)</div><div>MaskMat = M['Mask']</div><div>attr = SampleAttributes(AttrFile)</div><div><br></div><div># create masked data set</div><div>PyDat = MaskedDataset(samples=Data,labels=attr.labels,chunks=attr.chunks,mask=MaskMat)</div><div>zscore(PyDat,perchunk=True,targetdtype='float32')</div><div># PyDat is: <Dataset / float32 440 x 649 uniq: 8 chunks 3 labels></div><div><br></div><div># Now: feature selection:</div><div><br></div><div>splitter = NFoldSplitter(cvtype=1)</div><div>rfesvm_split = SplitClassifier(LinearCSVMC(),splitter) </div><div>FtSelClf = FeatureSelectionClassifier(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span># use a linear SVM classifier:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>clf = LinearCSVMC(),</div><div><span class="Apple-tab-span" style="white-space:pre"> </span># on features selected via RFE </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>feature_selection = RFE(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span># based on sensitivity of a clf which does splitting internally </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>sensitivity_analyzer=rfesvm_split.getSensitivityAnalyzer(), #transformer=None<span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>transfer_error=ConfusionBasedError( </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>rfesvm_split, </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>confusion_state="confusion"), </div><div><span class="Apple-tab-span" style="white-space:pre"> </span># and whose internal error we use </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>feature_selector=FractionTailSelector( </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>0.2, mode='discard', tail='lower'), </div><div><span class="Apple-tab-span" style="white-space:pre"> </span># remove 20% of features at each step </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>enable_states=['feature_ids'],</div><div><span class="Apple-tab-span" style="white-space:pre"> </span># update sensitivity at each step </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>update_sensitivity=True), </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>descr='LinSVM+RFE(splits_avg)')</div><div><br></div><div># Option 1: simple training and check on feature IDs</div><div><div>print FtSelClf.trained # prints "False"</div><div>FtSelClf.train(PyDat)</div><div>print FtSelClf.trained # prints "True"</div><div>print FtSelClf.feature_ids</div></div><div># (Generates error - see below)</div><div><br></div><div># Option 2: Run cross-validated transfer error</div><div>terr = TransferError(FtSelClf)</div><div>splitter = NFoldSplitter(cvtype=1)</div><div>cvterr = CrossValidatedTransferError(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>terr,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>splitter)</div><div>Err = cvterr(PyDat)</div><div>print Err</div><div># (Also generates error - having NOT run option 1)</div><div><br></div><div>To be clear - I only used EITHER Option 1 or Option 2 (one or the other was always commented out when I ran the code). </div><div><br></div><div>Option 1 gives the error: </div><div><br></div><div><div><font class="Apple-style-span" color="#FF0000">Traceback (most recent call last):</font></div><div><font class="Apple-style-span" color="#FF0000"> File "./FeatureSelection_Example.py", line 77, in <module></font></div><div><font class="Apple-style-span" color="#FF0000"> print FtSelClf.feature_ids</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/misc/state.py", line 1099, in __getattribute__</font></div><div><font class="Apple-style-span" color="#FF0000"> return collections[known_attribs[index]].getvalue(index)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/misc/state.py", line 353, in getvalue</font></div><div><font class="Apple-style-span" color="#FF0000"> return self._items[index].value</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/misc/attributes.py", line 66, in _getVirtual</font></div><div><font class="Apple-style-span" color="#FF0000"> return self._get()</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/misc/attributes.py", line 227, in _get</font></div><div><font class="Apple-style-span" color="#FF0000"> raise UnknownStateError("Unknown yet value of %s" % (self.name))</font></div><div><font class="Apple-style-span" color="#FF0000">mvpa.misc.exceptions.UnknownStateError: Exception: Unknown yet value of feature_ids</font></div><div><br></div></div><div><br></div><div>And Option 2 gives the error: </div><div><br></div></div><div><div><font class="Apple-style-span" color="#FF0000">Traceback (most recent call last):</font></div><div><font class="Apple-style-span" color="#FF0000"> File "./FeatureSelection_Example.py", line 81, in <module></font></div><div><font class="Apple-style-span" color="#FF0000"> Err = cvterr(PyDat)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/measures/base.py", line 105, in __call__</font></div><div><font class="Apple-style-span" color="#FF0000"> result = self._call(dataset)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/algorithms/cvtranserror.py", line 173, in _call</font></div><div><font class="Apple-style-span" color="#FF0000"> result = transerror(split[1], split[0])</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/clfs/transerror.py", line 1283, in __call__</font></div><div><font class="Apple-style-span" color="#FF0000"> self._precall(testdataset, trainingdataset)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/clfs/transerror.py", line 1239, in _precall</font></div><div><font class="Apple-style-span" color="#FF0000"> self.__clf.train(trainingdataset)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/clfs/base.py", line 354, in train</font></div><div><font class="Apple-style-span" color="#FF0000"> result = self._train(dataset)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/clfs/meta.py", line 1058, in _train</font></div><div><font class="Apple-style-span" color="#FF0000"> self.__testdataset)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/featsel/rfe.py", line 268, in __call__</font></div><div><font class="Apple-style-span" color="#FF0000"> wdataset = wdataset.selectFeatures(selected_ids)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/datasets/mapped.py", line 130, in selectFeatures</font></div><div><font class="Apple-style-span" color="#FF0000"> sdata = Dataset.selectFeatures(self, ids=ids, sort=sort)</font></div><div><font class="Apple-style-span" color="#FF0000"> File "/opt/local/lib/python2.5/site-packages/mvpa/datasets/base.py", line 1018, in selectFeatures</font></div><div><font class="Apple-style-span" color="#FF0000"> new_data['samples'] = self._data['samples'][:, ids]</font></div><div><font class="Apple-style-span" color="#FF0000">IndexError: <b>index (2) out of range (0<=index<1) in dimension 1</b></font></div><div><br></div><div><br></div></div><div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="font-size: 12px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><div><div>~~~~~~~~~~~~~~~~~~~~~~~~~~</div><div><br></div><div>Mark Lescroart</div><div>(say it LESS-qua)</div><div><br></div><div>University of Southern California</div><div>Neuroscience Graduate Program</div><div>Image Understanding Lab</div><div>Email: <a href="mailto:mark.lescroart@usc.edu">mark.lescroart@usc.edu</a></div><div>Cell: (213) 447-0752</div></div></div></div></div></span> </div><br></div></body></html>