[pymvpa] Some RFE Questions

[Yaroslav Halchenko]

Let me start with:
>    Also, has anyone tried to get sensitivity maps as described in Hanson,
>    Halchenko 2008?

but there were nothing special -- just regular RFE on a binary
task ranking features based on their {w_i}^2 (or alternatively on
abs(w)).  The only difference in the flow was: because we had multiple
subjects we had not decided on the 'stopping' point per each subject,
but rather obtained RFE curves for all of them to find somewhat stable
number of features necessary to obtain mean min error.

>    For a multiclass problem, if an SVM is the classifier and the
>    SVMWeights are the sensitivity measure, are the weights for all the
>    child SVMs that must be trained get averaged together to produce a new
>    weight to rank features by?

yeap, as ad-hoc as that ;)

> Having used RFE on some multiclass
>    datasets, it's clear that classification performance is improving as
>    features are removed but am unsure what the criterion for removal
>    really means in the multi-class scenario.

well, you could describe it with just a generic sense of: "total
aggregate contribution to all classification tasks", so if you have
a feature which contributes significantly to all pair-wise
classifications, you would expect that its mean of the coefficients
(^2 or abs) would also be high in the ratings.

such approach could indeed be improved, e.g. you do not mean, but rather
rank features based on the max of w_i across all pair-wise
classifications, thus preserving features which are found to be
important at least in some binary classification.

or you could come up with other alternative schemes.  May be someone
aware of a systematic approach?

-- 
=------------------------------------------------------------------=
Keep in touch                                     www.onerussian.com
Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic