[pymvpa] Sensitivity analysis

[Yaroslav Halchenko]

Hi Roberto, sorry for the delays with replies -- our other baby,
NeuroDebian, urgently needing our attention ;)

>    1) In PyMVPA we have a sensitivity analyzer for each classifier which
>    gives us the importance of dataset features, in the form of a vector of
>    #feature values. These values indicates if a feature is enrolled in the
>    classification task, but not if a feature is more sensitive to a class
>    rather than others. Is there a procedure to understand this or I'm
>    misunderstanding sensitivity analysis?

well, in general it is correct, especially for binary classifiers.  Moreover,
those sensitivities for linear classifiers most often are just coefficients of
the separating hyperplane.  Thus they have no notion of 'class' but rather hint
on importance of that feature for discrimination between participating classes;
thus cannot be univocaly attributed to one or another class.  Depending on
preprocessing, and what actual data you give for classification, the sign
of such coefficient might be indicative of favoring higher values (higher
activation) for one class than another in a specific feature.

>    2) Do you know some papers/lectures/book chapter/ books where can I
>    learn how to understand classifier feature importance? not only in
>    neuroimaging analysis but in general.

Well, someone might recommend some nice overview on variable
importance/sensitivity estimations.  Otherwise there are short of too many
papers/methods.

As for SVMs you might like to have a look at

@Article{ Rakotomamonjy03,
    Author = "A. Rakotomamonjy",
    Title = "Variable Selection Using {SVM}-based Criteria",
    Journal = "Journal of Machine Learning Research",
    Volume = "3",
    Pages = "1357--1370",
    bibsource = "DBLP, http://dblp.uni-trier.de",
    ee = "http://www.jmlr.org/papers/v3/rakotomamonjy03a.html",
    year = 2003,
    keywords = "Support Vector Machines",
    url = "http://www.jmlr.org/papers/volume3/rakotomamonjy03a/rakotomamonjy03a.pdf"
}

which provides few approaches to features ranking in SVMs.

Also of interest might be

Kienzle, W., Franz, M. O., Schölkopf, B. & Wichmann, F. A. (In press). Center-surround patterns emerge as optimal predictors for human saccade targets. Journal of Vision.
    This paper offers an approach to make sense out of feature sensitivities of non-linear classifiers. 

Sato, J. R., Mourão-Miranda, J., Martin, M. d. G. M., Amaro, E., Morettin, P. A. & Brammer, M. J. (2008). The impact of functional connectivity changes on support vector machines mapping of fMRI data. Journal of Neuroscience Methods, 172, 94–104.
    Discussion of possible scenarios where univariate and multivariate (SVM) sensitivity maps derived from the same dataset could differ. Including the case were univariate methods would assign a substantially larger score to some features.
    DOI: http://dx.doi.org/10.1016/j.jneumeth.2008.04.008


-- 
=------------------------------------------------------------------=
Keep in touch                                     www.onerussian.com
Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic