[pymvpa] New Elastic Net classifier

Per B. Sederberg persed at princeton.edu
Wed Feb 4 03:52:03 UTC 2009


Hi Folks:

I just wanted to let you know that I just added in a new
regression-based classifier to PyMVPA that I think is going to be very
effective: the elastic net (ENET).  It's a variant on least angle
regression (LARS), but it has both L1 and L2 priors so that it
sparsifies the features AND keeps redundant features, which helps with
both generalizability and with sensitivity analysis.

As with LARS, I simply wrapped the R code written by the authors of
this manuscript:

    Zou and Hastie (2005) 'Regularization and Variable Selection via
    the Elastic Net' Journal of the Royal Statistical Society, Series
    B, 67, 301-320.

So you have to install the elasticnet code in R and be sure to have
RPy installed and running (see the doc string).

The basic approach has been applied to neural data by some folks here
at Princeton and talked about in a recent NeuroImage paper:

http://www.ncbi.nlm.nih.gov/pubmed/18793733

I've been running some tests and I've had very good success by setting
a max_steps to around 500 or 1000, which means you'll be left with
that many features, and using a lambda (the lm variable) of 1.0, which
will let it bring in redundant features.  Sometimes I run an
OneWayAnova first to get it down to around 15000 features (you also
could take a percentage of good features) so that it runs faster:

clfr = FeatureSelectionClassifier(
    ENET(lm=1.0,max_steps=500,trace=True),
    SensitivityBasedFeatureSelection(
       OneWayAnova(),
       FixedNElementTailSelector(15000,mode='select',tail='upper')),
    descr="ENET on 15K best(ANOVA) features")

Anyway, try it out and write in with any successes or failures.  So
far I think it's out-performing any other classifier I've tried on the
Haxby dataset (I have not been very systematic.)  Finally, although I
have included minimal tests of everything, I have not plotted the
feature weights, yet, so there may be issues there because that part
of the code has not been verified.

Best,
Per



More information about the Pkg-ExpPsy-PyMVPA mailing list