<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Thank you for the quick reply Yaroslav, I substituted in GNB at your suggestion and it didn't seem to perform quite as well as the linearCSVMC method I was using without class weights. What kind of data have you been using it for? Perhaps it isn't so well
suited to EEG data</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Yaroslav Halchenko <yoh@onerussian.com><br>
<b>Sent:</b> 27 April 2016 21:41:52<br>
<b>To:</b> Development and support of PyMVPA; Ben McCartney<br>
<b>Subject:</b> Re: [pymvpa] Class weighting in unbalanced EEG data</font>
<div> </div>
</div>
<div>Quick one for now<br>
Did you try GNB? It has few weighting schemes and laplacian prior iirc was working for me quite well in a dataset with ratio of samples 100 to 1.<br>
<br>
<div class="gmail_quote">On April 27, 2016 4:07:03 PM EDT, Ben McCartney <bmccartney06@qub.ac.uk> wrote:
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif">
<p><span id="ms-rterangepaste-start"></span></p>
<p class="MsoNormal">Hello,</p>
<p></p>
<p class="MsoNormal"></p>
<p> </p>
<p class="MsoNormal">I am running an EEG experiment which involves presenting one of two images on the screen and recording the participant's neural activation. Right now I'm writing a script to take these recordings and attempt to decode from the EEG data
which image has been displayed. I have had success on similar scripts making use of PyMVPA's PLR, however in these cases the classes are balanced whereas in the experiment I'm trying to do now the ratio is closer to 1:4. The initial approach I had in mind
was to use asymmetrical error weights to penalize misclassification of the less numerous class more than the dominant class. I understand that there are various other approaches to handling unbalanced classes, such as over/undersampling or using a classifier
more robust in this regard such as random forests, but this seemed to me to be the most clean way to start.</p>
<p></p>
<p class="MsoNormal"></p>
<p> </p>
<p class="MsoNormal">I tried the class weight parameter in SKLearn as per this tutorial http://scikit-learn.org/stable/auto_examples/svm/plot_separating_hyperplane_unbalanced.html, and this seemed to work well for the data here, but failed to make any difference
when I used my EEG data. Upon increasing the number of dimensions in the sample data (changing the 2 in n_samples_1, 2), I noticed the difference between the predictions of the classifier with weights and the classifier without weights diminish. By the time
I had reached 7 features per training sample the predictions were the same. Does this parameter just not work well with many dimensions? My EEG data has around 200 trials (samples) and each sample has 180 datapoints in each of 20 electrode channels, so my
training data is shaped 200 * 3600. </p>
<p></p>
<p class="MsoNormal"></p>
<p> </p>
<p class="MsoNormal">I thought the weight parameter here was exactly what I was looking for http://www.pymvpa.org/generated/mvpa2.clfs.svm.LinearCSVMC.html#mvpa2.clfs.svm.LinearCSVMC, however when I tried to use it even with toy data I saw no difference, similar
to the issue in https://www.mail-archive.com/pkg-exppsy-pymvpa@lists.alioth.debian.org/msg02918.html. Some excerpts from my script follow, am I just overlooking something?</p>
<p></p>
<p class="MsoNormal"></p>
<p> </p>
<p class="MsoNormal">Thanks,</p>
<p></p>
<p class="MsoNormal">Ben</p>
<p></p>
<p class="MsoNormal"></p>
<p> </p>
<p class="MsoNormal">wclf = LinearCSVMC(weight=[2,8], weight_label=[True, False])</p>
<p></p>
<p class="MsoNormal">wanovaSelectedClassifier = M.FeatureSelectionClassifier(</p>
<p></p>
<p class="MsoNormal"> wclf,</p>
<p></p>
<p class="MsoNormal"> M.SensitivityBasedFeatureSelection(</p>
<p></p>
<p class="MsoNormal"> M.OneWayAnova(),</p>
<p></p>
<p class="MsoNormal"> M.FixedNElementTailSelector(numFeatures, mode='select', tail='upper')</p>
<p></p>
<p class="MsoNormal"> )</p>
<p></p>
<p class="MsoNormal">)</p>
<p></p>
<p class="MsoNormal">wfoldwiseCvedAnovaSelectedSVM = M.CrossValidation(</p>
<p></p>
<p class="MsoNormal"> wanovaSelectedClassifier,</p>
<p></p>
<p class="MsoNormal"> M.NFoldPartitioner(),</p>
<p></p>
<p class="MsoNormal"> enable_ca=['samples_error','stats', 'calling_time','confusion']</p>
<p></p>
<p class="MsoNormal">)</p>
<p></p>
<p class="MsoNormal">dataset = M.dataset_wizard(samples=X, targets=y, chunks=np.mod(np.arange(0,len(X)),numFolds))</p>
<p></p>
<p class="MsoNormal">wresults = wfoldwiseCvedAnovaSelectedSVM(dataset)</p>
<p></p>
<p class="MsoNormal">print wfoldwiseCvedAnovaSelectedSVM.ca.stats.as_string()</p>
<p></p>
<span id="ms-rterangepaste-end"></span><br>
<p></p>
</div>
<p style="margin-top:2.5em; margin-bottom:1em; border-bottom:1px solid #000"></p>
<pre class="k9mail"><hr><br>Pkg-ExpPsy-PyMVPA mailing list<br>Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org<br><a href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa" id="LPlnk508375">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a></pre>
</blockquote>
</div>
<br>
-- <br>
Sent from a phone which beats iPhone.</div>
</div>
</body>
</html>