<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Marco,<div class=""><br class=""></div><div class="">looking at the code, the chi-square being run is a test of independence, and not goodness-of-fit.The actual confusion matrix is tested against the expected values were rows and column independent. In the case of balanced classes (i.e., marginal row count is equal to marginal column count for each row and column), the expected value will be a matrix with identical values (row marginal * column marginal / number of observations; or, as it is computed in the code, number of observations/number of cells).</div><div class=""><br class=""></div><div class="">Hope this helps,</div><div class="">Matteo<br class=""><div><blockquote type="cite" class=""><div class="">On May 11, 2017, at 08:53, marco tettamanti <<a href="mailto:mrctttmnt@gmail.com" class="">mrctttmnt@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="content-type" content="text/html; charset=utf-8" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<pre class="">Dear Yaroslav,
thank you for your reply.
I might be wrong in the specific case of MVPA, but I think the 1-dimension Goodness-of-fit test is appropriate in case
you have something like one dice and you are expecting each of the 6 sides to occur with equal frequencies.
The N x N confusion matrix rather reflects the case in which you can a variable with N classes (targets) and you
measure how frequent these classes distribute across the levels of a different variable (predictions). In such a case,
a 2-dimension Pearson's test seems more appropriate.
Best,
Marco
On Thu, 11 May 2017, marco tettamanti wrote:
><i class=""> Dear all,
</i>><i class=""> I apologize if this has been asked before, or else is too trivial.
</i>
><i class=""> I have been trying to understand how the the pymvpa2 toolbox calculates
</i>><i class=""> the chi-square test of a confusion matrix.
</i>
><i class=""> In a cross-validation (e.g., cvte.ca.stats), it seems that by default this
</i>><i class=""> is done by means of a one-dimensional Goodness-of-fit chi-square test with
</i>><i class=""> expected uniform frequency distribution.
</i>
><i class=""> I was wondering whether the bi-dimensional Pearson's chi square wouldn't
</i>><i class=""> be more appropriate, as it seems to me that this would more closely
</i>><i class=""> reflect the "predictions vs targets N x N" matrix structure.
</i>
Hi Marco,
might as well be -- I would need to read on/check... IIRC we were just
following instructions on chi-square test to be done on contingency
tables.
--
Yaroslav O. Halchenko
Center for Open Neuroscience <a href="http://centerforopenneuroscience.org/" class="">http://centerforopenneuroscience.org</a>
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: <a href="http://www.linkedin.com/in/yarik" class="">http://www.linkedin.com/in/yarik</a>
</pre>
<br class="">
<pre class="moz-signature" cols="80">--
Marco Tettamanti, Ph.D.
Nuclear Medicine Department & Division of Neuroscience
IRCCS San Raffaele Scientific Institute
Via Olgettina 58
I-20132 Milano, Italy
Phone ++39-02-26434888
Fax ++39-02-26434892
Email: <a class="moz-txt-link-abbreviated" href="mailto:tettamanti.marco@hsr.it">tettamanti.marco@hsr.it</a>
Skype: mtettamanti</pre>
</div>
_______________________________________________<br class="">Pkg-ExpPsy-PyMVPA mailing list<br class=""><a href="mailto:Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org" class="">Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org</a><br class="">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</div></blockquote></div><br class=""><div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">--</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">Matteo Visconti di Oleggio Castello<br class="">Ph.D. Candidate in Cognitive Neuroscience<br class="">Dartmouth College<br class=""><br class="">+1 (603) 646-8665<br class=""><a href="http://mvdoc.me" class="">mvdoc.me</a> || <a href="http://linkedin.com/in/matteovisconti" class="">linkedin.com/in/matteovisconti</a> || <a href="http://github.com/mvdoc" class="">github.com/mvdoc</a><br class=""></div></div>
</div>
<br class=""></div></body></html>