<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Dear Matteo,<br>
thank you for your reply!<br>
This was also my understanding when I looked at the
mvpa2.clfs.transerror.chisquare function's help.<br>
However, the reality seems to be somewhat different, at least in my
hands.<br>
<br>
Given any confusion matrices as yielded by the pymvpa2 toolbox:<br>
      print cvte.ca.stats.matrix<br>
      [[22 19]<br>
      [26 29]]<br>
<br>
The calculated chi square:<br>
      print 'Chi^2: %.6f (p=%.6f)' % cvte.ca.stats.stats["CHI^2"]<br>
      Chi^2: 2.416667 (p=0.490540)<br>
<br>
Produces the same result as the 1-dimensional chisquare test:<br>
      from scipy.stats import chisquare<br>
      chisquare([22, 26, 19, 29], f_exp=[24, 24, 24, 24])<br>
      Power_divergenceResult(statistic=2.416666666666667,
pvalue=0.49053966890385647)<br>
<br>
      help(chisquare)<br>
      Help on function chisquare in module scipy.stats.stats:<br>
      chisquare(f_obs, f_exp=None, ddof=0, axis=0) Calculates a
one-way chi square test.<br>
<br>
And not as would be with the 2-dimensional independence test:<br>
      from scipy.stats import chi2_contingency<br>
      obs = np.array([[22, 26], [19, 29]])<br>
      g, p, dof, expctd = chi2_contingency(obs, correction=False)<br>
      print g, p, dof, expctd<br>
      0.383148558758 0.535922979308 1 [[ 20.5 27.5] [ 20.5Â
27.5]]<br>
 <br>
      help(chi2_contingency)<br>
      Help on function chi2_contingency in module
scipy.stats.contingency:<br>
      chi2_contingency(observed, correction=True, lambda_=None)<br>
      Chi-square test of independence of variables in a
contingency table.<br>
<br>
<br>
Am I doing something wrong?<br>
Thank you and best wishes,<br>
Marco<br>
<br>
<span style="white-space: pre;">> *Matteo Visconti di Oleggio Castello* matteo.visconti at gmail.com
> <a class="moz-txt-link-rfc2396E" href="mailto:pkg-exppsy-pymvpa%40lists.alioth.debian.org?Subject=Re%3A%20%5Bpymvpa%5D%20mvpa2.clfs.transerror.chisquare&In-Reply-To=%3CEAED4394-634F-4DED-A42F-99FFBCC9E7F5%40gmail.com%3E"><mailto:pkg-exppsy-pymvpa%40lists.alioth.debian.org?Subject=Re%3A%20%5Bpymvpa%5D%20mvpa2.clfs.transerror.chisquare&In-Reply-To=%3CEAED4394-634F-4DED-A42F-99FFBCC9E7F5%40gmail.com%3E></a>
>
> </span><br>
/Thu May 11 17:04:46 UTC 2017/<br>
<span style="white-space: pre;">> ------------------------- Hi Marco,
>
> looking at the code, the chi-square being run is a test of
> independence, and not goodness-of-fit.The actual confusion matrix is
> tested against the expected values were rows and column independent.
> In the case of balanced classes (i.e., marginal row count is equal to
> marginal column count for each row and column), the expected value
> will be a matrix with identical values (row marginal * column
> marginal / number of observations; or, as it is computed in the code,
> number of observations/number of cells).
>
> Hope this helps, Matteo
>> / On May 11, 2017, at 08:53, marco tettamanti <mrctttmnt at
>> gmail.com
>> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa"><http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa></a>>
>> wrote:
> />/ />/ Dear Yaroslav, />/ thank you for your reply. />/ I might be
> wrong in the specific case of MVPA, but I think the 1-dimension
> Goodness-of-fit test is appropriate in case />/ you have something
> like one dice and you are expecting each of the 6 sides to occur with
> equal frequencies. />/ The N x N confusion matrix rather reflects the
> case in which you can a variable with N classes (targets) and you />/
> measure how frequent these classes distribute across the levels of a
> different variable (predictions). In such a case, />/ a 2-dimension
> Pearson's test seems more appropriate. />/ />/ Best, />/ Marco />/
> />/ On Thu, 11 May 2017, marco tettamanti wrote: />/ />/ >Â Â Â Dear
> all, />/ >Â Â Â I apologize if this has been asked before, or else is
> too trivial. />/ />/ >Â Â Â I have been trying to understand how the
> the pymvpa2 toolbox calculates />/ >Â Â Â the chi-square test of a
> confusion matrix. />/ />/ >Â Â Â In a cross-validation (e.g.,
> cvte.ca.stats), it seems that by default this />/ >Â Â Â is done by
> means of a one-dimensional Goodness-of-fit chi-square test with />/ >
> expected uniform frequency distribution. />/ />/ >Â Â Â I was wondering
> whether the bi-dimensional Pearson's chi square wouldn't />/ >Â Â Â be
> more appropriate, as it seems to me that this would more closely />/
> >Â Â Â reflect the "predictions vs targets N x N" matrix structure. />/
>Â />/ Hi Marco, />/ />/ might as well be -- I would need to read
> on/check... IIRC we were just />/ following instructions on
> chi-square test to be done on contingency />/ tables. />/ />/ -- />/
> Yaroslav O. Halchenko />/ Center for Open Neuroscience
> <a class="moz-txt-link-freetext" href="http://centerforopenneuroscience.org">http://centerforopenneuroscience.org</a>
> <a class="moz-txt-link-rfc2396E" href="http://centerforopenneuroscience.org/"><http://centerforopenneuroscience.org/></a> />/ Dartmouth College, 419
> Moore Hall, Hinman Box 6207, Hanover, NH 03755 />/ Phone: +1 (603)
> 646-9834Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Fax: +1 (603) 646-1419 />/ WWW:
> <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/yarik">http://www.linkedin.com/in/yarik</a> <a class="moz-txt-link-rfc2396E" href="http://www.linkedin.com/in/yarik"><http://www.linkedin.com/in/yarik></a>
>Â />/ />/ -- />/ Marco Tettamanti, Ph.D. />/ Nuclear Medicine
> Department & Division of Neuroscience />/ IRCCS San Raffaele
> Scientific Institute />/ Via Olgettina 58 />/ I-20132 Milano, Italy
> />/ Phone ++39-02-26434888 />/ Fax ++39-02-26434892 />/ Email:
> tettamanti.marco at hsr.it
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa"><http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa></a>
> <a class="moz-txt-link-rfc2396E" href="mailto:tettamanti.marcoathsr.it"><mailto:tettamanti.marco at hsr.it
></a> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa"><http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa></a>>
>
> </span><br>
/>/ Skype: mtettamanti<br>
<span style="white-space: pre;">> />/ _______________________________________________ />/
> Pkg-ExpPsy-PyMVPA mailing list />/ Pkg-ExpPsy-PyMVPA at
> lists.alioth.debian.org
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa"><http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa></a>
>
> </span><br>
/>/
<a class="moz-txt-link-freetext" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a><br>
<span style="white-space: pre;">> / -- Matteo Visconti di Oleggio Castello Ph.D. Candidate in Cognitive
> Neuroscience Dartmouth College
>
> +1 (603) 646-8665 mvdoc.me || linkedin.com/in/matteovisconti ||
> github.com/mvdoc
>
> -------------- next part -------------- An HTML attachment was
> scrubbed... URL:
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20170511/8e8c7f55/attachment.html"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20170511/8e8c7f55/attachment.html></a>
>
> </span><br>
-------------------------<br>
<span style="white-space: pre;">>
> * Previous message: [pymvpa] mvpa2.clfs.transerror.chisquare
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/003618.html"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/003618.html></a>
>
> </span><br>
* Next message: [pymvpa] mvpa2.clfs.transerror.chisquare
<a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/003620.html"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/003620.html></a><br>
<span style="white-space: pre;">> * *Messages sorted by:* [ date ]
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/date.html#3619"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/date.html#3619></a>
> [ thread ]
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/thread.html#3619"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/thread.html#3619></a>
> [ subject ]
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/subject.html#3619"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/subject.html#3619></a>
> [ author ]
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/author.html#3619"><http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2017q2/author.html#3619></a>
>
>
> ------------------------- More information about the
> Pkg-ExpPsy-PyMVPA mailing list
> <a class="moz-txt-link-rfc2396E" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa"><http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa></a></span><br>
</body>
</html>