[pymvpa] multi-class major voting scheme paradox?

[Vadim Axel]

Fine, thanks.
So, then, the interpretation of mutli-class confusion can be misleading.
I have one more related question:
Does it make sense to interpret the confusion matrix off-diagonal values as
a tuning? For example, if my three classes are the circles of three
different size. For the row of the large circle I get 0.6 classified as
large (the correct one), 0.3 as a medium and 0.1 as a small. Can I say that
I get sort of tuning to circle size?

Thanks again.


On Fri, May 14, 2010 at 4:29 PM, <josef.pktd at gmail.com> wrote:

> On Fri, May 14, 2010 at 9:20 AM, Vadim Axel <axel.vadim at gmail.com> wrote:
> > Hi guys,
> >
> > I apply multi-class major voting scheme for three classes (all pairs
> > classification). I try to understand how the confusion matrix should look
> > like when two classes in a pair classification are not discriminated
> (chance
> > level). Consider pathological case where classes 1,2 and 2,3 are
> classified
> > with 100% and 1,3 are at chance level (50%). The confusion matrix I which
> > get looks like:
> > 0.584    0.083    0.333
> > 0    1    0
> > 0.327    0.071    0.602
> >
> > So, all of sudden it seems that classes 1 and 3 are discriminated. Isn't
> it
> > paradoxical?
> >
> > When I checked out how I get this result, I have found that it indeed
> makes
> > sense. Consider class 1 as a correct label:
> > pair 1: the classification of classes 1,2 always results in '1' (we are
> at
> > 100%, by definition)
> > pair 2: the classification of classes 1,3 results in half trials in '1'
> and
> > other half in '3' (we are at chance by definition).
> > pair 3: the classification of classes 2,3 results in half trials in '2'
> and
> > other in '3' (in case that classes are unrelated, the classifier should
> be
> > at chance here).
> >
> > The bottom line: since all (1) pairs and half (2) pairs results in '1', I
> am
> > already at 50% hit rate for correct class.
> >
> > What do you think about all this? Is there any flaw in my logic?
> > If someone is interested, I can send my matlab simulation.
>
> looks right to me if the tie-breaker is unbiased
>
> Probs if 1 is tru:
>
> >>> 0.5*0.5 + 0.5*0.5/3.   # 3 wins, by majority and tie-breaker
> 0.33333333333333331
> >>> 0.5*0.5/3.   # 2 wins, by tie-breaker
> 0.083333333333333329
> >>> 1-(0.5*0.5 + 0.5*0.5/3. + 0.5*0.5/3.)  # 1 wins
> 0.58333333333333337
>
> Josef
>
> >
> > Thanks for help,
> > Vadim
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Pkg-ExpPsy-PyMVPA mailing list
> > Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> > http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
> >
> >
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20100514/7ced7388/attachment.htm>