[pymvpa] multi-class major voting scheme paradox?

[josef.pktd at gmail.com]

On Fri, May 14, 2010 at 9:20 AM, Vadim Axel <axel.vadim at gmail.com> wrote:
> Hi guys,
>
> I apply multi-class major voting scheme for three classes (all pairs
> classification). I try to understand how the confusion matrix should look
> like when two classes in a pair classification are not discriminated (chance
> level). Consider pathological case where classes 1,2 and 2,3 are classified
> with 100% and 1,3 are at chance level (50%). The confusion matrix I which
> get looks like:
> 0.584    0.083    0.333
> 0    1    0
> 0.327    0.071    0.602
>
> So, all of sudden it seems that classes 1 and 3 are discriminated. Isn't it
> paradoxical?
>
> When I checked out how I get this result, I have found that it indeed makes
> sense. Consider class 1 as a correct label:
> pair 1: the classification of classes 1,2 always results in '1' (we are at
> 100%, by definition)
> pair 2: the classification of classes 1,3 results in half trials in '1' and
> other half in '3' (we are at chance by definition).
> pair 3: the classification of classes 2,3 results in half trials in '2' and
> other in '3' (in case that classes are unrelated, the classifier should be
> at chance here).
>
> The bottom line: since all (1) pairs and half (2) pairs results in '1', I am
> already at 50% hit rate for correct class.
>
> What do you think about all this? Is there any flaw in my logic?
> If someone is interested, I can send my matlab simulation.

looks right to me if the tie-breaker is unbiased

Probs if 1 is tru:

>>> 0.5*0.5 + 0.5*0.5/3.   # 3 wins, by majority and tie-breaker
0.33333333333333331
>>> 0.5*0.5/3.   # 2 wins, by tie-breaker
0.083333333333333329
>>> 1-(0.5*0.5 + 0.5*0.5/3. + 0.5*0.5/3.)  # 1 wins
0.58333333333333337

Josef

>
> Thanks for help,
> Vadim
>
>
>
>
>
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
>
>