[pymvpa] sphere_gnbsearchlight & Monte Carlo Testing

[wolf zinke]

Hi,
> Meanwhile, on your data, could you confirm that there is some variance
> in the obtained performances in a "good/lengthy" code, i.e. that values
>
> np.std(distr_est.ca.dist_samples.samples[0], axis=1)
Thanks for the reply. Indeed, using a sphere_searchlight with the Monte 
Carlo Testing, this gives me zeros only.

Hope this helps,
wolf

On 10/23/2012 10:08 PM, Yaroslav Halchenko wrote:
> Thanks for the patience... unfortunately for now (after staring at the
> code for a while in the evening and then having Aha moment outside) I
> just disabled (it would throw an exception) providing any partitioners
> which would change the targets in gnb or m1nn adhoc-searchlights.  The
> reason is simple - they were coded with an assumption of targets
> being "constant" across partitioning schemes and that only partitions
> would change...  That is why you were getting the same values -- since
> all values in the dist_samples were the same as the original
> classification performance.
>
> I will try to get into fixing it "properly" later today/tomorrow.
>
> Meanwhile, on your data, could you confirm that there is some variance
> in the obtained performances in a "good/lengthy" code, i.e. that values
>
> np.std(distr_est.ca.dist_samples.samples[0], axis=1)
>
> are not all 0s (or nearby)... either my testing dataset is too small or there
> might be yet another bug hiding ;-)
>
> On Mon, 22 Oct 2012, wolf zinke wrote:
>
>> Hi Yaroslav,
>> I hope you did survive the SfN well. If you have any ideas regarding
>> the constant value I get with my Monte Carlo testing of the
>> gnb_searchlight, I would be very happy. SO far I dod not try any
>> further to solve this issue, but tried a common Searchlight in
>> combination with Monte Carlo testing. And this takes an awful time
>> to get some results...
>> cheers,
>> wolf
>
>> On 10/11/2012 06:23 PM, Yaroslav Halchenko wrote:
>>> On Thu, 11 Oct 2012, wolf zinke wrote:
>>>>     Hi,
>>>>     With some great input from Michael I figured out now, that contrary to the
>>>>     common sphere_searchlight, the sphere_gnbsearchlight already implements a
>>> sorry for being silent since I guess blame should be on me here:
>>> sphere_gnbsearchlight and sphere_m1nnsearchlight indeed somewhat of a
>>> crafty beasts to achieve super-fast performance with those "naive"
>>> classification schemes -- they do splitting internally BUT should be
>>> parametrized with a partitioner to be instructed on how to "split".
>>> I will try looking in details of your case later on whenever rush of
>>> preparation for sfn goes down... sorry about the delay
>>> cheers
>
>> _______________________________________________
>> Pkg-ExpPsy-PyMVPA mailing list
>> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
>> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>