[pymvpa] Mask in from_wizard generates no data.

[Thomas Nickson]

My main problem is that this isn't at all consistent with how the function
for constructing fmri datasets works. Which way do you consider to be
correct? I think the fmri way of handling it makes more sense. This way I
have to understand that the mask needs to be flattened and I need to
provide an array of nonzero indices. The other way does not work. No other
way works.  I can't just provide a flattened boolean mask, if by that you
mean 1's and 0's. I can't provide a mask of the same shape as the original
data. This is confusing because the original data will be automatically
flattened but the mask won't. Why does that make any sense? Surely you
would at least want the inputs to be consistent for the mask and the
original data? It's neither consistent within the function itself nor is it
consistent between other functions that do the same thing.

The fmri data provided for the tutorial does exactly the same thing. It
hasn't got anything to do with the data. I can provide the code.

Anyway, I don't mean to non-constructively criticise but I found it
confusing and frustrating and thought it might be nice to alleviate some of
that.

#Open the main VBM data - some functions to help

import numpy as np
import nibabel as nb
import os
from functools import partial
from operator import mul
import pickle as p
from mvpa2.datasets.base import Dataset
from mvpa2.mappers.zscore import zscore
from mvpa2.datasets.mri import fmri_dataset

"""4ddata is fddata and 2ddata is tddata"""

def getImages(imagePath):
    """load a nifti"""
    return nb.load(imagePath)

def writeDatatoImages(writePath, Data):
    """Write a numpy array to a nifti1 image"""
    N = nb.Nifti1Image(Data, np.eye(4))
    N.to_filename(writePath)

def getImageData(imagePath):
    """load a nifti as numpy data"""
    return getImages(imagePath).get_data()

def rotateShape(FDData):
    """we need to rotate the shape of the data as other wise the time
dimension (here denoting the subjects) is actually last"""
    if len(FDData.shape) == 4:
        FDData = np.rollaxis(FDData, -1)
    return FDData

def reshapeFlatten(FDData):
    """Reshape the data and then flatten each member of the array"""
    Data = rotateShape(FDData)
    return Data.reshape(Data.shape[0],reduce(mul,Data.shape[1:],1))

#======================================Helper functions
above=====================================

DATA_LOCATION = ./pymvpa/PyMVPA/tutorial_data/data/bold.nii.gz'

MASK = './pymvpa/PyMVPA/tutorial_data/data/mask_brain.nii.gz'



#Load the metadata pickle
#metaData = p.load(open('VBMMeta.pkl'))

#get image data and reshape
rotatedImageData = rotateShape(getImageData(DATA_LOCATION))

#Get mask data
mask = getImageData(MASK).flatten()#.nonzero()[0]#.flatten().nonzero()[0]
#mask = np.float32(mask)
#print type(mask)

#Load dataset
ds = Dataset.from_wizard(rotatedImageData,
mask=mask)#targets=metaData['diagnosesT1'], mask=mask)
#ds = fmri_dataset(rotatedImageData)#, targets=metaData['diagnosesT1'],
mask=mask)
#ds = fmri_dataset(DATA_LOCATION, mask=mask)#,
targets=metaData['diagnosesT1'], mask=mask, chunks=np.array(range(133)))

#zscore normalise the data
#zscore(ds, chunks_attr=None)

On Wed, Dec 10, 2014 at 3:38 PM, Yaroslav Halchenko <debian at onerussian.com>
wrote:

>
> On Wed, 10 Dec 2014, Thomas Nickson wrote:
>
> >    Ah, I see what I have done now, after I loaded the fmri dataset and
> >    checked it seems that the from_wizard function wants as input for the
> mask
> >    the set of indices from the flattened mask array and I was not doing
> this.
>
> can't be that the only way -- boolean mask must work too.
> we even have a unittest ;)
>
> @reseed_rng()
> def test_feature_masking():
>     mask = np.zeros((5, 3), dtype='bool')
>     mask[2, 1] = True
>     mask[4, 0] = True
>     data = Dataset.from_wizard(np.arange(60).reshape((4, 5, 3)),
>                                targets=1, chunks=1, mask=mask)
>
>     # check simple masking
>     ok_(data.nfeatures == 2)
>     ...
>
>
> >    I wonder if anyone knows if the numpy array function .flatten() is
> >    equivalent to how the data mapper handles the flattening or I should
> do
> >    this some other way?
>
> I believe so (test!) -- it doesn't use 'flatten' but 'reshape' though
>
> >    Also this function is confusingly constructed.
>
> I am just still confused on what is the  problem here since boolean masks
> should have worked.  Once again -- if you just shared ds/mask -- we could
> look
> and resolve the mystery.
>
> > Should I restructure it so
> >    that it just accepts nifti files? Or are people happy with it this
> way?
>
> why to limit applicability?
>
> --
> Yaroslav O. Halchenko, Ph.D.
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Research Scientist,            Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20141210/dca82e4b/attachment-0001.html>