[Pkg-xen-devel] Bug#646987: /usr/bin/xinit: xinit fails with xkbcomp could not be invoked

[Brenda J. Butler]

The function that returns ENOMEM is actually copy_mm.  It returns
ENOMEM when dup_mm returns 0.


copy_mm receives clone_flags and task_struct * tsk.  I think tsk
is pointing to the new task (the process being forked).
copy_mm assigns some values into *tsk, then:

    tsk->mm = NULL;
    tsk->active_mm = NULL;

then we check if current process has ->mm; if not, return 0
(effectively copying the "no mm" status into the new task_struct, is
this true?).  Presumably 0 return is "success", because failure
returns are things like -ENOMEM.

If we haven't returned, then we check if clone_flags includes
CLONE_VM; if so, we set our mm = the current process' ->mm,
set some elements in ->mm and ->tsk and return.

If clone_flags does not include CLONE_VM, we try to set

    mm = dup_mm(tsk)

which presumably means we are trying to duplicate the
->mm of the task being created?!? and if that fails (0 return)
then we return with -ENOMEM.

So presumably the X server is trying to fork xkbcomp and
is coming to this bit of code - current->mm is not 0,
but clone_flags & CLONE_VM is false.  And I imagine that
tsk->mm is still 0 (as we are trying to create it in copy_mm,
is that true?) so copy_mm fails.

I need to know more about clone_flags and who sets it.

bjb