[buildd-tools-devel] Bug#877710: Bug#877710: /usr/sbin/sbuild-createchroot: fails to create stretch arm64 schroot on unstable

Wed Oct 4 20:21:45 UTC 2017

On Wed, 04 Oct 2017 21:04:05 +0200
Johannes Schauer <josch at debian.org> wrote:

> Hi Neil,
> 
> Quoting Neil Williams (2017-10-04 19:27:43)
> > Setting up passwd (1:4.4-4.1) ...
> > duplicate group entry
> > delete line 'sbuild:x:121:neil'? No
> > group sbuild: no user neil
> > delete member 'neil'? No
> > duplicate group entry
> > delete line 'sbuild:x:121:neil'? No
> > group sbuild: no user neil
> > delete member 'neil'? No
> > grpck: no changes
> > Multiple entries named 'sbuild' in /etc/group. Please fix this with
> > pwck or grpck. grpconv: failed to prepare the new /etc/group entry
> > 'sbuild' qemu: uncaught target signal 11 (Segmentation fault) -
> > core dumped Segmentation fault  
> 
> This segfault also looks fishy...
> 
> > /bin/chown: cannot access '/etc/gshadow': No such file or directory
> > /bin/chmod: cannot access '/etc/gshadow': No such file or directory
> > Please correct the error and rerun `/sbin/shadowconfig on'
> > dpkg: error processing package passwd (--configure):
> >  subprocess installed post-installation script returned error exit
> > status 1 Setting up login (1:4.4-4.1) ...
> > dpkg: libuuid1:arm64: dependency problems, but configuring anyway
> > as you requested: libuuid1:arm64 depends on passwd; however:
> >   Package passwd is not configured yet.
> > 
> > The line:
> > Multiple entries named 'sbuild' in /etc/group. Please fix this with
> > pwck or grpck. would seem to indicate a problem within sbuild
> > rather than debootstrap.  
> 
> Your conclusion might be correct. sbuild-createchroot
> fills /etc/group by running "getent group sbuild" on the parent
> system.

I tried the recommended fix using grpck but that got into a whole new
world of trouble. apt had not been installed at this point and although
the .deb existed in /var/cache/apt/archives inside the chroot,
installing it (with manually identified dependencies) didn't result in
a usable system due to some other problem with dpkg.

> The following output would be interesting to debug this
> further:
> 
>  - what you get when you run "getent group sbuild" yourself

neil at sylvester:~$ getent group sbuild
sbuild:x:121:neil

> 
>  - the actual content of /etc/group inside the chroot

# cat /etc/group
sbuild:x:121:neil
sbuild:x:121:neil
root:*:0:
daemon:*:1:
bin:*:2:
sys:*:3:
adm:*:4:
tty:*:5:
disk:*:6:
lp:*:7:
mail:*:8:
news:*:9:
uucp:*:10:
man:*:12:
proxy:*:13:
kmem:*:15:
dialout:*:20:
fax:*:21:
voice:*:22:
cdrom:*:24:
floppy:*:25:
tape:*:26:
sudo:*:27:
audio:*:29:
dip:*:30:
www-data:*:33:
backup:*:34:
operator:*:37:
list:*:38:
irc:*:39:
src:*:40:
gnats:*:41:
shadow:*:42:
utmp:*:43:
video:*:44:
sasl:*:45:
plugdev:*:46:
staff:*:50:
games:*:60:
users:*:100:
nogroup:*:65534:

> Might it also be possible that chroot directory was not empty before
> you executed sbuild-createchroot?

It was definitely empty. It was a new directory and after
investigating, I removed it and re-ran the same command before filing
the bug report.

> I'm just puzzled that this problem occurs when you use qemu-static and
> --foreign. It's a weird effect...

The difference with --foreign is that the build has to halt to allow
the bin format handler to be added, then the second stage handled
manually. sbuild could borrow from other debootstrap wrappers and
support a --bin-fmt option which takes the path to qemu-$ARCH-static
and copies it into the chroot itself.

Note the presence of the sbuild user at the top of /etc/group with the
bug. With a native schroot (freshly created using the same version
using a tmp suffix), /etc/group contains:

# cat /etc/group
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:
tty:x:5:
disk:x:6:
lp:x:7:
mail:x:8:
news:x:9:
uucp:x:10:
man:x:12:
proxy:x:13:
kmem:x:15:
dialout:x:20:
fax:x:21:
voice:x:22:
cdrom:x:24:
floppy:x:25:
tape:x:26:
sudo:x:27:
audio:x:29:
dip:x:30:
www-data:x:33:
backup:x:34:
operator:x:37:
list:x:38:
irc:x:39:
src:x:40:
gnats:x:41:
shadow:x:42:
utmp:x:43:
video:x:44:
sasl:x:45:
plugdev:x:46:
staff:x:50:
games:x:60:
users:x:100:
nogroup:x:65534:
sbuild:x:121:neil

Is this related to sbuild appending to what is initially an empty file?

Also, when using --foreign, sbuild-createchroot isn't particularly helpful when it stops. I tried to replicate the bug with an armhf stretch chroot:

W: The selected architecture and the current architecture do not match
W: (armhf versus amd64).
I: You probably need to add a personality option (see schroot(1)).
I: You may want to report your use case to the sbuild developers so that
I: the appropriate option gets automatically added in the future.

I: sudo chroot configuration linked as /etc/sbuild/chroot/stretch-armhf-sbuild.
/usr/sbin/chroot: failed to run command ‘/bin/sh’: No such file or directory
/usr/sbin/chroot: failed to run command ‘/bin/sh’: No such file or directory
E: Failed to create build directory /build
Failed to set up chroot
E: Error creating chroot session: skipping apt update
I: Successfully set up stretch chroot.

First, if --foreign is supplied, sbuild-createchroot should already
know that chroot will fail without a bin format handler and it hasn't
asked for one, so why does it try? Then when it does try and fails, it
then emits the final message: Successfully set up chroot.

This needs fixing at the same time - this is not a successfully set up
chroot, this is a chroot which currently needs manual intervention
(which is mentioned in the manpage but should also be in the message).

Also exiting zero in this situation is not helpful:
neil at sylvester:~$ echo $?
0

sbuild-createchroot can Recommends: qemu-static (or alternatively
Suggests:) and fail early (as soon as it parses --foreign has been
used) if a bin format handler isn't specified or the one specified
isn't found.

$ sudo cp /usr/bin/qemu-arm-static /srv/chroot/armhf-tmp/usr/bin/
neil at sylvester:~$ sudo chroot /srv/chroot/armhf-tmp/
I have no name!@sylvester:/# /debootstrap/debootstrap --second-stage

This doesn't seem to be arch dependent, the armhf schroot fails in
exactly the same way:

I: Configuring passwd...
I: Configuring libuuid1:armhf...
I: Configuring mount...
I: Configuring libfdisk1:armhf...
I: Configuring e2fsprogs...
I: Configuring libblkid1:armhf...
I: Configuring sysvinit-utils...
I: Configuring libmount1:armhf...
I: Configuring util-linux...
I: Configuring libc-bin...
W: Failure while configuring required packages.
W: See //debootstrap/debootstrap.log for details (possibly the package passwd is at fault)

Setting up passwd (1:4.4-4.1) ...
duplicate group entry
delete line 'sbuild:x:121:neil'? No
group sbuild: no user neil
delete member 'neil'? No
duplicate group entry
delete line 'sbuild:x:121:neil'? No
group sbuild: no user neil
delete member 'neil'? No
grpck: no changes
Multiple entries named 'sbuild' in /etc/group. Please fix this with pwck or grpck.
grpconv: failed to prepare the new /etc/group entry 'sbuild'
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
/bin/chown: cannot access '/etc/gshadow': No such file or directory
/bin/chmod: cannot access '/etc/gshadow': No such file or directory
Please correct the error and rerun `/sbin/shadowconfig on'

I already have a jessie armhf schroot, so this is new behaviour. Sadly,
I don't know of a way to determine which version of sbuild created the
chroot and I don't build these often.

vmdebootstrap has some useful options for this, including the ability
to have "slow build tests" which actually run test bootstraps and check
things like exit values and the presence of certain files in the
chroot. The test language itself is interpreter agnostic, it will work
as well with perl scripts as it does with python. A local mirror is
advisable when running the slow build tests.

-- 

Neil Williams
=============
http://www.linux.codehelp.co.uk/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20171004/7d65f898/attachment-0001.sig>