Bug#489608: mdadm: hung task timeout when doign data check
Alex Samad
alex at samad.com.au
Mon Jul 7 00:57:44 UTC 2008
Package: mdadm
Version: 2.6.4-2
Severity: normal
Hi
I have started to received these errors in dmesg when the monthly array
check happens
[569708.399818] md: data-check of RAID array md0
[569708.399823] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[569708.399826] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for data-check.
[569708.399831] md: using 128k window, over a total of 979840 blocks.
[569708.411831] md: delaying data-check of md1 until md0 has finished
(they share one or more physical units)
[569708.414774] md: delaying data-check of md2 until md0 has finished
(they share one or more physical units)
[566995.265070] md: delaying data-check of md3 until md2 has finished
(they share one or more physical units)
[569708.414984] md: delaying data-check of md1 until md3 has finished
(they share one or more physical units)
[569708.414997] md: delaying data-check of md2 until md0 has finished
(they share one or more physical units)
[566995.265119] md: delaying data-check of md3 until md2 has finished
(they share one or more physical units)
[567006.347885] md: md0: data-check done.
[569721.828687] md: delaying data-check of md1 until md3 has finished
(they share one or more physical units)
[567006.369737] md: delaying data-check of md3 until md2 has finished
(they share one or more physical units)
[569721.828687] md: data-check of RAID array md2
[569721.828687] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[569721.828687] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for data-check.
[569721.828687] md: using 128k window, over a total of 1959808 blocks.
[567006.405012] RAID1 conf printout:
[567006.405012] --- wd:2 rd:2
[567006.405012] disk 0, wo:0, o:1, dev:sdb1
[567006.405012] disk 1, wo:0, o:1, dev:sda1
[567027.013594] md: md2: data-check done.
[569749.143037] md: delaying data-check of md1 until md3 has finished
(they share one or more physical units)
[567027.032374] md: data-check of RAID array md3
[567027.032378] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[567027.032381] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for data-check.
[567027.032386] md: using 128k window, over a total of 964052544 blocks.
[569749.205917] RAID1 conf printout:
[569749.205917] --- wd:2 rd:2
[569749.205917] disk 0, wo:0, o:1, dev:sdb3
[569749.205917] disk 1, wo:0, o:1, dev:sda3
[580719.308717] INFO: task md1_resync:30573 blocked for more than 120
seconds.
[580719.308726] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[580719.308729] md1_resync D ffffffff80587a00 0 30573 2
[580719.308736] ffff81005136fdb0 0000000000000046 ffffffff8815e800
ffffffff802293e3
[580719.308747] ffff81004d502e40 ffff8100529e29c0 ffff81004d5030b0
0000000100000001
[580719.308757] 0000000000000282 0000000000000003 ffff81005136fdb0
ffffffff8022be74
[580719.308766] Call Trace:
[580719.308808] [<ffffffff802293e3>] __wake_up_common+0x41/0x74
[580719.308820] [<ffffffff8022be74>] __wake_up+0x38/0x4e
[580719.308843] [<ffffffff881504f8>] :md_mod:md_do_sync+0x21d/0x92c
[580719.308850] [<ffffffff80229959>] update_curr+0x44/0x6c
[580719.308858] [<ffffffff80229de8>] dequeue_entity+0x1a/0x83
[580719.308865] [<ffffffff80229d22>] __dequeue_entity+0x25/0x68
[580719.308874] [<ffffffff8020a7e2>] __switch_to+0x8d/0x2e6
[580719.308880] [<ffffffff802301f7>] hrtick_set+0x88/0xf7
[580719.308887] [<ffffffff802469f4>] autoremove_wake_function+0x0/0x2e
[580719.308909] [<ffffffff88152ffd>] :md_mod:md_thread+0xd7/0xed
[580719.308928] [<ffffffff88152f26>] :md_mod:md_thread+0x0/0xed
[580719.308932] [<ffffffff802468cb>] kthread+0x47/0x74
[580719.308939] [<ffffffff8020cdf8>] child_rip+0xa/0x12
[580719.308958] [<ffffffff80246884>] kthread+0x0/0x74
[580719.308963] [<ffffffff8020cdee>] child_rip+0x0/0x12
[580719.308968]
-- Package-specific info:
--- mount output
/dev/md1 on / type ext3 (rw,errors=remount-ro,acl)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
none on /selinux type selinuxfs (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
/dev/md0 on /boot type ext3 (rw)
/dev/mapper/raid0-tmp on /tmp type ext3 (rw,noexec,nosuid,nodev,noatime,nodiratime)
/dev/mapper/raid0-varlog on /var/log type ext3 (rw,noatime,nodiratime,acl)
/dev/mapper/raid1-home on /home type ext3 (rw,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-alexhome on /home/alex type ext3 (rw,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-anastazia on /home/anastazia type ext3 (rw,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-kathrinhome on /home/kathrin type ext3 (rw,sync,noatime,nodiratime,acl)
/home/alex/Maildir on /home/alex/Maildir type none (rw,bind)
/home/kathrin/Maildir on /home/kathrin/Maildir type none (rw,bind)
/dev/mapper/raid1-bblxfw01 on /home/bblxfw01 type ext3 (rw,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid0-squid on /var/cache/squid type ext3 (rw,noexec,nosuid,nodev,noatime,nodiratime,acl)
/dev/mapper/raid1-apt--cacher on /var/cache/apt-cacher type ext3 (rw,noexec,nosuid,nodev,noatime,nodiratime,acl)
/dev/mapper/raid0-apt on /var/cache/apt type ext3 (rw,noexec,nosuid,nodev,noatime,nodiratime,acl)
/dev/mapper/raid1-music on /exports/music type ext3 (rw,noexec,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-repository on /exports/repository type ext3 (rw,noexec,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-pictures on /exports/Pictures type ext3 (rw,noexec,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-cvs on /exports/cvs type ext3 (rw,noexec,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-webdav on /exports/webdav type ext3 (rw,noexec,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-doc on /usr/share/doc type ext3 (rw,noexec,nosuid,nodev,noatime,nodiratime,acl)
/dev/mapper/raid1-usrlocal on /usr/local type ext3 (rw,nodev,noatime,nodiratime,acl)
/dev/mapper/raid1-www on /var/www type ext3 (rw,noexec,nosuid,nodev,sync,noatime,nodiratime,acl)
/dev/mapper/raid1-spoolexim on /var/spool/exim4 type ext3 (rw,noexec,nosuid,nodev,sync,acl)
/dev/mapper/raid1-musicVideo on /exports/musicVideo type xfs (rw,noexec,nosuid,nodev,sync,noatime,nodiratime)
/dev/mapper/raid1-backups on /backups type ext3 (rw,nosuid,nodev,sync,acl)
/dev/mapper/raid1-chroot on /chroot type ext3 (rw,acl)
proc on /chroot/etch-ia32/proc type proc (rw)
/tmp on /chroot/etch-ia32/tmp type none (rw,bind)
/home on /chroot/etch-ia32/home type none (rw,bind)
/home/alex on /chroot/etch-ia32/home/alex type none (rw,bind)
/exports/downloads on /chroot/etch-ia32/exports/downloads type none (rw,bind)
/exports/video on /chroot/etch-ia32/exports/video type none (rw,bind)
/exports/music on /chroot/etch-ia32/exports/music type none (rw,bind)
/exports/shared on /chroot/etch-ia32/exports/shared type none (rw,bind)
/exports/Pictures on /chroot/etch-ia32/exports/Pictures type none (rw,bind)
nas.hme1.samad.com.au:/exports/video on /exports/video type nfs (rw,nosuid,nodev,sync,noatime,rsize=32768,wsize=32768,addr=192.168.11.14)
nas.hme1.samad.com.au:/exports/shared on /exports/shared type nfs (rw,nosuid,nodev,sync,noatime,rsize=32768,wsize=32768,addr=192.168.11.14)
nas.hme1.samad.com.au:/exports/downloads on /exports/downloads type nfs (rw,nosuid,nodev,sync,noatime,rsize=32768,wsize=32768,addr=192.168.11.14)
nfsd on /proc/fs/nfsd type nfsd (rw)
nas.hme1.samad.com.au:/exports/VideoCamera on /exports/VideoCamera type nfs (rw,nosuid,nodev,sync,noatime,rsize=32768,wsize=32768,addr=192.168.11.14)
--- mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#
# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions
# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes
# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR root
# definitions of existing MD arrays
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=04698cd3:a02af09f:069cb42f:03f72d27
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=ba8c4627:6e74a6c4:1a2e6c15:22feafcf
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=6afd4ab6:ce0ef994:2f0e35eb:2949c9ef
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=5782df0f:22160d41:2f0e35eb:2949c9ef
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=04698cd3:a02af09f:069cb42f:03f72d27
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=ba8c4627:6e74a6c4:1a2e6c15:22feafcf
#ARRAY /dev/md2 level=raid5 num-devices=3 UUID=98c007c6:3f751b9b:96bc634e:8ff321a3
--- /proc/mdstat:
Personalities : [raid1]
md3 : active raid1 sdb4[0] sda4[1]
964052544 blocks [2/2] [UU]
md2 : active raid1 sdb3[0] sda3[1]
1959808 blocks [2/2] [UU]
md0 : active raid1 sdb1[0] sda1[1]
979840 blocks [2/2] [UU]
md1 : active raid1 sdb2[0] sda2[1]
9767424 blocks [2/2] [UU]
unused devices: <none>
--- /proc/partitions:
major minor #blocks name
1 0 65536 ram0
1 1 65536 ram1
1 2 65536 ram2
1 3 65536 ram3
1 4 65536 ram4
1 5 65536 ram5
1 6 65536 ram6
1 7 65536 ram7
1 8 65536 ram8
1 9 65536 ram9
1 10 65536 ram10
1 11 65536 ram11
1 12 65536 ram12
1 13 65536 ram13
1 14 65536 ram14
1 15 65536 ram15
8 0 976762584 sda
8 1 979933 sda1
8 2 9767520 sda2
8 3 1959930 sda3
8 4 964052617 sda4
8 16 976762584 sdb
8 17 979933 sdb1
8 18 9767520 sdb2
8 19 1959930 sdb3
8 20 964052617 sdb4
8 32 488386584 sdc
8 33 488384001 sdc1
9 1 9767424 md1
9 0 979840 md0
9 2 1959808 md2
9 3 964052544 md3
253 0 20971520 dm-0
253 1 1048576 dm-1
253 2 20971520 dm-2
253 3 5242880 dm-3
253 4 29097984 dm-4
253 5 20971520 dm-5
253 6 89128960 dm-6
253 7 5242880 dm-7
253 8 73400320 dm-8
253 9 102400 dm-9
253 10 1048576 dm-10
253 11 5242880 dm-11
253 12 5242880 dm-12
253 13 53248 dm-13
253 14 3145728 dm-14
253 15 2621440 dm-15
253 16 10485760 dm-16
253 17 512000 dm-17
253 18 2097152 dm-18
253 19 53248 dm-19
253 20 62914560 dm-20
253 21 41943040 dm-21
253 22 1048576 dm-22
--- initrd.img-2.6.25-2-amd64:
39429 blocks
etc/mdadm
etc/mdadm/mdadm.conf
lib/modules/2.6.25-2-amd64/kernel/drivers/md/dm-mod.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/dm-snapshot.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/dm-mirror.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/md-mod.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/linear.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/multipath.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/raid0.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/raid1.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/raid456.ko
lib/modules/2.6.25-2-amd64/kernel/drivers/md/raid10.ko
sbin/mdadm
scripts/local-top/mdadm
--- /proc/modules:
dm_mirror 31748 0 - Live 0xffffffff88183000
dm_snapshot 22472 0 - Live 0xffffffff8817c000
dm_mod 68536 49 truecrypt,dm_mirror,dm_snapshot, Live 0xffffffff8816a000
raid1 28032 4 - Live 0xffffffff88162000
md_mod 87204 5 raid1, Live 0xffffffff8814b000
--- volume detail:
--- /proc/cmdline
root=/dev/md1 ro console=tty0 selinux=1
--- grub:
kernel /vmlinuz root=/dev/md1 ro console=tty0 selinux=0
kernel /vmlinuz root=/dev/md1 ro console=tty0 selinux=0 single
kernel /vmlinuz.old root=/dev/md1 ro console=tty0 selinux=0
kernel /vmlinuz.old root=/dev/md1 ro console=tty0 selinux=0 single
kernel /vmlinuz-2.6.25-2-amd64 root=/dev/md1 ro console=tty0 selinux=0
kernel /vmlinuz-2.6.25-2-amd64 root=/dev/md1 ro console=tty0 selinux=0 single
kernel /vmlinuz-2.6.24-1-amd64 root=/dev/md1 ro console=tty0 selinux=0
kernel /vmlinuz-2.6.24-1-amd64 root=/dev/md1 ro console=tty0 selinux=0 single
-- System Information:
Debian Release: lenny/sid
APT prefers testing
APT policy: (500, 'testing'), (500, 'stable'), (100, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.25-2-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_AU, LC_CTYPE=en_AU (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/dash
Versions of packages mdadm depends on:
ii debconf 1.5.22 Debian configuration management sy
ii libc6 2.7-12 GNU C Library: Shared libraries
ii lsb-base 3.2-12 Linux Standard Base 3.2 init scrip
ii makedev 2.3.1-88 creates device files in /dev
ii udev 0.114-2 /dev/ and hotplug management daemo
Versions of packages mdadm recommends:
ii exim4-daemon-heavy [mail-tran 4.69-5+b1 Exim MTA (v4) daemon with extended
ii module-init-tools 3.4-1 tools for managing Linux kernel mo
-- debconf information:
* mdadm/autostart: true
* mdadm/initrdstart: /dev/md1
mdadm/initrdstart_notinconf: false
mdadm/initrdstart_msg_errexist:
mdadm/initrdstart_msg_intro:
mdadm/initrdstart_msg_errblock:
* mdadm/warning:
* mdadm/start_daemon: true
* mdadm/mail_to: root
mdadm/initrdstart_msg_errmd:
mdadm/initrdstart_msg_errconf:
mdadm/autocheck: true
More information about the pkg-mdadm-devel
mailing list