[Pkg-samba-maint] Bug#578503: /sbin/mount.cifs: Wierd CIFS packet fragmentation
Dave Fennell
dave at unluckypixie.com
Tue Apr 20 11:49:10 UTC 2010
Package: smbfs
Version: 2:3.2.5-4lenny9
Severity: normal
File: /sbin/mount.cifs
I'm trying to optimize my CIFS network traffic for jumbo frames and either I'm mis-using TCPDUMP or CIFS is doing something funky.
My MTU is 7000 (due to other device limits on the network) the actual machine can do 9000.
Here are the mount options in use:
uid=1000,gid=34,file_mode=0660,dir_mode=0770,user=guest,password=password,dom=DOMAIN,rsize=56000,wsize=56000
When I copy a large file (test file is about 250mb) this is what I get from TCPDUMP (just looking at the traffic FROM my client).
12:03:56.511357 IP (tos 0x0, ttl 64, id 30413, offset 0, flags [DF], proto TCP (6), length 48688) mysql.here.lan.41013 > readynas1.here.lan.microsoft-ds: .
230432486:230481122(48636) ack 221076 win 143 <nop,nop,timestamp 93748678 89470853>
12:03:56.511365 IP (tos 0x0, ttl 64, id 30420, offset 0, flags [DF], proto TCP (6), length 4732) mysql.here.lan.41013 > readynas1.here.lan.microsoft-ds: P
230481122:230485802(4680) ack 221076 win 143 <nop,nop,timestamp 93748678 89470853>
12:03:56.512915 IP (tos 0x0, ttl 64, id 30421, offset 0, flags [DF], proto TCP (6), length 48688) mysql.here.lan.41013 > readynas1.here.lan.microsoft-ds: .
230485802:230534438(48636) ack 221127 win 143 <nop,nop,timestamp 93748678 89470855>
12:03:56.512923 IP (tos 0x0, ttl 64, id 30428, offset 0, flags [DF], proto TCP (6), length 4732) mysql.here.lan.41013 > readynas1.here.lan.microsoft-ds: P
230534438:230539118(4680) ack 221127 win 143 <nop,nop,timestamp 93748678 89470855>
12:03:56.514478 IP (tos 0x0, ttl 64, id 30429, offset 0, flags [DF], proto TCP (6), length 48688) mysql.here.lan.41013 > readynas1.here.lan.microsoft-ds: .
230539118:230587754(48636) ack 221178 win 143 <nop,nop,timestamp 93748678 89470856>
12:03:56.514488 IP (tos 0x0, ttl 64, id 30436, offset 0, flags [DF], proto TCP (6), length 4732) mysql.here.lan.41013 > readynas1.here.lan.microsoft-ds: P
230587754:230592434(4680) ack 221178 win 143 <nop,nop,timestamp 93748678 89470856>
It does that repeatedly, large packet small packet, large packet small packet they add up to 53420 which is around the figure I entered.
So trying to optimise the fragmentation I asjust the buffer sizes ... I get exactly the same packet sizes for 55500, 55000, and for 54000
If I change the rsize and wsize options to this:
rsize=53000,wsize=53000
I get:
12:34:20.723729 IP (tos 0x0, ttl 64, id 7367, offset 0, flags [DF], proto TCP (6), length 48688) mysql.here.lan.42583 > readynas1.here.lan.microsoft-ds: .
163903334:163951970(48636) ack 170484 win 143 <nop,nop,timestamp 93931099 91295031>
12:34:20.723738 IP (tos 0x0, ttl 64, id 7374, offset 0, flags [DF], proto TCP (6), length 636) mysql.here.lan.42583 > readynas1.here.lan.microsoft-ds: P
163951970:163952554(584) ack 170484 win 143 <nop,nop,timestamp 93931099 91295031>
12:34:20.725203 IP (tos 0x0, ttl 64, id 7375, offset 0, flags [DF], proto TCP (6), length 48688) mysql.here.lan.42583 > readynas1.here.lan.microsoft-ds: .
163952554:164001190(48636) ack 170535 win 143 <nop,nop,timestamp 93931099 91295033>
12:34:20.725211 IP (tos 0x0, ttl 64, id 7382, offset 0, flags [DF], proto TCP (6), length 636) mysql.here.lan.42583 > readynas1.here.lan.microsoft-ds: P
164001190:164001774(584) ack 170535 win 143 <nop,nop,timestamp 93931099 91295033>
12:34:20.726919 IP (tos 0x0, ttl 64, id 7383, offset 0, flags [DF], proto TCP (6), length 48688) mysql.here.lan.42583 > readynas1.here.lan.microsoft-ds: .
164001774:164050410(48636) ack 170586 win 143 <nop,nop,timestamp 93931100 91295034>
12:34:20.726928 IP (tos 0x0, ttl 64, id 7390, offset 0, flags [DF], proto TCP (6), length 636) mysql.here.lan.42583 > readynas1.here.lan.microsoft-ds: P
164050410:164050994(584) ack 170586 win 143 <nop,nop,timestamp 93931100 91295034>
So it looks like it's close to optimal another small tweak and the litter fragment will go away? but no ... I get the same for 52000.
But for 51000 I get:
2:37:26.101798 IP (tos 0x0, ttl 64, id 24548, offset 0, flags [DF], proto TCP (6), length 34792) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: .
229316714:229351454(34740) ack 238263 win 143 <nop,nop,timestamp 93949637 91480406>
12:37:26.102380 IP (tos 0x0, ttl 64, id 24553, offset 0, flags [DF], proto TCP (6), length 13948) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: .
229351454:229365350(13896) ack 238263 win 143 <nop,nop,timestamp 93949637 91480406>
12:37:26.102630 IP (tos 0x0, ttl 64, id 24555, offset 0, flags [DF], proto TCP (6), length 636) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: P
229365350:229365934(584) ack 238263 win 143 <nop,nop,timestamp 93949637 91480407>
12:37:26.103304 IP (tos 0x0, ttl 64, id 24556, offset 0, flags [DF], proto TCP (6), length 34792) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: .
229365934:229400674(34740) ack 238314 win 143 <nop,nop,timestamp 93949637 91480407>
12:37:26.103881 IP (tos 0x0, ttl 64, id 24561, offset 0, flags [DF], proto TCP (6), length 13948) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: .
229400674:229414570(13896) ack 238314 win 143 <nop,nop,timestamp 93949637 91480408>
12:37:26.104119 IP (tos 0x0, ttl 64, id 24563, offset 0, flags [DF], proto TCP (6), length 636) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: P
229414570:229415154(584) ack 238314 win 143 <nop,nop,timestamp 93949637 91480408>
12:37:26.104800 IP (tos 0x0, ttl 64, id 24564, offset 0, flags [DF], proto TCP (6), length 34792) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: .
229415154:229449894(34740) ack 238365 win 143 <nop,nop,timestamp 93949637 91480409>
12:37:26.105378 IP (tos 0x0, ttl 64, id 24569, offset 0, flags [DF], proto TCP (6), length 13948) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: .
229449894:229463790(13896) ack 238365 win 143 <nop,nop,timestamp 93949637 91480409>
12:37:26.105616 IP (tos 0x0, ttl 64, id 24571, offset 0, flags [DF], proto TCP (6), length 636) mysql.here.lan.45347 > readynas1.here.lan.microsoft-ds: P
229463790:229464374(584) ack 238365 win 143 <nop,nop,timestamp 93949637 91480410>
Now it's getting really silly. Three different fragments repeating that just makes no sense at all.
Jumping a few bytes (I don't have all day)...
If I change the rsize and wsize options to this:
rsize=32768,wsize=32768
I get:
12:20:02.845407 IP (tos 0x0, ttl 64, id 55326, offset 0, flags [DF], proto TCP (6), length 27844) mysql.here.lan.49160 > readynas1.here.lan.microsoft-ds: .
208279482:208307274(27792) ack 324147 win 143 <nop,nop,timestamp 93845311 90437169>
12:20:02.845414 IP (tos 0x0, ttl 64, id 55330, offset 0, flags [DF], proto TCP (6), length 5096) mysql.here.lan.49160 > readynas1.here.lan.microsoft-ds: P
208307274:208312318(5044) ack 324147 win 143 <nop,nop,timestamp 93845311 90437169>
12:20:02.846528 IP (tos 0x0, ttl 64, id 55331, offset 0, flags [DF], proto TCP (6), length 27844) mysql.here.lan.49160 > readynas1.here.lan.microsoft-ds: .
208312318:208340110(27792) ack 324198 win 143 <nop,nop,timestamp 93845312 90437170>
12:20:02.846536 IP (tos 0x0, ttl 64, id 55335, offset 0, flags [DF], proto TCP (6), length 5096) mysql.here.lan.49160 > readynas1.here.lan.microsoft-ds: P
208340110:208345154(5044) ack 324198 win 143 <nop,nop,timestamp 93845312 90437170>
12:20:02.847664 IP (tos 0x0, ttl 64, id 55336, offset 0, flags [DF], proto TCP (6), length 27844) mysql.here.lan.49160 > readynas1.here.lan.microsoft-ds: .
208345154:208372946(27792) ack 324249 win 143 <nop,nop,timestamp 93845312 90437171>
12:20:02.847671 IP (tos 0x0, ttl 64, id 55340, offset 0, flags [DF], proto TCP (6), length 5096) mysql.here.lan.49160 > readynas1.here.lan.microsoft-ds: P
208372946:208377990(5044) ack 324249 win 143 <nop,nop,timestamp 93845312 90437171>
Still the same problem.
If I change the rsize and wsize options to this:
rsize=7000,wsize=7000
I get:
12:08:32.673994 IP (tos 0x0, ttl 64, id 62896, offset 0, flags [DF], proto TCP (6), length 4216) mysql.here.lan.53542 > readynas1.here.lan.microsoft-ds: P
220118102:220122266(4164) ack 2696616 win 143 <nop,nop,timestamp 93776294 89747010>
12:08:32.673492 IP (tos 0x0, ttl 64, id 62895, offset 0, flags [DF], proto TCP (6), length 4216) mysql.here.lan.53542 > readynas1.here.lan.microsoft-ds: P
220122266:220126430(4164) ack 2696667 win 143 <nop,nop,timestamp 93776294 89747010>
12:08:32.673994 IP (tos 0x0, ttl 64, id 62896, offset 0, flags [DF], proto TCP (6), length 4216) mysql.here.lan.53542 > readynas1.here.lan.microsoft-ds: P
220126430:220130594(4164) ack 2696718 win 143 <nop,nop,timestamp 93776294 89747011>
12:08:32.674492 IP (tos 0x0, ttl 64, id 62897, offset 0, flags [DF], proto TCP (6), length 4216) mysql.here.lan.53542 > readynas1.here.lan.microsoft-ds: P
220130594:220134758(4164) ack 2696769 win 143 <nop,nop,timestamp 93776294 89747011>
2:08:32.674492 IP (tos 0x0, ttl 64, id 62897, offset 0, flags [DF], proto TCP (6), length 4216) mysql.here.lan.53542 > readynas1.here.lan.microsoft-ds: P
220130594:220134758(4164) ack 2696769 win 143 <nop,nop,timestamp 93776294 89747011>
Nice consistant sized packets but much smaller than they could be so is this the most efficient setting?...
I have two questions.
1) Is it really sending massive packets? e.g. 48688 byte packet? That makes no sense, my MTU is 7000. I think this must be me reading TCPDUMP wrongly.
2) My real bug (which I would like fixed) would be that CIFS seems to not consider the MTU at all when looking at packet sizes. I have read that the internal
buffers are all based upon multiples of 4096. But surely it should consider the MTU of the interface it's listening on (there is only one in this instance
maybe if there is more than one it could make sure the buffers are a nice multiple of them all). Maybe the old behaviour is still optimal on 1500 MTU
but if the MTU is larger perhaps it could detect and make sure it fragments packets sensibly? So at least it is possible to get optimal network throughput.
Thanks.
Dave Fennell
-- System Information:
Debian Release: 5.0.4
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.33.2 (SMP w/2 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages smbfs depends on:
ii libc6 2.7-18lenny2 GNU C Library: Shared libraries
ii libcomerr2 1.41.3-1 common error description library
ii libkeyutils1 1.2-9 Linux Key Management Utilities (li
ii libkrb53 1.6.dfsg.4~beta1-5lenny2 MIT Kerberos runtime libraries
ii libldap-2.4-2 2.4.11-1+lenny1 OpenLDAP libraries
ii libpopt0 1.14-4 lib for parsing cmdline parameters
ii libtalloc1 1.2.0~git20080616-1 hierarchical pool based memory all
ii libwbclient0 2:3.2.5-4lenny9 client library for interfacing wit
ii netbase 4.34 Basic TCP/IP networking system
ii samba-common 2:3.2.5-4lenny9 Samba common files used by both th
smbfs recommends no packages.
Versions of packages smbfs suggests:
ii smbclient 2:3.2.5-4lenny9 a LanManager-like simple client fo
-- no debconf information
More information about the Pkg-samba-maint
mailing list