[Pkg-xen-devel] Recent hypervisor update on Debian Wheezy breaks domU networking

[Gavin]

Hi,

Firstly I apologise for the cross-post, however I don't expect to get as
quick a response from the package maintainers as I do from the Debian
community, and this issue affects a service that I've got scheduled to go
live at midnight this evening. :(

A recent update from xen-hypervisor-4.1-amd64 version 4.1.3-7, to version
4.1.3-8 on Debian Wheezy has caused all vm's on this host to not receive
their arp replies anymore and as such they cannot reach their gateways and
are now isolated from the network.

There was a more recent update as well (4.1.4-2) which I have now since
applied however this particular issue persists.

The arp replies are received by the host and passed all the way up to the
bridge (br200) being used by Xen, however they are not seen on the vif
(vif2.0) created for the particular vm.

If I statically add the arp entry to the vm all starts working, ie: vm is
no longer isolated and is now connected to the world, but we all know that
this is not an ideal workaround.

This was working perfectly before this update. :(

1) Please let me know if I should roll-back this particular xen update,
kernel and all, and what those steps may be, or if this is a known issue
with a particular workaround that I can apply.

2) Would moving to openvswitch be another possible workaround?

My config:-

Bonded ethernet connected to trunks on Cisco 3750 stack with connection as
follows:-

eth0 --> bond0
eth1 --> bond0 --> br200 --> vif2.0

/etc/network/interfaces:-

iface bond0 inet manual
        slaves eth0 eth1
        bond_mode 5
        bond-miimon 100
        bond-downdelay 200
        bond-updelay 200

auto br200
iface br200 inet static
        address 172.31.1.66
        gateway 172.31.1.65
        netmask 255.255.255.240
        bridge_ports bond0
        bridge_maxwait 0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off

root at scjhb01:/home/gavin# brctl show
bridge name bridge id STP enabled interfaces
br200 8000.d4bed9f309a1 no bond0
vif2.0

root at scjhb01:/home/gavin# tcpdump -i bond0 'arp'
tcpdump: WARNING: bond0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bond0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:26:00.287489 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:00.287524 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:00.287669 ARP, Reply 172.31.1.49 is-at 00:09:0f:09:21:0e (oui
Unknown), length 46
11:26:01.303484 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:01.303518 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:01.303674 ARP, Reply 172.31.1.49 is-at 00:09:0f:09:21:0e (oui
Unknown), length 46
11:26:02.303484 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:02.303518 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:02.303579 ARP, Reply 172.31.1.49 is-at 00:09:0f:09:21:0e (oui
Unknown), length 46


root at scjhb01:/home/gavin# tcpdump -i br200 'arp'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br200, link-type EN10MB (Ethernet), capture size 65535 bytes
11:26:15.367489 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:15.367514 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:15.367580 ARP, Reply 172.31.1.49 is-at 00:09:0f:09:21:0e (oui
Unknown), length 46
11:26:16.383476 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:16.383511 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:16.383592 ARP, Reply 172.31.1.49 is-at 00:09:0f:09:21:0e (oui
Unknown), length 46
11:26:17.383486 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:17.383520 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:17.383616 ARP, Reply 172.31.1.49 is-at 00:09:0f:09:21:0e (oui
Unknown), length 46


root at scjhb01:/home/gavin# tcpdump -i vif2.0 'arp'
tcpdump: WARNING: vif2.0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vif2.0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:26:31.463481 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:31.463521 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:32.463480 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:32.463521 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:33.463477 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:33.463515 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:34.479482 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:34.479523 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46
11:26:35.479478 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 28
11:26:35.479515 ARP, Request who-has 172.31.1.49 tell 172.31.1.50, length 46

Thanks and Regards.
Gavin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20130218/b7ab704f/attachment.html>