|
| News | Profile | Code | Photography | Looking Glass | Projects | System Statistics | Uncategorized |
Blog |
For about two weeks, I've been trying to troubleshoot some odd host-only networking issues with VirtualBox. It turned out to be a configuration-related bug in a file marked DO NOT EDIT THIS FILE, which, of course, I ultimately had to edit.
I previously had two VirtualBox VMs, dax (FreeBSD) and adria (Linux) connected together using intnet networking. The setup worked, but I wanted to convert the adria VM to an LXC instance since I didn't actually need full VM emulation. I spun up the LXC with the following in its configuration file:
# Networking lxc.network.type = veth lxc.network.name = eth0 lxc.network.link = lxcbr0 lxc.network.hwaddr = 00:02:bc:56:cc:99
Then, I put the following in /etc/network/interfaces on the host:
# lxcbr0: bridges vboxnet1/vtnet2 on dax to adria
# This cannot be auto because vboxnet1 does not exist on boot
#auto lxcbr0
iface lxcbr0 inet manual
bridge_ports vboxnet1
bridge_fd 0
bridge_maxwait 0
Since VirtualBox's bridging doesn't play nice directly with LXC veth interfaces (explained here), I decided to convert dax's interface to host-only networking (vboxnet1) and use Linux's native bridging on the host to connect the VM and LXC. I ended up with the following:
(excalibur:14:09:EDT)% ifconfig vboxnet1
vboxnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.57.1 netmask 255.255.255.0 broadcast 192.168.57.255
ether 0a:00:27:00:00:01 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 13 overruns 0 frame 0
TX packets 19796 bytes 29052799 (27.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
(excalibur:14:09:EDT)% ifconfig vethGEVFJP
vethGEVFJP: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::fca5:45ff:fee5:ecb2 prefixlen 64 scopeid 0x20<link>
ether fe:a5:45:e5:ec:b2 txqueuelen 1000 (Ethernet)
RX packets 15215 bytes 1209437 (1.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 19678 bytes 29042119 (27.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
(excalibur:14:09:EDT)% brctl show
bridge name bridge id STP enabled interfaces
lxcbr0 8000.0a0027000001 no vboxnet1
vethGEVFJP
(excalibur:14:09:EDT)% sudo lxc-ls -f
NAME STATE AUTOSTART GROUPS IPV4 IPV6
adria RUNNING 0 - 10.3.7.217 2620:6:2000:12e:202:bcff:fe56:cc99
(excalibur:14:10:EDT)%
Looks good, right? Wrong. Connectivity worked, but only in one direction, and it seemed to be stopping on vboxnet1. After lots of tcpdumps, I figured out the following.
LXC ---> VM (GOOD) VM ---> LXC (DROPPED)
More specifically:
LXC -- vethGEVFJP --> lxcbr0 -- vboxnet1 --> VM (GOOD)
VM -- vboxnet1 --> lxcbr0 -- vethGEVFJP --> LXC (DROPPED)
|
|---- DROPPED HERE
Here are all of the things I tried, but none worked:
It seemed that the problem was on the VirtualBox side between vboxnet1 and the actual VM (vtnet2 on FreeBSD). This was odd, because I also have vboxnet0 connected to the host and the configuration is identical:
(excalibur:14:18:EDT)% VBoxManage showvminfo dax|grep NIC|head -n3 NIC 1: MAC: 0002BC56CB39, Attachment: Bridged Interface 'eth0', Cable connected: on,\ Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps,\ Boot priority: 0, Promisc Policy: allow-all, Bandwidth group: none NIC 2: MAC: 0002BC56CB99, Attachment: Host-only Interface 'vboxnet0', Cable connected: on,\ Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps,\ Boot priority: 0, Promisc Policy: allow-all, Bandwidth group: none NIC 3: MAC: 0002BC56CB29, Attachment: Host-only Interface 'vboxnet1', Cable connected: on,\ Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps,\ Boot priority: 0, Promisc Policy: allow-all, Bandwidth group: none (excalibur:14:18:EDT)%
So, what's the problem, here? I started looking into configuration files. Here's what I found in ~/VirtualBox VMs/dax/dax.vbox:
<Network>
<Adapter slot="0" enabled="true" MACAddress="0002BC56CB39" cable="true" promiscuousModePolicy="AllowAll" type="virtio">
<DisabledModes>
<InternalNetwork name="intnet"/>
<NATNetwork name="NatNetwork"/>
</DisabledModes>
<BridgedInterface name="eth0"/>
</Adapter>
<Adapter slot="1" enabled="true" MACAddress="0002BC56CB99" cable="true" promiscuousModePolicy="AllowAll" type="virtio">
<DisabledModes>
<InternalNetwork name="intnet"/>
<NATNetwork name="NatNetwork"/>
</DisabledModes>
<HostOnlyInterface name="vboxnet0"/>
</Adapter>
<Adapter slot="2" enabled="true" MACAddress="0002BC56CB29" cable="true" promiscuousModePolicy="AllowAll" type="virtio">
<DisabledModes>
<BridgedInterface name="tap0"/>
<InternalNetwork name="intnet"/>
<NATNetwork name="NatNetwork"/>
</DisabledModes>
<HostOnlyInterface name="vboxnet1"/>
</Adapter>
<Adapter slot="3" cable="true" type="82540EM"/>
<Adapter slot="4" cable="true" type="82540EM"/>
<Adapter slot="5" cable="true" type="82540EM"/>
<Adapter slot="6" cable="true" type="82540EM"/>
<Adapter slot="7" cable="true" type="82540EM"/>
The only thing that was different between vboxnet0 and vboxnet1 was the BridgedInterface under the DisabledModes tag. It must have been added during my original try to get LXC connectivity working and then modified when I used the TAP device during troubleshooting. It shouldn't matter because it's disabled. Also, there's a big warning on the top of the file stating:
<!-- ** DO NOT EDIT THIS FILE. ** If you make changes to this file while any VirtualBox related application ** is running, your changes will be overwritten later, without taking effect. ** Use VBoxManage or the VirtualBox Manager GUI to make changes. -->
Well, I made changes to it and removed that one BridgedInterface line, but when the VM was stopped. Bingo, that fixed it. I now have bidirectional networking through the Linux bridge between my VM and LXC instance.
This smells like a bug. VirtualBox must erroneously apply some networking state when it reads through DisabledModes when it really shouldn't. As a result, I opened ticket 17022.
New comments are currently disabled for this entry.
     |
This HTML for this page was generated in 0.001 seconds. |