Sunday, September 29, 2019

Virtualization on FreeBSD Host and Oracle VirtualBox Guests with VDE Networking

Supratim Sanyal's Blog: Oracle VirtualBox Guest Networking using VDE / VDE2 (Virtual Distributed Ethernet)


I described the basic setup for VirtualBox virtualization on FreeBSD 12-RELEASE as a host in an earlier post. Here are some notes on setting up VDE (Virtual Distributed Ethernet) guest networking with Oracle VirtualBox hypervisor version 5.2.32 running on FreeBSD 12.0-RELEASE-p10.

vde2 User-mode virtual ethernet infrastructure was installed from FreeBSD binary package repository using "sudo pkg install vde2". Keep in mind vde2 should be installed, not the "vde" package which is also available.

VirtualBox was built from the FreeBSD ports sources using "cd /usr/ports/emulators/virtualbox-ose/ && make install clean" with VDE networking option selected from the configuration screens produced by the build script.

The following setup script vde-switch.sh is invoked at boot time from /etc/rc.local to set the VDE switch and tap interfaces up:


It is to be noted that on FreeBSD, rc.local should source /etc/rc.conf first, i.e. the following should appear first before invoking other startup scripts:
. /etc/rc.conf

Once the VDE switch is created, the Oracle VirtualBox virtual machines are configured to use the VDE switch. This is accomplished by a series of "vboxmanage modifyvm" commands, as in the example below which configured my IBM OS/2 Warp 4.5 for eBusiness virtual machine:

$ vboxmanage modifyvm OS2-Warp-4.5-Server --nic1 generic
$ vboxmanage modifyvm OS2-Warp-4.5-Server --nicgenericdrv1 VDE
$ vboxmanage modifyvm OS2-Warp-4.5-Server --nicproperty1 network=/tmp/vde-ip.ctl

To identify the VDE switch to be used for virtual machine networking, the "network=" specification in the last command above uses the same socket specified in the "-s" switch when setting up the VDE switch at boot time.

After configuring the VirtualBox virtual appliance for VDE using the commands, the VirtualBox GUI settings show the networking configuration as the screenshot at the top of this post.

It may be possible to use the GUI and choose the VDE switch-related parameters graphically, but I have not tried that personally.

VDE networking for VirtualBox appliances works flawlessly once configured. I have multiple Virtualization hosts and connect all the VirtualBox virtual machines across all of them to one logical Ethernet switch using VDE's Ethernet-over-SSL tunneling capability. A typical command line to set up a Ethernet-over-SSL tunnel to another VM host looks like:

/usr/local/bin/dpipe /usr/local/bin/vde_plug /tmp/vde-ip.ctl = ssh vde0@$REMOTE vde_plug /tmp/vde-ip.ctl

For the above to work, obviously password-less SSH login to the "vde0" account on the remote host specified by the REMOTE environment variable had to be set up.

I also usually enable VDE switch's fstp (Fast Spanning Tree Protocol) on all of my VDE switches in the hope of avoiding Ethernet packet flooding-related Network issues. It appears once in a while the VDE port that is the end-point of the SSL tunnel to the remote VDE switch is designated "root" sporadically, potentially impacting connectivity to other VDE switches. When this happens, I manually change the SSL tunnel source port to and edge port using the "fstp/setedge" command from unixterm (the control program for VDE switch). Here is a typical example of such a session, where port 3 had transitioned to a "root" port which I force back to edge port:

$ unixterm /tmp/vde-ip.mgmt 
VDE switch V.2.3.2
(C) Virtual Square Team (coord. R. Davoli) 2005,2006,2007 - GPLv2

vde$ port/allprint
0000 DATA END WITH '.'
Port 0001 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: NONE Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0007 module tuntap      : tap0
Port 0002 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: root Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0003 module unix prog   : vde_plug: user=root PID=1043
Port 0003 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: localuser Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0009 module unix prog   : vde_plug: user=localuser PID=1120
Port 0004 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: localuser Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0011 module unix prog   : VirtualBOX user=localuser PID=1747 SSH=10.100.0.123
Port 0005 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: localuser Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0013 module unix prog   : VirtualBOX user=localuser PID=1748 SSH=10.100.0.123
Port 0006 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: localuser Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0015 module unix prog   : VirtualBOX user=localuser PID=3725 SSH=10.100.0.123
Port 0007 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: localuser Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0017 module unix prog   : VirtualBOX user=localuser PID=1750 SSH=10.100.0.123
.
1000 Success

vde$ fstp/print
0000 DATA END WITH '.'
FST DATA VLAN 0000  
 ++ root 80:00:00:ff:4d:44:85:2d
 ++ designated 80:00:00:ff:4d:44:85:2d
 ++ rootport 0003 cost 20000000 age 1 bonusport 0000 bonuscost 0
 -- Port 0001 tagged=0 portcost=20000000 role=Designated
 -- Port 0002 tagged=0 portcost=20000000 role=Designated
 -- Port 0003 tagged=0 portcost=20000000 role=Root
 -- Port 0004 tagged=0 portcost=20000000 role=Designated
 -- Port 0005 tagged=0 portcost=20000000 role=Designated
 -- Port 0006 tagged=0 portcost=20000000 role=Designated
 -- Port 0007 tagged=0 portcost=20000000 role=Designated
.
1000 Success

vde$ fstp/setedge 0 3 1
1000 Success

vde$ fstp/print
0000 DATA END WITH '.'
FST DATA VLAN 0000  
 ++ root 80:00:00:ff:4d:44:85:2d
 ++ designated 80:00:00:ff:4d:44:85:2d
 ++ rootport 0003 cost 20000000 age 6 bonusport 0000 bonuscost 0
 -- Port 0001 tagged=0 portcost=20000000 role=Designated
 -- Port 0002 tagged=0 portcost=20000000 role=Designated
 -- Port 0003 tagged=0 portcost=20000000 role=Edge
 -- Port 0004 tagged=0 portcost=20000000 role=Designated
 -- Port 0005 tagged=0 portcost=20000000 role=Designated
 -- Port 0006 tagged=0 portcost=20000000 role=Designated
 -- Port 0007 tagged=0 portcost=20000000 role=Designated
.
1000 Success

vde$ logout


In the above example case, after setting the edge port, the remote VDE switch (the other end of the Ethernet-over-SSL tunnel) correctly show it's own Port 2 connected to Port 3 over SSH:

$ unixterm /tmp/vde-ip.mgmt 
VDE switch V.2.3.2
(C) Virtual Square Team (coord. R. Davoli) 2005,2006,2007 - GPLv2

vde$ port/allprint
0000 DATA END WITH '.'
Port 0001 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: NONE Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0007 module tuntap      : vde-ip-tap0
Port 0002 untagged_vlan=0000 ACTIVE - Unnamed Allocatable
 Current User: vde0 Access Control: (User: NONE - Group: NONE)
  -- endpoint ID 0003 module unix prog   : vde_plug: user=vde0 PID=6352 SSH=10.100.0.13
.
1000 Success

vde$ fstp/print
0000 DATA END WITH '.'
FST DATA VLAN 0000 ROOTSWITCH 
 ++ root 80:00:00:ff:4d:44:85:2d
 ++ designated ff:ff:ff:ff:ff:ff:ff:ff
 ++ rootport 0000 cost 0 age 1287834 bonusport 0000 bonuscost 0
 -- Port 0001 tagged=0 portcost=20000000 role=Designated
 -- Port 0002 tagged=0 portcost=20000000 role=Designated
.
1000 Success

vde$ logout



No comments:

Post a Comment

"SEO" link builders: move on, your spam link will not get posted.

Note: Only a member of this blog may post a comment.