The Open edX devstack expects to run on Vagrant with VirtualBox. With a few tweaks, you can use Vagrant with KVM instead.
The instructions here are specific to the Eucalyptus release. If you need Ficus, look here.
At Scampersand we SSH into a shared development server that provides a stable environment and regular backups without relying on individual hardware or configuration. That server runs Fedora and hosts docker containers and KVM virtual machines.
KVM and VirtualBox are mutually exclusive on Linux because they each supply a kernel module that handles the CPU virtualization instructions, so only one can be loaded at a time. Since we have an existing infrastructure based on KVM, we’d rather run the devstack on it than need to bring up a separate host for Virtualbox.
If you’d like to run the Open edX devstack on KVM, these are the steps you’ll need. I’ll explain these in detail below.
- Install Vagrant 1.8.6 or later
- Install the vagrant-libvirt plugin
- Enable user for vagrant with libvirt
- Mutate the devstack box
- Download the modified devstack Vagrantfile
- Fire it up!
Install Vagrant 1.8.6 or later
Vagrant prior to 1.8.6 has a bug that affects old Ubuntu boxes. The current release of Open edX—known as Eucalyptus—is based on Ubuntu 12.04 which is affected by this bug.
You can check what version of Vagrant you have:
$ vagrant -v Vagrant 1.8.6
If the version reported is older than 1.8.6 (or if you don’t have Vagrant already) go ahead and get the latest from vagrantup.com.
Also note that if you’re planning for more than one user to run devstack on the same host, you’ll actually need the version after 1.8.6 to fix a bug with NFS exports. Unfortunately that version isn’t released as I’m writing this, but if you’re impatient then you can find a workaround in my pull request. This isn’t necessary for single-user systems, so don’t sweat it in that case.
Install the vagrant-libvirt plugin
“Wait, libvirt? I thought we were using KVM.”
Right, so let’s step back and review how this works. Vagrant is a tool for
controlling a virtual machine. The configuration for the VM is described in
Vagrantfile, and Vagrant depends on a virtualization provider to do the heavy
lifting. Most of the time the provider is Virtualbox, but Vagrant can also use
VMware, Hyper-V, Docker and others.
KVM is a kernel-level virtualization technology that’s managed by libvirtd in Linux distributions. In fact there used to be a KVM-specific provider plugin for Vagrant, but that’s been deprecated in favor of the libvirt provider plugin.
So in order to use Vagrant with KVM, you need the vagrant-libvirt plugin. You can install it like this:
$ sudo dnf install libvirt-devel # Fedora $ sudo apt-get install libvirt-dev # Ubuntu $ vagrant plugin install vagrant-libvirt
This approach installs it
from rubygems.org, which gives you
the most recent version the author has released. The distributions also package
the plugin so you can install it with
dnf (Fedora) or
apt-get (Ubuntu), but
they’re usually a few versions behind.
Enable user for vagrant with libvirt
The libvirt daemon will need to know that you’re authorized to manage virtual machines. The easiest way to do this is to add yourself to the libvirt group:
$ sudo usermod -aG libvirt $USER
There’s no need to log out and back in, since libvirtd checks group membership directly rather than via process inheritance.
It’s a good idea to make sure that
virsh works properly before proceeding:
$ virsh --connect=qemu:///system sysinfo
If that command produces some XML output, you should be good to go. If it doesn’t, then you might need to investigate since vagrant-libvirt isn’t likely to work either.
Mutate the devstack box
The box image provided by edX is specific to to VirtualBox. Use the vagrant-mutate plugin to convert it for libvirt/KVM.
$ vagrant plugin install vagrant-mutate Installing the 'vagrant-mutate' plugin. This can take a few minutes... Installed the plugin 'vagrant-mutate (1.2.0)'!
$ vagrant box add --name eucalyptus-devstack-2016-09-01 --provider virtualbox \ http://files.edx.org/vagrant-images/eucalyptus-devstack-2016-09-01.box ==> box: Box file was not detected as metadata. Adding it directly... ==> box: Adding box 'eucalyptus-devstack-2016-09-01' (v0) for provider: virtualbox box: Downloading: http://files.edx.org/vagrant-images/eucalyptus-devstack-2016-09-01.box ==> box: Successfully added box 'eucalyptus-devstack-2016-09-01' (v0) for 'virtualbox'!
$ vagrant mutate eucalyptus-devstack-2016-09-01 libvirt Converting eucalyptus-devstack-2016-09-01 from virtualbox to libvirt. (100.00/100%) The box eucalyptus-devstack-2016-09-01 (libvirt) is now ready to use.
You can inspect what this does on the filesystem in your
$ tree ~/.vagrant.d/boxes /home/aron/.vagrant.d/boxes └── eucalyptus-devstack-2016-09-01 └── 0 ├── libvirt │ ├── Vagrantfile │ ├── box.img │ └── metadata.json └── virtualbox ├── Vagrantfile ├── box-disk1.vmdk ├── box.ovf ├── metadata.json └── vagrant_private_key
Notice that the
vagrant_private_key wasn’t included in the migration. While it
would be possible to copy this manually, the easier workaround is to set
config.ssh.password, and this is the approach already taken in the custom
Vagrantfile you’ll download next.
Download the modified devstack Vagrantfile
The edX-provided Vagrantfile assumes Virtualbox, so I’ve provided a modified Vagrantfile that supports libvirt/KVM. You can inspect the differences here.
You should download this into a new directory where the sources will also eventually be checked out.
$ mkdir devstack $ cd devstack $ curl -OL https://raw.githubusercontent.com/scampersand/configuration/scampersand/eucalyptus.libvirt-kvm/vagrant/release/devstack/Vagrantfile
Fire it up!
At this point you can start devstack. If you will have more than one user
running on the same host, then you should start it with
avoid listening on a bunch of localhost ports that will collide between users.
Instead you can access the VM by its own IP address as shown below.
$ VAGRANT_NO_PORTS=1 vagrant up --provider=libvirt Bringing machine 'default' up with 'libvirt' provider... ==> default: Creating image (snapshot of base box volume). ==> default: Creating domain with the following settings... ==> default: -- Name: devstack_default_1478018111_5f69c2128844670d27e2 ==> default: -- Domain type: kvm ==> default: -- Cpus: 2 ==> default: -- Memory: 4096M ==> default: -- Management MAC: ==> default: -- Loader: ==> default: -- Base box: eucalyptus-devstack-2016-09-01 ==> default: -- Storage pool: default ==> default: -- Image: /var/lib/libvirt/images/devstack_default_1478018111_5f69c2128844670d27e2.img (80G) ==> default: -- Volume Cache: default ==> default: -- Kernel: ==> default: -- Initrd: ==> default: -- Graphics Type: vnc ==> default: -- Graphics Port: 5900 ==> default: -- Graphics IP: 127.0.0.1 ==> default: -- Graphics Password: Not defined ==> default: -- Video Type: cirrus ==> default: -- Video VRAM: 9216 ==> default: -- Keymap: en-us ==> default: -- TPM Path: ==> default: -- INPUT: type=mouse, bus=ps2 ==> default: -- Command line : ==> default: Creating shared folders metadata... ==> default: Starting domain. ==> default: Waiting for domain to get an IP address... ==> default: Waiting for SSH to become available... default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... ==> default: Configuring and enabling network interfaces... ==> default: Exporting NFS shared folders... ==> default: Preparing to edit /etc/exports. Administrator privileges will be required... Redirecting to /bin/systemctl status nfs-server.service ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled) Drop-In: /run/systemd/generator/nfs-server.service.d └─order-with-mounts.conf Active: active (exited) since Thu 2016-09-15 17:33:21 EDT; 1 months 16 days ago Main PID: 1470 (code=exited, status=0/SUCCESS) Tasks: 0 (limit: 512) Memory: 0B CPU: 0 CGroup: /system.slice/nfs-server.service Sep 15 17:33:20 gargan.jupiter systemd: Starting NFS server and services... Sep 15 17:33:21 gargan.jupiter systemd: Started NFS server and services. ==> default: Mounting NFS shared folders... ==> default: Running provisioner: shell... default: Running: inline script ==> default: stdin: is not a tty ==> default: ==> default: PLAY [all] ******************************************************************** ...
When this finishes, you can start Studio or LMS just like the edX-provided instructions:
$ vagrant ssh vagrant@precise64:~$ sudo su edxapp edxapp@precise64:~/edx-platform$ paver devstack studio # or lms
If you ran with
VAGRANT_NO_PORTS=1 then you’ll need the IP address of the
$ vagrant ssh-config Host default HostName 192.168.121.155 User vagrant Port 22 UserKnownHostsFile /dev/null StrictHostKeyChecking no PasswordAuthentication no IdentityFile /home/aron/devstack/.vagrant/machines/default/libvirt/private_key IdentitiesOnly yes LogLevel FATAL
Use the IP address shown in the output construct the URL. For example, in the
output above, the LMS will be on
http://192.168.121.155:8000 and Studio is on
Note that this IP address is a private IPv4 address as defined by RFC 1918, so if you’re running on a server separate from your own workstation, you’ll need some additional network hookup. I’ll show you what I use in a following article.