Tag Archives: ruby

Vagrant and Ruby: Creating multiple VMs programmatically

Vagrant is an excellent tool to quickly spin up a (Linux) VM to test all kinds of things: a new distribution, a configuration management tool, a new version of a software, you name it.

But Vagrant can do more: you can describe multiple VMs and their configuration in a single Vagrantfile. I’ve been using Vagrant for many years to develop various applications and test and verify deployments, and being able to spin up a small, but otherwise complete cluster is a godsend.

Manually defining multiple VMs

A simple example might look like this (directly stolen from the documentation):

Vagrant.configure("2") do |config|
config.vm.provision "shell", inline: "echo Hello"

config.vm.define "web" do |web|
web.vm.box = "apache"

config.vm.define "db" do |db|
db.vm.box = "mysql"

This Vagrantfile creates two VMs: one for MySQL, and another for Apache. This works well if the VMs you’d like to create are quite different from each other, and you need to configure them in significantly different ways.

Creating multiple similar VMs, and a problem

But sometimes, the cluster you’re setting up consists of multiple VMs that are very similar, or even identical, in configuration. I am working on trying out k3s together with Rook, MetalLB, and a current version of Traefik. For a minimal cluster with k3s and Rook Ceph, three nodes are needed. The first node, the k3s master, has a lot of configuration, but the additional nodes basically only need k3s installed; all configuration will be handled by k3s. So a naive approach could look like this:

Vagrant.configure("2") do |config|
config.vm.define "master" do |master|
master.vm.box = "debian/buster"
master.vm.network "private_network", ip: ""
 master.vm.provision "shell", path: "provision-master.sh"

for i in 1..2
config.vm.define "node-#{i}" do |node|
node.vm.box = "debian/buster"
node.vm.network "private_network", ip: "192.168.33.#{10+i}"
node.vm.provision "shell", path: "provision-node.sh", args: [ i.to_s ]

The first VM is the master, the second and third are created identically with a for loop. In both blocks, a second network interface is added as a VirtualBox Host-Only network, so the cluster nodes can communicate with each other, and the host can access each node directly. The provision-node.sh will also receive the index variable so it can adjust names and objects to the node that is currently being provisioned.

When you vagrant up this configuration, you will be surprised to see that both node-1 and node-2 will received the IP address, and provision-node.sh will be called with 2 in both cases.

What’s happening?

To understand why Vagrant is misreading our intentions, we have to dive into Ruby a little bit. Vagrant is implemented in Ruby, and the Vagrantfile is in fact a Ruby program. The line

config.vm.define "node-#{i}" do |node|

defines a Ruby block (everything from do to the following end line) that is passed to the define method of the vm object of the config object. The block parameter |node| is provided by the define method when the block is executed. Inside the block, you can access properties of the node object, such as the vm object, and set values or call methods, to create the configuration that you need.

Also, the block has access to the variables that are defined in the main program. That’s why we can use Ruby string interpolation to format the node name ("node-#{i}") or the IP address ("192.168.33.#{10+i}") by using the loop variable i. So if this is all as it should be, why has the loop variable 2 as a value in both loops? When we look at the output of vagrant status, we see three VMs, each with it’s own unique name:

Current machine states:
master                    not created (virtualbox)
node-1                    not created (virtualbox)
node-2                    not created (virtualbox)

The reason the loop variable has the same value inside the block for both iterations is when the block is executed. You can be forgiven thinking that the code should execute right where it appears in the code: inside the loop, when config.vm.define is called. However, that is only when it is defined. Vagrant will run the Vagrantfile program and construct an internal configuration from it, collecting the blocks for various properties. Only when the complete definition of the configuration has been collected, these blocks are executed and their results taken into consideration.

By the time the blocks for each of the two nodes are executed, the for loop has completed, and the loop variable simply has the last value. Since the variable is only resolved when the block is executed (not when it is defined), the value is 3 for both versions of the blocks. This is really surprising for many people not familiar with this Ruby idiom and the way it is often used. While other languages have the capability to store closures for later use, I have not seen this pattern there, only in Ruby-based applications.

Creating a configuration object

So, how can we overcome this? Since the loop variable is referenced in both instances of the block definition, but doesn’t have the right value by the time the code finally is executed, we somehow need to make sure we have a different variable for each round. One way to achieve this is to define our own config class, and create a config object for each iteration that stores all the info that we’re interested in, and can define the block in such a way that only the config class instance is referenced.

Let’s look at the example:

class Cfg
def initialize(i)
@name = "node-#{i}"
@ip = "192.168.33.#{10 + i}"

def configure(config)
config.vm.define @name, primary: @primary do |v|
v.vm.hostname = @name
v.vm.network "private_network", ip: @ip

Vagrant.configure("2") do |config|
# ...

for i in 1..2
cfg = Cfg.new(i)

First, we define a Ruby class Cfg that receives two properties, name and ip, when the instances is created (the initialize method). Then, the configure method runs the Vagrant code and uses the instance properties. Finally, in the for loop, we create a new instance of the Cfg class for each iteration, and call each instances configuration method.

Further reading

The code in this post is simplified. If you would like to see the real-world code that stands up a k3s cluster with additional components, head over to github.com/stblassitude/k3s-rook.

OpenStack: fixing “No socketless chef-zero server”

I’ve started playing with various incarnations of “in a box” OpenStack setups to familiarize myself with OpenStack, among them DevStack and Ubuntu OpenStack. The OpenStack Chef Repo looks promising to me, as it gives you a good starting point, but can easily be extended to more complex lab setups, or even full productions environments.

The only problem? This error:

$ git clone https://github.com/openstack/openstack-chef-repo
$ export REPO_OS=centos7.1
$ cd openstack-chef-repo
$ chef exec rake berks_vendor
$ chef exec rake aio_neutron
chef exec chef-client --force-formatter -z vagrant_linux.rb aio-neutron.rb
[2015-08-09T23:41:58+02:00] INFO: Started chef-zero at chefzero://localhost:8889 with repository at /Users/stb/working/xxx/openstack-chef-repo
  One version per cookbook

[2015-08-09T23:41:58+02:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.3.0
[2015-08-10T00:02:19+02:00] INFO: Converging controller because 'converge true' is set ...
[2015-08-10T00:02:19+02:00] INFO: Port forwarded: local URL chefzero://localhost:8889 is available to as chefzero://localhost:8889 for the duration of this SSH connection.
[2015-08-10T00:02:19+02:00] INFO: Executing sudo chef-client -l info on vagrant@

    [controller] [2015-08-09T22:02:18+00:00] INFO: Forking chef instance to converge...
                 Starting Chef Client, version 12.4.1
                 [2015-08-09T22:02:18+00:00] INFO: *** Chef 12.4.1 ***
                 [2015-08-09T22:02:18+00:00] INFO: Chef-client pid: 13637
                 Chef encountered an error attempting to load the node data for "controller"
                 Unexpected Error:
                 ChefZero::ServerNotFound: No socketless chef-zero server on given port 8889
                 Running handlers:
                 [2015-08-09T22:02:20+00:00] ERROR: Running exception handlers
                 Running handlers complete
                 [2015-08-09T22:02:20+00:00] ERROR: Exception handlers complete
                 Chef Client failed. 0 resources updated in 2.194757362 seconds
                 [2015-08-09T22:02:20+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
                 [2015-08-09T22:02:20+00:00] ERROR: No socketless chef-zero server on given port 8889
                 [2015-08-09T22:02:20+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[2015-08-10T00:02:23+02:00] INFO: Completed chef-client -l info on vagrant@ exit status 1

I’m far from an expert in Chef, so it took me a bit of searching and experimenting. Luckily, the solution is rather simple:

$ chef gem list -d '^chef-provisioning$'

*** LOCAL GEMS ***

chef-provisioning (1.1.1)
    Author: John Keiser
    Homepage: http://github.com/chef/chef-provisioning/README.md
    Installed at: /opt/chefdk/embedded/lib/ruby/gems/2.1.0

    A library for creating machines and infrastructures idempotently in
$ chef gem update chef-provisioning
Updating installed gems
Updating chef-provisioning
Fetching: chef-provisioning-1.3.0.gem (100%)
WARNING:  You don't have /Users/stb/.chefdk/gem/ruby/2.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed chef-provisioning-1.3.0
Gems updated: chef-provisioning
$ chef gem list -d '^chef-provisioning$'

*** LOCAL GEMS ***

chef-provisioning (1.3.0, 1.1.1)
    Author: John Keiser
    Homepage: http://github.com/chef/chef-provisioning/README.md
    Installed at (1.3.0): /Users/stb/.chefdk/gem/ruby/2.1.0
                 (1.1.1): /opt/chefdk/embedded/lib/ruby/gems/2.1.0

    A library for creating machines and infrastructures idempotently in

The next version of ChefDK will likely come with this updated version of chef-provisioning, but until then, this should do the trick.