Berkshelf & Application Cookbooks

We’ve been successfully using Chef, along with Vagrant, for quite a while now and it has pretty much changed my life. If you’re not already using Vagrant you’re doing it wrong. (Probably.)

Chef has a fantastic set of community cookbooks - a set of open source, versioned and community maintained cookbooks covering most major server applications/tools from nginx to node. At the time of writing there are about 1,000 available to pick up and use to get you up and productive in no time.

Bookshelves at Russell Books

When we first started out with Chef our workflow could be summarised as:

  • git clone a community cookbook
  • copy it into myapp/cookbooks
  • edit it to meet our application’s needs
  • commit it to my applications git repo

This was (and is) a pretty naive approach. It worked for sure but by disconnecting our copy of the cookbook from the community we lost the ability to bring in the latest fixes and features easily whilst preserving our application specific changes. I guess we could’ve forked the community cookbook, made changes in our fork and then used a git submodule to hook it into our application - but then my application is split across 1 + (1*cookbooks) repos which is a mess.

Berkshelf

Berkshelf is basically npm for cookbooks. You add a Berksfile to your project that defines the cookbooks (and versions) your project requires, where to get them from (community by default), run berks install and hey presto your cookbooks (and their dependencies) are all downloaded and version managed for you. Let’s look at a basic example: getting a vanilla nginx server up and running with Vagrant.

$ mkdir berkshelf-demo && cd berkshelf-demo$ touch Berksfile && touch Vagrantfile

Berksfile:

Vagrantfile:

$ berks install$ vagrant up

We now have an nginx server and up running on http://192.168.60.10 - so what just happened?

In our Berksfile we told Berkshelf that we wanted to use version >= 1.7.0 && < 1.8.0 (ie. pessimistic) of the nginx cookbook as defined at the opscode community site. When we ran berks install, Berkshelf downloaded the nginx cookbook and all it’s dependencies into ~/.berkshelf (in a similar manner to ~/.npm).

In our Vagrant file we defined a precise64 box that we wanted to be mounted on a private IP address locally. We enabled the Berkshelf plugin that hijacks Vagrant’s cookbooks_path variable (along with a few others) to allow it to provide Vagrant with the source of the cookbooks. We then defined the recipes we wanted to use to provision our VM. In a nutshell; run apt-get update, then install nginx via apt-get.

As an interesting side note, the inline shell script that is run before provisioning via the recipes ensures our VM is running v11.x of Chef. The precise64 box I’m using is still running v10.x and quite a few of the latest community cookbooks are now aligned to v11.x which introduced breaking changes.

So with 16 lines of code and 2 commands we’re up and running with a vanilla install of nginx, we don’t have to commit the cookbooks to our git repo and we can easily pick-up the latest version of the cookbook as it becomes available. But who wants vanila? How do we tweak the cookbook to meet our applications requirements without breaking this newfound cookbook management workflow?

Application Cookbooks

Application cookbooks (aka wrapper cookbooks) is an emerging pattern for applying application specific overrides to a community cookbook cleanly. I gleaned most of what follows from a bunch of great posts, which are summarised at the bottom of this post.

Your application cookbook basically wraps a community cookbook (by depending on it and thus creating a dependency chain that Berkshelf can follow) which only includes the overrides that you want to apply. Let’s adapt our example above to change some of the vanilla options and behaviours of the community cookbook:

$ vagrant destroy -f$ mkdir cookbooks && cd cookbooks$ berks cookbook nginx-app$ touch nginx-app/templates/default/default-site.erb

cookbooks/nginx-app/metadata.rb:

cookbooks/nginx-app/recipes/default.rb:

cookbooks/nginx-app/templates/default/default-site.erb:

Berksfile:

Vagrantfile:

$ cd ..$ vagrant up

Once more we’ve got a version of nginx up and running, but this time gzip is turned off (via an attribute override) and it’s on port 8080 (via a template override). In other words - we’ve customised our nginx install without touching the base community cookbook. So what just happened this time?

We used Berkshelf to create a skeleton application cookbook called nginx-app within our applications cookbooks directory. Crucially we made our application cookbook depend on nginx by specifying it in metadata.rb (this creates the dependency chain Berkshelf can follow). Note that our application cookbook itself is also versioned (it’s just a cookbook after all) which will allow us to control which version is used to provision our various environments down the line should we wish.

The default recipe of our application cookbook applies our overrides to the gzip attribute and the default-site template, pointing it instead to the template within our cookbook which we can customise to our hearts content. In this case we simply changed the port from 80 to 8080, but you get the idea.

Finally we updated our Berksfile and Vagrant file to use our application cookbook instead of the vanila nginx cookbook directly. Note again in our Berksfile we can specify the version of our application cookbook we want to use.

Our cookbooks directory (which only contains our application cookbooks), along with our Berksfile and Vagrantfile can now be committed to our git repo since they all only contain configuration relevant to our application. We have complete control over the version of the community cookbook we’re using and can easily update it at any point by editing the metadata.rb file in our application cookbook and rerunning berks install.

A note on librarian-chef

I thought I’d give a quick mention to librarian-chef since this is where my journey started and I only really stumbled across Berkshelf whilst researching it.

There are a few points where Berkshelf appears to offer me some nice differentiators:

  • librarian-chef takes over your projects /cookbooks directory meaning that need to add /cookbooks to your .gitignore. It also means that application cookbooks have to go in another directory, such as /site-cookbooks. Furthermore librarian-chef also creates /tmp folder within your project which you also have to ignore. Compared to Berkshelf, that’s a lot more cruft.
  • Berkshelf helps support the application cookbook pattern by facilitating the creation of skeleton cookbooks.
  • Berkshelf has a vagrant plugin which really brings the whole workflow together.

All that said, librarian-chef is still a very worthy tool and is absolutely worth checking out to see if it fits your needs better.

Further Reading