Dynamic hosts file using Chef

Note: This post is quite old. There are probably better methods for this now.

There are a number of ways to setup your infrastructure so that you can refer to machines by hostname. I currently prefer the “dynamically generated hosts file” approach because it’s simple to understand and setting up a DNS server is intimidating (as well as a single point of failure).

Shlomo Swidler has a great article comparing different DNS configurations as well as some sample code for dynamically updating hosts files. However, if you’re already using Chef you can achieve the same thing with a very simple cookbook.

First, create a new cookbook:

$ cd chef-repo/
$ rake new_cookbook COOKBOOK=hosts CB_PREFIX=site-

Place the following in site-cookbooks/hosts/recipes/default.rb:

# Find all nodes, sorting by Chef ID so their
# order doesn't change between runs.
hosts = search(:node, "*:*", "X_CHEF_id_CHEF_X asc")

template "/etc/hosts" do
  source "hosts.erb"
  owner "root"
  group "root"
  mode 0644
  variables(
    :hosts => hosts,
    :fqdn => node[:fqdn],
    :hostname => node[:hostname]
  )
end

Then create the template, site-cookbooks/hosts/templates/default/hosts.erb:

127.0.0.1   localhost

<% @hosts.keys.sort.each do |fqdn| %>
<%= @hosts[fqdn][:ipaddress] %> <%= fqdn %> <%= @hosts[fqdn][:hostname] %>
<% end %>

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Add the recipe to your node’s run_list and run chef-client and your /etc/hosts should contain all the Chef nodes on your network. Note that one downside to this approach is that updates will be slow (since chef-client only runs every 30 minutes by default).

Now what if you wanted to point at a specific service, like having chef.example.com point at your Chef master? Just search for it in your recipe (and add it to the variables list):

chef_server = search(:node, 'run_list:recipe\[chef\:\:server\]')

And add a line to the template:

# Chef master server
<%= @chef_server[:ipaddress] %> chef.example.com

Sure beats setting up a DNS server!

Redis plugin for collectd

Last weekend I wrote a small collectd plugin for monitoring Redis and put it on GitHub: http://github.com/powdahound/redis-collectd-plugin.

If you haven’t heard of Redis before, it’s a key-value store similar to memcached with support for more complex data types like lists, sets, and hashes. collectd is a stats collection daemon which recently added support for plugins written in Python. This is great because there’s no way I’d write a plugin in C! Anyway, you can use this plugin to collect data about your Redis server like number of active connections, database size, commands processed per second, and memory usage. You can then use a tool like drraw to generate nice graphs:

More info and installation instructions on the GitHub page.

Tricky Apache configuration bug

When using Apache’s DirectoryIndex directive make sure you don’t separate the values with commas. If you do, Apache will be looking for filenames ending in commas!

# Incorrect
DirectoryIndex index.html, index.php, index.cgi

# Correct
DirectoryIndex index.html index.php index.cgi