Written by on 23 April 2015

Managing the application settings for multiple environments

At Coolblue an application will be run on a variety of environments. Besides production most of our applications will be placed on an acceptance environment for quality assurance, but most likely also will be delivered on a test or development environment.

With a growing amount of applications being deployed on different environments we found out that maintaining these configurations in the puppet modules itself wasn’t useful and a pain to manage. So we went searching for a maintainable solution to configure application settings per environment. This article will describe how we tackled our problem.

Configuration management @ Coolblue

Before we dive into our solution it is useful to describe how we manage our environments. We currently use puppet to manage our nodes configuration. The big advantage of using configuration management is that configuration is known and documented. No more hidden configuration settings that are only known by the administrator that left the company last month. It makes system configuration predictable and ultimately reduces administration effort in the long term.

Our complete environment is described in puppet files. In order to make components of these configuration reusable we’ve decided to make use of the roles and profiles pattern. The roles and profiles pattern ensures that specific node configuration will not be tightly coupled to the module you have developed. Basically, it’s adding an abstraction level so modules are easier to reuse and isolated from node specific needs.

Extracting node configurations from the modules

The roles and profiles pattern forces us to store node specific configuration in a different element other than the module itself. We chose to use a hierarchical datastore to store node specific configurations. That might sound complicated, but in fact it isn’t. In fact, most of the time it are just a set of yaml files.

Puppet provides a hierarchical database out of the box which is called hiera. Hiera is a key/value store mechanism which has the notion of hierarchies. It is that particular mechanism which is the base for the way we manage application settings across various environments. This article is not about the workings of hiera, but it is useful to have a high overview of its workings.

Explaining hiera

Like I already mentioned, in our case the hiera store is just a set of yaml files. As an example lets take a look at a very simple hiera file.

---
# Define a read and write datastore
datastores:
 read:
   user: <USER>
   pass: <PASS>
 write:
   user: <USER>
   pass: <PASS>

In our puppet code we have a mysql profile which reads this configuration and creates the configuration elements accordingly.

class coolblue::profiles::mysql_config {
 $datastores = hiera_hash('datastores', {})
 
# Write configuration
 create_resources(..., $datastores)
}

A huge advantage with this way of working is that we can configure each node that uses the mysql profile in a different way. For example while node A has one mysql configuration for the application, the application on node B might have two mysql configurations. This is not described in the mysql profile, but in the hiera configuration of that particular node.

Earlier I’ve mentioned that hiera is a hierarchical datastore. With this mechanism it is possible to have a common file which contains the general values for your variables on all hosts and override a variable per environment, hostgroup, or host.

That might sound a bit abstract, so lets display how this works in our setup. Our hiera structure looks a bit as listed below:

  • /hieradata
    • /production
      • nodejs-01-01.eyaml
    • /acceptance
      • nodejs-01-01.eyaml
    • /development
      • dev-nodejs.yaml
    • common.yaml

Based on the node and the environment the puppet agent is executed on the correct values will be collected from the set yaml files and be used for the configuration. As an example take a look at the configuration table below.

Hiera file Property Value
/production/nodejs-01-01.eyaml database::write::user read_nodejs
/acceptance/nodejs-01-01.eyaml
/development/dev-nodejs.yaml
common.yaml database::write::user development

This will result in the following collected value for that property per environment.

Environment Property Value
Production database::write::user read_nodejs
Acceptance database::write::user development
Development database::write::user development

Exact details describing the hierarchy mechanism and the ordering can be found in the puppet documentation.

Wrapping up

In conclusion we can state that we have two important components which helps us managing application, and even node, configuration in a easy and maintainable way.

The first one is implementing the roles and profiles pattern. This pattern enforces us to separate node configuration out of the modules we create. The other one being the use of hiera and the hierarchy it provides to cope with several environments. Together these components will offer you a great mechanism for configuration management across various environments.

COMMENTGive your two cents.

*