Puppet

# Change Management

Many environments require some kind of change management workflow

-- changes must pass through multiple phases (e.g., development,

testing, production) and often require approval to move between

phases. How might such a system be built into Puppet, or built

externally and integrated with Puppet?

This problem is usually known as "Change Management", and it must

be solved for multiple problem spaces, including software

development. We should be able to take examples from those other

problem spaces, but I have not done sufficient research on those

other spaces at this point (other than trawling the 'net but not

finding much in the way of examples).

# Potential Solutions

Rather than going through a complicated analysis of all of the

options and what effect the different choices have, I'm just going

to lay out what I consider to be the basic options, to provide a

framework for discussion. From there, we can hopefully reduce the

scope of discussion to the more obviously appropriate solutions.

## Parse Trees in a Database

Because Puppet's language is declarative, it's relatively

straightforward to model it with a database. It wouldn't

necessarily be easy, but it would at least be straightforward. With

everything in a database, you could provide an interface for

selecting values based on whatever criteria you want, and those

decisions could be implemented via conditional structures in the

database. The approval process could also be relatively

straightforward, because they become approvals of an SQL

statement.

The downside of this solution, of course, is that it's a heckuva

lot of work. First is creating the database modelling and the

necessary interfaces into it, but you are also suddenly limited in

code-sharing because you have to translate back and forth between

SQL and Puppet code, and the scope of your solution is almost

entirely open-ended -- where does your change management

application end and Puppet begin?

Because of the open-ended nature of this solution, and the large

amount of work involved, I do not think this is a good idea.

## Functional Approval Lookup

The simplest solution is just to connect Puppet to an external

approval mechanism via a Puppet function and simple keyword lookup.

The approval system would have registered values that could be

modified as necessary, and then the appropriate values could be

looked up in the Puppet language:

    package { apache:

      ensure => approved("apache-version")

    }

The biggest problem with this solution is that systems in different

service levels vary by much more than just simple values --

sometimes a different package version has different dependencies or

renames a file, and re-factoring an implementation often looks

different enough from the original that the difference cannot be

modelled with simple values.

The function could be enhanced to support returning code segments

to apply, but then you're back to modelling Puppet elements in the

database.

## Version Control Solutions

Probably the easiest way to handle differences between machines

with slightly different configurations is to create a branch in

your version control system for each service level, and then create

a mechanism for approving individual changes and merges between

branches. This allows you to reuse the version control model for

everything except the approval process. This approval process

should not be too complicated, although it will almost certainly be

tied to a specific version control system. You could use

post-commit hooks for registering changes and merges and then have

the approval system migrate approved changes and merges into a

separate branch.

The big upside to this approach is that it's the most minimalist

approach and thus the easiest, and it should support creating a

decoupled approval system that could be used with any text-based

system, and it should be relatively easy to hook into different

version control systems. The downside is that it's entirely

unsemantic -- approvers will get plain diffs, rather than semantic

modifications (e.g., they'll get two lines mentioning the package

version, rather than a single statement along the lines of

"changing version from X to Y").

## Filtering

This isn't really a complete solution, but it's an idea, anyway.

There are multiple points at which a configuration could be

filtered through an approval process.

The parse tree could be passed through an approval filter that

modified it in some way, probably something like creating

conditional structures where there previously were none.

Alternatively, each individual host's compiled configuration could

be passed through a filter which could fill in values based on the

results of approvals. This option probably limits what can be done

via the approval mechanism, because again it mostly affects values,

not the elements themselves.

You could also potentially directly modify the database cache of

host configurations via the approval mechanism, but I think that

would again limit your flexibility while requiring too much

knowledge of Puppet's internals.

# Conclusion

This document is meant more as an entry point to discussion, rather

than a real solution, so it should not be seen as a complete

listing of options nor should it be expected to be complete.

Hopefully it's sufficient to continue discussion, anyway.