Puppet

# Keep your Puppet manifests under version control

It's a good idea to keep your Puppet manifests and other

configuration files under version control, for example Subversion

or CVS. To do this, just set up the Puppet Master the way you want,

and then import the whole of /etc/puppet into Subversion:

    $ svn import /etc/puppet https://www.your-svn-server-here.com/svn/puppet/trunk

Once you have this imported, you will need to remove the files from /etc/puppet 

and check it out from your SVN repo, to make it a working copy, so that

svn up /etc/puppet works.

    $ svn co https://www.your-svn-server-here.com/svn/puppet/trunk /etc/puppet

In order to edit your puppet config, you can then check out a working copy somewhere else:

    $ svn co https://www.your-svn-server-here.com/svn/puppet/trunk puppet

Edit it, then commit your changes, and update the master copy on

the server:

    $ svn up /etc/puppet

The Puppet master automatically detects that its configuration

files have changed.

Remember when you create new certificates, you are modifying the

Puppet master's working copy, so you need to commit these changes

every so often. This has the added benefit that if you should lose

the Puppet master server, you can easily recreate it by just

checking out a copy of the Puppet tree into /etc/puppet.

## Configuration Versioning

From version 0.25.0, a new configuration option, config\_version,

is now available:

    config_version = /usr/local/bin/return_version

The option allows you to specify a command that returns a version

for the configuration that is being applied to your hosts. The

command should return a string, such as a version number or name.

One such example of a command to return a version number would be:

    svn info /etc/puppet/ | grep Revision: | egrep -o [0-9]+

Puppet then runs this command at compile time. Each resource is

marked with the value returned from this command. This value is

also added to the log instance, serialised and sent along with any

report generated. This allows you to parse your report output and

ascertain which configuration version was used to generate the

resource.

# Using Hooks

Hooks let you extend the value of Subversion (or Git, CVS, etc) to

perform error checking, stage files and even produce audit trails.

## SVN Hooks

### SVN Pre-Commit Hook

To catch syntax errors and other basic problems, you can use a

Subversion pre-commit hook like this:

    #!/bin/sh

    # SVN pre-commit hook to check Puppet syntax for .pp files

    # Modified from http://mail.madstop.com/pipermail/puppet-users/2007-March/002034.html

    REPOS="$1"

    TXN="$2"

    tmpfile=`mktemp`

    export HOME=/

    SVNLOOK=/usr/bin/svnlook

    $SVNLOOK changed -t "$TXN" "$REPOS" | awk '/^[^D].*\.pp$/ {print $2}' | while read line

    do

        $SVNLOOK cat -t "$TXN" "$REPOS" "$line" > $tmpfile

        if [ $? -ne 0 ]

        then

            echo "Warning: Failed to checkout $line" >&2

        fi

        puppet --color=false --confdir=/tmp --vardir=/tmp --parseonly --ignoreimport $tmpfile >&2

        if [ $? -ne 0 ]

        then

            echo "Puppet syntax error in $line." >&2

            exit 2

        fi

    done

    res=$?

    rm -f $tmpfile

    if [ $res -ne 0 ]

    then

        exit $res

    fi

If you get errors like this when committing:

    /usr/lib/ruby/site_ruby/1.8/puppet/defaults.rb:102:in `handle': private method `split' called for nil:NilClass (NoMethodError)

It's because the PATH environmental variable doesn't exist. It

doesn't matter what it is, but puppet wants it to be there. Try

adding PATH="" somewhere in the top of the pre-commit script.

### SVN Post-Commit Hook

Using a post-commit hook can be handy if you want your commits to

automatically be seen by Puppet, e.g. you don't have to do the last

step shown above (svn up). Also, by integrating

[cvsspam](http://blog.thinkphp.de/archives/239-Using-CVSSpam-with-Subversion.html)

you can provide an audit trail with nicely formatted, coloured

diffs.

Here's a simplified example of a post-commit that simply updates

the files (previous checked-out) in /etc/puppet.

    #!/bin/sh

    REPOS="$1"

    REV="$2"

    svn up /etc/puppet

Of course much more can be done here, but that is a nice start.

## Git Hooks

### Git Update Hook

To catch syntax errors and other basic problems, you can use a

server-side Git update hook like this:

    #!/bin/bash

    NOBOLD="\033[0m"

    BOLD="\033[1m"

    BLACK="\033[30m"

    GREY="\033[0m"

    RED="\033[31m"

    GREEN="\033[32m"

    YELLOW="\033[33m"

    BLUE="\033[34m"

    MAGENTA="\033[35m"

    CYAN="\033[36m"

    WHITE="\033[37m"

    # V +1007

    # Peff helped:

    # http://thread.gmane.org/gmane.comp.version-control.git/118626

    syntax_check="puppet --color=false --confdir=/tmp --vardir=/tmp --parseonly --ignoreimport"

    tmp=$(mktemp /tmp/git.update.XXXXXX)

    log=$(mktemp /tmp/git.update.log.XXXXXX)

    tree=$(mktemp /tmp/git.diff-tree.XXXXXX)

    git diff-tree -r "$2" "$3" > $tree

    echo

    echo diff-tree:

    cat $tree

    exit_status=0

    while read old_mode new_mode old_sha1 new_sha1 status name

    do

      # skip lines showing parent commit

      test -z "$new_sha1" && continue

      # skip deletions

      [ "$new_sha1" = "0000000000000000000000000000000000000000" ] && continue

      # Only test .pp files

      if [[ $name =~ [.]pp$ ]]

      then

        git cat-file blob $new_sha1 > $tmp

        set -o pipefail

        $syntax_check $tmp 2>&1 | sed 's|/tmp/git.update.......:\([0-9]*\)$|JOJOMOJO:\1|'> $log

        if [[ $? != 0 ]]

        then

          echo

          echo -e "$(cat $log | sed 's|JOJOMOJO|'\\${RED}${name}\\${NOBOLD}'|')" >&2

          echo -e "For more details run this: ${CYAN} git diff $old_sha1 $new_sha1 ${NOBOLD}" >&2 

          echo

          exit_status=1

        fi

      fi

    done < $tree

    rm -f $log $tmp $tree

    exit $exit_status

### Git Pre-Commit Hook

To catch syntax errors and other basic problems, you can use a

client-side Git pre-commit hook like this:

    #!/bin/sh

    syntax_errors=0

    error_msg=$(mktemp /tmp/error_msg.XXXXXX)

    if git rev-parse --quiet --verify HEAD > /dev/null

    then

        against=HEAD

    else

        # Initial commit: diff against an empty tree object

        against=4b825dc642cb6eb9a060e54bf8d69288fbee4904

    fi

    # Get list of new/modified manifest and template files to check (in git index)

    for indexfile in `git diff-index --diff-filter=AM --name-only --cached $against | egrep '\.(pp|erb)'`

    do

        # Don't check empty files

        if [ `git cat-file -s :0:$indexfile` -gt 0 ]

        then

            case $indexfile in

                *.pp )

                    # Check puppet manifest syntax

                    git cat-file blob :0:$indexfile | puppet --color=false --parseonly --ignoreimport > $error_msg ;;

                *.erb )

                    # Check ERB template syntax

                    git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg > /dev/null ;;

            esac

            if [ "$?" -ne 0 ]

            then

                echo -n "$indexfile: "

                cat $error_msg

                syntax_errors=`expr $syntax_errors + 1`

            fi

        fi

    done

    rm -f $error_msg

    if [ "$syntax_errors" -ne 0 ]

    then

        echo "Error: $syntax_errors syntax errors found, aborting commit."

        exit 1

    fi

## Bazaar (bzr) Hooks

### Bazaar Pre-Commit Hook

This is a client side pre-commit hook to catch basic syntax errors.

    #!/usr/bin/env python

    #

    # BZR pre-commit hook, which will run some basic syntax checking on manifests

    # in the current branch.

    #

    # To use this script, place it in your ~/.bazzar/plugins directory (create

    # this directory if it doesn't exist).

    #

    from bzrlib import branch

    import os

    import sys

    import subprocess

    def get_branch_root(directory):

        """Find the root directory of the current BZR branch."""

        while os.path.exists(directory):

            if os.path.exists(os.path.join(directory, '.bzr')):

                return directory

            if directory == '/':

                break

            (parent, dir) = os.path.split(directory)

            directory = parent

        print "Commit FAILED:  Can't locate BZR Root."

        sys.exit(1)

    def check_puppet_syntax(local, master, old_revno, old_revid, future_revno,

                           future_revid, tree_delta, future_tree):

        """This will run some basic syntax checking on the puppet manifests."""

        # Check syntax on changed files

        errors = []

        os.chdir(get_branch_root(os.getcwd()))

        print "\n" # make some space so we aren't clobbered by bzr's status msgs

        for file in tree_delta.added + tree_delta.renamed + tree_delta.modified:

            file = file[0]

            if file.endswith(".pp"):

                print "Checking syntax in: %s" % (file)

                try:

                    process = subprocess.Popen(["puppet", "--parseonly", file],

                        stderr=subprocess.STDOUT, stdout=subprocess.PIPE)

                    process.wait()

                    if process.returncode != 0:

                        errors.append((file, ''.join(process.stdout.readlines())))

                except OSError, e:

                    print "\n\n Error: failed to execute 'puppet': %s" % (e)

                    sys.exit(1)

        if errors:

            print "\nSyntax errors were found:\n"

            for error in errors:

                print "%s: %s" % (error[0], error[1]),

            print "\nCommit FAILED"

            sys.exit(1)

        else:

            print "\nAll syntax checks PASSED"

    # This is where the magic happens

    branch.Branch.hooks.install_named_hook('pre_commit', check_puppet_syntax,

                                           'Check puppet manifests for syntax errors.')