History

After discussion on the mailing list and IRC,

WARNING: `DavidSchmitt`:trac: is not a valid TracLink

volunteered to spear-head a “Common Modules Project”. This page describes possible workflows with such a central repository. This page is currently in alpha status. Tests, Edits or Feedback on the mailing list highly appreciated! The plan is to wrap the git commands into some frontend script to manage the various action, when the workflows received a bit testing in real live. Currently, that central repository is the one detailed [[Complete Configuration]] , managed by David. It could eventually be folded into modules.reductivelabs.com.

Contents

Definitions

First, let’s put aside some vocabulary issues.

Roles

Responsible: coalesces and publishes contributions to a given module Contributor: adds value to a given module by contributing patches, testing, documentation User: uses a published module

Branches

Development: Anything goes. Here new stuff can be tested. Configurations are only deployed manually in restricted environments. Testing: Finished development is first deployed to a staging area. From here test hosts and second-line systems receive their configuration. While nothing should break, this stage is used to catch slip-ups. Production: After a burn-in in Testing, modules can be pushed to Production. Here nothing should happen anymore.

Repositories

Each repository contains a single module, thus collapsing the notions of repositories and modules. Within a repository, multiple branches can exist. Deploying a branch means checking out the files into a working directory. Using git, every checkout is a working repository too.

To reduce confusion, there should be one branch per environment. Here is how I (David Schmitt) set up my repos. My modules all go to /srv/puppet/module-$environment/ on my puppetmaster.

Creating the repositories:

MODULENAME=newmodule
for i in production development; do
   mkdir -p /srv/puppet/modules-$i/$MODULENAME
   ( cd /srv/puppet/modules-$i/$MODULENAME; git-init )
done

Setting up the development repository:

cd /srv/puppet/modules-development/$MODULENAME
mkdir manifests
touch manifests/init.pp
git add manifests/init.pp
git-commit -m 'empty module'
git-checkout --no-track -b development
git-checkout -b production
git-branch -d master
git-config --add remote.publish.url ssh://david@git.black.co.at/srv/git/module-$MODULENAME
git-config --add remote.production.url /srv/puppet/modules-production/$MODULENAME
git-checkout development
git-push production production # prime the production repo

Now I can start adding stuff and committing them to the development branch, running puppetd —test —environment=development now receives the new changes.

If I’m satisfied with my current level of testing I can merge the work done into my production branch:

git-push . development:production

Since working in the production branch is a big No-No, this should always be a fast-forward merge, which just adjusts the tip of the branch to the current status.

Finally this can be pushed into the production environment and publish it on my webpage:

git-push production
git-push publish

Again, working in the production repo is a even bigger No-No, so this should work always. Also, I didn’t show the creating of the repo on my webhost, but that is really straight forward (just use git::repo and git::web::export from my git module ;)

Having only a small installation, one staging area is good enough. Other people might be interested in more complicated structures like multiple staging and developing areas. Multiple, staggered staging areas can be achieved by simply adding more top-level directories and chaining them together with more remote.*.url config entries. git-push(1) has a list of all valid repository URLs.

Configuration

[This needs 0.24 or later]

To setup the environments, add these values to your puppet.conf:

[main]
# This is the default environment for all clients
environment=production

[puppetmasterd]
# specify allowed environments
environments=production,testing,development

# configure environments
[development]
modulepath=$confdir/modules-development:/usr/share/puppet/modules

[testing]
modulepath=$confdir/modules-testing:/usr/share/puppet/modules

[production]
modulepath=$confdir/modules-production:/usr/share/puppet/modules

For detailed Configuration Issues see: [[Using Multiple Environments]]

Common Modules Workflow

[todo: review http://wiki.procode.org/cgi-bin/wiki.cgi/StGIT_Tutorial for contributor/user usage]

“Upstream” Release and Development

  • one of the responsibles publishes module to public repo with branches master (production), testing and development

Contribution

  • contributor clones repo locally
  • contributor commits local changes and publishes them (repo or patch)
  • responsible integrates submitted changes
  • contributor tests modules' various branches

Usage

[This needs 0.24 or later]

  • user clones repo to $localdatadir/puppet/modules-development/$modulename
  • local changes are committed and tested there, using puppetd —test —environment development —noop
  • good modifications are pushed to $localdatadir/modules-production/$modulename

Usage, integrating new release

[This needs 0.24 or later]

  • user clones repo to $localdatadir/puppet/modules-development/$modulename
  • local changes can be committed there to branch “development”
  • testing against branch “development”, using puppetd —test —environment development —noop
  • create production branch and apply local changes: “git checkout -b production origin/master && git merge development”
  • testing against branch “production”, still in -development, again using puppetd —test —environment development —noop but with a different checkout
  • modifications are pulled to $localdatadir/modules-production/$modulename
  • git checkout development => further changing/testing cycles

After new upstream release:

  • git rebase origin/master development; after resolving conflicts now the development branch should be ready to test/fix up and then merge into production as above [ this breaks if the repo is used locally to push onto the production working directory, use merge instead. For better upstream cooperation, perhaps use a separate repo/branch to prepare patches by cherry-picking them and creating new commits ]