Puppet

1

# Testing against a Branch Revision

2

1

3

1

This document describes how to quickly set-up additional

4

1

puppetmasterd servers for testing purposes.

5

1

6

1

Author: Jeff McCune \<mccune@…>

7

1

8

1

## Motivation

9

1

10

1

We all know it's a good idea to test changes before deploying them

11

1

to a production environment. This can be difficult in a site

12

1

configuration system because of the overhead setting up a new

13

1

server, authentication, etc...

14

1

15

1

With Puppet, it's relatively easy to fork off a branch of your

16

1

production configuration and start an additional service, serving

17

1

only the testing manifest. This allows you to test a limited number

18

1

of clients against the untested policy before merging it back into

19

1

the production tree.

20

1

21

1

## Assumptions

22

1

23

1

-   Your Puppet configuration files and manifests are under

24

1

    revision control.

25

1

-   Your revision control system supports branching and merging.

26

1

    e.g. [subversion](http://subversion.tigris.org/).

27

1

-   You're running Puppet in a client-server environment.

28

1

-   /etc/puppet is a subversion working copy on the master.

29

1

30

1

## Module Organisation

31

1

32

1

As of version 0.22.2, Puppet now supports a configurable search

33

1

path to load modules. This greatly helps the management of site

34

1

specific manifest alongside manifest information from external

35

1

sources. This document should be updated to reflect this new

36

1

feature. Generally speaking, the module search path allows us to

37

1

direct a puppet master directly to VCS working copies of isolated

38

1

pieces of our manifest.

39

1

40

1

Please see [[Module Organisation]] for more information on this

41

1

excellent feature.

42

1

43

1

## Procedure

44

1

45

1

### Fork production configuration

46

1

47

1

I assume the puppetmasterd's confdir is under revision control, and

48

1

is a svn working copy.

49

1

50

1

In our personal working copy, we need to copy the production

51

1

configuration to a testing one. We'll make our changes in testing,

52

1

test them, then merge them back into production once we're

53

1

satisfied.

54

1

55

1

svn copy puppet puppet.testing

56

1

57

1

Once you do this, you should have a layout that looks like this:

58

1

59

1

    puppet/manifests/site.pp

60

1

    puppet/puppetca.conf

61

1

    puppet/puppetd.conf

62

1

    puppet/puppetmasterd.conf

63

1

    puppet.testing/manifests/site.pp

64

1

    puppet.testing/puppetca.conf

65

1

    puppet.testing/puppetd.conf

66

1

    puppet.testing/puppetmasterd.conf

67

1

68

1

My repository looks like:

69

1

70

1

    Path: .

71

1

    URL: file:///Volumes/svn/repos/management/master

72

1

    Repository Root: file:///Volumes/svn/repos/management

73

1

    Revision: 1

74

1

    Node Kind: directory

75

1

    Schedule: normal

76

1

    Last Changed Author: mccune

77

1

    Last Changed Rev: 1

78

1

    Last Changed Date: 2007-02-11 12:23:34 -0500 (Sun, 11 Feb 2007)

79

1

80

1

### Make changes to testing configuration

81

1

82

1

At this point, you need to edit puppetmasterd.conf to reflect the

83

1

new location of your testing configuration.

84

1

85

1

I change all instances of **/etc/puppet** to

86

1

**/etc/puppet.testing**, and other runtime directories. You should

87

1

keep the same **SSL** setup so you don't have to re-sign client

88

1

keys.

89

1

90

1

Finally, change **masterport = 8141**. This allows you to run a

91

1

second server on a non-default port.

92

1

93

1

Commit these changes back into the repository.

94

1

95

1

### Second puppetmasterd

96

1

97

1

Now that we have a second copy of the configuration, with relevant

98

1

changes in the repository, we can start the second server:

99

1

100

1

**svn co file:///Volumes/svn/repos/management/master/puppet.testing /etc/puppet.testing**

101

1

102

1

I include a simple shell script to start the test puppetmasterd in

103

1

the foreground:

104

1

105

1

    #!/bin/bash

106

1

    export RUBYLIB="/etc/puppet.testing/lib/ruby:$RUBYLIB"

107

1

    exec puppetmasterd --verbose --confdir=/etc/puppet.testing --masterport=8141 $*

108

1

109

1

### Making changes

110

1

111

1

Now that you have a branched configuration, you make changes in

112

1

your personal svn working copy within the puppet.testing tree,

113

1

commit them to the repository, then update the runtime

114

1

configuration with **svn up /etc/puppet.testing**. This represents

115

1

your basic work cycle.

116

1

117

1

### Client Runs

118

1

119

1

To test a client against this isolated server, you merely need to

120

1

call it as normal, adding --masterport=8141.

121

1

122

1

**puppetd --test --masterport=8141**

123

1

124

1

### Merging back into Production

125

1

126

1

It's been suggested that Svnmerge.py might be better at handling

127

1

merges like this procedure describes. You might be better off

128

1

taking a look at it:

129

1

[http://www.orcaware.com/svn/wiki/Svnmerge.py](http://www.orcaware.com/svn/wiki/Svnmerge.py)

130

1

131

1

Now here's the slightly tricky part. We'll assume that the testing

132

1

basic work cycle has gone on for some time, and changes have been

133

1

occurring in both puppet and puppet.testing.

134

1

135

1

This merge procedure is well documented in the SVN book:

136

1

137

1

[http://svnbook.red-bean.com/nightly/en/svn.branchmerge.html](http://svnbook.red-bean.com/nightly/en/svn.branchmerge.html)

138

1

139

1

First, we need to find the revision we forked **puppet.testing**

140

1

from.

141

1

142

1

svn log --verbose --stop-on-copy

143

1

file:///Volumes/svn/repos/management/master/puppet.testing

144

1

145

1

    ------------------------------------------------------------------------

146

1

    r3 | mccune | 2007-02-11 12:53:56 -0500 (Sun, 11 Feb 2007) | 1 line

147

1

    Changed paths:

148

1

       A /master/puppet.testing (from /master/puppet:1)

149

1

       A /master/puppet.testing/manifests/classes (from /master/puppet/manifests/classes:2)

150

1

       A /master/puppet.testing/manifests/site.pp (from /master/puppet/manifests/site.pp:2)

151

1

       A /master/puppet.testing/puppetd.conf (from /master/puppet/puppetd.conf:2)

152

1

       D /master/puppet.testing/ssl

153

1

154

1

    forked puppet to testing branch

155

1

    ------------------------------------------------------------------------

156

1

157

1

So, we can see that !r3 is the epoch of the testing branch.

158

1

--stop-on-copy is very useful here.

159

1

160

1

Here's the final procedure for merging back:

161

1

162

1

    $ cd master/puppet

163

1

    $ svn update

164

1

    At revision 4.

165

1

    $ svn merge -r3:4 file:///Volumes/svn/repos/management/master/puppet.testing

166

1

    U    manifests/site.pp

167

1

168

1

At this point, your personal working copy of the production

169

1

configuration has been modified with the changes you made to the

170

1

testing branch. You should review any conflicts, make sure

171

1

everything merged cleanly with changes that happened to the

172

1

production branch after the revision you copied from, then commit

173

1

your changes back:

174

1

175

1

**svn ci -m 'Merged testing back into production'**

176

1

177

1

That's it.

178

1

179

1

# Regarding the Configuration Files

180

1

181

1

If you're using branch testing, you may only want to merge certain

182

1

sections of the tree back and forth.

183

1

184

1

For example, I personally maintain discrete copies of the

185

1

configuration files, but try and keep my manifest directory merged

186

1

together.

187

1

188

1

I'll first commit a change for testing into

189

1

puppet.testing/manifests, test the change on the second

190

1

puppetmaster, then merge it into puppet.production/manifests.

191

1

192

1

This allows me to keep discrete configuration files;

193

1

/etc/puppet/\*.conf and /etc/puppet.testing/\*.conf for each of the

194

1

puppetmaster daemons as working copies on the server. Only the

195

1

manifest directory is a moving target, the configuration files

196

1

remain unchanged and discrete.

197

1

198

1

# Persistent Branches

199

1

200

1

I maintain two branches in subversion, and I always test and commit

201

1

against the puppet.testing branch. Once I'm satisfied, I merge.

202

1

This makes for a lot of merging over time, so it helps to keep the

203

1

last merged revision and the URL of where you're merging from as

204

1

SVN properties.

205

1

206

1

I find this incredibly useful for the manifests directory.

207

1

208

1

For example: (Set last revision of testing which was merged into

209

1

production)

210

1

211

1

    cd /Volumes/svn/puppet/branches/puppet.production/

212

1

    svn propset merge:rev '2432' manifests

213

1

    svn propset merge:url 'https://manage.math.ohio-state.edu/svn/puppet/branches/puppet.testing/manifests' manifests

214

1

215

1

Then, to actually merge, it's as simple as: (Note, we're in the

216

1

production working directory, merging manifests from the testing

217

1

repository.)

218

1

219

1

    cd /Volumes/svn/puppet/branches/puppet.production/manifests

220

1

    svn merge -r$(svn propget merge:rev):HEAD $(svn propget merge:url)

221

1

222

1

Once you've tested your merged changes in the production branch,

223

1

you can commit and update the merge revision property:

224

1

225

1

    cd /Volumes/svn/puppet/branches/puppet.production/manifests

226

1

    PRODUCT_REVISION=$(svn propget merge:rev)

227

1

    TESTING_REVISION=$(svn info $(svn pg merge:url) | grep Revision | awk '{print $2}')

228

1

    svn propset merge:rev $TESTING_REVISION ./

229

1

    svn update

230

1

    svn status

231

1

    svn commit -m "MERGED: ${PRODUCT_REVISION}:${TESTING_REVISION} from $(svn pg merge:url)"

232

1

233

1

If you do this each time, it's trivial to track what has and has

234

1

not been merged into production from testing.

Branch Testing