Puppet

So I may actually have described this incorrectly.

Perhaps the actual case is that clients are now always required to talk to the CA on every run, but if so, that’s a change in behavior.

It also doesn’t make a lot of sense, as the puppetmasterd process was started with —no-ca on testserver.mydomain, so it should really have failed to talk to that server as a CA if this is the case, but perhaps —no-ca isn’t working.

So far, I haven’t been able to reproduce this, on neither 0.25.x nor 0.25.0rc1 I suppose there has to be an interaction with another setting, but I haven’t had any insight into what that might be.

So I just hit this again.

I’m in the middle of switching from 0.24.8 to 0.25.4 on the servers, so am doing a fair bit of testing of all my clients against 0.25.4 (including 0.25.4 clients).

If my puppetd.conf looks like this on a client:

[puppetd]
  user            = root
  group           = wheel
  server          = my_puppet_server
  masterport      = 9140
  ca_server    = my_ca_server
  ca_port       = 9150
  listen          = false
  configtimeout   = 360
  pidfile         = /var/run/puppetd.pid
  vardir          = /var/puppet
  pluginsync      = true
  factpath        = $vardir/lib/facter
  runinterval     = 3600
  syslogfacility  = local7
  certname        = [redacted]

I noticed when moving to plugins in modules that I was getting ERROR 405: eval_generate, method not allowed errors when attempting to pluginsync.

As this indicates I’m talking to a 0.24.8 server with a 0.25.4 client, I removed the ca_server/ca_port lines, the error goes away, and pluginsync works correctly again.

I’ll try to repro a simple case and attach traces.

Config file with separate ca_server and ca_port:

[puppetd]
  user            = root
  group           = wheel
  server          = my_test_025_server
  ca_server       = my_024_ca_server
  ca_port         = 9150
  masterport      = 9140
  listen          = false
  configtimeout   = 360
  pidfile         = /var/run/puppetd.pid
  vardir          = /var/puppet
  pluginsync      = true
  factsync        = false
  environment     = test_environment
  factpath        = $vardir/lib/facter
  runinterval     = 3600
  syslogfacility  = local7
  certname        = [redacted]

puppet run against a test environment that doesn’t do much:

# puppetd --test --trace
info: Retrieving plugin
/Library/Ruby/Site/1.8/puppet/indirector/rest.rb:55:in `deserialize'
/Library/Ruby/Site/1.8/puppet/indirector/rest.rb:69:in `find'
/Library/Ruby/Site/1.8/puppet/indirector/indirection.rb:195:in `find'
/Library/Ruby/Site/1.8/puppet/indirector.rb:51:in `find'
/Library/Ruby/Site/1.8/puppet/ssl/host.rb:208:in `ssl_store'
/Library/Ruby/Site/1.8/puppet/network/http_pool.rb:56:in `cert_setup'
/Library/Ruby/Site/1.8/puppet/network/http_pool.rb:100:in `http_instance'
/Library/Ruby/Site/1.8/puppet/indirector/rest.rb:65:in `network'
/Library/Ruby/Site/1.8/puppet/indirector/rest.rb:73:in `search'
/Library/Ruby/Site/1.8/puppet/indirector/indirection.rb:240:in `search'
/Library/Ruby/Site/1.8/puppet/indirector.rb:59:in `search'
/Library/Ruby/Site/1.8/puppet/type/file.rb:595:in `perform_recursion'
/Library/Ruby/Site/1.8/puppet/type/file.rb:562:in `recurse_remote'
/Library/Ruby/Site/1.8/puppet/type/file.rb:561:in `collect'
/Library/Ruby/Site/1.8/puppet/type/file.rb:561:in `recurse_remote'
/Library/Ruby/Site/1.8/puppet/type/file.rb:483:in `recurse'
/Library/Ruby/Site/1.8/puppet/type/file.rb:385:in `eval_generate'
/Library/Ruby/Site/1.8/puppet/transaction.rb:349:in `send'
/Library/Ruby/Site/1.8/puppet/transaction.rb:349:in `generate_additional_resources'
/Library/Ruby/Site/1.8/puppet/transaction.rb:193:in `eval_generate'
/Library/Ruby/Site/1.8/puppet/transaction.rb:240:in `eval_children_and_apply_resource'
/Library/Ruby/Site/1.8/puppet/transaction.rb:207:in `eval_resource'
/Library/Ruby/Site/1.8/puppet/transaction.rb:296:in `evaluate'
/Library/Ruby/Site/1.8/puppet/util.rb:418:in `thinmark'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/Library/Ruby/Site/1.8/puppet/util.rb:417:in `thinmark'
/Library/Ruby/Site/1.8/puppet/transaction.rb:295:in `evaluate'
/Library/Ruby/Site/1.8/puppet/transaction.rb:289:in `collect'
/Library/Ruby/Site/1.8/puppet/transaction.rb:289:in `evaluate'
/Library/Ruby/Site/1.8/puppet/resource/catalog.rb:142:in `apply'
/Library/Ruby/Site/1.8/puppet/configurer/downloader.rb:32:in `evaluate'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'
/Library/Ruby/Site/1.8/puppet/configurer/downloader.rb:31:in `evaluate'
/Library/Ruby/Site/1.8/puppet/configurer/plugin_handler.rb:12:in `download_plugins'
/Library/Ruby/Site/1.8/puppet/configurer.rb:85:in `prepare'
/Library/Ruby/Site/1.8/puppet/configurer.rb:152:in `run'
/Library/Ruby/Site/1.8/puppet/agent.rb:53:in `run'
/Library/Ruby/Site/1.8/puppet/agent/locker.rb:21:in `lock'
/Library/Ruby/Site/1.8/puppet/agent.rb:53:in `run'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/sync.rb:229:in `synchronize'
/Library/Ruby/Site/1.8/puppet/agent.rb:53:in `run'
/Library/Ruby/Site/1.8/puppet/agent.rb:134:in `with_client'
/Library/Ruby/Site/1.8/puppet/agent.rb:51:in `run'
/Library/Ruby/Site/1.8/puppet/application/puppetd.rb:103:in `onetime'
/Library/Ruby/Site/1.8/puppet/application.rb:226:in `send'
/Library/Ruby/Site/1.8/puppet/application.rb:226:in `run_command'
/Library/Ruby/Site/1.8/puppet/application.rb:217:in `run'
/Library/Ruby/Site/1.8/puppet/application.rb:306:in `exit_on_fail'
/Library/Ruby/Site/1.8/puppet/application.rb:217:in `run'
/usr/sbin/puppetd:159
err: /File[/var/puppet/lib]: Failed to generate additional resources using 'eval_generate': Error 405 on SERVER: Method Not Allowed
info: Caching catalog for [redacted]
info: Applying configuration version '1265160940'
notice: //base/File[/tmp/test_environment_dynamic]/ensure: created
notice: //base/File[/tmp/test_environment_static]/ensure: content changed '{md5}0d1dca493040cfceb342313b03a7ea29' to '{md5}0d1dca493040cfceb342313b03a7ea29'
notice: Finished catalog run in 0.40 seconds

Now I comment out the ca_server and ca_port lines:

# puppetd --test --trace
info: Retrieving plugin
notice: /File[/var/puppet/lib/facter/puppet_certname.rb]/ensure: content changed '{md5}f3f7a82b554d9a0ae22851e7066ed7ea' to '{md5}f3f7a82b554d9a0ae22851e7066ed7ea'
info: Loading downloaded plugin /var/puppet/lib/facter/puppet_certname.rb
info: Loading facts in puppet_certname
info: Loading facts in puppet_certname
info: Caching catalog for [redacted]
info: Applying configuration version '1265160940'
--- /tmp/test_environment_dynamic   2010-02-02 17:35:41.000000000 -0800
+++ /tmp/puppet-diffing.23781.0 2010-02-02 17:37:34.000000000 -0800
@@ -1 +1 @@
-Tue Feb  2 17:35:41 PST 2010
+Tue Feb  2 17:37:34 PST 2010
notice: //base/File[/tmp/test_environment_dynamic]/content: content changed '{md5}657238cc81979d8883dc1a7ca457904a' to 'unknown checksum'
notice: Finished catalog run in 0.60 seconds

PS. Tested with 0.25.4 on Mac OS X 10.5 and Ubuntu Hardy with exactly the same results.

ok, so it turns out this is a misleading error message.

err: /File[/var/puppet/lib]: Failed to generate additional resources using 'eval_generate': Error 405 on SERVER: Method Not Allowed

Several repeated tests are showing that the problem isn’t this directory. It’s the crl.pem on the client. If I switch to a 0.25.x CA, everything is fine, I switch back to my 0.24.8 CA, everything continues to work for a certain period of time. If I delete the crl.pem file on the client at this time, then the problems occur again.

So this is ok. I don’t mind having to provision my 0.25. CA before the other servers, but there’s a bug in the logging system somewhere.

I have no idea what this bug should be renamed to….

I’m not sure what the correct status should really be, but it’s not needs more information.

I’m unaccepting this bug as the issue seems to be in the logging system, not in my original report, but I’m still unsure what this should be renamed to.