Puppet

We still have a lot of problems with puppetmasterd crashing because it runs out of memory, but we’re noticing that puppetd on clients crashes too, in those cases. It first runs the stored config and then crashes with the following trace (after I add —trace):

/usr/lib/ruby/1.8/timeout.rb:54:in `rbuf_fill': execution expired (Timeout::Error)
        from /usr/lib/ruby/1.8/timeout.rb:56:in `timeout'
        from /usr/lib/ruby/1.8/timeout.rb:76:in `timeout'
        from /usr/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill'
        from /usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
        from /usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'
        from /usr/lib/ruby/1.8/net/http.rb:2020:in `read_status_line'
        from /usr/lib/ruby/1.8/net/http.rb:2009:in `read_new'
        from /usr/lib/ruby/1.8/net/http.rb:1050:in `request'
         ... 42 levels...
        from /usr/lib/ruby/1.8/puppet/network/client/master.rb:254:in `run'
        from /usr/lib/ruby/1.8/sync.rb:229:in `synchronize'
        from /usr/lib/ruby/1.8/puppet/network/client/master.rb:236:in `run'
        from /usr/sbin/puppetd:417

We can simulate this behaviour with sending a “kill -STOP” to the puppetmasterd, starting the puppetd on the client and wait for several minutes until it times out. The “kill -STOP” should simulate a crash, since it keeps the port open, but simply makes the puppetmasterd not respond anymore.

It’s easy to work around this, of course, by having a cronjob that regularly checks if the puppetd is still running. But it would be better if this was fixed in the code, since it’s probably not the last time that a connection from puppetd to puppetmasterd times out.

Tested in Debian Etch with 0.24.5-1 packages from testing.

(Note for those who try, you can restart the puppetmasterd after you send it the STOP signal by sending it a CONT signal.)