Puppet

Although this doesn’t affect if a single module/service is deployed. But if a bunch of modules are being deployed with these services as dependencies, then those modules aren’t deployed.

What causes the 5 second delay? Is there a way to tell runit to apply changes immediately instead?

runsvdir monitors the services directory ( /etc/service ) for new services. http://smarden.org/runit/runsvdir.8.html

So, as soon as runsvdir detects a new service, runit will start/supervise it. 5 seconds is the time interval after which runsvdir checks the services directory. So, within a 5 second cycle of runsvdir, if I create a new service at the 3rd second, the service will start in 2 seconds. Brice Figureau told on irc sometime back that Puppet’s earlier versions (0.24.x) could work with such a mechanism.

I got caught up at work and couldn’t really follow up to check.

Can you paste a debug run of a single manifest to manage this service?

http://smarden.org/runit/sv.8.html

From looking at the code, we’re running “sv start ” which should wait 7 seconds.

Can you clarify what you were saying about the differences with Puppet 0.24.x?

I think Brice was the maintainer/author for these providers – daemontools and runit. He told that in earlier Puppet versions, the providers behaved properly.

‘sv start’ won’t wait for 7 seconds explicitly unless the service scripts are broken or there are other errors. 7 seconds is just the upper limit on the time after which if the service doesn’t start, runit will give up on it. Usually services will start instantly. The problem with the runit provider is that without waiting for runsvdir to pick up a new service in /etc/service, it will issue the ‘sv stat’ command, which will give an error because runsvdir hasn’t detected the service yet.

Will post the manifest/debug info later, but its fairly easy to reproduce.

That’s a very different reading of “sv start” to the one I’m getting…

start: Same as up, but wait up to 7 seconds for the command to take effect. Then report the status or timeout. If the script ./check exists in the service directory, sv runs this script to check whether the service is up and available; it’s considered to be available if ./check exits with 0.

A debug run would be helpful.

Sorry for the delay. Hopefully this will be useful. The run script was placed under/etc/sv/test manually.

The manifest:

class test { 
service { "test" :
enable => "true",
ensure => "running",
provider => "runit",
hasstatus => "true",
hasrestart => "true",
}
}

The run script for the service:

#!/bin/sh
while [ 1 ]; do
echo "Hello" > /tmp/hello
done

Debug Information:

root@services:~# puppetd --test --debug
debug: Failed to load library 'rubygems' for feature 'rubygems'
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Puppet::Type::User::ProviderPw: file pw does not exist
debug: Failed to load library 'ldap' for feature 'ldap'
debug: Puppet::Type::User::ProviderLdap: feature ldap is missing
debug: Puppet::Type::User::ProviderDirectoryservice: file /usr/bin/dscl does not exist
debug: Puppet::Type::User::ProviderUser_role_add: file roleadd does not exist
debug: Puppet::Type::File::ProviderMicrosoft_windows: feature microsoft_windows is missing
debug: /File[/etc/puppet/puppet.conf]: Autorequiring File[/etc/puppet]
debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/crl.pem]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/clientbucket]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/public_keys/services.deeproot.in.pem]: Autorequiring File[/var/lib/puppet/ssl/public_keys]
debug: /File[/var/lib/puppet/ssl/private_keys]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/log/puppet/http.log]: Autorequiring File[/var/log/puppet]
debug: /File[/var/lib/puppet/client_yaml]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/client_data]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/state/state.yaml]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/ssl/private]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/private_keys/services.deeproot.in.pem]: Autorequiring File[/var/lib/puppet/ssl/private_keys]
debug: /File[/etc/puppet/namespaceauth.conf]: Autorequiring File[/etc/puppet]
debug: /File[/var/lib/puppet/state/classes.txt]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/state/graphs]: Autorequiring File[/var/lib/puppet/state]
debug: /File[/var/lib/puppet/ssl/certs/ca.pem]: Autorequiring File[/var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/public_keys]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/certs/services.deeproot.in.pem]: Autorequiring File[/var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/ssl/certificate_requests]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/reports]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/certs]: Autorequiring File[/var/lib/puppet/ssl]
debug: Finishing transaction -613690018
debug: /File[/var/lib/puppet/ssl/private_keys/services.deeproot.in.pem]: Autorequiring File[/var/lib/puppet/ssl/private_keys]
debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/certs/ca.pem]: Autorequiring File[/var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/ssl/certs/services.deeproot.in.pem]: Autorequiring File[/var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/ssl/certs]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/log/puppet/http.log]: Autorequiring File[/var/log/puppet]
debug: /File[/etc/puppet/namespaceauth.conf]: Autorequiring File[/etc/puppet]
debug: /File[/var/lib/puppet/ssl/crl.pem]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/private]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/reports]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/public_keys]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/certificate_requests]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/public_keys/services.deeproot.in.pem]: Autorequiring File[/var/lib/puppet/ssl/public_keys]
debug: /File[/var/lib/puppet/ssl]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/private_keys]: Autorequiring File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/puppet]
debug: Finishing transaction -614148408
debug: Using cached certificate for ca
debug: Using cached certificate for services.deeproot.in
notice: Ignoring --listen on onetime run
debug: Finishing transaction -614371868
debug: Loaded state in 0.01 seconds
debug: Using cached certificate for ca
debug: Using cached certificate for services.deeproot.in
debug: Using cached certificate_revocation_list for ca
debug: catalog supports formats: b64_zlib_yaml dot marshal pson raw yaml; using pson
info: Caching catalog for services.deeproot.in
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderLaunchd: file /bin/launchctl does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
debug: Creating default schedules
debug: Loaded state in 0.02 seconds
debug: /Stage[pre]/before: requires Stage[main]
info: Applying configuration version '1296567479'
debug: Puppet::Type::Service::ProviderRunit: Executing '/usr/bin/sv status /etc/sv/test'
debug: Puppet::Type::Service::ProviderRunit: Executing '/usr/bin/sv status /etc/sv/test'
debug: Puppet::Type::Service::ProviderRunit: Executing '/usr/bin/sv status /etc/sv/test'
notice: Enabling /etc/service/test: linking /etc/sv/test -> /etc/service/test
debug: Puppet::Type::Service::ProviderRunit: Executing '/usr/bin/sv start /etc/service/test'
err: /Stage[main]/Test/Service[test]/ensure: change from stopped to running failed: Execution of '/usr/bin/sv start /etc/service/test' returned 1: fail: /etc/service/test: runsv not running
debug: Finishing transaction -614020658
debug: Storing state
debug: Stored state in 0.08 seconds
notice: Finished catalog run in 0.22 seconds
debug: Using cached certificate for ca
debug: Using cached certificate for services.deeproot.in
debug: Using cached certificate_revocation_list for ca
debug: Value of 'preferred_serialization_format' (pson) is invalid for report, using default (marshal)
debug: report supports formats: b64_zlib_yaml marshal raw yaml; using marshal

I do not have to manually invoke runsv, that is done by runsvdir ( http://smarden.org/runit/runsvdir.8.html ) . What puppet does – is the correct procedure ( except the timing considerations, I guess ) – which yes, if done manually ( symlink and then start ) works.

I see the same behaviour as Mohit. Let me explain.

To create a service in runit, you first create a service directory containing a run script and, optionally, a log dir containing a run script for the log service. Then, to enable the service, you symlink it into the directory that runsvdir is monitoring (usually /service or /var/service). runsvdir is the process that checks for new services every 5 seconds. The relevant section of man 8 runsvdir:

At  least  every five seconds runsvdir checks whether the time of last modifica-
tion, the inode, or the device, of the services directory dir has  changed.   If
so, it re-scans the service directory, and if it sees a new subdirectory, or new
symlink to a directory, in dir, it starts a new runsv(8)  process;  if  runsvdir
sees a subdirectory being removed that was previously there, it sends the corre-
sponding runsv(8) process a TERM signal, stops monitoring this process,  and  so
does not restart the runsv(8) process if it exits.

puppet should wait at least 5 seconds before checking the status of a newly created service to give runsvdir time to see it and start it.

See also #6718 for another runit provider issue.

I have just submitted a simple patch to the runit provider in this pull request: https://github.com/puppetlabs/puppet/pull/422

I also found some newsgroup discussion suggesting that it might be possible to force runit to check the service directory immediately by sending a SIGCONT (or SIGALARM?) but this is likely platform dependent? I’m not an expert in signal handling so I didn’t want to attempt the more complex fix, at least not yet.