Puppet

I have an example. Lets say I have an exec, which updates apt:

  exec {
    'refresh_apt':
      command => '/usr/bin/apt-get update && sleep 1',
      refreshonly => true,
      subscribe => [ File['/etc/apt/apt.conf.d'], Config_file['/etc/apt/sources.list'] ];

when I install packages, i want to make sure that my apt information is up-to-date before I do the install. So how can I trigger this exec before a package resource is realized? I cannot do “Package { require => Exec[refresh_apt] }” because ‘refreshonly’ is only triggered by ‘notify’ or ‘subscribe’. I can’t use ‘notify’ because that will be run after the package resource has been realized. Subscribing to the exec isn’t going to do anything either.

So most people seem to be ok with the cost of a single apt-get update on each run. I take it this is not acceptable Micah and you have a slow update process thus wanting the “refreshonly”?

Another method people seem to use is to have an onlyif/unless that checks a timestamp you drop at the end of the exec.

Are either of these workarounds acceptable?

Nigel, it is not sufficient to run “apt-get update” at the start of the run — new repos can have been added in the interim. The simplest workaround is to put all repo-handling in a separate stage.

My case isn’t related to Apt at all, I just want a method where I can have a chain of Exec’s without duplicating the “onlyif” terms in each and every instance.

So this is exactly the problem I used to solve by making sure that repositories were added before apt-get update, and that packages were added after. like:

AptRepo { before => Class["apt::update"] }
Package { require => Class["apt::update"] }

You can use stages to solve this, you can also use resource defaults or defined types.

Great. Now solve the problem in the issue description :–)

:) That’s why I said:

So most people seem to be ok with the cost of a single apt-get update on each run. I take it this is not acceptable Micah and you have a slow update process thus wanting the “refreshonly”?

We don’t currently have a solution other than:

• wear the cost of the single run each time
• make the apt-get update dump a timestamp file, use onlyif/unless to only run if older than X hrs.
• Dump complex logic around apt-get update into a script that does all the checking internally if you have multiple conditions. The script runs each time, but it may not actually run apt-get update.

My use case has nothing to do with “apt-get”. Exec[“prereq”] can have destructive qualities if the onlyif in Exec[“main”] is not satisfied. This means that the onlyif has to be duplicated, and if there is a long chain of such Execs, the onlyifs will progressively become more complicated.

The current workaround is to put the logic in a shell script, since it is so much simpler than reams of duplicated code, but IMHO this is not a bad fit for the Puppet DSL to handle on its own.

Can you provide a more concrete example? It feels as if there is a better way to achieve what you want, but it’s a little difficult at a purely theoretical level.

I think I understand what Kjetil wants. Given these resources:

exec { "A":
    command => "A.command",
    onlyif => "A.onlyif",
    require => Exec["B"],
}
exec { "B":
    command => "B.command",
}

the current behaviour is:

run B.command;
run A.onlyif;
if A.onlyif was sucessful {
    run A.command
}

I believe that Kjetil wants a way to express the following behaviour:

run A.onlyif;
if A.onlyif was sucessful {
    run B.command;
    run A.command;
}

I believe that Kjetil wants a way to express the following behaviour:

[…]

If you didn’t miss the refreshonly on B by accident, I don’t think that this change, as stated by you, would be a good idea.

If you missed the refreshonly, then yes, this could probably be changed and would make sense, although the name refreshonly would then be wrong.

Peter Meier wrote:

If you didn’t miss the refreshonly on B by accident, I don’t think that this change, as stated by you, would be a good idea.

I was simply attempting to explain his desired behaviour, and that he wants some way to be able to express the desired behaviour; I did not intend to make any suggestions about what syntax should be used to express the desired new behaviour, and I certainly did not intend to imply that the existing behaviour should be changed.

I think that the existing behaviour with the existing syntax is sensible. If new behaviour is desired, then I think it needs new syntax, but I have no suggestions for the new syntax. I do not think that overloading “refreshonly” for this purpose would be a good idea.

Nigel Kersten wrote:

So most people seem to be ok with the cost of a single apt-get update on each run. I take it this is not acceptable Micah and you have a slow update process thus wanting the “refreshonly”?

I dont mind a single apt-get update that is run before a package is to be installed, but having an apt-get update run on every puppet run is a bit much, when its not needed. There are various problems it exacerbates: a. development offline is impossible, if apt-get update is mandatory; b. slow internet connections make puppet run times take very long; c. unnecessary querying of the archive on every run by every client is not being a nice netizen, I dont like to involuntarily DDoS the upstream servers if possible, especially if you have hundreds of machines doing this on every puppet run; d. while puppet is doing its pointless apt-get update, no other apt operations can be done either through scripts/cronjobs on the system, or by administrators because the package db is locked. This is annoying if you are running something like cron-apt which periodically cant do its work because puppet is running, so you get cron errors. Its annoying as an admin because you cant do anything until the puppet run is finished, it would be acceptable if the run was actually doing the update because it was going to do a package operation, but to do it because of a limitation in puppet is annoying.

Another method people seem to use is to have an onlyif/unless that checks a timestamp you drop at the end of the exec.

I’m not sure what this solves except to make it so the exec is only run at certain intervals, which is what cron-apt does. What we want puppet to do is to run an apt-get update before a package is installed, not at random times, and not at every run.

Are either of these workarounds acceptable?

I admin that this is a bit of a corner case… but it is surprising that this is not possible, when it seems like a reasonable thing to want to do.