Puppet

So your intention is to mix template sources and plain file sources. Right? Indeed that would be interesting.

However, there is one architectural problem: templates are evaluated on the server at compile time while file sources are passed as is to the client and the client then requests each file-source until the server returns a file.

This could probably be solved by treating erb:///... as special url, which will then trigger a template compile on the master. However, then we would still need to have all the variables in the current scope available and also the facts would be missing. So this would still not work.

An option might be the recent discussed post-compile processor, which could also [optimize the file sources]:http://groups.google.com/group/puppet-dev/browse_thread/thread/1c8ac2c2d6fab46#2c8d8baa39d1d717 and would be a general framework to do such things. Still at this point we would need to have the possibility to jump back at the actual scope of this file resource to get all the variables right.

Another option might be to do it right at compile time, which would probably increase the complexity of the compile process but would solve all the other problems.

So I was just writing down some ideas that came up. I’m not sure if I got everything right?!

Martin, I don’t get why the existing template system via a function doesn’t work for you? Can you perhaps post a use case that shows why it is insufficient?

Isn’t delivering templates via a remote source, when templates by definition are going to differ between hosts going to be difficult? How do we checksum?

The file extension is not an option. It’s perfectly reasonable to want to deliver an .erb file as is, without templating being involved.

Counter-question: I don’t understand why puppet distinguishes between source and content for the file type. It seems to be an implementation detail that percolated into the API and now serves to confuse users. content may well have good use-cases, but I don’t see a reason why the fileserver couldn’t evaluate files it serves using ERB under certain circumstances. If the first template in an array does not exist, return failure to the client and let it request the next one.

Content ends up in the catalog at catalog request time. Source does not, and is served directly by the puppetmaster to the client at catalog application time.

Depending on your situation, there may be significant advantages in one approach over the other. You may wish to update files separately to manifests, or you may not wish to deliver file content at catalog request time.

It’s not an implementation detail difference. There are quite different use cases, particularly if you manage mobile clients.

If you’re claiming things “should” be a certain way, the onus is on you to explain what is unsatisfactory about the current implementation and the use case that has lead you to want this.

I think there are problems with the current file delivery situation. I’m not convinced by what you’ve said here so far that this is one of them though.

This comes up often, ‘i want to create a file with a template but want to be able to specify a specific source for certain clients’ ie. [“puppet:///foo/foo.${fqdn}”, template(“foo/foo”)]

Templates are fine and all, but sometimes you just need to do something entirely different on a specific machine and playing the whole inheritance game for a simple one off is just too much effort. This feature request would make that kind of thing much easier. Combined with the fact that template() doesnt do source selection like source does it really is pretty hard to do

I also don’t think this is such a massive hurdle as people think, you could send the template content out in the catelog – as you would need to with compile time validation etc – but if there’s a potential override source specified attempt to fetch it, if it 404s put the template content down else use the file.

Very simple: there is no way to combine the “multi-select” logic available when passing an array to source with templates, and I can think of numerous cases where it would make sense to be able to override a template with a site-, domain-, or fqdn-specific one.

Isn’t this really then a feature request to make the template function support the multi-select logic then?

Maybe. It sounds about as trivial as extending the fileserver with ERB evaluation features, but I don’t know the code. In any case, I still kinda fail to understand why the user has to be confronted with the underlying design; the user just wants to put content into a file and doesn’t really care whether that content is stored in a manifest. But content generation is either template-based or static, or generated otherwise with a function.

As I pointed out above, the user does care whether files are delivered directly in the catalog, or whether they are retrieved at catalog application time.

Delivering in catalog: * increases catalog size * delivers files every time client requests catalog * guarantees file contents can be applied offline

Retrieving at application time: * allows you to change files separately to manifests * reduces catalog size * delivers files to clients only when needed * guarantees file contents cannot be applied offline

It’s not a matter of exposing an underlying design. There are advantages and disadvantages to each approach.

Nigel, thanks for following up. Yes, I think having a template-multi-select would be great. I suppose you could even combine it with existing syntax: an array provides alternatives for a first-found-is-used basis (like source select), and multiple arguments cause concatenation as it does right now.

Respecting the functional difference between content and source, it would still be nice if the syntax were unified:

1. both source and content can do multi-select (see above);
2. both source and content can concatenate multiple files;
3. both source and content can use all kinds of URIs (I don’t see a reason why a template could not be fetched from remote by the server before being evaluated).