Bug #2675
File path comparisons sensitive to trailing slashes
| Status: | Closed | Start date: | 09/24/2009 | |
|---|---|---|---|---|
| Priority: | Low | Due date: | ||
| Assignee: | - | % Done: | 0% |
|
| Category: | file | |||
| Target version: | - | |||
| Affected Puppet version: | 0.25.1rc2 | Branch: | http://github.com/jes5199/puppet/tree/ticket/0.25.x/2675 | |
| Keywords: | ||||
| Votes: | 0 |
Description
I just noticed this error on 0.25.x. It used to work on 0.24.8.
$ cat /tmp/test1.pp
file { "/tmp/dir1": ensure => directory }
file { "/tmp/dir1/file1": ensure => present, require => File["/tmp/dir1"] }
$ cat /tmp/test2.pp
file { "/tmp/dir2/": ensure => directory }
file { "/tmp/dir2/file2": ensure => present, require => File["/tmp/dir2"] }
$ cat /tmp/test3.pp
file { "/tmp/dir3/": ensure => directory }
file { "/tmp/dir3/file3": ensure => present, require => File["/tmp/dir3/"] }
$ puppet /tmp/test1.pp
notice: //File[/tmp/dir1]/ensure: created
notice: //File[/tmp/dir1/file1]/ensure: created
$ puppet /tmp/test2.pp
notice: //File[/tmp/dir2]/ensure: created
notice: //File[/tmp/dir2/file2]/ensure: created
$ puppet /tmp/test3.pp
Could not find dependency File[/tmp/dir3/] for File[/tmp/dir3/file3] at /tmp/test3.pp:2
I think I have a working patch. I’ll submit it in a few minutes.
Related issues
History
Updated by Marc Fournier over 2 years ago
This bug was introduced in commit cc09c1af. Using File.split on a filename ending with a slash drops this ending slash. A resource name ending with a slash gets silently “renamed” and this breaks the dependencies.
Updated by James Turnbull over 2 years ago
- Status changed from Unreviewed to Needs Decision
- Assignee set to Luke Kanies
Updated by Luke Kanies over 2 years ago
This is kind of the inverse of #1042, and I think it has the right approach.
Really, the correct approach is that the / shouldn’t have an affect either way — I absolutely despise the way tools like cp behave differently depending on whether there’s a trailing slash.
I.e., I think this is a duplicate of #1042, and this patch will only work to the extent that you know exactly whether the file has a trailing slash.
Updated by Luke Kanies over 2 years ago
- Status changed from Needs Decision to Duplicate
I’m closing this as a dupe of #1042, since they really are the same problem.
This is actually a pretty hard problem to fix, and a very easy problem to work around, so I don’t think it qualifies as High priority.
Updated by Marc Fournier over 2 years ago
- Status changed from Duplicate to Re-opened
- Priority changed from High to Low
- Affected Puppet version changed from 0.25.1rc1 to 0.25.1rc2
Sorry for reopening this bug. I would like to draw attention to this thread: http://groups.google.com/group/puppet-dev/browse_thread/thread/d36450a5747be593/25d9edad436dbe51?lnk=gst&q=2675
Maybe the solution is just that people fix their manifests with file resources ending with a slash. In this case this issue indeed is a duplicate of #1042. If not, it deserves to stay open as a reminder.
Updated by James Turnbull over 2 years ago
- Target version changed from 0.25.1 to 0.25.2
Updated by Markus Roberts over 2 years ago
- Branch set to http://github.com/MarkusQ/puppet/tree/ticket/master/2675
Branch at http://github.com/MarkusQ/puppet/tree/ticket/master/2675
Updated by Markus Roberts over 2 years ago
- Status changed from Re-opened to Code Insufficient
- Assignee changed from Luke Kanies to Markus Roberts
No one (including me) was thrilled with the rush patch intended for 0.25.1, so I’ve got another, more general/cleaner solution in the queue (coming soen).
Updated by Markus Roberts over 2 years ago
New branch, rebased off of 0.25.1 and restructured as per discussions, making parameters (rather than types) responsible for value canonicalization.
Updated by Markus Roberts over 2 years ago
- Status changed from Code Insufficient to In Topic Branch Pending Review
Updated by Markus Roberts about 2 years ago
- Status changed from In Topic Branch Pending Review to Ready For Checkin
I just updated (rebased) the branch & ported the key potentially conflicting change (m flag on the regexp) from #2770 to simplify the eventual merge conflict.
Updated by James Turnbull about 2 years ago
- Status changed from Ready For Checkin to Code Insufficient
This doesn’t apply to 0.25.x cleanly.
Updated by Markus Roberts about 2 years ago
- Branch changed from http://github.com/MarkusQ/puppet/tree/ticket/master/2675 to http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/2675
Doh, my bad. Try the rebased one.
http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/2675
Updated by James Turnbull about 2 years ago
Still no go.
Auto-merged lib/puppet/resource/catalog.rb Auto-merged lib/puppet/resource/reference.rb CONFLICT (content): Merge conflict in lib/puppet/resource/reference.rb Automatic cherry-pick failed. After resolving the conflicts, mark the corrected paths with 'git add' or 'git rm ' and commit the result. When commiting, use the option '-c f3e48f9' to retain authorship and message.
Updated by Markus Roberts about 2 years ago
This wasn’t a rebasing problem, it was the expected conflict with #2770. Fixed now.
Updated by Markus Roberts about 2 years ago
- Status changed from Code Insufficient to Ready For Checkin
Updated by James Turnbull about 2 years ago
- Status changed from Ready For Checkin to Closed
Pushed in commit:ca56aa7e5849a5489e8d38e29b25ea934caafcd7 in branch 0.25.x
Updated by Marc Fournier about 2 years ago
- Status changed from Closed to Re-opened
I just tried to update my puppetmaster to 0.25.2rc2, and another variant of this bug resurfaced. Now every client, even those < 0.25, fail with this error:
info: Retrieving plugins info: Retrieving facts err: Could not retrieve catalog: Could not find resource File[/etc/pki/] when converting to resource resources on node host.example.com [...]
The resource triggering the error is quite simple:
file { ["/etc/pki/", "/etc/pki/rpm-gpg/"]:
ensure => directory,
}
Rewriting it like this solves the problem:
file { ["/etc/pki", "/etc/pki/rpm-gpg"]:
ensure => directory,
}
As I have dozens of resources ending with a slash, my initial question was: should people edit every “file” resource ending with a slash and stripping it off ? Or should puppet treat File[“/etc/pki”] and File[“/etc/pki/”] as the same ?
Using the 3 demo files described in the inital report can be used to reproduce the error:
$ puppet test1.pp notice: //File[/tmp/dir1]/ensure: created notice: //File[/tmp/dir1/file1]/ensure: created $ puppet test2.pp Could not find resource File[/tmp/dir2/] when converting to resource resources on node host.example.com $ puppet test3.pp Could not find resource File[/tmp/dir3/] when converting to resource resources on node host.example.com
Updated by Markus Roberts about 2 years ago
- Status changed from Re-opened to Investigating
Updated by Jesse Wolfe about 2 years ago
- Status changed from Investigating to In Topic Branch Pending Review
- Assignee changed from Markus Roberts to Jesse Wolfe
- Branch changed from http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/2675 to http://github.com/jes5199/puppet/tree/ticket/0.25.x/2675
There was an intermittent bug in Puppet::Parser::Resource::Reference,
during initialization, and object could sometimes have its title set
before its type is set. This prevented the title from going through
type-specific canonicalization.
Updated by James Turnbull about 2 years ago
- Assignee changed from Jesse Wolfe to Marc Fournier
Marc – can you confirm so we can merge and release please! Thanks!
Updated by Markus Roberts about 2 years ago
- Status changed from In Topic Branch Pending Review to Ready For Checkin
This was also biting us as a test indeterminacy.
Updated by James Turnbull about 2 years ago
- Status changed from Ready For Checkin to Closed
Pushed in commit:13cbf043c6e16c14b0ab9fccd5738a8c9e5925b3 in branch 0.25.x
Updated by Jeroen Meeuwen about 2 years ago
- Status changed from Closed to Re-opened
A trailing slash in a File resource is still being stripped off. I found a working workaround to be to not strip off the trailing slash.
http://git.ogd.nl/people?p=jeroenm2/public_git/puppet.git;a=commitdiff;h=524e85160c275effaf8a5550dbb0c22a0fa1e651 (or remote http://git.ogd.nl/~jeroenm2/puppet.git, branch ‘tickets/2675’
Updated by Markus Roberts about 2 years ago
- Subject changed from ending slash in directory name gets stripped off to File path comparisons sensitive to trailing slashes
- Status changed from Re-opened to Needs More Information
- Assignee deleted (
Marc Fournier)
I’m unclear on what that patch is supposed to accomplish; it appears that it would simply reintroduce the bug, by making “/some/path/” and “/some/path” be internally distinct when they are not supposed to be.
Updated by James Turnbull about 2 years ago
- Target version changed from 0.25.2 to 0.25.3
Bumping this and discussion – my inclination is to what for feedback before closing though.
Updated by Jeroen Meeuwen about 2 years ago
Markus Roberts wrote:
I’m unclear on what that patch is supposed to accomplish; it appears that it would simply reintroduce the bug, by making “/some/path/” and “/some/path” be internally distinct when they are not supposed to be.
The former solution provided to the problem as you state it though, is not allowing to use a trailing slash in a directory path at all. Usually though, directory paths do have a trailing slash (documentation, even freaking tab completion, etc. and thus why not puppet manifests also)?
The use of this trailing slash is to be able to significantly distinct between a file and a directory path when using dependencies; the ‘require => File[“/some/path/”]’ may be somewhere other then the ‘file { “/some/path/”: ensure => directory }’.
I’d say that reintroducing the original “bug” where Filecommit:“/some/path/”] and File[“/some/path” are not the same while they should be, is irrelevant if you consider the solution provided in 0.25.x is eliminating the possibility to conform to very common standards, and therefore actually preventing stuff from happening.
Updated by Jeroen Meeuwen about 2 years ago
Is it reasonable to look at lib/puppet/type/file/ensure.rb to mangle/canonicalize (or alias?) file/link/directory paths, rather then not allowing trailing slashes to all paths for all types of ensure?
Updated by Jeroen Meeuwen about 2 years ago
- Status changed from Needs More Information to Closed
Mysteriously (or not so mysteriously) upgrading both the server and the client to the 0.25.2 release solved all my problems. It seems I was under the mistaken impression that trailing slashes could no longer be used -as that was what I thought my error messages indicated not finding resource dependencies anymore- I now understand how that would not have made any sense. Apologies to all!
Updated by James Turnbull about 2 years ago
- Target version deleted (
0.25.3)