Bug #9112

"Duplicate generated resource; skipping" when using Tidy with File.

Added by Trevor Vaughan 9 months ago. Updated 15 days ago.

Status:Accepted Start date:08/18/2011
Priority:Urgent Due date:
Assignee:Daniel Pittman % Done:

0%
Category:file
Target version:-
Affected Puppet version: Branch:
Keywords:tidy, file, duplicate resource
Votes: 1

Description

The following code produces the ‘info’ statement “Duplicate generated resource; skipping” when run under ‘puppet agent’. ‘puppet apply’ does not appear to have the same issue.

file { '/tmp/foo':
  ensure => 'directory',
  mode => '644'
}

file { [
  '/tmp/foo/bar',
  '/tmp/foo/bar1',
  '/tmp/foo/bar2'
  ]:
  ensure => 'file',
  owner => 'root',
  group => 'root',
  mode => '644',
  content => "test\n"
}

exec { 'make_baz':
  command => '/bin/echo "test" > /tmp/foo/baz',
  require => File['/tmp/foo']
}

tidy { '/tmp/foo':
  size => '0b',
  recurse => 'true',
  rmdirs => 'true'
}

Output:

notice: /Stage[main]//Node[test.test.net]/Tidy[/tmp/foo]: Tidying File[/tmp/foo]
notice: /Stage[main]//Node[test.test.net]/Tidy[/tmp/foo]: Tidying File[/tmp/foo/bar1]
notice: /Stage[main]//Node[test.test.net]/Tidy[/tmp/foo]: Tidying File[/tmp/foo/baz]
notice: /Stage[main]//Node[test.test.net]/Tidy[/tmp/foo]: Tidying File[/tmp/foo/bar]
notice: /Stage[main]//Node[test.test.net]/Tidy[/tmp/foo]: Tidying File[/tmp/foo/bar2]
info: /File[/tmp/foo/bar2]: Duplicate generated resource; skipping
info: /File[/tmp/foo/bar1]: Duplicate generated resource; skipping
info: /File[/tmp/foo/bar]: Duplicate generated resource; skipping
info: /File[/tmp/foo]: Duplicate generated resource; skipping

tidy_fix.patch (667 Bytes) Trevor Vaughan, 04/23/2012 09:11 am

History

#1
Updated by James Turnbull 9 months ago

  • Category set to file
  • Status changed from Unreviewed to Accepted
  • Assignee set to Matt Robinson
  • Priority changed from Normal to High

#2
Updated by Trevor Vaughan 8 months ago

Any chance of this getting into 2.7.4?

#3
Updated by Daniel Pittman 8 months ago

Trevor Vaughan wrote:

Any chance of this getting into 2.7.4?

G'day Trevor.

We have this on the list of things to take a look at on Friday, and should be able to make a decision around it then.

Thanks for the report, and sorry it has taken so long to get attention.

#4
Updated by Trevor Vaughan 8 months ago

  • Affected Puppet version changed from 2.7.3 to 2.7.5

#5
Updated by Trevor Vaughan 7 months ago

  • Affected Puppet version changed from 2.7.5 to 2.7.6

#6
Updated by James Turnbull 7 months ago

  • Assignee changed from Matt Robinson to Daniel Pittman

#7
Updated by Trevor Vaughan 6 months ago

  • Affected Puppet version changed from 2.7.6 to 2.7.7rc2

I’m still not certain if this is causing issues but I’m still tracking it.

#8
Updated by Trevor Vaughan 6 months ago

  • Priority changed from High to Urgent

I was finally able to pin this down a bit more and there is a definitely serious issue with the combination of file and tidy.

Pre-Setup:

/root/foo /root/foo/blah1 /root/foo/blah2 /root/foo/blah3

Example Code:

tidy { "/root/bar":
  size => '0b',
  matches => '*',
  recurse => 'true'
}

file { "/root/bar":
  ensure => 'directory',
  source => 'file:///root/foo',
  recurse => 'true'
}

file { "/root/bar/test3":
  ensure => 'file',
  content => 'blah'
}

After creating this code and running it once with ‘puppet apply’, remove the file ‘blah2’ from the directory ‘bar’ and run ‘puppet apply’ again.

With subsequent runs of puppet, it will remove random files from ‘bar’ and create random other ones. This causes extremely non-deterministic behaviour.

However, this does not always manifest itself. It appears to only happen when a file that is usually there gets removed by some other means. However, I think that it’s related to the ‘duplicate generated resource’ material above even though this code doesn’t spawn that warning.

The use case for doing something like this is to ensure that the only files in a directory are those placed by puppet. I know that this can be done using ‘file’ with ‘purge’, but this did not always work properly in legacy versions of puppet and the presented combination should either throw an error or work in a deterministic manner.

#9
Updated by Trevor Vaughan 3 months ago

  • Affected Puppet version changed from 2.7.7rc2 to 2.7.11

#10
Updated by Trevor Vaughan about 1 month ago

Any update on this?

#11
Updated by Trevor Vaughan about 1 month ago

  • File tidy_fix.patch added
  • Affected Puppet version changed from 2.7.11 to 2.7.14rc1

Added a patch that fixes this issue. Probably not as elegant as it could be but verified to have proper functionality.

#12
Updated by Max Martin 19 days ago

  • Affected Puppet version deleted (2.7.14rc1)

Ping-Daniel, several months ago you indicated this was going to be entering the queue. Has this been patched? Still open and marked as “Urgent” at the moment.

#13
Updated by Daniel Pittman 15 days ago

So, I looked over the code that Trevor submitted, and it looked correct but worrying.

Specifically, it only generates things that are not in catalog – but I am not certain if you can generate duplicate entries after the fact, based on the order we call the code that generates things through local recursion.

I wanted to work out how that worked, and see if this was a valid concern or not before this got merged.

My concrete concern is that something like this might – or, worse, sometimes might – still fail:

tidy { "/tmp": } -> file { "/tmp/foo": recurse => local }

Also available in: Atom PDF