Bug #2701
Memory Leak in Puppet state
| Status: | Accepted | Start date: | 10/06/2009 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% |
|
| Category: | plumbing | |||
| Target version: | 2.7.x | |||
| Affected Puppet version: | 0.24.8 | Branch: | ||
| Keywords: | ||||
| Votes: | 0 |
Description
I have a puppet daemon running tidy to clean up a very busy directory (cleans several thousand files each week). Using a tidy object:
tidy { “/tps/reports”: age => 1d, recurse => true, rmdirs => true, loglevel => info, type => mtime, }
The tidy works well, but every file gets recorded into state.yaml, even the ones that were removed several months ago. When Puppet reads in this yaml file it gets progressively bigger which consumes a significant chunk of memory on the host (I just killed a Puppet daemon with approx 900Mb resident, for example – this had been running approx. 2 weeks)
My workaround for now is that I wipe the state.yaml file and restart puppet periodically – usually every couple of weeks or so… Another workaround I have is to run the attached script periodically and restart puppet. The script culls all references to Tidy objects from the database – preserving the rest of the state database.
History
Updated by Greg Boug over 2 years ago
- File deleted (
CleanState.txt)
Updated by Luke Kanies over 2 years ago
- Category set to plumbing
- Status changed from Unreviewed to Accepted
- Target version set to 2.6.0
The only real choice here is to replace the yaml file with a database of some kind that can do partial loading. This is pretty complicated, and likely won’t actually make it into rowlf, but I’d like to try, anyway.
Updated by Markus Roberts over 2 years ago
- Assignee set to Markus Roberts
Updated by Trevor Vaughan over 2 years ago
Would it be possible to have an internal flag in Type or File that could prevent an object from being written to the yaml file in the first place?
It seems like in both this case, and the case of a recursive file operation in general, you want to record the top level object, but then let nature take its course on the file(s) below the top level.
I’m not entirely sure what this would do to performance though.
It would almost be nice to bypass the creation of the various file objects altogether and just use the native ruby constructs for recursive actions.
Updated by James Turnbull over 2 years ago
- Target version changed from 2.6.0 to 2.7.x
Updated by Joshua Lifton 5 months ago
- Assignee deleted (
Markus Roberts)
This issue was assigned to a former Puppet Labs employee. Adding back to the pool of unreviewed issues.
Updated by Joshua Lifton 5 months ago
This issue was assigned to a former Puppet Labs employee. Adding back to the pool of unreviewed issues.
Updated by Ben Hughes 4 months ago
- Description updated (diff)
- Status changed from Accepted to Unreviewed
Updated by Luke Kanies 4 months ago
- Description updated (diff)
- Status changed from Unreviewed to Accepted
I don’t know why this was marked as unreviewed – nothing has changed on it that I can tell.