Bug #5200
Many-to-many implementation of stages needlessly inefficient
| Status: | Closed | Start date: | 11/03/2010 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | % Done: | 0% |
||
| Category: | performance | |||
| Target version: | 2.7.x | |||
| Affected Puppet version: | 2.6.0 | Branch: | ||
| Keywords: | ||||
| Votes: | 2 |
Description
Trevor Vaughan reports:
Stages appear, by default, to generate a many-to-many dependency graph. By adding 'pre' and 'post' stages that contain a single item, the graphs are vastly reduced in complexity and we're seeing something like a 3X speedup in graph execution in simple test cases. I've attached a couple of puppet files illustrating the issue with their associated graphs. If the graphs could be auto-reduced to a single point prior to execution automatically, it would be quite helpful.
I believe the savings could be even greater in complex situations. We had discussed addressing this internally by creating a whit (null resource) but the idea apparently did not make it into code. It should be an easy win.
Related issues
History
Updated by Markus Roberts over 1 year ago
- Assignee set to Markus Roberts
- Target version set to 2.7.x
Luke, Teyo & Nigel have all agreed that this would be very nice to get in Statler.
Updated by Markus Roberts over 1 year ago
Just to clarify, I linked the two autorequire tickets due to implicit conceptual collision between the issues discussed on the respective threads on puppet-dev; specifically, those tickets propose expanding / extending the autorequire functionality while Luke’s refactor/master/3691-no_relationship_graph branch, which he suggested as an alternate resolution on the thread for this ticket removes autorequire functionality and it’s not immediately clear (to me at least) how it could be restored in that model.
I think eliminating the second graph as Luke’s branch does is the right idea, and I think representing containment with bracket node as Tervor, Jesse and other have advocated is better than adding edge types. But I’m not yet seeing how either idea can work in actual code. Just adding bracket nodes as an added complication in splice! looks like the easiest, though significantly less efficient. In short, I’m finding myself in agreement with pretty much everyone on the details and with almost no one on the big picture.
Updated by Markus Roberts about 1 year ago
To update: Jesse & I have a working sketch of the bracket solution which keeps the “two graphs” structure for now. It’s turning out to be a needed step for #6911 and will probably be folded in with that.
Updated by Markus Roberts about 1 year ago
The branch on #6911 now contains a sketch of the fix for this, with one known bug and a small amount of cleanup left to do.
Updated by Joshua Lifton 5 months ago
- Assignee deleted (
Markus Roberts)
This issue was assigned to a former Puppet Labs employee. Adding back to the pool of unreviewed issues.
Updated by Joshua Lifton 5 months ago
This issue was assigned to a former Puppet Labs employee. Adding back to the pool of unreviewed issues.
Updated by Ben Hughes 4 months ago
- Description updated (diff)
- Status changed from Accepted to Unreviewed
Updated by Nigel Kersten 4 months ago
- Description updated (diff)
- Status changed from Unreviewed to Needs More Information
- Assignee set to Nick Lewis
Nick, didn’t this get resolved when we had the same issue with class –> class relationships and the Whit implementation was meant to solve it?
Updated by Nick Lewis about 1 month ago
- Status changed from Needs More Information to Closed
This has been resolved for a year or so.