Bug #1844
"Too many open files" when copying directory structure
| Status: | Closed | Start date: | 12/29/2008 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% |
|
| Category: | - | |||
| Target version: | - | |||
| Affected Puppet version: | 0.24.7 | Branch: | ||
| Keywords: | ||||
| Votes: | 0 |
Description
My goal is to sync an entire directory structure required for deploying an application: we currently use rsync for this.
My testing manifest reads (with some obfuscation):
class policy::solaris { file { “/destination/destdir”:
source => "puppet:///policy/test/sourcedir",
recurse => true
} }
This copies the contents of modules/policy/files/test/sourcedir/ into /destination/destdir/ on my target system. The sourcedir/ has 5282 files underneath it.
The puppetd on my destination system starts chewing through this, but eventually starts filling the log with the like messages:
err: //Node[default]/baseclass/policy::solaris/File[/destination/destdir/foo/bar/include]: Failed to retrieve current state of resource: Too many open files – socket(2) Could not describe /policy/test/sourcedir/foo/bar/include: Too many open files – socket(2) err: Could not call fileserver.list: #<Errno::EMFILE: Too many open files – socket(2)>
No messages are seen on the puppetmasterd log, except that it is sending the files to the destination puppetd.
Both systems are Solaris 10 Update 6 on x86.
Related issues
History
Updated by Luke Kanies about 3 years ago
- Status changed from Unreviewed to Needs More Information
If you look at an lsof, what files does it actually have open?
Either way, though, I highly recommend you stick with rsync for now. Hopefully someday Puppet will be fast enough to replace it, but it’s definitely not yet.
Updated by Ed Powell about 3 years ago
I ran lsof against the master and the client as it ran into the problem.
On the client side, it had 249 TCP connections to the master: 194 in CLOSE_WAIT, 56 in ESTABLISHED. All connections were to the master’s port 8140.
The master only showed 56 connections to the client, all established. Filtering out the TCP connections from the lsof output showed 51 entries on the master, 43 on the client, all of which appear to be normal output.
Time to investigate an rsync type, if one hasn’t already been worked out… :)
Updated by Luke Kanies almost 3 years ago
This should be fixed now that #961 is fixed.
Updated by Luke Kanies almost 3 years ago
- Status changed from Needs More Information to Closed