Bug #1454

puppetd restart, sometimes failed.

Added by Vadim Vatlin almost 4 years ago. Updated about 1 year ago.

Status:Closed Start date:07/28/2008
Priority:Normal Due date:
Assignee:Nigel Kersten % Done:

0%
Category:-
Target version:-
Affected Puppet version:0.24.7 Branch:
Keywords:
Votes: 0

Description

Hi. I created module puppetd, which restart puppetd itself:

service { "puppetd":
                ensure     => running,
                enable     => true,
                path       => [ "/etc/rc.d",
                                "/usr/local/etc/rc.d" ],
                pattern   => "puppetd",
                hasstatus  => true,
                hasrestart => true;
        }

But it doesn’t work: tail -f /var/log/messages

Jul 14 16:32:57 hostname puppetd[580]: (//Node[fqdn]/base_ac/puppetd/Service[puppetd]/enable) enable changed 'false' to 'true'
Jul 14 16:32:57 hostname puppetd[580]: (//Node[fqdn]/base_ac/puppetd/Service[puppetd]) Triggering 'refresh' from 1 dependencies
Jul 14 16:32:57 hostname puppetd[580]: Caught TERM; shutting down
Jul 14 16:32:57 hostname puppetd[580]: Shutting down
Jul 14 16:33:18 hostname puppetd[580]: Reparsing /usr/local/etc/puppet/puppet.conf
Jul 14 16:33:18 hostname puppetd[580]: Could not run Puppet::Network::Client::Master: Thread(#) not locked.

ps aux -www | grep puppetd

root    580  0.0  1.8 65736 65252  ??  Ss   Sat11AM   0:55.87 /usr/local/bin/ruby18 /usr/local/bin/puppetd --rundir /var/run/puppet
root  58855  0.0  0.0  1868  1460  ??  Is   Sat12PM   0:00.01 /bin/sh /usr/local/etc/rc.d/puppetd restart
root  58862  0.0  0.0  1868  1492  ??  S    Sat12PM   0:24.50 /bin/sh /usr/local/etc/rc.d/puppetd restart

OS: FreeBSD 6.3

Related issues

duplicated by Puppet - Bug #2267: puppetd "thread not locked" on CentOS 4 Duplicate 05/14/2009

History

#1
Updated by Luke Kanies almost 4 years ago

  • Status changed from Unreviewed to Accepted

Does this happen every time? Is anyone else experiencing this problem?

#2
Updated by Vadim Vatlin almost 4 years ago

luke wrote:

Does this happen every time? Is anyone else experiencing this problem?

When I managed a few resources puppetd restarted successfully. But When I began to manage about 200 resources, puppetd failed to restart with such error.

I have two different installation and this problem occur on both of them.

#3
Updated by Vadim Vatlin almost 4 years ago

silent-ion wrote:

luke wrote:

Does this happen every time? Is anyone else experiencing this problem?

When I managed a few resources puppetd restarted successfully. But When I began to manage about 200 resources, puppetd failed to restart with such error.

I have two different installation and this problem occur on both of them.

And with puppet 0.24.5 I get the same result.

#4
Updated by Luke Kanies almost 4 years ago

  • Target version set to 0.24.6

#5
Updated by Andrew Shafer over 3 years ago

  • Status changed from Accepted to Needs More Information
  • Assignee set to Andrew Shafer

I’m having a hard time reproducing, but I think I can see what is happening.

Can someone reproduce this behavior with —trace and share the stacktrace?

Or give me a simple mechanism to reproduce…

#6
Updated by Andrew Shafer over 3 years ago

I sent an email to the person who opened this issue, but I haven’t got a reply yet and have no way to reproduce this. Maybe they are on holiday?

If someone can reliably reproduce, we can make a run at fixing it.

Looking at the original bug, it looks like it might be a problem with the rc script or something in the puppet manifest that is trying to restart it twice.

Looking at the ps output, you can see there are two restart processes.

The Thread not locked exception is from synchronize, and it looks like the only way that could happen is if the unlocking thread is not the thread that locked.

This will sit open until someone can work with me to reproduce.

#7
Updated by James Turnbull over 3 years ago

  • Target version changed from 0.24.6 to 4

#8
Updated by Rob Daugherty about 3 years ago

  • Affected Puppet version changed from 0.24.4 to 0.24.7

I am seeing this behavior during the shutdown process of puppetd. I would be happy to provide any traces I can for resolution. Currently I have to kill -9 the process manually in order for it to shut down.

#9
Updated by achemze - about 3 years ago

I’m also seeing this problem. What do i need to do to help out?

#10
Updated by Luke Kanies about 3 years ago

achemze wrote:

I’m also seeing this problem. What do i need to do to help out?

Please run puppetd with —debug —trace, and provide the stack trace when it fails to restart.

#11
Updated by David Bishop over 2 years ago

  • Assignee deleted (Andrew Shafer)

I had the same issues as this ticket and #2267, which was solved by fixing my puppet.conf. I had a malformed libdir definition ( libdir = $libdir:/usr/local/puppet-dashboard/lib/puppet, if you must know). Fixing that solved my problem. Obviously, I shouldn’t have screwed up the definition, but a nicer error message would have been handy.

#12
Updated by Nigel Kersten over 1 year ago

  • Status changed from Needs More Information to Closed
  • Assignee set to Nigel Kersten

As per the below thread, we’re more aggressively closing tickets whose state is unsure, particularly old tickets with little to no inactivity in a long time.

You are free to reopen them.

http://groups.google.com/group/puppet-users/browse_thread/thread/a040cb9bc5c5b647

#13
Updated by James Turnbull about 1 year ago

  • Target version deleted (4)

Also available in: Atom PDF