Bug #2889

puppetrun gives error 500 with no explanation

Added by Alex Urbanowicz over 2 years ago. Updated about 1 year ago.

Status:Closed Start date:12/04/2009
Priority:Low Due date:
Assignee:James Turnbull % Done:

0%
Category:-
Target version:-
Affected Puppet version:0.24.8 Branch:
Keywords:puppetrun 500
Votes: 0

Description

Hello!

We have problem using puppetrun:

[root@stonka alex]# puppetrun --trace --debug --host hostname.fqdn
debug: Parsing /etc/puppet/puppet.conf
debug: Puppet::Network::Client::Runner: defining puppetrunner.run
Triggering hostname.fqdn
debug: Calling puppetrunner.run
err: Could not call puppetrunner.run: #
Host hostname.fqdn failed: HTTP-Error: 500 Internal Server Error 
hostname.fqdn finished with exit code 2
Failed: hostname.fqdn

on the puppet the —trace —debug —verbose log looks as follows:

Dec  4 18:06:57 hostname puppetd[30452]: (access[fileserver]) allowing puppet.fqdn access

namespaceauth.conf contents is:

[fileserver]
    allow puppet.fqdn

[puppetmaster]
    allow puppet.fqdn

[puppetrunner]
    allow *.fqdn

[puppetbucket]
    allow *.fqdn

[puppetreports]
    allow puppet.fqdn

[resource]
    allow puppet.fqdn

Strace gives unconclusive results. I suspect the problem is somewhat related to the network setup (the puppet I try to trigger has no access to reverse dns of the fqdn domain, and uses /etc/hosts lookups) but I found no way to confirm this.

dev24-namespaceauth.conf (482 Bytes) konrad rzentarzewski, 12/04/2009 07:20 pm

dev24-puppetd.strace (86 kB) konrad rzentarzewski, 12/04/2009 07:20 pm

History

#1
Updated by Alex Urbanowicz over 2 years ago

unmangled namespaceauthd.conf:

[fileserver]

allow puppet.fqdn

[puppetmaster]

allow puppet.fqdn

[puppetrunner]

allow *.fqdn

[puppetbucket]

allow *.fqdn

[puppetreports]

allow puppet.fqdn

[resource]

allow puppet.fqdn

#2
Updated by konrad rzentarzewski over 2 years ago

seems to be similiar problem to (unanswered) http://markmail.org/message/ltinzpv63l7hv6ll#query:puppetrun%20Overriding%20with%20cert%20name+page:1+mid:kma4cw7sautesjnd+state:results

or completely unrelated.

it’s certain that “500 Internal Server Error” is not the best feedback from client. it might be authentication issue, framework bug or infrastructure problem, with no distinction on error received from client (puppetd).

#3
Updated by konrad rzentarzewski over 2 years ago

i’ve found obscured exception in process strace which turns out to namespaceauth related bug with pattern matching:

write(4, “n puppet.looney.acme.com at line 2 of

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:122:in `parse\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:117:in `each\'
    /usr/lib/ruby/site_ruby/1.8/puppet/net

work/authconfig.rb:117:in `parse\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:100:in `each\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:100:in `parse\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authcon

fig.rb:97:in `open\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:97:in `parse\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:86:in `read\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:59:in `in

itialize\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:10:in `new\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:10:in `main\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authorization.rb:14:in `authconfig\

'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authorization.rb:28:in `authorized?\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authorization.rb:74:in `verify\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/xmlrpc/processor.rb:40:in `p

rocess\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/xmlrpc/webrick_servlet.rb:68:in `service\'
    /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service\'
    /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run\'
    /usr/lib/ruby/1.8/web

rick/server.rb:173:in `start_thread\'

    /usr/lib/ruby/1.8/webrick/server.rb:162:in `start\'
    /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread\'
    /usr/lib/ruby/1.8/webrick/server.rb:95:in `start\'
    /usr/lib/ruby/1.8/webrick/s

erver.rb:92:in `each\'

    /usr/lib/ruby/1.8/webrick/server.rb:92:in `start\'
    /usr/lib/ruby/1.8/webrick/server.rb:23:in `start\'
    /usr/lib/ruby/1.8/webrick/server.rb:82:in `start\'
    /usr/lib/ruby/site_ruby/1.8/puppet.rb:293:in `start

\'

    /usr/lib/ruby/site_ruby/1.8/puppet.rb:144:in `newthread\'
    /usr/lib/ruby/site_ruby/1.8/puppet.rb:143:in `initialize\'
    /usr/lib/ruby/site_ruby/1.8/puppet.rb:143:in `new\'
    /usr/lib/ruby/site_ruby/1.8/puppet.rb:143:in `newthread

\'

    /usr/lib/ruby/site_ruby/1.8/puppet.rb:291:in `start\'
    /usr/lib/ruby/site_ruby/1.8/puppet.rb:290:in `each\'
    /usr/lib/ruby/site_ruby/1.8/puppet.rb:290:in `start\'
    /usr/sbin/puppetd:437

[2009-12-04 20:09:39] DEBUG close: 10.0. 4.2:36567 [2009-12-04 20:11:24] DEBUG accept: 10.0.4.2:37430 [2009-12-04 20:11:24] DEBUG Puppet::Network::XMLRPC::WEBrickServlet is invoked. [2009-12-04 20:11:24] ERROR Puppet::ConfigurationError: Invalid pattern puppet.looney.acme.com at line 2 of

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:122:in `parse\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:117:in `each\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:117:in `parse\'\

n /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:100:in `each\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:100:in `parse\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:97:in `open\'
    /usr/lib/ru

by/site_ruby/1.8/puppet/network/authconfig.rb:97:in `parse\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:86:in `read\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:59:in `initialize\'
    /usr/lib/ruby/site_ru

by/1.8/puppet/network/authconfig.rb:10:in `new\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authconfig.rb:10:in `main\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/authorization.rb:14:in `authconfig\'
    /usr/lib/ruby/site_ruby/1.8/pu

ppet/network/authorization.rb:28:in `authorized?\'

    /usr/lib/ruby/site_ruby/1.8/puppet/network/authorization.rb:74:in `verify\'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/xmlrpc/processor.rb:40:in `process\'
    /usr/lib/ruby/site_ruby

/1.8/puppet/network/xmlrpc/webrick_servlet.rb:68:in `service\'

    /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service\'
    /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run\'
    /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread

\'

    /usr/lib/ruby/1.8/webrick/server.rb:162:in `start\'
    /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread\'
    /usr/lib/ruby/1.8/webrick/server.rb:95:in `start\'
    /usr/lib/ruby/1.8/webrick/server.rb:92:in ", 4096) = 4096

(full strace and namespaceauth attached)

#4
Updated by James Turnbull over 2 years ago

  • Status changed from Unreviewed to Needs Decision
  • Assignee set to Luke Kanies
  • Priority changed from High to Normal

#5
Updated by Luke Kanies over 2 years ago

  • Assignee changed from Luke Kanies to James Turnbull
  • Priority changed from Normal to Low

Ok, so looks like it’s a problem on the client and it’s not propagating its error correctly.

This problem becomes moot with rowlf, because we’re moving puppetrun to REST, which already has much better error propagation.

Should we just ignore the problem, or does someone feel like tracking this down sufficiently to fix it in 0.25.2 or something?

#6
Updated by Nicholas Veeser over 2 years ago

So I did some digging in case anyone cares.
I am curious because my namespaceauth.conf file is wrong and I would like to fix it, but debugging the parsing of it is becoming difficult.

Here is my description so far, correct as necessary:

Using Webrick, we have Puppet::Network::XMLRPC::WEBrickServlet It “include"s Puppet::Network::XMLRPCProcessor which "include"s Puppet::Network::Authorization

(I understand what ruby does mechanically here, but I am not sure how to talk about it in OO terms I can say “The WEBrickServlet is now also a XMLRPCProcessor. Is it also an "Authorization”)

Request comes in, sent to Puppet::Network::XMLRPCProcessor.process(data, request) That calls verify(request).

This looks up the authconfig member, which uses the “initialize object on first call” pattern (better name?).

This presumably loads and parses the namespaceconfig.conf file. Which has a parse error and throws an exception.

Well the XMLProcessor is not expecting any config parsing to be going on, so it ignores the exception, which (I believe) that Webrick just catches and turns into an undefined 500 Server error.

Seems like the problem is that Puppet::Network::Authorization needs to be initialized at some point. Idealy for me, the user, somewhere before it starts taking requests. That way it can tell me I did something foolish with my ns.conf file and tell me what and where.
However it’s a Module so I don’t know the expected ruby like pattern to initialize a Module.

Isn’t this an error that should come out on the server side, at initialization, not the client side as a response. That is, I would expect the webrick daemon not to start if the file is there and not valid.

I am experimenting with patches…and looking for guidance.

#7
Updated by James Turnbull about 1 year ago

  • Status changed from Needs Decision to Closed

Nicholas – given the unlikeliness of another 0.24.x release I am going to close this.

Also available in: Atom PDF