Puppet

# Using Mongrel with Apache

Alternatively, see: 

* [[Using Mongrel On Debian]] 

* [[Using Mongrel On Enterprise Linux]] 

* [[Using Mongrel Pound]] 

* [[Using Mongrel Nginx]]

* [[Using_Thin_Nginx]]

* [[Using Passenger]]

Puppet daemons default to using WEBrick for http serving, but

puppetmasterd can be used with Mongrel instead. This requires

setting up a web proxy in front of Mongrel to handle the SSL

connections, and then you can start as many puppetmasterd instances

as you need. Note that Puppet uses Mongrel differently than most --

usually, you would have a mongrel\_cluster command that managed

multiple Mongrel instances. I planned to implement it this way, but

it ended up being easier in the short term to stick with

puppetmasterd. Please contact us if you want to work on

getting mongrel\_cluster to work with Puppet -- but we generally

recommend folks use Passenger or Unicorn instead these days...

Mongrel will work with any modern version of puppet; 0.23.1 is the minimum

version with appropriate support. (Earlier

versions couldn't do certificate signing because they didn't

support the X-Client-Verify header.)

Mongrel 1.1.5 or later is strongly recommended, and while 0.3.15 will

theoretically work we don't test older versions of Mongrel extensively.

This document only describes setting up Apache as a proxy in front

of Mongrel. Any other http proxy should work, as long as it

supports validating client certificates.

# Why You'd Do This

Mongrel scales much better than WEBrick, at least partially because

it allows you to run multiple processes serving the same pool of

clients on the same host. WEBrick only uses Ruby's threading, which

does not scale beyond one processor, and it appears that WEBrick

starts dropping connections beyond about 2 concurrent connections.

If you're getting connection-reset or End-of-file errors, you

should try Mongrel. As more people try it and it proves to be

stable, it will eventually become the preferred serving platform

for the master.

# Known Issues

We recommend Mongrel 1.1.5 or later, although it will theoretically work

as far back as 0.3.15 or maybe 0.3.14.  We have not extensively tested between

those versions, however, and would strongly encourage you to upgrade.

SLES 10 ships with Mongrel 0.3.13, which is too old to work out of the box.

The Puppet file server (and other parts of puppet?) will not see

connections coming from the puppet client but from the reverse

proxy (ie. Apache). FileServingConfiguration (Security section)

gives more detail.

There is a problem with Apache and the certificate revocation list

(CRL) as generated by the puppet CA. If you encounter this problem,

you are likely to see lines like this in your error log:

    [Mon Nov 10 08:47:20 2008] [warn] Invalid signature on CRL

    [Mon Nov 10 08:47:20 2008] [error] Certificate Verification: Error (8): CRL signature failure

Puppet will also complain:

    err: Could not retrieve catalog: Certificates were not trusted: tlsv1 alert decrypt error

Yet everything looks good in your puppet.conf.

If so, you will need to create a CRL directory and point your Apache server to this path by changing

    SSLCARevocationFile     /Library/Puppet/Generated/Server/SSL/ca/ca_crl.pem

To

    SSLCARevocationPath     /Library/Puppet/Generated/Server/SSL/CRL_DIRECTORY

To create this directory, you should take the following steps. For this example, we will assume that the directory will be created in the standard SSL directory that puppet uses.

    $mkdir /Library/Puppet/Generated/Server/SSL/crl

    $cd /Library/Puppet/Generated/Server/SSL/crl

    $ln -s /Library/Puppet/Generated/Server/SSL/ca/ca_crl.pem `openssl crl -in /Library/Puppet/Generated/Server/SSL/ca/ca_crl.pem -hash -noout`.0

Then, restart your Apache server.

If, for some reason, you are still getting the old error, you will need to disable CRL checking by commenting out all lines starting with 'SSLCARevocation'.

# Apache Configuration

Here is a complete apache configuration, intended for use only with

puppet:

    # Jeff McCune <mccune@math.ohio-state.edu>

    # 2007-09-14

    #

    # Minimal Apache Configuration for Apache+Mongrel+Puppetmaster

    #

    #  Host System Setup and Configuration:

    #  - Add puppet/puppet user/group

    #  - Use the following configuration file.

    #  - /Library/Puppet/Generated/Server is owned by puppet/puppet

    #

    #  - If you have a system which doens't provide Apache 2.2.X packages:

    #    Apache may be built with:

    #

    #    ./configure --enable-so \

    #       --enable-ssl=shared --enable-proxy=shared --enable-proxy_http=shared \

    #       --enable-proxy_balancer=shared --enable-headers=shared \

    #       --enable-authz_host=shared --enable-log_config=shared \

    #       --prefix=/Library/Puppet/Resources/httpd-2.2.4

    #

    # - Finally, start the SSL proxy with:

    #   /Library/Puppet/Resources/httpd-2.2.4/bin/httpd -f \

    #   /Library/Puppet/Versioned/Server/cluster-orange/httpd/puppetmasterd.conf

    Listen 8140

    PidFile /Library/Puppet/Generated/Server/balancer.pid

    User puppet

    Group puppet

    LoadModule proxy_module modules/mod_proxy.so

    LoadModule proxy_http_module modules/mod_proxy_http.so

    LoadModule proxy_balancer_module modules/mod_proxy_balancer.so

    LoadModule headers_module modules/mod_headers.so

    LoadModule ssl_module modules/mod_ssl.so

    LoadModule authz_host_module modules/mod_authz_host.so

    LoadModule log_config_module modules/mod_log_config.so

    <Directory />

        Options FollowSymLinks

        AllowOverride None

        Order deny,allow

        Deny from all

    </Directory>

    <Proxy balancer://puppetmaster>

      BalancerMember http://127.0.0.1:18140 keepalive=on max=2 retry=30

      BalancerMember http://127.0.0.1:18141 keepalive=on max=2 retry=30

    </Proxy>

    <VirtualHost *:8140>

        SSLEngine on

        SSLCipherSuite SSLv2:-LOW:-EXPORT:RC4+RSA

        SSLCertificateFile      /Library/Puppet/Generated/Server/SSL/host_cert.pem

        SSLCertificateKeyFile   /Library/Puppet/Generated/Server/SSL/host_key.pem

        SSLCertificateChainFile /Library/Puppet/Generated/Server/SSL/ca/ca_crt.pem

        SSLCACertificateFile    /Library/Puppet/Generated/Server/SSL/ca/ca_crt.pem

        # Using the technique from above.

        SSLCARevocationPath     /Library/Puppet/Generated/Server/SSL/crl

        SSLVerifyClient require

        SSLVerifyDepth  1

        SSLOptions +StdEnvVars

        # The following client headers allow the same configuration to work with Pound.

        RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e

        RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e

        RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

        <Location />

            SetHandler balancer-manager

            Order allow,deny

            Allow from all

        </Location>

        ProxyPass / balancer://puppetmaster:8140/ timeout=180

        ProxyPassReverse / balancer://puppetmaster:8140/

        ProxyPreserveHost on

        SetEnv force-proxy-request-1.0 1

        SetEnv proxy-nokeepalive 1

        ErrorLog  /Library/Puppet/Generated/Server/balancer_error.log

        CustomLog /Library/Puppet/Generated/Server/balancer_access.log combined

        CustomLog /Library/Puppet/Generated/Server/balancer_ssl_request.log \

                      "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

    </VirtualHost>

Note that you'll have to customize it to match the name of your

puppet server. Note also the cluster member declarations -- this is

where you tell Apache how to contact the Mongrel instances -- you

need a separate line for every instance, and (of course) each

instance needs to be started on a separate port.

This configuration starts Apache on Puppet's normal server port, so

clients do not need to be reconfigured.

# Another Apache configuration

Here's another happy apache configuration, without all the OSXey

stuff ('''Note:''' You still need to load the apache modules from

the configuration above. Except for ''log\_config'' which seems to

be OS X specific):

    Listen 8140

    ProxyRequests Off

    <Proxy balancer://puppetmaster>

        BalancerMember http://127.0.0.1:18140

    </Proxy>

    <VirtualHost *:8140>

        SSLEngine on

        SSLCipherSuite SSLv2:-LOW:-EXPORT:RC4+RSA

        SSLCertificateFile /etc/puppet/ssl/certs/backup4.foo.com.pem

        SSLCertificateKeyFile /etc/puppet/ssl/private_keys/backup4.foo.com.pem

        SSLCertificateChainFile /etc/puppet/ssl/ca/ca_crt.pem

        SSLCACertificateFile /etc/puppet/ssl/ca/ca_crt.pem

        # Using the technique from above.

        SSLCARevocationPath     /Library/Puppet/Generated/Server/SSL/crl

        SSLVerifyClient require

        SSLVerifyClient require

        SSLVerifyDepth  1

        SSLOptions +StdEnvVars

        RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e

        RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

        <Location />

           SetHandler balancer-manager

           Order allow,deny

           Allow from all

        </Location>

        ProxyPass / balancer://puppetmaster:8140/

        ProxyPassReverse / balancer://puppetmaster:8140/

        ProxyPreserveHost on

    </VirtualHost>

# Configuring Puppet

The only change to Puppet is that you need to start puppetmasterd

with --servertype mongrel and choose a separate masterport for each

instance. You will also need to specify a separate pidfile for each

instance.

The attached example init script for RedHat systems, or this helper

script might be useful:

    #!/bin/bash

    #

    # Start a Puppet Master Server instance.

    EZPUPPET_CONFDIR="${EZPUPPET_CONFDIR:=${EZPUPPET_BASE:=/Library/Puppet}/Versioned/Server/${EZPUPPET_SERVER:=cluster-orange}/puppet}"

    if ! [[ "$1" -gt 0 ]]; then

      echo "ERROR: You must provide a port to run this puppet master on."

      echo "Start at 18140 and increment upward."

      echo "  The apache load balancer is configured to talk to these servers"

      echo "  so the port numbers are NOT arbitrary."

      exit 1

    fi

    MASTERPORT="$1"

    shift

    puppetmasterd \

      --confdir="${EZPUPPET_CONFDIR}" \

      --pidfile=${EZPUPPET_BASE}/Generated/Server/var/run/puppetmaster."${MASTERPORT}".pid \

      --servertype=mongrel \

      --masterport="${MASTERPORT}" \

      $*

As mentioned earlier, this would be easier if it were done with

mongrel\_start, but it will take a significant amount of work to

get all of Puppet's existing code working in that framework, and

I'm unlikely to do it except as part of contract development.

Clearly, patches that provided this would be accepted.

'''NOTE''': The current (as of at least 0.24.6)

/etc/init.d/puppetmaster script on RedHat and compatible systems

already supports multiple mongrel instances in a similar way to

above. Simply specify your Mongrel instance ports in

/etc/sysconfig/puppetmaster in BASH array format as per the

following example:

    PUPPETMASTER_PORTS=([0]=18140 [1]=18141 [2]=18142 [3]=18143 [4]=18144)

# Generating Certificates

In version 0.23.2 I found that puppetmasterd would not generate

certificates when started using Mongrel. I worked around the

problem by using puppetca to generate the certificates as part of

the install process, like this:

    puppetca --generate <DOMAIN>

# Client certificate verification 

With the apache option:

    SSLVerifyClient optional|require

you control how Apache wil check the client certification. You can either, set the different certificate request path to not require verification, or disable it (unless you have another ca server or another way of distributing the certificates). So:

 * by enabling require, you basically say - the client is already suppose to have a certificate that I could verify. If certificate is not signed in puppetmaster you will get the error:

    err: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read finished A: sslv3 alert handshake failure

 * "optional" will simulate the behaviour of an default standalone puppetmaster (without apache).