Puppet

# Using Mongrel with nginx

Alternatively, see: [[Using Mongrel]] and [[Using Mongrel Pound]]

This solution freshly baked by Jeff McCune.

Date: 2007-11-27

## References

-   [English Language nginx Wiki](http://wiki.codemongers.com/Main)

## Why choose nginx

-   Performance: nginx is very small and fast. Many claim it's

    faster than Pound.

-   Logging, Debugging: nginx appears to be "cleaner" than Pound.

-   Flexibility: nginx handles SSL client verification at the

    application layer, rather than terminating the SSL connection.

-   You don't have to patch pound if you don't want to upgrade

    Mongrel.

-   You don't have to upgrade Mongrel if you don't want to patch

    Pound.

-   nginx works "out of the box"

-   Despite russian documentation, the configuration syntax is

    intuitive.

## Why NOT choose nginx

-   I see no way to configure a certificate revocation list. If you

    need revocation, consider Pound or Apache for your SSL proxy.

## Overview

This sample configuration is based off nginx-0.5.33.

The back end puppet servers are configuration-compatible with the

[[Using Mongrel Pound]] documentation. That is, you must configure

ssl\_client\_header = HTTP\_X\_SSL\_SUBJECT and take note of ticket

[#906](http://projects.reductivelabs.com/issues/906).

## Sample Configuration

This configuration appears to be working well for about 300 clients

in production:

    user  daemon daemon;

    worker_processes  4;

    error_log       /var/log/nginx-puppet.log notice;

    pid             /var/run/nginx-puppet.pid;

    events {

        worker_connections  1024;

    }

    http {

        # include /etc/mime.types;

        default_type  application/octet-stream;

        # no sendfile on OSX uncomment 

        #this if your on linux or bsd

        sendfile        on;

        tcp_nopush      on;

        # Look at TLB size in /proc/cpuinfo (Linux) for the 4k pagesize

        large_client_header_buffers     16      4k;

        proxy_buffers                   128     4k;

        # if you adjust this setting to something higher

        # you should as well update the proxy_read_timeout 

        # in the server config part (see below)

        # Otherwise nginx will rerequest a manifest compile.

        keepalive_timeout  65;

        tcp_nodelay        on;

        ssl                     on;

        ssl_certificate         /Library/Puppet/Generated/Server/SSL/host_cert.pem;

        ssl_certificate_key     /Library/Puppet/Generated/Server/SSL/host_key.pem;

        ssl_client_certificate  /Library/Puppet/Generated/Server/SSL/ca/ca_crt.pem;

        ssl_ciphers             SSLv2:-LOW:-EXPORT:RC4+RSA;

        ssl_session_cache       shared:SSL:8m;

        ssl_session_timeout     5m;

        upstream puppet-production {

            server 127.0.0.1:18140;

            server 127.0.0.1:18141;

            server 127.0.0.1:18142;

            server 127.0.0.1:18143;

        }

        upstream puppet-test1 {

            server 127.0.0.1:28140;

        }

        upstream puppet-test2 {

            server 127.0.0.1:38140;

        }

        upstream puppet-test3 {

            server 127.0.0.1:48140;    

        }

        server {

            listen                  8140;

            ssl_verify_client       on;

            root                    /var/empty;

            access_log              /var/log/nginx/access-8140.log;

            rewrite_log             /var/log/nginx/rewrite-8140.log;

            # Variables

            # $ssl_cipher returns the line of those utilized it is cipher for established SSL-connection

            # $ssl_client_serial returns the series number of client certificate for established SSL-connection

            # $ssl_client_s_dn returns line subject DN of client certificate for established SSL-connection

            # $ssl_client_i_dn returns line issuer DN of client certificate for established SSL-connection

            # $ssl_protocol returns the protocol of established SSL-connection

            location / {

                proxy_pass          http://puppet-production;

                proxy_redirect      off;

                proxy_set_header    Host             $host;

                proxy_set_header    X-Real-IP        $remote_addr;

                proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;

                proxy_set_header    X-Client-Verify  SUCCESS;

                proxy_set_header    X-Client-DN      $ssl_client_s_dn;

                proxy_set_header    X-SSL-Subject    $ssl_client_s_dn;

                proxy_set_header    X-SSL-Issuer     $ssl_client_i_dn;

                proxy_read_timeout  65;

            }

        }

        server {

            listen                  8141;

            ssl_verify_client       off;

            root                    /var/empty;

            access_log              /var/log/nginx/access-8141.log;

            rewrite_log             /var/log/nginx/rewrite-8141.log;

            location / {

                proxy_pass  http://puppet-production;

                proxy_redirect     off;

                proxy_set_header   Host             $host;

                proxy_set_header   X-Real-IP        $remote_addr;

                proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

                proxy_set_header   X-Client-Verify  FAILURE;

                proxy_set_header   X-Client-DN      $ssl_client_s_dn;

                proxy_set_header   X-SSL-Subject    $ssl_client_s_dn;

                proxy_set_header   X-SSL-Issuer     $ssl_client_i_dn;

                proxy_read_timeout  65;

            }

        }

        server {

            listen                  8150;

            ssl_verify_client       on;

            root                    /var/empty;

            access_log              /var/log/nginx/access-8150.log;

            rewrite_log             /var/log/nginx/rewrite-8150.log;

            location / {

                proxy_pass  http://puppet-test1;

                proxy_redirect     off;

                proxy_set_header   Host             $host;

                proxy_set_header   X-Real-IP        $remote_addr;

                proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

                proxy_set_header   X-Client-Verify  SUCCESS;

                proxy_set_header   X-Client-DN      $ssl_client_s_dn;

                proxy_set_header   X-SSL-Subject    $ssl_client_s_dn;

                proxy_set_header   X-SSL-Issuer     $ssl_client_i_dn;

                proxy_read_timeout  65;

            }

        }

        server {

            listen                  8160;

            ssl_verify_client       on;

            root                    /var/empty;

            access_log              /var/log/nginx/access-8160.log;

            rewrite_log             /var/log/nginx/rewrite-8160.log;

            location / {

                proxy_pass  http://puppet-test2;

                proxy_redirect     off;

                proxy_set_header   Host             $host;

                proxy_set_header   X-Real-IP        $remote_addr;

                proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

                proxy_set_header   X-Client-Verify  SUCCESS;

                proxy_set_header   X-Client-DN      $ssl_client_s_dn;

                proxy_set_header   X-SSL-Subject    $ssl_client_s_dn;

                proxy_set_header   X-SSL-Issuer     $ssl_client_i_dn;

                proxy_read_timeout  65;

            }

        }

        server {

            listen                  8170;

            ssl_verify_client       on;

            root                    /var/empty;

            access_log              /var/log/nginx/access-8170.log;

            rewrite_log             /var/log/nginx/rewrite-8170.log;

            location / {

                proxy_pass  http://puppet-test3;

                proxy_redirect     off;

                proxy_set_header   Host             $host;

                proxy_set_header   X-Real-IP        $remote_addr;

                proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

                proxy_set_header   X-Client-Verify  SUCCESS;

                proxy_set_header   X-Client-DN      $ssl_client_s_dn;

                proxy_set_header   X-SSL-Subject    $ssl_client_s_dn;

                proxy_set_header   X-SSL-Issuer     $ssl_client_i_dn;

                proxy_read_timeout  65;

            }

        }

    }

## Certificate Signing

-   You will need to add **--ca\_port 8141** to your first puppetd

    run in order to have the puppetmaster sign your certificate.

## Nginx > 0.8.7

Since Nginx 0.8.7 or any 0.7 version with the patches from:

[http://www.masterzen.fr/2009/07/21/new-ssl-features-for-nginx/](http://www.masterzen.fr/2009/07/21/new-ssl-features-for-nginx/) (Note: not all 0.7 version need these patches, from a certain version on they are integrated upstream)

Nginx supports out of the box

\* Optional client verification, like apache \* CRL certificate

verification

Thus for those versions, the configuration becomes much simpler,

and you don't need to instruct your clients to use another port

(--ca\_port is not needed anymore):

    ...

    upstream puppet-production {

      server 127.0.0.1:18140;

      server 127.0.0.1:18141;

    }

    server {

      listen 8140;

      ssl                     on;

      ssl_session_timeout     5m;

      ssl_certificate         /var/lib/puppet/ssl/certs/puppetmaster.pem;

      ssl_certificate_key     /var/lib/puppet/ssl/private_keys/puppetmaster.pem;

      ssl_client_certificate  /var/lib/puppet/ssl/ca/ca_crt.pem;

      # choose any ciphers

      ssl_ciphers             SSLv2:-LOW:-EXPORT:RC4+RSA;

      # allow authenticated and client without certs

      ssl_verify_client       optional;

      # obey to the Puppet CRL

      ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;

      root                    /var/tmp;

      location / {

        proxy_pass              http://puppet-production;

        proxy_redirect         off;

        proxy_set_header    Host             $host;

        proxy_set_header    X-Real-IP        $remote_addr;

        proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;

        proxy_set_header    X-Client-Verify  $ssl_client_verify;

        proxy_set_header    X-Client-DN      $ssl_client_s_dn;

        proxy_set_header    X-SSL-Subject    $ssl_client_s_dn;

        proxy_set_header    X-SSL-Issuer     $ssl_client_i_dn;

        proxy_read_timeout  65;

      }

    }

    ...

## Fair Proxy Balancer

There is a module for nginx available that improves how requests are balanced to the backend mongrel servers: [http://www.brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel](http://www.brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel). Integrating and activating this module for your nginx can improve performance of puppet runs further. The reason is that some puppet requests are very long living such as the compile requests. As the mongrel instance is busy during that time compiling your catalog it won't serve any other requests and they are stuck in nginx's queue until this request is finished. If you have 4 mongrel instances running and one is currently busy compiling a huge catalog every 4th request from another client that is for example asking for file metadata will be stuck at the mongrel instance compiling the catalog. This will slowdown the puppet runs on the second client.

The fair proxy balancer will keep track which of your mongrel instances are busy and will direct new requests to other mongrel instances, which are probably free.

All you need to do is to compile nginx with this module and setup your proxy balancer like that:

    ...

    upstream puppet-production {

      fair;

      server 127.0.0.1:18140;

      server 127.0.0.1:18141;

    }

    ...