Puppet

Further investigation (with the help of Brice) shows that my example code doesn’t contain the CRL verifications.

removing the CRL verification from the client seems to be “solving” the issue. e.g.:

diff --git a/lib/puppet/ssl/host.rb b/lib/puppet/ssl/host.rb
index 9d016c8..ebe170a 100644
--- a/lib/puppet/ssl/host.rb
+++ b/lib/puppet/ssl/host.rb
@@ -213,7 +213,7 @@ class Puppet::SSL::Host
 
             # If there's a CRL, add it to our store.
             if crl = Puppet::SSL::CertificateRevocationList.find("ca")
-                @ssl_store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK_ALL|OpenSSL::X509::V_FLAG_CRL_CHECK
+#                @ssl_store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK_ALL|OpenSSL::X509::V_FLAG_CRL_CHECK
                 @ssl_store.add_crl(crl.content)
             end
             return @ssl_store

if I understand correctly, one can split the problem into a few parts:

1. CRL will always be generated on each and every CA.
2. CRL will be pushed to the client (if none exits) – but will not be updated anymore (regardless of changes to it)

may I suggest that: 1. CRL verification should be configurable on the client. 2. CRL source may be another CA server (e.g. upstream CA server) 3. CRL on the client should be re-synced every interval 4. Master CA will not auto generate a CRL (or at-least allow to configure it) 5. consider switching to OCSP.

On further investigation it appears that the problem may be that the flag settings are incorrect. There was an OpenSSL bug many years ago (openssl <= 0.9.7a) in which the flags were reversed, and the conjunction was used as a workaround for this bug.

It appears (though I have not verified this) that the combination hack fails to work as expected on newer (>= 0.9.8) versions. While we may ultimately have to set the flags based on the openssl version, it may be that OR-ing in the flag we want will suffice:

diff --git a/lib/puppet/ssl/host.rb b/lib/puppet/ssl/host.rb
index 9d016c8..ebe170a 100644
--- a/lib/puppet/ssl/host.rb
+++ b/lib/puppet/ssl/host.rb
@@ -213,7 +213,7 @@ class Puppet::SSL::Host

             # If there's a CRL, add it to our store.
             if crl = Puppet::SSL::CertificateRevocationList.find("ca")
-                @ssl_store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK_ALL|OpenSSL::X509::V_FLAG_CRL_CHECK
+                @ssl_store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK
                 @ssl_store.add_crl(crl.content)
             end
             return @ssl_store

Can you try this, and report back:

1. If it works
2. If CRLs still work
3. The version of openssl you tested on

In the meantime, I’ll continue to investigate.

— Markus

The openssl change log may prove to be a useful resource if we wind up having to do something version sensitive:

http://www.openssl.org/source/exp/CHANGES

Looking at the code for openssl 0.9.8k, it appears we may also need to define:

V_FLAG_ALLOW_PROXY_CERTS = 0x40

And write:

@ssl_store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK|V_FLAG_ALLOW_PROXY_CERTS

although this is not what I would characterize as “obvious” and I’d like to find some support for it from the documentation.

— Markus

P.S. It is possible (though not as likely) that the correct combination is all three: check, check_all, and allow_proxy_certs, though I’m even more dubious of that path).

I’ve put up a simple branch that disables the CRL for both new and old SSL:

luke/tickets/0.25.x/3640-disable_certificate_revocation

I had to disable it in the old SSL code because the CA handler was automatically creating it.

This is at least a hack fix.

I didn’t see Luke’s patch on the list, so I’ll respond here: +1 as a stopgap for 0.25.5, provided it works for Ohad, but I’d like to see a real solution for Rowlf and/or 0.25.6

In the my comment, the P.S. appears to have been correct and the body just an example of why I generally drink coffee before programming, instead of just starting typing away.

— Markus

as expected, Luke’s patch is working http://github.com/lak/puppet/commit/bb07664e22ef7c035576e25c8ed699990365626d

I took an existing 0.24.8 client, upgrade it to 0.25.5 and got the client working fine.

however, when I’m trying to use the 0.25.5 master to sign a new ca, it fails. it seems, that puppet is really unhappy if the certs/ca.pem file contain other certificates. afaik, this is the place to put the top level ca pub key, however, the puppetmaster keeps replacing the content of this file to match to the local ca pub key (ca/ca_crt.pem) from the log:

Removing file Puppet::SSL::Certificate ca at '/var/lib/puppet/ssl/certs/ca.pem'
Retrieved certificate does not match private key
Signed certificate request for ca
Starting Puppet server version 0.25.5

What if the top-level CA has all of the certs in it?

At this point, I really don’t think we can fix chained certs in this release.

I think we cut 0.25.5 as soon as the yaml thing is fixed, and then we focus on a new release with chained certs working.

We should be able to get a patch working relatively quickly once the release is out, but given how we keep finding little things, we should not hold the release up for this.