Bug #3751

0.25.5rc2 server fails with 0.24-8 client

Added by Ohad Levy about 2 years ago. Updated about 2 years ago.

Status:Closed Start date:05/10/2010
Priority:Normal Due date:
Assignee:James Turnbull % Done:

0%

Category:serialization
Target version:0.25.5
Affected Puppet version:0.25.5rc2 Branch:http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/3751
Keywords:
Votes: 0

Description

During tests on RC2, I’ve got the following error message on a 0.24-8 client:

    warning: Configuration could not be instantiated: interning empty string

this is the trace from the client

    /usr/lib/ruby/site_ruby/1.8/puppet/metatype/manager.rb:114:in `to_sym'
    /usr/lib/ruby/site_ruby/1.8/puppet/metatype/manager.rb:114:in `type'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:90:in `to_type'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:198:in `to_catalog'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:126:in `each'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:126:in `each'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:195:in `to_catalog'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:204:in `call'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:204:in `to_catalog'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:126:in `each'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:126:in `each'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:195:in `to_catalog'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:210:in `call'
    /usr/lib/ruby/site_ruby/1.8/puppet/transportable.rb:210:in `to_catalog'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/client/master.rb:172:in `getconfig'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/client/master.rb:245:in `run'
    /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:426:in `thinmark'
    /usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
    /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime'
    /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:425:in `thinmark'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/client/master.rb:244:in `run'
    /usr/lib/ruby/1.8/sync.rb:229:in `synchronize'
    /usr/lib/ruby/site_ruby/1.8/puppet/network/client/master.rb:237:in `run'
    /usr/sbin/puppetd:417

on the server side, there are no errors e.g:

Compiled catalog for certname in 1.14 seconds

Related issues

related to Puppet - Bug #3766: Remove yaml-fixups no longer needed with zaml Closed 05/13/2010

History

Updated by Markus Roberts about 2 years ago

  • Status changed from Unreviewed to Investigating

Updated by Markus Roberts about 2 years ago

Ohad —

So it appears that a resource is getting serialized without a type, but it’s not clear which resource, or why, and most especially, how. I’ve tried reproducing with a variety of resource types but haven’t been able to get this result. So it’s question time:

1) How repeatable is the problem? 2) How pervasive is the problem? 3) Do you know (or suspect) what resource is causing it? 4) Do you know what catalog serialization format is being used?
5) Does the problem persist if you try a different format? 6) Is either machine running ruby 1.8.1? 7) Is the manifest in question small enough and non-sensitive enough that you could share it? 8) Can you think of anything else that might give us a handle on the problem?

— Markus

Updated by Ohad Levy about 2 years ago

1) How repeatable is the problem?

afaik on every run

3) Do you know (or suspect) what resource is causing it?

so far I was unable to find the exact manifest which causes it.

4) Do you know what catalog serialization format is being used?

5) Does the problem persist if you try a different format?

yaml, switching to marshel on the client solves the problem

6) Is either machine running ruby 1.8.1?

yes, but that was observed on newer versions of ruby too(1.8.5 RH5)

7) Is the manifest in question small enough and non-sensitive enough that you could share it?

I didn’t find it yet :(

8) Can you think of anything else that might give us a handle on the problem?

  • it does work against a 0.25.4 master.
  • could it be that the default serialization was marshel, and with the introduction of zaml the order was changed?

Updated by Markus Roberts about 2 years ago

Ohad —

Can you try it with this patch on the client:

diff --git a/lib/puppet/transportable.rb b/lib/puppet/transportable.rb
index 41c51fd..7afded1 100644
--- a/lib/puppet/transportable.rb
+++ b/lib/puppet/transportable.rb
@@ -87,6 +87,7 @@ module Puppet
         end
 
         def to_type
+            p [:to_type,type,name,file,line]
             if typeklass = Puppet::Type.type(self.type)
                 return typeklass.create(self)
             else

which will spew a lot of diagnostics, and should provide enough information to figure out which resource it’s choking on?

— Markus

Updated by Ohad Levy about 2 years ago

bingo :)

[:to_type, nil, "addons", "/etc/puppet/modules/development/services/redhat/manifests/yum.pp", 26]

and the file content is:

# adds http://puppet as source to the yum repository and enable the addons repo
class redhat::yum {
  $nfsgold=$gi ? { "2" => "goldenimage_2.0", "4" => "goldenimage_4.0", "5" => "goldenimage_5" }
  $webgold="gold_$gi"
  $gipath=$gi ? { "2" => "_ws", default => ""}

  yumrepo { 'base':
    enabled => '1',
    baseurl => "file:///opt/$nfsgold/\$basearch$gipath/
              http://puppet/opt/$nfsgold/\$basearch$gipath/
              http://yum.klu.infineon.com/yum/$webgold/base/\$basearch$gipath/",
    failovermethod => 'priority'
  }
  yumrepo { 'stable':
     baseurl => "file:///opt/$nfsgold/repo/\$basearch/stable/
              http://puppet/opt/$nfsgold/repo/\$basearch/stable/
              http://yum.klu.infineon.com/yum/$webgold/repo/\$basearch/stable/",
    failovermethod => 'priority'
  }
  yumrepo { 'addons':
    enabled => '1',
    baseurl => "file:///opt/$nfsgold/repo/\$basearch/addons/
              http://puppet/opt/$nfsgold/repo/\$basearch/addons/
              http://yum.klu.infineon.com/yum/$webgold/repo/\$basearch/addons/",
    failovermethod => 'priority'
  }
}

Updated by Markus Roberts about 2 years ago

  • Category set to serialization
  • Status changed from Investigating to Accepted

Given the information above, I was able to produce a variant of the symptoms which led to a theory as to the cause. We have a number of hacks/odd ways of doing things to deal with the fact that ruby’s built in YAML doesn’t properly encode certain data structures. When we switched to zaml these bugs went away, but the fixups to work around them were left in place.

As an example, with stock ruby’s yaml we get this amazing error:

> irb
>> require "yaml"
=> false
>> YAML.load(YAML.dump("\n"))
=> ""

vs. the correct results with zaml:

> irb
>> require "lib/puppet/util/zaml.rb"
=> true
>> YAML.load(ZAML.dump("\n"))
=> "\n"

The long-term correct fix is probably to find and remove the no-longer valid fixups, but for the moment the cleanest solution is to patch zaml so that it no longer produces the cases that will trigger them.

This patch:

diff --git a/lib/puppet/util/zaml.rb b/lib/puppet/util/zaml.rb
index dcf0e19..22045e1 100644
--- a/lib/puppet/util/zaml.rb
+++ b/lib/puppet/util/zaml.rb
@@ -234,7 +234,7 @@ class String
                     (self =~ /\s$/) or
                     (self =~ /^[>|][-+\d]*\s/i) or
                     (self[-1..-1] =~ /\s/) or 
-                    (self =~ /[\x00-\x08\x0B\x0C\x0E-\x1F\x80-\xFF]/) or
+                    (self =~ /[\x00-\x08\x0A-\x1F\x80-\xFF]/) or
                     (self =~ /[,\[\]\{\}\r\t]|:\s|\s#/) or 
                     (self =~ /\A([-:?!#&*'"]|<<|%.+:.)/) 
                     )

This causes zaml to always escape \r and \n when they occur in strings, thus resulting in slightly less human-readable output, but guarantees the line endings won’t get “corrected” thus preserving the string’s value.

— Markus

Ohad —

Could you try this patch on your 0.25.5pre puppetmaster on top of Luke’s fix for #3640, and see if it works with your 0.24.8 client without needing the —preferred_serialization_format option?

— Markus

Updated by Markus Roberts about 2 years ago

  • Status changed from Accepted to In Topic Branch Pending Review
  • Branch set to http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/3751

Branch with the above patch up at http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/3751

Note that this is intended for 0.25.5 and should NOT be applied to master / Rowlf. See #3766 for the alternative.

Updated by Ohad Levy about 2 years ago

  • Status changed from In Topic Branch Pending Review to Ready For Checkin
  • Assignee set to James Turnbull
  • Target version set to 0.25.5

I can confirm that the patch above seems to be solving the issue I was facing

Thank you!

Updated by James Turnbull about 2 years ago

  • Status changed from Ready For Checkin to Closed

Pushed in commit:9d3e98b8cb2b980b61f3cd17af471f2e9d04841f in branch 0.25.x

Also available in: Atom PDF