Bug #4025

"ralsh service" fails with "Unimplemented element" when using launchd provider if certain plists are binary

Added by Clay Caviness almost 2 years ago. Updated over 1 year ago.

Status:Closed Start date:06/16/2010
Priority:Normal Due date:
Assignee:Paul Berry % Done:

0%

Category:OSX
Target version:2.6.2
Affected Puppet version:0.25.5 Branch:http://github.com/stereotype441/puppet/tree/ticket/2.6.x/4025
Keywords:
Votes: 0

Description

On 10.6.3, all system-level LaunchDaemons are shipped as text files (xml1 format). However, as they are updated, they may be converted to binary (binary1 format).

I suspect this is because ralsh is using Plist::parse_xml, which can’t handle binary plists.

:) fox:~ crc$ file {/System,}/Library/Launch{Daemons,Agents}/*.plist | grep binary
/System/Library/LaunchDaemons/com.apple.syslogd.plist:
Apple binary property list
:( fox:~ crc$ sudo ralsh --debug --trace service com.openssh.sshd
debug: Puppet::Type::Service::ProviderDebian: file /usr/sbin/update-rc.d does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not exist
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
debug: Puppet::Type::Service::ProviderDebian: file /usr/sbin/update-rc.d does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not exist
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
/Library/Ruby/Site/1.8/facter/util/plist/parser.rb:104:in `parse'
/Library/Ruby/Site/1.8/facter/util/plist/parser.rb:28:in `parse_xml'
/Library/Ruby/Site/1.8/puppet/provider/service/launchd.rb:65:in `jobsearch'
/Library/Ruby/Site/1.8/puppet/provider/service/launchd.rb:61:in `each'
/Library/Ruby/Site/1.8/puppet/provider/service/launchd.rb:61:in `jobsearch'
/Library/Ruby/Site/1.8/puppet/provider/service/launchd.rb:59:in `each'
/Library/Ruby/Site/1.8/puppet/provider/service/launchd.rb:59:in `jobsearch'
/Library/Ruby/Site/1.8/puppet/provider/service/launchd.rb:87:in `instances'
/Library/Ruby/Site/1.8/puppet/type.rb:1006:in `instances'
/Library/Ruby/Site/1.8/puppet/type.rb:1005:in `collect'
/Library/Ruby/Site/1.8/puppet/type.rb:1005:in `instances'
/Library/Ruby/Site/1.8/puppet/application/ralsh.rb:93:in `main'
/Library/Ruby/Site/1.8/puppet/application.rb:226:in `send'
/Library/Ruby/Site/1.8/puppet/application.rb:226:in `run_command'
/Library/Ruby/Site/1.8/puppet/application.rb:217:in `run'
/Library/Ruby/Site/1.8/puppet/application.rb:306:in `exit_on_fail'
/Library/Ruby/Site/1.8/puppet/application.rb:217:in `run'
/usr/bin/ralsh:89
Could not run: Unimplemented element
:( fox:~ crc$ sudo plutil -convert xml1 /System/Library/LaunchDaemons/com.apple.syslogd.plist 
:) fox:~ crc$ sudo ralsh -d service com.openssh.sshd
debug: Puppet::Type::Service::ProviderDebian: file /usr/sbin/update-rc.d does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not exist
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
debug: Puppet::Type::Service::ProviderDebian: file /usr/sbin/update-rc.d does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not exist
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update does not exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/chkconfig does not exist
debug: Service path /etc/init.d does not exist
debug: Puppet::Type::Service::ProviderLaunchd: Executing '/bin/launchctl list'
debug: Puppet::Type::Service::ProviderLaunchd: Executing '/bin/launchctl list'
service { 'com.openssh.sshd':
    enable => 'true',
    ensure => 'running'
}

Related issues

related to Puppet - Bug #5112: Launchd Service broke in 2.6.2 with OS X 10.4 Clients. Closed 10/27/2010

History

Updated by Nigel Kersten almost 2 years ago

We have several options here:

  1. skip binary plists
  2. convert binary plists to xml1
  3. switch to using Foundation calls with osx/cocoa

1) means we simply can’t cope with binary plists, so anytime anyone uses ‘defaults’ it will make the job such that puppet can’t cope with it 2) means we are changing the state of the system even when simply checking the state of a service resource 3) means we’re dropping support for 10.4

When I first wrote this, there were no binary launchd plists shipped by Apple. That seems to have changed.

Updated by Clay Caviness almost 2 years ago

I haven’t verified on a clean retail image, but I think the launchd plists as shipped are all text, but they may be converted to binary as they’re changed.

This would be a change in the provider, right? So if we skip binary plists, we’d no longer be able to control them with service resources in a puppet manifest?

I vote for option 3; changing things just to query them seems icky, and I’d rather not skip binary plists.

Updated by Nigel Kersten almost 2 years ago

I can’t see what’s changing them though? It’s like something is using NSDefaults to modify them… which I’m not doing in the provider at all for precisely this reason.

Surely launchtl isn’t converting them to binary now ?

Updated by James Turnbull almost 2 years ago

  • Category set to OSX
  • Status changed from Unreviewed to Investigating

Updated by Clay Caviness over 1 year ago

Diff below. Is there a better method to get this patched?

diff --git a/lib/puppet/provider/service/launchd.rb b/lib/puppet/provider/service/launchd.
index 9703595..716afa3 100644
--- a/lib/puppet/provider/service/launchd.rb
+++ b/lib/puppet/provider/service/launchd.rb
@@ -1,4 +1,4 @@
-require 'facter/util/plist'
+require 'osx/cocoa'
 
 Puppet::Type.type(:service).provide :launchd, :parent => :base do
   desc "launchd service management framework.
@@ -62,7 +62,7 @@ Puppet::Type.type(:service).provide :launchd, :parent => :base do
           next if f =~ /^\..*$/
           next if FileTest.directory?(f)
           fullpath = File.join(path, f)
-          job = Plist::parse_xml(fullpath)
+          job = OSX::NSDictionary.dictionaryWithContentsOfFile(fullpath).to_ruby
           if job and job.has_key?("Label")
             if job["Label"] == label
               return { label => fullpath }
@@ -118,7 +118,7 @@ Puppet::Type.type(:service).provide :launchd, :parent => :base do
   def plist_from_label(label)
     job = self.class.jobsearch(label)
     job_path = job[label]
-    job_plist = Plist::parse_xml(job_path)
+    job_plist = OSX::NSDictionary.dictionaryWithContentsOfFile(job_path)
     raise Puppet::Error.new("Unable to parse launchd plist at path: #{job_path}") if not 
     [job_path, job_plist]
   end
@@ -200,7 +200,7 @@ Puppet::Type.type(:service).provide :launchd, :parent => :base do
     job_plist_disabled = job_plist["Disabled"] if job_plist.has_key?("Disabled")
 
     if self.class.get_macosx_version_major == "10.6":
-      overrides = Plist::parse_xml(Launchd_Overrides)
+      overrides = OSX::NSDictionary.dictionaryWithContentsOfFile(Launchd_Overrides).to_ru
 
       unless overrides.nil?
         if overrides.has_key?(resource[:name])
@@ -227,14 +227,14 @@ Puppet::Type.type(:service).provide :launchd, :parent => :base do
   # versions this is stored in the job plist itself.
   def enable
     if self.class.get_macosx_version_major == "10.6"
-      overrides = Plist::parse_xml(Launchd_Overrides)
+      overrides = OSX::NSMutableDictionary.dictionaryWithContentsOfFile(Launchd_Overrides
       overrides[resource[:name]] = { "Disabled" => false }
-      Plist::Emit.save_plist(overrides, Launchd_Overrides)
+      overrides.writeToFile_atomically_(Launchd_Overrides, true)
     else
       job_path, job_plist = plist_from_label(resource[:name])
       if self.enabled? == :false
         job_plist.delete("Disabled")
-        Plist::Emit.save_plist(job_plist, job_path)
+        job_plist.writeToFile_atomically_(job_path, true)
       end
     end
   end
@@ -242,13 +242,13 @@ Puppet::Type.type(:service).provide :launchd, :parent => :base do
 
   def disable
     if self.class.get_macosx_version_major == "10.6"
-      overrides = Plist::parse_xml(Launchd_Overrides)
+      overrides = OSX::NSMutableDictionary.dictionaryWithContentsOfFile(Launchd_Overrides
       overrides[resource[:name]] = { "Disabled" => true }
-      Plist::Emit.save_plist(overrides, Launchd_Overrides)
+      overrides.writeToFile_atomically_(Launchd_Overrides, true)
     else
       job_path, job_plist = plist_from_label(resourcecommit::name])
       job_plist["Disabled" = true
-      Plist::Emit.save_plist(job_plist, job_path)
+      job_plist.writeToFile_atomically_(job_path, true)
     end
   end
 

Updated by Nigel Kersten over 1 year ago

Clay, have you looked at this?

http://projects.puppetlabs.com/projects/puppet/wiki/Development_Development_Lifecycle

I’d really like to see you guys commit more patches, and this is an easier way, happy to sit down at work and run through this if you have any issues.

I’m pretty sure we’ll need to update some unit tests too.

Updated by Nigel Kersten over 1 year ago

  • Status changed from Investigating to Accepted
  • Assignee set to Nigel Kersten

Updated by James Turnbull over 1 year ago

  • Target version set to 2.6.2

Updated by James Turnbull over 1 year ago

  • Status changed from Accepted to In Topic Branch Pending Review

Updated by Nigel Kersten over 1 year ago

  • Status changed from In Topic Branch Pending Review to Tests Insufficient

I’ll do a full submission with unit test updates, as Clay has gone on vacation…

Updated by Nigel Kersten over 1 year ago

Clay, how do you feel about this patch?

http://github.com/nigelkersten/puppet/commit/9b87bb8f25d896c41010f0469e75d1a875d016dd

I’ll mail it to the dev-list if it passes your test, but I did some quick on the fly conversions and it worked fine.

(This should also patch against 2.6.x cleanly)

Updated by Clay Caviness over 1 year ago

That’s a nice solution. Looks good to me.

Updated by James Turnbull over 1 year ago

  • Status changed from Tests Insufficient to In Topic Branch Pending Review

Updated by Paul Berry over 1 year ago

  • Target version changed from 2.6.2 to 52

Setting target version to “queued”, since we would prefer to limit 2.6.x releases to fixing bugs that were introduced in 2.6.

Updated by Nigel Kersten over 1 year ago

The situation isn’t that clear though Paul.

Apple weren’t shipping binary plists or converting xml plists to binary during the 0.25.x lifecycle, they only started doing this after 2.6.0 was released.

This is a bug that pretty much breaks the Service provider on OS X. We’re not going to fix this in 2.6.x at all?

Updated by James Turnbull over 1 year ago

What Nigel said. :)

Updated by Paul Berry over 1 year ago

  • Target version changed from 52 to 2.6.2

Whoops, sorry. You guys are right. This deserves to go in 2.6.2.

Updated by Paul Berry over 1 year ago

  • Assignee changed from Nigel Kersten to Paul Berry

The patch isn’t applying cleanly against 2.6.x. I’ll work on the conflicts and hopefully post a patch against 2.6.x to github today.

Updated by Nigel Kersten over 1 year ago

My git-fu isn’t so amazing, but the patch should at least apply reasonably cleanly? I didn’t think we’d really made any changes to the provider in 2.6.x….

Updated by Paul Berry over 1 year ago

It was mostly failing because of whitespace changes we made between 0.25 and 2.6 (we switched from 4-space indentation to 2-space indentation), and that confused git.

I’ve now fixed all the whitespace problems (I think) and cherry-picked the patch to 2.6.x, and I’m getting spec test failures. I’m happy to track them down, Nigel, but you probably are more familiar with this corner of the code than I am. Do you want to have a look? I uploaded my cherry-pick to http://github.com/stereotype441/puppet/tree/wip/4025 if you’re interested.

Updated by Paul Berry over 1 year ago

  • Status changed from In Topic Branch Pending Review to Accepted

It looks like the spec test failures are due to threading problems related to rubycocoa. The key clue is this message:

Ruby threads cannot be used in RubyCocoa without patches to the Ruby interpreter

which gets output every time a thread is created after requiring ‘osx/cocoa’.

I’m going to experiment this afternoon to see if I can rework the patch to shell out to plutil rather than using rubycocoa.

Updated by Paul Berry over 1 year ago

  • Status changed from Accepted to In Topic Branch Pending Review
  • Branch set to http://github.com/stereotype441/puppet/tree/ticket/2.6.x/4025

Ok, I’ve pushed a version that uses plutil to translate plists from binary to xml format. It works and passes spec tests on my system. Nigel or Clay, would you mind having a look at this to make sure it works for you?

Updated by Markus Roberts over 1 year ago

  • Status changed from In Topic Branch Pending Review to Ready For Checkin

Nigel has apparently signed off on this version so…

Updated by Markus Roberts over 1 year ago

  • Status changed from Ready For Checkin to Code Insufficient

The appears to cause two test failures under Kubuntu:

3)
'Puppet::Type::Service::ProviderLaunchd when checking whether the service is enabled on OS X 10.6 should return true if the job plist says disabled is true and the global overrides says disabled is false' FAILED
expected: :true,
     got: :false (using ==)
./spec/unit/provider/service/launchd_spec.rb:102:
/home/markus/projects/puppet/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:21:in `run'
/home/markus/projects/puppet/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:16:in `each'
/home/markus/projects/puppet/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:16:in `run'

4)
'Puppet::Type::Service::ProviderLaunchd when checking whether the service is enabled on OS X 10.6 should return false if the job plist says disabled is false and the global overrides says disabled is true' FAILED
expected: :false,
     got: :true (using ==)
./spec/unit/provider/service/launchd_spec.rb:108:
/home/markus/projects/puppet/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:21:in `run'
/home/markus/projects/puppet/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:16:in `each'
/home/markus/projects/puppet/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:16:in `run'

Probably a missing confine on the tests.

Updated by Paul Berry over 1 year ago

  • Status changed from Code Insufficient to In Topic Branch Pending Review

You’re right, Markus. It was missing a confine. I’ve updated the branch.

Updated by Markus Roberts over 1 year ago

  • Status changed from In Topic Branch Pending Review to Ready For Checkin

Updated by Markus Roberts over 1 year ago

  • Status changed from Ready For Checkin to Closed

Pushed to 2.6.x as commit:a7fb9b112025e328304743c6453ece5d6ae81275

Updated by Paul Berry over 1 year ago

  • Status changed from Closed to Ready For Checkin

The spec test changes for my patch failed on non-OSX systems because of a missing stub. Added the missing stub so that the spec tests can now run on all systems.

Updated by Markus Roberts over 1 year ago

  • Status changed from Ready For Checkin to Closed

Second part pushed to 2.6.x as commit:ea49d13192fce5e891a5ea767ecf05fc3107a411

Also available in: Atom PDF