The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Feature #11004

Solaris 11 GA & pkg

Added by Robert van Veelen over 2 years ago. Updated over 1 year ago.

Status:ClosedStart date:11/22/2011
Priority:NormalDue date:
Assignee:Rahul Gopinath% Done:

0%

Category:package
Target version:3.0.0
Affected Puppet version: Branch:https://github.com/puppetlabs/puppet/pull/1046
Keywords:solaris pkg package provider upgradeable versionable

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This issue is currently not available for export. If you are experiencing the issue described below, please file a new ticket in JIRA. Once a new ticket has been created, please add a link to it that points back to this Redmine ticket.


Description

Oracle has changed the output format of pkg again for the final release of s11. The “status” field is no longer present in the output of “pkg list -H”. There is now an IFO field that has the following status flags (from the man page):

     The last column contains a set of flags  that  show  the
     status of the package:

         o    An i in the I column shows that the package  is
              installed.

         o    An f in the F column shows that the package  is
              frozen.

         o    An o in the O column shows that the package  is
              obsolete.  An  r in the O column shows that the
              package has been renamed  (a  form  of  obsole-
              tion).

Using the work done for Bug #7986 , I have tried the following with some success (though this only uses the ‘I’ column):

pkg.rb.diff Magnifier (2.92 KB) Robert van Veelen, 11/22/2011 03:18 am

solaris11pkgs.txt Magnifier - Solaris 11 pkg -Ha list output (178 KB) Ben Hughes, 04/26/2012 01:51 pm


Related issues

Related to Puppet - Bug #7986: Solaris 11/OpenSolaris pkg provider needs to accept exit ... Closed 06/19/2011

History

#1 Updated by Robert van Veelen over 2 years ago

sorry. adding the patch as a file.

#2 Updated by Daniel Pittman over 2 years ago

  • Tracker changed from Bug to Feature
  • Description updated (diff)
  • Category set to package
  • Status changed from Unreviewed to Requires CLA to be signed

G'day Robert. Thanks so much for this submission. It doesn’t look like we have a CLA for you – or, at least, RedMine suggests to me that we don’t, but please forgive me if that isn’t on the money.

If we get that we should be able to look at rolling this change in; if not, we are likely to have to go through and rewrite it, because lawyers are awesome but grumpy. :)

#3 Updated by Ben Hughes almost 2 years ago

I’ve been suffering this myself on our Solaris 11 box.

They have indeed changed the output of pkg -H, because that’s useful.

So, I’ve written something quite quite horrible which kinda works.

diff --git i/lib/puppet/provider/package/pkg.rb w/lib/puppet/provider/package/pkg.rb
index 399a9df..6e71689 100644
--- i/lib/puppet/provider/package/pkg.rb
+++ w/lib/puppet/provider/package/pkg.rb
@@ -22,7 +22,12 @@ Puppet::Type.type(:package).provide :pkg, :parent => Puppet::Provider::Package d
     packages
   end
 
+  # Sol 11 version
+  # x11/library/libxcb  1.7-0.175.0.0.0.0.1215     i--
+  #
+
   self::REGEX = /^(\S+)(?:\s+\(.*?\))?\s+(\S+)\s+(\S+)\s+\S+$/
+  self::BACKUPREGEX = /^(\S+)\s+(\S+)\s+(\S+)$/
   self::FIELDS = [:name, :version, :status]
 
   def self.parse_line(line)
@@ -35,14 +40,29 @@ Puppet::Type.type(:package).provide :pkg, :parent => Puppet::Provider::Package d
 
       hash[:provider] = self.name
 
-      if hash[:status] == "installed"
+      if hash[:status] == "installed" or hash[:status] == "i--"
         hash[:ensure] = :present
       else
         hash[:ensure] = :absent
       end
     else
-      warning "Failed to match 'pkg list' line #{line.inspect}"
-      return nil
+      if match = self::BACKUPREGEX.match(line)
+
+        self::FIELDS.zip(match.captures) { |field,value|
+          hash[field] = value
+        }
+
+        hash[:provider] = self.name
+
+        if hash[:status] == "installed" or hash[:status] == "i--"
+          hash[:ensure] = :present
+        else
+          hash[:ensure] = :absent
+        end
+      else
+        warning "Failed to match smelly 'pkg list' line #{line.inspect}"
+        return nil
+      end
     end
 
     hash
@@ -59,6 +79,8 @@ Puppet::Type.type(:package).provide :pkg, :parent => Puppet::Provider::Package d
         return v
       when "installed"
         version = v
+      when "i--"
+        version = v
       else
         Puppet.warn "unknown package state for #{@resource[:name]}: #{v}"
       end

Which is utterly filthy, but means package/pkg works on my solaris install. I tried to work out if I can just combine that regex, but man was it ugly. If the good Mr. Van Veelan doesn’t sign the CLA, shall I work this in to a better patch? I’ve not looked at his diff, so legally we’re above board there I think.

#4 Updated by Stefan Schulte almost 2 years ago

Hi Ben,

I also had a look on this issue and came up with a quite simple regex change I think:

https://github.com/stschulte/puppet/tree/feature/2.7.x/11004

#5 Updated by Ben Hughes almost 2 years ago

I’ve updated mine and it’s at https://github.com/barn/puppet/commit/71707b9b5123415f4a7e3b9ca00c106eeb311f56

Mine works with the previous version, as it detects it (poorly at present) but I don’t know what other Solarises do. Your way is cleaner (and has tests, yay), but is it backwards compatible?

I’m happy for either/any/all, I just want it to work. (;

#6 Updated by Stefan Schulte almost 2 years ago

Ben Hughes wrote:

is it backwards compatible?

Yes it is. At least I don’t get any spec failures (I dont have any solaris around) and the spec tests do already test the correct parsing of solaris 10 output.

The new regex marks the status field as optional. If it is not present in the output it will try to set the status manually to installed or known depending on the flags field.

#7 Updated by Stefan Schulte almost 2 years ago

  • Status changed from Requires CLA to be signed to In Topic Branch Pending Review
  • Branch set to https://github.com/puppetlabs/puppet/pull/719

opened a pull request

#8 Updated by Ben Hughes almost 2 years ago

Trying it out now on our Solaris [sparc] 11 box.

[root@mundilfari:puppet]# puppet agent -t --environment sunosbase                                                                    puppet:stschulte/feature/2.7.x/11004:6aa8441
info: Retrieving plugin
info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
info: Caching catalog for sparc.example.org
info: Applying configuration version '6b5d038'
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "library/python-2/pycurl-26                        7.19.0.1-0.175.0.0.0.2.537 i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "network/dns/bind                                  9.6.3.4.3-0.175.0.0.0.2.537 i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "print/cups/filter/foomatic-db                     0.20080903-0.175.0.0.0.2.537 i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "print/cups/filter/foomatic-db-engine              0.20080903-0.175.0.0.0.2.537 i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "service/network/dns/bind                          9.6.3.4.3-0.175.0.0.0.2.537 i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "service/network/ftp                               1.3.3.0.5-0.175.0.0.0.2.537 i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "service/network/ntp                               4.2.5.200-0.175.0.0.0.2.537 i--"
err: /Stage[main]/Ssh/Package[network/ssh]: Could not evaluate: Could not get latest version: undefined method `parse_line' for #
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "service/network/ntp                               4.2.5.200-0.175.0.0.0.2.537 i--"
err: /Stage[main]/Ntp/Package[ntp]/ensure: change from absent to present failed: Execution of '/usr/bin/pkg install ntp' returned 4: No updates necessary for this image.

May need to expand the regex. (should I comment here or the pull request?)

#9 Updated by Stefan Schulte almost 2 years ago

Hi Ben,

you are right the regex was wrong because I marked the status field as optional, while the field delemiters (\s+) were not optional. So if there is only one whitespace between version and flags my regex failed. Should be fixed now.

(should I comment here or the pull request?)

I get notifications for both. As long as we keep the discussion in one place I don’t care were it happens ;–)

#10 Updated by Ben Hughes almost 2 years ago

Giving it another go, I now get:

[root@mundilfari:~]# puppet agent -t
info: Retrieving plugin
info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
info: Caching catalog for mundilfari.example.org
info: Applying configuration version '4f2dfc4'
err: /Stage[main]/Ssh/Package[network/ssh]: Could not evaluate: Could not get latest version: undefined method `parse_line' for #

That’s on a package that is defined as:

   if $kernel == 'SunOS' {
    package { 'network/ssh':
        ensure => latest,
    }
 

So the latest function in pkg.rb can’t see parse_line. My version has ‘self.class.parse_line’ in the latest function. See https://github.com/barn/puppet/commit/71707b9b5123415f4a7e3b9ca00c106eeb311f56#L0R88

#11 Updated by Stefan Schulte almost 2 years ago

Hi Ben,

so I guess latest is broken since https://github.com/puppetlabs/puppet/commit/54e5d18. I remember that I did not really understand how latest should work with the pkg provider anyways. So your fix seems to be right of course (calling self.class.parse_line) but having some sample output of pkg list -Ha some_package where

  • some_package is currenty not installed
  • some_package is installed, but a newer package is available
  • some_package is already installed in the latest version

would be cool. This way we can create some specs to cover the latest method. But I think this really should be a seperate ticket because it doesnt seem to be a solaris 11 specific issue.

#12 Updated by Ben Hughes almost 2 years ago

See attached. That has obsolete packages in it, installed packages and uninstalled packages in it.

I can’t find how to get updated packages out of Solaris/pkg… Urg.

#13 Updated by Marco Walther almost 2 years ago

I used Ben’s version and updated a little bit more.

  • A new way to find the pkg version/list format
  • Updated the SOL11REGEX to include the publisher again

https://github.com/mw46d/puppet/commit/297f4c68a3affbd23f9e2244792e8466a3775ae6

#14 Updated by Ben Hughes almost 2 years ago

So running that I get:

[root@mundilfari:puppet]# git lg -n 1                                                                                                   puppet:mw46d/ticket/11004/Solaris:297f4c6
* 297f4c6 Marco Walther 19 hours ago New idea to find the pkg version  (HEAD, mw46d/master, mw46d/ticket/11004/Solaris)
[root@mundilfari:puppet]# puppet agent -t                                                                                               puppet:mw46d/ticket/11004/Solaris:297f4c6
info: Retrieving plugin
info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
info: Caching catalog for mundilfari.example.org
info: Applying configuration version '4b5b4e7'
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "archiver/gnu-tar                                  1.26-0.175.0.0.0.2.537     i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "compress/bzip2                                    1.0.6-0.175.0.0.0.2.537    i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "compress/gzip                                     1.3.5-0.175.0.0.0.2.537    i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "compress/p7zip                                    9.20.1-0.175.0.0.0.2.537   i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "compress/unzip                                    6.0-0.175.0.0.0.2.537      i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "compress/zip                                      3.0-0.175.0.0.0.2.537      i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/SunVTS/SunVTS-incorporation         0.5.11-0.172.0.0.0.0.0     i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/X/X-incorporation                   0.5.11-0.175.0.0.0.0.1215  i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/admin/admin-incorporation           0.5.11-0.175.0.0.0.2.0     i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/cacao/cacao-incorporation           0.5.11-0.174.0.0.0.0.0     i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/cde/cde-incorporation               0.5.11-0.175.0.0.0.0.0     i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/cns/cns-incorporation               0.5.11-0.175.0.0.0.1.0     i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "consolidation/dbtg/dbtg-incorporation             0.5.11-0.175.0.0.0.2.0     i--"
[...snip...]
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "x11/library/toolkit/libxt                         1.0.9-0.175.0.0.0.0.1215   i--"
warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "shell/bash                                        4.1.9-0.175.0.0.0.2.537    i--"
err: /Stage[main]/Packages::Shells/Package[shell/bash]/ensure: change from absent to present failed: Execution of '/usr/bin/pkg install shell/bash' returned 4: Recursing into linked image: zone:smelly

Returning from linked image: zone:smelly

No updates necessary for this image.

warning: Puppet::Type::Package::ProviderPkg: Failed to match 'pkg list' line "shell/zsh                                         4.3.12-0.175.0.0.0.2.537   i--"

etc…

#15 Updated by Daniel Pittman almost 2 years ago

  • Status changed from In Topic Branch Pending Review to Merged - Pending Release

https://github.com/puppetlabs/puppet/pull/719 has been merged.

#16 Updated by Daniel Pittman almost 2 years ago

  • Status changed from Merged - Pending Release to Code Insufficient

Daniel Pittman wrote:

https://github.com/puppetlabs/puppet/pull/719 has been merged.

I missed the discussion here that showed this was incomplete. I have reverted the code until it is finished up.

#17 Updated by Stefan Schulte almost 2 years ago

As far as I get the discussion:

  • the spec failures are not generated by the code you’ve merged but are generated by Marco’s solution
  • the latest method is broken in the code that you’ve merged. But it was also broken before and is not Solaris 11 related. In my opinion this should be fixed in a separate ticket

Is this correct?

#18 Updated by Ben Hughes almost 2 years ago

Is latest broken on everything. It seems to work fine on Linux/debian for me? Can you provide an example of what you mean with that please Stefan?

#19 Updated by Stefan Schulte almost 2 years ago

Ben Hughes wrote:

Is latest broken on everything. It seems to work fine on Linux/debian for me? Can you provide an example of what you mean with that please Stefan?

I’m a bit confused now. I am talking about the pkg provider and the bug that you mentioned in https://projects.puppetlabs.com/issues/11004#note-10. The latest method was not touched by the pull request that was merged. The bug was introduced before: https://github.com/puppetlabs/puppet/commit/54e5d18#L0R56. That’s why I said it is not solaris 11 related and why I suggested to open a new ticket for that issue or reopen #7986 for that matter.

#20 Updated by Ben Hughes almost 2 years ago

Latest fails as it’s not parsing the line correctly. The latest function is fine I believe.

The reason it exits with 4 is because it doesn’t think it’s installed, so tries to install it again.

If Oracle ever update any packages I could actually test it though…

#21 Updated by Stefan Schulte almost 2 years ago

Ben Hughes wrote:

Latest fails as it’s not parsing the line correctly. The latest function is fine I believe.

I am referring to the error message you saw

err: /Stage[main]/Ssh/Package[network/ssh]: Could not evaluate: Could not get latest version: undefined method `parse_line' for #

in https://github.com/puppetlabs/puppet/commit/54e5d18#L0R56 (ticket #7986) the latest method was changed to use the parse_line classmethod to reduce code duplication. But latest itself is no classmethod so it has to use self.class.parse_line not parse_line. That’s why I said the latest method is broken since #7986. The lineparsing itself which was changed to handle solaris 11 output (so pull request https://github.com/puppetlabs/puppet/pull/719) does work I guess.

#22 Updated by Ben Hughes almost 2 years ago

No idea from me. This is all Puppet/Ruby internals way out of my depth I’m afraid.

#23 Updated by Moses Mendoza almost 2 years ago

https://github.com/puppetlabs/puppet/commit/e37b536397568ffa3e6fe9383bc8a19a0d3765e5, referencing this ticket, was released with puppet 2.7.15rc1. Can this issue be closed?

#24 Updated by Ben Hughes almost 2 years ago

Seeing as that commit got reverted… https://github.com/puppetlabs/puppet/commit/24d4bb51d07f1c643d6c2277b4901f2044a51a6d and there is no fix in the ticket yet, so please do not close it.

#25 Updated by John Wiegley almost 2 years ago

So what is the status of this? I cannot update packages on OpenIndiana via Puppet, so this is still a serious issue. Here’s the output I see:

err: /Stage[main]/Ssh::Server/Package[pkg:/network/ssh/ssh-key]: Could not evaluate:
Could not get latest version: undefined method `parse_line' for
Package[pkg:/network/ssh/ssh-key](provider=pkg):Puppet::Type::Package::ProviderPkg

Changing the call to “parse_line” in latest to “self.class.parse_line” makes things work for me.

#26 Updated by Rahul Gopinath almost 2 years ago

  • Assignee set to Rahul Gopinath

changing parse_line to self.class.parse_line works. Here is my test case and patch.

--- a/lib/puppet/provider/package/pkg.rb
+++ b/lib/puppet/provider/package/pkg.rb
@@ -69,7 +69,7 @@ Puppet::Type.type(:package).provide :pkg, :parent => Puppet::Provider::Package d
   def latest
     version = nil
     pkg(:list, "-Ha", @resource[:name]).each_line do |line|
-      v = parse_line(line.chomp)[:status]
+      v = self.class.parse_line(line.chomp)[:status]
       case v
       when "known"
         return v
# cat ../t.pp 
Package { provider => 'pkg' }

package { 'text/gnu-patch':
  ensure => latest,
}

file {'tst':
 path => '/tmp/xxx',
 content => 'Hai',
 require => Package['text/gnu-patch']
}

#27 Updated by Rahul Gopinath almost 2 years ago

  • Status changed from Code Insufficient to Merged - Pending Release
  • Branch changed from https://github.com/puppetlabs/puppet/pull/719 to <https://github.com/puppetlabs/puppet/pull/958>

#28 Updated by Rahul Gopinath almost 2 years ago

  • Status changed from Merged - Pending Release to Code Insufficient
  • Branch changed from <https://github.com/puppetlabs/puppet/pull/958> to https://github.com/puppetlabs/puppet/pull/958

#29 Updated by Rahul Gopinath almost 2 years ago

  • Status changed from Code Insufficient to In Topic Branch Pending Review

#30 Updated by Stefan Schulte almost 2 years ago

Rahul, can you please open a seperate ticket for that issue (ensure => latest is broken on pkg provider). While your patch is correct of course it does not solve the solaris 11 issue.

#31 Updated by Rahul Gopinath almost 2 years ago

As suggested, this is now a new issue. #15644

#32 Updated by Rahul Gopinath over 1 year ago

Hi Ben, Stefan, I refactored the pkg provider in https://github.com/puppetlabs/puppet/pull/1046, and was wondering if this issue still existed. Could you please give it a go? (updated pull.)

#33 Updated by Josh Cooper over 1 year ago

  • Status changed from In Topic Branch Pending Review to Merged - Pending Release
  • Target version set to 3.0.0
  • Branch changed from https://github.com/puppetlabs/puppet/pull/958 to https://github.com/puppetlabs/puppet/pull/1046

Merged in https://github.com/puppetlabs/puppet/commit/75c0640ec8bf41ca0679e3744163af3319d3ef4a

The solaris `:pkg` provider is now versionable and upgradeable. For Solaris
version 5.11, it is now the default provider, instead of `:sun`.

#34 Updated by Josh Cooper over 1 year ago

  • Keywords set to solaris pkg package provider upgradeable versionable

#35 Updated by Matthaus Owens over 1 year ago

  • Status changed from Merged - Pending Release to Closed

Released in Puppet 3.0.0-rc4

Also available in: Atom PDF