Bug #2964

updated resources cannot be collected until they are exported twice

Added by Dan Bode 7 months ago. Updated 7 months ago.

Status:Closed Start:12/19/2009
Priority:Normal Due date:
Assigned to:Jesse Wolfe % Done:

0%
Category:-
Target version:0.25.2
Affected version:0.25.1 Branch:http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/2964
Keywords:
Votes: 0

Description

I noticed the following behavior. When an exported resource already exists in the database, and I want to update a param, puppet has to run twice to update the param.

  1. The first run removes the param_values row
  2. The second run updates teh param_values row.

Is this the expected bahavior?

Below is a detailed example:

Here is the basic file

site.pp

node exporter {
  @@host { 'test':
           ip => "127.0.0.1",
  }
}
node collector {
  Host<<| |>>
}

export for the first time

[root@exporter manifests]# puppetd —certname exporter —test

and here is the db

mysql> select * from param_values;
+----+-----------+---------------+------+-------------+---------------------+---------------------+
| id | value     | param_name_id | line | resource_id | updated_at          | created_at          |
+----+-----------+---------------+------+-------------+---------------------+---------------------+
| 45 | 127.0.0.1 |             1 | NULL |          52 | 2009-12-18 17:30:14 | 2009-12-18 17:30:14 | 
+----+-----------+---------------+------+-------------+---------------------+---------------------+
1 row in set (0.00 sec)

mysql> select * from resources where title='test';
+----+-------+---------+---------+----------------+----------+------+---------------------+---------------------+
| id | title | restype | host_id | source_file_id | exported | line | updated_at          | created_at          |
+----+-------+---------+---------+----------------+----------+------+---------------------+---------------------+
| 52 | test  | Host    |      14 |              2 |        1 |    4 | 2009-12-18 17:30:14 | 2009-12-18 17:30:14 | 
+----+-------+---------+---------+----------------+----------+------+---------------------+---------------------+

now change the ip address to 127.0.0.2

this removed the param values from the db????

[root@exporter manifests]# puppetd —certname exporter —test

mysql> select * from param_values;
Empty set (0.00 sec)
mysql> select * from resources where title='test';
+----+-------+---------+---------+----------------+----------+------+---------------------+---------------------+
| id | title | restype | host_id | source_file_id | exported | line | updated_at          | created_at          |
+----+-------+---------+---------+----------------+----------+------+---------------------+---------------------+
| 52 | test  | Host    |      14 |              2 |        1 |    4 | 2009-12-18 17:30:14 | 2009-12-18 17:30:14 | 
+----+-------+---------+---------+----------------+----------+------+---------------------+---------------------+
1 row in set (0.00 sec)

of coarse, we cant properly collect now [root@collector manifests]# puppetd —certname collector —test info: Caching catalog for collector info: Applying configuration version ‘1261186363’ notice: Finished catalog run in 0.09 seconds

now if I run it a second time, the ip is updated in the db

[root@exporter manifests]# puppetd —certname exporter —test

mysql> select * from param_values;
+----+-----------+---------------+------+-------------+---------------------+---------------------+
| id | value     | param_name_id | line | resource_id | updated_at          | created_at          |
+----+-----------+---------------+------+-------------+---------------------+---------------------+
| 46 | 127.0.0.2 |             1 |    4 |          52 | 2009-12-18 17:35:30 | 2009-12-18 17:35:30 | 
+----+-----------+---------------+------+-------------+---------------------+---------------------+
1 row in set (0.00 sec)

now collection works

[root@collector manifests]# puppetd —certname collector —test info: Caching catalog for collector info: Applying configuration version ‘1261186363’ notice: //Node[collector]/Host[test]/ip: ip changed ‘127.0.0.1’ to ‘127.0.0.2’ info: Filebucket[/var/lib/puppet/clientbucket]: Adding /etc/hosts(2fff89c170a0f6eed1af1d711d189720) notice: Finished catalog run in 0.10 seconds

is it supposed to work like this?

Related issues

related to Puppet - Bug #2949: host can be exported without ipaddress Rejected 12/17/2009
related to Puppet - Bug #2942: store configs error: After adding a new external node, al... Closed 12/16/2009

Associated revisions

Revision 49530adaaa302c4b4e3b9eefb05e9551503a0ba6
Added by Markus Roberts 7 months ago

Fixing #2964 updated resources cannot be collected until they are exported twice

This logic had a bug where it would not insert data if it had just been deleted.

Signed-off-by: Jesse Wolfe jes5199@gmail.com

History

Updated by Markus Roberts 7 months ago

  • Status changed from Unreviewed to Investigating
  • Target version set to 0.25.2

Mythinks these are all one based on one underlying issue.

Updated by Markus Roberts 7 months ago

  • Status changed from Investigating to Needs more information

Dan —

Playing around with this on 0.25.x for a while yesterday and again this morning, I’m not able to reproduce. If I retry directly without waiting or restarting the puppetmaster, it just retains the old compiled catalog. If I wait (or restart the puppetmaster) it seems to work as I’d expect (updates the hosts file with the new IP, no errors or anything).

Any hints about factors I may be missing?

— Markus

Updated by Markus Roberts 7 months ago

  • Assigned to set to defunt Bode

Updated by Markus Roberts 7 months ago

  • Assigned to changed from defunt Bode to Dan Bode

Updated by Dan Bode 7 months ago

I grabbed origin/0.25.x for this just to clarify.

a few point not mentioned before 1. exporter is the same node as the server 2. first, I clean out the db using ruby puppetstoredconfigclean.rb exporter collector 3. related config

[puppetmasterd]
    storeconfigs = true
    dbadapter = mysql
    dbuser = puppet
    dbpassword = password
    dbserver = localhost
    dbsocket = /var/lib/mysql/mysql.sock

site.pp

node exporter {
  @@host { 'test':
           ip => "127.0.0.2",
           alias  => 'test-alias',
  }
  Host<<||>>
}

first run

[root@exporter manifests]# !pu
puppetd --certname exporter --test
warning: SELinux not installed
info: Connecting to mysql database: puppet
info: Caching facts for exporter
info: Caching catalog for exporter
info: Applying configuration version '1261339485'
notice: //Node[exporter]/Host[test]/ensure: created
info: Filebucket[/var/lib/client//clientbucket]: Adding /etc/hosts(3844956ab64e6edcf0a89ab84b4f1f4d)
notice: Finished catalog run in 0.05 seconds

mysql> select * from param_values;
+-----+------------+---------------+------+-------------+---------------------+---------------------+
| id  | value      | param_name_id | line | resource_id | updated_at          | created_at          |
+-----+------------+---------------+------+-------------+---------------------+---------------------+
| 166 | test-alias |             3 | NULL |         129 | 2009-12-20 12:04:45 | 2009-12-20 12:04:45 | 
| 167 | 127.0.0.2  |             1 | NULL |         129 | 2009-12-20 12:04:45 | 2009-12-20 12:04:45 | 
+-----+------------+---------------+------+-------------+---------------------+---------------------+
2 rows in set (0.00 sec)

now change ip and run again

node exporter {
  @@host { 'test':
           ip => "127.0.0.3",
           alias  => 'test-alias',
  }
  Host<<||>>
}

[root@exporter manifests]# !pu
puppetd --certname exporter --test
warning: SELinux not installed
info: Connecting to mysql database: puppet
info: Caching catalog for exporter
info: Applying configuration version '1261339742'
notice: //Node[exporter]/Host[test]/ip: ip changed '127.0.0.2' to '127.0.0.3'
info: Filebucket[/var/lib/client//clientbucket]: Adding /etc/hosts(b25ac177b4286e8f747a38f9a450d5a8)
notice: Finished catalog run in 0.05 seconds

mysql> select * from param_values;
+-----+------------+---------------+------+-------------+---------------------+---------------------+
| id  | value      | param_name_id | line | resource_id | updated_at          | created_at          |
+-----+------------+---------------+------+-------------+---------------------+---------------------+
| 166 | test-alias |             3 | NULL |         129 | 2009-12-20 12:04:45 | 2009-12-20 12:04:45 | 
+-----+------------+---------------+------+-------------+---------------------+---------------------+
1 row in set (0.00 sec)

[root@exporter manifests]# puppetd --certname exporter --test
warning: SELinux not installed
info: Connecting to mysql database: puppet
info: Caching catalog for exporter
info: Applying configuration version '1261339742'
notice: Finished catalog run in 0.02 seconds

mysql> select * from param_values;
+-----+------------+---------------+------+-------------+---------------------+---------------------+
| id  | value      | param_name_id | line | resource_id | updated_at          | created_at          |
+-----+------------+---------------+------+-------------+---------------------+---------------------+
| 166 | test-alias |             3 | NULL |         129 | 2009-12-20 12:04:45 | 2009-12-20 12:04:45 | 
| 168 | 127.0.0.3  |             1 |    6 |         129 | 2009-12-20 12:10:29 | 2009-12-20 12:10:29 | 
+-----+------------+---------------+------+-------------+---------------------+---------------------+
2 rows in set (0.00 sec)

Updated by Dan Bode 7 months ago

  • Assigned to changed from Dan Bode to Markus Roberts

Updated by Markus Roberts 7 months ago

  • Assigned to changed from Markus Roberts to Jesse Wolfe

I’m still not able to reproduce. Jesse’s going to try, then confer with Dan.

Updated by Markus Roberts 7 months ago

  • Status changed from Needs more information to Ready for Testing
  • Branch set to http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/2964

Branch with Jesse’s fix up on my repo at http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/2964

Updated by Jesse Wolfe 7 months ago

  • Assigned to changed from Jesse Wolfe to Dan Bode

I thought I updated this ticket yesterday, but I guess it didn’t really happen.

Updated by Jesse Wolfe 7 months ago

  • Assigned to changed from Dan Bode to Jesse Wolfe

Updated by Markus Roberts 7 months ago

  • Status changed from Ready for Testing to Ready for Checkin

Updated by James Turnbull 7 months ago

  • Status changed from Ready for Checkin to Closed

Pushed in commit:“49530adaaa302c4b4e3b9eefb05e9551503a0ba6” in branch 0.25.x

Also available in: Atom PDF