Bug #2816
Storeconfigs - MySQL server has gone away
| Status: | Closed | Start date: | 11/13/2009 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% |
|
| Category: | Rails | |||
| Target version: | 0.25.2 | |||
| Affected Puppet version: | 0.25.1 | Branch: | http://github.com/jes5199/puppet/tree/ticket/0.25.x/2816 | |
| Keywords: | ||||
| Votes: | 1 |
Description
I am receiving the following error from storeconfigs with a MySQL backend.
Mysql::Error: MySQL server has gone away
Respawning one client in a loop doesn’t produce the error. Spawning all clients (17 on this particular network) at once from cron reproduces the error every time. Several of the clients then fail to complete. Attached is a puppetmasterd log of such a run. Observing MySQL’s processlist at the time shows that it holds up to 18 idle connections open until the point at which the errors are logged and then connections are closed.
The puppetmaster is setup like so.
- Gentoo Linux
- Puppet 0.25.0
- Mongrel 1.1.5 (5 puppetmasterd processes)
- Nginx 0.8.17
- Rails 2.3.4
- ActiveRecord 2.3.4
- MySQL/Ruby 2.7.5
- MySQL 5.0.70 (unix socket, default settings)
I have tried combinations of the following with no success.
- Downgrading to Rails 2.2 and ActiveRecord 2.2.3 (with patch from #2508).
- Upgrading to MySQL/Ruby 2.8.1
I have noticed that amending @rails.rb@ to set the ActiveRecord connection option @:reconnect = true@ seems to resolve the issue. However I’m not clear whether such an approach is transaction safe for Puppet to use.
Related issues
History
Updated by Markus Roberts about 2 years ago
- Status changed from Unreviewed to Investigating
- Target version set to 0.25.2
Updated by Dan Carley about 2 years ago
I did take a look at #2764 previously. It doesn’t apply because @allow_concurrency@ was deprecated in ActiveRecord 2.2 and later.
It might also be worth noting that the same behaviour occurs with a single Webrick instance as it does multiple Mongrel instances.
Updated by Adam Gibbins about 2 years ago
- Affected Puppet version changed from 0.25.0 to 0.25.1
I’m receiving this exact same problem. All machines received this error:
Nov 17 17:47:34 puppet.example.com puppetmasterd[19059]: Mysql::Error: MySQL server has gone away: SELECT * FROM `hosts` WHERE (`hosts`.`name` = 'slave1.example.com') LIMIT 1
Restarting puppetmaster/mongrel resolved this instantly however it reoccurs again 5 minutes later.
Puppet Version (Master and Client): 0.25.1 MySQL Version: MySQL 5.0.75-0ubuntu10.2 Operating System: Ubuntu Jaunty 9.04 Mongrel: 1.1.5 (10 puppetmasterd processes) Apache: 2.2.11 (Prefork 64bit) Ruby: ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
Please let me know if you require any additional details (My ruby knowledge is practically non existent).
Updated by Brice Figureau about 2 years ago
Adam Gibbins wrote:
I’m receiving this exact same problem. All machines received this error: […] Restarting puppetmaster/mongrel resolved this instantly however it reoccurs again 5 minutes later.
Puppet Version (Master and Client): 0.25.1 MySQL Version: MySQL 5.0.75-0ubuntu10.2 Operating System: Ubuntu Jaunty 9.04 Mongrel: 1.1.5 (10 puppetmasterd processes) Apache: 2.2.11 (Prefork 64bit) Ruby: ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
Please let me know if you require any additional details (My ruby knowledge is practically non existent).
You forgot to print the most important information: rails version. I highly suspect you run 2.1 and thus you’re seeing #2764.
Updated by Luke Kanies about 2 years ago
- Assignee set to Jesse Wolfe
Updated by Jesse Wolfe about 2 years ago
- Status changed from Investigating to In Topic Branch Pending Review
- Branch set to http://github.com/jes5199/puppet/tree/ticket/0.25.x/2816
This patch sets :reconnect to true, as suggested by the ticket.
Updated by Markus Roberts about 2 years ago
- Assignee deleted (
Jesse Wolfe)
Updated by James Turnbull about 2 years ago
Added this to the commit.
diff --git a/spec/unit/rails.rb b/spec/unit/rails.rb
index 0302873..f61288f 100755
--- a/spec/unit/rails.rb
+++ b/spec/unit/rails.rb
@@ -90,7 +90,7 @@ end
describe Puppet::Rails, "when initializing a mysql or postgresql connection" do
confine "Cannot test without ActiveRecord" => Puppet.features.rails?
- it "should provide the adapter, log_level, and host, username, password, and database arguments" do
+ it "should provide the adapter, log_level, and host, username, password, database, and reconnect arguments" do
Puppet.settings.stubs(:value).with(:dbadapter).returns("mysql")
Puppet.settings.stubs(:value).with(:rails_loglevel).returns("testlevel")
Puppet.settings.stubs(:value).with(:dbserver).returns("testserver")
@@ -105,11 +105,12 @@ describe Puppet::Rails, "when initializing a mysql or postgresql connection" do
:host => "testserver",
:username => "testuser",
:password => "testpassword",
- :database => "testname"
+ :database => "testname",
+ :reconnect => true
}
end
- it "should provide the adapter, log_level, and host, username, password, database, and socket arguments" do
+ it "should provide the adapter, log_level, and host, username, password, database, socket, and reconnect arguments" do
Puppet.settings.stubs(:value).with(:dbadapter).returns("mysql")
Puppet.settings.stubs(:value).with(:rails_loglevel).returns("testlevel")
Puppet.settings.stubs(:value).with(:dbserver).returns("testserver")
@@ -125,7 +126,8 @@ describe Puppet::Rails, "when initializing a mysql or postgresql connection" do
:username => "testuser",
:password => "testpassword",
:database => "testname",
- :socket => "testsocket"
+ :socket => "testsocket",
+ :reconnect => true
}
end
end
Updated by James Turnbull about 2 years ago
- Status changed from In Topic Branch Pending Review to Closed
Pushed in commit:b86decc0ea274eb6d9ffa3170fd4ec81d735519f in branch 0.25.x