Feature #1866

Support for specification of multiple servers

Added by Ian Turner over 3 years ago. Updated 2 months ago.

Status:Investigating Start date:01/13/2009
Priority:Normal Due date:
Assignee:- % Done:

0%

Category:plumbing
Target version:3.x Estimated time:4.00 hours
Affected Puppet version:0.24.7 Branch:
Keywords:load balanced, multiple masters, high availability
Votes: 8

Description

At the moment it is only possible to tell puppetd about one puppetmaster: Engaging multiple servers requires load balancing equipment such as that employed by the reporter of Feature #1557. The simplest (for administrators) way to address this issue is to enable round-robin server access within puppetd itself. This is the technique employed extremely effectively in openvpn.

I propose two configuration changes: The “server” directive should accept a comma-seperated list of servers, and a new “randomserver” boolean directive indicates how the server list is traversed.

Pseudocode to be executed at the start of each puppet run:

# if randomserver and lastserver = NIL
  # serverlist <- rotateleft(serverlist, rand(length(serverlist))
# nextserver <- NIL
# trialcount <- length(serverlist)
# loop until nextserver ≠ NIL
  # trialserver <- first(serverlist)
  # if retrievecatalogok(trialserver)
    # nextserver <- trialserver
  # else
    # server <- rotateleft(server, 1)
    # trialcount <- trialcount - 1
    # if trialcount = 0
      # error("no working servers")    
# lastserver <- nextserver

History

Updated by Ian Turner over 3 years ago

As an addendum, I would propose that randomserver’s default value be false.

Also, I should give credit where due and state that this entire proposal is copied from the way that openvpn does things; to quote from an example openvpn configuration:

# The hostname/IP and port of the server.
# You can have multiple remote entries
# to load balance between the servers.
remote server-a.example.com 1194
remote server-b.example.com 1194
remote server-c.example.com 1194

# Choose a random host from the remote
# list for load-balancing. Otherwise
# try hosts in the order specified.
;remote-random

Updated by James Turnbull over 3 years ago

  • Status changed from Unreviewed to Needs Decision
  • Assignee set to Luke Kanies
  • Target version set to 4
  • 3 changed from Easy to Medium

Updated by Luke Kanies over 3 years ago

  • Status changed from Needs Decision to Accepted

I don’t know exactly where this code should be, though. Would you want every operation to load balance, or would you want a given client to pick a single server and stick with it?

Updated by Ian Turner over 3 years ago

I don’t have a strong opinion on this question; what seems both easiest and most reliable to me would be to pick a single server for each run. That is, a single invocation of —onetime would go to a single server, but the server could change after every runinterval.

It also makes sense to me to stick to a single server unless it has a problem, so that we don’t keep trying to hit dead servers (that’s what this “lastserver” variable is about), but I don’t really view that as essential. You could argue that it’s better to try a random server each time so as to quickly redistribute the load after a failure.

Updated by Nigel Kersten over 3 years ago

How will this work with environments? Isn’t this going to hit exactly the same problem as #1557 ? Where the server may not have any environment information for a given client on requests subsequent to the initial connection?

Updated by Ian Turner over 3 years ago

It shouldn’t be an issue as long as the server is not changed within a run.

Updated by Luke Kanies about 3 years ago

  • Target version changed from 4 to 2.6.0

I’ll try to get this into the next release for a customer.

Updated by Wolfgang Schricker almost 3 years ago

  • Assignee deleted (Luke Kanies)

Hello, my vision is a scenario with multiple (here two) servers:

  • puppetmaster1.my.domain gives basic configurations for all clients
  • puppetmaster2.my.domain gives additional any classes for any clients

The configuration should be “AND combined”.

Updated by Ian Turner almost 3 years ago

Wolfgang,

What you are proposing is very different from the proposal in this Wishlist entry. Also, your request would require major changes to the existing Puppet paradigm. If the ability to combine manifests from multiple sources is very important to you, and you are willing to dedicate a lot of time and money to that feature, then I would suggest you open a new Feature request and/or get in touch with Reductive Labs for paid work.

Cheers,

—Ian

Updated by Luke Kanies almost 3 years ago

I agree with Ian – that’s a big change, and it shouldn’t be part of this ticket.

You can already get that to some extent by splitting file serving and node management onto separate servers.

Updated by Luke Kanies over 2 years ago

  • Priority changed from Normal to High

I promised this to a customer, so hopefully we can get it in this release.

Updated by James Turnbull over 2 years ago

  • Target version changed from 2.6.0 to 2.7.x

Updated by Mario Verbelen almost 2 years ago

this is looks like a important feature for scaling I’m interested for high availability in this case

Think about the multiple certification in this case (http://projects.puppetlabs.com/projects/puppet/wiki/Multiple_Certificate_Authorities)

If I see the last note on that there is also work in progress about that I don’t now if you should include that into this ticket or create a new one?

Updated by Nigel Kersten about 1 year ago

For the people who care about this issue, would you be satisfied with #3669 and having this be implemented as part of the SRV record work?

Updated by Nigel Kersten about 1 year ago

  • Priority changed from High to Normal
  • Target version changed from 2.7.x to 3.x

Updated by Rich Rauenzahn 9 months ago

I was just looking for a similar feature — but my use case is a little different.

I don’t want round robin — I only want the alternate puppetmasters as failovers if the primary isn’t responding, and then failback when the primary is back up.

If it is possible to incorporate that into this feature, that would be great.

(Our use case, btw, is running puppetd apply from cron.)

Updated by Nigel Kersten 9 months ago

Rich, would priorities in SRV records work well enough for that purpose for you?

I’m considering closing this feature request in favor of doing it as SRV records.

Updated by Rhys Morgan 6 months ago

This would be highly useful for an implementation we are currently considering where we essentially have tiers of puppetmasters and would like to have a comma delimited list of puppetmasters in the agent config which will be attempted in order, moving up a tier each time.

Updated by Nigel Kersten 6 months ago

For everyone commenting on this, please think about whether SRV record support that we’ve merged will suffice for what you need.

Updated by Nigel Kersten 6 months ago

  • Status changed from Accepted to Investigating

Updated by Wil Cooley 4 months ago

Reviewed for triage-a-thon 21-01-2012; ticket looks ok to me for 2.7.9.

Also available in: Atom PDF