Overview

What?

A simple pluggable Hierarchical Database.

Why?

Hierarchical data is a good fit for the representation of infrastructure information. Consider the example of a typical company with 2 datacenters and on-site development, staging etc.

All machines need:

  • ntp servers
  • sysadmin contacts

By thinking about the data in a hierarchical manner you can resolve these to the most correct answer easily:

     /------------- DC1 -------------\             /------------- DC2 -------------\
    | ntpserver: ntp1.dc1.example.com |           | ntpserver: ntp1.dc2.example.com |
    | sysadmin: dc1noc@example.com    |           |                                 |
    | classes: users::dc1             |           | classes: users::dc2             |
     \-------------------------------/             \-------------------------------/
                                \                      /
                                  \                  /
                           /------------- COMMON -------------\
                          | ntpserver: 1.pool.ntp.org          |
                          | sysadmin: sysadmin@%{domain}       |
                          | classes: users::common             |
                           \----------------------------------/

In this simple example machines in DC1 and DC2 have their own NTP servers, additionaly DC1 has its own sysadmin contact – perhaps because its a remote DR site – while DC2 and all the other environments would revert to the common contact that would have the machines domain fact expanded into the result.

The classes variable can be searched using the array method which would build up a list of classes to include on a node based on the hierarchy. Machines in DC1 would have the classes users::common and users::dc1.

The other environment like development and staging would all use the public NTP infrastructure.

This is the data model that extlookup() have promoted in Puppet, Hiera has taken this data model and extracted it into a standalone project that is pluggable and have a few refinements over extlookup.

Enhancements over Extlookup?

Extlookup had just one backend, Hiera can be extended with your own backends and represent a few enhancements over the base Extlookup approach thanks to this.

Multiple backends are queried

If you have a YAML and Puppet backend loaded and your users provide module defaults in the Puppet backend you can use your YAML data to override the Puppet data. If the YAML doesnt provide an answer the Puppet backend will get an oppertunity to provide an answer.

More scope based variable expansion

Extlookup could parse data like %{foo} into a scope lookup for the variable foo. Hiera retains this ability and any Arrays or Hashes will be recursively searched for all strings that will then be parsed.

The datadir and defaults are now also subject to variable parsing based on scope.

No CSV support by default

We have not at present provided a backward compatible CSV backend. A converter to YAML or JSON should be written. When the CSV backend was first chosen for Puppet the Puppet language only supports strings and arrays of strings which mapped well to CSV. Puppet has become (a bit) better wrt data and can now handle hashes and arrays of hashes so it’s a good time to retire the old data format.

Array Searches

Hiera can search through all the tiers in a hierarchy and merge the result into a single array. This is used in the hiera-puppet project to replace External Node Classifiers by creating a Hiera compatible include function.

Issue tracking

View all issues

Members

Manager: Daniel Pittman, Ken Barber, Moses Mendoza
Developer: Adrien Thebo, Aimee Fahey, Alanna Brown, Amy Hoang, Andrew Parker, Ann Franck, Arnan Outhaythip, Ben Ford, Ben Hughes, Branan Purvine-Riley, Brent Caldwell, Carl Caum, Carrie Odell, Celia Cottle, Char Moe, Charlotte Caswell, Chris Price, Chris Spence, Cody Herriges, Dan Bode, Daniel Pittman, Daniel Sauble, David Gwilliam, Deepak Giridharagopal, Devon St.Clair, Dominic Maraglia, Dustin Thomas, Eric Shamow, Fred Lifton, Fritz Siebert, Garrett Honeycutt, Gary Larizza, Hailee Kenney, Harry Pan, Heidi Page, Hunter Haugen, Jack Norman, James Sweeny, James Turnbull, Jason McKerr, Jeff McCune, Jeff Weiss, Jill Burrows, Joe Henderson, Joe Lokis, John Grim, Jose Palafox, Josh Cooper, Joshua Lifton, Julie Crawford, Justin Stoller, Kate Kockler, Kelly Perkel, Kelsey Hightower, Ken Barber, Ken Johnson, Kyle Smith, Luke Kanies, Matt Robinson, Matthaus Litteken, Max Martin, Michael Kincaid, Michael Stahnke, Michelle Carroll, Moses Mendoza, Nan Liu, Nick Fagerlund, Nick Lewis, Nigel Kersten, Patrick Carlisle, Pieter Loubser, Pieter van de Bruggen, Puppet Labs Security, R.I. Pienaar, Ralph Luchs, Randall Hansen, Rick Bradley, Ryan Coleman, Sam Myers, Scott Campbell, Scott Johnston, Teyo Tyree, Tinashe Mashungu, Zach Leslie
Reporter: Adrien Thebo, Aimee Fahey, Alanna Brown, Amy Hoang, Andrew Parker, Ann Franck, Arnan Outhaythip, Ben Ford, Ben Hughes, Branan Purvine-Riley, Brent Caldwell, Carl Caum, Carrie Odell, Celia Cottle, Char Moe, Charlotte Caswell, Chris Price, Chris Spence, Cody Herriges, Dan Bode, Daniel Pittman, Daniel Sauble, David Gwilliam, Deepak Giridharagopal, Devon St.Clair, Dominic Maraglia, Dustin Thomas, Eric Shamow, Fred Lifton, Fritz Siebert, Garrett Honeycutt, Gary Larizza, Hailee Kenney, Harry Pan, Heidi Page, Hunter Haugen, Jack Norman, James Sweeny, James Turnbull, Jason McKerr, Jeff McCune, Jeff Weiss, Jill Burrows, Joe Henderson, Joe Lokis, John Grim, Jose Palafox, Josh Cooper, Joshua Lifton, Julie Crawford, Justin Stoller, Kate Kockler, Kelly Perkel, Kelsey Hightower, Ken Barber, Ken Johnson, Kyle Smith, Luke Kanies, Matt Robinson, Matthaus Litteken, Max Martin, Michael Kincaid, Michael Stahnke, Michelle Carroll, Moses Mendoza, Nan Liu, Nick Fagerlund, Nick Lewis, Nigel Kersten, Patrick Carlisle, Pieter Loubser, Pieter van de Bruggen, Puppet Labs Security, R.I. Pienaar, Ralph Luchs, Randall Hansen, Rick Bradley, Ryan Coleman, Sam Myers, Scott Campbell, Scott Johnston, Teyo Tyree, Tinashe Mashungu, Zach Leslie
Support Staff: Adrien Thebo, Aimee Fahey, Alanna Brown, Amy Hoang, Andrew Parker, Ann Franck, Arnan Outhaythip, Ben Ford, Ben Hughes, Branan Purvine-Riley, Brent Caldwell, Carl Caum, Carrie Odell, Celia Cottle, Char Moe, Charlotte Caswell, Chris Price, Chris Spence, Cody Herriges, Dan Bode, Daniel Pittman, Daniel Sauble, David Gwilliam, Deepak Giridharagopal, Devon St.Clair, Dominic Maraglia, Dustin Thomas, Eric Shamow, Fred Lifton, Fritz Siebert, Garrett Honeycutt, Gary Larizza, Hailee Kenney, Harry Pan, Heidi Page, Hunter Haugen, Jack Norman, James Sweeny, James Turnbull, Jason McKerr, Jeff McCune, Jeff Weiss, Jill Burrows, Joe Henderson, Joe Lokis, John Grim, Jose Palafox, Josh Cooper, Joshua Lifton, Julie Crawford, Justin Stoller, Kate Kockler, Kelly Perkel, Kelsey Hightower, Ken Barber, Ken Johnson, Kyle Smith, Luke Kanies, Matt Robinson, Matthaus Litteken, Max Martin, Michael Kincaid, Michael Stahnke, Michelle Carroll, Moses Mendoza, Nan Liu, Nick Fagerlund, Nick Lewis, Nigel Kersten, Patrick Carlisle, Pieter Loubser, Pieter van de Bruggen, Puppet Labs Security, R.I. Pienaar, Ralph Luchs, Randall Hansen, Rick Bradley, Ryan Coleman, Sam Myers, Scott Campbell, Scott Johnston, Teyo Tyree, Tinashe Mashungu, Zach Leslie

Contributor License Signers: Luke Kanies, Dan Bode, Jose Palafox, Nigel Kersten, Daniel Pittman, Ken Barber, Jeff McCune, Cody Herriges, Nick Lewis, Matt Robinson, Teyo Tyree, Deepak Giridharagopal, Dominic Maraglia, Zach Leslie, Michael Stahnke, R.I. Pienaar, Pieter Loubser, James Turnbull, Rick Bradley, Amy Hoang, Hunter Haugen, Garrett Honeycutt, Nick Fagerlund, Eric Shamow, Randall Hansen, Gary Larizza, Ben Hughes, Adrien Thebo, Max Martin, Michael Kincaid, Josh Cooper, Matthaus Litteken, Scott Campbell, Scott Johnston, Daniel Sauble, Jason McKerr, Nan Liu, Kelsey Hightower, Carl Caum, Joshua Lifton, Pieter van de Bruggen, Devon St.Clair, Patrick Carlisle, Ryan Coleman, Justin Stoller, Jeff Weiss, Ben Ford, Ralph Luchs, Chris Price, Chris Spence, Fred Lifton, Celia Cottle, David Gwilliam, Hailee Kenney, Branan Purvine-Riley