Using Puppet Automation to Configure and Deploy Oracle Solaris Zones

Picture1I just got back from Oracle OpenWorld 15 where I co presented on this topic with our CTO, Brian Bream.  I wanted to share this with you folks as I ramp up on puppet, docker and openStack in general.

 

What is Puppet?

  • Open source configuration management
  • Written in Ruby
  • New C++ implementation of the core server – puppetmaster
  • Defines Server configurations in code
  • Puppet is used to ensure configuration consistency across the data center
  • Used to install and configure applications
  • Scales to large environments including cloud based solutions
  • A few servers to very large server environments
  • Multi OS support
  • Solaris, Linux, and Windows

 

Puppet is produced by puppet labs which was founded in 2005 by Luke Kanies.  It is written in Ruby and is released as free software under the GNU General Public License.  There is a commercial version of Puppet called Puppet Enterprise which has a GUI front end and other enhancements.

It is supported on multiple flavors of linux including debian based and RedHat based distros.

 

The popular IT automation software, Puppet, is included in Oracle Solaris 11.2.  Puppet helps you manage IT infrastructure by automating repetitive tasks, deploying critical applications rapidly, and proactively managing changes required in a system.  Puppet automates tasks such as provisioning, configuration, compliance, and software management. Puppet can scale from simple deployments to complex infrastructure, from on-premise to cloud deployments.  With enhanced support for Oracle Solaris technologies, administrators can host their Puppet masters on a mission-critical environment and extend their automation to managing a completely heterogeneous data center environment.

 

Picture2

 

As you can see from the depiction above, puppet is supported across a number of heterogeneous operating systems.  You can have a Puppet Master on AIX, and have agents on Solaris and Oracle Linux for example.

 

Puppet can be useful in maintaining Configuration Management:

  • Maintain a consistent configuration across multiple platforms. Don’t worry about OS differences, credentials, etc.
  • Puppet agent can self heal client configuration anomalies automatically. Big brother really is watching!
  • Commit changes organization wide in minutes- across multiple OS platforms
  • Timely security vulnerability remediation (poodle, GHOST, Venom, HeartBleed etc..)
  • Easily maintain compliance by enforcing your security policies (SOX, HIPAA, PCI, FERPA, FISMA)

 

It can also be leveraged as a capable Organization wide monitoring tool:

  • Gain visibility into all nodes and detect malicious patterns. Be able to react with variable decision making.

 

Puppet can also be used to facilitate Provisioning:

  • Deploy numerous workloads (linux/solaris containers, AWS, VMware, OpenStack, Hadoop, etc) in a fraction of the time it used to take.

 

Puppet can also react based on the value of a variable. For example, “If I see permissions on /etc/shadow have changed, kick off a special script that does a deep level scan for other attempts at breaking in”.

 

Picture3

Puppet is a tool designed to manage the configuration of Unix-like and Microsoft Windows systems declaratively. The user describes system resources and their state, either using Puppet’s declarative language or a Ruby DSL (domain-specific language). This information is stored in files called “Puppet manifests”. Puppet discovers the system information via a utility called Facter, and compiles the Puppet manifests into a system-specific catalog containing resources and resource dependency, which are applied against the target systems. Any actions taken by Puppet are then reported.

 

 

Picture4

Resources are used to model system configuration.  For example:

  • Components of a package that should be installed
  • A system service that should be enabled
  • A file that should exist with specific permissions

 

Puppet’s Resource Abstraction Layer (RAL) consists of

  • High level model called a type
  • A platform specific implementation called a provider

 

Manifests can be identified by any of the following:

  • Files that contain puppet code. Saved with a .pp extension
  • Site manifest (site.pp) typically used to declare default settings organization wide

Ownership/permissions on /etc/shadow, /etc/sysctl.conf, /bin/login, etc

  • Node manifest can be used to tune specific hosts in a more modular approach

Validate IP address, enforce security settings (selinux/iptables), monitor filesystem capacity

  • Most manifests are contained in Modules
  • Can dynamically adjust behavior based on variable

I found something, because of that I need to perform X, Y and Z

  • Use classes and resources to describe objects and their intended state

The Site.pp manifest file is normally used for things like permissions on critical files, validating contents of other critical files etc.  In contrast, the Node.pp would describe a number of classes and resources specific to that node.  Other manifests are typically contained inside a module such as an Apache module, MySQL module or a sendmail module.  Below is a sample hierarchy of how all the parts of a sample Apache module fit together:

Apache module -> apache manifest -> web server class -> httpd.conf resource

Modules are self contained bundles of code and data. Nearly all Puppet manifests belong in modules (except for the site manifest)

  • They are a hierarchical description of relevant entities such as file contents, web server virtual host definition, /etc file configurations
  • Lots of modules are available for download at puppet forge– don’t reinvent the wheel
  • If you really need to create your own module, use the generate command to create new modules – it does heavy lifting for you such as creating the filesystem directory structure and placing the appropriate skeleton files in place for you
  • Plugins can be used to deploy custom “facts” and “types” to clients – resides inside module. Specific directory structure inside module determines where information goes
  • Modules can contain Classes that are modular and can be reused elsewhere by using the include directive.  Here are some examples:

Apache module
-> class: install and enable Apache
-> class: enable PHP for use with Apache
-> class: turn on mod_rewrite

Some products that are comparable to puppet would be for example:

Tivoli Endpoint Manager (IBM BigFix)

Enterprise solution, costs $, can manage patching, software distribution, OS deployment, mobile devices too

Oracle Ops Center

More of a focus on Oracle Solaris and Oracle Linux, patching, provisioning, compliance reporting.

Vagrant

Focused on deploying virtual environments. Relies on puppet or other configuration management software to complete solution

SaltStack Enterprise

CLI based, utilizes push method for communication with clients, very resilient and scalable, appeals more to sysadmin audience

Chef

Similar to Puppet, push feature still “immature”, requires other components such as workstation and GIT in place to manage the configuration as well as knowledge of Ruby programming. Appeals more to development crowd.

 

You can utilize Puppet to manage Solaris Containers.  Below I will walk you through the high level steps needed to deploy a container automatically using puppet code:

 

  • Install your puppet master and puppet agent(s). Container will usually live on agent(s)
  • Create zfs filesystem on agent(s) for containers to live in
  • Create zone config file describing the container (zonepath, IP address, shared filesystems, etc)
  • Create entry in site manifest for agent node(s) where containers will live
  • Restart puppet agent service on agent to kick off container build
  • Install puppet agent in container, register with master and add stanza in site manifest

 

-- Master
# pkg install puppet
# svcadm enable puppet:master

-- Agent
# pkg install puppet
# svccfg -s puppet:agent setprop config/server=puppet-master
# svccfg -s puppet:agent refresh
# svcadm enable puppet:agent
# puppet agent --test

-- Master
# puppet cert list
"puppet-agent1" (SHA256) 09:8B:A0:55:17:73:8E:9D:32:17:E3:E2:EB:29:FF:2D:12:44:3A:46:7C:01:B1:6A:29:A3:8B:29:BF:FA:E6:61
# puppet cert sign puppet-agent1

Puppet Master and Agent communication have now been established.

 

Create the ZFS filesystem on the agent where the container will live. Can be a separate pool if you desire

# zfs create -o mountpoint=/zones rpool/zones
# chmod 700 /zones

Create /etc/puppet/assets/mywebserver.cfg on Master with contents below:

create -b
set zonepath=/zones/alpha1
set autoshutdown=shutdown
set autoboot=true
add anet
set linkname=net0
set lower-link=auto
set configure-allowed-address=true
set link-protection=mac-nospoof
set mac-address=auto
end

Add stanza for agent machine in the site manifest file on master (/etc/puppet/manifests/site.pp)

node puppet-agent1 {
          zone { 'mywebserver':
               zonecfg_export => '/etc/puppet/assets/mywebserver.cfg',
               name => 'alpha1',
               iptype => 'exclusive',
               ensure => 'running',
          }
}

Restart the puppet:agent service on agent1 to start the container build process

# svcadm restart puppet:agent
.
.
.
# zoneadm list –ivc

ID      NAME      STATUS      PATH                BRAND      IP
0       global    running     /                   solaris    shared
1       alpha1    running     /zones/alpha1       solaris    excl

 

Log into the container and install, configure and start the puppet agent

# pkg install puppet
# svccfg -s puppet:agent setprop config/server=puppet-master
# svccfg -s puppet:agent refresh
# svcadm enable puppet:agent
# puppet agent --test

 

Troubleshooting Container Deployment:

  • Verify syntax of zone configuration file ahead of time by creating a zone with that file. Make sure to completely mark inactive, uninstall and delete the zone before letting puppet run or it will fail.
  • When managing the container, recall that we told puppet to ensure that the zone was running. This means that if you halt the zone, the next time the agent fires it will boot the zone and attempt to put it into the running state. Disable the agent during maintenance to ensure no surprises!
  • Review the puppet log file located at /var/log/puppet/puppet.log for clues as to why things aren’t working. Puppet does a decent job at telling you exactly what the problem is.

 

Just a few of the additional Puppet modules available at Puppet Forge:

Apache – create and configure a web server
MySQL – install, configure and manage a MySQL database
Samba – create a fileserver that can be accessed by Windows clients
PHP – configure and manage your PHP installations
DHCP – install and manage a DHCP server instance
NTP – configure NTP server(s) for time management
SMTP – install and configure a postfix Mail Transfer Agent
Over 3,600 pre-built modules to choose from. Or build your own!

You’ll find a huge assortment of modules at puppet forge. It’s a community driven collection of modules that can perform a variety of tasks. Check there before making your own module.

Website: http://forge.puppetlabs.com

 

Some additional resources to be aware of:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s