learning puppet chapter 2

LEARNINGPUPPET - 02

SlideBook (Book on slides)

Inspired by Slidedoc -

http://www.duarte.com

- Vishal Biyani

www.vishalbiyani.com

Puppet language language for “No Ruby” person!

www.vishalbiyani.com Learning Puppet

Puppet language

Puppet is a DSL based on Ruby – an object oriented programming

language. So do you need to know Ruby in and out to work with

Puppet ? Not necessarily! The whole idea of this chapter is to

introduce enough puppet & ruby to work with it.

Puppet is declarative which has two implications

• You define the final desired state not how to achieve it

• The sequence in which tasks are carried out is not guaranteed unless you explicitly specify it.

You don’t have to master all of the concepts right here – but do

spend some time understanding syntax in this chapter. You can

3|

spend some time understanding syntax in this chapter. You can

always come back to understand any specific part which might

not be clear.

As with any language – there are sometimes multiple ways to do the

same thing and this is true for Puppet/Ruby too.

Lastly covering an entire language in such short span is a challenge.

So do refer to guides & references:

Some useful links:

• https://docs.chef.io/ruby.html provides ruby basics needed for Chef framework. A good quick introduction

• Similar quick guide from Puppet documentation: http://docs.puppetlabs.com/puppet/3.7/reference/lang_visual_index.html

• For the super curious once for quickly learning Ruby there is no better source than https://rubymonk.com/ that I know of!

Some language features are best explained within a context – with a use case or a problem. Hence those

parts of puppet language have been intentionally skipped here and will be covered as we go.


Variables & Expressions

Lot of times we also need to resolve variables within a string for ex:

Variables can be declared with names starting

with $ and value can be a literal value or a

function/expression which resolves to a value.

1 $name = "vishal"

2 $number = 10+2

As long as the value that a variable resolves to is

compatible with data type of intended target, it

can be used as part of expression, function or

attribute! You will also see variables in following

6 $f_name = "${conf_dir}/.conf"

3 $users = ['us1','us2']

4 $users += ['us3']

Here we are resolving variable named conf_dir within a string – called

interpolation. Similarly addition of variables of same type is easy:

Now coming to one of most important features of variables – you can

assign a variable only once in a given scope. In a new scope you can assign a

new value to same variable, for example below the $name is reassigned but

in a new scaope – scope of the class testClass:

1 $name = "Default Name"

2 class testClass {

3 $name = "Test Class Name"

4 }

4|

attribute! You will also see variables in following

form:

4 $tomcat::config::tc_port

What does this mean? It means a variable called

“tc_port” from class tomcat::config. We will see

the classes & modules shortly. You can access

variables from parent scopes but not from child

scope (And we will scope in detail soon). For

example you can access a top level variable from

class level scope but not other way around. Also

there is a special notation for variables at top

scope – the scope is blank. In following example

we are getting “osfamily” from top scope

4 $::osfamily

4 }

Lastly – a variable can not be resolved unless it is defined. Which means

unlike rest of Puppet – order matters while defining variables.

Expressions

Most expressions in puppet are usual suspects of any programming

language like == (Equality), != (Non Equality), ,< , > , <= , >= , and, or, ! , +, -,

/, *, % etc. Couple of them that need some attention are not found in some

languages are:

=~ Left hand is a string & right hand is a regex – if match is found then returns true

!~ Left hand is a string and right is a regex – if match is foundreturns false

in Left hand operand should be string and right hand could be string, array, hash. It checks if string on left side is “in” the right side variable. Returns true if found


Of Conditions

We have quite a few friends in conditional

statements in Puppet – if, unless, case &

selectors. This part is quite easy so let’s run over it

quick!

Our if, elseif & else block is dead simple1 if $osfamily == 'linux'{

2 # Do something

3 }

4 elseif $osfamily == 'windows'{

5 # Do something else

6 }

7 else

8 # Do something horrible here

But if we use regular expression in if block – there

Unless is exact opposite of if – but without additional else & regex magic:

1 unless $osfamily == 'linux'{

2 # Only for Linux family things

3 }

1 case $osfamily {

2 'Linux' : {include role::linux}

3 'Windows': {include role::windows}

4 default : {include role::default}

5 }

Case statement is also similar to other languages, almost like having

multiple if conditions & a default condition which will be executed if none of

previous conditions meet. And oh they also support regex capture variables if

you are using a regular expression to evaluate the case expression!

Finally the selectors! They are a special case of case conditional expression

and differ in behavior. In case statement we execute a piece of code once the

5|

But if we use regular expression in if block – there

is some black magic happening. If you look at

regular expression below – it is expecting string

some and varying digits after it – like some01,

some02 etc. Now these might be your hostnames

and would be good to get them inside the

execution block – as shown in comments within

the block. They have a name: “Regex Capture

variables”

10 if $hostname =~ /^some(\d+)\./ {

11 # Magic happens here with $0 and $1

12 # If some2 is passed then $1 has value

of 2

13 # and $0 has value of some2 - entire

match

14 }

and differ in behavior. In case statement we execute a piece of code once the

condition in a block is met, but in case of selector statement when the

expression evaluates to true – a value is returned. This value can be assigned

to a variable and then used later in flow. Imagine a situation where your

windows user name and linux username for certain operation is different –

and you want to use a selector to determine the user name dynamically

based on OS as shown in example below:

1 $user_name = $osfamily ? {

2 'windows' => 'win_admin',

3 'linux' => 'root_user',

4 default => 'root',

5 }

In above code – if $osfamily is windows then the username is different and if

it is linux then it is a different user.


Functions & Classes Functions when it comes to puppet have a special

meaning. They are already defined pieces of code

which you can use to get things done. But more

importantly – functions are evaluated at

compilation time. Which means when your

manifest convert to catalog – the functions either

return their values or modify catalog based on

function! We have already used a function on

previous slide – which includes roles:

A class is a logical grouping of resources which can be accessed by class

name. There are two parts to it- first is you define the class and secondly you

call (Called declare) it. The execution happens only when you make a call.

Let’s first see how to define classes:

2 'Linux' : {include role::linux}

Another function which returns a value of true or

false based on input:

1 class test {

2 # Resource declarations here

3 }

4 class testParam ($name = 'default_value') {

5 # You can use the $name in any resource declaration

6 }

In first example above we defined a class which does not take any

parameters. Second one takes a parameter called “name” and if the caller

does not provide any value then “default_value” is chosen. This is just to

safeguard in case no value is provided & is a good practice to have a default.

The resources inside class are defined as you would do normally and they

6|

false based on input:

1 str2bool($is_virtual)

Functions have following characteristics:

• Rvalues type of function retuen some value (Like

str2bool)

•Statements kind of function will do some work but

wont return anything (Like include)

•Functions can take arguments – based on their

definition of what kind of & how many arguments

they take.

•Functions are run at compilation time and hence

on master – so any agent specific data/code should

be done using functions. Instead use

resource/custom facts (Which we will see soon)!

The resources inside class are defined as you would do normally and they

are belong/related to class.

The classes should be in files with name of class & ideally should be in

manifests directory within a module. You can of course place classes within

main manifest or manifests which are imported into main manifest, but for

now we won’t look at those cases.

Puppet also has concept of class inheritance but it should be used sparingly.

We will see the concept and when to use it in later chapters including how to

override parent resource attributes


Declaring ClassesBefore we deep dive into declaring/calling classes

let’s understand a fundamental difference due

the way this evolved with Puppet:

•Include-like way: This was introduced in 3.x &

later versions. You can declare a class any number

of times but it will be called only once. It uses a

combination of external data & defaults to set

values of parameters. (Good practice of

externalizing configuration data from code). It is

suggested to use this way.

•Resource-like way: This is old way – in which

Let’s say you want to include a class into another one, this can be done

using “require” funcion which BTW is different than require used for

ordering. The require’s behavior is include like and you can pass list

separated by comma, array etc.

There is third include like way of declaring classes which uses hiera but we

will see that when we talk about hiera!

Now let’s look at resource like way of declaring classes which is not

suggested after version puppet 3. It is as good as declaring any other

resource:

6 define nix::tomcat(){

7 require java

8 }

10 class {‘nix::tomcat’}

7|

•Resource-like way: This is old way – in which

you declare a class the way you declare a

resource. You MUST DECLARE ONLY ONCE! You

can override parameters else they will be taken

from external data or lastly form defaults!

Now let’s see in how many different ways we can

declare classes.

Include: this is of course include like way to

declare classes and can take many forms

There is another way to assign classes to some nodes is via something called

ENC - External Node Classifier! Why would you do that?

External node classification is process of defining which node belong to

which classes. Doing it separate from automation tool provides loose

coupling & separation of concerns. ENC can be an external script or a

system which when called gives data of which nodes need to be applied

which classes and the automation tool accordingly applies the same. For

example many organizations store the node data in LDAP and in such case

you will fetch data from LDAP and then run puppet based on which nodes

needs which class. This is a topic which needs more detailed treatment and

we will not go into much details on this one at the moment.

1 include nix::tomcat

2 include nix::tomcat, nginx

3 clas_list = ['nix::tomcat','nginx']

4 include clas_list

10 class {‘nix::tomcat’}


Resource & types

Some more ground rules for resources:

•You can not declare the same resource twice! Imagine if in one place you

said file should be present and other place absent. Since puppet is declarative

language – it has to have a clear state defined. Although you can add

attributes to already defined resource for example:

1 file { "/home/vagrant/testFile"

2 ensure => present,

3 owner => 'root',

4 group => 'root',

5 mode => '0400',

6 }

Lifecycle of a resource:

When a resource declaration is executed, following things happen:

• The resource’s current state is read.

• If there is a different in current state vs. desired state then the change the

resource to desired state.

• The change in state is logged as event. This event will appear in log files

& reports.

The title of type in this case is

Every resource has a type – in our example below

it is file. It can be a service you want to manage or

a package that needs installed etc.

8|

attributes to already defined resource for example:

1 file { '/home/vagrant/testFile'


3 }

4 File['/home/vagrant/testFile']{

5 owner => 'root',

6 group => 'root',

7 mode => '0400',

8 }

The title of type in this case is

‘/home/vagrant/testFile’ & rest of the details are

”attributes” with values (Equivalent to

parameters in context of resources). Now the

“ensure” attribute will ensure that the file is

“present” – meaning it exists. The value of ensure

might vary from type to type – for example for a

service you might say “running” . So we are

defining the desired state of the a resource. How

to achieve it is what puppet does it for us on

multiple platforms. So in short a resource has

primarily three things – type, title & one or more

attributes. Attributes also might vary form type

to type.

•The order in which you declare resources in file does not matter. Meaning

during execution the order might change for execution optimization. So if you

want to introduce sequence in resource execution – you will have to use

relationships, which we will see shortly.


Objects & Scope 1/2 Now in real world scenarios when you configure nodes/machine – one

node might be Database server while another might be a Application

Server and so on. So you need to define what is a node’s identity! (Also

called node classification). In Puppet we can define a node’s identity in a

file /etc/puppet/manifests/site.pp – and this is our next level of structure.

Let’s say you have a Database node – you might want to configure OS

level resources first – which will be common across all type of nodes. After

that you might apply database specific resource definitions. These two

should be ideally in separate modules. So our node definition looks like:

Remember that everything in Puppet is

resource? But as we add more resources – there

is a need of a structure to manage them. It’s

almost like game of lego – you first get smaller

blocks and combine them to make bigger

structures.

We can combine related resources together to

create a class. And we can combine smaller

classes to create bigger classes. For example we

50 #/etc/puppet/manifests/site.pp

51 node 'database.mycompany.com' {

52 include os_common

53 include database

9|

classes to create bigger classes. For example we

might have a class for managing a file:

42 class base::home {

43 file { '/home/vagrant':

44 owner => 'root',

45 group => 'root',

46 mode => '0644',

47 }

48 }

Next we combine classes & configuration data to

create modules. These modules & classes can be

called from outside. Again we won’t go to depths

of module right away but consider that as next

logical level of structure.

53 include database

54 }

Here the modules os_common & and database is applied on a node

named “database.mycompany.com”

And now we have last structural element – “top”! It will be more clear

when we talk about scope in next slide but for now it is a level which

applies to all nodes in your infrastructure.


Objects & Scope 2/2 What is scope? A way to isolate & limit reach of things. In case of puppet

specifically scope applies to variables & resource defaults. Let’s checkout various

scope levels and their interplay:

Top Scope: is what you define at top most level and is accessible everywhere!

Node Scope: Things that you define in your node definition exist at node scope.

Code you define at node scope is available to classes included in that node but

not at top scope.

Local scope is something local to a class, resource declaration or a method etc.

Overriding is the means to override a variable's value form parent scope. Let’s

say we define “message” at node scope and need to override in a class:

We saw four levels in last slide: Resource, Class,

Node & top. Module is just a structural element

which encompasses classes and is not a scope level

really. Scopes form a hierarchy – for example you

can access top scope code at node scope but not

other way around. Basically a parent scope

code/variable can be accessed at child level but child

level can not access things at parent level. Below

picture illustrates the scopes at various levels.

topTop Scope

1 node example.com {

2 $message = "This is node"

3 include

4 }

10|

resource

resource

resource

resource

resource

classclass

module

class filesconfig

Node definition

Local Scopes

Node Scope

4 }

5 class test_class {

6 $message = "This is class overriding node"

7 notify{"Test message = $message":}

8 }

What will get printed above? Message from class due to overriding.

The scopes of top & class get names – i.e. they can be references using a

notation like $::variable etc. But node & local scopes don’t get names hence

they can not be referenced. Finally the variables which are not in your scope and

have name (i.e. do not belong to node/local) can be accessed with their full

names. For example let’s say the class tomcat has a parameter named port, we

can call it with it’s fully qualified name:

1 include tomcat::config

2 $port = $tomcat::config::port_num


Relations & Ordering We saw that the sequence of execution of resources

can be different than what we see in manifest. But

there are times when you need to enforce order –

and that’s where ordering attributes help us. There

are four attributes – or metaparameters which

enable ordering:

Attribute Effect

before Execute current resource before and

Let’s say we want to ensure package “ssh” is installed first and then the file

“/etc/ssh/ssh_config” is configured. So there are two ways to do it (With only

relevant attributes:10 file {'/etc/ssh/ssh_config'


12 require => Package[ssh]

13 }

14 package {'ssh'

15 ensure => installed,

16 }

18 file {'/etc/ssh/ssh_config'


20 }

21 package {'ssh'

22 ensure => installed,

23 before => File['/etc/ssh/ssh_config']

24 }

Similarly let’s say we want the service SSH to be running only after the file

“/etc/ssh/ssh_config” is configured. In addition the service SSH should be

11|The notation -> (ordering) and ~> (Ordering + Refresh) are equivalent to the four attributes above based on direction you use them. You can use them or be more explicit and use the attribute names. These arrows are lovingly called “chaining arrows”

before Execute current resource before and target resource later

require Execute current resource later and target resource before

notify Execute current resource before, target resource later & refresh target resource of current one changes

subscribe Execute target resource before, then execute current resource, and refresh current if target has changed

As you have noticed – before and require are same

things but depends on where you declare it. Ditto

with notify & subscribe but with a refresh included.

Quite confusing? – let’s look at some example.

“/etc/ssh/ssh_config” is configured. In addition the service SSH should be

restarted every time the file changes. Again two ways we can configure it:26 file {'/etc/ssh/ssh_config'


28 notify => Service['ssh'],

29 }

30 service {'ssh'

31 ensure => running,

32 }

34 file {'/etc/ssh/ssh_config'


36 }

37 service {'ssh'

38 ensure => running,

39 subscribe => File['/etc/ssh/ssh_config']

40 }

• You can refer an array of resources in value for attributes. For example you can

restart SSH and some other service as file changes.

• There are more complex scenarios where you might want to use “resource

collectors” – but more on that later ☺


What did we learn? Although we covered quite a few

things, we have omitted many

details and certain concepts

altogether in this chapter. This is

intentional and we will pick up

those concepts with examples so

that they make lot more sense

than bunch of theory concepts.

Let’s start building stuff now!

This chapter seems like lot of

theory but as stated in

introduction – do a sanity round

and come back to this chapter

every time you come across a

certain concept and need help

implementing it. We have

covered quite a few things and

we will start using them in

12|

Let’s start building stuff now!we will start using them in

upcoming chapters.

learning puppet chapter 2

Software