• We will be using a virtual machine in the faculty's cloud.
• When creating a virtual machine follow the steps in this tutorial.
• When creating a virtual machine in the Launch Instance window:
  • For Availability zone, choose CAMPUS, CI or hp
  • Select Boot from image in Instance Boot Source section
  • Select SCGC Template in Image Name section
  • Select a flavor that is at least m1.medium.
• The username for connecting to the VM is student
• First, download the laboratory archive:

  [student@scgc ~] $ cd scgc
  [student@scgc ~/scgc] $ wget --user=<username> --ask-password https://repository.grid.pub.ro/cs/scgc/laboratoare/lab-07.zip
  [student@scgc ~/scgc] $ unzip lab-07.zip

After unzipping you should have a KVM image file (puppet.qcow2) and a script used to start the VM (lab07-start-kvm).

To start the VM, run the startup script:

student@scgc:~/scgc$ ./lab07-start-kvm

The KVM virtual machine for the lab will boot (can take up to 2-3 minutes).

In order to access the VM, use the following IP address (the password is student):

student@scgc:~/scgc$ ssh student@10.0.0.2

Puppet is a configuration management tool. In order to describe the necessary configurations, Puppet uses its own declarative language. Puppet can manage both Linux and Windows systems.

Puppet uses a resource as an abstraction for most entities and operations to be performed on a system. As an example, the state of a service (running/stopped) is defined in Puppet as a resource.

Use the puppet resource service command to see the system services from Puppet's perspective.

[root@puppet ~]# puppet resource service
service { 'apparmor':
  ensure => 'running',
  enable => 'true',
}
service { 'apparmor.service':
  ensure => 'running',
  enable => 'true',
}
service { 'autovt@':
  ensure => 'stopped',
  enable => 'true',
}
service { 'autovt@.service':
  ensure => 'stopped',
  enable => 'true',
}
service { 'bootlogd.service':
  ensure => 'stopped',
  enable => 'false',
}
service { 'bootlogs.service':
  ensure => 'stopped',
  enable => 'false',
}
...

The previous command syntax is:

• puppet command - used for accessing most of Puppet's features
• resource subcommand - interacts with available Puppet resources
• service parameter - type of the resources to be shown

Besides services, other Puppet resource examples are:

• users
• files or directories
• packages (software)

Show the resource representing the root user account, using the command: puppet resource user root

[root@puppet ~]# puppet resource user root
user { 'root':
  ensure             => 'present',
  comment            => 'root',
  gid                => 0,
  home               => '/root',
  password           => '$6$SWpfJK2ozbQ.bFA9$/[...]',
  password_max_age   => 99999,
  password_min_age   => 0,
  password_warn_days => 7,
  shell              => '/bin/bash',
  uid                => 0,
}

The resource structure contains the following elements:

• Type of resource: for this example, user
• Name of the resource: 'root'
• Attributes of the resource: ensure, comment, gid, home etc.
• Each attribute has a certain value

The previous syntax forms the “resource declaration”.

Besides services and users, Puppet implements a lot of other types of resources. In order to show them, use the command puppet describe –list

[root@puppet ~]# puppet describe --list
These are the types known to puppet:
augeas          - Apply a change or an array of changes to the  ...                                                                                          
computer        - Computer object management using DirectorySer ...                                                                                          
cron            - Installs and manages cron jobs                                                                                                             
exec            - Executes external commands                                                                                                                 
file            - Manages files, including their content, owner ...                                                                                          
filebucket      - A repository for storing and retrieving file  ...                                                                                          
group           - Manage groups
...

Using the puppet resource command, we can create new resources. Generic syntax is:

puppet resource type name attr1=val1 attr2=val2

If we want to create the user gigel so that:

• the user home directory is /home/gigel
• the default shell is /bin/sh

The Puppet command for this is:

[root@puppet ~]# puppet resource user gigel ensure=present shell="/bin/sh" home="/home/gigel"
Notice: /User[gigel]/ensure: created
user { 'gigel':
  ensure => 'present',
  home   => '/home/gigel',
  shell  => '/bin/sh',
}

Open the /etc/passwd file and check if the user has been created.

In order to remove a resource, the ensure attribute must be set to absent.

As an example, to remove the user gigel that we have previously created:

[root@puppet ~]# puppet resource user gigel ensure=absent
Notice: /User[gigel]/ensure: removed
user { 'gigel':
  ensure => 'absent',
}

Chech the /etc/passwd file to see if the user was actually removed.

Even though we can create, modify or remove resources from the command line, using puppet resource commands, this is not a scalable approach and not appropriate for complex scenarios.

A better solution would be:

• declaring resources in a (text) file
• applying the described modifications using Puppet

Files containing Puppet resource declarations are called manifests and usually have the .pp file extension.

We are going to write a manifest that describes a (text) file resource. The file is going to have the following properties:

• name and path: /tmp/my_file
• access rights: 0604
• file content: “File created using Puppet”

Resource declaration has the following syntax:

file {'my_file':
  path    => '/tmp/my_file',
  ensure  => present,
  mode    => '0640',
  content => "File created using Puppet.",
}

Save the previously described code in a manifest file called my_file_manif.pp

Applying a manifest is done with the command: puppet apply

[root@puppet ~]# puppet apply my_file_manif.pp

Notice: Compiled catalog for puppet in environment production in 0.18 seconds
Notice: /Stage[main]/Main/File[my_file]/ensure: defined content as '{md5}b4fdf30d694de5a5d7fe7a50cda27851'
Notice: Finished catalog run in 0.38 seconds

Check that the file has been created and the content and access rights are correct.

Try to apply the same manifest one more time:

[root@puppet ~]# puppet apply my_file_manif.pp

Notice: Compiled catalog for puppet in environment production in 0.16 seconds
Notice: Finished catalog run in 0.38 seconds

Notice that if the resource is already in the state described by the manifest, Puppet does not execute any action.

Change the access rights of the file to 755 and then apply the manifest again.

[root@puppet ~]# chmod 755 /tmp/my_file 
[root@puppet ~]# puppet apply my_file_manif.pp

Notice: Compiled catalog for puppet in environment production in 0.18 seconds
Notice: /Stage[main]/Main/File[my_file]/mode: mode changed '0755' to '0640'
Notice: Finished catalog run in 0.38 seconds

Change the content of the file and then apply the manifest again.

[root@puppet ~]# echo "This is not my file" > /tmp/my_file 
[root@puppet ~]# puppet apply my_file_manif.pp

Notice: Compiled catalog for puppet in environment production in 0.18 seconds
Notice: /Stage[main]/Main/File[my_file]/content: content changed '{md5}7225302b0d15d4a2562c2ab55e45d4cc' to '{md5}b4fdf30d694de5a5d7fe7a50cda27851'
Notice: Finished catalog run in 0.41 seconds

Notice that if the attributes of the resource are different from the ones described in the manifest, applying the manifest brings the resource back to the desired state.

The ensure attribute usually specifies if the resource:

• must exist (ensure ⇒ present)
• must NOT exist (ensure ⇒ absent)

Some types of resources define additional states for this attribute. File resources can have, in addition, the following values for ensure:

• directory
• link
• file

Define a manifest that creates a symbolic link to the /tmp/my_file file.

In a manifest, define a resource with the type ssh_authorized_key.

The resource must allow the user student from the physical machine to authenticate as the student user on the VM, without a password.

A Puppet manifest can contain declarations for multiple resources, but the order in which they are applied is not strictly enforced.

There are cases in which we have to make sure that a resource is applied before another (as an example, a package is installed before starting the service).

In these situations, we have to define resource dependencies.

We modify the previously created manifest:

file {'my_file':
  path    => '/tmp/my_file',
  ensure  => present,
  mode    => '0640',
  content => "File created using Puppet.",
}

notify {'my_notify':
  message => "File /tmp/my_file has been synced",
  require => File['my_file'],
}

• The notify resource defines a message that will be shown when its declaration is evaluated
• The dependency between resources is defined with the require attribute. In the previous example, the my_file resource is evaluated before the my_notify resource.

Modify the /tmp/my_file file and then apply the manifest described above. Notice the order in which the resources are evaluated.

An equivalent syntax would be to use the before attribute in the my_file resource:

file {'my_file':
  path    => '/tmp/my_file',
  ensure  => present,
  mode    => '0640',
  content => "File created using Puppet.",
  before  => Notify['my_notify'],
}

notify {'my_notify':
  message => "File /tmp/my_file has been synced",
}

For some resources we need a “refresh” action (as an example, a service that has to be restarted).

If in addition to resource dependency, we want to “refresh” the second resource when the first one is changed, we must:

• use notify instead of before, or
• use subscribe instead of require

An example would be restarting the SSH service when its configuration file has been changed:

file { '/etc/ssh/sshd_config':
  ensure => file,
  mode   => '0600',
  source => '/root/examples/sshd_config',
}
service { 'sshd':
  ensure    => running,
  enable    => true,
  subscribe => File['/etc/ssh/sshd_config'],
}

Create a Puppet manifest with the previous code, then modify the /etc/ssh/sshd_config file and apply the manifest.

Instead of before / require or notify / subscribe, we can use the operators: ”->” or ”~>”

Example:

file {'my_file':
  path    => '/tmp/my_file',
  ensure  => present,
  mode    => '0640',
  content => "File created using Puppet.",
}
->
notify {'my_notify':
  message => "File /tmp/my_file has been synced",
}

In many situations, Puppet is used to make sure that a certain system service is installed, started and with the appropriate configuration.

The above use case can be implemented with 3 resources:

• package
• file
• service

Between the first 2 we have a “before / require” relation, and between the last 2 there is a “notify / subscribe”.

Create the following manifest which implements this design pattern for the SSH service, and then apply the manifest:

package { 'openssh-server':
  ensure => present,
}
->
file { '/etc/ssh/sshd_config':
  ensure => file,
  mode   => '600',
  source => '/root/config-files/sshd_config',
}
~>
service { 'sshd':
  ensure     => running,
  enable     => true, 
}

Modify various states of the “package / file / service” triplet and reapply the manifest. Examples:

• uninstall/purge the packet
• change the configuration file
• stop the service

Create a “package / file / service” manifest for the Apache service.

In order to define a variable in Puppet, we use the syntax $variable, both for assignment and referencing.

We change the manifest for the my_file file, defining the contents of the file as a variable.

$my_content = "File created using Puppet."

file {'my_file':
  path    => '/tmp/my_file',
  ensure  => present,
  mode    => '0640',
  content => $my_content,
}

In addition to user-defined variables, Puppet defines some system variables. These are called facts. In order to see all of these variables, we use the command facter.

[root@puppet ~]# facter 
disks => {
  fd0 => {
    size => "4.00 KiB",
    size_bytes => 4096
  },
  sda => {
    model => "QEMU HARDDISK",
    size => "8.00 GiB",
    size_bytes => 8589934592,
    vendor => "ATA"
  },
  sr0 => {
    model => "QEMU DVD-ROM",
    size => "1.00 GiB",
    size_bytes => 1073741312,
    vendor => "QEMU"
  }
}
dmi => {
  bios => {
    release_date => "04/01/2014",
    vendor => "SeaBIOS",
    version => "1.10.2-1ubuntu1"
  },
...

An example of using system variables is when taking decisions based on the value of some of them.

The following manifest ensures that the NTP service:

• is started if the system is a physical machine
• is stopped if the system is a virtual machine

The decision is taken based on the value of the $is_virtual system variable.

if str2bool("$is_virtual") {
  service {'ntp':
    ensure => stopped,
    enable => false,
  }
}
else {
  service { 'ntp':
    name       => 'ntp',
    ensure     => running,
    enable     => true,
    hasrestart => true,
    require => Package['ntp'],
  }
}

Apply the manifest and notice the state of the NTP service.

First, uninstall the NTP server from the virtual machine.

Then, write a manifest that:

• installs the NTP server packet
• ensures that the NTP server is started (the service name depends on the Linux distro)

Use the case conditional statement.

Ansible is a configuration management and provisioning tool, similar to Puppet. It uses SSH to connect to servers and run the configured Tasks.

As opposed to Puppet, where each host manages its own data (services, users, files, etc.), and can optionally connect to a remote host to retrieve manifest files for configuration, Ansible is used to push the configuration from a central system to other hosts. An advantage of Ansible is that it does not require a specific service daemon to be installed before being able to configure the hosts. Operation is achieved through Python scripts for remote Linux hosts, or Powershell scripts for remote Windows hosts.

On the SCGC VM we are going to install and configure Ansible.

student@scgc:~$ sudo apt update
student@scgc:~$ sudo apt install -y ansible
# Required to use password authentication. By default, ansible requires authentication through SSH keys
student@scgc:~$ sudo apt install -y ansible

Check that the package was successfully installed by running the command:

student@scgc:~/scgc$ ansible --version
ansible 2.5.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/student/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.17 (default, Nov  7 2019, 10:07:09) [GCC 7.4.0]

Ansible has a default inventory file used to define which servers it will be managing. After installation, there's an example one you can reference at /etc/ansible/hosts (initially fully commented out).

We are going to add 2 labels to the hosts file. One containing the local address, named thishost and another one of the previously used VM, named remote.

student@scgc:~$ cat /etc/ansible/hosts | grep -A 5 thishost
--
--
[thishost]
127.0.0.1

[remote]
10.0.0.2
--
--

Let's start running Tasks against a server.

Running against localhost:

student@scgc:~$ ansible thishost --connection=local -m ping
127.0.0.1 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

Running against the remote VM:

student@scgc:~$ ansible --ask-pass --user=student remote -m ping
SSH password: 
10.0.0.2 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

In either case, we can see the output we get from Ansible is some JSON which tells us if the Task (our call to the ping module) made any changes and the result.

Let's cover these commands:

• thishost, remote - Use the servers defined under this label in the hosts inventory file
• --connection=local - Run commands on the local server, not over SSH
• -m ping - Use the “ping” module, which checks if the host can be accessed. Using ping with --connection=local does not make sense, as the option is used when running commands on the host that is issuing commands. It normally attempts to connect to the host via SSH.
• --ask-pass --user=student - SSH connection parameters: interactive password input, login as student user