TrustedSec - Automation Testing With Ansible, Molecule, and Vagrant

There is an old rule that if you find yourself doing anything more than twice, you should automate it. For developers, this may be software builds or the environments into which they will be deployed; for penetration testers, it may be the need to create a phishing host or a lab environment for testing. In any case, the goal is reliability and repeatability without destroying existing environments. My preferred tools for creating environments are Ansible, Molecule, and Vagrant. In this article, I will demonstrate my basic setup and how to get started testing your automation.

1. Prerequisites

• Python 3
• Python Virtual Environment
  • https://docs.python-guide.org/dev/virtualenvs/#lower-level-virtualenv
• Vagrant
  • https://www.vagrantup.com/downloads.html

2. Set Up Virtual Environment

Run the following commands:

$ cd /home/username/new-project
$ virtualenv new-project

3. Installation

3.1 Setting up Molecule

Run the following commands:

$ cd /home/username/new-project
$ source bin/activate
$ pip install ansible molecule python-vagrant
$ molecule init role -r project-role --driver-name vagrant
$ rm -rf project-role/meta

What is happening:

We change the directory to the ‘new-project’ folder. We set the source to bin/activate, which means that anything you run in this shell until running ‘deactivate’ will take place in the virtual environment. We use pip to install ansible, molecule, and python-vagrant. Pip handles all dependencies. Running the command molecule init will create the skeleton of the project with all of the subdirectories in place and a basic configuration file. Finally, we remove the meta directory because the default is set up to publish to Galaxy, which is not what we want to do with this project. If you are already familiar with Ansible and have an existing role called project-role, you can init over it with:

$ cd existing_role_directory
$ molecule init scenario --role-name existing_role_directory --driver-name vagrant

Once everything is installed ,we need to edit the project-role/molecule/default/molecule.yml file. Here, we will concentrate on the driver and platform. The driver controls what platform with which Molecule will test. By default, it will use Docker containers.

Note: There are additional drivers for all major cloud services, if you require them, including AWS, GCE, Azure, and Linode. The full list of drivers can be found at https://ift.tt/2LjvnnJ.

In this case, we will use Vagrant and we will also change the platform to spin up an Ubuntu test server. Modify the molecule.yml file so it looks like the following:

Modify project-role/molecule/default/molecule.yml

dependency:
  name: galaxy
driver:
  name: vagrant
  provider:
    name: virtualbox
lint:
  name: yamllint
platforms:
  - name: new-project-server
    box: ubuntu/bionic64
    instance_raw_config_args:
      - "vm.network 'forwarded_port', guest: 80, host: 8088"
provisioner:
  name: ansible
  lint:
    name: ansible-lint
verifier:
  name: testinfra
  lint:
    name: flake8

We forward 8088/TCP to 80/TCP because we are going to install nginx later. It is unlikely that 8088 is used on your machine, but it can be modified if needed. Nginx, by default, will run on port 80 inside the Vagrant machine. To ensure the configuration is set up correctly, run the following command:

$ molecule check

3.2 Ansible

We will start our Ansible configuration by ensuring that the server we build is running nginx. Because this is Ubuntu, we will use the apt module. Edit the project-role/tasks/main.yml file to look like the following:

Modify project-role/tasks/main.yml

# tasks file for project-role

- name: Install nginx
  apt:
    name: nginx
    state: present

Now, we want to lint the file and make sure the Ansible configurations pass. The linter will not only check for syntax errors but will also recommend modules and best practices.

$ molecule lint

If all passes, it will return ‘Lint completed successfully.’ Then we run a full test with the following command:

$ molecule test

This command does the following steps:

• Downloads the Ubuntu image
• Spins up a new virtual machine (VM) with that image
• Sets up networking
• Adds a Vagrant user
• Installs python on the VM for Ansible
• Runs the Ansible role
• Reruns the Ansible role
• Destroys the VM

In this case, the first time it runs the role, the test fails with something like the following:

fatal: [instance]: FAILED! => {“changed”: false, “cmd”: “apt-get install nginx”, “msg”: “E: Could not open lock file /var/lib/apt/lists/lock – open (13: Permission denied)…

The linter cannot catch everything. Here, we forgot that apt requires root. So, we add become and become_method. There are other, more efficient ways to do this on larger projects, but this will work for now.

Edit project-role/tasks/main.yml

# tasks file for project-role

- name: Install nginx
  apt:
    name: nginx
    state: present
  become: true
  become_method: sudo

Run the molecule test command again and confirm that it passes this time. Some will notice that it runs the role twice. The reason is idempotence, which just means that the same actions can be applied multiple times without changing the result beyond the first run. This test ensures that when it runs the role the second time, no changes are made. If you run it twice in a row and something changes on the second time, you need to find another way to do it. Idempotency problems are most common when using the ‘command’ or ‘shell’ modules. In those cases, checks to see if those steps need to be run should be added. We will address ways to do that in a follow-up post. Most other modules are idempotent by design. If an apt update is not needed, for example, it will just return ‘ok’. This will get you 95% to a finished role.

4. Bringing it all Together

Finally, you need to test the finished product. Run the molecule converge command to spin up the VM and run the role against it. Because it is not a full test, the VM will stay running for testing. Once running, open your browser and navigate to http://localhost:8088/. Remember that in your configuration, we set port 8088 to forward to port 80. You should see the default nginx page. This is a simple install, but in more complicated roles, it is common to find something that does not work without an obvious cause. Running the command molecule login will log you into the VM and allow you to troubleshoot.

There is much more that can be done with Ansible and Vagrant, including spinning up multiple systems to test their ability to work together. Running playbooks that start up front ends, databases, and APIs to do full end-to-end tests can be deployed in minutes, helping to ensure that you are not delayed when you actually need them. Vagrant can be configured to use any number of distributions, allowing you to test on, for example, both Ubuntu and Red Hat with a single change. Ansible can detect the OS and change its behavior on the fly. We will delve into Ansible variables and build a more functional host in a later post, but this should get you started on your journey.

The post Automation Testing With Ansible, Molecule, and Vagrant appeared first on TrustedSec.

from TrustedSec https://www.trustedsec.com/blog/automation-testing-with-ansible-molecule-and-vagrant/