Here is how we have used goss and Ansible templates to run system tests after building syslog-ng servers

Intro

We are using Ansible to deploy Splunk environment(s) at one of our customers’ site. We were looking for a  way to run system tests after the deployment of the splunk environment is complete and stumbled upon goss.  goss is a self contained binary that allows you to run different tests’ type, like

• is a service running
• is a port listening
• does the file/directory exists
• running any shell command and compare against an expected output
• a lot more…

Folder structure

ansible_goss/
├── README.md
├── ansible.cfg
├── files
├── goss_deploy.yml
├── goss_run_test.yml
├── group_vars
│   └── all
│       └── vars.yml
├── inventory
│   ├── dev
│   ├── hosts
│   ├── prd
│   └── sit
├── roles
│   ├── hf
│   │   └── templates
│   │       └── goss_test_00.yml
│   ├── syslog
│   │   └── templates
│   │       ├── goss_test_00.j2
│   │       ├── goss_test_01.j2
│   │       └── goss_test_02.j2
│   └── uf
│       └── templates
│           └── goss_test_00.j2
└── test.yml

Splitting and ordering tests

I had to split goss tests into multiple files since goss doesn’t naturally support the ability to order tests, but rather executes them sequentially in the alphabetical order of the test type, .i.e addr, dns, command, file, etc..

In my case here is how I’ve broken down the tests

• goss_test_00
  • general service, port and file tests
• goss_test_01
  • executing a command (to send test syslog packets locally)
• goss_test_02.
  • file tests that verifies the files were created as a result of the command that were run as part goss_test_01

Then the playbook that runs the test will search for all the goss_test files that are relevant to the current role and will run them in a “numbered” order

One note is that I had to add an empty files folder in the “root” as Ansible will try to search relative to all the files folders inside it’s structure

---

- name: run goss test on remote
  hosts: "{{ splunk_group }}"
  roles:
    - syslog
    - hf
    - uf
  gather_facts: no
  tasks:
    - name: save goss files list
      set_fact:
        goss_files : "{{ lookup('fileglob', '../roles/{{splunk_group}}/templates/*.j2', wantlist=True) | sort }}"
    - name: print
      debug:
        msg: 
          - "{{ goss_files }}"

    - name: Copy goss test from template to remote
      template:
        src: "{{ item }}"
        dest: /tmp/{{ item | basename | regex_replace('\.j2','.yml') }}
        mode: '0644'
      with_items:
        - "{{ goss_files }}"

    - name: Run goss
      command: /usr/local/bin/goss --gossfile /tmp/"{{ item | basename | regex_replace('\.j2','.yml') }}" validate --format tap
      with_items:
        - "{{ goss_files }}"
      ignore_errors: yes
      become: yes
      register: goss_result

    - name: Print Goss Results
      debug:
        msg: "{{ item.stdout_lines }}" 
      with_items: "{{ goss_result.results }}"

goss, Ansible and Templates

I have also used the “power” of Ansible variables and Jinja2 (j2) templates to populate the the goss_test.yaml files.

For example I have a variables file that contains all the ports’ information

port:
  dns:
    tcp: 7001
    udp: 7101
  dual:
    tcp: 8001
    udp: 8101
  relay:
    tcp: 9001
    udp: 9101

As you see, there are 6 ports here (for now) and it will be tedious to each time populate the goss_test*.yaml files “manually” not to mentioned that the ports information might change, so instead I i’ve decided to “combine” goss ansible and templates using jinja loops.  This snippet in j2 file

port:
{% for feed in port %}
{% for protocol, portnum in port[feed].iteritems() %}
  {{ protocol }}:{{ portnum }}:
    listening: true
{% endfor %}
{% endfor %}

will be transformed into this yaml

port:
  udp:7101:
    listening: true
  tcp:7001:
    listening: true
  udp:8101:
    listening: true
  tcp:8001:
    listening: true
  udp:9101:
    listening: true
  tcp:9001:
    listening: true

 

Please excuse me if the Ansible is not written in “proper” way, but it did work for me.

You can find the full solution here https://github.com/ilyaresh/ansible_goss