Permanent Links

Poll

What should be the topic for the next Impossibly Stupid poll?

A Town Square Poll Space

Tech Corner

See Also

[ICO]NameLast modifiedSizeDescription

[PARENTDIR]Parent Directory  -  
[TXT]README.html2018-02-17 13:23 11K 
[   ]info.json2018-02-11 12:16 35  
[   ]tags=auto2015-03-26 18:30 0  
[   ]tags=meta2015-03-26 18:30 0  

Extra automation employing Ansible

When I wrote about configuring Exim, the Ruby scripts I gave were just the tip of the automation iceberg. It’s only after you’ve taken those first baby steps that you start to realize how many other parts of the process there are that still require human intervention. Then the questions come in a rush:

Who knows which physical servers are supposed to be handling email? Are we sure they’re running Exim? What about Ruby? Do they already have these custom scripts on them? What about the zone files that feed the scripts? How does a person access each machine to run the scripts in the first place?

It’s all those sorts of issues that either have you stopping cold at basic scripting, or feverishly looking for some kind of tool that does it all. At Impossibly Stupid, I advocate a more reasonable approach, and so today I’m going to suggest you consider using Ansible for your next steps in automation. Why?

It’s declarative

I think a good way to start doing operations automation is by simply answering that list of questions with the basic facts that describe what you have and/or what you want, and that’s what Ansible encourages. There are a lot of configuration management choices, and many of them emphasize writing code as the first step. I think that’s a poor way to handle automation because it represents an unnecessary barrier to entry. To get the most out of automation, you want to be able to present it as an option that anyone at your organization can take advantage of.

It scales well

A lot of those other configuration management options also have extremely steep learning curves. Yes, the ability to do some amazingly large orchestration for a global enterprise is unquestionably very valuable, but anyone who is just starting to introduce automation needs to be able to show its value on smaller scales first. Ansible nicely allows you to not only do that big stuff eventually, but it also scales all the way down to running simple scripts or even basic commands ad-hoc on your various machines.

It’s language agnostic

Because Ansible doesn’t rush you into coding, it doesn’t force you in to a particular coding language, either. It’s written in Python, but I mainly write my scripts in Bash or Ruby. I’ve also used some of those configuration management tools that are Ruby-centric, too, and the architecture of Ansible really allows you to see how silly some of the language requirements are for the other solutions. Especially those that make the further unapproachable choice of demanding their code be written in a custom domain specific language!

But enough of the talky-talk about what I like about Ansible, let me show it in action. We again look at the example of configuring an Exim server based on our zone files. What we can start by doing is just sitting down and writing out the facts we know about our email setup. The file format Ansible uses for that information follows the YAML standard, which is something that even non-technical people should be able to understand. For what we want to do, let’s call the file mx.yml, it might look something like this:

hosts:
  - smtp1.example.net
  - smtp2.example.net
packages:
  - exim4
exim4:
  config:
    domains: /etc/exim4/domains.virtual
    catchall: /etc/exim4/catchall.virtual
zones:
  host: prepbox.local
  user: zonemaster
  path: /var/named/*.db
  extractor: /home/zonemaster/bin/exim_virtual_domains.rb

Translated to English, it basically starts by saying We have these two machines that are our mail servers, so they should be running Exim, and the configuration files for it that we want to modify are at these two locations. Then you really have to start thinking about things when it comes to those zone files!

The largest issue that needs to be resolved when implementing widespread automation is figuring out how all the various pieces of the puzzle fit together. You have a network of different machines with different capabilities, some of which can connect to others and some of which are unreachable, and you have to figure out a way to coordinate them all to get a task done. With just one or two machines to work with, the solutions are pretty straightforward.

If, for example, I kept my zone information on either my current machine (the one running Ansible) or on the mail servers, it would just be a case of running the script to get the output and you’re done. But when it’s all off on another machine (like in a Git repository or on a separate DNS host), you actually have to think about how everything is best going to work together.

Once you’ve put in that very-necessarily-human effort, you can write down the details of what you decided makes the most sense for your current setup. For me, the English translation of the above zones: entry is We will be getting our MX information from our staging server, which has the script that the zonemaster account can run.

Now, keep in mind, that none of these labels/names for things are formalized in any way. The format is YAML, but how you use what you’ve written down is up to you. As you do more and more automation, you’re naturally going to want to start using some common conventions for the things that you do a lot, but there’s no need to worry about that just yet.

Instead, we can get right into coding an Ansible-compatible script (known as a Playbook) based on the information we just wrote down. This also takes the form of a YAML file (but this time with very specific labels for everything), and for my example of doing the Exim configuration, let’s call it zone2exim.yml, it looks like this:

- hosts: '{{ hosts }}'

  tasks:
  - name: run the MX extraction script and capture the output
    shell: '{{ zones.extractor }} {{ zones.path }} | sort -u'
    delegate_to: '{{ zones.host }}'
    remote_user: '{{ zones.user }}'
    register: extraction
  - name: push the extraction output to the server
    copy:
      content: '{{ extraction.stdout }}'
      dest: '{{ exim4.config.domains }}'
    register: reconfig
    become: yes
    notify: restart MX
  - name: run the subdomain catch-all updater
    shell: 'grep -v "^\w*\.\w*$" {{ exim4.config.domains }} | sed -E s/"(.*\.)(\w*)(\..*)"/"\@\1\2\3 : \2"/g > {{ exim4.config.catchall }}'
    when: reconfig | changed
    become: yes
    notify: restart MX

  handlers:
  - name: update exim configuration
    command: update-exim4.conf
    become: yes
    listen: restart MX
  - name: restart exim
    service: 
      name: exim4
      state: restarted
    become: yes
    listen: restart MX

Hopefully that’s somewhat readable even if you don’t have any experience writing Ansible playbooks. Regardless, let me break down the little bits and translate them to English for you.

- hosts: '{{ hosts }}'

This is the list of hosts we want to configure. You’re going to see these {{ something.here }} snippets a lot. All they are is substitutions (i.e., variables) so that you can use the information from the first file (in this case, the SMTP hosts) in this second file.

  tasks:

Then we establish the set of tasks that will accomplish our desired outcome. These tasks are normally going to be performed on each machine given in hosts:, but we already know we have to look elsewhere for the zone details.

  - name: run the MX extraction script and capture the output
    shell: '{{ zones.extractor }} {{ zones.path }} | sort -u'
    delegate_to: '{{ zones.host }}'
    remote_user: '{{ zones.user }}'
    register: extraction

The first task is what creates the list of MX servers. It uses the zones: information we gave earlier to connect from the local Ansible machine to the remote zone machine (so not involving the mail server at all) and run the script there, saving the results into the extraction variable, which will be available on the local machine.

  - name: push the extraction output to the server
    copy:
      content: '{{ extraction.stdout }}'
      dest: '{{ exim4.config.domains }}'
    register: reconfig
    become: yes
    notify: restart MX

The next task takes that extraction output and writes it to the Exim config file we gave in xm.yml. Since this is a privileged file, we need to become root to do that. Note that Ansible will only modify the file if there actually are changes to the configuration. If that’s the case, though, we then want Exim to restart so that it is using the new set of domains.

  - name: run the subdomain catch-all updater
    shell: 'grep -v "^\w*\.\w*$" {{ exim4.config.domains }} | sed -E s/"(.*\.)(\w*)(\..*)"/"\@\1\2\3 : \2-catchall"/g > {{ exim4.config.catchall }}'
    when: reconfig | changed
    become: yes

And the last task takes care of updating the list of catch-all addresses. It also only runs if there was an actual change in the configuration.

  handlers:
  - name: update exim configuration
    command: update-exim4.conf
    become: yes
    listen: restart MX
  - name: restart exim
    service: 
      name: exim4
      state: restarted
    become: yes
    listen: restart MX

And these last two entries handle the steps needed to get Exim to use the new configuration, running them after all the main tasks are done. It’s definitely a lot more verbose than the direct update-exim4.conf && service exim4 restart command I gave in my last post, but it’s that way because it spells out all the implicit assumptions that humans would otherwise take for granted.

So now you’ve set out what needs to be automated, but you still need to run it somehow! That’s what Ansible does, of course, with a command that might look something like this:

ansible-playbook -e '@mx.yml' zone2exim.yml

Keep in mind that this is all about how you might start automating your infrastructure beyond basic scripting. Ansible has many more complex conventions that you’ll want to use if you start scaling up. Stay tuned for future posts discussing more advanced topics.