Path: TUTORIAL.txt
Last Update: Fri Mar 20 04:47:12 -0700 2009

A hands-on tutorial for learning AutomateIt

AutomateIt is an open source tool for automating the setup and maintenance of servers, applications and their dependencies.

This hands-on guide will teach you to use AutomateIt and explain where to find more detailed instructions.

It‘s recommended that you see the Screenshots of AutomateIt in action to get a quick idea of what AutomateIt is all about.

AutomateIt is feature-complete, exceeds the capabilities of similar products, and ensures its quality with a self-test suite. However, this is a young product and users are expected to be technically proficient, willing to accept rough spots, to work through problems and upgrade frequently.

Please sign up for RSS change notifications so you know when upgrades are available.


AutomateIt is written using the Ruby programming language. If you haven‘t used Ruby, you‘ll find it easy to learn and a pleasure to use. If you already know the basics of Perl, Python, PHP or Java, you‘ll be able to pick up Ruby almost instantly. Although AutomateIt provides much of the structure and commands needed, you still need to know the basic Ruby syntax to get by.

Some Ruby resources:

Typographical conventions

AutomateIt‘s technical documentation uses the following typographical conventions:

  • automateit — A file, command or variable.
  • :verbosity — A symbol, usually used in an options hash.
  • AutomateIt::ShellManager — A class, with a link to its documentation. You can also find a link to this class‘s documentation in the "Classes" pane on the left.
  • AutomateIt::ShellManager#sh — A method, with a link to its documentation. You can also find a link to this method‘s documentation in the "Methods" pane on the left.
  • Ruby code:
 puts "Ruby code"
  • Unix shell command, although the prompt may be left off when obvious:
 you@host:myproject> echo "Unix command run from the 'myproject' directory"
  • AutomateIt interactive shell command, although the prompt may be left off when obvious:
 ai> puts "I'm in the Interpreter"


AutomateIt uses a number of unique terms:

  • Recipe — A file that contains AutomateIt commands.
  • Project — A special directory that contains related recipes and helper files.
  • Interpreter — The part of AutomateIt that runs commands.
  • Plugin — A part of AutomateIt that describes related features.
  • Driver — An implementation of a plugin. There are often multiple drivers per plugin.

Interactive shell

AutomateIt comes with an interactive shell, which is useful for exploring commands and developing recipes.

Here‘s what an interactive shell session looks like (text following the # symbol explains the line and is not actually part of the session):

 you@host:tmp> automateit      # Start the AutomateIt interactive shell
 => AutomateIt Shell v0.5.0    # Welcome message from AutomateIt
 ai> puts "Hello world!"       # AutomateIt's prompt and a user command
 Hello world!                  # Output from the "puts" command
 => nil                        # Return value of the "puts" command
 ai> self.class                # Prompt and another user command
 => AutomateIt::Interpreter    # Return value of "self.class"
 ai> <CTRL-D>                  # Press <CTRL-D> to exit the shell
 you@host:tmp>                 # We're back to the Unix shell

Recipe files

The Interpreter can run recipe files that contain AutomateIt commands.

For example, create a /tmp/hello.rb file that contains:

 puts "Hello, I'm an #{self.class}"

Then run the recipe:

 you@host:tmp> automateit /tmp/hello.rb
 Hello, I'm an AutomateIt::Interpreter

String evaluation

The Interpreter can also evaluate commands as strings:

 you@host:tmp> automateit -e 'puts self.class'

Exploring the Interpreter‘s unique methods

The Interpreter provides some unique methods:

 ai> unique_methods
 => ["account_manager", "address_manager", "cd", "chmod", ...

The names are Interpreter methods. Names ending with _manager are methods for accessing plugins, while the rest are normal methods. The AutomateIt::Interpreter documentation provides a list of these methods and links to their individual documentation.

For example, one of the commands listed is pwd, which can be used like this:

 ai> pwd
 => "/tmp"

AutomateIt provides many commands that work just like the Unix shell commands you already know, so you‘ll be productive quickly.

Conditional execution

AutomateIt executes only the commands needed to achieve a desired state, which makes recipes repeatable, reusable and maintainable.

Eliminating the distinction between setup and maintenance means the same command can perform both tasks without concern for the host‘s condition. A properly-written recipe can be run and re-run immediately afterwards, and it will do nothing the second time because all changes have already been applied.

An example can make this clearer — again the text following the # symbol explains the commands and is not part of the session:

 you@host:tmp> automateit   # Start the AutomateIt interactive shell
 => AutomateIt Shell v0.5.0 # Welcome message
 ai> mkdir "asdfasdf"       # Create a directory called "asdfasdf"
 ** mkdir asdfasdf          # Message showing that directory was created
 => ["asdfasdf"]            # Return value with array of directories created
 ai> mkdir "asdfasdf"       # Try to create the same directory again
 => false                   # Nothing happened, the directory already exists
 ai> rmdir "asdfasdf"       # Remove the directory
 ** rmdir asdfasdf          # Message showing that directory was removed
 => ["asdfasdf"]            # Return value with array of directories removed
 ai> rmdir "asdfasdf"       # Try to remove the directory again
 => false                   # Nothing happened, there's no directory to remove

Notice how the second time mkdir was run above, it returned false and didn‘t create a directory? That‘s the conditional execution in action, realizing the action has already been performed. Similarly, the rmdir only ran the first time, but returned false and took no action when the directory was already gone. You can use the return values of these commands to write sophisticated logic of your own based on what actions took place.


AutomateIt uses an extensible plugin architecture to group together related commands:

Plugins can be accessed from the Interpreter like this:

 ai> shell_manager.pwd
 => "/tmp"

The most common plugin methods have aliased shortcuts. For example, pwd is the alias for shell_manager.pwd. These are documented in AutomateIt::Interpreter‘s "Aliased methods."


Each plugin has one or more drivers that implement its functionality.

For example:

  • AutomateIt::ShellManager — A plugin for running shell commands.
  • AutomateIt::ShellManager::Portable — A portable but limited-functionality driver that implements the ShellManager‘s methods.
  • AutomateIt::ShellManager::Unix — A full-featured driver implementing ShellManager‘s methods that only runs on Unix-like systems.

Plugins provide APIs, drivers provide implementations

Plugins describe consistent interfaces for related features, and drivers implement this API for different tools.

For example, AutomateIt provides a common API for all packaging tools: AutomateIt::PackageManager. It provides drivers for packaging tools like APT, YUM, Gem, Egg and others. To install a package called foo with the apt driver:

 ai> package_manager.install "foo", :with => :apt

AutomateIt will check if foo is installed, install it if needed, or do nothing if the package is present. This API is the same for all packaging tools, making it easy to get work done by using high-level AutomateIt commands instead of cryptic tool-specific commands. Although AutomateIt uses the low-level tools, it uses them with best-practices approaches and hides the senseless complexity from the user.

What‘s the big deal? Consider how one would install packages from the Unix shell. Most packaging tools are pathologically dysfunctional and make it bafflingly difficult to programmatically install or uninstall packages, or tell if a package is installed. Many don‘t use exit values and require complex output parsing. Others require user-input even when it‘s obvious that none is needed. Almost all make it necessary to write conditional code because they‘ll either fail with errors if told to install an existing package, destroy an existing setup, or install duplicate packages. Writing Unix shell code to handle all these quirks is frustrating and risky.

Here‘s a sample Unix shell command for installing a package using one of the simplest, most reasonable tools available — although note that unlike AutomateIt, this shell command can‘t handle multiple packages, is slower, can‘t be previewed, and has no consistent error handling:

 if dpkg-query -W --showformat '${Status}\n' foo 2>&1 | \
     egrep -q '(^| )installed'; then
   apt-get install -y -q some_package_name < /dev/null

Now compare that hideous command to the simple, clear and consistent AutomateIt command before it. AutomateIt‘s plugins and drivers are easy to install and write. As more are written, more people will hopefully be freed from needlessly convoluted low-level commands, and able to get simple things done simply. AutomateIt‘s consistent API, multiple drivers, sane defaults and conditional-checking make it easier to write clear and maintainable recipes than using the low-level directly commands from the Unix shell.

Driver auto-detection

AutomateIt can automatically detect the most suitable driver for each plugin command.

For example, the AutomateIt::PackageManager plugin has drivers called AutomateIt::PackageManager::APT and AutomateIt::PackageManager::YUM. On a Debian system, which uses the apt-get packaging tool, AutomateIt will default to using the AutomateIt::PackageManager::APT driver:

 ai> package_manager.installed? "apache2"
 => true

Using a specific driver

Sometimes it‘s necessary to specify the driver to use. The recommended way to do this is to pass a :with => :driver_name option to the plugin command.

For example, tell the package manager to use the Gem driver:

 ai> package_manager.installed? "automateit", :with => :gem
 => true

You can also completely bypass the plugin and its auto-detection to directly interact with the driver:

 ai> package_manager[:gem].installed? "automateit"
 => true


A project is a special directory that contains related recipes and helper files. Although it‘s possible to run recipe files without a project, a project provides many useful features, described in the AutomateIt::Project documentation.

A project is created by specifying the directory to create:

 you@host:tmp> cd /tmp
 you@host:tmp> automateit --create myproject
 ** mkdir -p myproject
 => Creating AutomateIt project at: myproject
 ** cd myproject
 ** mkdir config
 ** cd config
 => Rendering 'tags.yml' because of it doesn't exist
 => Rendering 'fields.yml' because of it doesn't exist
 => Rendering 'automateit_env.rb' because of it doesn't exist
 ** cd -
 ** mkdir dist
 ** cd dist
 => Rendering 'README.txt' because of it doesn't exist
 ** cd -
 ** mkdir lib
 ** cd lib
 => Rendering 'README.txt' because of it doesn't exist
 ** cd -
 ** mkdir recipes
 ** cd recipes
 => Rendering 'README.txt' because of it doesn't exist
 ** cd -
 => DONE!

This creates a /tmp/myproject directory with the newly-created project. It contains directories, each with a README.txt file explaining the directory‘s purpose, and individual files like tags.yml that contain comments with basic usage instructions.

The project creator is an AutomateIt recipe, so it‘s smart enough to only execute the commands needed. If the project creator is re-run against an existing project, it won‘t make any changes because none are needed:

 you@host:tmp> automateit --create myproject
 => Found AutomateIt project at: myproject
 => DONE!

Project recipes

To associate a recipe with a project, put it into the project‘s recipes directory.

For example, create a recipes/hello_project.rb file with these contents:

 puts "Hello, this is: " + project

And run it:

 you@host:myproject> automateit recipes/hello_project.rb
 Hello, this is: /tmp/myproject

The Interpreter automatically loads the project. The project method contains the project path and is only available when executing a recipe associated with a project.


A project‘s config/fields.yml file is meant to store configuration constants, like custom paths for applications. Fields abstract configuration variables for recipes, improving maintainability by separating data and logic. Fields can also be easily queried from Unix using the aifield command. More information about fields and aifield can be found in AutomateIt::FieldManager.

For example, consider a fields file with the following contents:

 myuser: dhh
   path: /var/www/rails

These fields can be used from the interactive shell like this:

 you@host:myproject> pwd
 you@host:myproject> automateit -p .    # (1) Load project from current directory
 => AutomateIt Shell v0.5.0
 ai> lookup :myuser                     # (2) Lookup string value by symbol key
 => "dhh"
 ai> lookup "myuser"                    # (3) Lookup string value by string key
 => "dhh"
 ai> lookup "myapp"                     # (4) Lookup hash value by string key
 => {"path"=>"/var/www/rails"}
 ai> lookup "myapp#path"                # (5) Lookup string value by compound key
 => "/var/www/rails"

To load a project‘s fields, the AutomateIt interactive shell needs to know which project to load. On the line annotated (1), automateit -p . tells it to load the project from the "." directory, the current directory. The argument can be an absolute or relative path. The command accepts other arguments and environmental options, run automateit —help for details.

Fields can be queried by string (2) or symbol (3) keys. The lookup command can return any associated data, usually a string, but it can be a complex type, such as a hash (4). The compound-key (5) syntax is a convenient syntax for looking up nested keys in a hash. For example, lookup "myapp#path" is a shortcut for <tt>lookup("myapp")["path"] — these query the myapp hash and return the value of its path key.

The Interpreter loads a project and its fields automatically for recipes, so there is no need to specify the -p option. For example, a project recipe file recipes/hello_fields.rb contains:

 puts lookup("myapp#path")

This recipe will load its project and fields automatically:

 you@host:myproject> automateit recipes/hello_fields.rb


A project‘s config/tags.yml file describes tags assigned to hosts, grouping together hosts by their roles and attributes.

For example, this tags file describes a "desktops" tag with three hosts named "satori", "sunyata" and "michiru", and another tag named "notebooks" with other hosts:

    - satori
    - sunyata
    - michiru
    - rheya
    - avijja

The tags can be accessed like this when run on a host named "satori":

 you@satori:myproject> automateit -p .
 ai> tags
 => ["satori", "desktops", "localhost", ...]        # Tags for this host
 ai> tagged?("desktops")                            # Is this host tagged with "desktops"?
 => true
 ai> tagged?("notebooks")
 => false
 ai> tagged?(:satori)                               # Strings and symbols are treated the same.
 => true
 ai> tagged?("satori")
 => true
 ai> tagged?("satori || desktops")                  # Query by simple boolean expression
 => true
 ai> tagged?("(satori || desktops) && !notebooks")  # Query by complex boolean expression
 => true

Role-based behavior can be demonstrated by creating a file called recipes/hello_tags.rb:

 if tagged?("localhost")
  puts "I'm a localhost!"
 if tagged?("desktops")
  puts "Special commands to execute on desktops"
 elsif tagged?("notebooks")
  puts "Special commands to execute on notebooks"

Here are the results when executed on a host called "satori":

 you@satori:myproject> automateit recipes/hello_tags.rb
 I'm a localhost!
 Special commands to execute on desktops

Notice how the notebooks section wasn‘t executed? Tags make it possible to create sophisticated recipes that run commands on only the appropriate systems.

More documentation on tags can be found in AutomateIt::TagManager.

Previewing commands

AutomateIt lets you preview commands the recipe will run without actually letting them actually modify the system.

The Interpreter has a boolean that determines if it‘ll make changes to your system. This one variable can be called from two different ways:

  • writing? — Will it write changes?
  • noop? — Will it not write changes and just preview them? The noop means "no-operation"

Here‘s an example with the interactive shell, with comments for annotations:

 ai> noop true                # Enter noop mode
 ai> noop?                    # Currently in noop mode?
 => true                      # Yes, in noop mode
 ai> writing?                 # Writing to disk? The opposite of noop
 => false                     # No, not writing
 ai> mkdir "foo"              # Try to create a directory
 ** mkdir foo                 # Message showing directory will be creating
 => ["foo"]                   # Return value with directories to create
 ai> File.directory? "foo"    # Was the directory actually made?
 => false                     # No, noop mode prevented the change

Recipes can take advantage of the noop mode as well, consider a mkdir_example.rb recipe:

 mkdir "foo"

To run this recipe with noop mode, pass the -n option to the Interpreter shell:

 you@host:tmp> automateit -n mkdir_example.rb
 ** mkdir foo
 you@host:tmp> ls foo
 ls: foo: No such file or directory

Notice how the directory wasn‘t actually created? This is great because you can see exactly what commands AutomateIt will run without actually having it apply the changes.

WARNING: Previewing code can be dangerous. Read previews.txt for instructions on how to write code that can be safely previewed.


I hope you enjoy working with AutomateIt and look forward to hearing about your experiences with it. Drivers, patches, documentation and ideas are welcome.

Thank you for taking the time to read this!

  • Igal Koshevoy (igal@pragmaticraft.com)