| Path: | TUTORIAL.txt |
| Last Update: | Fri Mar 20 04:47:12 -0700 2009 |
AutomateIt is an open source tool for automating the setup and maintenance of servers, applications and their dependencies.
This hands-on guide will teach you to use AutomateIt and explain where to find more detailed instructions.
It‘s recommended that you see the Screenshots of AutomateIt in action to get a quick idea of what AutomateIt is all about.
AutomateIt is feature-complete, exceeds the capabilities of similar products, and ensures its quality with a self-test suite. However, this is a young product and users are expected to be technically proficient, willing to accept rough spots, to work through problems and upgrade frequently.
Please sign up for RSS change notifications so you know when upgrades are available.
AutomateIt is written using the Ruby programming language. If you haven‘t used Ruby, you‘ll find it easy to learn and a pleasure to use. If you already know the basics of Perl, Python, PHP or Java, you‘ll be able to pick up Ruby almost instantly. Although AutomateIt provides much of the structure and commands needed, you still need to know the basic Ruby syntax to get by.
Some Ruby resources:
AutomateIt‘s technical documentation uses the following typographical conventions:
puts "Ruby code"
you@host:myproject> echo "Unix command run from the 'myproject' directory"
ai> puts "I'm in the Interpreter"
AutomateIt uses a number of unique terms:
AutomateIt comes with an interactive shell, which is useful for exploring commands and developing recipes.
Here‘s what an interactive shell session looks like (text following the # symbol explains the line and is not actually part of the session):
you@host:tmp> automateit # Start the AutomateIt interactive shell => AutomateIt Shell v0.5.0 # Welcome message from AutomateIt ai> puts "Hello world!" # AutomateIt's prompt and a user command Hello world! # Output from the "puts" command => nil # Return value of the "puts" command ai> self.class # Prompt and another user command => AutomateIt::Interpreter # Return value of "self.class" ai> <CTRL-D> # Press <CTRL-D> to exit the shell you@host:tmp> # We're back to the Unix shell
The Interpreter can run recipe files that contain AutomateIt commands.
For example, create a /tmp/hello.rb file that contains:
puts "Hello, I'm an #{self.class}"
Then run the recipe:
you@host:tmp> automateit /tmp/hello.rb Hello, I'm an AutomateIt::Interpreter
The Interpreter can also evaluate commands as strings:
you@host:tmp> automateit -e 'puts self.class' AutomateIt::Interpreter
The Interpreter provides some unique methods:
ai> unique_methods => ["account_manager", "address_manager", "cd", "chmod", ...
The names are Interpreter methods. Names ending with _manager are methods for accessing plugins, while the rest are normal methods. The AutomateIt::Interpreter documentation provides a list of these methods and links to their individual documentation.
For example, one of the commands listed is pwd, which can be used like this:
ai> pwd => "/tmp"
AutomateIt provides many commands that work just like the Unix shell commands you already know, so you‘ll be productive quickly.
AutomateIt executes only the commands needed to achieve a desired state, which makes recipes repeatable, reusable and maintainable.
Eliminating the distinction between setup and maintenance means the same command can perform both tasks without concern for the host‘s condition. A properly-written recipe can be run and re-run immediately afterwards, and it will do nothing the second time because all changes have already been applied.
An example can make this clearer — again the text following the # symbol explains the commands and is not part of the session:
you@host:tmp> automateit # Start the AutomateIt interactive shell => AutomateIt Shell v0.5.0 # Welcome message ai> mkdir "asdfasdf" # Create a directory called "asdfasdf" ** mkdir asdfasdf # Message showing that directory was created => ["asdfasdf"] # Return value with array of directories created ai> mkdir "asdfasdf" # Try to create the same directory again => false # Nothing happened, the directory already exists ai> rmdir "asdfasdf" # Remove the directory ** rmdir asdfasdf # Message showing that directory was removed => ["asdfasdf"] # Return value with array of directories removed ai> rmdir "asdfasdf" # Try to remove the directory again => false # Nothing happened, there's no directory to remove
Notice how the second time mkdir was run above, it returned false and didn‘t create a directory? That‘s the conditional execution in action, realizing the action has already been performed. Similarly, the rmdir only ran the first time, but returned false and took no action when the directory was already gone. You can use the return values of these commands to write sophisticated logic of your own based on what actions took place.
AutomateIt uses an extensible plugin architecture to group together related commands:
Plugins can be accessed from the Interpreter like this:
ai> shell_manager.pwd => "/tmp"
The most common plugin methods have aliased shortcuts. For example, pwd is the alias for shell_manager.pwd. These are documented in AutomateIt::Interpreter‘s "Aliased methods."
Each plugin has one or more drivers that implement its functionality.
For example:
Plugins describe consistent interfaces for related features, and drivers implement this API for different tools.
For example, AutomateIt provides a common API for all packaging tools: AutomateIt::PackageManager. It provides drivers for packaging tools like APT, YUM, Gem, Egg and others. To install a package called foo with the apt driver:
ai> package_manager.install "foo", :with => :apt
AutomateIt will check if foo is installed, install it if needed, or do nothing if the package is present. This API is the same for all packaging tools, making it easy to get work done by using high-level AutomateIt commands instead of cryptic tool-specific commands. Although AutomateIt uses the low-level tools, it uses them with best-practices approaches and hides the senseless complexity from the user.
What‘s the big deal? Consider how one would install packages from the Unix shell. Most packaging tools are pathologically dysfunctional and make it bafflingly difficult to programmatically install or uninstall packages, or tell if a package is installed. Many don‘t use exit values and require complex output parsing. Others require user-input even when it‘s obvious that none is needed. Almost all make it necessary to write conditional code because they‘ll either fail with errors if told to install an existing package, destroy an existing setup, or install duplicate packages. Writing Unix shell code to handle all these quirks is frustrating and risky.
Here‘s a sample Unix shell command for installing a package using one of the simplest, most reasonable tools available — although note that unlike AutomateIt, this shell command can‘t handle multiple packages, is slower, can‘t be previewed, and has no consistent error handling:
if dpkg-query -W --showformat '${Status}\n' foo 2>&1 | \
egrep -q '(^| )installed'; then
apt-get install -y -q some_package_name < /dev/null
done
Now compare that hideous command to the simple, clear and consistent AutomateIt command before it. AutomateIt‘s plugins and drivers are easy to install and write. As more are written, more people will hopefully be freed from needlessly convoluted low-level commands, and able to get simple things done simply. AutomateIt‘s consistent API, multiple drivers, sane defaults and conditional-checking make it easier to write clear and maintainable recipes than using the low-level directly commands from the Unix shell.
AutomateIt can automatically detect the most suitable driver for each plugin command.
For example, the AutomateIt::PackageManager plugin has drivers called AutomateIt::PackageManager::APT and AutomateIt::PackageManager::YUM. On a Debian system, which uses the apt-get packaging tool, AutomateIt will default to using the AutomateIt::PackageManager::APT driver:
ai> package_manager.installed? "apache2" => true
Sometimes it‘s necessary to specify the driver to use. The recommended way to do this is to pass a :with => :driver_name option to the plugin command.
For example, tell the package manager to use the Gem driver:
ai> package_manager.installed? "automateit", :with => :gem => true
You can also completely bypass the plugin and its auto-detection to directly interact with the driver:
ai> package_manager[:gem].installed? "automateit" => true
A project is a special directory that contains related recipes and helper files. Although it‘s possible to run recipe files without a project, a project provides many useful features, described in the AutomateIt::Project documentation.
A project is created by specifying the directory to create:
you@host:tmp> cd /tmp you@host:tmp> automateit --create myproject ** mkdir -p myproject => Creating AutomateIt project at: myproject ** cd myproject ** mkdir config ** cd config => Rendering 'tags.yml' because of it doesn't exist => Rendering 'fields.yml' because of it doesn't exist => Rendering 'automateit_env.rb' because of it doesn't exist ** cd - ** mkdir dist ** cd dist => Rendering 'README.txt' because of it doesn't exist ** cd - ** mkdir lib ** cd lib => Rendering 'README.txt' because of it doesn't exist ** cd - ** mkdir recipes ** cd recipes => Rendering 'README.txt' because of it doesn't exist ** cd - => DONE!
This creates a /tmp/myproject directory with the newly-created project. It contains directories, each with a README.txt file explaining the directory‘s purpose, and individual files like tags.yml that contain comments with basic usage instructions.
The project creator is an AutomateIt recipe, so it‘s smart enough to only execute the commands needed. If the project creator is re-run against an existing project, it won‘t make any changes because none are needed:
you@host:tmp> automateit --create myproject => Found AutomateIt project at: myproject => DONE!
To associate a recipe with a project, put it into the project‘s recipes directory.
For example, create a recipes/hello_project.rb file with these contents:
puts "Hello, this is: " + project
And run it:
you@host:myproject> automateit recipes/hello_project.rb Hello, this is: /tmp/myproject
The Interpreter automatically loads the project. The project method contains the project path and is only available when executing a recipe associated with a project.
A project‘s config/fields.yml file is meant to store configuration constants, like custom paths for applications. Fields abstract configuration variables for recipes, improving maintainability by separating data and logic. Fields can also be easily queried from Unix using the aifield command. More information about fields and aifield can be found in AutomateIt::FieldManager.
For example, consider a fields file with the following contents:
myuser: dhh myapp: path: /var/www/rails
These fields can be used from the interactive shell like this:
you@host:myproject> pwd
/tmp/myproject
you@host:myproject> automateit -p . # (1) Load project from current directory
=> AutomateIt Shell v0.5.0
ai> lookup :myuser # (2) Lookup string value by symbol key
=> "dhh"
ai> lookup "myuser" # (3) Lookup string value by string key
=> "dhh"
ai> lookup "myapp" # (4) Lookup hash value by string key
=> {"path"=>"/var/www/rails"}
ai> lookup "myapp#path" # (5) Lookup string value by compound key
=> "/var/www/rails"
To load a project‘s fields, the AutomateIt interactive shell needs to know which project to load. On the line annotated (1), automateit -p . tells it to load the project from the "." directory, the current directory. The argument can be an absolute or relative path. The command accepts other arguments and environmental options, run automateit —help for details.
Fields can be queried by string (2) or symbol (3) keys. The lookup command can return any associated data, usually a string, but it can be a complex type, such as a hash (4). The compound-key (5) syntax is a convenient syntax for looking up nested keys in a hash. For example, lookup "myapp#path" is a shortcut for <tt>lookup("myapp")["path"] — these query the myapp hash and return the value of its path key.
The Interpreter loads a project and its fields automatically for recipes, so there is no need to specify the -p option. For example, a project recipe file recipes/hello_fields.rb contains:
puts lookup("myapp#path")
This recipe will load its project and fields automatically:
you@host:myproject> automateit recipes/hello_fields.rb /var/www/rails
A project‘s config/tags.yml file describes tags assigned to hosts, grouping together hosts by their roles and attributes.
For example, this tags file describes a "desktops" tag with three hosts named "satori", "sunyata" and "michiru", and another tag named "notebooks" with other hosts:
desktops:
- satori
- sunyata
- michiru
notebooks:
- rheya
- avijja
The tags can be accessed like this when run on a host named "satori":
you@satori:myproject> automateit -p .
ai> tags
=> ["satori", "desktops", "localhost", ...] # Tags for this host
ai> tagged?("desktops") # Is this host tagged with "desktops"?
=> true
ai> tagged?("notebooks")
=> false
ai> tagged?(:satori) # Strings and symbols are treated the same.
=> true
ai> tagged?("satori")
=> true
ai> tagged?("satori || desktops") # Query by simple boolean expression
=> true
ai> tagged?("(satori || desktops) && !notebooks") # Query by complex boolean expression
=> true
Role-based behavior can be demonstrated by creating a file called recipes/hello_tags.rb:
if tagged?("localhost")
puts "I'm a localhost!"
end
if tagged?("desktops")
puts "Special commands to execute on desktops"
elsif tagged?("notebooks")
puts "Special commands to execute on notebooks"
end
Here are the results when executed on a host called "satori":
you@satori:myproject> automateit recipes/hello_tags.rb I'm a localhost! Special commands to execute on desktops
Notice how the notebooks section wasn‘t executed? Tags make it possible to create sophisticated recipes that run commands on only the appropriate systems.
More documentation on tags can be found in AutomateIt::TagManager.
AutomateIt lets you preview commands the recipe will run without actually letting them actually modify the system.
The Interpreter has a boolean that determines if it‘ll make changes to your system. This one variable can be called from two different ways:
Here‘s an example with the interactive shell, with comments for annotations:
ai> noop true # Enter noop mode ai> noop? # Currently in noop mode? => true # Yes, in noop mode ai> writing? # Writing to disk? The opposite of noop => false # No, not writing ai> mkdir "foo" # Try to create a directory ** mkdir foo # Message showing directory will be creating => ["foo"] # Return value with directories to create ai> File.directory? "foo" # Was the directory actually made? => false # No, noop mode prevented the change
Recipes can take advantage of the noop mode as well, consider a mkdir_example.rb recipe:
mkdir "foo"
To run this recipe with noop mode, pass the -n option to the Interpreter shell:
you@host:tmp> automateit -n mkdir_example.rb ** mkdir foo you@host:tmp> ls foo ls: foo: No such file or directory
Notice how the directory wasn‘t actually created? This is great because you can see exactly what commands AutomateIt will run without actually having it apply the changes.
WARNING: Previewing code can be dangerous. Read previews.txt for instructions on how to write code that can be safely previewed.
I hope you enjoy working with AutomateIt and look forward to hearing about your experiences with it. Drivers, patches, documentation and ideas are welcome.
Thank you for taking the time to read this!