Project

machines

0.0
No commit activity in last 3 years
No release in over 3 years
Install and configure Ubuntu desktops, laptops, servers and cloud instances. Install software, configure settings, preferences, keys, projects, applications, scripts, etc...
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies
 Project Readme

Machines

A custom Ubuntu system in less than 15 minutes

Setup customised Ubuntu development and server Machines locally or in the cloud for developing and hosting Ruby, Rails and related technologies.

Write commands in Ruby like:

sudo install %w(build-essential zlib1g-dev libpcre3-dev)
sudo write "127.0.1.1\t#{$conf.hostname}", :to => '/etc/hosts'
sudo append "192.168.1.2\tserver", :to => '/etc/hosts'
run install $conf.nginx.url
run create_from 'nginx/nginx.conf.erb', :to => File.join($conf.nginx.path, 'conf', 'nginx.conf')

Example to upgrade passenger:

machines build phil_workstation passenger_nginx

RubyDoc.info documentation

Status

July 2013

  • Released 0.5.6 gem
  • Some gem updates, cleaned up API
  • Working development and server builds.

Working on:

  • Cloud deployments

Features

  • An opinionated Ubuntu configuration script with sensible defaults
  • Easily override the defaults with configuration options and custom ruby
  • Default template supports Nginx, Passenger, Ruby, Rails, MySQL, Git, Monit, Logrotate
  • Preconfigured Ruby & Rails minimal development environment (Openbox, Subtle, dwm)
  • Bring up new instances fully configured in less than 15 minutes
  • The best form of documentation for your architecture
  • More secure (by installing only what you want)

What it's Not

  • It does not replace the base installation (Yet). A minimal Ubuntu must be installed prior to running the Machines install (instructions provided below)
  • It's not a full blown configuration management tool such as Puppet or Chef although it can be used to run packages on the local machine to install new versions

Motivation

Configuration management is a complex topic. I wanted to reduce some of the variables (single target platform, single development environment) to provide a simpler solution.

Overview

The top level script is the Machinesfile. This contains a list of packages to run. Packages contain one or more tasks and the tasks contain the commands to run. Default packages are provided by Machines. Default packages can be overridden and new ones created.

Commands are added to a queue with sudo or run. lib/packages contains the packages you can add in the Machinesfile in Machines. Once the build starts the commands are run and shown with the current progress.

Installation and Configuration

Install the gem

gem install machines

Generate an example project

machines new example

Configure your deployment

Take a look at the generated project. It contains several folders with templates and configuration settings for various programs, your Machinesfile and the various .yml files.

  • machines.yml is your architecture. Here you'll add all the computers in your setup.
  • webapps.yml contains the web applications you develop and maintain.
  • config.yml contains settings for various packages. Versions, Cloud setup, paths, etc.
  • users/ contains user specific preferences, dotfiles, etc

So here is the recommended approach to configuring your environment:

  • Create your architecture in machines.yml (see notes in file)
  • Add you webapps in webapps.yml
  • Edit the build script (Machinesfile)
  • Add your websites certificates and amazon private key (if required)
  • Edit settings in config.yml
  • Run machines override <package> to copy a default package to your project for alteration (Use machines packages to see a list)
  • Setup your users/ folders
  • Add ~/.ssh/id_rsa.pub public key from all users machines that need access, to the users/www/authorized_keys file

Prepare the target machine

Grab a suitable ISO from https://help.ubuntu.com/community/Installation/MinimalCD (Tested with 13.04)

Ubuntu 13.04 ISO works with CD or USB

  • To copy to USB

    dd if=mini.iso of=/dev/sdX where sdX is your USB device (use dmesg to get this)

Older ISOs do not allow USB stick install. You need to download the image instead.

  • Grab an ISO or image to install a minimal Ubuntu 12.04 (LTS)
  • Images can be written to USB with gunzip boot.img.gz && sudo dd if=boot.img of=/dev/sdX where sdX is your USB device (use dmesg to get this)

Insert the USB stick and boot from it to install Ubuntu (See Setting up a test VM below for installing on a Virtual Machine)

  • Install SSH Server & note the IP address

    sudo apt-get update && sudo apt-get -y install openssh-server && ifconfig

Check the Machinesfile

machines dryrun <machine>
cat log/output.log

Check the machine

ssh-keygen -R <host ip> # remove host from known_hosts file (handy when testing)
$ ssh <IP ADDRESS>      # Make sure you can connect to the machine

Build the machine

$ machines build <machine>

Console output:

  • Running commands are displayed in gray
  • Tasks show in blue
  • Successfully completed commands are displayed in green
  • Failures show in red
  • Yellow indicates there was no check for the command

While running open another terminal to view detailed output:

tail -f log/output.log

or debug:

tail -f log/debug.log

Commandline Options

machines COMMAND
COMMAND can be:
  htpasswd                 Generates basic auth in webserver/conf/htpasswd
  new <DIR>                Generates an example machines project in DIR
  dryrun                   Logs commands but does not run them
  tasks                    Lists the available tasks
  build <machine> [task]   Builds your chosen machine. Optionally, build just one task
  packages                 Lists the available packages
  override <PACKAGE>       Copies the default package into project/packages so it can be edited/overidden

Global settings

Machines uses a gem I wrote called app_conf. It allows settings to be loaded from YAML and also set using Ruby. Machines uses it both internally and for package settings. Some of the settings set and used by Machines are:

  • $conf.appsroot - Where applications are cloned
  • $conf.commands - All the commands that are to be run
  • $conf.environment - Environment of the machine (also at $conf.machine.environment)
  • $conf.machine - Configuration for the selected machine
  • $conf.machine_name - Name of the selected machine
  • $conf.roles - List of roles for selected machine (also available as $conf.machine.roles)
  • $conf.tasks - Names of the tasks - Used to check dependencies and display tasks the help
  • $conf.user - The selected user name
  • $conf.user_home - Users home folder e.g. /home/phil
  • $conf.users - A list of the available users
  • $conf.webapps - A hash of webapps keyed from the name of the webapp specified in webapps.yml

The following settings are set rbenv, rvm, passenger and used by passenger_nginx packages:

  • $conf.ruby.gems_path - e.g. .rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems
  • $conf.ruby.executable - e.g. .rbenv/versions/1.9.3-p448/bin/ruby
  • $conf.passenger.root - Is just the ruby gems path plus passenger-version
  • $conf.passenger.ruby - Is simply the absolute path to the ruby executable

Take a look at template/*.yml for more.

Commands

Commands you can use with run or sudo are in the following modules (links to rdoc):

When creating your own commands the Checks module should be useful.

Setting up a test VM

Make sure you've downloaded one of the ISOs from the list in Prepare the target machine.

  1. Download and install VirtualBox
  2. Oracle VirtualBox -> New
  • Name: machinesvm
  • Operating System: Linux
  • Version: Ubuntu (64 bit)
  1. Settings -> Network -> Adapter 2 -> Enable Network Adapter
  • Attached to: Host-only Adapter
  1. Start VM -> Next -> Media Source folder icon (next to dropdown) -> mini.iso -> Next -> Start
  2. Ubuntu Installer boot menu -> Install -> Accept defaults except for:
  • Select correct country/keyboard
  • Primary network interface: eth0
  • Full name for the new user: user
  • Username for your account: user
  • Choose a password for the new user: password
  • Re-enter password to verify: password
  • YES to partition disks (choose partition entire disk if asked)
  • Add OpenSSH from application installer menu
  1. Devices -> CD/DVD Devices -> Uncheck mini.iso
  2. Reboot and login
  • login: user
  • Password: password
  1. Run ifconfig and make a note of the eth1 inet addr. It should be 192.168.56.101 if this is the first VM you've installed
  • If you don't see eth1 then sudo nano /etc/network/interfaces and add the lines auto eth1 and iface eth1 inet dhcp as with eth0. Save and exit. Run ifconfig again and note the inet addr of eth1
  1. Take a snapshot (restore before each test run)

What's happening under the hood

  • An SSH connection is established to send all commands and uploads (similar to Capistrano) to the target machine
  • SSH uses the specified user and then sudo is added to commands that require it
  • When sudo is needed for file uploads. The file is uploaded to /tmp then sudo cp'd to the destination
  • When package is called in the Machinesfile that file is loaded either from the projects packages folder or from the Machines packages if not found in the project
  • It is executed within the context of the Core class. This class defines basic commands such as task, run, sudo and includes all the Commands modules so they are available to the running package

Limitations

  • Only one user per machine. Although other users could be setup with additional build runs.
  • One environment per machine - Again additional machines could be configured to use the same physical machine (although could be problems with some environment settings)
  • Servers use www (by default) for nginx/apache, passenger and deployments
  • The system has been designed to allow a certain flexibility in the configuration although some things may not yet be totally configurable it should be possible to add or modify the relevant package
  • We are currently focused on Ruby 1.9.3 (Moving to 2.0 soon), Rails 3 and Passenger 3
  • Some commands may not properly escape quotes when used with sudo (e.g. append and replace). This may be addressed in a future release
  • Tasks not configured to run for a particular setup will not be available to run explicitly from the commandline

Development, Patches, Pull Requests

  • Fork the project
  • Test drive your feature addition or bug fix
  • Commit, do not mess with Rakefile, version, or history
  • Send me a pull request. Please use topic branches
  • Feel free to add, enhance or update packages and submit pull requests
  • Package tests are a bit of a pain but do catch a lot of potential issues so please add these

References

APIs

General

RVM

Ubuntu minimal install guides

Acknowledgements

Thanks to all the people that published the hundreds of articles, blog posts and APIs I've read.

Copyright

Copyright (c) 2010, 2011, 2012, 2013 Phil Thompson. See LICENSE for details.