O'Reilly    
 Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples


How to Build Simple Console Apps with Ruby and ActiveRecord

by Gregory Brown
06/21/2007

If you're coming into Ruby via Rails, you've probably noticed that there is a whole lot to learn within the framework itself. In fact, the integration between Rails (the framework) and Ruby (the programming language) is so tight, that it might be tough to easily see where Rails ends and Ruby begins.

In terms of productivity and learning curve, this is a great feature. It means that you can learn both at once without encountering the chicken and egg problem you might find elsewhere. Still, if you're either a control freak, a tinkerer, or someone with needs that don't comfortably fit within a web browser, you've probably thought about exploring more of Ruby outside of Rails.

The really attractive thing about branching out into writing Ruby code outside of Rails is you don't need to throw away all of your Rails experience. Most of the features found in Rails are broken out into separate packages, which means you can cherry pick from your favorites and use them in your Ruby applications.

Introducing EarGTD, the Easy ActiveRecord GTD System for Ruby

Database driven web applications are great, but nothing brings back the power and glory of 1986 like a database driven console application. To illustrate this, we'll be walking through a simple Ruby application with some Rails goodness sprinkled on top.

Rather than taking the approach of building up an application as we work through the article tutorial style, this time, we'll be treating the source code like a cheap magazine, flipping back and forth to the interesting parts, and skipping some of the mundane details. This means your experience will be a whole lot more enjoyable if you grab the EarGTD source package so you can follow along and tinker with it as we explore its different parts.

A Quick GTD Primer

Many programmers I know are big fans of David Allen's Getting Things Done (GTD) productivity system. Though the system is quite comprehensive in all its practices and habits, I've always looked at it as mostly a "Mega To Do List."

The power of using this system is that all of your outstanding tasks end up in a single sink, and then you can cleverly filter them to be able to work on what makes the most sense productivity-wise at a given point in time.

I tend to prefer a minimalist approach for all of this, and EarGTD reflects that. I find that you can get pretty far with having just two ways to organize your tasks: by project and by context.

A project consists of several inter-related tasks. This can range anywhere from something code related to something like "Clean the garage."

A context is a conditional filter that lets you quickly decide whether a task is actionable at a given time. Certain things can be done anywhere, other things most definitely cannot. For example, "Mow the lawn" might have the context of "Home."

This topic can go a whole lot deeper, but since this is a Ruby article and not a productivity article, we'll leave it at that for now.

EarGTD in Action

EarGTD implements a very simple command-line interface that lets you record and manage tasks by project and context. It doesn't do a whole lot beyond that, but it is a perfect example of what you might build using plain old Ruby.

The following sample session shows how you might use this tool:

$ earGTD @
Looks like you have nothing to do.

$ earGTD + "Check in accounting module <Foo> [work]" 
$ earGTD + "Add tests for credit card validation <Foo> [work]" 
$ earGTD + "Call Joe Frasier [work]" 
$ earGTD + "Plant a tree <Beautification> [home]" 
$ earGTD + "Weed the garden <Beautification> [home]" 
$ earGTD + "Paint the shed [home]" 
$ earGTD + "Play tennis for 60 hours"   

$ earGTD @                           
1. Check in accounting module <Foo> [work]
2. Add tests for credit card validation <Foo> [work]
3. Call Joe Frasier [work]
4. Plant a tree <Beautification> [home]
5. Weed the garden <Beautification> [home]
6. Paint the shed [home]
7. Play tennis for 60 hours          

$ earGTD @c work
1. Check in accounting module <Foo> [work]
2. Add tests for credit card validation <Foo> [work]
3. Call Joe Frasier [work] 

$ earGTD @p Beautification
4. Plant a tree <Beautification> [home]
5. Weed the garden <Beautification> [home]   

$ earGTD - 5     

$ earGTD @p Beautification
4. Plant a tree <Beautification> [home] 

$ earGTD @                
1. Check in accounting module <Foo> [work]
2. Add tests for credit card validation <Foo> [work]
3. Call Joe Frasier [work]
4. Plant a tree <Beautification> [home]
6. Paint the shed [home]
7. Play tennis for 60 hours

Though it's not particularly fancy, even this simple feature set shows that it's somewhat useful for recording and filtering tasks.

Learning by Example

We're going to take a walk through EarGTD's implementation. We'll start by taking a look at how to get ActiveRecord hooked up without the aid of code generators. We'll then dive into some of the application's more interesting parts.

Along the way, we'll cover several different concepts, including console and file I/O, simple modular structure, and some other useful idioms for building Ruby console applications.

Of course, leaving Rails behind isn't without its sacrifices…

Living Without script/generate

One thing that's great about working with Rails is that it's really easy to generate the boilerplate for your project, and you also have a well defined directory structure to start with. When writing standalone scripts, you'll need to do most of that stuff by hand. Luckily, it's not a particularly hard thing to do.

What follows are essentially several recipes for how to handle some of the tasks that the generators typically do for you. Feel free to skip by the ones you already are comfortable with.

Directory Structure

Before hooking everything up, it is worthwhile to set up a basic directory layout for your project. EarGTD's looks like this:

data/
  ear_gtd.db 
  test_ear_gtd.db
lib/
  ear_gtd.rb           
test/
  test_ear_gtd.rb

Rakefile
earGTD

For a simple app, this is about all we need. Usually when code is actually distributed, the scripts you can run from the command line will go in a folder called bin/, but we skipped that step for simplicity.

Otherwise, this structure is fairly common among Ruby projects: lib/ holds your library files, data/ holds any of the projects data files, test/ your unit tests, and your Rakefile will go in the top-level directory.

There's nothing mandating this project layout, if you want to put all your code in one big file, including your tests and data, Ruby will not stop you. Still, the conventions are handy, and many tools rely on them in order to function properly.

Connecting to a Database

When working with Rails, you normally stash your configuration information in config/database.yml, but when using ActiveRecord standalone, you have to do a tiny bit more leg work.

Below is an example of how to convert the YAML configuration to a Ruby call.

development:
  adapter: sqlite3
  database: db/my_database.sqlite3

becomes

ActiveRecord::Base.establish_connection(
  :adapter  => :sqlite3,
  :database => "db/my_database.sqlite3" 
)

As long as this code is defined before you try to work with your model data, ActiveRecord (AR) will work as normal.

Though it's not really necessary, it's also pretty trivial to emulate the rails style config file approach by just feeding a parsed YAML file to establish_connection.

Fun Ruby stuff: The YAML library

Let's say you have a YAML file called config/database.yml that looks like this:

adapter: sqlite3
database: data/ear_gtd.db

If you wanted to use this to configure AR, you'd just do:

require "yaml" 
ActiveRecord::Base.establish_connection(YAML.load_file("config/database.yml"))

The loaded YAML file actually just represents a Ruby Hash, so you can pass it directly to establish_connection.

Of course, it's often fine just to keep everything in Ruby and not worry about configuration files.

Database Schema Definition

For small standalone apps, you probably won't need the power of migrations. Still, it is nice to not have to write SQL when you can avoid it. This is where ActiveRecord::Schema comes in.

The relationships for EarGTD are extremely simple. Contexts and Projects both have many tasks. The code below is what I'm using for my schema definition:

  ActiveRecord::Schema.define do 
    create_table :tasks do |t|
      t.column :description, :string
      t.column :status, :string
      t.column :context_id, :integer
      t.column :project_id, :integer
    end

    create_table :contexts do |t|
      t.column :name, :string
    end

    create_table :projects do |t|
      t.column :name, :string
    end
  end

As you can see, aside from the explicit method call, the code you write is identical to that which would go in migrations.

Between ActiveRecord::Schema.define and ActiveRecord::Base.establish_connection, you have the tools you'll need to use AR outside of Rails. Model definitions work exactly the same way as they do in Rails, so there is nothing special to worry about there.

Writing a Rakefile by Hand

One thing Rails users are sure to be familiar with is using rake for various project automation tasks. However, if you haven't written your own custom tasks before, you might not be familiar with how to create a Rakefile from scratch.

You can learn a whole lot by taking a peek at the Rakefile that is generated by Rails, but one of the most simple needs you'll have that's easy to handle with Rake is creating a test runner.

The following code shows what I'm using in EarGTD.

File: Rakefile
require "rubygems" 
require "rake" 
require "rake/testtask" 

task :default => [:test]

Rake::TestTask.new do |test|
  test.libs << "test" 
  test.test_files = Dir[ "test/test_*.rb" ]
  test.verbose = true
end

This should be fairly intuitive just by reading it. This uses libraries built into Rake to set up a test task that will run anything in the test/ dir that has a file name like test_something.rb

The code above sets the test task as default, so running rake anywhere within your project will run your test suite.

Though this is all fairly pedestrian stuff, I'd like to mention that the Dir class rocks. It's part of core ruby and lets you treat file pattern matches as Enumerable objects. Below is a quick irb session that shows why this is interesting.

Fun Ruby stuff: Dir

>> Dir["EarGTD/*"]
=> ["EarGTD/data", "EarGTD/earGTD", "EarGTD/lib", "EarGTD/Rakefile", "EarGTD/test"]
>> Dir["EarGTD/*"].map { |f| f.reverse }
=> ["atad/DTGraE", "DTGrae/DTGraE", "bil/DTGraE", "elifekaR/DTGraE", "tset/DTGraE"]
>> Dir["EarGTD/*"][0]                   
=> "EarGTD/data"

Code for Console Apps Doesn't Need to Be Ugly

It's really tempting to build little console apps in a way similar to how you'd write a shell script: as a procedural series of commands that form a ball o' code. Though this works fine in a pinch, it makes the code difficult to test and prevents it from playing well with others.

Still, how much is too much? You probably don't want a massive class heirarchy for a simple tool like EarGTD. Usually in cases like this, I find myself leaning towards a simple structure that borrows some ideas from functional programming.

Using module_function to Create a "Bag o' Functions"

You may have seen modules in Ruby used as mix-ins. This is when you define some functionality in a module and then include it in a class for use.

The following example will add the instance method b to the class C.

module A
  def b
    puts "this is b" 
  end
end

class C
  include A
end

d = C.new
d.b #=> "this is b"

Still, if you aren't really dealing with maintaining any state, why bother with classes? By using module_function, the following code could be simplified so that the method may be called directly on the module.

module A          

  module_function

  def b
    puts "this is b" 
  end

end

A.b #=> "this is b"

Since all the state for EarGTD is stored in the database, this is the approach we use. The result is a very clean modular interface. In fact, the EarGTD script itself becomes quite minimal in part because of this.

File: EarGTD
#!/usr/bin/env ruby
require "lib/ear_gtd"        

EarGTD.connect
EarGTD.process_command(ARGV)

You can see that all of the heavy lifting is essentially being done by a single method, process_command.

Creating a Simple Command Processor

An approach that helps keep things clean when building console apps in Ruby is to build some simple command processing. This could either be a method or a class that manipulates the data that is passed in from the command line.

In the case of EarGTD, all that needs to be handled are the arguments passed to the script. In Ruby, this is stored in a special variable called ARGV.

Above, you saw it used like this:

EarGTD.process_command(ARGV)

To set up my command processor, I simply peel off the first argument and store the rest of the arguments in an array, as Example 3 shows.

File: lib/ear_gtd.rb
def process_command(cmd)
  args = Array(cmd)
  command = args.shift
  case(command)
  when "+" 
    add_task(args[0])     
  when "@" 
    t = tasks 
    puts t.empty? ? "Looks like you have nothing to do.\n" : t

  # ... other commands omitted

  else
    puts "Que?" 
  end
end

Above is a simplified version of EarGTD's command processor. You can see that it's just using a case statement to figure out what command is being called and then delegating to some functions to do the actual tasks.

This allows you to treat the rest of your coding task as if you were writing a small function library, and cleanly separates that code from the user interface. Any interface that manipulates the input into something the command processor understands will work fine.

Simple Parsing of Input

You might have noticed that all data is entered as a single string to EarGTD.

For example, you can enter a task like:

$ earGTD + "a trivial task <a project> [a context]"

This is meant to simplify command processing. Since Ruby's regular expression support is quite good, the task of parsing that data is easy. Though it's possible to do it all in one sweep, I've opted to break it out into three different methods for clarity:

def parse_project(string)
  string =~ /<(.*)>/ && $1.strip
end

def parse_task(string)
  string.gsub(/[<\[].*[>\]]/,"").strip
end

def parse_context(string)
  string =~ /\[(.*)\]/ && $1.strip
end

Regular Expressions often look like line noise to the uninitiated, but they're really quite simple, and essential for pattern matching tasks.

Let's break these down, one by one.

parse_project looks for any amount of characters (.*) between angle braces, and stores them in the first "group." If the pattern matches, this is stored in $1, and then the method returns the results with all leading and trailing whitespace stripped.

The tests for this show what the method does:

@message = " do the foo < protocol 4 > [context]"  
expected = "protocol 4" 
assert_equal expected, EarGTD.parse_project(@message)

parse_context does the same thing, but for angle braces instead. The added noise with the back slashes in the pattern are because [] is a special construct in regexp, so they need to be escaped to match literals. This is quite similar to how you need to escape quotes in string, like \"this\"

The tests for parse_context are straightforward:

@message = " do the foo < protocol 4 > [context]"  
expected = "context" 
assert_equal expected, EarGTD.parse_context(@message)

parse_task

This code actually uses the [] construct we just mentioned. This is a character class, which will match any of the characters within it once. It also does a substitution via gsub. In english, the command:

  string.gsub(/[<\[].*[>\]]/,"").strip

means, "Take the string and anywhere you find some text between angle braces or square brackets, replace them with an empty string".

Here are the tests which show it in action:

@message = " do the foo < protocol 4 > [context]"    
expected = "do the foo" 
assert_equal expected, EarGTD.parse_task(@message)

Using these three helper methods, you can see the add_task method is actually quite simple.

def add_task(description)

  task_desc = parse_task(description)
  task = Models::Task.create(:description => task_desc)

  if (project=parse_project(description))
    Models::Project.find_or_create_by_name(project).tasks << task
  end

  if (context=parse_context(description))
    Models::Context.find_or_create_by_name(context).tasks << task
  end

end

If things got a little heavier, you might want to build a custom parser class, but this is about perfect for our needs. By breaking the parsers out into their own functions, we can change how they work without having to change the add_task code.

File and Console I/O

I/O is something that often trips up newbies in Ruby, but it's really quite simple. For example, printing a line to the screen is just puts "something" and reading a file is just File.read("somefile").

A simple example of writing to file is EarGTD's dump functionality. You can dump tasks to a file, as a backup that can be imported later if needed.

The following sessions shows how this works, after displaying the current tasks.

$ ./earGTD @
1. Check in accounting module <Foo> [work]
2. Add tests for credit card validation <Foo> [work]
3. Call Joe Frasier [work]
4. Plant a tree <Beautification> [home]
6. Paint the shed [home]
7. Play tennis for 60 hours

$ ./earGTD dump foo.txt

$ cat foo.txt 
Check in accounting module <Foo> [work]
Add tests for credit card validation <Foo> [work]
Call Joe Frasier [work]
Plant a tree <Beautification> [home]
Paint the shed [home]
Play tennis for 60 hours

The example above calls EarGTD.dump('foo.txt'). The implementation is shown below for reference:

def dump(file=nil)
  content = Models::Task.dump
  if file
    File.open(file,"w") { |f| f.puts(content) }
  end
  return content
end

We're basically just printing out the array of Task objects that is returned by Models::Task.dump.

This code is actually a little more clever than it looks, though. The object we're printing isn't a string, which means Ruby is making some assumptions about how to represent our data.

Fun Ruby stuff: puts()

As a print function, puts is actually very powerful.

It will iterate through an array printing the elements one at a time, and calling to_s on them along the way. Below is a simple example of how this works

>> class A
>>   def initialize(name)
>>     @name = name
>>   end
>>   def to_s
>>     @name.reverse
>>   end
>> end

>> a = [A.new("greg"),A.new("pete"),A.new("paul")]

>> puts a
gerg
etep
luap

puts will also try to call to_s on any object you pass it.

>> puts A.new("nick")
kcin

EarGTD's Task model implements a to_s method, which is why printing an array of tasks just works.

This can come in handy whenever you want to represent an object in a particular way for output. No explicit conversions to string representation are necessary.

Because of features like these, I/O is fairly comfortable to work within Ruby.

The End of the Tour

We might be able to squeeze some more lessons out of EarGTD, but we've covered enough ground for now.

Though this article is unlikely to make you an instant Ruby guru, it has hopefully exposed some interesting idioms to you for further exploration. Leveraging the libraries that make up the Rails underbelly can be a really powerful skill, and allows you to gain the best of both worlds for a lot of different problems.

I should mention that for more complex console application needs, you'll often want to use a tool that will help handle the low level bits for you, such as Ruby's standard library OptionParser, or for interactive applications that need input validation, HighLine.

Of course, there's nothing wrong with keeping it simple, either!

Gregory Brown is a New Haven, CT based Rubyist who spends most of his time on free software projects in Ruby. He is the original author of Ruby Reports.


Return to Ruby.

Copyright © 2009 O'Reilly Media, Inc.