Page 1 of 1

Rake A Ruby Based Build Tool

#1 dorknexus  Icon User is offline

  • or something bad...real bad.
  • member icon

Reputation: 1255
  • View blog
  • Posts: 4,618
  • Joined: 02-May 04

Posted 13 January 2010 - 01:47 AM

Attached File  rake_tutorial.pdf (126.93K)
Number of downloads: 779

Rake: A Ruby-Based Build Tool
Justin Kenworthy
January 13, 2010

Contents
1 Overview
2 Embedded Domain Specific Language
3 Dependencies
4 Rakefiles
5 The Rake Build Language
5.1 Tasks
5.1.1 Parametric Tasks
5.1.2 Descriptions
5.2 Declaring Dependencies
5.2.1 Dependency Lists
5.2.2 Open Dependency Lists
5.2.3 Parallel Dependencies
5.3 File Tasks
5.3.1 Directory Tasks
5.4 Synthesizing Tasks
5.5 Importing
5.6 Namespaces
6 Utilities and Extensions
6.1 FileList
6.2 File Path Extensions
6.3 Cleaning
6.4 Stack Trace
7 Documentation and Support

1 Overview
What exacttly is a build tool? A build tool allows for developers to automate
the many build processes they encounter on a regular basis. They help to
automate and streamline lengthy build processes which might be present in
even small scale projects. These processes include compiling, linking, testing,
packaging, document generation, and even deployment. Some build tools are
very automated and require very little manual configuration or scripting. These
tools look at the source files themselves to determine dependencies and resolve
them. Other tools depend entirely on scripts which inform the build system of
the dependencies a project might have. A few examples of popular build tools
are Make and Apache Ant. If you are already familiar with either then Rake
should be fairly easy to pick up as it is similar.

Who needs software build tools? This is really for developers to decide but
in reality just about anyone can use them. Even non-developers can find use for
such tools in non-programming situations. In fact, I am using Rake to construct
this document which is writtein in LaTeX. If you find yourself spending large
amounts of time building projects or if you just wish there was a way to expedite
the process then build tools are for you. This tutorial/reference assumes you
are already familiar with a little bit of Ruby. If not, you shouldn’t encounter
too many problems but things may be a little harder to understand.

You might have already guessed, but Rake is a software build tool. It is
written in Ruby and simply extends the Ruby language. This fact makes it an
embedded domain specific language (EDSL) which makes it very versatile and
powerful. We’ll talk about what exactly an EDSL is in the next section. For
now just understand that Rake is similar to Make and will help you to automate
all of your build process needs.

2 Embedded Domain Specific Language
This sounds pretty fancy but it is a simple concept. An embedded domain
specific language is a language which is embedded inside of another language
(usually a general purpose language). In this case, the Rake Build Language
is embedded inside of the Ruby programming language (neat!). This is really
powerful because it allows us to use the build language alongside the power of
Ruby. In this sense, Rake is an extension of Ruby. Rake is considered domain
specific because it was specifically made for automating build processes and it
isn’t good for much else. Rake itself is only a small extension of Ruby so most
of Rake’s allure comes from the ability to use Ruby to help us do our scripting.

As stated before, it is not a requirement to know any large amount of Ruby to
use Rake. However, knowing Ruby will significantly increase your productivity
with Rake as a build tool. Plus, you should learn Ruby anyways because it is
really awesome.

3 Dependencies
Rake does both dependency analysis and resolution. This means that Rake
will determine which items in a project are dependant on other items, and
then resolve those dependencies. To better illustrate this concept let us use an
example with some graphics:

Attached Image

In this example we can see that a calculator program is dependant on three
libraries and its own header file: LinkedList.h, StringLib.h, Math.h, and Cal-
culator.h. Each of these libraries themselves has their own dependants (their
header files). Finally, the executable is dependant upon the calculator source
file. If we were in a UNIX-like environment, assembling this project might look
something like this:

$  gcc   -Wall  -c  LinkedList.c
$  gcc   -Wall  -c  StringLib.c
$  gcc   -Wall  -c  Math.c
$  gcc   -Wall  -c  Calculator.c
$  gcc   -Wall  -o  Calculator Calculator.o LinkedList.o StringLib.o Math.o


We first compile our libraries into object files, then compile the program
itself into an object file, and, finally, link all of the object files together. This is
a fairly straightforward dependency graph for this example but you can imagine
how a graph might look for a larger project (operating systems, etc.). In this
example it is easy to figure out what objects are dependencies but it may not
be so easy in larger ones. The analysis of dependencies is important because
if we make a change to a single file we do not want to have to re-compile the
entire project. We only want to re-compile the file itself and the portions which
are dependant upon that file. More importantly, dependencies determine the
order in which the build must occur which will mean the difference between a
successful build or compiler/linker errors.

This example also helps to illustrate the point that even small projects can
become tedious to build. Imagine debugging this piece of software and re-
compiling it frequently without the aid of a build tool like Rake. Each time you
would have to issue five commands which is going to get old quick.

4 Rakefiles
A rakefile is simply a ruby script which contains the Rake build language. A
project can consist of multiple rakefiles or just one. The usual naming scheme
for these files is ”Rakefile” or ”rakefile” but, of course, you can name it whatever
you like. Rake will, by default, search for a file named ”Rakefile” and ”rakefile”
(among some others) if you do not specify an input file for it to use as a build
script. You can refer to the Rake manpage to see which files it looks for and
in what order. If you really feel like renaming the rakefile to something custom
then you will have to execute Rake like so:

$ rake -f your_rakefile


It is recommended that you use the default naming scheme, however, as
it is fairly standard. For this tutorial, we assume all build scripts are placed
in a rakefile named ”Rakefile”. The execution context for the rakefile is the
directory in which the rakefile appears (rather than the directory from which
Rake is executed). This means you can use paths relative to the rakefile.

5 The Rake Build Language
Rake itself offers a small set of extensions to Ruby and those extensions are
what we are going to discuss next.

5.1 Tasks
Simples tasks are the basic and main way of accomplishing work with a build
script. Tasks have names associated with them which are usually symbols but
can be strings as well. Tasks can also have dependencies (also called prequisites)
and actions to actually process a build. We will discuss dependencies in a later
section but for now, this is how you would declare and name a task:

task :default


In this example, the name of the task is ”:default” which is actually a Ruby
symbol. When executing Rake at the command line it will, by default, execute
the ”:default” task. If you do not declare this task in your Rakefile you will
have to specify which task to execute on the command line. Although we have
defined a new task associated with the symbol ”:default”, it doesn’t do anything.
Making a task actually do something is the responsibility of the task’s associated
action. If you don’t already know, all Ruby methods can have an optional
annoynmous block associated with them when they are invoked. In Rake, that
optional anonymous code block is how you give a task an action to be executed.
You associate an action with a task like so:

task :default do
	  puts "Hello, from the default task!!"
end


In Ruby, the ”do” keyword is how we define an anonymous code block. If
we were to execute Rake on this rakefile its output would be as follows:

$ rake
(in /home/darknexus/projs/rake_tutorial)
Hello, from the default task!!


Neat! Unfortunately, this doesn’t accomplish anything particularly useful so
let’s keep moving to find out how we can achieve more. Something that is fairly
common for most builds is that we want to execute some sort of shell-bound
commands. One of the extensions that Rake offers (although you can do it in
standard Ruby just as easily) is the ”sh” method, which executes the strings
you pass it in your shell environment. So let’s make our task more useful by
compiling a single HelloWorld.c program:

task :default do
	  sh "gcc -Wall -o hello_world HelloWorld.c"
end


When this rakefile is executed, the HelloWorld.c file will be compiled into a
binary form. Hooray! Still not entirely useful but this illustrates the purpose
and basic usage of tasks.

5.1.1 Parametric Tasks
A useful feature of tasks is that they can be parameterized. In other words,
Rake allows you to pass arguments to tasks which can augment their behavior
however you choose. The syntax is only a little different and looks something
like this:

task :default, [:arg1, :arg2] do |t, args|
	  puts args.arg1 + ’ ’ + args.arg2
end


In this case we have to allow our anonymous block to take arguments itself.
The first argument ”t” is the task object itself so that we can manipulate and
extract data from it (more on that later). The second argument ”args” is to
collect the arguments passed from the commandline. This allows us to interact
with the build script on the command line like so:

$ rake ’default[Hello,World!!]’
(in /home/darknexus/projs/rake_tutorial)
Hello World!!


5.1.2 Descriptions
Sometimes it’s nice to be able to describe what a task’s purpose is. In Rake this
can be easily accomplished by utilizing the ”desc” method. The method takes
a string and associates it with the next defined task. In a way this allows you
to comment each task. Here’s an example of adding a description to the default
task in our rakefile:

desc ’Run the default task’
task :default do
	 puts ’Hello, from the default task!!’
end


The Rake command line utility allows us to view all available tasks as well
as their descriptions this is done with the ”-T” or ”–tasks” argument. So let’s
examine the available tasks in our rakefile:

$ rake --tasks
(in /home/darknexus/projs/rake_tutorial)
rake default # Run the default task


5.2 Declaring Dependencies
After our discussion about dependencies we then proceeded to completely ignore
them. Well, prepare to introduce some dependencies! This is a really easy task
in Rake and only involves listing the names of the other tasks or files which a
task depends on. Here is a trivial example but this should give you the idea:

task :default => :first do
	 puts "Completed the default task!"
end
task :first => :second do
	 puts "Completed the first task!"
end
task :second do
	 puts "Completed the second task!"
end


In plain english, we are stating that ”The default task depends on the first
task. The first task depends on the second task.” Here is what executing this
rakefile will give us:

$ rake
(in /home/darknexus/projs/rake_tutorial)
Completed the second task!
Completed the first task!
Completed the default task!


Rake has done the dependency analysis for us and resolved them in the
proper order, starting with the second task.

5.2.1 Dependency Lists
Rake also allows for multiple dependencies to be declared for a single task. This
time we will give our default task multiple dependencies like so:

task :default => [:first, :second] do
	 puts "Completed the default task!"
end
task :first do
	 puts "Completed the first task!"
end
task :second do
	 puts "Completed the second task!"
end


Which, when rake’ed, gives us the following output:

$ rake
(in /home/darknexus/projs/rake_tutorial)
Completed the first task!
Completed the second task!
Completed the default task!


Not suprisingly, this is the output we expect given how we setup our depen-
dencies.

5.2.2 Open Dependency Lists
Convieniently, dependency lists are left ”open” after their associated tasks are
defined. In other words, we can continue to add dependencies to a task even
after we’ve declared it. In the previous example we associated two dependencies
with the default task. This can be accomplished in the same way using open
dependencies like so:

task :default do
	 puts "Completed the default task!"
end
	
task :first do
	  puts "Completed the first task!"
end
task :second do
	  puts "Completed the second task!"
end
task :default => :first
task :default => :second


This is handy because we may not always know at the time we declare a task
what all of its dependencies will be. This construct also allows us to place all
dependency declarations in a seperate file so they can be easily read. Of course
that would require us to later import the seperate file but we will talk about
that in a later section.

5.2.3 Parallel Dependencies
In the interest of effeciency, Rake offers the ability to execute dependency reso-
lution in parallel. Rake does this by giving each dependant task its own thread
of execution so they all operate simultaneously. Rake’s provided data structures
are all thread safe. Rake will look at the provided dependencies and if those
dependencies have common dependencies of their own, Rake will first wait to
resolve the parallel dependencies until those common prequisites are fulfilled.
Here is a quick example of using the ”multitask” method to setup parallel de-
pendency resolution:

multitask :mytask => [:task1, :task2, :task3] do
	  puts "Completed parallel execution of tasks 1 through 3."
end


5.3 File Tasks
Many times, we want to define our tasks in terms of files. This is useful because
we can define how certain output files needs to be produced. We can accomplish
this in one of two ways. The first way is with standard tasks:

task :myfile do
	  src = ’src/myfile.c’
	  target = ’build/myfile.o’
	  unless uptodate?(target, src)
		   sh "gcc -Wall -c myfile.c"
	  end
end


This method will convert the dependant source file into the target file (which
, in this case, is an object file). This task also has a nice feature which checks
to see if the source file has been modified since the generation of the target file.
If so, it will recompile the source file, otherwise it will do nothing. This saves
useless compilation steps because there is no sense in re-compiling something if
its source has not been changed. The second way to create a file-based task is
like so:

file ’build/myfile.o’ => ’src/myfile.c’ do
	  sh "gcc -Wall -c -o build/myfile.o myfile.c’
end


This method is exactly like the first. It has a dependant task (in this case
a file task) and it compiles the source file into an object file. Also like the
first method it will only perform its associated action if the source file has been
modified since the target file was generated. As you can see, this method is
much more concise so this should be the preferred method for declaring file-
based tasks. Just like regular Rake tasks, you can associate more than one
dependency with each file task. You can also give file tasks descriptions

5.3.1 Directory Tasks
Like source files, we may also wish to create directory-based dependencies. For
instance we may want to require that a build directory be created before actually
building any portion of the project. Directory tasks are specialized file tasks.
They will only create the specified directory if that directory does not already
exist. Here is a quick example of a directory dependency:

directory ’build’
file ’build/myfile.o’ => [’build’, ’myfile.c’] do
	  sh "gcc -Wall -c -o build/myfile.o myfile.c
end


5.4 Synthesizing Tasks
In programs like Make, we have the ability to write implicit rules which create
tasks on the fly during the build process. The ability to generate tasks pro-
gramattically is pretty handy because, remember, we should let programs write
programs whenever possible (or something like that). Rake offers a construct
called rules which informs the Rake system how to handle files that are listed
as dependencies but have no task to explain how to resolve them. Here is an
example:

rule ’.o’ => [’.c’] do |t|
	  sh "gcc -Wall -c #{t.source}"
end


With this rule in place, any dependant ”.o” file without a defined task will
be generated from a ”.c” file with the same name. This rule assumes the cor-
responding ”.c” files actually exists in the same directory as the ”.o” file. If
that file does not exist then it will attempt to construct it from other rules or
tasks. Now lets look at another example, assuming our previous rule example
has already been defined:

task :default => [’StringLib.o’, ’LinkedList.o’, ’Math.o’] do
	  sh "gcc -Wall -o calculator StringLib.o LinkedList.o Math.o"
end


The rule allows us to simply define the default task with the dependant ”.o”
files. For each dependant ”.o” file, Rake will use our previously defined rule to
synthesize a file task to generate that ”.o” file. Rules can also accept regular
expressions to create more advanced rules and file paths.

5.5 Importing
For larger projects, we do not always want our entire build process to be handled
by a single rakefile. Even for smallers projects it is helpful to list tasks in one file
and dependencies in another. For these purposes, Rake enables us to include
other rakefiles. Of course, we could easily accomplish this with the ’require’
method in Ruby but that presents us with a problem: what if the file we want
to import is the product of a task? This means that the external rakefile will
not exist until a file task (or some other task) is executed. Rake provides an
”import” method to overcome this obstacle. If you import a file whose path
matches a task name then it will first execute that task before importing file.
Here is an example of usage:

file "depends.rake" do
	  # ...Actions to produce depends.rake
end
import "depends.rake"


As stated before, if ”depends.rake” does not exist when the import call is
made, then Rake will attempt to produce it by executing the file task for that
file. It will then try to include the file into the calling rakefile at that point. If
the file still does not exist it will complain.

5.6 Namespaces
Being that we are on the topic of dealing with larger projects, we should discuss
the utility of namespaces. Namespaces give us the ability to avoid naming col-
lisions. As projects become larger we are bound to encounter naming collisions
for tasks. To overcome this problem, Rake gives us the ”namespace” method
which lets us create named compartments to construct tasks. Name collisions
cannot occur across namespaces because of this compartmentalization. so, let’s
creat a few namespaces to illustrate their utility:

namespace "test" do
	 task :build do
		  # ...Actions to produce unit tests...
	 end
end
namespace "proj" do
	 task :build do
		  # ...Actions to produce the project...
	 end
end


To invoke these tasks at the commandline we would denote the namespace
of interest like so:

$ rake test:build
$ rake proj:build


6 Utilities and Extensions
6.1 FileList

If a project is composed of many files it becomes a chore to generate all of the
needed file tasks manually. Part of the Rake module includes a FileList class
which allows us to aggregate file names into a nice array. The list’s constructors
can accept glob patterns as arguments to construct a list that, say, contains
anything ending in ”.c”. Using FileList’s, we can then synthesize file tasks.
Here is an example which will dynamically generate file tasks for all ”.c” files:

SRC = FileList[’.c’]
BUILD_DIR = ’build’
task :default do
	 puts "Completed compiling all C files into object files."
end
SRC.each do |source|
	 target = File.join(BUILD_DIR, source.sub(/.c$/, ’.o’))
	 file target => source do
		  sh "gcc -Wall -c -o #{target} #{source}"
	 end
	 task :default => target
end


This composes a FileList which consists of all files that end with ”.c”. We
set our build directory to ”build”. Next a default task is created to handle the
default invocation of Rake. Next we generate a file task for each ”.c” file in
the ”SRC” FileList. We produce a target path which consists of the ”.c” file’s
basename with ”.o” appended to it and ”build/” prepended to denote it belongs
in the build directory. After generating each file task, we also list the new target
file as a dependency for the default task.

The nice thing about this approach of synthesizing tasks is that we do not
have to modify the rakefile each time we add a new ”.c” file. Any new ”.c” files
will be automatically included for compilation. This saves us time and shifts
the work to the build tool which is where it belongs.

6.2 File Path Extensions
To help you create rakefiles, Rake provides some extensions to the existing
String class in Ruby. One of these exensions is the ”ext” method which makes
changing up a file’s extension quick and easy. For example, we might want to
convert a source file name into a target file name. We can easily do that like so:
"myfile.c".ext "o"

This will convert the file extension of the ”.c” file to a ”.o” file. Take note
that this method modifies the String in place so the object itself will be modified
rather than just returning a modified String. For more advanced path manip-
ulation you can use the ”pathmap” method which takes in a format string to
explain how you would like to change the path of a file. Here is an example of
the more advanced ”pathmap” method:

"myfile.c".pathmap "%X.o"


This accomplishes the same task as the previous example using ”ext.” You
can look up more format flags in the official Rake documentation which is dis-
cussed briefly at the end of this tutorial.

6.3 Cleaning
Because cleaning up from the build process is a very common need, Rake has
these sorts of tasks built-in. First you will have to import the support for
automated cleanup like so:

require ’rake/clean’


However, there may be a little bit of confusion between the the two most
popular types of cleanup Rake offers: clean and clobber. Clean is usually used
to cleanup the intermediate products of a build process. Clobber is usually used
to clean up all products of the build process. The official Rake documentation
states that the clean directive will only remove those files listed in the CLEAN
FileList. The clobber directive will cause Rake to remove those files listed in
both the CLEAN and CLOBBER FileList’s. To include a file for cleanup, you
simply include it in one of the existing FileList’s:

CLEAN.include(’*.o’, ’build.log’)
CLOBBER.include(’binary1’, ’binary2’, ’binary3’)


Take note that you are able to utilize glob expressions for file names as we
did in this example which includes all files ending in ”.o” for cleaning. Once
you have setup those FileList’s the way you want, you simply invoke Rake on
the command line like so to engage the cleanup tasks:

$ rake clean
$ rake clobber


6.4 Stack Trace
Because one day your rakefiles are bound to produce unexpected operations, we
are able to generate stack traces to catch a glimpse at some of the internal work
Rake does on our behalf. By default Rake keeps quiet about this sort of stuff
but you can pass the ”–trace” flag on the command line to get a stack trace for
the build process associated with your rakefile. Here’s an example:

$ rake --trace


7 Documentation and Support
For more documentation you can refer to the following great articles and refer-
ences which I used to assist in writing this tutorial:
Using the Rake Build Language
http://martinfowler....icles/rake.html
Rakefile Format
http://rake.rubyforg...efile_rdoc.html
RAKE – Ruby Make
<a href="http://rake.rubyforge.org/" target="_blank">http://rake.rubyforge.org/</a>

This post has been edited by Dark_Nexus: 13 January 2010 - 04:32 AM


Is This A Good Question/Topic? 1
  • +

Replies To: Rake

#2 EdwinNameless  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 120
  • View blog
  • Posts: 710
  • Joined: 15-October 09

Posted 09 February 2010 - 01:39 AM

Wow, excellent stuff, thanks for that!
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1