Rails Performance Diary

Although this blog post is titled Rails Performance Diary, it actually has more to do with SQL. For many websites, much of the work is being done by the database server. Such is the case for a simple dog site that I am working on. Having done quite a few websites using Ruby on Rails, I was surprised to find this simple site giving me some major performance problems. Pages were loading slow and sometimes timing out and there was no traffic to the site, other than the developers.

Fortunately, the Rails log outputs all of the queries by default, which is always the first place that I look. And here's what I found -- 2 queries:

1) SELECT * FROM breeds

2) SELECT breeds.id AS t0r0, breeds.name AS t0r1, breeds.alias AS t0r2, breeds.lookup AS t0r3, breeds.info AS t0r4, breeds.line_breaks AS t0r5, breeds.has_pics AS t0r6, breeds.has_videos AS t0r7, pictures.id AS t1r0, pictures.breed_id AS t1r1, pictures.original AS t1r2, pictures.large AS t1r3, pictures.medium AS t1r4, pictures.thumbnail AS t1r5, pictures.caption AS t1r6, pictures.email AS t1r7, pictures.notes AS t1r8, pictures.disabled AS t1r9, pictures.order_field AS t1r10, pictures.created_at AS t1r11, pictures.filename AS t1r12, pictures.breed_lookup AS t1r13, videos.id AS t2r0, videos.breed_id AS t2r1, videos.title AS t2r2, videos.description AS t2r3, videos.code AS t2r4, videos.created_at AS t2r5, videos.orig_url AS t2r6, videos.embed_src AS t2r7, videos.thumbnail AS t2r8, videos.time_length AS t2r9, videos.length_seconds AS t2r10, videos.youtube_id AS t2r11, videos.lookup AS t2r12 FROM breeds LEFT OUTER JOIN pictures ON pictures.breedid = breeds.id LEFT OUTER JOIN videos ON videos.breedid = breeds.id WHERE (breeds.lookup = 'airedaleterrier' )

The next step was to run each of these queries in the MySQL Query Browser, so I could see how long it took the queries to run. The first query took 13 seconds to execute, so I felt that I had found the problem. Much to my chagrin, however, I found the second query to take over a minute!!! Clearly, neither one of these queries are acceptable, even with caching.

The First Query : SELECT * FROM breeds

This query was being used simply to get a list of the dog breeds so that I could link to each of their info pages, yet the query was returning the entire record, which includes a long "description" column containing a lot of text. This query was generated by ActiveRecord from the following code: @breeds = Breed.find(:all) To solve the problem, let's force a specific query that doesn't include the description field: @breeds = Breed.find_by_sql("SELECT id, name FROM breeds") Immediately, the query dropped from 13 seconds to a fraction of a second.

Second Query: SELECT breeds.id AS t0_r0, breed....

Here was the stumper. This vast expanse of SQL was generated from the following Ruby: @breeds = Breed.find_by_lookup(lookup, :include => [pictures, videos]) Typically, including relations speeds things up because it allows an SQL join to be used rather than multiple SQL queries. But, in this case, the join only reduces 3 queries to 1 query and incurs a large overhead to do so. Join tables multiply the number of records for each table joined, so in this case the number of results was extremely large. Now, I'm sure a veteran database wizard could enlighten you with SQL magic to solve this problem. But, not being such a wizard, I simply broke the query into 3 separate queries as shown below: @breeds = Breed.find_by_lookup(lookup) @pictures = Picture.find(:all, :conditions => "breed_id = #{@breed.id}") @videos = Video.find(:all, :conditions => "breed_id = #{@breed.id}") Voila! Each of the 3 queries executes in less than a second!

Note: although it's not mentioned above, I also added appropriate indexes to each of the database tables to speed up the queries. These did not help much, but are always a good practice.

Problems w/ Zlib, Ruby, Gems, Rails?

If you experience problems with zlib and rubygems you are not alone. With practically every Rails installation that I’ve done, I have encountered some sort of problem with zlib and rubygems. So, I wanted to highlight my latest installation woes in case anyone else out there could benefit from the steps that I took. Here we go…

I’m starting with a hefty 2 x dual core AMD 64 Opteron machine and this is what I do. - I grab the latest Ruby source (1.8.5). - I’m also a big fan of Yum, the nifty package manager that makes installation and maintenance of Linux machines a breeze (well, not always), so I use Yum to grab needed developer packages like gcc, gcc-cpp, zlib, and the corresponding “-devel” packages. - I configure, make, make install Ruby - I grab the latest RubyGems source (0.9.2) and install it - I then attempt to install Rails with dependencies (gem install rails—include-dependencies), but I get the following error: gzip error installing rake-0.7.2.gem - After searching on the web and finding nothing helpful, I decide to try removing the current ruby gems installation and using rubygems 0.9.0 - Unfortunately, this just causes another error related to zlib: gzip error installing sources-0.0.1.gem - Finally, here’s what fixed the problem: 1) uninstall ruby (recursively delete the main ruby folder) 2) install zlib 1.2.3 from source 3) re-install Ruby 4) re-install RubyGems

Voila—it works! So, the moral of the story is that sometimes the Yum package manager doesn’t install zlib properly and you may need to install from the source.

Speed up your Ruby Code

Ever wish you could embed C code directly in Ruby? Me neither, but sometimes you have to take one for the team in order to speed up your code. RubyInline allows you to do exactly that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

class MyTest

  def factorial(n)
    f = 1
    n.downto(2) { |x| f *= x }
    f
  end

  inline do |builder|
    builder.c "
    long factorial_c(int max) {
      int i=max, result=1;
      while (i >= 2) { result *= i--; }
      return result;
    }"
  end
end

For a full guide on optimizing Ruby code including profiling, benchmarking, and using RubyInline, please check out this tutorial

A New Way to Scrape

I’m a big fan of Ruby Mechanize and the built-in Regular Expressions—it makes screen scraping so much easier than it could be. Even so, I’m pleasantly surprised to see an innovative approach taken by a project called ARIEL – A Ruby Information Extraction Language.

According to the project home page:

That sure beats developing, testing, and re-testing complex regular expressions!

The process involves… 1) Creating the structure that you’d like it to create (example below):

1
2
3
4
5
6
7
8
9
10
11

structure = Ariel::Node::Structure.new do |r|
  r.item :title
  r.item :body
  r.list :comments do |c|
    c.list_item :comment do |d|
      d.item :author
      d.item :body
    end
  end
end

2) Find some example data and label each example with tags such as , , etc. in the relevant places.

1
2

Ariel.learn structure, labeled_file1, labeled_file2, labeled_file3

4) Begin extracting your data!

1
2

extractions = Ariel.extract structure, unlabeled_file1, unlabeled_file2

The full article is here

RadiantCMS - Best Ruby Content Management System?

Even the most dynamic websites usually have quite a few pages of content. Sometimes a small shop with an all-technical staff can get away with editing files directly, but most companies and organizations find themselves in need of software to make editing and managing their online content—a Content Management System (CMS). While hundreds of open source CMSs exist, finding one for Ruby is another story. Before discovering Radiant, I had taken a peek at other wikis and CMSs out there, but most seemed too early in development, or too complex for a simple site. Radiant struck a good balance between keeping things simple and doing things right, and I’m happy to say that I had an opportunity to recently take Radiant on a test run with a PAID project.

Why Radiant?

Even the simple sites require some sort of additional functionality. Contact forms requiring sending email come to mind. Adding additional sections to an admin. Some sort of custom widget that gets embedded in your layout. So, it seems that no matter what CMS you use, you can’t escape the fact that you will need to extend it. Quite simply, Radiant has the features that you need, but not the bloat that gets in the way of adding on to it. In fact, Radiant is just a Rails app, so you needn’t fear making changes. It also includes a tagging library called Radius, which allows special functionality to be inserted easily. And, for more heavy work, you can create Behaviors, which have their own Radius tags and other special features.

Getting Started

To use Radiant, simply install using a gem if you prefer:

gem install --include-dependencies radiant

The gem hides much of the Rails code in the gem itself, so I recommend downloading Radiant from the website if you need to make customizations. It simply unzips as any other Rails webapp and you can begin editing the models, controllers, and views as needed. I ended up switching from the gem install to the file install once I realized that Radiant doesn’t include a WYSIWYG editing component. Not to worry though, the simplicity of RadiantCMS allowed me to add WYSIWYG HTML editing to the site pages in about an hour. Not bad…

I won’t go over all of the installation because it can be found right here: Radiant Installation

Creating Content

After you get things rolling, you’ll want to create one or more layouts. I found the fact that layouts can be composed of snippets, page parts, and tags to allow me to create multiple layouts with very little repetition of code. Here’s an example layout:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
    <meta http-equiv="Content-Type" content="text/html">
<title><r:title /></title>
<link href="/stylesheets/styles.css" media="all" rel="Stylesheet" type="text/css" />
<script src="/javascripts/jquery.js" language="javascript"></script>
</head>

<body>
<r:snippet name="header" />

<table id="table-secondary" width="800" cellspacing="0" cellpadding="0">
<tr>
<td valign="top">
<r:content part="left-nav" inherit="true" />
</td>
<td valign="top">
<div id="secondary-text">

<r:content />

</div>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>
<r:snippet name="footer-nav" />
<br />
<br />
</td>
</tr></table>
</div>
</body>
</html>

After that, I began adding pages in Radiant’s nice hierarchichal structure. Child pages can inherit Behaviors, layouts, and page parts from their parent page, which makes it easy to enforce a consistent look and functionality across various sections of a site.

Well, I hope this article has encouraged you to check out this wonderful CMS. I know that I will be using it for many future websites.