System Metrics is not intended to be a replacement for performance monitoring solutions such as New Relic. However, it is especially handy for quickly identifying performance problems in a development environment. It's also a great alternative for private networks disconnected from the Internet.

You can find more information about System Metrics on the System Metrics site. Please kick the tires and let us know what you think.

Generating the Lucene Index

public class IndexBuilder {

    public static void main(String[] args) throws Exception {
        String namesFile = "names.csv";
        Document doc = new Document();
        Field[] fields = new Field[]{new Field("firstName", "", Field.Store.YES, Field.Index.ANALYZED_NO_NORMS),
                new Field("lastName", "", Field.Store.YES, Field.Index.ANALYZED_NO_NORMS),
                new Field("address", "", Field.Store.YES, Field.Index.ANALYZED_NO_NORMS),
                new Field("email", "", Field.Store.YES, Field.Index.ANALYZED_NO_NORMS)};
        addFieldsToDocument(doc, fields);

        BufferedReader reader = new BufferedReader(new FileReader(namesFile));

        IndexWriter indexWriter = new IndexWriter(FSDirectory.open(new File("blog-index")),new IndexWriterConfig(Version.LUCENE_31, new StandardAnalyzer(Version.LUCENE_31)));

        String line;
        while ((line = reader.readLine()) != null) {
            String[] personData = getPersonData(line);
            setFieldData(personData, fields);
            indexWriter.addDocument(doc);
        }
        indexWriter.optimize();
        indexWriter.close();
    }

    private static String[] getPersonData(String line) {
        return line.split(",");
    }

    private static void setFieldData(String[] data, Field[] fields) {
        int index = 0;
        for (Field field : fields) {
            field.setValue(data[index++]);
        }
    }

    private static void addFieldsToDocument(Document doc, Field[] fields) {
        for (Field field : fields) {
            doc.add(field);
        }
    }
}

Creating the .thrift File

//all generated java code will have the following for package name
namespace java bbejeck.thrift.gen

//this is the person object 
struct Person {
  1: string firstName,
  2: string lastName,
  3: string address,
  4: string email
}

//exception used to send meaningful error messages back to user
exception LuceneSearchException {
  1: string message
}

//service definition used by client and server
service LuceneSearch { 
    list<Person> search(1: string query) throws (1:LuceneSearchException error) 
}

• $ thrift --gen java lucene_search.thrift
• $ thrift --gen rb lucene_search.thrift

Thrift Server - Java

public class LuceneThriftServer {
    private static final int PORT = 9090;
    private static int numberThreads = 5;

    public static void main(String[] args) throws Exception {
        TServerSocket serverSocket = new TServerSocket(PORT, 100000);
        LuceneSearch.Processor searchProcessor = new LuceneSearch.Processor(new SearchHandler(args[0]));
        if (args.length > 1) {
            numberThreads = Integer.parseInt(args[1]);
        }
        TThreadPoolServer.Args serverArgs = new TThreadPoolServer.Args(serverSocket);
        serverArgs.maxWorkerThreads(numberThreads);
        TServer thriftServer = new TThreadPoolServer(serverArgs.processor(searchProcessor).protocolFactory(new TBinaryProtocol.Factory()));
        thriftServer.serve();
    }

Iface Implementation

public class SearchHandler implements LuceneSearch.Iface {
    private IndexSearcher searcher;
    private QueryParser queryParser;
    private static final int MAX_RESULTS = 1000;

    public SearchHandler(String indexPath) {
        try {
            searcher = new IndexSearcher(FSDirectory.open(new File(indexPath)), true);
            queryParser = new QueryParser(Version.LUCENE_31, null, new StandardAnalyzer(Version.LUCENE_31));
            queryParser.setAllowLeadingWildcard(true);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    public List<Person> search(String query) throws LuceneSearchException {
        List<Person> results = new ArrayList<Person>();
        try {
            Query q = queryParser.parse(query);
            TopDocs topDocs = searcher.search(q, MAX_RESULTS);
            for (ScoreDoc sd : topDocs.scoreDocs) {
                Document document = searcher.doc(sd.doc);
                results.add(getPersonFromDocument(document));
            }
        } catch (Exception e) {
            throw new LuceneSearchException(e.getMessage());
        }
        return results;
    }

    private Person getPersonFromDocument(Document document) {
        Person p = new Person();
        p.firstName = document.get("firstName");
        p.lastName = document.get("lastName");
        p.address = document.get("address");
        p.email = document.get("email");

        return p;
    }
}

Thrift Client - Ruby

module ThriftConnection

  class LuceneClient

    def initialize(host='localhost', port=9090)
      socket = Thrift::Socket.new(host, port)
      @transport = Thrift::BufferedTransport.new(socket)
      protocol_factory = ::Thrift::BinaryProtocolFactory.new
      protocol = protocol_factory.get_protocol(@transport)
      @transport.open
      @client = LuceneSearch::Client.new(protocol)
    end

    def search(query)
      @client.search(query)
    end

    def close
      @transport.close
    end

  end
end

Running With Scissors

#added to translate from symbol to expected search format
  SEARCH_KEYS_MAPPING = {:first_name => 'firstName',
                                            :last_name => 'lastName',
                                            :email => 'email',
                                            :address => 'address'}


  def self.method_missing(method_name, *args)
    lucene_client = ThriftConnection::LuceneClient.new
    query = ""
        #handles find_by_first_name etc
    if method_name.to_s =~ /^find_by_([a-z]+_?[a-z]*)$/
      query = "#{SEARCH_KEYS_MAPPING[$1.to_sym]}:#{args[0]}"
        #handles find_by_first_name_or_last_name, find_by_first_name_and_email 
    elsif method_name.to_s =~/^find_by_([a-z]+_[a-z]+)_([a-z]+)_([a-z]+_?[a-z]*)$/
      query ="#{SEARCH_KEYS_MAPPING[$1.to_sym]}:#{args[0]} #{$2.upcase} #{SEARCH_KEYS_MAPPING[$3.to_sym]}:#{args[1]}"
    else
       raise ArgumentError.new("search method pattern #{method_name} not recognized")
    end

    results = lucene_client.search(query)
    lucene_client.close
    results
  end

Testing

head names.csv

cat names.csv | grep -iE 'elizabeth|krause' | wc -l 

class SearchTest < Test::Unit::TestCase

  def setup
    @lucene_client = ThriftConnection::LuceneClient.new
  end


  def teardown
    @lucene_client.close
  end

  def test_search_client_first_name
    persons = @lucene_client.search("firstName:Tia")
    assert_equal(5, persons.length)

    persons.each do |person|
      assert_equal("Tia", person.firstName)
    end
  end

  def test_search_person_class_first_name
    persons = Person.find_by_first_name("Tia")
    assert_equal(5, persons.length)

    persons.each do |person|
      assert_equal("Tia", person.firstName)
    end
  end

  def test_search_client_first_name_email_domain
    persons = @lucene_client.search("+firstName:Elizabeth +email:*pookmail.com")
    assert_equal(59, persons.length)
  end

  def test_search_person_class_first_name_email_domain
    persons = Person.find_by_first_name_and_email("elizabeth", "*pookmail.com")
    assert_equal(59, persons.length)
  end

  def test_search_client_first_name_and_last_name
    persons = @lucene_client.search("+firstName:Elizabeth +lastName:Krause")
    assert_equal(1, persons.length)
    person = persons[0]

    assert_equal("Elizabeth", person.firstName)
    assert_equal("Krause", person.lastName)
  end

  def test_search_person_class_first_name_and_last_name
    persons = Person.find_by_first_name_and_last_name("elizabeth", "krause")
    assert_equal(1, persons.length)
    person = persons[0]

    assert_equal("Elizabeth", person.firstName)
    assert_equal("Krause", person.lastName)
  end

  def test_search_person_class_first_name_or_last_name
    persons = Person.find_by_first_name_or_last_name("elizabeth", "krause")
    assert_equal(289, persons.length)
  end

  def test_invalid_search
    assert_raises ArgumentError do
      Person.find_person_by_name("tia")
    end
  end

end

Conclusion

Resources

• For more information on thrift the thrift wiki is a great start
• More information on Lucene can be found here

The model has a simple one-to-many relationship (parent/children)

{parent = { id: 1, name: 'Jim', children: 
[{id: 2, name: 'Larry',parent_id:1},
{id: 3, name:'Curly',parent_id:1},
{id:4, name: 'Moe',parent_id:1}] } 

class Parent < ActiveRecord::Base
  attr_accessible :name
  has_many :children 
end

class Child < ActiveRecord::Base
  belongs_to :parent
end

My expectation was that when I save the parent data, via update_attributes, both the parent and child data save together in a single transaction.

class ParentsController< ApplicationController
  ...
  def update
    parent = Parent.find(params[:id])
    parent.update_attributes(params)
    ...
  end
end

Being new to developing Rails apps in the "real world" I had not dealt with this, what I thought to be, common scenario. I found that out of the box the parent data saves, but not the child data. I first explored the book "Agile Web Development with Rails", but didn't find what I needed. Next I checked out the Rails Guides, in particular the section on associations The first setting I found was the :autosave setting:

If you set the :autosave option to true, Rails will save any loaded members and destroy members that are marked for destruction whenever you save the parent object.

Seemed like a good fit.

has_many :children, :autosave => true

However, the children still did not save when calling update_attributes (I'm still not sure exactly what this is used for)

Next stop, Google. Some searching brought me to the ActiveRecord "accepts_nested_attributes_for" method. It turns out this is exactly what I needed. So I added "accepts_nested_attributes_for :children" to Parent

class Parent < ActiveRecord::Base
  attr_accessible :name
  has_many :children
  accepts_nested_attributes_for :children
end

However, just adding "accepts_nested_attributes_for :children" wasn't enough. It turns out that with the accepts_nested_attributes_for method, the Parent class receives a method called "children_attributes=(attributes)". This allows for the parent's children attributes to be updated. This also means I have to update my JSON so that the "children" property is called "children_attributes".

parent = { id: 1, name: 'Jim', children_attributes: 
[{id: 2, name: 'Larry',parent_id:1},
{id: 3, name:'Curly',parent_id:1},
{id:4, name: 'Moe',parent_id:1}] } 

Since I am using "attr_accessible" I also have to include "children_attributes" in the list

class Parent < ActiveRecord::Base
  attr_accessible :name, :children_attributes
  has_many :children
  accepts_nested_attributes_for :children
end

Now calling "parent.update_attributes" saves both the parent and its children, within a single transaction. Rails will handle inserts/updates/deletes appropriately. If the child has an id, the child is updated. If the child is missing an id, a new child is added. If the child has the "_destroy" attribute set, the child is removed.

The last todo was to serialize the JSON back to the client upon success. When trying this:

render :json => parent.to_json(:include => :children)

my JSON included "children", not "children_attributes":

{parent = { id: 1, name: 'Jim', children: 
[{id: 2, name: 'Larry',parent_id:1},
{id: 3, name:'Curly',parent_id:1},
{id:4, name: 'Moe',parent_id:1}] } 

For the save to work correctly, the JSON needs "children_attributes", (which is a bit of a disconnect between relationship and nested attribute naming conventions) I was not able to figure out how to get Rails and "to_json" to use "children_attributes". For my solution I did not ":include => :children" in my call to "to_json" and I overrode "as_json". In my "as_json" I manually add the "children attributes" to the parent JSON.

  def as_json(options={})
    json = super(options)
    json[:children_attributes]=[]
    
    self.children.each do |child|
      json[:children_attributes] << {:id => child.id, :name => child.name}
    end
    json
  end

Looking back on getting the save to work, it seems rather simple. However, several things made this a little harder than it should have been. The first being the documented coverage. Surprisingly, this was not covered in the book "AWDR" or Rails Guides. It is spelled out quite well in the API docs, however, but you need to know about the NestedAttributes module The ":autosave" setting misled me a bit. I will need to go back and do a bit more research on this. Lastly, the naming convention for nested attributes threw me off and the fact that I had to "rig" as_json. Perhaps there is a way to handle this by "convention".

For further reading check out Ryan's blog or the Nested Attributes API docs.

There were several requirements that I had to keep in mind when I was designing this

1. The cap tasks should use the environment configuration files that we already have.
2. Because there are a large number of tasks we should be able to run them individually, or with a single command we should be able to run the entire suite.
3. The cap script should produce well formatted, easy to read output, so that it's clear what's broken and where.

Control Support Methods

While building the validation tasks I found that I was having to do the same basic operations fairly often:

1. Issue a command on a remote host.
2. Parse the output of that command.
3. Issue more commands based on the output.

Or sometimes I just wanted to run a series of commands on one host, verifying that it was correct before moving on to the next. It's possible to make Capistrano work this way, but it's not well documented. By default, if you give Capistrano multiple commands to run on multiple hosts, the it will run the commands in host order. However what I really wanted is for Capistrano to run the commands in command order.

For instance: given that we have host-A, host-B, host-C, and I want to run commands doA, doB, and doC by default Capistrano will run:

    HostA > doA
    HostB > doA
    HostC > doA
    HostA > doB
    HostB > doB
    HostC > doB
    HostA > doC
    HostB > doC
    HostC > doC

But what I needed it to do was:

    HostA > doA
    HostA > doB
    HostA > doC
    HostB > doA
    HostB > doB
    HostB > doC
    HostC > doA
    HostC > doB
    HostC > doC

By using the roles that we already have configured for our deploy tasks with Capistrano we were able to create a few support methods that issue commands in the order that we want.

each_host

The each_host method is used to iterate through our configured hosts. The method prints out the hostname (which is important for the look of the script's output) sets the current host, then yields the block that was passed into it.

  def each_host
    roles[:web].each do |host|
      print_hostname host
      set(:current_host, host.host)
      yield host
    end
  end

run_serial

The run_serial method runs the given command, but only on the current host, yielding the output of the command and the name of the output stream (:out or :err). The ssh channel we don't have much use for.

  def run_serial(command)
    run command, :hosts => fetch(:current_host) do |chan, stream, data|
      yield stream, data
    end
  end

run_primary

We also have a primary server set in our configuration. We can create a few more support methods that take advantage of the primary host, for issuing command on one host only.

  def run_primary(command)
    run command, :hosts => fetch(:primary) do |chan, stream, data|
      set_current_host channel
      yield stream, data
    end
  end

currently_primary?

Having a method that tells us if we're currently on the primary host is helpful as well.

  def currently_primary?
    fetch(:current_host) == fetch(:primary).host
  end

set_current_host

The current host can also be set from the ssh channel. This method is called from the run_primary method, but this needs to be called when we call the default run method.

  def set_current_host(channel)
    host = channel.properties[:host]
    print_hostname host
    set(:current_host, host)
  end

Now we can write tasks that execute in in command order, execute tasks just on the primary host, and as we're preforming checks we print out the current host, so we're able to keep track of where we are.

Printing Support Methods

Because printing the script output out to the command line is an important part of the script I'm going to go over the printing support next. When running the server checks I wanted the output to look like rspec or cucumber. As each check is performed I wanted to see a colorful pass or fail message with the values that I was checking against to be highlighted. Also once the checks were finished I wanted to see a list of everything that failed, grouped by host.

print_hostname

This is the print_hostname method that was mentioned above. Everytime we move to a different host this method prints out the hostname.

  def print_hostname(name)
    puts "    Host: #{grey(name)} >"
  end

step

Also at the beginning of every task I wanted to print the task name and the number of the task.

  def step(name)
    step = fetch(:step)
    puts "\n[#{step}] #{name}"
    set(:step, step+1)
  end

failure

The failure method is called when a check fails. It prints the failure message, and stores it under the current hostname so that it can be printed again after all the checks have been done. If the expected or actual parameters are included it prints those along with the message, otherwise just the message is printed.

  def failure(message, expected=nil, actual=nil)
    host = fetch(:current_host)
    complete = (expected || actual) ?
      "#{message} Expected: #{grey(expected)} Actual: #{grey(actual)}" :
      message
    errors = fetch(:errors)
    unless errors[host]
      errors[host] = []
    end
    errors[host] << message
    log_item(complete, :fail)
  end

pass

The pass method is called when a check passes.

  def pass(message)
    log_item(message, :pass)
  end

log_item

The log_item method is called by the pass and failure methods. The method includes a warn and empty status which can also be used by the tasks when printing their status.

  def log_item(message, flag=nil)
    status = case flag
      when :pass : "      [#{green("PASS")}] "
      when :fail : "      [#{red("FAIL")}] "
      when :warn : "      [#{orange("WARN")}] "
      else "      "
    end
    puts "#{status}#{message}"
  end

colorize

The methods below handle the coloring of the text as it's printed to the command line. This was actually pretty fun figuring out. Every script should have fancy colorized output. (and now every script of mine will, muaha ha ha)

  def red(text)
    colorize(text, 31)
  end

  def orange(text)
    colorize(text, 33)
  end

  def green(text)
    colorize(text, 32)
  end

  def grey(text)
    colorize(text, 37)
  end

  def colorize(text, color_code)
    "\e[#{color_code}m#{text}\e[0m"
  end

Start and Finish Tasks

Now that all that's taken care of we can start writing some actual Capistrano tasks to take advantage of our fine grained control and fancy printing.

  TASK_LIST = [
    "check:all",
    "check:environment_variables",
    "check:middleware_versions",
    "check:apache_configuration"]

  on :start, 'check:setup', :only => TASK_LIST
  on :finish, 'check:print_errors', :only => TASK_LIST

setup

The setup task handles whatever specific setup needs to be done, but at the very least the step and errors variables need to be defined. They're used by the printing methods. Also, all of the tasks below are inside of the check namespace.

  namespace :check do

    task :setup do
      set :step, 1
      set :errors, {}
    end

print_errors

The print errors method is run after all the checks have been run to print a convenient list of errors grouped by host. This keeps errors from being lost in the output.

  task :print_errors do
    errors = fetch(:errors)
    if (errors.size() > 0)
      print "\n ==== #{red 'Errors'} ===="
      errors.each do |host,list|
        print_hostname host
        list.each {|e| puts "      #{e}"}
      end
    end
  end

Validation Tasks

The meat of the script are in the validation tasks. Out current script has fifteen different tasks that do everything from checking environment variables, to verifying directory and file permissions, to checking network and database statuses. If it can be automated, we'll find a way to get it in. Rather then include concrete examples though I'm just going to include some skeleton methods to show the method structure to illustrate how the support methods are used together in the Capistrano tasks.

Simple Tasks

This task executes one command on each server and validates the output from a list of expected results. This form is used by our script to check environment variables, commands on the sudo list, and programs in the cron, all of which can be read and verified with one command. I'm just including the validateVariable, and getVariable methods here to show the pass and failure methods.

  VARIABLES = [
    { :key => 'KEY', :value => /Expected Value/ },
    { :key => 'KEY', :value => /Expected Value/ },
    { :key => 'KEY', :value => /Expected Value/ }]

  task :simple do
    step "A Simple Check"
    run "env" do |channel, stream, data|
      set_current_host channel
      VARIABLES.each do |map|
        validateVariable(data, map)
      end
    end
  end

  def validateVariable(data, map)
    value = getVariable(data, map[:key])
    (value.match map[:value]) ?
      pass("Env #{grey(map[:key])} is set to #{grey(value)}") :
      failure("Env #{grey(map[:key])} is incorrect", map[:value].inspect, value)
  end

  def getVariable(data, key)
    result = data.match /^#{key}=(.*)/
    unless result
      failure "Env #{grey(key)} is not set."
      return nil
    end
    result[1]
  end

Tasks that run multiple commands on the same host

Most of the tasks in our validation script run multiple commands on the same host, using the each_host and run_serial methods. This allows us to read files and act on the values in those files. It's alse used because the script output looks better organized when running tasks in host order. The task below itterates through a list of standard directories, then a list of configured directories. For each directory a run_serial command is executed to see if the directory exists, then once the lists have been gone through it's done again on the next host.

  task :serial_example do
    step "Run multiple commands on a host"
    each_host do
      DIRECTORIES.each do |dir|
        verify_directory dir
      end

      [:deploy_to, :transfer_path, :cache_path].each do |key|
        dir = fetch(key)
        if (dir)
          verify_directory dir
        else
          falure "No directory configured for #{grey(key)}"
        end
      end

    end
  end

  def verify_directory(path)
    command = "[ -d #{path}] && echo 'true' || echo 'false'"
    run_serial command do |stream, data|
      if (data.strip == 'true')
        pass "Verified that #{grey(path)} exists."
      else
        failure "Directory at #{grey(path)} does not exist."
      end
    end
  end

Tasks using different server combinations

Here's a command that runs on the primary host, reads a file, then acts across multiple hosts. We do something like this to verify that the Apache configuration is correct. We know the httpd.conf is the same across hosts, but we want to verify that the files and directories that are in the configuration are actually on the host in question.

  task :primary_example do
    step "Verifying Certificates"
    path = fetch(:path_to_httpd_conf)
      certs = {}
      run_primary "grep SSLC[eA] #{path}" do |stream,data|

        if (stream == :err)
          failure "No http configuration file at #{grey(path)}"
          break
        end

        keys = [
          'SSLCertificateFile',
          'SSLCertificateKeyFile',
          'SSLCACertificatePath',
          'SSLCARevocationPath']

        keys.each do |key|
          certs[key] = read_http_conf_value(data,key)
        end

        pass "Read Certificate Paths from HTTP Configuration"
      end

      each_host do
        verify_file certs['SSLCertificateFile']
        verify_file certs['SSLCertificateKeyFile']
        verify_directory certs['SSLCACertificatePath']
        verify_directory certs['SSLCARevocationPath']
      end
    end

And finally we need a task that runs the whole suite of validation tasks. The all task does nothing itself, it just runs all of the other tasks.

  desc "Run all of the validation tasks"
  after "check:all",
    "check:environment_variables",
    "check:middleware_versions",
    "check:apache_configuration"

  task :all do
  end

Then to execute the cap task you would type is:

cap -q production check:all

And watch as all the beautiful passes and fails scroll by, though hopefully more of the former.

RJava lets you write code in Java and run it on a Ruby VM! And you still get the full benefit of the Java compiler to ensure your code is 100% correct. Of course with Java you also get checked exceptions and proper interfaces and abstract classes to ensure compliance with your design. You no longer need to worry about whether an object responds to a random message, because the Java compiler will enforce that it does.

You get all this and more but on the power and flexibility of a Ruby VM. And because Java does not support closures, you are ensured that everything is properly designed since you'll be able to define interfaces and then implement anonymous inner classes just like you're used to doing! Even when JDK 8 arrives sometime in the future with lambdas, you can rest assured that they will be statically typed.

As a first example, let's see how you could filter a collection in RJava to find only the even numbers from one to ten. In Ruby you'd probably write something like this:

evens = (1..10).find_all { |n| n % 2 == 0 }

With RJava, you'd write this:

List<Integer> evens = new ArrayList<Integer>();
for (int i = 1; i <= 10; i++) {
  if (i % 2 == 0) {
    evens.add(i);
  }
}

This example shows the benefits of declaring variables with specific types, how you can use interfaces (e.g. List in the example) when declaring variables, and shows how you also get the benefits of Java generics to ensure your collections are always type-safe. Without any doubt you know that "evens" is a List containing Integers and that "i" is an int, so you can sleep soundly knowing your code is correct. You can also see Java's powerful "for" loop at work here, to easily traverse from 1 to 10, inclusive. Finally, you saw how to effectively use Java's braces to organize code to clearly show blocks, and semi-colons ensure you always know where lines terminate.

I've just released RJava on GitHub, so go check it out. Please download RJava today and give it a try and let me know what you think!

<?mso-application progid="Word.Document"?>
<w:wordDocument xml:space="preserve" w:embeddedObjPresent="no">
<o:DocumentProperties>...</o:DocumentProperties>
<o:CustomDocumentProperties>...</o:CustomDocumentProperties>
<w:fonts>...</w:fonts>
<w:lists>...</w:lists>
<w:styles>...</w:styles>
<w:docPr>...</w:docPr>
<w:body>{table}</w:body>
</w:wordDocument>

        <w:tbl>
           <w:tblPr>
               <w:tblStyle w:val="Table1"/>
               <w:tblW w:w="9973.2467" w:type="dxa"/>
               <w:tblInd w:w="0" w:type="auto"/>
               <w:jc w:val="left"/>
           </w:tblPr>
           <w:tblGrid>
               <w:gridCol w:w="3324.3676"/>
               <w:gridCol w:w="3324.3676"/>
               <w:gridCol w:w="3324.3676"/>
           </w:tblGrid>
           <w:tr>
               <w:trPr/>
               <w:tc>
                   <w:tcPr>
                       <w:tcW w:type="dxa" w:w="3324.3676"/>
                       <w:tcMar/>
                       <w:tcBorders>
                           <w:top w:val="single" w:sz="0" w:color="000000"/>
                           <w:bottom w:val="single" w:sz="0" w:color="000000"/>
                           <w:left w:val="single" w:sz="0" w:color="000000"/>
                           <w:right w:val="none" w:sz="0" w:color="auto"/>
                       </w:tcBorders>
                   </w:tcPr>
                   <w:p>
                       <w:pPr>
                           <w:pStyle w:val="Table_20_Contents"/>
                       </w:pPr>
                       <w:r>
                           <w:t>Header 1</w:t>
                       </w:r>
                   </w:p>
               </w:tc>
               <w:tc>
                   <w:tcPr>
                       <w:tcW w:type="dxa" w:w="3324.3676"/>
                       <w:tcMar/>
                       <w:tcBorders>
                           <w:top w:val="single" w:sz="0" w:color="000000"/>
                           <w:bottom w:val="single" w:sz="0" w:color="000000"/>
                           <w:left w:val="single" w:sz="0" w:color="000000"/>
                           <w:right w:val="none" w:sz="0" w:color="auto"/>
                       </w:tcBorders>
                   </w:tcPr>
                   <w:p>
                       <w:pPr>
                           <w:pStyle w:val="Table_20_Contents"/>
                       </w:pPr>
                       <w:r>
                           <w:t>Header 2</w:t>
                       </w:r>
                   </w:p>
               </w:tc>
               #another w:tc defined here for Header 3...
           </w:tr>
        {rows}
        </w:tblPr>
        <w:p>
            <w:pPr>
                <w:pStyle w:val="Standard"/>
            </w:pPr>
        </w:p>
        <w:sectPr>
            <w:type w:val="next-page"/>
            <w:pgSz w:w="12241.5302" w:h="15841.9803" w:orient="portrait"/>
            <w:pgMar w:top="1133.9978" w:bottom="1133.9978" w:left="1133.9978" w:gutter="0" w:right="1133.9978"/>
            <w:pgBorders w:offset-from="text"/>
        </w:sectPr>

            <w:tr>
                <w:trPr/>
                <w:tc>
                    <w:tcPr>
                        <w:tcW w:type="dxa" w:w="3324.3676"/>
                        <w:tcMar/>
                        <w:tcBorders>
                            <w:top w:val="none" w:sz="0" w:color="auto"/>
                            <w:bottom w:val="single" w:sz="0" w:color="000000"/>
                            <w:left w:val="single" w:sz="0" w:color="000000"/>
                            <w:right w:val="none" w:sz="0" w:color="auto"/>
                        </w:tcBorders>
                    </w:tcPr>
                    <w:p>
                        <w:pPr>
                            <w:pStyle w:val="Table_20_Contents"/>
                        </w:pPr>
                        <w:r>
                            <w:t>{column_1_value}</w:t>
                        </w:r>
                    </w:p>
                </w:tc>
                <w:tc>
                    <w:tcPr>
                        <w:tcW w:type="dxa" w:w="3324.3676"/>
                        <w:tcMar/>
                        <w:tcBorders>
                            <w:top w:val="none" w:sz="0" w:color="auto"/>
                            <w:bottom w:val="single" w:sz="0" w:color="000000"/>
                            <w:left w:val="single" w:sz="0" w:color="000000"/>
                            <w:right w:val="none" w:sz="0" w:color="auto"/>
                        </w:tcBorders>
                    </w:tcPr>
                    <w:p>
                        <w:pPr>
                            <w:pStyle w:val="Table_20_Contents"/>
                        </w:pPr>
                        <w:r>
                            <w:t>{column_2_value}</w:t>
                        </w:r>
                    </w:p>
                </w:tc>
                #another w:tc defined with {column_3_value}...
            </w:tr>
        </w:tbl>

#create a new empty xml document
document = File.new(some_filename, 'a')

#query for data
table_data = SomeObject.find(:all, :conditions => 'some conditions...')

document_template = File.open('path/to/document-template.xml').read

table_template = File.open('path/to/table-template.xml').read
rows = []
table_data.each do |row_data|
    row_template = File.open('path/to/row-template.xml').read
    row_template.gsub!('{column_1_value}', row_data.col_1) #substitute in whatever returns the data you need to put in the Col 1 column.
    row_template.gsub!('{column_2_value}', row_data.col_2) #substitute in whatever returns the data you need to put in the Col 2 column.
    row_template.gsub!('{column_3_value}', row_data.col_3) #substitute in whatever returns the data you need to put in the Col 3 column.
    
    #add the row to the master array of rows
    rows << row_template
end

#substitute the dynamically generated rows into the table template
table_template.gsub!('{rows}', rows.join(''))

#substitute the dynamically generated table into the document template
document_template.gsub!('{table}', table)

#write everything to the new xml document.  As I said before you'll probably want to write to the xml document more frequently than 
#this example to avoid Out-Of-Memory errors.
document.puts document_template
document.close

var prepareExport = function(gridStore, gridColumnModel) {
    var sort = null;
    var direction = null;
    if (gridStore.getSortState()) {
        sort = gridStore.getSortState().field();
        direction = gridStore.getSortState().direction;
    }
    
    var columnOrder = []; //ordered list of columns reflecting the GridPanel's state
    var columnsToTitles = {}; //map of ext field mappings to column titles
    var columnsToWidths = {}; //width of each column
    var totalWidth = 0;
    var groupByField = gridStore.groupField; //the store's current grouping field ('false' if not grouped)
    
    var storeReaderMapping = gridStore.fields.items;
    
    //check to see if the groupField has a corresponding mapping
    for (var i = 0; i < gridReaderMapping.length; i++) {
        if (gridReaderMapping[i].name == gridStore.groupField && gridReaderMapping[i].mapping != null) {
            groupByField = gridReaderMapping[i].mapping;
            break;
        }
    }
    
    //get the total width of the displayed columns (this will be used later when generating the Word XML)
    gridColumnModel.getColumnsBy(function(c) {
        if (!c.hidden) {
            totalWidth = totalWidth + c.width;
        }
    });
    
    //iterate over the grid's column model.  For displayed columns, get the ext field names (mappings) and column titles
    gridColumnModel.getColumnsBy(function(c) {
        if (!c.hidden) {
            for (var i = 0; i < gridReaderMapping.length; i++) {
                if (gridReaderMapping[i].name == c.dataIndex && gridReaderMapping[i].mapping == null) {
                    columnOrder.push(gridReaderMapping[i].name);
                    var key = gridReaderMapping[i].name;
                    columnsToTitles[key] = c.header;
                    columnsToWidths[key] = c.width/totalWidth;
                    break;
                }
                else if (gridReaderMapping[i].name == c.dataIndex && gridReaderMapping[i].mapping != null) {
                    //if the name and mapping are different (i.e. a mapping exists), we need to push the mapping and not the name
                    columnOrder.push(gridReaderMapping[i].mapping);
                    var key = gridReaderMapping[i].mapping;
                    columnsToTitles[key] = c.header;
                    columnsToWidths[key] = c.width/totalWidth;
                    break;
                }
            }
        }
    });
    
    return [columnOrder, columnsToTitles, groupByField, sort, direction, columnsToWidths];
};

var gridReaderMapping = [
    {name: 'id'},
    {name: 'col_one'},
    {name: 'col_two', mapping: 'some_mapping'},
    {name: 'col_three'}
];

var gridColumnModel = new Ext.grid.ColumnModel({
    defaults: {sortable: true},
    columns: [
        {header: 'ID', dataIndex: 'id'},
        {header: 'Column One', dataIndex: 'col_one'},
        {header: 'Column Two', dataIndex: 'col_two'},
        {header: 'Column Three', dataIndex: 'col_three'}
    ]
});

var store = new Ext.data.GroupingStore({
    proxy: new Ext.dataHttpProxy({
        url: 'give/me/my/data'
        method: 'GET'
    }),
    reader: new Ext.data.JsonReader({
        root: 'data',
        totalProperty: 'total',
        messageProperty: 'message'
    }, gridReaderMapping)
});

var grid = new Ext.grid.GridPanel({
    id: 'gridpanel',
    ds: gridStore,
    cm: gridColumnModel,
    sm: new Ext.grid.RowSelectionModel({
        singleSelect: true
    }),
    tbar: [exportButton],
    view: etc, etc, etc, etc...
});

var exportButton = new Ext.Button({
    renderTo: this.wrap,
    listeners: {
        click: function (node, event) {
            var exportConfig = prepareExport(grid.getStore(), grid.getColumnModel());
            
            Ext.Ajax.request({
                url: 'url/to/query/for/data'
                params: {
                    'columnsToTitles': Ext.util.JSON.encode(exportConfig[1]),
                    'columnsToWidths': Ext.util.JSON.encode(exportConfig[5]),
                    'columnOrder[]': exportConfig[0],
                    'groupByExport': exportConfig[2],
                    'groupBy': gridStore.groupField,
                    'groupDir': 'ASC',
                    'sort': exportConfig[3],
                    'dir': exportConfig[4],
                    'export': true
                    'query': build_some_query_params_to_pass_to_the_controller
                },
                success: function(response, opts) {
                    window.location.href = 'path/to/exporter_controller/export_word_XML'
                },
                failure: function(response, opts) {
                    //output error message
                },
                scope: this
            }); 
        }
    }
});

class ExportController < ApplicationController
    def export_word_XML
        filename = "#{X_SENDFILEPATH}/word_doc.xml"
        if (File.exist?(filename))
            send_file filename, :x_sendfile => true
        else
            render :nothing => true
        end
    end
end

  1 def as_json(options={})
  2   options[:methods] ||= []
  3   options[:methods] << :full_name
  4   super(options)
  5 end

When this method was executed in order to convert the object into a JSON representation, I received this error (with line numbers adjusted for this example):

  NoMethodError: undefined method '[]' for nil:NilClass
    from MyModel:2:in 'as_json'

I stared at this for a while until it dawned on me that explicitly passing 'nil' to as_json might not result in options being set to the default empty hash, and indeed it does not. I suppose it's technically correct, since nil is an instance of NilClass in Ruby and not some special value as it is in many other languages. It's just not what I was expecting and not what I want. For all the books, blog posts, and tutorials I've read, you'd think I would have picked up on this somewhere along the way.

Curiously though, can anyone think of a valid use case for not assigning the default value to a nil argument? I'm sure there are some, but they're not coming to mind.

You downloaded and installed the latest 64-bit DMG from MySQL and now you’re getting “could not load driver” errors in DBI (Ruby or Perl) and Rails.

The Cause

In Ruby, the ‘mysql’ and ‘mysql-2’ gems were compiled against older version of the MySql library. Ditto DBD:Mysql module in Perl.

The Solution

Update 6/6/2011: Google lead me to a better solution:

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/

That should work for everything! But in case it doesn’t, this will fix Rails:

sudo install_name_tool -change libmysqlclient.16.dylib \ 
/usr/local/mysql/lib/libmysqlclient.16.dylib \
/usr/local/lib/ruby/gems/1.9.1/gems/mysql2-0.2.6/lib/mysql2/mysql2.bundle

And this will fix Ruby’s DBI:

sudo install_name_tool -change libmysqlclient.16.dylib  \ 
/usr/local/mysql/lib/libmysqlclient.16.dylib \
/usr/local/lib/ruby/gems/1.9.1/gems/mysql-2.8.1/lib/mysql_api.bundle

Note: you’ll need to substitute both the appropriate path to the gems, and the appropriate version of the MySQL library.

Fixing Perl’s DBI is a little trickier. You need to download the MySql DBD source and run the install_name_tool on the bundle before you install. Something like:

sudo install_name_tool -change libmysqlclient.16.dylib \ 
/usr/local/mysql/lib/libmysqlclient.16.dylib \
/path-to-my-sql-dbd-source-folder-with-the-bundle-file/mysql.bundle

class Photo < ActiveRecord::Base
  has_attached_file :image,
    :styles => { :medium => '350x350>', :thumb => '100x100#' },
    :url    => ':class/:id/:style.:extension',
    :path   => ':rails_root/assets/:class/:id_partition/:style.:extension'
end

I love how easy it is to specify an attachment, but I don't like having to specify the :url and :path on all attachments for two reasons:

1. It's not DRY. With the Paperclip interpolations I'm using, the :url and :path are never going to change within an environment no matter how many attachments I have across all my models.
2. Between environments I do need to change the path. In development I keep the attachments in :rails_root/assets but in production I want them somewhere else.

Fortunately, changing the Paperclip attachment defaults per environment is really easy. In your environment config files, do something like the following.

# development.rb (or use environment.rb to setup defaults for all)
Paperclip::Attachment.default_options[:url] = ':class/:id/:style.:extension'
Paperclip::Attachment.default_options[:path] = ':rails_root/assets/:class/:id_partition/:style.:extension'

# production.rb
Paperclip::Attachment.default_options[:url] = ':class/:id/:style.:extension'
Paperclip::Attachment.default_options[:path] = '/usr/local/assets/:class/:id_partition/:style.:extension'

Adding those defaults in your environment configuration files means you can keep your models DRY and free of any ugly code used to change options depending on the environment. Isn't this nicer?

class Photo < ActiveRecord::Base
  has_attached_file :image, 
    :styles => { :medium => '350x350>', :thumb => '100x100#' }
end

The best conferences aren't about learning at all. They're about inspiration. I finally realized this at RailsConf last week, despite having attended dozens of conferences during the past ten years. Surprisingly, I found most of the topics rather boring, most of the presenters unpolished, and most of the presentations poorly constructed and minimally rehearsed. The things I enjoyed most were the keynotes from Dave Thomas and David Heinemeier Hansson, a performance-related presentation by Aaron Patterson, an entrepreneurial story by Tom Preston-Werner, and a reflection talk by Keavy McMinn on her journey from artist to programmer. Why did I enjoy them? Because they caused me to look at things differently, forced me to think critically about my choices, and motivated me to act. Only one of these was technical in nature by the way, just one!

A good conference should inspire you to do something. No talk is long enough to teach you anything immediately useful. The best you can hope for is exposure to lots of stuff you might want to look into after the conference. Of course you could get the same level of exposure by reading the conference agenda, Googling each topic, and reading through some documentation on each homepage. What you can't get is the inspiration from a great speaker, motivational story, or engaged community.

So the next time you contemplate attending a conference, do your best to judge the inspiration potential. After all, if you're not going to do anything different after the conference, why bother going?

Our application has some services that it connects to that require secure connections so in order to comply and still develop we had modified the WEBrick configuration under 2.3.5 to make it run under SSL. This turned out to be very easy to accomplish with 2.3.5, we just cracked open the script/server file and added the appropriate SSL options that WEBrick already knows about.

Rails 3 proved to be a little bit more difficult to modify. This is due to the fact that the script folder only contains a rails script now and all of the old scripts are now contained deep in the rails build. I didn’t really want to crack open rails and start modifying (especially not when it is only for development), so I was looking for another option. I spent a few days searching the internet with no luck. Today I came into work and after 6+ hours and some pair programming a solution was figured out. You can find the solution that we came up with below, and if there is a more elegant solution out there I would love to hear about it.

All you have to do to get WEBrick running in SSL is modify the script/rails file and add the following lines (above the APP_PATH variable declaration):

require 'rubygems'
require 'rails/commands/server'
require 'rack'
require 'webrick'
require 'webrick/https'

module Rails
    class Server < ::Rack::Server
        def default_options
            super.merge({
                :Port => 3000,
                :environment => (ENV['RAILS_ENV'] || "development").dup,
                :daemonize => false,
                :debugger => false,
                :pid => File.expand_path("tmp/pids/server.pid"),
                :config => File.expand_path("config.ru"),
                :SSLEnable => true,
                :SSLVerifyClient => OpenSSL::SSL::VERIFY_NONE,
                :SSLPrivateKey => OpenSSL::PKey::RSA.new(
                       File.open("path/to/key").read),
                :SSLCertificate => OpenSSL::X509::Certificate.new(
                       File.open("/path/to/crt").read),
                :SSLCertName => [["CN", WEBrick::Utils::getservername]]
            })
        end
    end
end