Jim Clark - Blogs at Near Infinity

ActiveRecord: Nested Attributes

Jim Clark — Mon, 02 May 2011 23:25:38 -0400

My project recently started developing a web app using Rails 3 and Extjs, a pure JavaScript frontend, i.e. no ERB, haml, etc. We have several multi-page forms where we use a JavaScript model to hold the form data and upon save serialize the JSON back to the server. If successful, the updated data is rendered as JSON and sent back to the client.

The model has a simple one-to-many relationship (parent/children)

{parent = { id: 1, name: 'Jim', children: 
[{id: 2, name: 'Larry',parent_id:1},
{id: 3, name:'Curly',parent_id:1},
{id:4, name: 'Moe',parent_id:1}] }

class Parent < ActiveRecord::Base
  attr_accessible :name
  has_many :children 
end

class Child < ActiveRecord::Base
  belongs_to :parent
end

My expectation was that when I save the parent data, via update_attributes, both the parent and child data save together in a single transaction.

class ParentsController < ApplicationController
  ...
  def update
    parent = Parent.find(params[:id])
    parent.update_attributes(params)
    ...
  end
end

Being new to developing Rails apps in the "real world" I had not dealt with this, what I thought to be, common scenario. I found that out of the box the parent data saves, but not the child data. I first explored the book "Agile Web Development with Rails", but didn't find what I needed. Next I checked out the Rails Guides, in particular the section on associations The first setting I found was the :autosave setting:

If you set the :autosave option to true, Rails will save any loaded members and destroy members that are marked for destruction whenever you save the parent object.

Seemed like a good fit.

has_many :children, :autosave => true

However, the children still did not save when calling update_attributes (I'm still not sure exactly what this is used for)

Next stop, Google. Some searching brought me to the ActiveRecord "accepts_nested_attributes_for" method. It turns out this is exactly what I needed. So I added "accepts_nested_attributes_for :children" to Parent

class Parent < ActiveRecord::Base
  attr_accessible :name
  has_many :children
  accepts_nested_attributes_for :children
end

However, just adding "accepts_nested_attributes_for :children" wasn't enough. It turns out that with the accepts_nested_attributes_for method, the Parent class receives a method called "children_attributes=(attributes)". This allows for the parent's children attributes to be updated. This also means I have to update my JSON so that the "children" property is called "children_attributes".

parent = { id: 1, name: 'Jim', children_attributes: 
[{id: 2, name: 'Larry',parent_id:1},
{id: 3, name:'Curly',parent_id:1},
{id:4, name: 'Moe',parent_id:1}] }

Since I am using "attr_accessible" I also have to include "children_attributes" in the list

class Parent < ActiveRecord::Base
  attr_accessible :name, :children_attributes
  has_many :children
  accepts_nested_attributes_for :children
end

Now calling "parent.update_attributes" saves both the parent and its children, within a single transaction. Rails will handle inserts/updates/deletes appropriately. If the child has an id, the child is updated. If the child is missing an id, a new child is added. If the child has the "_destroy" attribute set, the child is removed.

The last todo was to serialize the JSON back to the client upon success. When trying this:

render :json => parent.to_json(:include => :children)

my JSON included "children", not "children_attributes":

{parent = { id: 1, name: 'Jim', children: 
[{id: 2, name: 'Larry',parent_id:1},
{id: 3, name:'Curly',parent_id:1},
{id:4, name: 'Moe',parent_id:1}] }

For the save to work correctly, the JSON needs "children_attributes", (which is a bit of a disconnect between relationship and nested attribute naming conventions) I was not able to figure out how to get Rails and "to_json" to use "children_attributes". For my solution I did not ":include => :children" in my call to "to_json" and I overrode "as_json". In my "as_json" I manually add the "children attributes" to the parent JSON.

  def as_json(options={})
    json = super(options)
    json[:children_attributes]=[]
    
    self.children.each do |child|
    json[:children_attributes] << {:id => child.id, :name => child.name}
    end
    json 
  end

Looking back on getting the save to work, it seems rather simple. However, several things made this a little harder than it should have been. The first being the documented coverage. Surprisingly, this was not covered in the book "AWDR" or Rails Guides. It is spelled out quite well in the API docs, however, but you need to know about the NestedAttributes module The ":autosave" setting misled me a bit. I will need to go back and do a bit more research on this. Lastly, the naming convention for nested attributes threw me off and the fact that I had to "rig" as_json. Perhaps there is a way to handle this by "convention".

For further reading check out Ryan's blog or the Nested Attributes API docs.

Introduction to PDF::TechBook

Jim Clark — Mon, 30 Apr 2007 22:44:39 -0400

Recently I had an interest in producing PDF documents using Ruby on Rails. A quick Google search brought me to a Ruby PDF library called PDF::Writer for Ruby written by Austin Ziegler. As far as I can tell from the online manual PDF::Writer is a fairly robust library. The manual was well written and generated by PDF::Writer itself, very impressive. By reading Austin's introductory PDF::Writer article I was able to get up and running quickly. The article demonstrates creating text, tables, drawings and inserting images. However, I didn't need to do anything fancy like create charts or drawings (see the PACMAN example). All I wanted was some basic header, text and code styling. Out of the box PDF::Writer allows you to do these sort of things but it requires some additional work on your part. PDF::Writer gives you the primitives to work with to create whatever you want, but I didn't have the time to go through the effort to create my own styling from scratch. Fortunately, Austin extended PDF::Writer to create PDF::TechBook. TechBook gives you the ability to mark up and style your content that can then be used to generate a PDF document. This is a brief introduction to TechBook.

The TechBook class extends PDF::Writer and is an interpreter that interprets a lightweight markup language. You feed the TechBook class a string or a file that contains content and TechBook markup to generate a PDF document. TechBook markup provides styling for basic headings, preformatted blocks, code sections, columns, table of contents, bulleted lists and a few other items. I equate this to something like an HTML-lite or textile for PDFs. The markup uses directives to indicate the start and end of a block of text. Some directives are line oriented like headers that only require a start directive. Other directives can span multiple lines like bulleted lists. We'll start with a basic example.

Headers, similar to the <H1-5> tags in HTML, are supported up to five headers by using the somewhat awkward format #<heading-text>, e.g.

    1&lt;Level 1&gt;
    2&lt;Level 2&gt;
    3&lt;Level 3&gt;
    4&lt;Level 4&gt;

produces...

With TechBook you can create bulleted lists using "discs" or standard bullets. Here's an example showing how to create a bulleted list using the disc bullets. (To create standard bullets replace "disc" with "bullet")

    .blist disc
    Line item 1 with disc
    .endblist
    .blist disc
    Line item 2 with disc
    .endblist
    .blist disc
    Line item 3 with disc
    .endblist

produces...

In addition to headings and lists, TechBook allows you to preformat text and identify code sections. E.g.

    .pre
           Pre line 1
           Pre line 2
    .endpre

produces...

and

    .code
    pdf = PDF::Writer.new
    pdf.select_font "Times-Roman"
    pdf.text "Hello Ruby", :font =&gt;14, :justification =&gt; :left
    pdf.save_as("e:/hello_ruby.pdf")
    .endcode

produces...

After you've created your content using the TechBook markup, it's easy to create a new instance of the PDF::TechBook class and produce the PDF file. The following is an example, where "my_content" is your marked-up content

    pdf = PDF::TechBook.new
    pdf.select_font "Times-Roman"
    pdf.techbook_parse <span style="color: blue;">my_content</span>
    pdf.save_as("c:/temp/hello_ruby.pdf")

Download a sample program

There are other features supported by TechBook such as table of contents and the eval directive, but hopefully this will give you a start to using TechBook. If you view the raw version of the PDF::Writer manual that comes with gem you can get a good idea of what you can accomplish with PDF::Writer.

I'm interested in other solutions. Leave me a comment if you know of other ways and/or better ways to create PDF documents using Ruby or have additional tips using PDF::Writer.

Finding a Ruby IDE

Jim Clark — Mon, 16 Apr 2007 21:49:33 -0400

Recently I've spent some time revisiting and learning more about Ruby on Rails. In doing so I wanted a decent editor to work with. This article is about my "adventure" in finding an editor and/or IDE that I could use in exploring and learning Ruby and Ruby on Rails and ultimately what I ended up selecting.

Everything I read online talked about TextMate the editor of choice for Mac Ruby developers. I have a Windows machine so I didn't have that option. I decided to stick close to home by seeing if IntelliJ (my favorite Java editor) had a plug-in that supported Ruby. At the time no such plug-in existed. I gave SciTE and FreeRIDE (the editors that come with the Ruby distribution) a spin but wanted something a little closer to what I was use to in a Java IDE. I remember hearing something about RadRails, a Ruby IDE based on Eclipse. I was familiar enough with Eclipse so I decided to use that for my dabbling. I used this for a while and it was OK for what I was doing.

In the meantime I came across a Windows TextMate-like editor called "e". I first installed a beta version of "e" which stands for the "E Text Editor ". I spent some time trying to make this editor work for me. I wanted to see what all the excitement was about in the TextMate world, even if it was on Windows :-) . To use "e" I first needed an updated installation of cygwin since "e" relies on cygwin to function. I wasn't overly pleased that "e" relied on cygwin to be a functional editor. Nevertheless I installed a recent copy of cygwin and was up and running with "e". "e" is in beta and with beta software comes annoyances and partially functional software. I really wanted to make this editor work. However, I ran in to a few bugs that prohibited me from using "e" and caused me to resume using RadRails. About this time, the IntelliJ folks released a Ruby plug-in.

The IntelliJ plug-in worked fairly well for what I needed. I could use it in an environment that I was very familiar with and was happy to be "home". Initially I experienced some minor bugs. The biggest problem I had was the plug-in can only be used in IntelliJ 6+ and I currently run 5.1 and wasn't prepared to upgrade. Unlike other editors I had looked at IntelliJ is relatively more expensive. After my 30 day license expired, I headed back to RadRails, but then I ran in to this blog post about the future of RadRails:

"The bottom line to all of this is that Matt and I can't provide the same kind of commitment that we once could. Like or not, RadRails is not a business. We've been working our asses off on a start-up for the past few months and RadRails has suffered as a result. I wish things were different and we could sit back and work on open source all day but that is just not the reality anymore."

I wanted to use software where I felt there was community support. I continued my search to see what else was available. I came across InType, a very basic editor that is suppose to be TextMate-like, however, at the time of the writing they were still in an alpha version and didn't seem to have much momentum. At the time I used the software it was still very immature (e.g. no tab support though you can open multiple windows). Then one day I read a blog entry about Ruby support in NetBeans. IÃ•d always associated NetBeans with the so-so Java IDE currently produced by Sun.

I downloaded a milestone release of NetBeans release 6 and installed support for Ruby and Ruby on Rails. Almost instantly I was wowed by their support for Ruby and Rails. I had no trouble becoming productive with the IDE and have been happy with the product since. Current features include:

Code completion
Occurrence highlighting
Integrated documentation pop-ups for Ruby API calls
Parameter completion
Mongrel support
Rake migrations
And much more

Additionally, NetBeans supports development in either the reference implementation of Ruby or in JRuby, a Java implementation of the Ruby interpreter. Most recently the NetBeans folks have begun adding support for TextMate bundles. They have also created a stripped down version of the IDE that is strictly for Ruby coding (though this still seems a little rough around the edges). As you can see the NetBeans folks are really putting forth a great deal of effort to create a solid, robust IDE for creating Ruby and Ruby on Rails applications. While the RadRails IDE is now supported by Aptana and IntelliJ 7 will include built in Ruby support (release expected by the end of the year), NetBeans has made a big leap ahead of the rest of the pack and nothing appears to be getting in their way. For now NetBeans is the choice for me when doing Ruby development.

Update: I forgot to mention this recent IDE comparison of NetBeans, IntelliJ and RadRails.

IntelliJ Type Renderers

Jim Clark — Wed, 04 Apr 2007 14:30:27 -0400

When debugging in IntelliJ you may need to inspect byte arrays as these byte arrays may contain meaningful text. Out of the box individual byte array values will be rendered as numeric byte values. This makes it difficult to decipher what it is you're debugging.

Fortunately IntelliJ gives a way to customize your data views when debugging. The way to handle this is create a custom Type Renderer. While inspecting values during a debug session, in the Frame tab right click and select "Customize Data Views...". A dialog will appear, select the Type Renderers tab. IntelliJ allows you to create a renderer for objects of a particular type. Apply renderer to a byte[] and use the expression "new String(this)".

This will render the byte array as a String:

For cases when you don't want this type renderer enabled, IntelliJ allows you to disable the renderer by unchecking the checkbox next to the renderer name.

Thumbnail Generation Gotchas

Jim Clark — Tue, 26 Dec 2006 15:21:48 -0500

For a recent release of our software we had a requirement to generate and store thumbnail images. We have documents that come in to our system with image attachments and we need to generate a thumbnail image for each of the attachments that have a supported MIME type (gif, jpg, and png). When I signed up for this assignment I thought it was a good opportunity to re-learn JavaÃ•s image APIs (itÃ•s been almost seven years since doing any Swing or AWT work!). A lot of information is available on the web for generating thumbnail images, however, the information is scattered and there is more than one way to perform this task. Additionally, there are several gotchas that I experienced rather than reading about. Hopefully this article will get you started in the right direction and help you avoid some of the problems I faced.

Scaling Images

Before reading and writing images, I first needed to figure out how to actually scale an image, i.e. take an original image and size it to an appropriate thumbnail size. Digging around the AWT package and the Image class I found the java.awt.Image.getScaledInstance method. Piece of cake, just do something like:

    Image scaledInstance = image.getScaledInstance(thumbWidth, thumbHeight, hint);

This worked fine and I used it for a couple of weeks when one of my coworkers pointed out that this was no longer the best way to scale images (see Bug 6196792) Instead, use the 2D Graphics API and an AffineTransform for a better solution.

public static BufferedImage getThumbnail(BufferedImage image, 
                                         int maxThumbWidth, 
                                         int maxThumbHeight, 
                                         RenderingHints hints)
{
   BufferedImage thumbnail = null;
   if(image != null)
   {
       AffineTransform tx = new AffineTransform();
       // Determine scale so image is not larger than the max height and/or width. 
       double scale = scaleToFit(image.getWidth(), 
                                 image.getHeight(), 
                                 maxThumbWidth, maxThumbHeight);
                 
       tx.scale(scale, scale);

       double d1 = (double) image.getWidth() * scale;
       double d2 = (double) image.getHeight() * scale;
       thumbnail = new BufferedImage(
               ((int) d1) < 1 ? 1 : (int)d1,  // don't allow width to be less than 1
               ((int) d2) < 1 ? 1 : (int)d2,  // don't allow height to be less than 1
               image.getType() == BufferedImage.TYPE_CUSTOM ? 
                                  BufferedImage.TYPE_INT_RGB : image.getType());
       Graphics2D g2d = thumbnail.createGraphics();
       g2d.setRenderingHints(hints);
       g2d.drawImage(image, tx, null);
       g2d.dispose();
   }
   return thumbnail;
}


private static double scaleToFit(double w1, double h1, double w2, double h2) {
        double scale = 1.0D;
        if (w1 > h1) {
            if (w1 > w2)
                scale = w2 / w1;
            h1 *= scale;
            if (h1 > h2)
                scale *= h2 / h1;
        } else {
            if (h1 > h2)
                scale = h2 / h1;
            w1 *= scale;
            if (w1 > w2)
                scale *= w2 / w1;
        }
        return scale;
    }

Using this code I was able to scale an image, i.e. create a thumbnail, and now I needed to save it.

Storing Thumbnails

Since I was working with J2SE 1.4, which only allows for creating images in JPEG and PNG formats, we decided to use the JPEG format for thumbnails for its smaller file size. I needed a way to write my thumbnail to an output stream since we have a separate API for storing files. Fortunately the javax.imageio.ImageIO package provided an easy way to create JPEG images and put the image in to an OutputStream:

    ImageIO.write(myImage, "jpg", myOutputStream);

I incorporated my changes in to the build and left for a week of training, everything seem to be working ok. Upon returning I found out that thumbnails were "killing" our system. Naturally, I questioned, "are you sure itÃ•s thumbnails causing the problem?" ItÃ•s just image creation. Sure enough, the problem was caused by thumbnail generation. It turns out the garbage collector went in to full GC mode when thumbnails were being created, specifically when hundreds or thousands of thumbnails were being generated. Ugh, back to the drawing board.

After a little research, it was discovered that there are memory leaks in SunÃ•s ImageIO JPEG libraries (Bug 5015137 and Bug 4827358). As our system was creating hundreds of thumbnails, everything else came to a screeching halt thanks to garbage collection. What to do, what to do?

Supposedly these bugs are fixed in J2SE 1.5, however, upgrading wasn't an option.

One solution we looked at was creating thumbnails as PNG files. Fine, this works, no problems with performance. YouÃ•ll just need to go out and buy a few extra hard drives. I was finding the PNGs to be 4, 5, 6 times larger than JPEGs. I looked at ways to compress the PNGs via Java but didnÃ•t have much luck getting them down to a reasonable size.

We also talked about having the thumbnail creation occur offline after the document was brought in to our system. This seemed more complicated than we wanted to make it. Plus we were running out of time as the code deadline was approaching.

Next a coworker sent a snippet of code, for encoding images to JPEG, that talked directly to com.sun JPEG classes. This always seemed a no-no, i.e. Sun says: "Note that the classes in the com.sun.image.codec.jpeg package are not part of the core Java APIsÃ‰". Since we werenÃ•t planning to upgrade the JDK any time soon we went with this approach for writing JPEGs.

    JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(myByteArrayOutputStream);
    JPEGEncodeParam encodeParams = encoder.getDefaultJPEGEncodeParam(myImage);
    encodeParams.setQuality(0.8F, false);
    encoder.setJPEGEncodeParam(encodeParams);
    encoder.encode(myImage);

Everyone was happy; we could create JPEG thumbnails by circumventing the ImageIO API and talking directly to the com.sun classes. UntilÃ‰

Reading Images

I noticed that certain thumbnails werenÃ•t being rendered correctly; rather they were rendered as black rectangles.

Now what? Was this something to do with the usage of the JPEGImageEncoder, i.e. talking directly to the com.sun classes? After a little research it turns out the problem existed previously when directly using ImageIO to encode and write images. Therefore, no new problems were introduced. It seemed to be a problem only with GIF files. I ran a couple of tests on Windows (dev environment) using my test GIFs and they worked fine. I tested animated GIF files on Windows and they worked correctly. I tested the animated GIFs on Solaris. These images were being rendered as black images. The animated GIF images that were "thumbnailed" on Solaris were not created correctly. I was now able to repeat the problem.

To read images from the file system, the images I would create thumbnails from, I was using the AWT tookit:

    Image image = Toolkit.getDefaultToolkit().createImage(myFileBytes);

Unlike Windows, the Solaris toolkit apparently didnÃ•t know how to handle images that were made up of multiple images (GIF89a). I needed a way to create images, in my case from a byte array, that didn't result in a black image. Back to the ImageIO class. Turns out they have a read method that can take an InputStream

    image = ImageIO.read(new ByteArrayInputStream(myImageByteArray));

However, I was a little leery of the ImageIO class due to the problems I had seen with the write method. I did find a bug related to ImageIO.read method (Bug 4867874), but it pertained to JPEGs. I decided to use ImageIO.read for GIF files and continue using the AWT toolkit for JPEGs and other types. This was a fix for our black image problem.

It seems as if our thumbnail processing is working as expected and we havenÃ•t seen noticeable degradation in performance or the black image problem.

If youÃ•ll be doing image processing, such as thumbnail creation, make sure your application can handle the processing requirements. Test a worse case scenario, e.g. create 1000 thumbnails. Test with many different image samples with different image formats (e.g. animated gifs) and sizes (large image files). Unfortunately you'll have to actually view the images to verify expected output, but now you are hopefully aware of some of the issues surrounding the generation of thumbnails using the ImageIO APIs and the AWT toolkit.