For a recent release of our software we had a requirement to generate and store thumbnail images. We have documents that come in to our system with image attachments and we need to generate a thumbnail image for each of the attachments that have a supported MIME type (gif, jpg, and png). When I signed up for this assignment I thought it was a good opportunity to re-learn Java’s image APIs (it’s been almost seven years since doing any Swing or AWT work!). A lot of information is available on the web for generating thumbnail images, however, the information is scattered and there is more than one way to perform this task. Additionally, there are several gotchas that I experienced rather than reading about. Hopefully this article will get you started in the right direction and help you avoid some of the problems I faced.

Scaling Images

Before reading and writing images, I first needed to figure out how to actually scale an image, i.e. take an original image and size it to an appropriate thumbnail size. Digging around the AWT package and the Image class I found the java.awt.Image.getScaledInstance method. Piece of cake, just do something like:

    Image scaledInstance = image.getScaledInstance(thumbWidth, thumbHeight, hint);

This worked fine and I used it for a couple of weeks when one of my coworkers pointed out that this was no longer the best way to scale images (see Bug 6196792) Instead, use the 2D Graphics API and an AffineTransform for a better solution.

public static BufferedImage getThumbnail(BufferedImage image, 
                                         int maxThumbWidth, 
                                         int maxThumbHeight, 
                                         RenderingHints hints)
{
   BufferedImage thumbnail = null;
   if(image != null)
   {
       AffineTransform tx = new AffineTransform();
       // Determine scale so image is not larger than the max height and/or width. 
       double scale = scaleToFit(image.getWidth(), 
                                 image.getHeight(), 
                                 maxThumbWidth, maxThumbHeight);
                 
       tx.scale(scale, scale);

       double d1 = (double) image.getWidth() * scale;
       double d2 = (double) image.getHeight() * scale;
       thumbnail = new BufferedImage(
               ((int) d1) < 1 ? 1 : (int)d1,  // don't allow width to be less than 1
               ((int) d2) < 1 ? 1 : (int)d2,  // don't allow height to be less than 1
               image.getType() == BufferedImage.TYPE_CUSTOM ? 
                                  BufferedImage.TYPE_INT_RGB : image.getType());
       Graphics2D g2d = thumbnail.createGraphics();
       g2d.setRenderingHints(hints);
       g2d.drawImage(image, tx, null);
       g2d.dispose();
   }
   return thumbnail;
}


private static double scaleToFit(double w1, double h1, double w2, double h2) {
        double scale = 1.0D;
        if (w1 > h1) {
            if (w1 > w2)
                scale = w2 / w1;
            h1 *= scale;
            if (h1 > h2)
                scale *= h2 / h1;
        } else {
            if (h1 > h2)
                scale = h2 / h1;
            w1 *= scale;
            if (w1 > w2)
                scale *= w2 / w1;
        }
        return scale;
    }

    

Using this code I was able to scale an image, i.e. create a thumbnail, and now I needed to save it.

Storing Thumbnails

Since I was working with J2SE 1.4, which only allows for creating images in JPEG and PNG formats, we decided to use the JPEG format for thumbnails for its smaller file size. I needed a way to write my thumbnail to an output stream since we have a separate API for storing files. Fortunately the javax.imageio.ImageIO package provided an easy way to create JPEG images and put the image in to an OutputStream:

    ImageIO.write(myImage, "jpg", myOutputStream);

I incorporated my changes in to the build and left for a week of training, everything seem to be working ok. Upon returning I found out that thumbnails were "killing" our system. Naturally, I questioned, "are you sure it’s thumbnails causing the problem?" It’s just image creation. Sure enough, the problem was caused by thumbnail generation. It turns out the garbage collector went in to full GC mode when thumbnails were being created, specifically when hundreds or thousands of thumbnails were being generated. Ugh, back to the drawing board.

After a little research, it was discovered that there are memory leaks in Sun’s ImageIO JPEG libraries (Bug 5015137 and Bug 4827358). As our system was creating hundreds of thumbnails, everything else came to a screeching halt thanks to garbage collection. What to do, what to do?

Supposedly these bugs are fixed in J2SE 1.5, however, upgrading wasn't an option.

One solution we looked at was creating thumbnails as PNG files. Fine, this works, no problems with performance. You’ll just need to go out and buy a few extra hard drives. I was finding the PNGs to be 4, 5, 6 times larger than JPEGs. I looked at ways to compress the PNGs via Java but didn’t have much luck getting them down to a reasonable size.

We also talked about having the thumbnail creation occur offline after the document was brought in to our system. This seemed more complicated than we wanted to make it. Plus we were running out of time as the code deadline was approaching.

Next a coworker sent a snippet of code, for encoding images to JPEG, that talked directly to com.sun JPEG classes. This always seemed a no-no, i.e. Sun says: "Note that the classes in the com.sun.image.codec.jpeg package are not part of the core Java APIs…". Since we weren’t planning to upgrade the JDK any time soon we went with this approach for writing JPEGs.

    JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(myByteArrayOutputStream);
    JPEGEncodeParam encodeParams = encoder.getDefaultJPEGEncodeParam(myImage);
    encodeParams.setQuality(0.8F, false);
    encoder.setJPEGEncodeParam(encodeParams);
    encoder.encode(myImage);

Everyone was happy; we could create JPEG thumbnails by circumventing the ImageIO API and talking directly to the com.sun classes. Until…

Reading Images

I noticed that certain thumbnails weren’t being rendered correctly; rather they were rendered as black rectangles.

Now what? Was this something to do with the usage of the JPEGImageEncoder, i.e. talking directly to the com.sun classes? After a little research it turns out the problem existed previously when directly using ImageIO to encode and write images. Therefore, no new problems were introduced. It seemed to be a problem only with GIF files. I ran a couple of tests on Windows (dev environment) using my test GIFs and they worked fine. I tested animated GIF files on Windows and they worked correctly. I tested the animated GIFs on Solaris. These images were being rendered as black images. The animated GIF images that were "thumbnailed" on Solaris were not created correctly. I was now able to repeat the problem.

To read images from the file system, the images I would create thumbnails from, I was using the AWT tookit:

    Image image = Toolkit.getDefaultToolkit().createImage(myFileBytes);

Unlike Windows, the Solaris toolkit apparently didn’t know how to handle images that were made up of multiple images (GIF89a). I needed a way to create images, in my case from a byte array, that didn't result in a black image. Back to the ImageIO class. Turns out they have a read method that can take an InputStream

    image = ImageIO.read(new ByteArrayInputStream(myImageByteArray));

However, I was a little leery of the ImageIO class due to the problems I had seen with the write method. I did find a bug related to ImageIO.read method (Bug 4867874), but it pertained to JPEGs. I decided to use ImageIO.read for GIF files and continue using the AWT toolkit for JPEGs and other types. This was a fix for our black image problem.

It seems as if our thumbnail processing is working as expected and we haven’t seen noticeable degradation in performance or the black image problem.

If you’ll be doing image processing, such as thumbnail creation, make sure your application can handle the processing requirements. Test a worse case scenario, e.g. create 1000 thumbnails. Test with many different image samples with different image formats (e.g. animated gifs) and sizes (large image files). Unfortunately you'll have to actually view the images to verify expected output, but now you are hopefully aware of some of the issues surrounding the generation of thumbnails using the ImageIO APIs and the AWT toolkit.