Recently about Testing
Recently we have been using the excellent FakeFS (fake filesystem) gem in some specs to test code that reads and writes files on the filesystem. We are using the latest release version of this gem which is 0.2.1 as I am writing this. Some of the code under test uses the IO each_line method to iterate lines in relatively largish files. But we found out quickly that is a problem, since in version 0.2.1 the FakeFS::File class does not extend StringIO and so you don't get all its methods such as each_line. (The version on master in GitHub as I write this does extend StringIO, but it is not yet released as a formal version.)
As an example suppose we have the following code that prints out the size of each line in a file as stars (asterisks):
def lines_to_stars(file_path)
File.open(file_path, 'r').each_line { |line| puts '*' * line.size }
end
Let's say we use FakeFS to create a fake file like this:
require 'fakefs/safe'
require 'stringio'
FakeFS.activate!
File.open('/tmp/foo.txt', 'w') do |f|
f.write "The quick brown fox jumped over the lazy dog\n"
f.write "The quick red fox jumped over the sleepy cat\n"
f.write "Jack be nimble, Jack be quick, Jack jumped over the candle stick\n"
f.write "Twinkle, twinkle little star, how I wonder what you are\n"
f.write "The End."
end
So far, so good. But now if we call lines_to_stars we get an error:
NoMethodError: undefined method `each_line' for #<FakeFS::File:0x000001012c22b8>
Oops. No each_line. If you don't want to use an unreleased version of the gem, you can add each_line onto FakeFS::File using the following code:
module FakeFS
class File
def each_line
File.readlines(self.path).each { |line| yield line }
end
end
end
Basically all it does is define each_line so that it reads all the lines from a (fake) file on the (fake) filesystem and then yields them up one by one, so you can have code under test that iterates a file and work as expected. So now calling lines_to_stars gives a nice pretty bar chart containing the line sizes represented by stars:
******************************************** ******************************************** *************************************************************** ******************************************************* ********
Since we're using RSpec, to make this work nicely we added the above code that defines each_line into a file named fakefs.rb in the spec/support directory, since spec_helper requires supporting files in the spec/support directory and its subdirectories. So now all our specs automatically get the each_line behavior when using FakeFS.
On my current project we have been involved in converting some of the hundreds of manual tests that are run by our Test Team every release into a suite of automated Selenium RC tests.
During the course of this adventure my crew found several instances where XPath and native JavaScript were not sufficiently expressive to find elements in some of our more complicated interfaces.
Since our web app uses the Prototype/Scriptaculous JavaScript framework we wanted to find a way to make the locating power of Prototype available within Selenium RC.
We developed approaches for both Selenium 0.9.2 and Selenium 1.0.3 (which had better programmatic support for adding JavaScript user extensions to Selenium).
Selenium 0.9.2
Selenium RC provides the capability to add "user extensions" to augment its JavaScript core.
See http://seleniumhq.org/docs/08_user_extensions.html for detailed information on how to set this up.
We wrote the following user-extension.js file:
Selenium.prototype.setupProtoypeJS = function() {
id = selenium.browserbot.getCurrentWindow().$;
css = selenium.browserbot.getCurrentWindow().$$;
}In our code we have a base class for all our test cases. To this we added our own waitForPage() method:
public void waitForPage() {
selenium.waitForPage('60000')
proc.doCommand('setupPrototypeJS', [])
}Thus every time the page reloads (which clears the JavaScript context) we call waitForPage() and this command is re-executed. It creates two global variables (id and css) and binds them to Prototype's $ and $$ functions respectively.
Note: The reason we choose id and css instead of $ and $$ was that Groovy considers $ in Strings to be a special character and we would have had to escape it each time it was used.
The Prototype selectors can now be used in Selenium RC like this:
selenium.click("dom=id('foo')")
selenium.click("dom=css('.bar')")
selenium.click("dom=css('span.foo a.baz')")Note: You still have to specify the dom locator type so Selenium RC will know to execute your locator string as JavaScript.
Selenium 1.0.3
In more recent version of Selenium RC the project added the setExtensionJs() method. This allows you to set extension JavaScript programmatically prior to starting the selenium client:
selenium = new DefaultSelenium(...)
selenium.setExtensionJs('...')
selenium.start()
This made it much easier to implement our Prototype bindings. The only trick was that the JavaScript seems to be executed prior to having access to a page context and is also only executed once. This required us to take a different approach.
We created id and css as global functions instead of variables. This allowed us to defer accessing the current window until the functions were actually invoked.
selenium.setExtensionJs('''
id = function(value) {
return selenium.browserbot.getCurrentWindow().$(value);
}
css = function(value) {
return selenium.browserbot.getCurrentWindow().$$(value);
}
'''); Note: The above code snippet is written in Groovy which allows multiline Strings.
The Prototype selectors are used in Selenium RC like before:
selenium.click("dom=id('foo')")
selenium.click("dom=css('.bar')")
selenium.click("dom=css('span.foo a.baz')")I would be interested in hearing feedback from anyone who has a chance to use this technique!
- "Programming is a social exercise" - I thought this was a really good point. It was mentioned in the context of pair-programming, but I think it has far-reaching implications. Software development is more than just running through a bunch of formulas and crunching out an answer. Interaction within the team and with domain experts is crucial, to not only build the software right, but build the right software.
- "A refactoring is something that takes a few seconds, or a few minutes at most" - I was really impressed with the importance of automated refactorings in his discussions. I think most of the time that I spend "refactoring" is in small, manual edits, whereas most of the time that he spends is in using automated refactoring to "chunk" his edits. I definitely need to learn these keystrokes and refactorings better. Maybe I should start a "Refactoring Driven Development" movement...
- "Don't put refactoring on the schedule; do it all the time" - Simple, but effective. My tendency is to want to spend all my time refactoring, but this curbs that, because it forces me to refactor while I'm delivering user stories.
- "There are 3 essential design skills: nose, vision, and plan" - A nose for recognizing design smells, a vision for seeing a good design for your codebase, and an ability to come up with a plan to get from point A to point B.
- "Testing trumps good design" - This bothered me at first, but I think it's a really good point. The idea here is not to say that design is not important. But, rather, if you are forced to choose between between a "bad" design that allows better test coverage (e.g., less encapsulation), and "good" design which is hard to test, choose testability. The reason here is that the biggest roadblock to changing your codebase is not bad design, but FEAR of breaking something. If you know you will know when you've broken something, then you can retrofit a better design later.
- "There is self-worth tied up in "finishing" something" - He also drew a distinction between having something working (which some developers will call "finishing" it), and finishing it - making it not only work, but be thoroughly tested, maintainable, etc.
- Presentation layer - he talked about having the thinnest possible UI layer, which talks to a presentation layer to find out everything about how it should render. Then test the UI and business logic completely separately
- "You aren't doing agile development unless you are tracking your velocity and remaining story points in terms of passing, automated acceptance tests" - I balked at this at first, feeling like it was too easy to use this as a performance to beat the team over the head with. After I thought about it, though, it's really about the definition of done. We are done if all the features we said would be working are working. And how can we know this? Only by testing them. Continuously. Which means it should be automated.
- Acceptance tests don't need to be end-to-end, and, in fact, shouldn't be. This is another one that made me hesitate. After all, how do you know the feature is really working unless you go all the way from the UI to the database and back? Well, in short, because you're the developer. There's more value in being able to test features fast, constantly, than in being able to truly test them all the way from one end to the other. Mock/stub out what you need to to make that happen.
- FitNesse is cool. This is the second time I've played around with that tool, and the second time I've been impressed with its power and simplicity. I definitely need to play around with it some more.
So I stumbled across Griffon not too long ago:
http://griffon.codehaus.org/
It is a new project on Codehaus.org whose unique aim is to bring the Grails framework and way of doing things to the desktop application development world. I am not yet convinced this is a good idea, but since I still have many fond memories of Swing development I was intrigued to see what MVC for a desktop application might look like.
Like Grails, Griffon is built on Groovy, so if you are not familiar with Groovy you might want to take a look at:
http://groovy.codehaus.org/
On my current project we are starting to develop a suite of Selenium tests to help automate manual tests our test team are performing every release. Anyone who has used Selenium will probably agree with me that it can be finicky. When I write a selenium test I find myself frequently having to tweak a little XPath here, rerun the whole test, tweak a waitForCondition() there, rerun it again, etc.
Inspired by this drudgery I wrote a Griffon app that would give me a sandbox in which I could play with Selenium live instead of in the context of a test.
If you are interested in Griffon, I will walk you through some of the basics and you can see what you think. If you just want the app I wrote feel free to download it. I hope it will ease some of the pain of using Selenium!
Installing Griffon is not too hard. Download it, unzip it, create a GRIFFON_HOME system variable, and then glom $GRIFFON_HOME/bin to your PATH.
Like Grails, Griffon provides an application scaffolding script. Simply type griffon create-app and you have a skeleton for an application.
Of interest to this little tutorial are the models, views, and controllers folders, as well as lifecycle and conf.
MODEL
A Griffon model is just a POGO. It does not have to inherit from any particular class. It is associated with a controller and a view via the naming convention.
So for example, here is the model from my Selenium Console project:
class SeleniumConsoleModel {
@Bindable String host
@Bindable String port
@Bindable String command
@Bindable String baseUrl
@Bindable boolean serverRunning = false
@Bindable String scriptSource
@Bindable String scriptResult
@Bindable boolean enabled = true
}Griffon will automatically associate this model object with a view named SeleniumConsoleView and a controller named SeleniumConsoleController.
Note the use of the @Bindable annotation. This is a very interesting feature that will be put to good use in the view.
VIEW
Griffon uses Groovy SwingBuilder syntax to create a view. Here is a snippet from the SeleniumConsoleView:
application(title:'Selenium Console', pack:true, locationByPlatform:true, iconImage: imageIcon('/griffon-icon-48x48.png').image,
iconImages: [imageIcon('/griffon-icon-48x48.png').image,
imageIcon('/griffon-icon-32x32.png').image,
imageIcon('/griffon-icon-16x16.png').image]
) {
panel() {
borderLayout()
panel(constraints:CENTER) {
borderLayout()
panel(constraints: NORTH, border: raisedBevelBorder(), background: Color.DARK_GRAY) {
flowLayout(alignment:FlowLayout.LEFT)
panel(opaque: false) {
button("Start Selenium", actionPerformed: controller.&startSelenium, enabled: bind { model.enabled }, visible:bind(source:model, sourceProperty:'serverRunning', converter:{ !it }))
button("Stop Selenium", actionPerformed: controller.&stopSelenium, enabled: bind { model.enabled }, visible:bind(source:model, sourceProperty:'serverRunning'))
}
...
By making good use of Groovy's dynamic features, the SwingBuilder syntax allows you to declaratively build your layouts. Each "method call" (such as panel() or button() is actually interpreted by methodMissing and the corresponding Swing component is constructed and added in whatever scope it occurs. The named parameters correspond to properties of the component which can be set inline.
A useful reference for all the valid named parameters that can be called on each component:
http://groovy.codehaus.org/Alphabetical+Widgets+List
A closure containing any subcomponents to be added can be passed in as well, allowing you to anonymously nest components inside of each other.
Two properties are injected into the scope of the view: model and controller, allowing you to access them where appropriate.
BINDING
Groovy / Griffon allow you to "bind" one property to another. This mechanism acts like an action listener, when a property changes, any property that is bound to it also changes. This is extremely useful because you can bind GUI widgets (such as a checkbox or a testarea) to your model. As the user interacts with the GUI, the state of the GUI is automatically captured on your model object.
This can also work in reverse. You can set portions of your GUI to respond to changes in your model object.
A recent project started out life as an all-Java project that used Maven as the build tool. Initially we used Atlassian Clover to measure unit test coverage. Clover is a great product for Java code, but unfortunately it only works with Java code because it works at the Java source level. (This was the case as of Spring 2009, and I haven't checked since then.) As we started migrating existing code from Java to Groovy and writing new code in Groovy, we started to lose data about unit test coverage because Clover does not understand Groovy code. To remedy this problem we switched from Clover to Cobertura, which instruments at the bytecode level and thus works with Groovy code. Theoretically it would also work with any JVM-based language but I'm not sure whether or not it could handle something like Clojure or not.
In any case, we only cared about Groovy so Cobertura was a good choice. With the Cobertura Maven plugin we quickly found a problem, which was that even though the code coverage was running, the reports only showed coverage for Java code, not Groovy. This blog shows you how to display coverage on Groovy code when using Maven and the Cobertura plugin. In other words, I'll show how to get Cobertura reports to link to the real Groovy source code in Maven, so you can navigate Cobertura reports as you normally would.
The core problem is pretty simple, though it took me a while to figure out how to fix it. Seems to be pretty standard in Maven: I know what I want to do, but finding out how to do it is the really hard part. The only thing you need to do is tell Maven about the Groovy source code and where it lives. The way I did this is to use the Codehaus build-helper-maven-plugin which has an add-source goal. The add-source goal does just what you would expect; it adds a specified directory (or directories) as a source directory in your Maven build. Here's how you use it in your Maven pom.xml file:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/groovy</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
In the above code snippet, we're using the "build-helper-maven-plugin" to add the src/main/groovy directory. That's pretty much it. Run Cobertura as normal, view the reports, and you should now see coverage on Groovy source code as well as Java.

