Last April I attended the first annual Selenium conference in San Francisco. It was an awesome conference with a great line up speakers that included many of the original creators of the Selenium project! It was a neat opportunity to have a lot of perspective into the history of Selenium and what they want it to be in the future.

One great talk I heard while I was there was by Jason Huggins the original creator of Selenium IDE. I had just had an email discussion with a person on another project in our company about whether or not they should standardize on Selenium IDE or Selenium RC and I was interested to hear what Jason had to say about the future of Selenium IDE.

When asked whether or not one should use Selenium IDE over RC he replied with something to the effect of "choose Selenium RC hands down". He went on to say that there was even discussion among the project creators about whether or not to remove Selenium IDE from the project completely.

Very intriguing coming from the creator of Selenium IDE himself! And I have to agree with him.

Note: For those new to Selenium, Selenium IDE is a Firefox browser plugin that allows you to record a test script that includes actions (such as clicks or typing text into forms) and assertions (such as verifying text on a page) to be replayed it at a later date. Selenium RC on the other hand is an API with Java bindings (and several other languages as well). It allows you to start a browser from code and issues commands to it or make assertions from it via a Java program.

On the surface Selenium IDE is appealing. It is fast and easy to get started. Testers with no programming background can create test scripts with relative easy. In a short time you can have a suite of tests to show for your efforts.

But unfortunately the returns are very short lived. The fundamental problems with Selenium IDE (which Jason pointed out in his talk) are the inherent brittleness of the scripts, the lack of variables, conditionals, or looping support, and the lack of ability to reuse portions of the scripts.

(1) Selenium IDE scripts are brittle. As soon as the page layout changes, the scripts will break. The Selenium IDE plugin does its best to locate page elements using id values or xpath selectors, but if the structure of the page changes at all the tests break.

Selenium RC is not immune to this problem, but since the programmer is writing the selector it is easier to use selectors that are more resilient to change. For example, instead of the Selenium IDE generating an xpath selector like "//div/span[3]/div[2]/span[4]/div[2]/a" which is bound to break as soon as the page structure changes, Selenium RC allows a programmer to write something like "$('foo').down('bar')" which is much more resilient.

(2) Selenium IDE lacks the support that a programming language has for variables, conditionals and looping. Imagine you have a list on a page with a large number of items in it. You might want to assert that each expected item appears on the page. But SeleniumIDE has no capacity for repeating the same action over and over again unless you manually record that action multiple times you need.

If you have an application where the appearance of the application can vary based on the environment it is deployed to then you would have to write one Selenium IDE script for each environment. You have no way of performing "if in this environment then check this condition" kind of logic.

Other kinds of simple assertions are difficult. Imagine trying to assert that a timestamp is no less than a day old. In Selenium IDE you have no way of providing the logic to parse a timestamp and verify it falls within the expected range.

Selenium RC is executing in the context of a full featured programming language. Loops, conditionals, and variables are all at the disposal of the test writer and can be used to make rich assertions about the state of the application without needing to create many scripts that are variations on the same thing. The power and expressiveness you have in Selenium RC lets you write tests in entirely different ways and much less tediously than recording long scripts over and over again.

(3) Selenium IDE lacks the ability to reuse portions of your test. In most applications there are common steps (such as logging in or searching for an item) that are repeated in many different tests. In Selenium IDE you have to repeat all those steps for each test script you write.

Selenium RC is again executing in the context of a programming language so common portions of the test can be refactored into methods that can be reused. This dramatically reduces maintenance effort because when an application changes the tests only need to be updated in one or two places instead of being recreated altogether.



These three comparisons are really just part of the debate. There are other things that make Selenium RC attractive in the long run (including easy deployments, and the ability to interact with the application in more ways than just via a web browser). The debate boils down to the fact that while Selenium RC requires a higher learning curve and requires test writers to be programmers, the results are far less brittle, far more powerful, and dramatically easier to maintain than a massive collection of Selenium IDE scripts.

In an organization where developers have the trust and respect of management and application testing is not strictly assigned to a test team, Selenium RC is the clear choice of technology.

Thanks for reading! I am on Twitter (@marlhammer) and you can contact me there or through email (smouring@nearinfinity.com) if you have any questions or comments!