In this post we want to give an example of how Visual Validation can be integrated into a Selenium QA project in Java. By Visual Validation we mean the comparison of the rendered user interface to a given baseline. In its most simplistic form it is a pixel-by-pixel comparison of screenshots. If any deviation is detected, the test will fail. One way to do this kind of automated testing is by utilizing a tool called Ocular:

Ocular is a simple utility which helps us to add the visual validation feature into the existing WebDriver test automation frameworks. Ocular uses Arquillian RushEye for the actual image comparison.

Project setup

We are going to create a test that checks Amazon’s home page for differences. We will use:

  • Eclipse
  • Maven
  • jUnit
  • Selenium
  • Ocular

We create a new Maven project in Eclipse and define the pom.xml with the dependencies needed:

The structure of the test code is simple and contained within a single class. We use jUnit annotations to execute methods that will configure Ocular and the WebDriver used to run the browser. In this example, we execute the test in Chrome, so we need a ChromeDriver. The ChromeDriver works as a bridge between Selenium and the Chrome browser and it can be found here. The location of the ChromeDriver executable needs to be specified in a system variable. We also need to configure Ocular. We set the location of the baseline images and the results and set it to create a new baseline image, if a baseline does not exists beforehand. Furthermore, Ocular can be instructed to ignore certain parts of the web page. This can be useful where dynamic content is loaded. In this example we will mask out the logo of Amazon (top right corner) of the screen.

Results

In order to show the functionality of the test, we have created a baseline image by running the test once and having Ocular generate a screenshot. Then we have edited the screenshot to create artificial differences that will be highlighted in the second execution. The following is the edited baseline screenshot along with the results from the second execution:

Snapshot of the user interface of Amazon.com, used as a baseline. We have added certain elements to the page.

Results of the comparison between the baseline and the current screenshot. Notice differences highlighted in red. Notice also that the Amazon logo has been masked out.

Ocular generates a result image that highlights the differences between the baseline image and the current screenshot. This result is in black and white with the differences highlighted in red. The blue rectangular encompasses all the differences that were discovered. Notice how the logo of Amazon is masked out by a black rectangular shape. An interesting side note is that there is a slight difference in the location of the categories listed at the top of the page. This is a real difference between the two loads of Amazon’s front page.

Conclusion

We have seen how to incorporate Visual Validation into a Selenium QA testing environment by using the Ocular tool. This can add great value to your team’s test suite and help to catch discrepancies in the user interface that are difficult to discover by using traditional value. We will conclude with a few tips and tricks:

  • How stable is the site to be tested? Highly volatile user interface will prove very difficult to test in this way.
  • When using Ocular ensure that zoom levels of the browser and the operating system are set to 100%. Otherwise you can experience discrepancies when trying to locate specific elements.
  • During automated testing with Selenium it is important to ensure that a site is fully loaded before executing tests. For obvious reasons this is absolutely crucial for Visual Validation.
  • Find each dynamic element (orderID, date, marketing spot, …) and mask it, effectively ignoring it when comparing screenshots.
  • Ocular allows for a threshold to be set before failing a test. This can give some flexibility and stability to test execution.