Hi, I’m Will Gordon, a QA engineer on the basket and checkout teams. I’ve been with AO for a little over 5 years and during that time I’ve worked on or developed several different frameworks for automating some of the testing of our applications, including Protractor, and Selenium and SpecFlow with C#.
I’m going to share five tips that many beginners (and some veterans) may find useful for improving their automated tests. These tips are written specifically with Selenium WebDriver in mind, but can be applied more broadly to other test automation frameworks.
#1 Condensing test scenarios for quicker execution
Many people design their test suites so that each test is checking one particular piece of functionality. While this can make a test suite easier to read and navigate, it often introduces a lot of repetition between test scenarios.
You could try condensing your test scenarios down so that each one checks several areas of functionality in one pass, as appropriate. This way there are fewer test scenarios to run, with little repetition between them. This will reduce the run-time of your tests, and reduce the number of redundant actions.
Example: In the case of an e-commerce checkout, you would have tests which check several different types of address (flats, businesses, etc), and each different accepted payment method. Instead of having a specific test for each address type and each payment method, these could be condensed into tests which each test an address type and a payment method. In this limited example, you could halve the number of tests. Assuming each payment method test would require an address being entered, you are also greatly reducing redundancy in the test suite.
These tests, while simple, contain several repeated steps.
The same tests are being performed here, but in fewer scenarios and fewer steps.
#2 Better element locators to make tests more deterministic
Choosing a good locator is very important to do carefully – it will define how reliable, readable, maintainable, and durable your tests are going to be, and how likely they are to cope with future changes to the UI of the page you’re testing.
The stated “best practice” for Selenium element locators is ID > name > CSS > XPath, but that doesn’t always apply. Here are a few things to keep in mind when choosing an element locator:
- “Data-oriented” elements and attributes are preferable to “layout-oriented”. Try not to depend on page design choices. For example, in the absence of good ID’s and names, you could select a button by contained text rather than by a CSS class that may change. An XPath of “//*[contains(text(),’Confirm)]” can be more reliable than a CSS selector of “.btnBlue” even though it’s less readable, and against best practices.
- Don’t depend on the technology of the UI implementation. For example, if the page uses AngularJS, try not to use internal angular classes, such as ng-scope or ng-binding.
- Use the HTML structure of the page as little as possible. The more elements you have in your path to the selected element, the higher the chance that a UI change would break your locator.
#3 JavaScript execution when all else fails
JavaScript execution is one of Selenium’s most versatile features, and one that many people seem to be unaware of. If WebDriver doesn’t have a capability you need for interacting with the page, then JavaScript probably can. You can execute JavaScript directly on the page Selenium is testing using the following code (assuming your instance of the WebDriver is named _driver):
This example code simply forces a page refresh using JavaScript execution
Depending on the JavaScript you are executing, this can make tests less readable and maintainable, so use with care, and only where other WebDriver functionality is lacking for what you need.
Uses for JavaScript execution that I have needed in the past include:
- Forcing a click on an element that WebDriver doesn’t seem to want to click on. This can happen for a number of reasons depending on web browser, version, and the underlying technology of the page you are testing.
- Scrolling elements into view
- Removing elements from the page (for example if they’re immaterial to the test and causing issues for testing certain functionality)
#4 HTML reporting to make life easier
If you have ever spent long and frustrating amounts of time sifting through test logs trying to find why a certain test failed, then you’ll appreciate how much easier the addition of an HTML report will make your life.
HTML reporting usually consists of a dashboard collating all of your test scenarios with
a pass/fail mark against each, and an expandable view to see where the failures are. With many reporting frameworks, you can extend them to add a screenshot from the browser window as well, for even easier diagnosis of issues.
There are many reporting frameworks available for various test frameworks, and many newer automation frameworks such as Cypress come with one built in. Specifically for Selenium using C# and SpecFlow, Extent seems to be the preferred reporting framework. It can be fiddly to integrate, and a time investment, but it will pay itself off in a relatively short space of time.
Something like this is definitely easier on the eyes than sifting through text logs
#5 Performance tweaks for faster feedback
Fast feedback is often key to good test automation, which means minimising the run-time of your tests can be an important task. There are many factors which can inflate this time more than you might expect, and most people don’t consider many of them when writing tests.
- Implicit waits. These are a great idea in theory, meaning that your tests will automatically wait a specific amount of time before throwing a failure in any case where expected conditions are not met. However, consider using only explicit waits, which wait for specific conditions in specific cases instead of any time conditions are not met. While more difficult upfront, this will result in less time spent waiting, errors returning more quickly when they should, all resulting in that fast feedback. Always weigh up whether the ease of adding implicit waits is worth the cost in run-time.
- Just say “No” to thread sleeps. Thread sleeps are guaranteed to inflate the run-time of your tests by an exact amount. Wherever possible, these should be replaced with explicit waits. It may be that on average a certain wait will take as much time as the thread sleep it replaced, but will at least have the chance of taking up less time.
- Not all element locators are created equal. Some methods of locating elements are slower than others. The fastest is usually either ID or CSS selector (and there has been some debate online over this), and the slowest by far is XPath. In a small test suite this difference may only add up to seconds at most, but in a large and long-running suite testing a complex web application, you may see a larger reduction by optimising your element locators.
Thanks for reading, I hope these tips can help you optimise your test automation.