Test Automation Frameworks- Then & Now!

Implementing automated tests is something that everybody wants to do. If you ask any tester – they aim to be an automation tester. Implemented well, automated testing not only complements agile development processes, it reduces cost of test execution, enables developers to focus on core operations by automating repetitive tasks, and eventually it increases testing accuracy as well as helps to deliver a better solution faster. It is the golden target that every tester aims for, but only a few of them take pains to assess the required knowledge. Over the years, test automation tools and processes have matured to simplify the process of developing test automation and creating reliable test scripts.

This blog will give you an insight on test automation frameworks and revisit why they faded into oblivion.

Record and Playback

Even though this involves either no or minimal code writing, manageability of automation code makes Record/Playback only viable on small scale. The fundamental problem of this approach is literally the problem of scale. If we want another automated test, we’ll record another script which eventually results in two scripts to be maintained as the AUT’s interface has changed with time. The more tests we record, the more automation code needs to be maintained – which will become too much of a burden on testing budget. In record and playback each automated test case is a sequence of actions with test data hard coded in to it. This, firstly is not a good approach as per the software engineering principles. Secondly, payback in automation starts when we playback the same automation to test the subsequent versions of the application. But, in this approach each test is run once per release and is not used repeatedly to increase the test coverage. Therefore, to manage such automation code makes this approach impractical for large scale automation. Therefore, it is only due to the cost of maintenance, test automation has moved past Record and Playback.

Data-Driven tests

Data-driven tests focus on data and inputs/outputs. Usually, there is a test case defined and set of test data that will be passed into it as arguments. This way we can execute one scenario with different inputs or outputs for example when testing some sort of calculations or algorithms. Test data can be defined in external file – in this way we separate test data from test code. But this is not necessary. The TestNG framework, allowing implementation of unit tests  (and not only) for Java applications, comes with the Data Provider feature letting us to create data sets that can be used and shared among the test method. Using this framework let us remove redundancy from tests when we have many similar tests differing with data only. Moving the data to external source facilitates users, customer, business to create tests and a better collaboration.

Keyword Driven

This takes test automation to the next level. It is not the script now that directs testing but it is the test data itself. The test data with use of keywords sequences out the actions to be followed. When the automated test case runs, it will read through the test data and call for the relevant script specified by the keyword, passing AUT the data for that line. Hence, keywords are scripts written by test specialists to perform all necessary actions to test the business/functional task for which this script was written. With this approach, the testers have the full control over what to do and in which order. But, development of automation code is still AUT specific. The rate and the amount at which the UI of AUT undergoes change so will the maintenance of the scripts. Therefore, for large scale test automation, we still need to find ways to decrease the need to regularly maintain scripts.

The concept of the keyword-driven framework may seem to be complicated especially to implement. Fortunately there are tools/frameworks already implemented that can be integrated with yours. For example Google’s Robot Framework can be easily integrated with Selenium/Webdriver. Also commercial tools have such capabilities. If not, they provide sets of helper objects that can facilitate this task. But still the tester needs to demonstrate development skills in order to get all of these working.

UI Object Map Based

In pursuit to make test automation better, UI Object Map framework solves all the three challenges of test automation. It resolves the maintainability, reliability, and the ease of development of test scripts. This framework takes instruction from test data, recognizes the class of object to be acted upon, and then performs the specified action on that object by calling a script for that particular object class, passing actions and data to it. This means the scripts are no more AUT specific but are UI object class specific. No script for specific instance of an object but for class of the object. Once the script for a class of UI object is written, it can be reused in any automation project where this class of UI object is being used. Through this framework scripts automate the class of UI object. And whenever the AUT undergoes change, you need to only change object map and the data, not the scripts for the UI object class. The only time this needs to be changed is when new UI objects are introduced or when the behavior of existing UI-object-class changes. A UI Object Map lists the objects in each window/page. Each UI object is given a logical name and also contains the information that the test tool needs to identify that object uniquely. The test tool script uses the logical name in scripts when performing actions on UI Objects.