Skip to main content

How to choose what tests to automate

Today I wanted to discuss the basic automation question - what exactly do we want to automate?
We can of course be extreme and say "everything" or "nothing" but both of those options are a very costly choices, first takes a lot of resources the other get you pay for it at the end.
I was consulting a client once where they had a complex process of ROI calculation that resulted in them not automating anything as the process to choose what to automate was time consuming by itself and ironically enough was not part of the Return Of Investment formula.

With this client we had to find and streamline the process to make a quick and easy decision regarding the automation candidates and I wanted to share this process with you.

To better understand the process we will need to answer two simple questions - Why and What:
  1. Why we automate
  2. What we automate

Why we automate?

Imagine yourself in an epic competition Human against the Machine in a battle of hammering nails into the wood. 
You prepare yourself by performing a set of complex stretches, warming your hands and feeling the weight of the hummer, the pure fresh air of a spring morning gives you the bust of energy that you need. While the other team of lab folks fine tuning the shining robot and adjusting the configurations before the start signal. Then the announcement coming in, but not a one that everyone expects. 
"Instead of a long, straight, board with nails to hummer", says the voice through the loadspeaks "we decided to introduce a small change - the board will be snake shaped"
The crowed murmurs, the lab team scrambles in a frenetic panic and you smile to yourself.
Then the boards are being replaced to a super long snake shaped boards with hundred of nails to hummer that will probably take you hours to finish and the gong  announces the beginning of the competition.
You are jumping on the first nail with the crowd supportive cheers, but you competitor stands still. What happened? You competitor, it seems, cannot start as all the tuning and the calculation had to be adjusted to the new terrain of the game. 
By the time you are half way through with the nails the lab team finally got their act together, finished the new calculations, applied the configuration and the robot is on its way.
"If the boards were shorter I would have won now", you think to yourself with a sigh, but unfortunately the things are not that easy. With a blink of the eye the robot catching up, passing you, and in another several minutes finishing the competition as an absolute winner.
"Rough start but an amazing finish" announcing the voice. And the crowd roars.

So from this story we can clearly see why we want to automate:  

We Automate To Save Time In The Long Term

If we have a short term project should we automate - probably No. If we are in a multi-level long term project should we automate - absolutely Yes.  

What we automate?

Imagine that in the previous story the competition was not about hammering nails but about inventing a story based on 10 randomly requested subjects. While you could come up with some sort of answers easily enough the lab team would probably needed hours if not days to program the robot to tell the new story each time.

So we can easily see that the best candidate for automation is:

Repetitive tasks

Back to my client, when we were able to narrow down our interest to "repetitive tests that we will require to execute for many releases in the future" we had easily arrived to to the right answer every time, keeping the decision making short and simple.


Popular posts from this blog

7 Most Popular Test Types in Software Testing

Today we are going to return back to basics of software testing and discuss the 7 most popular test types that are being used in every software testing effort. Those different test types cover all the levels of the software to make sure that the final result matching the expectations from every possible angle. Here is our list: Unit testingSmoke testingRegression testingFunctional testingIntegration testingUser Acceptance TestingPerformance Testing

Now let’s have a deeper dive into each one of those by using a simple example of an imaginary system that was created in order to manage warehouse activity including shipments, inventory and goods receptions from suppliers.
Unit Testing This type of testing is usually performed by the developers and is covering the very basic development component. In this test developers are testing the straight forward functionality of a functional piece of code to make sure that it is performing according to their expectations. In our example: This test w…

Story Points estimation for Scrum with Fibonacci vs Shirt Sizes vs Linear - 7 minute guide

It is all began long time ago when Development Teams were constantly asked to provide estimate and they were having a hard time to properly face the task. Let's admit it, there are so many things that can change, happen, and simply go wrong during the development process that one can hardly expect a proper estimation of hours for each task. That why a relative estimation with Story Points came along.

Story Points Estimation
Its a different way to estimate the effort of the Scrum Development Team with-in Agile methodology, which means that instead of estimating hours of work the team estimates each effort relatively to other efforts in the project.
Let's assume that a developer knows that specific 'Task 1' is much harder than another 'Task 2' it is hard for him/her to quantify that harder feeling in hours of additional work but it is possible to say that it much more work. This situation is being address by Story Points when each story point is representing som…

What is the velocity of an Agile scrum methodology?

Let's discuss some of the important measurements in Agile, and that is the Velocity of the Scrum team work.
Based on Wikipedia definition Velocity is " ...the rate of change of its position with respect to a frame of reference and is a function of time...", which when transferred to the scrum world can be summarized as: The amount of work that the scrum team completed in a single measure of time - in a sprint.

How we Calculate Velocity?
Velocity is actually a very simple to calculate, it is done but totaling the number of story points of fully completed user stories from the sprint backlog. So if a current sprint included 4 user stories: 2 with 8 story points each, one with 3 story points and one with 32 story points. and by the end of the sprint the 32 one was not fully done the velocity calculation will be:
Note: the 32 story points are not part of the velocity calculation as this user story was not completed.

What Velocity is used for?
The velocity is used i…