Biology Driven Testing

Hello Everyone!

It has been a while since my last post but today I would like to share with you my vision about the future of automation testing.

Introduction

My experience of over 8 years prompts me so that whole conception of automation testing needs to be changed in order to be able to adapt to constantly growing user requirements and the speed of project development.

ISTQB definition of automation testing says that they are not finding any incidents at all. This can be true if we are looking from the perspective of a static code that is doing only limited amount of operations. For me a word "automation" is equal to "do something for me but you can use my all of my current experience". I wonder if you are slowly getting what I am trying to tell you :).

Common problems

So far, while I was working on various projects, I found several problems that needed to be addressed. Unfortunately, I could not find any good application or agile methodology which could help me to solve them relatively fast.

1. Limited amount of time

I hope that there is no need to elaborate on point :).

2. Limited human resources

This point is related to a situation when one person is trying to do several different tasks and as a result the quality of his work is heavily reduced.

3. Limited human knowledge

It is related to a situation when one person knows everything about one part of a project. A good example are user data which need to be input in order to log in to an application.

4. Limited human communication

It describes a situation when there is a problem with basic communication between people. For instance, when a new version has been deployed for testing and no one is familiar with changes it contains.

5. Limited testing environment stability

It presents a situation when environment is constantly changing and we are lacking the information about parts of the project which are currently under heavily modifications.

Of course, you can say that some of them are really trival ones to solve but when you add to them a presure generated by stakeholders and other factors then you can see that sometimes you are not able to solve them at all.

This is only a fragment of problems which a tester needs to fight every single day.

Common solutions

There are plenty of opensource applications, methodologies and other technologies which testers, who are related to a part of automation proccess, are really familiar with. You probably know or maybe you have heared about Test Driven Development, Behavior Driven Development or Keyword Driven Testing. They can be really helpful when it comes to produceing high quality tests but at the very same time they are limited by their functionality.

What I am talking about?
- you have to prepare some global data and a solution for them to work
- you have to prepare some input data and you have to know the output data in order to make a simple test
- you have to always update already prepared tests because of the fact that the testing environment is constantly changing
- ...

ISTQB says that a good tester should be able to overcome any difficulties in the project. This is true. No doubts about it. However, because of some random real life situations (for example someone with big domain knowledge is leaving the project), the whole concept of testing automation is one big question mark.

Biology

This is not a simple and trival problem to solve. Nevertheless, while thinking about the world which surrounds us, we can come to a conclusion that we are not alone with this kind of problems. Plants or animals also contantly need to adapt to the changing environment, without the possibility to modify their bodies. So why can't we try using their methods to solve our problems?

Graphic taken from: http://animalsadda.com/wp-content/uploads/2013/06/Ants-3.jpg

You have probably heard about ants and some algorithms which are trying to simulate their behaviour. How could we use it in our testing proccess? First of all, you should think about their number. They can do huge amount of operations at same time. They can be anywhere. When it comes to finding a path, they leave some kind of trail behind them, so that others of their kind can find the correct way.

We could implement their behavior and "attack" our project from different perspective. At the very same time, we could check UI as well as do some white box code testing. They could help us to solve some non deterministic problems by leaving a direct information about their path and everything what they have done so far, in order to produce an incident. Sounds great, right?

Ants are checking what is going on inside a class during testing process

One of many articles about ants algorithms: https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms

Graphic taken from: http://park-lake.org/wp-content/uploads/2011/07/youngPlant.jpg

And what about plants? They have some kind of their own perception. By using specific signals inside their body they are responding to the changing environment. The most suprising is that they are doing it not having any brain at all. What could we do with that?

We could implement a behavior that is checking and reacting directly to the changing environment. If there is a signal (incident) then we can react to it and provide the user with high quality information.

Plant is sitting next to the secondMethod and awaits for any incidents that can appear

One of many articles about plant perception: https://en.wikipedia.org/wiki/Plant_perception_(physiology)

Graphic taken from: http://www.nathab.com/uploaded-files/carousels/TRIPS/Photo-Wolves/2a-Wolves-photo-holdsworth.jpg

Predators? Smell is a one of many major signals for them and it is helping them to localize a prey. If any additional argument is present, as for example blood, then time needed to find new delicious food is shortened by a huge margin. How can we use it?

For example, if there was an incident of one type in some place, we could check the whole project or only limited area around it, because there is a high chance that a similar problem can be found in other place. We can also send them for a hunting mission with the aim of finding a defect of single type which can appear in unknown area. We could use information directly from the code and from black box testing methods in order to help to localize them.

Wolfs are sometimes working together as a pack. Keep that in mind :).

Wolf is searching for a hidden bug present in the firstMethod

"Sky is a limit", I would say. The only problem is current technology and limited machine resources.

Light!?

Although it can sound rather skeptically, I would like to share with you some good news. There is a very good and accurate article about AI which describes when it will be invented to help us with our tasks.

You can find it here: http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html

Invention of AI will change our whole world. Just before it, there will be time of huge computing powers which could be used by us. Testers will have to adapt and their role will have to modify.

Danger Zone

One more thing that I would like to share with you before we come to the final conclusion. Animals often know which areas should be avoided in order to stay alive. The same is for human beings. We are leaving warning signs for others.

Biology Driven Testing could use this mechanism by adding for instance a new type of annotations. Developers who are working over a method or a class could leave a trail behind which would say that a particular area should not be a part of testing proccess. This could directly impact a situation when testers are wasting their precious time on creating tests related to a part of aplication or code which is is to change soon or become obsolete.

Example:

Thus, our algorithm could avoid those areas and everything related to it and focus mainly on stable parts. Below you can find a simple graph showing how a system would be visible from the perspective of Biology Driven Testing.

Oracle

Major factor which would allow us to use this testing methology is a testing oracle containing all tester's knowledge and experience from the project. If we are lacking of those data, we could use some good and popular books, which describe common bugs appearing in applications, as our first major data base.

How to use it?

What is left to say about the concept of Biology Driven Testing is how to use it. The only single thing that any tester should do with this testing method would be to select an algorithm which would search for incidents in a specific for it traits way. If there would be a request to find as many bugs in the code and in the whole project as we can, then we could use Ants algorithm to do so. On the other side, if we would have a single incident to check, then we could use Predators features for that. Finally, if we would like to monitor our project for some behavior then Plants could come handy. And so on...

So the very simple algorithm would could look like this:

How could we upgrade it?

Depending on the company which you are currently working in, tester could or should not add any tips on how to solve incidents which had been found. We could improve Biology Driven Testing by adding some good and already popular problem solutions. There are plenty of static code analyse applications which could be useful when it comes to that part.

The final word

Ok everyone. That is all for today. I hope that some of my thoughts could be inspiring for you.

This is only a minor amount my ideas regarding the future of test automation. If you find it intresting or you would like to ask about something, then please leave a comment.

See you around!