TLDR; Automating Strategically is hard. Sometimes we need to work tactically and learn, before we move forward. Sometimes we automate for ourselves so that we benefit, before attempting to automate for the project.
Sometimes when people automate to support their testing process they shoot for the moon too fast. I’ve done that. And failed. I get lots of emails from people who are trying to implement a Strategic solution, in a tactical environment. And don’t realise it.
I see people who have not yet developed the skills and experience to build the “Test Automation” solution they have in mind. With the risk that they then fall back on concepts like “flaky tests”, or “tech debt” or “Test Automation doesn’t work” or “Never Automate through the GUI” to explain away the over-reach.
Automating Strategically is hard. In fact any kind of strategic operation is hard.
This is why Agile projects try to work in small iterations to help spot issues early and take corrective action quickly.
This is why Agile thought leadership stresses working as a whole team, to avoid over reliance on a subset of the skills and experience on the project.
Sometimes we need to work tactically and learn, before we move forward.
That often isn’t the working practice people want to follow.
When we aim too high, too fast, we often are not demonstrating the value we think we are, we often communicate the work we are doing as ’the end goal’ rather than ‘a learning process’. We are often working slower and making more mistakes than we ultimately will after we gain experience. The perception of the ’end goal’ value is reduced because are are still working towards it, but haven’t described it as a work in progress.
Aim high, but work within our capabilities and develop slowly. Sometimes there is more value in expanding the data coverage rather than functional scope. Make automated execution work robustly on a small scale first, then build on a solid foundation.
What follows is a series of small notes and ideas which have helped me in the past.
If you want to automate, then start tactical, small and simple
“If you want to automate, then start tactical, small and simple.”
I presented at SauceCon 2020 on Tactical vs Strategic mindset in terms of Test Automation. A distinction that I use to help me deal with organisation issues and Software Development problems.
I also use it when thinking about advice to offer.
For example, I receive a lot of emails and questions from testers who are:
- “struggling to get the team to see the value in test automation”
- “getting push back from programmers about test automation”
- “convincing management about the ROI value for test automation”
- etc.
Often, when you try to implement a Strategic Test Automation approach like the ‘best in class’ that you see in blog posts and articles before your team is ready. Possibly before your project, organisation, department, or even you yourself, is ready. Then you will hit resistance and will likely fail or struggle because the wider system will not commit the resources you need to succeed.
You may not be ready for that ‘best in class’ implementation. You might still be learning how to automate, you might still be learning how to work as an Agile team. It might not be the most appropriate time to automate to that level.
Start simple. Start small. Start personal.
- Posted to Linkedin on 20200609
When you see value in automating then start personal
If you, see value in automating. And you can see how to add value to your personal test process, then start there. Start tactically. Aim smaller. Automate to help you save time and augment your testing.
If the small experiments and implementations that you create do actually add value, and do actually save you time. Then you can spend the time saved expanding the scope, or demonstrating the benefit to other people as part of your ‘sales’ process for more strategic automating.
This doesn’t mean ‘give up all hope of automating strategically’. It means start small with the resources and knowledge you have to hand. This might mean - “use a tool” initially, rather than “write an automation framework”.
I tend to think in terms of:
“Don’t aim for Strategic ‘big’ automated execution as a first step. Instead, start small, with simple tactical implementations that can benefit you as an individual. And then build from those small steps.”
Not the real problem
When I’m asked:
- “struggling to get the team to see the value in test automation”
- “getting push back from programmers about test automation”
- “convincing management about the ROI value for test automation”
- etc.
There are a lot of reasons for all of this. Many of them are contextual.
Often the ‘issue’ that I’m being asked about isn’t the real issue e.g. they aren’t getting push back about Test Automation, they are often getting push back on Testing itself. They are often getting push back on communication, and working as a team. Very often this is symptomatic of an ineffective work place, rather than “Test Automation”.
“Test Automation” is a big picture concept. Doing something small to improve is pragmatic and hard to argue against, particularly when people can see an improvement.
Context Free Advice For Introducing Automation
Without knowing your context. Without knowing why you are getting push back on Test Automation. Without knowing why you are failing to sell the benefits of Test Automation to management or others in your department.
There is something you can try, regardless of any context.
Automate for yourself.
Identify quick wins that:
- don’t require a lot of up front time.
- don’t require a budget.
- don’t require a lot of training or research.
Look for tools that offer quick wins out of the box with minimal configuration.
If you can program then write quick and dirty utilities that increase the data coverage, and increase the possibility of hitting an edge case.
Automate for yourself so that whatever you do adds value to you.
Automate For Yourself
Automate to support your testing effort.
Identify quick wins for your own Personal Test Process.
Whatever “Test Automation” you put in place.
When starting out, the first customer and stakeholder is you.
The first person to receive value, is you.
You have no budget to spend, except your time. If you spend time automating, then it is time away from the project and you will have to make that time up somehow. So any ROI has to be a return on your time.
e.g. If you spend an hour automating then you need to return at least an hour of your time to break even.
Any waste, is going to come out of your time budget i.e. you may have to work over lunch, or stay late, because you are spending your time.
It is hard. You’re taking a risk.
So start small with simple tooling, and simple scripts. If these do save you time, then its time you can reinvest in automating more, or testing more.
How do you avoid spending your own time?
Get permission for your experiment or learning. This can often be hard when you don’t know what you are about to do, or how long it will take, or what you think the results will be.
Some environments will not support this.
It can be hard.
We avoid spending our time by spending our time on a sales process to get permission to spend team time.
Sometimes the sales process can be as time-consuming as the experiment.
How to Automate for Yourself
How can you automate for yourself?
Start tactical.
- Identify common bugs, and find tools that can identify them automatically.
- find tools that can find low-hanging fruit e.g. link checkers, W3c Standard compliance, SEO meta data validators
- Identify simple functional paths and find tools that can automate them
- find tools that can automate out of the box activities e.g. password fillers, form fillers
- find capture play back tools that are robust and can perform data driven activities
- Identify common processes and find tools that can automate them
If the tool causes re-work, or is intermittent in its execution then stop using it, because it is costing time, rather than saving it.
Automate Contextually. Where you are the context.
It is tempting to automate with code, and with a library like Selenium WebDriver.
I do that. I trained people to do that. But it takes time to learn, and build your experience. Ultimately it is a long term strategic set of learning that allows you to handle more variety.
When you are starting out, and spending your limited time budget. You need to achieve results fast.
Automate simply and at the level of your ability. You are a major part of the context when you are automating for yourself and your time is the only budget that you have.
- Experiment with keyword driven tools
- Focus on Data Coverage, rather than path coverage
- If you know how to code, then start with the simplest paths and tools for the language you know.
Automate so that you achieve value as quickly as possible.
Work Tactically
When you start by spending your own time, to automate your own process. You are not attempting to automate for the long term and create a large framework or suite of tests that will run in continuous integration.
You are attempting to automate simple paths, that can add maximum value, as fast as you can.
If you find that the tests execute intermittently, i.e. sometimes they work, sometimes they don’t. Then this is failure demand. This is costing you time. This is not adding value.
Quickly identify an alternative approach, or quickly find a solution.
Do not build on something that is intermittent.
If any of the tactics that you adopt work, then you can build on them, and perhaps they can become part of your strategic approach.
Strategic is the Sell
The point at which you need to move beyond a tactical approach and into a strategic approach, is the point at which you start ‘selling’ the benefits to others. Because this is the point at which you are trying to spend project time, or gain more involvement from other people on the team, project, department or company.
Because you have had success on a tactical basis. You have:
- results that you can show
- benefits that you can explain
- an understanding of what you need to move forward
Management
Unfortunately, many people do not work in teams that are operating effectively.
- their role is not valued
- they are viewed as an expense or overhead
- the focus is on features, not quality of the features
- the only valued output from testing is ‘bugs’ (and these are often frowned upon, “testing is delaying the project by finding too many bugs”)
- etc.
When a project is not operating effectively. Trying to automate ‘well’ and ‘properly’, using Strategic approaches, is likely to fail, and meet heavy resistance.
Unfortunately, and this is not advice that is often given, sometimes we have to look out for our own survival.
I know you want to do the best for the project and put in a world-class Test Automation system.
But… the project isn’t ready for it. So whatever you do, it has to add an advantage to your process, before you try and sell it to anyone else.
Personal Experience
I have worked on projects that did not value Testing, and did not value Test Automation. And where we made the mistake of building Strategic Long Term Automated Execution solutions.
They were a lot of work. There was a lot of pain and frustration involved. The cost on our own personal time budgets probably wasn’t worth it.
We learned a lot. But we could have learned the same, with less pain. By spending our personal time on training and personal automation projects.
Strategic approaches to Automated Execution, which are mostly how we talk about automating, require buy in from:
- the rest of the team
- the project
- the management
If you don’t have that commitment. Then start small on a tactical personal level first.
Build your personal experience. Then you know the limits to what you can do. You know what you need to take the automating to the next level. You know specifically what to ask for to take the automating forward, and you know what risks or time commitments are involved.
One of the hardest things about automating is that we often over-reach our capabilities as we are learning, as we are trying to demonstrate value to people who don’t believe in what we are doing. If we are still learning, then people have to buy in to ’learning more’ as one of the value outputs from the automating, not just the automated execution.
