Paul Maxwell-Walters

A British-Australian software tester and test lead based in Sydney, Australia with about 12 years of experience testing in agriculture, education, financial services, digital media and energy consultancy. Paul is acting chair and a committee member of Sydney Testers Meetup Group, along with having spoken at several conferences in Australia.

6 minute read

Written by - Paul Maxwell-Walters

There is a Test Automation LinkedIn group I have been a member of for a few years. Members mostly post marketing posts, job ads and dubious polls, however interspersed with them are occasional questions about the practice of test automation.


This week a member posted the following poll -



“Do you think Manual Testers will Extinct Soon?” (sic) - True? False?



Reading this sent me to the edge of a fit of despair - not because I actually think “Manual Testers” will somehow “extinct” themselves but because it is the return to a nonsensical debate that I heard at testing conferences and read online at the start of my testing career in the late 2000s - at least 14 years ago.

However despite what the automation tool vendors and evangelists may have wanted to happen since then, manual testing (typically deemed to be testing by not programming the browser or UI to interact with itself or some tool to send off API requests and check responses automatically) is a long way from being dead.

Many teams throughout the world do not test using automation - that doesn’t mean that they are failing or unsuccessful in their roles.

Much of my career has been involved in purely manual testing work. Even in places where test automation is mature and widespread, testers still spend much time manually testing, analysing and reporting on products - such that if you were not skilled enough to test without automation, you would be found out quickly.



Another thing I have noticed is a rise in posts and blogs evangelising test automation as a brand new concept (“Have you heard about test automation and how it will revolutionise your QA? etc”). I saw one advertising a book, with the tagline “Life is Too Short for Manual Testing”. If that were the case, I should start looking for a new career or find a way to win the lottery since I doubt I will be absolving myself of all manual testing any time soon. 😊😂



One example I saw this week is a post with the headline “Are you ready for the future of QA?”.

It then described how test automation is “on the verge of impulse growth” and that “automation is becoming increasingly important in the QA industry.. companies that are able to take advantage of automation tools will have a significant advantage over their competitors”…



Those who have been testers for any length of time will know that test automation - and people (vendors, evangelists, test automation consultants or people looking for clickbait) proselytising test automation have been around for decades.

We already know (or at least should) what it is, what it can do and where it is applicable.

Kaner, Bach and Pettichord’s “Lessons Learned in Software Testing” had a whole chapter devoted to it - equally as fresh and relevant now as it was then - and that was published in 2002.

James Bach’s brilliant, seminal critique of the promises of test automation and the reckless assumptions used to justify them, “Test Automation Snake Oil” was written back in 1999. However the next generation comes along and makes the same promises hoping we have forgotten, for the same deleterious outcomes.



One of the big testing buzzwords, codeless automation, has had a renaissance in the last few years - promising quick and easy test case creation and execution across environments without the need to learn any of that messy coding stuff, with a whole suite of competing proprietary products vying for our testing budgets.

There are also a plethora of AI-powered test frameworks that can smart recognise UI components that have changed, ostensibly making tests more robust, less brittle. Similarly, record-and-playback tools for generating tests, deemed an antipattern as early as 2002, never took off in the industry - however they returned in later versions of Selenium IDE.



Of course there is a place for them in the market however this experiment was tried more than ten years ago with tools that promised smart UI object recognition and an IDE to create tests without the need to code, such as TestComplete and HP UFT - yet the industry moved away from them to newer, more basic open source frameworks like Selenium. It seems that having to learn to code to implement our own test automation for free (although it was never for free and failures were costly) wasn’t such a burden for us after all. Fixating on these also mask the genuine innovations that have arrived in more recent automation, such as visual UI comparison tools and tools for data creation and automating API and security checks.



Test Automation, like fashion, works in ever shrinking concentric circles.



Interestingly the test automators, who may have gloated to see fears of manual testing being made redundant by test automation in years past, have recently had their own bout of schadenfreude. Numerous articles, blog posts and linkedIn posts have risen speculating about the chatbot model chatGPT, showing code and sample test cases generated by the tool and making spurious claims and hand wringing that this will put (typically UI) test automation devs out of work.

Nevertheless, the authors failed to figure out (or at least ignored it) that there is a huge difference between generating easy specimen code for trivial purposes in already existing public websites (like entering text and clicking search in the Google homepage, the closest we have to a UI automation “Hello World”) and what we actually have to do - study designs and requirements, prioritising work, deciding what to automate, setting up environments, creating test data, writing complex frameworks, dealing with brittle tests, integrating them into CI/CD pipelines, reporting our work meaningfully, logging bugs, working with devs, managing testing projects, contributing to dev teams…



Those who really think that AI tools are imminently about to replace either manual or automated test practitioners neither understand AI, testing or software development.



AI will have a role to play but it will replace us when it replaces all of software development, which is no time soon.

Copying and pasting code from chatGPT is about as useful currently as pasting code from Stack Overflow or any of the numerous tutorial articles written in the last decade. Fears of the end of writing test automation are greatly exaggerated.

However chatGPT can churn out some quite convincing generic and bland tech blog articles, so I would suggest that the authors of the above should be more worried about themselves.



The tendency for testing as a profession to agonisingly navel gaze on the automation issue takes us away from actual challenges we face. For example, despite it being a major talking point at industry conferences for the last several years, most of us would struggle to test machine learning and AI algorithms adequately - much less so for blockchain applications and in the growing VR and metaverse spaces.

Despite adherence to accessibility standards being a legal requirement in countries like the USA, UK, France, India and Australia and a moral imperative everywhere, most of us have poor knowledge of this.

We need to collectively decide on what we want from automation in testing and put our efforts towards the future, not the past.

Paul

Tags:

Categories:

Updated:

You may also enjoy