Why the Copilot Route Is a Flawed Strategy for Software Testing

I’ve spent over 25 years developing and bringing software testing products to market. During this time, we’ve witnessed significant changes — automated testing, continuous testing, low-code/no-code platforms, Centers of Excellence, and the shift-left approach. Yet, despite these advancements, one thing has remained constant: testing remains a time-consuming, inefficient bottleneck in the software development lifecycle.

Although testing has become less of a bottleneck over the years, we’ve never truly achieved seamless validation. Today, AI is at the forefront of the evolution of our testing products (as with many other vendors), but the key question remains: Will it make testing more efficient? What about the human element? Will AI reduce mundane tasks, or will it simply add another layer of complexity, as past “advancements” have done?

The Next Generation of Automated Testing

Many vendors have started incorporating AI into their testing platforms. Many have opted for AI copilots to assist in test creation, but this is only part of the solution. With AI advancing rapidly, we must look beyond copilots and focus on more meaningful accuracy, efficiency, and speed improvements.

While copilots can accelerate test creation, they don’t solve some of testing’s more significant challenges. Feedback from users has been mixed. Some developers find copilots helpful, but many report that they sometimes generate incorrect tests or create more tests that need ongoing maintenance. Instead of addressing core problems, copilots risk exacerbating them, simply layering AI over an already broken process.

Why Testing Is Still Broken

To understand why, let’s look at the evolution of automated software testing. The industry has developed various techniques for identifying objects on screen, enabling tests to run. While some methods work better than others, the fact remains that changes to the UI or application break tests necessitate constant maintenance — an estimated 30-40% of a tester’s time.

Take regression testing as an example: regression suites must be rerun whenever a change is made to detect new issues. In large applications, these tests can number in the thousands, yet it’s estimated that only 20% of tests in regression suites are valid. Many of these tests produce errors that require time-consuming analysis to verify their authenticity. It’s no wonder some enterprises still rely on manual testing, even though they know they can’t achieve full test coverage.

A New Approach To Testing

When discussing how to evolve web, mobile, and performance test suites, it was realized that while copilots assist in writing test scripts, they don’t address users’ most significant pain points: test maintenance and analysis. That’s why it’s best to focus on areas where value can be delivered, starting with test validation.

We need to rethink how to approach testing. Can the need for scripting and object-based locators be eliminated?

A Smarter Approach

The next evolution in automated testing is not simply visualizing what’s on the screen but using AI to understand it contextually and dynamically. By leveraging natural language processing, users can ask simple questions, and the system can analyze and validate tests accordingly without relying on objects or code or even needing to understand the underlying technologies of the applications. The result? Real-time validation that works across any platform and is usable by both technical and non-technical team members. It also survives massive changes to the code without manual intervention.

While the immediate benefits of AI-driven test validation are clear, the real value comes from hands-on experience and feedback from real users. Financial institutions, for example, have used this approach to validate data they could never validate before, such as checking whether on-screen graphs match the underlying tables. Retailers can now ensure product descriptions match product images. This AI-driven validation can work seamlessly, even as applications evolve, without requiring manual adjustments.

Moving Beyond Validation

But test validation is just the beginning. To truly transform testing, we must look at how AI can free testers and developers from routine tasks, allowing them to focus on more valuable work. The goal is to democratize testing — making it accessible and efficient for everyone. I am not a fan of the word “democratize”, but when used correctly, this is genuinely what AI will enable.

The future of software testing isn’t about applying AI for the sake of it or adopting short-term fixes. It’s about harnessing the most significant advancements in computing to solve real-world problems and finally fixing the broken processes plaguing software testing for years.

The post Why the Copilot Route Is a Flawed Strategy for Software Testing appeared first on The New Stack.