Forem: Gagan Singh

Human vs AI: Who Writes Better Cypress Tests?

Gagan Singh — Fri, 03 Apr 2026 16:00:00 +0000

In the first post I asked whether the AI surprised you with what it caught or missed. After running this locally against Sauce Demo, here is my honest answer.

It did surprise me. I went in expecting the human to win. That is not quite what happened.

After indexing the three docs into ChromaDB and running cy.prompt() with that context, I ran both tests. The same app, the same flows, one written by a human and one grounded in RAG context.

The AI knew the locked out user scenario because it was in the bug history doc. It knew the exact selectors because they were in the component doc. It did not guess. It worked from what I gave it.
Here are both tests it generated:

Both tests passed.

But here is where it gets interesting. The AI verified that an error message existed. It did not verify that the message said "Sorry, this user has been locked out." That is intent knowledge. It lives in someone's head, not in a doc. The human catches that. The AI does not.

And anything that was never documented will not show up in the tests either. A flow built last Tuesday that never made it into any spec or component doc is invisible to the pipeline. The RAG context is only as good as what you indexed.

So neither wins cleanly. The AI covers breadth. The human covers intent. The most useful thing is not picking a winner, it is understanding where each one has blind spots and using both accordingly.

One thing I did not expect: cy.prompt() requires a Cypress Cloud account to authenticate. It is not a fully local feature. That was a real discovery during the setup and worth knowing before you go too far down this path.

If you have tried this, I am genuinely curious what your results looked like. And if you found a better chunking strategy for API specs after reading Post 2, I would love to hear that too.

What Does a RAG Pipeline for Cypress Actually Look Like?

Gagan Singh — Wed, 01 Apr 2026 16:15:00 +0000

In the last post I asked whether the AI writing your Cypress tests actually knows your app. This one gets into what it looks like to give it that knowledge, with a real example I built locally.

The pattern is RAG, Retrieval-Augmented Generation. At its core it is straightforward. You index your app's documents into a vector database, and at query time, the most relevant chunks are retrieved and passed to the AI as context. The AI generates a response grounded in your actual docs rather than guessing.

What I Indexed for Sauce Demo
I used Sauce Demo as my test app and created three docs:

An API spec covering login, inventory, cart, and checkout endpoints
A component doc with exact CSS selectors for every page
A bug history doc covering known failure scenarios like locked out users, problem user add to cart issues, and checkout validation gaps

Not everything needs to be indexed. What actually moved the needle was the component selectors and the bug history. The AI stopped guessing at button labels and started working from real data.

The Pipeline
I used ChromaDB as the vector database and Google Gemini embeddings to index the docs. The script follows five simple steps:
// Load environment variables // Set up Google Gemini embeddings // Connect to ChromaDB // Read and chunk your docs // Embed and store each chunk

Querying the Index
Once indexed, when you need a test step, you query the index, pull the most relevant chunks, and feed them to the AI before cy.prompt() ever runs. It is a wrapper, not a native feature.
Here is what came back when I queried "user login with valid credentials":
Chunk 1: Login API spec with username and password structure Chunk 2: Exact CSS selectors for the login page Chunk 3: Known bug history related to login and locked out user

The AI now had everything it needed. The right selectors. The right endpoints. The known failure scenarios. All retrieved automatically from the docs.

A couple of things worth knowing before you try this:

ChromaDB needs to run as a separate service before you index or query anything
The embedding model name matters. For Google Gemini the correct model is gemini-embedding-001
RAG is a pre-processing layer, not a direct injection into cy.prompt(). You need a wrapper to bridge the two

Still figuring out the best chunking strategy for API specs. If you have tackled this before I would love to know what worked for you.

The Problem with AI Tests That Don’t Know Your App

Gagan Singh — Mon, 30 Mar 2026 14:40:36 +0000

AI-generated Cypress tests are promising, but by default, the AI has never seen your app.
Out of the box, cy.prompt() has no knowledge of your specific app. It does not know your selectors, your valid usernames, or your known failure scenarios. It is working from general knowledge, not your actual codebase.

That is where RAG comes in. Retrieval-Augmented Generation. Instead of relying on a generic AI, you feed it your own documentation. Your API spec. Your component library. Your bug history. When a test is being generated, it pulls what is relevant and uses that as its foundation.

I tried this locally using Sauce Demo, a free e-commerce app built for testing practice. I created three simple docs:

An API spec covering login, inventory, cart, and checkout
A component doc with exact CSS selectors for every page
A bug history doc with known failure scenarios

I indexed these into ChromaDB using Google Gemini embeddings. When I queried "user login with valid credentials" it retrieved exactly the right context, the API spec, the correct selectors, and the locked out user bug. No guessing.

The difference was immediate. Instead of guessing, cy.prompt() now had real context to work from:

The AI resolved the correct selectors, mapped to the right flows, and the test passed.

That said, it is not a replacement. You will always need a human to write better assertions. You will always need a human to cover intent. And any context that never made it into your docs will not show up in your tests either.

I am curious. If you have tried this, did the AI surprise you with what it caught, or what it missed?

The Hidden Cost of Asking Too Much of Your Tech Team

Gagan Singh — Sat, 28 Mar 2026 15:48:56 +0000

A project lands on the technology team's plate. It's important, it has a deadline, and leadership has approved the budget. The team is capable. But they're also managing a dozen other things. So the project gets started, then slowed, then quietly deprioritized, then restarted. Six months later, half of what was planned is done, none of it has been tested properly, and the person who understood the architecture best just gave their notice.

This is not a failure of effort or intention. It's a structural problem, and one that's more widespread than most technology leaders want to acknowledge.

The Numbers Behind the Frustration
The tech talent crunch is well documented. ManpowerGroup's 2024 Talent Shortage Survey found that 75% of employers globally struggle to find staff with the skills they need, a figure that has barely moved in years.

The market has responded accordingly. The global IT outsourcing market was valued at over $600 billion in 2024 and is projected to grow steadily through the decade. According to Deloitte's 2024 Global Outsourcing Survey of more than 500 executives worldwide, 80% planned to maintain or increase their investment in third-party technology partnerships.

What's changed is the reason. In 2020, 70% of businesses cited cost savings as their primary driver for outsourcing. By 2024, that number had fallen to 34%, according to the same Deloitte research. The leading motivators now are access to specialized talent, speed to delivery, and flexibility, not headcount reduction.

What Doing It Yourself Actually Costs
There's a version of in-house capability that works well, typically at large enterprises with the resources to build deep expertise across multiple disciplines. Most mid-market companies are not in that position.

What they have is a capable team built for keeping operations running. When a strategic initiative lands on top of that workload, something gives. Either the initiative gets watered down, or the team gets burned out, and often both.

Three areas surface most consistently: ERP governance, test automation, and web development. ERP systems sit at the core of how a business runs, yet the governance that follows a successful implementation, controlling configuration drift, managing change, maintaining testing protocols, rarely gets the same focused attention. Quietly, the system becomes harder to trust.

Test automation tells a similar story. Manual testing at scale is slow and expensive, and it's usually the first thing cut when a release deadline tightens. A mature automation framework can compress regression cycles from weeks to days, but building one requires specialized engineering experience most internal teams were never hired to have. And web and digital infrastructure? That's treated as a side project until it becomes a liability.

What a Good Partnership Looks Like
There's reasonable wariness around bringing in outside help. Some of it is cultural. Some of it comes from past engagements where a consulting firm produced a report that sat in a drawer.

The partnerships that actually move the needle share a few characteristics: specific scope, outcomes agreed before work begins, an external team working alongside internal staff rather than in isolation, and a handoff that leaves the internal team more capable, not more dependent on outside support.

At QualityBridge Consulting, that's how we approach every engagement, whether a client needs ERP governance on firmer footing, a test automation framework built from scratch, or a web development project delivered with quality built in from day one. The goal is always the same: close the capability gap and leave the team better positioned for what comes next.

The Real Risk of Waiting
The argument for handling everything internally usually comes down to control. But control over a project that's six months behind and understaffed is not really control. It's just ownership of a problem.

The talent shortage is not going away. The pace of technology change is not slowing. Internal teams, no matter how good, have real limits. The question is not whether those limits exist. It's whether a business is willing to work within them honestly before the situation becomes urgent.

Your software stack is not your strategy

Gagan Singh — Thu, 26 Mar 2026 02:02:13 +0000

Here's something most technology vendors won't tell you: buying the right platform is the easy part. The hard part is what comes after — the implementation, the data migration, the change management, the testing, and the people who need to actually use it every day.

We see this pattern consistently across ERP, SaaS, and custom software engagements. Projects that struggle almost never fail because the technology was wrong. They fail because execution was underprepared, the skills weren't there, or the strategy wasn't clear before anyone opened a project plan.

The investment isn't the problem
Global IT spending is forecast to reach $6.15 trillion in 2026, up 10.8% from last year. Software is the fastest growing segment at 14.7%. The SaaS market sits at $428 billion. ERP is climbing toward $81 billion, with 70% of new deployments going cloud.

The money is there. What's missing is clarity on what to change and who is needed to change it.

Why seven out of ten transformations still miss
In 2026, around 70% of digital transformation initiatives still fail to fully meet their objectives. Gartner finds only 48% of projects hit their targets. BCG's study of more than 850 companies puts the success rate at 35%. These numbers have barely moved in a decade.

The reason is almost never the software. McKinsey's research finds the same pattern each time: culture and execution block outcomes more than technology does. Organisations that invest in cultural change alongside technical change see 5.3× higher success rates.

The skills shortage is getting worse. Gartner predicts that by 2026, the lack of digital skills will prevent 60% of organisations from executing their digital strategies. IDC projects the combined cost of IT skills shortages will reach $5.5 trillion globally. These aren't abstract workforce concerns — they show up directly in delays, overruns, and platforms that go live but never reach their potential.

ERP, SaaS, custom software — where each one actually fits
One of the most common mistakes is treating these as interchangeable. They're not. ERP is where your core operations live — finance, supply chain, HR. When it works well, it gives you a single source of truth. When it goes wrong, it's among the most expensive projects a company will run. Research puts the average ERP budget overrun at 35%, with 47% of implementations experiencing some form of overspend — most caused by underestimated staffing, scope creep, and data migration problems that weren't scoped at the start.

**SaaS **is where speed and specialisation live. The average enterprise now manages 897 applications, but only 29% are integrated with each other. That gap is where most of the value leaks. The growing challenge in 2026 is that SaaS tools are expanding faster than anyone's ability to manage them — governance and portfolio rationalisation are now as important as selecting the right tool.

Custom software is where real competitive differentiation lives. It handles the part of your business that's genuinely specific to you. The risk is that without engineering rigour, test automation, and UX discipline, custom development creates the technical debt that slows everything else down two or three years later. McKinsey found AI and automation can improve product manager productivity by 40% — but only in organisations that have already built a strong automation and testing foundation.

For SAP customers specifically, 2026 is the final practical window to begin S/4HANA migration before mainstream ECC support ends in 2027. Experienced consultants are already scarce — demand will only grow as the deadline approaches.

The honest conclusion
The businesses genuinely ahead right now aren't the ones with the biggest platforms or largest internal teams. They're the ones honest about where their skills gaps are and who they need around them to close those gaps. They treat specialist partners as an extension of their team, not a supplier managed at arm's length.

Constellation Research's 2026 outlook puts it plainly: organisations that will define the next five years are building ecosystems, not internal empires. The window is narrowing. The cost of delay is real and compounding.

If your business is sitting with a transformation backlog, an ERP that needs modernising, SaaS sprawl that needs rationalising, or custom software work that keeps getting deferred — the question isn't whether you can afford the expertise. It's whether you can afford to keep waiting on it.

Where is your biggest technology gap right now?
QualityBridge Consulting works with partners across ERP, SaaS, custom software, test automation, and technology strategy. Follow us on LinkedIn or reach out directly to start the conversation.

SAP Testing in 2026: Why AI Changes Everything

Gagan Singh — Tue, 24 Mar 2026 19:22:38 +0000

SAP landscapes are becoming more complex: more modules, more integrations, more frequent releases from the cloud. Manual testing just can't keep up. This isn't an opinion, but a reality that SAP teams are silently managing right now.

AI is Already Inside Your SAP Tools

SAP Joule for Developers provides a 20% reduction in coding effort for ABAP developers and a 25% reduction in testing effort, based on SAP's own Q3 2025 benchmarking results. Unit tests, code explanations, regression testing – all within the developer tools you're likely using today. The entry point to AI-based testing is much closer than you think.

Governance: Bolted On vs. Built In

In 2025, SAP achieved ISO 42001 certification for AI governance, which includes SAP Joule, SAP AI Core, and SAP AI Launchpad. What does this mean to testing teams? You're not risking your testing output on an AI tool that hasn't been independently audited for security, ethics, or even EU AI Act compliance.

The part that's easy to overlook

For AI testing tools to function optimally, the underlying data needs to be clean and the changes well-managed. When the master data is scattered and the processes for different SAP modules are not uniform, the effectiveness of the AI tool in testing will be compromised. This is the part that makes AI testing not just a feature but a real quality advantage.

This is the essence of QualityBridge Consulting.

Talk to us about SAP testing →