80% of AI Projects in Banks Fail - Here’s Why (And How We Fixed It)

Swapneswar Sundar Ray — Sun, 26 Apr 2026 00:43:12 +0000

Banks invested billions in AI.

Fraud detection.

Credit scoring.

Customer experience.

Risk modeling.

The promise was massive.

But here’s the uncomfortable truth:

Most AI projects never make it to production.

Not because the models don’t work.

But because everything around them fails.

From my experience building AI systems in banking, the pattern is always the same.

The Real Problem

AI doesn’t fail at the model level.

It fails at the system level.

Let’s break it down.

Where AI Projects Break

1. The “Pilot Trap”

Every bank has this story:

Build a model
It works in a demo
Leadership is impressed

And then… silence.

Why?

No production infrastructure
No ownership after POC
No integration roadmap

Result:

Great demos. Zero impact.

2. Legacy Systems Kill Momentum

AI needs:

Clean data
Real-time access
APIs

Banks often have:

Data silos
Batch pipelines
Fragile integrations

AI becomes a side layer, not core infrastructure.

3. Data Reality Check

Everyone assumes:

“We have years of data—we’re ready.”

Reality:

Missing fields
Inconsistent formats
Historical bias

Garbage in → Garbage out

4. Compliance Slows Everything

Banking isn’t a startup.

Every model must be:

Explainable
Auditable
Fair

What happens:

Models get rejected late
Legal blocks rollout
Risk teams force simplification

Speed → Gone

Momentum → Gone

5. Business vs Tech Misalignment

AI teams build models.

Business teams expect ROI.

But:

No shared KPIs
No domain alignment
No clear success metric

Misalignment = failure.

6. No MLOps = No Product

Most teams stop at:

“Model trained”

But production needs:

Monitoring
Drift detection
Retraining
Versioning

Without MLOps, models decay fast.

The Reality (Simple View)

Typical AI Project Flow in Banks:

Idea → Pilot → Demo → Approval → Stuck → Dead

What Actually Works:

Idea → Data → Architecture → Integration → Deployment → Monitoring → Impact

What Actually Worked (In Production)

Here’s what changed everything for us:

1. Start With Business, Not Models

Instead of:

“Let’s build AI”

We asked:

“What business problem matters?”

Examples:

Reduce fraud loss by X%
Improve loan approval speed

AI became outcome-driven, not experiment-driven.

2. Fix Data Before Models

We invested in:

Clean pipelines
Standard schemas
Strong governance

Data became usable and reliable.

3. Build for Production From Day One

No throwaway pilots.

Every model had:

API endpoints
Integration plan
Deployment path

If it can’t scale, don’t build it.

4. Bring Compliance Early

Instead of late approvals:

Risk teams involved from day one
Explainability built-in
Documentation automated

Compliance became a partner, not a blocker.

5. Build Cross-Functional Teams

We combined:

Engineers
Data scientists
Domain experts
Risk & legal

Decisions got faster, clearer, and aligned.

6. Invest in MLOps

We implemented:

CI/CD for models
Monitoring dashboards
Automated retraining

Models stayed reliable in production.

The Outcome

Faster deployments
Lower failure rates
Higher reliability
Real business impact

Most importantly:

AI became a capability — not an experiment.

Final Thought

AI in banking isn’t failing because it’s too complex.

It’s failing because:

Organizations treat AI like a project.

Not like infrastructure.

Until that changes…

Failure rates won’t.

Stop Building One Giant Prompt: A Better Way to Design LLM Systems

Swapneswar Sundar Ray — Sat, 25 Apr 2026 18:56:55 +0000

## Most early LLM apps start the same way:

“Let’s just put everything into one prompt and let the model handle it.”

So we write a prompt that tries to:

validate input
transform data
generate output
summarize
add reasoning
handle edge cases

…and somehow do it all in one call.

It works—until it doesn’t.

The Problem with “God Prompts”

As the prompt grows:

Instructions start conflicting
Context becomes noisy
Accuracy drops
Outputs become inconsistent

You end up with:

a very expensive confusion engine

I’ve hit this multiple times while building AI systems.

What’s Actually Happening

You’re increasing what I call LLM cognitive load.

The more responsibilities you push into a single call:

the harder it is for the model to prioritize
the easier it is to miss instructions
the more likely it is to hallucinate

Even with better models, this pattern doesn’t go away.

A Better Approach: Think Like a System Designer

Instead of one big prompt, break the problem into smaller, focused steps.

Validate + transform + summarize + generate + explain everything

Do this:

Validation step (code)
Extraction step (LLM)
Transformation step (code or LLM)
Generation step (LLM)
Formatting step (code)

Use the Right Tool for the Right Job

Let code handle:

validation
parsing
routing
rules
state

Let LLM handle:

reasoning
interpretation
summarization
ambiguity

Treat LLM Calls Like Microservices

This mindset shift helped me a lot:

Each LLM call should have a single responsibility

Small input
Clear task
Predictable output

Then orchestrate them together.

Real-World Example

While working on API automation systems, we initially tried:

one prompt to validate specs + generate APIs + create mock data

It became unstable very quickly.

Splitting it into:

validation module
generation module
mock data module

made the system far more reliable.

LLMs are powerful—but they’re not a replacement for system design.

“Just add AI” is not an architecture pattern.

Design your system first.
Then use AI where it actually adds value.

Forem: Swapneswar Sundar Ray