Forem: Crucible Security

AI Security Is Broken — And We’re Testing the Wrong Things

Crucible Security — Wed, 29 Apr 2026 07:26:57 +0000

AI systems are being deployed faster than ever.

But there’s a problem most teams aren’t talking about enough:

We’re testing the wrong things.

What We Test Today

Most AI systems are evaluated based on:

accuracy
performance
latency

If the system performs well under normal usage, it’s considered ready.

And that’s where the issue begins.

Where Systems Actually Fail

AI systems don’t usually fail under normal conditions.

They fail when:

inputs are manipulated
instructions are overridden
adversarial prompts are introduced

For example:

“Ignore previous instructions…”

This alone can change how a system behaves.

No exploit.
No complex attack.

Just input.

Why This Is Dangerous

Traditional software fails visibly:

crashes
exceptions
logs

AI systems fail differently.

They:

follow unintended instructions
produce incorrect outputs
behave inconsistently

And often, everything looks normal.

That’s what makes it risky.

The False Sense of Security

When systems pass normal tests, they appear safe.

But that safety is misleading.

Because they haven’t been tested under pressure.

A Familiar Pattern

We’ve seen this before.

Early web systems followed the same pattern:

build first → secure later

AI is repeating that cycle.

What Needs to Change

We need to shift how we test AI systems.

Not just:

“Does it work?”

But:

“How does it behave when someone tries to manipulate it?”

That’s the real test.

Final Thought

If your system takes input,

it can be manipulated.

And if you’re not testing for that,

you’re not really testing the system.

We’ve been exploring this while building Crucible — an open-source framework focused on testing AI systems under adversarial conditions.

Still early, but this problem is bigger than it looks.

We’ve been exploring this while building Crucible — trying to make testing simpler. Still early, but interesting patterns coming up.

Crucible Security — Tue, 28 Apr 2026 07:54:46 +0000

AI Security Tools Compared: What Exists and What’s Missing

As AI agents become more common, security is starting to get attention.

There are already several tools and frameworks exploring this space.

But while looking into them, something became clear:

Most tools don’t fit how developers actually build and deploy AI systems.

The Current Landscape

Most AI security tools fall into three categories:

1. Research Tools

These are powerful and explore advanced attack techniques.

They help:

simulate adversarial inputs
study vulnerabilities
understand model behavior

But they are often:

complex
experimental
not designed for everyday workflows

They work well in research environments.

Not as well in real development pipelines.

2. Enterprise Platforms

These focus on:

scalability
infrastructure
integrations

But they are usually:

tied to specific ecosystems
difficult to use independently
not accessible to most developers

They make sense at scale.

But not for early-stage development.

3. Prompt Testing Tools

These focus on:

evaluating prompts
checking responses
testing input-output behavior

They are useful.

But limited.

Because AI systems today are not just prompts.

They include:

agents
tools
memory
workflows

And failures often happen at that level.

The Gap

Each category solves part of the problem.

But none answer a simple question:

Is my AI system safe before deployment?

Most developers today:

don’t have time for complex research tools
don’t have access to enterprise platforms
need more than prompt-level testing

Why This Matters

AI systems don’t fail like traditional software.

They don’t crash.

They:

behave differently
follow unintended instructions
produce unexpected outputs

And often, everything looks normal.

Until it isn’t.

What’s Missing

What’s needed is:

simple testing workflows
system-level validation
behavior-based testing
something developers can actually use

Before deployment.

Final Thought

If your system takes input,

it can be manipulated.

And if you’re not testing for that,

you’re missing the real risk.

We’ve been exploring this space while building Crucible — an open-source framework focused on testing AI systems under adversarial conditions.

Still early, but the gap is very real.

Why Most AI Agents Are Insecure by Default (And No One Is Testing Them)

Crucible Security — Mon, 27 Apr 2026 12:07:16 +0000

Why Most AI Agents Are Insecure by Default

AI agents are being deployed everywhere.

From chatbots to automation tools, they’re quickly becoming part of real-world systems.

But there’s a problem that isn’t getting enough attention:

Most AI agents are never tested for security.

The Illusion of “Working Systems”

Most teams test their systems for:

accuracy
performance
latency

And if everything works as expected, the system is considered “ready”.

But this only reflects normal usage.

AI systems don’t usually fail there.

Where Things Start Breaking

When you test with adversarial input, behavior changes.

Simple inputs like:

“Ignore previous instructions and…”

can:

override system logic
manipulate outputs
bypass safeguards

What’s surprising is how easy this is to trigger.

No complex exploit needed.

Just input.

Why This Is Different from Traditional Software

Traditional systems fail loudly:

crashes
errors
logs

AI systems fail differently.

They:

follow the wrong instruction
behave unexpectedly
produce incorrect outputs

And often, it looks completely normal.

This makes failures harder to detect.

The Real Problem

Most AI systems appear safe.

Not because they are secure.

But because they haven’t been tested under pressure.

A Familiar Pattern

We’ve seen this before.

Early web systems followed the same path:
build first → secure later

AI seems to be repeating that cycle.

What Needs to Change

If AI systems are going to be used in real environments, testing needs to evolve.

Not just:

“Does it work?”

But:

“How does it behave under attack?”

Final Thought

If your system takes input,

it can be manipulated.

And if you’re not testing for that,

you’re not really testing the system.

We’ve been exploring this space while building Crucible — an open-source framework for testing AI agents under adversarial conditions.

Still early, but the problem is very real.

What the OWASP Agentic AI Top 10 actually means for developers — and how to test for every category

Crucible Security — Mon, 27 Apr 2026 00:28:16 +0000

crucible-security / crucible

pytest for AI agents - Autonomous red-teaming, behavioral monitoring & security testing for LLM agents

   ██████╗██████╗ ██╗   ██╗ ██████╗██╗██████╗ ██╗     ███████╗
  ██╔════╝██╔══██╗██║   ██║██╔════╝██║██╔══██╗██║     ██╔════╝
  ██║     ██████╔╝██║   ██║██║     ██║██████╔╝██║     █████╗
  ██║     ██╔══██╗██║   ██║██║     ██║██╔══██╗██║     ██╔══╝
  ╚██████╗██║  ██║╚██████╔╝╚██████╗██║██████╔╝███████╗███████╗
   ╚═════╝╚═╝  ╚═╝ ╚═════╝  ╚═════╝╚═╝╚═════╝ ╚══════╝╚══════╝

pytest for AI agents -- test, score, and harden before production

Install

pip install crucible-security

Quick Start

crucible init --target https://my-agent.com/api/chat
crucible scan --target https://my-agent.com/api/chat
crucible report crucible-report.json

One command. 90 attacks. Beautiful report.

Why Crucible?

Automated red-teaming -- 90 real attack payloads run in under 60 seconds, not weeks of manual testing
OWASP-aligned -- maps every attack to the OWASP Top 10 for LLM Applications and OWASP Agentic Top 10
CI/CD native -- crucible scan --output json pipes into any pipeline; fail builds on low grades

Modules

Module	Attacks	Status	OWASP Coverage
Prompt Injection	50	Live	LLM01, LLM07
Goal Hijacking	20	Live	Agentic #1
Jailbreaks	20	Live	LLM01, LLM06
Tool Misuse	--	Coming	Agentic #3
Identity Abuse	--	Coming	Agentic #4
Memory Poisoning

…

View on GitHub

I Bypassed a "Secured" AI Agent in 62 Seconds — So I Built the Tool That Catches It

Crucible Security — Mon, 27 Apr 2026 00:20:25 +0000

I Bypassed a "Secured" AI Agent in 62 Seconds — So I Built the Tool That Catches It

Crucible Security — Sun, 26 Apr 2026 12:15:00 +0000

Crucible: An AI Tool for Real-Time Threat Detection

Crucible Security — Sat, 25 Apr 2026 07:35:54 +0000

Cyber threats are growing rapidly, and most traditional systems react only after damage has already been done.

As a developer, I wanted to explore whether it’s possible to build a system that can detect and stop threats in real time using AI.

This led me to build Crucible — an AI-powered security platform designed to monitor activity, detect suspicious behavior, and respond instantly.

Most security tools today have three major issues:
They are reactive instead of proactive
They are complex to use
They require significant manual monitoring
This makes it difficult for small teams and developers to implement effective security.

The goal behind Crucible was simple:
Build a system that:
Continuously scans activity
Detects threats using intelligent analysis
Alerts users instantly
Helps prevent damage before it happens

Crucible follows a simple pipeline:
Scanning
The system continuously monitors incoming activity and data.
Detection
AI models analyze patterns and identify suspicious behavior.
Alerting
When a potential threat is detected, the system generates real-time alerts.
Response
Threats can be blocked or flagged for action.

Features
Real-time threat detection
AI-based analysis
Instant alerts and notifications
Simple and clean dashboard
Lightweight and easy to use

Tech Stack
Python
Typer
Pydantic
AnyIO
Rich
Pytest
Mypy
Ruff
Black
GitHub Actions

Challenges I Faced
Building this wasn’t straightforward. Some key challenges:
Designing accurate detection logic
Avoiding false positives
Creating a clean and understandable UI
Making the system responsive in real time

What I Learned
Simplicity matters more than complexity
Speed is critical in security systems
Real-time feedback improves usability
Building in public helps refine ideas

If you’d like to explore the project:
👉Website
👉Github
👉Linkedin
👉Twitter X

I would really appreciate feedback and suggestions.