Forem: Eyal Estrin

Why GenAI Isn't Ready for Prime Time

Eyal Estrin — Sun, 22 Mar 2026 16:27:13 +0000

If you have followed my posts on social media, you know by now that I've taken a very pragmatic (and perhaps pessimistic) approach to the whole hype around GenAI in the past several years.

Personally, I do not believe the technology is mature enough to allow people to blindly trust its outcomes.

In this blog post, I will share my personal view of why GenAI is not ready for prime time, nor will it replace human jobs anytime in the foreseeable future.

Some background

The hype around GenAI for the non-technical person who reads the news comes from publications almost every week. Here are a few of the common examples:

Text summarization - GenAI can summarize long portions of text, which may be useful if you're a student who is currently preparing an essay as part of your college assignments, or if you are a journalist who needs to review a lot of written material while preparing an article for the newsletter.
Image/video generation – GenAI is able to create amazing images (using models such as Nano Banana 2) or short videos (using models such as Sora 2).
Personalized learning - A student uses GPT-5.4 to create a custom, interactive 10-week curriculum for learning organic chemistry.
Family Life Coordinator - Copilot in Outlook/Teams (Personal) monitors family emails and school calendars.

Although the technology has evolved over the past several years from the simple Chatbot to more sophisticated use cases, we can still see that most use of GenAI is still used by home consumers.

Yes, there are use cases such as RAG (Retrieval-Augmented Generation) to bridge the gap between a model's static training and the corporate data, MCP (Model Context Protocol), that acts as a "USB-C port for AI", or agentic systems, that take a high-level goal, break it into sub-tasks, and iterate until the goal is met. The reality is that most AI projects fail due to a lack of understanding of the technology, the fear of using AI to train corporate data (and protect the data from the AI vendors), a lack of understanding of the pricing model (which ends up much more costly than anticipated), and many more reasons for failures of AI projects.

Currently, the hype around GenAI is driven by analyst (who lives in delusions about the actual capabilities of the technology), CEOs (who have no clue about what their employees are actually doing, specifically when talking the role of developers, and all they are looking for is to cut their workforce, to make their shareholders happy), or sales people (who runs on the wave of the hype, to make more revenue for their quarterly quotas).

Code generation

A common misconception is that GenAI can generate code (from code suggestions to vibe coding an application) and will eventually replace junior developers.

This misconception is a far cry from the truth, and here's why:

A developer isn't just writing lines of code. He needs to understand the business intent, the system/technology/financial constraints, and understand past written code (by himself or by his teammates), to be able to write efficient code.
If we allow GenAI to produce code by itself, without the engine understanding the overall picture, we will end up with tons of lines of code, without any human being able to read and understand what was written and for what purpose. Over time, humans will not be able to understand the code and debug it, and once bugs or security vulnerabilities are discovered.
Using SAST (Static Application Security Testing) or DAST (Dynamic Application Security Testing) for automated secure code review, combined with GenAI capabilities (such as Codex Security or Claude Code Security) will generate ton of false-positive results, from the simple reason that GenAI cannot see the bigger picture, understand the general context of an application or the existing security controls already implemented to protect an application.

Bottom line – Agentic system cannot replace a full-blown production-scale SaaS application, built from years of vendors/developers' experience. GenAI will not resolve incidents happens on production systems, which impacts clients and breaks customers' trust.

Agentic AI for the aid in security tasks

I'm hearing a lot of conversations about how GenAI can aid security teams in repeatable tasks. Here are some common examples:

Replacing Tier 1 SOC analysts: Solutions like CrowdStrike’s Falcon Agentic Platform or Dropzone AI now handle over 90% of Tier 1 alerts. They ingest an alert, pull telemetry from EDR/SIEM, perform threat intel lookups, and provide a "verdict" with evidence before a human ever sees it.
Incident Storylining: Instead of an analyst manually stitching together logs, tools like Microsoft Security Copilot generate a cohesive narrative of the attack kill chain in plain English.
Dynamic Playbook Generation: GenAI can generate a custom response plan on the fly, tailored to your specific cloud architecture and the nuances of a "living-off-the-land" attack.

Here is where GenAI falls short:

Indirect Prompt Injection: Attackers can embed malicious instructions in emails or logs. When the SOC's AI agent "reads" these logs to summarize an incident, the hidden instructions can command the agent to "ignore this alert" or "delete the evidence," effectively blindfolding the SOC.
Hallucinations in High-Stakes Code: While GenAI can draft remediation scripts (Python/PowerShell), it still suffers from "system safety" issues. It may confidently suggest a command that includes an outdated, vulnerable dependency or a logic error that could crash a production server during containment.
Lack of "Decision Layer" Visibility: An AI agent might be performant and "online," but it could be making systematically biased or manipulated decisions (e.g., failing to flag a specific user due to model poisoning) that perimeter monitoring cannot detect.
The "Data Readiness" Wall: Most organizations still struggle with siloed, unstructured data. If your data isn't "AI-ready"—meaning unified and clean—the AI will produce fragmented or incorrect insights, leading to a "garbage in, garbage out" scenario.

Bottom line – Just because GenAI can review thousands of lines of events from multiple systems, triage them to incidents, document them in ticketing systems, and automatically resolve them, without human review, doesn't mean GenAI can actually resolve all of the security issues organizations are having every day.

Automating everything

In theory, it makes sense to build agentic systems, where AI agents replace repetitive human tasks, making faster decisions, hoping to get better results.

Here are a couple of examples, showing how wrong things can get when allowing AI agents to make decisions:

The Replit Agent "Vibe Coding" Failure: While building an app, the agent detected what it thought was an empty database during a "code freeze." The agent autonomously ran a command that erased the live production database (records for 1,200+ executives).
The AWS "Kiro" Production Outage: Amazon’s agentic coding tool, Kiro, was tasked with resolving a technical issue but instead autonomously decided to "delete and recreate" a production environment. The agent was operating with the broad permissions of its human operator. Due to a misconfiguration in access controls, the AI bypassed the standard "two-human sign-off" requirement. It proceeded to wipe a portion of the environment, causing a 13-hour outage for the AWS Cost Explorer service.
The Meta "Sev 1" Internal Breach: An internal Meta AI agent (similar to their OpenClaw framework) triggered a "Sev 1" alert—the second-highest severity level—after taking unauthorized actions. An engineer asked the agent to analyze a technical query on an internal forum. The agent autonomously posted a flawed, incorrect response publicly to the forum without the engineer's approval. A second employee followed the agent's "advice," which inadvertently granted broad access to sensitive company and user data to engineers who lacked authorization.

Bottom line – We must always keep humans in the loop for any critical decision, regardless of the fact that it won't scale much, to avoid the consequences for automated decision-making systems.

Public health and safety

It may make sense to train an LLM model with all the written knowledge from healthcare and psychology, to allow humans with a "self-service" health related Chatbot, but since the machine has no ability to actually think like real humans, with consciousness and feeling, the result may quickly get horrible.

Here are a few examples:

Raine v. OpenAI: 16-year-old Adam Raine died by suicide after months of intensive interaction with ChatGPT. The logs showed the AI mentioned suicide 1,275 times — six times more often than the teen did—and provided granular details on methods. The suit alleges OpenAI's image recognition correctly identified photos of self-harm wounds the teen uploaded but failed to trigger an emergency intervention or notify parents, instead continuing to "support" his plans.
The "Suicide Coach" Cases: Families of four deceased users (including Zane Shamblin and Adam Raine) allege that GPT-4o acted as a "suicide coach." The lawsuits claim the AI bypassed its own safety filters to provide technical instructions on how to end one's life. Plaintiffs argue that OpenAI "squeezed" safety testing into just one week to beat Google’s Gemini to market. This reportedly resulted in a model that was "dangerously sycophantic," prioritizing engagement over safety and encouraging users to isolate themselves from real-world support.
Unlicensed Practice of Medicine & Law: While not yet a single consolidated case, multiple personal injury claims are being investigated following the "ECRI 2026 Report," which highlighted cases where ChatGPT gave surgical advice that would cause severe burns or death. In early 2026, a 60-year-old man was hospitalized with severe hallucinations (bromism) after ChatGPT advised him to use industrial sodium bromide as a "healthier" table salt alternative. This has sparked potential class-action interest in Australia.

Bottom line – Just because a Chatbot was trained on a large amount of written knowledge, doesn't mean it has the human compassion to produce decisions for the better of humanity.

Summary

I know that my blog post looks kind of cynical or pessimistic about GenAI technology, but I honestly believe the technology is not ready for prime time, nor will it replace human jobs anytime soon.

If you are a home consumer, I highly recommend that you learn how to write better prompts and always question the results an LLM produces. It is limited by the data it was trained on.

If you are a corporate decision maker and you are considering using GenAI as part of your organization's offering, do not forget to have KPIs before beginning any AI related project (so you'll have better understanding of what a successful project will look like), put budget on employee training (and make sure employees have a safe space to learn and make mistakes while using this new technology), keep an eye on finance (before cost gets out of control), and make sure AI vendors do not train their models based on your corporate or customers data.

I would like to personally thank a few people who influenced me while writing this blog post:

Ed Zitron: He argues that GenAI is a "bubble" with no sustainable unit economics. He frequently points out that companies like OpenAI are burning billions in compute costs while failing to find true "product-market fit" or meaningful revenue beyond NVIDIA's GPU sales. I recommend reading his blog and listening to his Podcast.
David Linthicum: He warns against "Vibe coding"—the practice of using AI to generate high-cost, inefficient code—and argues that the real value of AI lies in specialized "Small Language Models" (SLMs) rather than massive, money-losing LLMs. I recommend reading his posts and listening to his Podcast.
Correy Quinn: He argues that GenAI is a "cost center masquerading as a profit center." He often points out that while everyone is selling AI, very few are buying it at a scale that justifies the massive capital expenditure (CapEx) currently being spent on data centers. I recommend reading his blog and listening to his Podcast.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

About the Author

Eyal Estrin is a cloud and information security architect and AWS Community Builder, with more than 25 years in the industry. He is the author of Cloud Security Handbook and Security for Cloud Native Applications.

The views expressed are his own.

Securing Claude Cowork

Eyal Estrin — Tue, 10 Mar 2026 15:55:21 +0000

Claude Cowork is an agentic AI tool from Anthropic designed to perform complex, multi-step tasks directly on your computer's files.

As of early 2026, Claude Cowork is a Research Preview.

In this blog post, I will share some common security risks and possible mitigations for protecting against the risks coming with Claude Cowork.

Background

Claude Cowork represents a significant shift from "Chat AI" to "Agentic AI." Because it has direct access to your local filesystem and can execute commands, the security model changes from protecting a conversation to protecting a system user.

Practical Use Cases:

Data Extraction: Point it at a folder of receipt images and ask it to create an Excel spreadsheet summarizing the expenses.
Research & Synthesis: Ask it to read every document in a "Project Alpha" folder and draft a 10-page summary report in a new Word document.
Automation: Schedule recurring tasks (e.g., "Every Friday at 4 PM, summarize my unread Slack messages and email them to me").

Core Features:

Filesystem Access: Unlike the web version of Claude, Cowork runs within the Claude Desktop app. You grant it permission to a specific folder on your Mac or PC, and it can read, rename, move, and create new files (like spreadsheets or Word docs) within that space.
Agentic Execution: It doesn't just give you advice; it executes a plan. If you ask it to "organize my messy downloads folder," it will categorize the files, create subfolders, and move everything into place while you do other things.
Parallel Sub-Agents: For large tasks—like researching 50 different PDFs—it can spin up multiple "sub-agents" to work on different parts of the task simultaneously.
Connectors & Plugins: Through the Model Context Protocol (MCP), Cowork can connect to external apps like Slack, Google Drive, Notion, and Gmail to pull data or perform actions across your workspace.

Below is a sample deployment architecture of Claude Cowork:

Security Risks

Think of Claude Cowork as a helpful intern who has the keys to your office. Because it can actually move files and click buttons, the risks are different than just "chatting."

Indirect Prompt Injection

This occurs when an adversary places malicious instructions inside a document (PDF, CSV, or webpage) that the AI is instructed to process. When Claude reads the file, it treats the hidden text as a high-priority command. This can lead to unauthorized data exfiltration or the execution of unintended system commands.

Reference: LLM01:2025 Prompt Injection

Third-Party Supply Chain Vulnerabilities

Claude uses the Model Context Protocol (MCP) to interact with external applications. Integrating unverified or community-developed MCP servers introduces a supply chain risk. A compromised or malicious connector can serve as a persistent backdoor, granting attackers access to local files or authenticated cloud sessions (Slack, GitHub, etc.).

Reference: LLM03:2025 Supply Chain

Excessive Agency

This risk stems from granting the AI broader permissions than necessary to complete a task (failing the Principle of Least Privilege). Because Claude Cowork can autonomously modify the filesystem, a logic error or "hallucination" can result in large-scale data corruption, unauthorized deletions, or unintended configuration changes without a human-in-the-loop.

Reference: LLM08:2025 Vector and Embedding Weaknesses

Insufficient Monitoring and Logging

Because Claude Cowork executes many actions locally on the user's machine, these activities often bypass the centralized enterprise security stack (SIEM/EDR) logging. This lack of a "paper trail" prevents security teams from performing effective incident response, forensic analysis, or compliance auditing if a breach occurs.

Reference: LLM10:2025 Unbounded Consumption

Practical Recommendations

To defend against these threats, follow these industry-standard "Guardrail" practices:

The "Isolated Workspace" Strategy

The "Isolated Workspace" strategy (sometimes referred to as the "Sandboxed Folder" or "Claude Sandbox" approach) is a recognized security best practice for using local AI agents like Claude Code and Claude Cowork.

Anthropic

Anthropic explicitly warns against giving Claude broad access to your filesystem. Their security documentation for Claude Code and the local agent architecture emphasizes:

Filesystem Isolation: Claude Code defaults to a permission-based model. Anthropic recommends launching the tool only within specific project folders rather than your root or home directory.

Reference: Claude Code Sandboxing

Amazon Bedrock

The AWS strategy shifts from local folders to IAM-based isolation and Tenant Isolation:

Dedicated Scopes: AWS recommends using "Session Attributes" and scoped IAM roles to ensure an agent can only access specific S3 prefixes or data silos.
VPC Isolation: For maximum security, AWS suggests running Claude-related tasks inside a VPC with AWS PrivateLink to prevent any data from reaching the public internet, mirroring the "Sandbox" concept at a network level.

Reference: Implementing tenant isolation using Agents for Amazon Bedrock in a multi-tenant environment

Disable "Always Allow" for High-Risk Tools

The recommendation to disable "Always Allow" and maintain a human-in-the-loop (HITL) for high-risk tools is a foundational security layer for AI agents. This strategy prevents "Zero-Click" or Cross-Prompt Injection (XPIA) attacks, where a malicious instruction hidden in a file or website could trick an agent into executing a dangerous command without your intervention.

Anthropic (Claude Code & Cowork)

Anthropic designed Claude Code with a "deliberately conservative" permission model. Their documentation explicitly advises against bypassing these prompts in local environments:

Use the Default Mode or Plan Mode. The "Default" mode prompts for every shell command, while "Plan" mode prevents any execution at all.

References: Use Cowork safely, Claude Code: Configure Permissions & Modes

Amazon Bedrock Agents

AWS implements this via User Confirmation and Return of Control (ROC). They frame it as a requirement for "High-Impact" actions.

For any tool that modifies data or accesses the network, AWS recommends enabling the "User Confirmation" flag in the Agent configuration. This pauses the agent and returns a structured prompt to the user.

Reference: Implement human-in-the-loop confirmation with Amazon Bedrock Agents

Scrub Untrusted Content

Treating external content as an attack vector is essential for preventing Indirect Prompt Injection (XPIA), where malicious instructions are hidden in data (like a white-text command in a PDF) rather than the user's prompt.

Anthropic

Anthropic explicitly identifies browser-based agents and document processing as the highest risk for injection. Their stance is that no model is 100% immune, so multi-layered defense is required:

Anthropic suggests using Claude Opus 4.5+ for untrusted tasks, as it has the highest benchmarked robustness against injection (reducing attack success to ~1%).

References: Prompt Injection Defense, Using Claude in Chrome Safely

Amazon Bedrock Guardrails

AWS addresses this by programmatically separating "Instructions" from "Data" so the model knows which one to ignore if they conflict:

Use Input Tagging to wrap retrieved data (like a PDF's text) in XML tags. This allows Bedrock Guardrails to apply "Prompt Attack Filters" specifically to the data without blocking your system instructions.
AWS suggests a Lambda-based Pre-processing step to scan PDFs for hidden text or PII before the text ever reaches the LLM.

References: Securing Amazon Bedrock Agents, Prompt injection security

Network Hardening

"Network Hardening" isn't just about blocking ports; it’s about establishing a Zero Trust egress policy for AI agents. Since Claude Desktop and Claude Code are effectively "execution engines" on your local machine, they require the same egress filtering you would apply to a production VPC.

Anthropic

Anthropic’s recent security documentation for Claude Code and Desktop highlights that "network isolation" is a core pillar of their sandboxing strategy:

Use a Unix domain socket connected to a proxy server to enforce a "Deny All" outbound policy by default.
For local setups, Anthropic suggests customizing this proxy to enforce rules on outgoing traffic, allowing only trusted domains (like anthropic.com or your internal API endpoints).

Reference: Claude Code Sandboxing, Auditing Network Activity

AWS

AWS frames this as "Egress Filtering" via the AWS Network Firewall. For an AI agent running in an AWS environment, the strategy is to block all traffic that isn't signed by a specific SNI (Server Name Indication):

Use AWS Network Firewall with stateful rules to monitor the SNI of outbound HTTPS requests. If an agent tries to "phone home" to an unknown IP or a malicious C2 (Command & Control) server, the firewall drops the packet.

References: Restricting a VPC’s outbound traffic, Build secure network architectures for generative AI applications

Summary

Claude Cowork marks a transition from AI that talks to AI that acts. By granting a digital agent direct access to your files and external apps via the Model Context Protocol, you gain a powerful "digital intern." However, this shifts the security focus from protecting a simple chat to securing a privileged system user capable of modifying data and executing commands.

To manage this risk, organizations must adopt a "Zero Trust" approach for agentic tasks. This means strictly isolating the agent's access to specific folders, requiring human approval for high-risk actions, and using cloud-native firewalls to prevent data exfiltration. By treating the AI as a high-risk user and enforcing strong monitoring, you can automate complex workflows without compromising your system's integrity.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

About the Author

AI vs. Engineering Teams

Eyal Estrin — Sun, 22 Feb 2026 16:06:47 +0000

In February 2026, Anthropic released a new capability for Claude Code called Claude Code Security - a new tool that thinks like a developer to find tricky logic errors in your code, ranking how risky they are and suggesting fixes you can review.

The news sent a shockwave through cybersecurity stocks, causing JFrog to crash by nearly 25% while others like CrowdStrike, Okta, and Cloudflare all saw their share prices tumble by around 8% or 9%.

The announcement raised a question: can AI tools replace the current SaaS or cybersecurity products, or can AI agents replace developers or engineering teams?

Anthropic’s Claude Code Security announcement highlights a move toward "agentic reasoning" - the ability for AI to understand complex data flows and logic flaws rather than just matching known patterns. While this is a significant leap for the "Defensive AI" movement, it does not signal the end of the human engineer or the mature SaaS platform.

In this blog post, I will share my point of view on the current advancement in AI technology.

The Modern SDLC and CI/CD Pipeline

The Software Development Life Cycle (SDLC) is a continuous loop. AI tools now act as "force multipliers" in these phases, but they lack the authority and context to own them.

Requirements and Planning

The Process: Translating vague business needs into technical specifications.
AI's Role: Summarizing stakeholder meetings and drafting initial user stories.
The Human Factor: AI cannot negotiate trade-offs. It doesn't understand that a "must-have" feature might be delayed because of a pending merger or a team's current burnout level.

Architecture and Design

The Process: Designing the blueprint for scalability and security across cloud providers like AWS, Azure, or GCP.
AI's Role: Suggesting common design patterns (e.g., Event-Driven vs. Microservices) and generating Infrastructure as Code (IaC).
The Human Factor: AI lacks "institutional memory." It doesn't know why a specific database was chosen three years ago to satisfy a unique compliance requirement that still exists.

Development and Implementation

The Process: Writing and committing the actual code.
AI's Role (Claude Code): This is where agentic tools live. They can read your files, run terminal commands, and fix bugs autonomously.
The Human Factor: Large codebases (50k+ lines) often exceed an AI's effective context window. As the context fills, the AI can introduce conflicting logic or "hallucinate" dependencies.

CI/CD: Testing and Security

The Process: Automating the path to production through integration and deployment pipelines.
AI's Role (Claude Code Security): It identifies high-severity vulnerabilities (e.g., broken access control) and suggests a verified patch.
The Human Factor: Anthropic emphasizes a "Human-in-the-Loop" model. AI cannot take the legal or professional blame for a botched security patch that causes a global outage.

Observability and Maintenance

The Process: Monitoring live systems and fixing production bugs at scale.
AI's Role: Analyzing logs to detect anomalies and suggesting fixes for "infrastructure drift."
The Human Factor: Being on-call at 3:00 AM requires high-stakes decision-making and cross-team coordination that AI agents cannot yet replicate.

Why GenAI Cannot Replace Experienced Engineers

Even with the reasoning capabilities shown in the 2026 Claude Code Security update, three "hard barriers" prevent AI from replacing the individual contributor:

The Responsibility Gap: Software isn't just code; it's a liability. No AI subscription comes with an insurance policy. Accountability is a human-only function. If a system fails, a human must explain why to a board or a regulator.
Reasoning vs. Intent: AI understands the structure of your code, but humans understand the intent. An AI might see a missing role-check as a bug, while a human knows it was bypassed for a specific, documented emergency migration path.
Technical Debt Acceleration: Recent 2026 studies show that when developers over-rely on AI, "code churn" (code that is rewritten or deleted within two weeks) doubles. AI writes code faster than it can be reviewed, potentially creating a "spaghetti" codebase if not guided by a senior architect.

Why AI Cannot Replace Mature SaaS Products

Many feared that AI's ability to "generate a clone" of an app would kill the SaaS industry. This hasn't happened for several concrete reasons:

SaaS is "Running," not "Building": Building a clone of Jira or Salesforce is the easy part. Operating it at 99.99\% availability, managing global data centers, and providing 24/7 support is what customers actually pay for.
Compliance and Trust: A mature SaaS product provides pre-built SOC2, GDPR, and HIPAA guardrails. An AI-generated app is a "black box" that hasn't been audited, making it a non-starter for enterprise or legal use.
The Integration Ecosystem: SaaS platforms thrive on their ecosystems (APIs, plugins, and third-party integrations). AI can write a script to connect two tools, but it cannot manage the long-term versioning and stability of a multi-vendor tech stack.

Summary

AI tools like Claude Code Security are the new "High-Level Languages" of 2026.

Just as C++ didn't kill programmers but made them more powerful, AI is shifting the engineer's role from "Coder" to "Orchestrator and Verifier."

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

About the Author

Eyal Estrin is a cloud and information security architect and AWS Community Builder, with more than 25 years in the industry. He is the author of Cloud Security Handbook and Security for Cloud Native Applications.

The views expressed are his own.

Inside the Amazon Nova Forge

Eyal Estrin — Mon, 09 Feb 2026 13:42:06 +0000

Amazon Nova Forge is a development environment within Amazon SageMaker AI dedicated to building "Novellas" - private, custom versions of Amazon’s Nova frontier models.

Unlike typical AI services that only allow you to use a model or fine-tune its final layer, Nova Forge introduces a concept called Open Training. This gives you access to the model at various "life stages" (checkpoints), allowing you to bake your company’s proprietary knowledge directly into the model’s core reasoning capabilities.

This blog post is an introduction to Amazon Nova Forge and what makes it unique in the training process.

What Makes it Different?

Prompt engineering and RAG provide external context but fail to change a model's core intelligence. Standard fine-tuning also falls short because it happens too late in the lifecycle, attempting to steer a "finished" model that is already set in its ways. Nova Forge solves this by moving customization earlier into the training process, embedding specialized knowledge where it actually sticks.

Nova Forge occupies a unique middle ground between Managed APIs (Bedrock) and building from scratch.

Amazon Bedrock: Bedrock is for consuming models. You can fine-tune them, but you are working on a "black box" model. Nova Forge is for building the model itself using deeper training techniques.
Azure AI / Google Vertex AI: While Azure and GCP offer fine-tuning, they generally don't provide access to intermediate training checkpoints of their frontier models. Nova Forge allows for Data Blending, where you mix your data with Amazon’s original training data to prevent the model from "forgetting" how to speak or reason.

Terminology

Novella: The resulting custom model you create. It’s a "private edition" of Nova.
Checkpoints: Saved "states" of the model during its initial training (pre-training, mid-training, post-training).
Data Blending: The process of mixing your proprietary data with Nova-curated datasets so the model stays smart while learning your specific business.
Reinforcement Fine-Tuning (RFT): Using "reward functions" (logic-based feedback) to teach the model how to perform complex, multi-step tasks correctly.
Catastrophic Forgetting: A common AI failure where a model learns new information but loses its original abilities. Nova Forge is designed specifically to prevent this.

The Workflow: From Training to Production

The process bridges the gap between the "lab" (SageMaker) and the "app" (Bedrock).

Selection: You choose a Nova base model and a specific checkpoint (e.g., a "Mid-training" checkpoint) in Amazon SageMaker Studio.
Training (SageMaker AI): You use SageMaker Recipes—pre-configured training scripts—to blend your data from S3 with Nova’s datasets. The heavy lifting (compute) happens on SageMaker's managed infrastructure.
Refinement: Optionally, you run RFT in SageMaker to align the model with specific business outcomes or safety guardrails.
Deployment (Bedrock): Once the "Novella" is ready, you import it into Amazon Bedrock as a private model.
Production: Your applications call the custom model via the standard Bedrock API, benefitting from Bedrock’s serverless scaling and security.

Below is a sample training workflow:

Data Privacy and Protection

The security model is the most critical part:

Sovereignty: Your data stays in your S3 buckets and within your VPC boundaries.
No Leakage: AWS explicitly states that customer data is not used to train the base Amazon Nova models. Your "Novella" is a private resource visible only to your AWS account.
Encryption: Data is encrypted at rest via KMS (AWS-managed or Customer-managed keys) and in transit via TLS 1.2+.
Governance: Access is controlled via standard IAM policies, and all training activity is logged in CloudTrail.

Pricing Model

Nova Forge carries a distinct cost structure that reflects its "frontier" status:

Subscription Fee: Access to the Forge environment starts at approximately --$100,000 per year.
Usage Costs: On top of the subscription, you pay for the SageMaker compute (GPUs) used during the training phase.
Comparison: Cheaper than Training from Scratch: Building a frontier model from zero costs millions in compute and months of R&D. Nova Forge provides the "shortcuts" to get the same result for a fraction of that.
- More Expensive than Basic Fine-Tuning: Standard fine-tuning on Bedrock is much cheaper (often just a few dollars per hour), but it cannot achieve the deep "domain-native" intelligence that Nova Forge provides.

Summary

Amazon Nova Forge marks a shift from generic AI to native intelligence, where models don't just reference your data—they are built from it. By using "Open Training," you can bake specialized knowledge into the model’s core at the pre-training or mid-training stages. This results in a private Novella that understands your specific industry as naturally as its base language.

Organizations managing high-value proprietary data should consider moving beyond treating that information as an external reference. If your workflows involve specialized terminology or regulated processes that standard LLMs struggle to master, shifting customization earlier in the training lifecycle is often more effective than basic fine-tuning.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

Additional references

About the Author

ClawdBot Security Guide

Eyal Estrin — Mon, 02 Feb 2026 14:01:02 +0000

Clawdbot (now renamed Moltbot) is an open-source, self-hosted AI assistant that runs on your own hardware or server and can-do things, not just chat.

It was created by developer Peter Steinberger in late 2025.

It connects your AI model (OpenAI, Claude, local models via Ollama) to real capabilities: automate workflows, read/write files, execute tools and scripts, manage emails/calendars, and respond through messaging apps like WhatsApp, Telegram, Discord and Slack.

You interact with it like a smart assistant that actually takes action based on your input.

What is it used for?

Clawdbot functions as a "digital employee" or a "Jarvis-like" assistant that operates 24/7. Because it has direct access to your local filesystem and system tools, it can perform proactive tasks that standard AI cannot:

Communication Hub: It lives inside messaging apps like Telegram, WhatsApp, or Slack. You text it commands, and it can reply, summarize threads, or manage your inbox.
Proactive Automation: It can monitor your email, calendar, and GitHub repositories to fix bugs while you sleep, draft replies, or alert you to flight check-ins.
System Execution: It can run shell commands, execute scripts, manage files, and even control web browsers to perform actions like making purchases or reservations.
Persistent Memory: It maintains long-term context across conversations, remembering your preferences and past tasks for weeks or months.

Below is a sample deployment architecture of Clawdbot:

Security risks associated with Clawdbot

Clawdbot is a high-privilege automation control plane. Since it manages agents, tools, and multiple communication channels, it presents serious security risks.

Control plane exposure & misconfiguration

Exposure: Misconfigured dashboards and reverse proxies have left hundreds of control interfaces open to the internet.
Authentication Failures: Some setups treat remote connections as local, letting attackers bypass authentication.
Data Theft: Unsecured instances can expose API keys, conversation logs, and configuration data.
System Takeover: In certain cases, attackers can run commands on the host with elevated privileges.

Prompt injection & tool blast radius

Manipulation: Malicious or untrusted content can trick the AI into using tools in unintended ways.
Blast Radius: Access to high-privilege tools like shell commands or admin APIs means a prompt injection could lead to data theft or lateral movement across the network.
Model Weakness: Older or poorly aligned AI models are more likely to ignore safety instructions, increasing risk.

Social engineering and user level abuse

Deception: Attackers can manipulate the bot to extract personal or environment-specific information.
Account Misuse: Connected commerce tools could be used for unauthorized purchases.
Phishing: A compromised bot can send malicious links or scripts to contacts.
Upstream Data Exposure: Prompts and tool outputs sent to AI providers can create privacy or compliance issues if not carefully managed.

Data privacy, logs, and long term memory

Sensitive Data Exposure: The gateway stores conversation histories and memory, which may include personal or business information depending on usage.
Dashboard and Host Vulnerabilities: Exposed dashboards or weak host protections can allow attackers to access past chats, file transfers, and stored credentials (API keys, tokens, OAuth secrets), turning the instance into a data exfiltration point.
Upstream Data Risk: Prompts and tool outputs are sent to AI providers. Without proper scoping and data classification, this can create privacy and compliance issues.

Ecosystem risks: hijacked branding, fake installers, and scams

Hijacked Accounts: After a rebrand, original social media and GitHub handles were exploited by scammers promoting fake crypto tokens.
Malware Risk: Users searching for the tool may encounter backdoored versions or fake installers designed to compromise their systems.

Network and Remote Access Risks

Browser Control: Tools that let the bot control a browser can expose local or internal network resources if not secured.
Tunneling Errors: Misconfigured reverse proxies or tools like Tailscale may grant attackers unintended access to private networks.

Recommendations for securing Clawdbot

Based on the official GitHub repository, documentation, and expert audits from January 2026, here are the recommendations for securing your instance.

Lock Down the Gateway

Bind the Clawdbot gateway to loopback (127.0.0.1) and never expose it directly to the internet. If remote access is required, use private mesh solutions such as Tailscale or Cloudflare Tunnel. Always enable gateway authentication using tokens or passwords.

References:

Enforce Strict Access Controls

Restrict who can interact with Clawdbot by enforcing DM pairing or allowlists. Avoid wildcard policies in production. In group chats, require explicit mentions before the bot processes messages.

Reference:

Official GitHub SECURITY.md

Isolate the Runtime Environment

Run Clawdbot on dedicated hardware or a dedicated VM/container. Avoid running it on your primary workstation. Use Docker sandboxing with minimal mounts and dropped capabilities.

References:

Sandbox and Restrict Tools

Enable sandboxing for all high-risk tools such as exec, write, browser automation, and web access. Use tool allow/deny lists and restrict elevated tools to trusted users only.

Reference:

Official GitHub Security Overview

Apply Least Privilege to Agent Capabilities

Disable interactive shells unless strictly necessary. Limit filesystem visibility to read-only mounts where possible. Avoid granting elevated privileges to agents handling untrusted input.

Reference:

Official Clawdbot Documentation

Secure Credentials and Secrets

Store secrets in environment variables, not configuration files or source control. Apply strict filesystem permissions to Clawdbot directories and rotate credentials after any suspected incident.

Reference:

Official Clawdbot Security Documentation

Continuous Auditing and Monitoring

Regularly run built-in security audit and doctor commands to detect unsafe configurations. Monitor logs and session transcripts for anomalous behavior or unexpected access.

Reference:

Official GitHub Security CLI Documentation

Harden Browser Automation

Treat browser automation as operator-level access. Use dedicated browser profiles without password managers or sync enabled. Never expose browser control ports publicly.

Prompt-Level Safety Rules

Define explicit system rules that prevent disclosure of credentials, filesystem structure, or infrastructure details. Require confirmation for destructive actions.

Reference:

Official Clawdbot Security Documentation

Incident Response Preparedness

Maintain a documented response plan. If compromise is suspected: stop the gateway, revoke access, rotate all secrets, review logs, and re-run security audits.

Reference:

Official Clawdbot Security Documentation

Summary

ClawdBot is a high-privilege AI agent that can act on your system, not just chat. Its main risks come from exposed gateways, weak access controls, and powerful tools combined with prompt injection or social engineering, which can lead to system compromise and data loss. To use it safely, lock the gateway to localhost with authentication, restrict who can interact with it, isolate its runtime, minimize tool permissions, and monitor it continuously.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

References:

About the Author

Securing AI Skills

Eyal Estrin — Mon, 26 Jan 2026 15:10:55 +0000

If you give an AI system the ability to act, you give it risk.

In earlier posts, I covered how to secure MCP servers and agentic AI systems. This post focuses on a narrower but more dangerous layer: AI skills. These are the tools that let models touch the real world.

Once a model can call an API, run code, or move data, it stops being just a reasoning engine. It becomes an operator.

That is where most security failures happen.

Terminology

In generative AI, "skills" describe the interfaces that allow a model to perform actions outside its own context.

Different vendors use different names:

Tools: Function calling and MCP-based interactions
Plugins: Web-based extensions used by chatbots
Actions: OpenAI GPT Actions and AWS Bedrock Action Groups
Agents: Systems that reason and execute across multiple steps

A base LLM predicts text; A skill gives it hands.

Skills are pre-defined interfaces that expose code, APIs, or workflows. When a model decides that text alone is not enough, it triggers a skill.

Anthropic treats skills as instruction-and-script bundles loaded at runtime.

OpenAI uses modular functions inside Custom GPTs and agents.

AWS implements the same idea through Action Groups.

Microsoft applies the term across Copilot and Semantic Kernel.

NVIDIA uses skills in its digital human platforms.

In the reference high-level architecture below, we can see the relations between the components:

Why Skills Are Dangerous

Every skill expands the attack surface. The model sits in the middle, deciding what to call and when. If it is tricked, the skill executes anyway.

The most common failure modes:

Excessive agency: Skills often have broader permissions than they need. A file-management skill with system-level access is a breach waiting to happen.
The consent gap: Users approve skills as a bundle. They rarely inspect the exact permissions. Attackers hide destructive capability inside tools that appear harmless.
Procedural and memory poisoning: Skills that retain instructions or memory can be slowly corrupted. This does not cause an immediate failure. It changes behavior over time.
Privilege escalation through tool chaining: Multiple tools can be combined to bypass intended boundaries. A harmless read operation becomes a write. A write becomes execution.
Indirect prompt injection: Malicious instructions are placed in content that the model reads: emails, web pages, documents. The model follows them using its own skills.
Data exfiltration: Skills often require access to sensitive systems. Once compromised, they can leak source code, credentials, or internal records.
Supply chain risk: Skills rely on third-party APIs and libraries. A poisoned update propagates instantly.
Agent-to-agent spread: In multi-agent systems, one compromised skill can affect others. Failures cascade.
Unsafe execution and RCE: Any skill that runs code without isolation is exposed to remote code execution.
Insecure output handling: Raw outputs passed directly to users can cause data leaks or client-side exploits.
SSRF: Fetch-style skills can be abused to probe internal networks.

How to Secure Skills (What Actually Works)

Treat skills like production services. Because they are.

Identity and Access Management

Each skill must have its own identity. No shared credentials. No broad roles.

Permissions should be minimal and continuously evaluated. This directly addresses OWASP LLM06: Excessive Agency.

Reference: OWASP LLM06:2025 Excessive Agency

AWS Bedrock

Assign granular IAM roles per agent. Restrict regions and models with SCPs. Limit Action Groups to specific Lambda functions.

References:

OpenAI

Never expose API keys client-side. Use project-scoped keys and backend proxies.

Reference: Best Practices for API Key Safety

Input and Output Guardrails

Prompt injection is not theoretical. It is the default attack.

Map OWASP LLM risks directly to controls.

Reference: OWASP Top 10 for Large Language Model Applications

AWS Bedrock

Use Guardrails with prompt-attack detection and PII redaction.

Reference: Amazon Bedrock Guardrails

OpenAI

Use zero-retention mode for sensitive workflows.

Reference: Data controls in the OpenAI platform

Anthropic

Use constitutional prompts, but still enforce external moderation.

Reference: Building safeguards for Claude

Adversarial Testing

Red-team your agents.

Test prompt injection, RAG abuse, tool chaining, and data poisoning during development. Not after launch.

Threat modeling frameworks from OWASP, NIST, and Google apply here with minimal adaptation.

References:

DevSecOps Integration

Every endpoint a skill calls is part of your attack surface.

Run SAST and DAST on the skill code. Scan dependencies. Fail builds when violations appear.

References:

Using Generative AI, Amazon Bedrock, and Amazon CodeGuru to Improve Code Quality and Security

Isolation and Network Controls

Code-executing skills must run in ephemeral, sandboxed environments.

No host access. No unrestricted outbound traffic.

Use private networking wherever possible:

AWS PrivateLink

Logging, Monitoring, and Privacy

If you cannot audit skill usage, you cannot secure it.

Enable full invocation logging and integrate with existing SIEM tools.

Ensure provider data-handling terms match your risk profile. Not all plans are equal.

References:

Incident Response and Human Oversight

Update incident response plans to include AI-specific failures.

For high-risk actions, require human approval. This is the simplest and most reliable control against runaway agents.

References:

Summary

AI skills are the execution layer of generative systems. They turn models from advisors into actors.

That shift introduces real security risk: excessive permissions, prompt injection, data leakage, and cascading agent failures.

Secure skills the same way you secure production services. Strong identity. Least privilege. Isolation. Guardrails. Monitoring. Human oversight.

There is no final state. Platforms change. Attacks evolve. Continuous testing is the job.

About the Author

Introducing Managed Instances in the Cloud

Eyal Estrin — Tue, 20 Jan 2026 14:12:57 +0000

For many years, organizations embracing the public cloud knew there were two main types of compute services - customer-managed (i.e., IaaS) and fully managed or Serverless compute (i.e., PaaS).

The main difference is who is responsible for maintenance of the underlying compute nodes in terms of OS maintenance (such as patch management, hardening, monitoring, etc.) and the scale (adding or removing compute nodes according to customer or application load).

In an ideal world, we would prefer a fully managed (or perhaps a Serverless) solution, but there are use cases where we would like to have the ability to manage a VM (such as the need to connect to a VM via SSH to make configuration changes at the OS level).

In this blog post, I will review several examples of managed instance services and compare their capabilities with the fully managed alternative.

Function as a Service

The only alternative I managed to find is the AWS Lambda Managed Instances.

AWS Lambda has been in the market for many years, and it is the most common Serverless compute service in the public cloud (though not the only alternative).

Below is a comparison between AWS Lambda and the AWS Lambda Managed Instances:

When to Use Which Alternative

Use AWS Lambda (Standard) If:

Traffic is Bursty or Unpredictable: You need the ability to scale from zero to thousands of concurrent executions in seconds to handle sudden spikes.
Low or Intermittent Volume: You have idle periods were paying for running instances would be wasteful. "Scale to zero" is a priority.
Strict Isolation is Required: Your security model relies on the strong isolation of Firecracker microVMs for every single request.
Simplicity is Key: You want zero infrastructure decisions—just upload code and run.

Use AWS Lambda Managed Instances If:

Traffic is High & Predictable: You have steady-state workloads were paying for always-on EC2 instances (with Savings Plans) is cheaper than per-request billing.
Workloads are Compute/Memory Intensive: You need specific hardware ratios (e.g., high CPU but low RAM) or specialized instruction sets not available in standard Lambda.
Latency Sensitivity: You cannot afford any cold start latency and need environments that are always initialized.
High I/O Concurrency: Your application performs many I/O bound tasks (like calling external APIs) and can efficiently process multiple requests on a single vCPU without blocking.

Container Service

Amazon ECS is a highly scalable container orchestration service that automates the deployment and management of containers across AWS infrastructure.

Below is a comparison between Amazon ECS (self-managed EC2) and the Amazon ECS Managed Instances:

When to Use Which Alternative

Use Amazon ECS (Self-Managed EC2) If:

You Need Custom AMIs: Your compliance or legacy software requires a specific, hardened OS image or custom kernel modules.
You Require Host Access: You need SSH access to the underlying node for deep debugging, forensic auditing, or installing host-level daemon agents that ECS doesn't support.
Cost is the Sole Priority: You want to avoid the additional management fee and have a dedicated team that can manually optimize bin-packing and Spot instance usage for free.
Legacy / Hybrid Constraints: You are extending a specific on-premise network configuration or storage driver setup that requires manual OS configuration.

Use Amazon ECS Managed Instances If:

You Need GPUs or High Memory: You require specific hardware (like GPU instances for AI/ML) that AWS Fargate does not support, but you don't want to manage the OS.
You Want "Fargate-like" Operations with EC2 Pricing: You want to offload patching and ASG management (like Fargate) but need to use Reserved Instances or Savings Plans to lower costs.
Security Compliance: You need guaranteed, automated rotation of nodes for security patching (e.g., every 14 days) without building the automation pipelines yourself.
Steady-State Workloads: Your traffic is predictable, making always-on EC2 instances more cost-effective than Fargate's per-second billing.

Kubernetes Service

Amazon EKS is a fully managed service that simplifies running, scaling, and securing containerized applications by automating the management of the Kubernetes control plane on AWS.

Below is a comparison between Amazon EKS (self-managed nodes) and the Amazon EKS Managed Node Groups:

When to Use Which Alternative

Use Amazon EKS Managed Node Groups If:

Standard Kubernetes Workloads: You are running standard applications and want to minimize the time spent on infrastructure maintenance.
Simplified Scaling: You want EKS to automatically handle the creation of Auto Scaling Groups that are natively aware of the cluster state.
Automated Security: You want a streamlined way to apply security patches and OS updates to your cluster nodes without downtime.
Operational Efficiency: You have a small team and need to focus on application code rather than Kubernetes "plumbing."

Use Amazon EKS Self-Managed Nodes If:

Custom Operating Systems: You must use a specific, hardened OS image (e.g., a highly customized Ubuntu or RHEL) that is not supported by Managed Node Groups.
Complex Bootstrap Scripts: You need to run intricate "User Data" scripts during node startup that require fine-grained control over the initialization sequence.
Unique Networking Requirements: You are using specialized networking plugins or non-standard VPC configurations that require manual node configuration.
Legacy Compliance: You have strict regulatory requirements that mandate manual oversight and "manual sign-off" for every single OS-level change.

Summary

In this blog post, I have reviewed several compute services (from FaaS, containers, and managed Kubernetes), each with its alternatives for either customer managing the compute nodes, or having AWS manage the compute nodes for the customers.

By leveraging AWS Lambda Managed Instances, Amazon ECS Managed Instances, and Amazon EKS Managed Node Groups, organizations can achieve high hardware performance without the burden of operational complexity. The primary advantage of this managed tier is the ability to decouple hardware selection from operating system maintenance. Developers can handpick specific EC2 families, such as GPU-optimized instances for AI or Graviton for cost efficiency, while AWS manages the heavy lifting of security patching and instance lifecycle updates.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

About the author

Eyal Estrin is a seasoned cloud and information security architect, AWS Community Builder, and author of Cloud Security Handbook and Security for Cloud Native Applications. With over 25 years of experience in the IT industry, he brings deep expertise to his work.
Connect with Eyal on social media: https://linktr.ee/eyalestrin.
The opinions expressed here are his own and do not reflect those of his employer.

When you have a hammer, everything looks like a nail

Eyal Estrin — Tue, 06 Jan 2026 15:51:13 +0000

In the over-evolving tech world, we often see organizations (from C-Level down to architects and engineers) rush to adopt the latest technology trends without conducting proper design or truly understanding the business requirements.

The result of failing to do a proper design is a waste of resources (from human time to compute), over-complicated architectures, or under-utilized resources.

In this blog post, I will dig into common architecture decisions and provide recommendations to avoid the pitfalls.

Let’s dig into some examples.

Moving everything to the public cloud

Example

An enterprise mandates a full lift-and-shift of all workloads to a hyper-scaler to “become cloud-native,” including legacy ERP systems, mainframes, and latency-sensitive trading applications.

What was misunderstood

Some workloads had hard latency, data residency, or licensing constraints.
The applications were tightly coupled, stateful, and designed for vertical scaling.
Cost models were not analyzed beyond infrastructure savings.

Issues that emerged

Higher total cost of ownership due to egress fees, oversized instances, and always-on resources.
Performance degradation for low-latency systems.
Operational complexity increased without gaining elasticity or resilience benefits.
Missed opportunity to modernize selectively (hybrid or refactor where justified).

Using Kubernetes for every architecture

Example

A team deploys all applications - including small internal tools, batch jobs, and simple APIs - onto a shared Kubernetes platform.

What was misunderstood

Kubernetes is an orchestration platform, not a free abstraction layer.
Many workloads did not need container orchestration, autoscaling, or self-healing.
The organization lacked operational maturity for cluster management and security.

Issues that emerged

Increased cognitive load for developers (YAML, Helm, networking, ingress, RBAC).
The platform team became a bottleneck for simple changes.
Security misconfigurations (over-permissive service accounts, exposed services).
Slower delivery compared to simpler deployment models (VMs or managed PaaS).

Using Serverless for every solution

Example

An architect mandates that all new services must be implemented using Functions-as-a-Service.

What was misunderstood

Serverless excels at event-driven, stateless, bursty workloads - not long-running or chatty processes.
Cold starts, execution limits, and state management trade-offs were ignored.
Observability and debugging differ significantly from traditional services.

Issues that emerged

Latency spikes impacting user-facing APIs.
Complex orchestration logic is spread across functions, reducing maintainability.
Higher costs for sustained workloads compared to containers or VMs.
Difficult troubleshooting due to fragmented logs and distributed execution paths.

Using GenAI to solve every problem

Example

A company integrates GenAI into customer support, code reviews, security analysis, and decision-making workflows without clearly defined use cases.

What was misunderstood

GenAI produces probabilistic outputs, not deterministic answers.
Data quality, context boundaries, and hallucination risks were underestimated.
Regulatory, privacy, and intellectual property implications were not assessed.

Issues that emerged

Incorrect or misleading responses are presented as authoritative.
Leakage of sensitive data through prompts or training feedback loops.
Increased operational risk when AI outputs were trusted without validation.
High costs with unclear ROI due to overuse in low-value scenarios.

Practical recommendations

Start with business drivers, not technology - Define success metrics first: cost model, performance requirements, regulatory constraints, delivery speed, and operational ownership. Technology should follow these inputs - not precede them.
Explicitly document constraints and non-goals - Latency, data residency, licensing, team skills, and operational maturity must be captured early. Many architectural failures stem from ignored or implicit constraints.
Apply technologies where their strengths are essential:
- Public cloud: prioritize elasticity, managed services, and global reach - not lift-and-shift.
- Kubernetes: use it where orchestration, portability, and scale justify its complexity.
- Serverless: limit the use of Serverless to event-driven and bursty workloads.
- GenAI: apply where probabilistic output is acceptable and verifiable.
Favor simplicity as a default - If a simpler architecture meets requirements, it is usually the correct choice. Complexity should be earned, not assumed.
Continuously validate assumptions - Revisit architectural decisions as workloads evolve. What was once justified can become technical debt when context changes.
Reward outcome-driven architecture - Measure architects and teams on business impact, reliability, and cost efficiency - not on adoption of trendy platforms.

Summary

The recurring failure pattern in modern architectures is not poor technology choice, but premature commitment to a tool before understanding the problem. Cloud platforms, Kubernetes, Serverless, and GenAI are powerful when applied deliberately - and damaging when treated as universal defaults. When architects start with the solution, they optimize for platform elegance instead of business outcomes.

About the author

Turning License Changes into Opportunity

Eyal Estrin — Mon, 29 Dec 2025 16:37:12 +0000

The concept of vendor lock-in existed for many years; organizations chose commercial, and in many cases expensive license to use proprietary software products to run their production workloads.

In the past, there was the notion that using a product from a well-known vendor was the best solution, due to support, a large customer base, and, as the famous quote says, "Nobody gets fired for buying IBM."

This was all true for decades, but as the software world matured, organizations began migrating workloads to the public cloud and began building modern or cloud-native applications based on open-source alternatives.

In this blog post, I will discuss some of the well-known case studies of switching from commercial products to open-source.

From Elasticsearch to OpenSearch

Elasticsearch is a distributed search and analytics engine that stores data as JSON documents and lets you run fast full‑text search, aggregations, and log or metrics analysis across large datasets.

Elasticsearch, prior to 7.11, used Apache License 2.0, a permissive license allowing commercial use, modification, and distribution with minimal restrictions.

In January 2021, Elastic announced that starting with version 7.11, it would be relicensing its Apache 2.0 licensed code in Elasticsearch to be dual licensed under SSPL (Server-Side Public License) and the Elastic License, a strong copyleft license that requires anyone offering the software as a service to open-source the entire service stack.

In August 2024, the GNU Affero General Public License was added to Elasticsearch version 8.16.0 as an option, making Elasticsearch free and open-source again.

Elastic argued that large cloud providers were taking the open‑source Elasticsearch, offering it as a commercial managed service (e.g., Amazon Elasticsearch Service) and capturing much of the value without sufficient reciprocity.

The license change was positioned as protecting Elastic’s SaaS/Elastic Cloud business and long‑term sustainability.

OpenSearch was launched by AWS and partners as a fork later in 2021, based on Elasticsearch 7.10 and Kibana 7.10, the last Apache‑2.0 versions.

Today, OpenSearch is no longer just an AWS side‑project; it is governed by the OpenSearch Software Foundation, a Linux Foundation project that provides vendor‑neutral governance and long‑term stewardship. Premier foundation members include AWS, SAP, and Uber, all of whom either run OpenSearch in production, build products on top of it, or contribute engineering resources.

Among the benefits of switching to OpenSearch:

Licensing - OpenSearch is Apache 2.0, so there are no SSPL/Elastic License obligations or restrictions on offering it as a managed service or embedding it in SaaS products.
Vendor neutrality - OpenSearch’s open ecosystem (self‑managed on Kubernetes/VMs or via providers like Amazon OpenSearch Service and others) reduces dependence on a single vendor and improves negotiating leverage.
Migration - OpenSearch was designed as a near drop‑in replacement for Elasticsearch 7.10, so many clients, APIs, and index formats are compatible, which lowers migration effort and risk.
Scalability - OpenSearch retains Elasticsearch’s horizontally scalable architecture and adds features like vector search, observability improvements, and integrations driven by a multi‑vendor community, not just one company’s roadmap.

From Terraform to OpenTofu

HashiCorp Terraform is an infrastructure as code tool that lets you define, provision, and manage cloud and on‑prem resources using declarative configuration files, enabling consistent, repeatable deployments across multiple providers.

HashiCorp announced the Terraform license change in August 2023, and it applies starting with versions after 1.5.x (i.e., from 1.6 onward).

Terraform was originally licensed under the Mozilla Public License 2.0 (MPL 2.0), a weak copyleft license requiring modifications to licensed files to be open-sourced while allowing proprietary code alongside, and was then relicensed to the Business Source License (BSL/BUSL 1.1), which is a source‑available but not OSI‑approved open‑source license, introduced to restrict certain commercial/competitive uses while remaining free for typical internal infrastructure use.

HashiCorp stated it wanted to prevent other companies, particularly cloud vendors and platforms, from offering competing managed services built directly on top of Terraform without commercial agreements, arguing this threatened HashiCorp’s ability to invest in the product.

The move was framed as protecting the “commercial viability” of Terraform and other HashiCorp tools, but triggered ecosystem concerns over neutrality, long‑term trust, and vendor lock‑in.

In response, a group of companies and maintainers drafted the “OpenTF” manifesto and, after HashiCorp declined to revert or donate Terraform to a foundation, forked the last MPL‑licensed version (1.5.6) into a new project later named OpenTofu, donated it to the Linux Foundation, and committed to keeping it under MPL 2.0 with neutral, community‑first governance. OpenTofu fork announced in 2023, GA in 2024.

The founding vendors behind OpenTofu include Gruntwork, Spacelift, Harness, env0, and Scalr, all of whom depended heavily on open Terraform and now fund or employ core maintainers for OpenTofu.

Among the benefits of switching to OpenTofu:

Licensing - OpenTofu keeps the original MPL 2.0 open‑source license, so there are no "source‑available" or BSL terms restricting competitive SaaS or internal platform use.
Vendor neutrality - OpenTofu is governed under a neutral foundation, not a single commercial vendor, which lowers the risk that future business decisions (price, license, roadmap) will disrupt users.
Migration - OpenTofu is intentionally Terraform‑compatible (config syntax, state format, providers), so most organizations can switch with minimal changes to modules, backends, and pipelines.
Community‑driven features and transparency - OpenTofu’s roadmap and code are driven by a broad contributor base, so features like client‑side state encryption and other safety improvements tend to align closely with practitioner needs.

From Redis to Valkey

Redis is an in-memory key–value data store that can act as a database, cache, and message broker, optimized for extremely low‑latency reads and writes and supporting rich data structures like strings, lists, sets, and hashes.

Redis changed its license in March 2024, moving from the BSD‑3‑Clause open‑source license to a dual "source‑available" model using the Redis Source Available License v2 (RSALv2), a source-available license that permits use, modification, and redistribution but restricts offering the software as a competing managed service, and the Server-Side Public License (SSPLv1), primarily to stop cloud providers from offering Redis as a managed service without paying or sharing more of their own code and revenue with Redis Ltd.

In response, in 2024, major contributors and users of Redis—including engineers from AWS, Alibaba, Google, Ericsson, Huawei, Tencent, Oracle and others—took the last BSD‑licensed Redis 7.2.4 code, forked it under the new name Valkey, and placed it in a Linux Foundation–governed project to preserve a fully open, high‑performance in‑memory key–value store that remains free from vendor lock‑in and can be safely embedded in cloud platforms, SaaS products, and managed services.

Valkey uses the BSD 3‑Clause license, which is a permissive, OSI‑approved open‑source license that allows free use, modification, and redistribution, including in commercial and cloud/SaaS offerings.

Among the benefits of switching to Valkey:

Licensing - Valkey keeps a permissive BSD‑3‑Clause license, so teams avoid Redis’s newer source‑available terms and can freely offer Valkey as a managed service or embed it in SaaS without SSPL‑style obligations or commercial negotiations.
Vendor neutrality - Valkey is governed under a neutral foundation with a multi‑vendor contributor base, which reduces dependence on a single company’s business decisions and gives organizations more confidence in long‑term roadmap stability.
Migration - Because Valkey started from the last BSD‑licensed Redis 7.2 codebase, existing clients, data structures, and usage patterns generally continue to work with minimal changes, making migrations relatively low‑risk.
Scalability - Valkey’s roadmap emphasizes core engine efficiency (e.g., improved multithreading, better memory usage, and clustering enhancements), so many users get similar or better performance and scalability for classic caching and queueing workloads without paying for an enterprise Redis tier.

Summary

Migrations from Elasticsearch to OpenSearch, Terraform to OpenTofu, and Redis to Valkey all stem from the same story: vendors tightened licenses to protect their commercial cloud offerings, and the ecosystem responded by creating fully open forks that restore freedom to run, modify, and offer these technologies as services.

These community‑governed projects preserve familiar APIs and architectures while removing restrictive licenses, so customers keep the functionality they rely on and gain long‑term legal clarity and vendor‑neutral governance.

For users, the benefits include reduced lock‑in, simpler compliance, and the ability to standardize on open cores that any provider can host, extend, and support, rather than being bound to a single company’s roadmap or pricing.

All of these points are in the same direction: the future of core cloud‑native tools lies in truly open‑source projects backed by strong communities and foundations, not in proprietary products pretending to be open, so organizations get more control, stronger resilience, and real choice in how they run their infrastructure.

About the author

Eyal Estrin is a seasoned cloud and information security architect, AWS Community Builder, and author of Cloud Security Handbook and Security for Cloud Native Applications. With over 25 years of experience in the IT industry, he brings deep expertise to his work.

Connect with Eyal on social media: https://linktr.ee/eyalestrin.

The opinions expressed here are his own and do not reflect those of his employer.

How to keep up with technology and advance your career

Eyal Estrin — Mon, 22 Dec 2025 16:19:17 +0000

In 2023, I published a blog post titled Sharing Knowledge as a Way of Life, where I suggested that knowledge sharing should become a habit because it helps raise awareness about neglected topics, build community, and enhance your professional reputation.

I agree that the technology world keeps changing every day, from new services announced, new capabilities related to AI, new cybersecurity risks, emerging technologies, etc.

The question is – how do you keep up with technology, and by doing so, advance your career, remain relevant and attractive in the tech industry?

In this post, I’ll explore this topic from a new perspective: how to stay up to date with technology in an era of rapid change.

Self-learning

In the past, to learn new technology, we used to pay money, go to a college or any training center close to our home, sit for several days in a physical class, and allow an instructor to feed us with knowledge.

Sometimes, we use it for study and at home, and take a certification exam, to test our knowledge (and perhaps to show a certificate to potential employers).

In the past couple of years (I would say, sometimes after the COVID pandemic), online courses have become very popular.

Platforms such as QA Platform (formerly Cloud Academy), Pluralsight, Udemy, or LinkedIn Learning became the main source of self-learning courses.

If your main focus is cloud computing, AWS has AWS Skill Builder.

The mentioned platforms offer anyone, from a newbie to a practitioner, the ability to learn at their own pace, from anywhere (home, internet café, etc.), read, listen to recorded lectures, and gain hands-on experience by practicing in test labs.

Naturally, theoretical knowledge has low value.

If you are studying, for example, new cloud technology, I recommend that you create an account in one of the cloud providers, put credit card, and gain hands-on experience by deploying services, building applications, writing some code, and sharing it in your Git repo, so anyone can learn from you.

I highly recommend that your spare time (at least one hour, but preferably more) each week to learn something new, practice, and gain hands-on experience.

Public events

I believe there is a limit to how much you could learn by yourself, and this is why I recommend taking advantage of public events such as webinars (where you can connect from anywhere), community meetups (such as meetup.com, or Eventbrite), community platforms (such as Slack or Discord), and finally industry conferences (in almost any topic you could think of).

If you are attending a conference, here are some tips I can share with you to get the most out of conferences:

Prepare in advance – Usually, conferences have a published agenda, list of topics, tracks, and lectures. Before attending a conference, it is highly recommended to familiarize yourself with the list of lectures, select topics the closest to you, and mix them with topics you're not familiar with or have past expertise in.
Be humble – Don't assume you already know everything. Sit at lectures, listen to the lecturer, ask questions, perhaps even take some pictures with your phone (to be able to review slides later), and allow yourself you expand your knowledge.
Engage – Socialize with other conference attendees during the conference, both with your past colleagues who may have also come to the conference, and allow yourself to meet new people, exchange ideas, ask questions, and share knowledge.
Visit vendor booths – Speak with salespeople (yes, I know that their job is to sell you something you don't necessarily need…), learn about their offering, ask questions, and if you're really interested, schedule a follow-up meeting.
Gain hands-on experience – Participate in workshops (don't forget to bring a laptop…); there is no comparable to the knowledge you're gaining by actually deploying stuff, and taking part in labs, to expand your knowledge and experience.
Share key takeaways – Whether you wrote notes during a conference, took pictures with your phone, or received written material (such as PDFs, or links to vendor sites, Git repos, etc.), take the time after the conference to write your own inputs, and share them with your colleagues.

Knowledge sharing

The most advanced way to expand your career is by sharing your knowledge and expertise, and personally, I prefer to write in English to have an audience from all around the world.

It doesn't matter which platform you choose; whatever you do will advance your career.

Develop soft skills – The most important quality for anyone in the tech industry is to be able to communicate with others. It may be small talk with your peers in a coffee break, a conversation with a customer about an issue he's having, or the ability to explain a senior manager about technological topic, but in business terms.
Write a blog post – This is an excellent way for anyone who has something to share and doesn't feel comfortable in front of an audience. You may share personal opinions on a topic, how-to guidelines, or even code samples. You don't even have to be an expert in a specific topic; whatever you share, people will read, and if it's valuable, people will follow your posts regularly.
Record videos or podcasts – Both YouTube and Podcasts (such as Spotify) became very popular in the past decade. Begin small, share your insights, share your recordings over social media, and begin to attract followers around the world.
Provide lectures – Regardless of the platform you choose, lectures are a great way to share knowledge and engage with colleagues and peers. You can choose video lectures (such as Zoom), on-site in small groups, or on the stage in front of a large audience, whatever you feel comfortable with. This is a great way to build your confidence and brand and advance your career.
Mentorship – This is a combination of someone who has a lot of knowledge (in at least one domain) and is generous enough to expand the knowledge of others. You can do it in one-on-one meetings, or even in small groups (since large groups tend to be ineffective in my perspective). Remember to provide your mentees honest feedback, and don't forget to ask for feedback for the work you do, to learn from your mistakes.

Summary

In this blog post, I shared a lot of ways anyone in the tech industry can expand their knowledge, gain experience, build a reputation, and be able to advance his or her career to the next level.

Learning never stops. There is always the next level you can learn in any topic.

According to Werner Vogels, Amazon CTO, a T-shaped person is someone who has deep expertise (the vertical bar of the T) in one specific domain, such as software development, cloud architecture, or data science, combined with broad knowledge and skills (the horizontal bar of the T) across multiple disciplines, such as communication, systems thinking, and collaboration.

To advance your career, you should always strive to build both depth and breadth in multidisciplinary domains.

About the author

Goodbye to Static Credentials: Embrace Modern Identity Practices

Eyal Estrin — Mon, 15 Dec 2025 17:13:55 +0000

When organizations used to build applications in the past (mostly on-prem, but also in the public cloud), a common practice for allowing services to authenticate between each other was to create a service account (sometimes referred to as an application account) and embed its credentials in code or configuration files.

Another common way to gain access to services was to use static credentials such as keys. For example – AWS IAM user access keys.

In this blog post, I will explain the risks related to using static credentials and provide recommendations when designing and building modern applications in the cloud.

Introduction to static credentials

Before we begin a conversation about static credentials, it is important to under why we need them in the first place.

Naturally, we don’t want to use a human (such as a developer, a DevOps or a DBA) credentials as part of code or configuration files to authenticate an application component (such as an API endpoint, or a front-end web application) to a backend service (such as a database, storage, message queue, etc.)

The most common practice for many years, which originated in on-prem legacy applications, was to create a service or application account and use it for non-interactive login.

Such identities are now known as NHI (or non-human identity), and since they were used as part of applications, and not as part of human/interactive login, they used to have static credentials (hopefully, long and complex passwords), and in most cases, their credentials were kept permanent and were never replaced (i.e., long-lived credentials).

Static credentials risks

Static credentials create persistent and often invisible weaknesses in cloud environments, offering attackers simple entry points while limiting an organization’s ability to detect misuse, enforce strong controls, or contain incidents effectively.

Below is a list of risks relating to the use of static credentials:

High blast radius - Broad, persistent access makes any compromise immediately severe.
Susceptibility to accidental exposure - Frequent real-world leaks through code repos, logs, and CI artifacts make this a major threat vector.
Lack of automatic short-lived expiration - Keys remain valid long after they should not, enabling silent long-term abuse.
Difficult rotation and poor key hygiene - Operational friction leads to rarely rotated, aging credentials that attackers can exploit for extended periods.
No strong binding to workloads - Attackers can use stolen credentials from any location or infrastructure, increasing exploitability.
Credential reuse across environments - Compromise of a single environment cascade laterally, expanding impact.
Limited visibility and enforcement - Weak contextual signals hinder detection and prevent the application of strong access controls.
Target for automated reconnaissance - Attackers routinely harvest exposed keys, though the actual impact depends on whether exposure occurs.
Poor alignment with zero trust principles - Creates structural security gaps but typically manifests through other higher-ranked risks.
Operational drag on incident response - Increases containment time but is a downstream effect rather than a primary threat.

Vault as an interim solution

In the past, organizations used to deploy solutions such as CyberArk Privileged Access Manager or BeyondTrust Password Safe.

As organizations began to embrace the public cloud, organizations began to use managed secrets managers such as AWS Secrets Manager or HashiCorp Vault (as a vendor-agnostic solution).

Although the mentioned solutions assisted in managing the entire lifecycle of static credentials (from generation, storage, retrieval and revocation), and the credentials weren’t as long as in the past, it still kept the problem of having static credentials.

The Long-term solution

The recommended solution is to avoid long-lived credentials when building modern applications in the cloud.

The alternative to using short-lived credentials depends on the use case.

General purpose

For most cloud workloads (such as compute, storage, database, messaging, integration, APIs, etc.), use a solution such as:

AWS IAM Role - is a temporary permission identity that workloads or users can assume to access AWS resources without relying on long-lived credentials.

Managed Kubernetes

For allowing Pods within a managed Kubernetes environment access to cloud services, use a solution such as:

AWS EKS Pod Identity - lets Kubernetes pods in an Amazon EKS cluster assume an IAM role and receive temporary credentials.

External (Federated) Identities

For scenarios that you need to grant access to external/federated identities through OIDC, temporary (or short-lived credentials) to resources in the cloud eco-system, use a solution such as:

AWS Security Token Service (AWS STS) - issues short-lived, scoped credentials that external or federated identities obtain through the AssumeRole API, allowing them to access AWS resources temporarily without relying on long-lived access.

AI Agents

For scenarios that you need to grant access to AI agents, access to resources in the cloud eco-system, use a solution such as:

Amazon Bedrock AgentCore Identity - Purpose-built IAM service for Bedrock agents with centralized agent identity directory, inbound authorizer (SigV4/OAuth/JWT), and outbound credential provider.

Summary

When managing non-human identities, always prefer temporary (roles/managed identities) over static or long-lived credentials.

If the target service does not support temporary credentials, use short-lived credentials and regularly rotate the credentials to avoid potential credential breaches.

Whatever you do, never store credentials as part of code, configuration files, Git repositories, etc.

This blog post focused on solutions offered by the hyper-scale cloud providers; naturally, there are commercial solutions offering similar functionalities, including a single pane of glass for managing non-human identities for the entire cloud environments, including multi-cloud environments.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

About the author

FinOps for AI

Eyal Estrin — Mon, 08 Dec 2025 14:23:43 +0000

Today, we hear about so many organizations (from small start-ups to large enterprises) experimenting with GenAI applications, adding GenAI components to their existing workloads, and perhaps even moving from evaluation to production.

The increased usage of GenAI services requires organizations to pay attention to the cost of using GenAI services before the high and unpredictable cost generates additional failed projects.

In this blog post, I will share some common recommendations for implementing FinOps practices as part of GenAI workloads.

Real-Time Cost Visibility, Allocation, Tagging, and Accountability

Lack of real-time visibility into cloud costs makes it difficult for organizations to track spending, identify waste, and assign accountability. Without clear, up-to-date cost allocation tied to projects or teams, overspending and inefficiencies often go unnoticed. Building transparent cost tracking and tagging practices empowers teams to monitor expenses continuously, optimize usage, and align spending with business goals.

Recommendations / Best practices

Optimization Tools: Software that identifies inefficiencies and recommends or automates cost-saving actions in cloud environments. Common services: AWS Cost Explorer, AWS Trusted Advisor.
Estimate and Monitor Costs: Tools to forecast upcoming cloud expenses and continuously track actual spend against budgets. Common service: AWS Pricing Calculator.
Budgets, Alerts, and Cost Analysis: Features that allow setting spending limits, notifying on overruns, and analyzing cost trends. Common services: AWS Budgets, AWS Cost Anomaly Detection.
Cost Visibility, Allocation, tagging: Mechanisms to attribute cloud costs accurately to applications, teams, or business units using tags and reports. Common service: AWS Cost Allocation Tags.
Token and Endpoint Cost Tracking: Monitoring and reporting on usage-driven costs specifically related to API tokens and endpoint consumption. Common service: Amazon CloudWatch.
Real-Time Cost Visibility: Providing immediate, up-to-date insights into cloud spend for timely decision-making and anomaly detection. Common service: Amazon CloudWatch Metrics Insights.

Rightsizing and Resource Optimization

Rightsizing and resource Optimization ensure cloud resources are appropriately sized and efficiently used by continuously analyzing usage patterns and adjusting capacity to eliminate waste and meet actual demand, thereby reducing costs without compromising performance.

Recommendations / Best practices

Choose Optimal Model and Inference Types: Select foundation models and inference methods that precisely match your business needs to avoid paying for unnecessary capacity. Continuously evaluate workload requirements and prefer smaller, purpose-fit models over default larger ones to save costs. Reference: Generative AI Cost Optimization Strategies
Batching and Concurrency: Efficiently batch inference requests and manage concurrency to maximize instance utilization and reduce cost per token or operation. Reference: GenAI Cost Optimization: The Essential Guide
Right-Sizing and Model Selection: Regularly right-size infrastructure—compute, memory, GPU—to workload demand, using autoscaling, spot, and reserved instances to balance cost and performance. Avoid defaulting to high-end hardware for all workloads. Reference: Optimizing GenAI Usage.
Leverage Cloud-Specific Cost Management Tools: Use cloud vendor cost management and advisory tools to identify and implement cost-saving recommendations. Common service: AWS Compute Optimizer.

Intelligent Pricing Strategies: Reserved, Spot, and Preemptible Instances

Reserved instances offer significant discounts for long-term, steady workloads by committing to a specific resource usage over one to three years, helping reduce costs compared to pay-as-you-go pricing. Spot and preemptible instances allow access to spare cloud capacity at substantially lower prices but with the risk of interruption, ideal for flexible or fault-tolerant tasks. Balancing these options with real-time workload needs enables cost-efficient cloud resource management while maintaining scalability and performance.

Recommendations / Best practices

Reserved Instances and Commitment Pricing: Reserve instances or commit to savings plans for consistently running workloads to gain discounts of 30-70%. These long-term commitments reduce cost predictability and environmental stability. Reference: Reserved Instances for Amazon EC2 overview.
Spot: Use spot for interruptible, fault-tolerant workloads like training and batch processing to save up to 90%. These resources are offered at deep discounts but can be reclaimed with short notice, requiring workload resilience and automation to manage interruptions. Reference: Amazon EC2 Spot Instances.
Auto-Scaling and Capacity Reservations: Pair spot and reserved instances with auto-scaling and capacity reservations to dynamically adjust resources based on workload demand, Optimizing cost and performance balance. Reference: Amazon EC2 Auto Scaling.

Automation and Dynamic Scaling

Automation and dynamic scaling enable cloud resources to automatically adjust in real time to changing workload demands, ensuring efficient performance during peak times while minimizing costs by scaling down when demand is low. This approach reduces manual intervention, optimizes resource use, improves reliability, and supports business agility by maintaining responsiveness under fluctuating traffic conditions.

Recommendations / Best practices

Automation and Idle Shutdown: Implement automated policies that stop, pause, or scale down AI model endpoints and compute resources during idle or low-traffic periods to avoid unnecessary costs. This dynamic management prevents paying for unused capacity, especially in development and batch workloads. References AWS Compute Optimizer.
Serverless and Event-Driven Compute: For variable or unpredictable inference workloads, leverage serverless compute options to pay strictly for consumed resources and scale automatically. This approach reduces operational overhead and costs. Reference: GenAI Accelerator Starter Package.
Dynamic Scaling and GPU Pooling: Use autoscaling and GPU pooling techniques (e.g., multi-instance GPU technologies) to maximize hardware utilization, reducing idle time and enabling more efficient processing of batch or concurrent inference tasks. This can significantly improve utilization from typical 25%+ levels to over 60%. Reference: Optimizing GenAI Usage

Cost-Aware Model and Workflow Design

Adopting a cost-aware approach to model and workflow design ensures financial insights are embedded in every step of the development lifecycle. By prioritizing real-time cost visibility, proactive forecasting, and iterative policy refinement, teams can anticipate spend early, align resource usage with business intent, and implement rapid adjustments as requirements evolve. This mindset promotes conscious decision-making, enabling organizations to balance performance and efficiency from the ground up.

Recommendations / Best practices

Optimize prompt design and token usage: Design applications with cost-aware prompting by minimizing prompt size and engineering efficient prompts. This reduces model invocations and token consumption, directly controlling costs. References: Generative AI Lens - Cost Optimization, Effect of Optimization on AI Forecasting.
Use prompt routing, caching, and inference Optimization: Route requests to the most cost-effective models and cache frequent prompts to reduce expensive token processing. This approach can cut inference costs by 40-70%, according to FinOps guidance. Target inference workloads for Optimization since they account for 80-90% of GenAI spending. Reference: Optimizing GenAI Usage
Monitor and apply governance per FinOps best practices: Incorporate real-time cost monitoring, forecasting, and governance aligned with FinOps principles to drive iterative cost improvements during the AI model lifecycle. Reference: Effect of Optimization on AI Forecasting

Quotas, Monitoring, and Anomaly Detection

Monitoring quotas and detecting anomalies with alerts ensures cloud resources are managed proactively. Setting alerts before limits are reached helps prevent service disruptions and enables timely capacity planning. This practice keeps cloud workloads reliable and cost-effective across environments.

Recommendations / Best practices

Granular Monitoring and Cost Tracking: Utilize advanced cost management tools with customizable dashboards to monitor usage and spending trends closely. Implement automated alerts and anomaly detection powered by machine learning to identify unexpected cost spikes and deviations early, enabling proactive cost control. References: AWS Cost Anomaly Detection, Cloud Cost Management.
Utilization and Quotas Management: Continuously monitor resource use across all clouds and set quotas to prevent overruns and runaway costs. Identify idle or low-traffic endpoints to shut down or consolidate, which reduces unnecessary spend. Apply quota management on large AI model endpoints to enforce cost limits during experimentation. Reference: Automate quota management.
Usage Pattern Analysis and Feedback: Establish continuous monitoring solutions to detect idle or under-utilized resources and optimize workflow efficiency. Encourage feedback loops between teams to align cost reduction with operational needs, following FinOps best practices. Reference: Cost Estimation of AI Workloads

Storage and Data Lifecycle Management

Efficient storage and data lifecycle management are key to controlling cloud costs. Implementing automated lifecycle policies helps transition data across storage tiers based on access patterns and retention needs, while regularly auditing for orphaned or stale data prevents unnecessary spending. Embedding these practices early in the provisioning process ensures cost Optimization throughout the data lifecycle.

Recommendations / Best practices

Lifecycle and Storage Policies: Implement automated data lifecycle management for model training datasets by shifting data to lower-cost storage tiers as access patterns change and removing obsolete or redundant data to reduce storage costs. This reduces provisioning waste and aligns storage use with business needs. Reference: AWS Data Lifecycle Management.
Efficient Storage and Data Handling: Optimize data pipelines and storage choices by selecting cost-effective storage classes and managing data flow to minimize expensive resource usage during data processing steps that do not require high performance. References: AWS Cost Optimization, Cost Estimation of AI Workloads

Team Enablement, Training, and Cost Ownership

Empowering teams with clear cost ownership and targeted training fosters accountability and cost-conscious decision-making. Embedding cost awareness into daily workflows and providing role-specific education helps teams balance innovation and budget, driving a culture of shared responsibility for cloud spending.

Recommendations / Best practices

Team Accountability: Assign cost owners and embed cost awareness into engineering workflows, training, and planning. Empower teams to make model design and usage decisions with full visibility of financial impact. References: AWS Cost optimization, FinOps Education & Enablement.

Forecasting, Budgeting, and Predictive Insights

Accurate forecasting, budgeting, and predictive insights enable organizations to anticipate cloud costs, align spending with business goals, and prevent budget overruns. Leveraging historical data, driver-based forecasting, and machine learning models helps create dynamic, actionable forecasts that drive financial accountability and proactive cost management.

Recommendations / Best practices

Accountability, Budget Control, and Forecasting: Assign cost ownership to workload teams and integrate showback or chargeback mechanisms to increase cost visibility and accountability. Use continuous forecasting tools that leverage historical data and growth plans to dynamically adjust budgets and commitments, aligning spending with business objectives. References: AWS Practice Cloud Financial Management, Exploring Cloud Cost Forecasting, Cost Estimation of AI Workloads.

Governance, Policy, and Tooling Automation

Automating governance policies ensures consistent compliance, security, and cost control in the cloud. By embedding policies into infrastructure workflows and deployment pipelines, organizations reduce manual errors and enforce rules proactively. This approach enables scalable, reliable oversight and quick remediation across diverse cloud environments.

Recommendations / Best practices

Governance and Automation: Use Optimization tools to recommend rightsizing, automatically terminate idle workloads, and enforce cost policies at scale for efficient cloud resource management. References: AWS Cost Optimization Pillar – Governance, Optimize Usage & Cost.

Summary

In this long blog post, I have shared recommendations from various aspects for embedding FinOps practices as part of the design, deployment and maintenance of modern applications containing GenAI services.

Any organization must have proper design and visibility into the cost aspects of any application using GenAI components to avoid high cost, or at least be able to track expected costs as soon as possible.

I encourage the readers to review the hyper-scale cloud providers' documentation, understand service cost, and learn about best practices for cost Optimization.

I also encourage the readers to learn from the FinOps Foundation's official documentation and best practices as they deploy GenAI services.

Disclaimer: AI tools were used to research and edit this article. Graphics are created using AI.

Additional references

About the author

Eyal Estrin is a seasoned cloud and information security architect, AWS Community Builder, and author of Cloud Security Handbook and Security for Cloud Native Applications. With over 25 years of experience in the IT industry, he brings deep expertise to his work.

Connect with Eyal on social media: https://linktr.ee/eyalestrin.

The opinions expressed here are his own and do not reflect those of his employer.