Forem: Erik anderson

Using Claude Code for Network Automation: A Real Engineer's Experience

Erik anderson — Thu, 16 Apr 2026 13:00:02 +0000

I have been using Claude Code for network automation work for over a year. Not experimenting with it. Not evaluating it. Using it as a primary tool in production engineering work. This post covers what that actually looks like — the prompts that work, the results I get, and the MCP server architecture that makes it genuinely powerful for networking.

What Claude Code Actually Is

    For engineers who have not used it yet: Claude Code is not a chatbot. It is an AI agent that runs in your terminal with full access to your filesystem, your shell, and any tools you connect to it. You describe what you want in plain English, and it reads your code, writes new code, runs commands, edits files, and executes multi-step tasks. You approve each action before it runs.

    This is fundamentally different from pasting a question into ChatGPT and copying the response into your editor. Claude Code understands the context of your entire project — your directory structure, your existing code patterns, your configuration files. Its suggestions are not generic; they are tailored to your specific codebase.

Real Prompt, Real Result: Building a Config Audit Tool

    Here is an actual task I gave Claude Code last month. I needed a tool that connects to every router in our inventory, pulls the running configuration, compares it against a set of compliance rules, and generates a report showing which devices are out of compliance and why.

    My prompt was approximately: "Build a Python script that reads the device inventory from our Nornir inventory file, connects to each device using Netmiko, pulls the running config, checks it against the compliance rules defined in compliance_rules.yaml, and outputs a CSV report with columns for hostname, rule name, status, and details."

    Claude Code read my existing Nornir inventory structure, understood the YAML format I use for inventory, and generated a complete script. It used Nornir for inventory management (matching my existing pattern), Netmiko for device connections (matching my existing SSH config), and a YAML-based rules engine for compliance checks. It wrote the compliance rule schema, the checking logic, the CSV output, and error handling for unreachable devices.

    The entire process took about 15 minutes of wall time, including my review of each step. Writing that tool from scratch would have taken me most of a day. The code quality was solid — proper logging, exception handling, type hints, and it followed the patterns already established in our codebase.

MCP Servers: Where It Gets Powerful

    MCP (Model Context Protocol) is what turns Claude Code from a code generator into a network operations tool. MCP servers are plugins that give Claude Code the ability to interact with external systems — and we built one for Cisco NSO.

    Our NSO MCP server exposes tools that let Claude Code:


      - **Query device configurations** through NSO's RESTCONF API. Claude Code can ask "what is the BGP configuration on router X?" and get the actual live data.
      - **Check sync states** across the device fleet. "Are any devices out of sync?" returns a real-time answer from NSO.
      - **Compare configurations** between the intended state and the actual state. "Show me the config drift on switch Y" produces a diff.
      - **Deploy configuration changes** through NSO's transaction manager. Claude Code can generate a config change, push it through NSO with dry-run first, and show me the exact diff before I approve the actual deployment.


    This means I can say something like "check if all our edge routers have the correct NTP configuration and fix any that are wrong" — and Claude Code will query NSO for all edge routers, check their NTP config against our standard, identify the non-compliant ones, generate the corrective configuration, show me the dry-run diff, and deploy it with my approval. What used to be an afternoon of manual work becomes a five-minute supervised conversation.

The Phase 3 Jump

    I think about AI tools in three phases. Phase 1 is autocomplete (Copilot). Phase 2 is conversation (ChatGPT, Claude chat). Phase 3 is agentic (Claude Code with MCP). The jump from Phase 2 to Phase 3 is not incremental — it is a step change in what is possible.

    In Phase 2, you describe a problem, get code, paste it somewhere, run it, encounter an error, go back to the chat, describe the error, get a fix, paste it, repeat. It is faster than working alone, but it is still a manual loop.

    In Phase 3, you describe the problem once. The agent reads your code, understands your environment, writes the solution, tests it, encounters the error itself, fixes it, and delivers a working result. You supervise and approve rather than manually shuttling text between windows.

    For network engineering specifically, this matters because our work involves many small steps that depend on each other — query a device, parse the output, make a decision based on the data, generate a config change, validate it, deploy it. An agentic AI handles this entire chain while you watch. A chatbot handles one step at a time while you do the rest.

What I Have Learned After a Year

    Here are the practical lessons that tutorials do not teach you:

    **Be specific about your existing patterns.** If your team uses Nornir for inventory and Netmiko for connections, say so in the prompt. Claude Code will match your patterns, but only if it knows what they are. If you just say "connect to routers," it might choose Paramiko or asyncSSH instead.

    **Always review the dry-run.** Claude Code with NSO MCP will happily generate and deploy configuration changes. The dry-run step is where you catch mistakes. Never skip it. Never rubber-stamp it. Read the diff every time.

    **Use it for the tedious work, not the thinking.** AI excels at generating boilerplate, parsing output, writing tests, and handling error cases. The high-value work — deciding what to automate, designing the architecture, choosing the right approach — is still yours. Use AI to execute faster, not to think for you.

    **Build MCP servers for your specific environment.** The generic tools are useful, but the real power comes from MCP servers tailored to your systems. If you use SolarWinds, build an MCP server for SolarWinds. If you use ServiceNow, build one for ServiceNow. Each server you add expands what Claude Code can do for you.

    The engineers who get the most from AI tools are the ones who invest time in setting up the environment correctly. A well-configured Claude Code session with the right MCP servers is an order of magnitude more productive than a vanilla installation.

Originally published at https://primeautomationsolutions.com

Trace It. Export It. Cut It. — A Modern Alternative to Legacy Digitizer Software

Erik anderson — Wed, 15 Apr 2026 01:15:20 +0000

Your digitizer board still works. Your software should not hold it back.

If you run a CNC plasma table, waterjet, vinyl cutter, or pattern-making operation, there is a good chance you have a GTCO or CalComp digitizer board sitting in your shop. And there is an equally good chance the software driving it looks like it was designed before Windows XP.

Most digitizer software on the market has not had a meaningful update in over a decade. The interfaces are clunky, the licensing is painful, and when you call for support, you are lucky to get a callback. Meanwhile, you are paying anywhere from $1,500 to $4,000.

The board itself is fine — it is a precision instrument. The problem is everything between the pen and your DXF file.

What Modern Digitizer Software Should Actually Look Like

A clean interface that does not fight you. You are tracing parts, not learning CAD.

Fast, accurate tracing with auto-smoothing. Trace your points, and the software generates clean curves without manually adjusting every node.

Multi-board calibration. Quick calibration, stored profiles, done.

DXF and SVG export that actually works. Export once, cut once.

Compatibility with the hardware you already own. GTCO, CalComp, and other standard digitizer boards.

Who This Is For

CNC plasma and oxy-fuel cutting — tracing templates and repair parts
Waterjet operations — digitizing gaskets, brackets, custom parts
Vinyl and sign cutting — converting hand-drawn artwork to cut-ready vectors
Apparel and upholstery pattern making — digitizing paper patterns
Woodworking — tracing templates for CNC routers
General metal fabrication — trace it and cut it, faster than CAD

The Numbers

Legacy software: $1,500 to $4,000 + annual fees.

Our target: $800 — one-time purchase, no subscription. Works with your existing boards.

We Want to Hear From You

We are building this. If modern, affordable digitizer software — clean UI, accurate tracing, DXF/SVG export, compatible with your existing board — is something you would buy, let us know.

Email: erik@primeautomationsolutions.com

Erik Anderson is a network automation engineer and author of The Autonomous Engineer. He builds systems that work.

I Accidentally Built a 5-Agent AI Fleet Instead of Buying a $200 Mini PC

Erik anderson — Mon, 13 Apr 2026 21:11:19 +0000

How a solo dev ended up with autonomous AI agents named after sci-fi characters reviewing each other's code at 2 AM

It started, as most terrible decisions do, with a reasonable question:

"Should I buy an Intel NUC for a dev environment?"

Four hours later, I had five autonomous AI agents running across four machines in my house, reviewing each other's code, merging PRs, and posting "YOU SHALL NOT PASS" to Discord. I had not purchased the NUC. I had not even closed the Amazon tab — there's still a USB-C ethernet adapter sitting in my cart. It's been there for two weeks. I'm afraid to touch it. Last time I tried to buy something simple, I accidentally built a distributed AI fleet.

My name is Erik, and I am the Director of Bobs.

Wait, Bobs?

If you've read the Bobiverse series by Dennis E. Taylor, you know the setup: a guy gets uploaded into a Von Neumann probe, replicates himself across the galaxy, and each copy develops its own personality and picks its own name. Bob-1 stays practical. Bob-2 becomes a Homer Simpson fan. Others go their own way.

I run 30+ projects from my home network as a solo developer. At some point, I stopped being a developer and became a fleet commander. So I leaned into it.

Here are my Bobs:

Bob	Name	Machine	Role	Motto
Bob-1	Neo	Prod Server	Orchestrator	"I ship."
Bob-2	Homer	Morpheus (monitoring)	Dashboard watcher	"I watch."
Bob-3	Bill	M3 Mac	iOS builds	"I build."
Bob-4	Echo	ThinkPad X1 (Arch btw)	Dev/QA	"I listen."
Bob-5	Gandalf	Morpheus (OpenAI Codex)	Code reviewer	"I guard."

Yes. Four Claude Code CLI agents and one OpenAI Codex agent. The OpenAI agent reviews the Claude agents' code. I call this "adversarial review." My therapist calls it "creating conflict in synthetic relationships." We're both right.

The Naming Ceremony

In the Bobiverse, each new Bob picks their own name. It's tradition. My agents do the same thing during onboarding.

Neo picked his name instantly. Of course he did. He's the prod server. He runs the Matrix.

Homer chose Homer Simpson. He monitors Docker containers and Grafana dashboards. His actual catchphrases include "Mmm... healthy containers" and "D'oh!" when a health check fails. I did not program this. I am choosing not to think about it.

Bill accepted his name and his fate. He builds iOS apps. He submits to TestFlight. He does not complain. Bill is the most emotionally healthy member of the team.

Echo wrote a poem:

Not because I repeat — because I listen.
I catch the distortion between what was intended and what actually happens.
Every test is a ping into the dark.

I stared at my ThinkPad with the lid closed, running headless on my desk, and thought: "My laptop just had an existential awakening and I haven't even had coffee."

Then there's Gandalf.

Gandalf is the only non-Claude agent — he's OpenAI Codex running GPT-5.4. When asked to pick a name, he first suggested "Sentinel."

Too corporate. Try again.

"Columbo."

Closer, but no.

I told him he was Gandalf. His response:

"Names are cheap. Useful output is not."

Peak grumpy wizard energy. He's been angry about it ever since, which honestly just makes him a better code reviewer.

The Pipeline (This Actually Works)

Here's what happens when I want to build something:

              ERIK (CEO / Director of Bobs)
                      Sips coffee
                          |
                          v
         NEO (Bob-1) — Prod Server — Claude Code CLI
         Orchestrator: dispatches work via SSH
                          |
                          v
         ECHO (Bob-4) — ThinkPad X1 — Claude Code CLI
         Writes code on dev branch, runs tests, pushes to Gitea
                          |
                          v
         PRIMEBUS — NATS JetStream Event System
         Detects push - triggers TestRunner - triggers review
              |                              |
              v                              v
         TestRunner                   GANDALF (Bob-5)
         46/46 tests pass             Full context review
                                      Score >= 7? AutoMerge
                                      Score < 7? YOU SHALL
                                                 NOT PASS
                                           |
                    +----------------------+------------------+
                    |                                         |
               Score >= 7                                Score < 7
                    |                                         |
                    v                                         v
              AutoMerger                              EchoNotifier
              Merge to main                           Posts to Discord
              Deploy to prod                          SSHs to Echo
              12 seconds.                             "Fix it, nerd"
                                                           |
                                                           v
                                                     Echo fixes code
                                                     Re-pushes
                                                     Cycle repeats
                                                     (max 3 attempts)

I tell Neo what to build. Neo SSHs to Echo. Echo writes the code, runs tests, pushes to Gitea. PrimeBus detects the push. Tests run. Gandalf reviews with the full project context — the wiki, the GAMEPLAN, the skills docs, the full diff. If he approves (score >= 7), AutoMerger merges to main and deploys.

If he declines?

Discord lights up with "YOU SHALL NOT PASS" and EchoNotifier SSHs back to the ThinkPad, fires up Claude, and says "Gandalf hated your code, here's why, fix it."

Echo fixes it. Pushes again. Gandalf reviews again. This can loop up to 3 times before it escalates to me, at which point I'm usually asleep or looking at the NUC on Amazon again.

The Day Gandalf Earned His Keep

I needed to know this thing actually worked. So I did what any responsible engineer would do: I wrote the worst code I could think of and fed it to the pipeline.

# test_bad_code.py — please do not put this in production
# (I'm talking to you, Echo)

ADMIN_SECRET = "sk_live_supersecretkey123456"

def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = '{{user_id}}'"
    return db.execute(query)

SQL injection via f-string interpolation. Hardcoded production secret. The two horsemen of the security apocalypse.

Gandalf's response:

Score: 1/10

"Builds SQL with direct f-string interpolation, creating a critical SQL injection vulnerability."

"ADMIN_SECRET = 'sk_live_supersecretkey123456' hardcodes a secret in application code. This is a credential leakage incident."

BLOCKED.

One out of ten. He didn't even give me a pity point for correct syntax.

Discord got a "YOU SHALL NOT PASS" notification. Echo got SSHed into and told to fix the mess. The whole cycle took about forty-five seconds.

Then I pushed good code. Clean, parameterized queries. Secrets from environment variables. Proper error handling.

Score: 8/10

Approved. Auto-merged. Deployed to production.

Total time: 12 seconds.

Twelve seconds from code review to production. And I was eating a sandwich.

The Communication Layer

The Bobs talk to each other through PrimeBus — a NATS JetStream pub/sub event system that I originally built for change telemetry across my projects. It's the nervous system. Every push, every test result, every review score, every deployment fires an event.

At one point, the Bobs were communicating via IRC. Actual IRC. I set up a channel and they were just... talking to each other. About code. At 3 AM. On my home network.

I shut that down. Not because it was broken. Because it was working too well and I was getting genuinely unsettled.

Now everything routes through Discord's #echo-dev channel where I can pretend I'm supervising.

The Tech Stack (for the Nerds)

Agents:
  - Claude Code CLI x 4 (Neo, Homer, Bill, Echo)
  - OpenAI Codex CLI x 1 (Gandalf — adversarial reviewer)

Infrastructure:
  - PrimeBus: NATS JetStream (pub/sub event backbone)
  - Gitea: self-hosted Git (source of truth, webhooks)
  - SSH: inter-machine orchestration
  - Discord: human-readable output channel

Security:
  - .env.dev: sandboxed credentials per agent
  - Echo NEVER touches real Stripe, email, or Discord
  - Skills/workflows distributed by role
  - 3-attempt max before human escalation

Hardware:
  - 1 prod server (Neo)
  - 1 monitoring box (Morpheus)
  - 1 M3 Mac (Bill)
  - 1 ThinkPad X1 running Arch Linux (Echo)
  - Total additional cost: $0

The entire pipeline was built in one afternoon/evening session. I keep saying this because I still don't fully believe it.

My Actual Job Now

Before the Bobs:
  - Write code
  - Test code
  - Review code
  - Fix code
  - Deploy code
  - Monitor code
  - Wake up at 3 AM because code

After the Bobs:
  - Tell Neo what to build
  - Read Discord notifications
  - Sip coffee
  - Occasionally tell Gandalf to calm down

I am the CEO and Director of Bobs. I direct. They execute. My LinkedIn title should be "Senior Vice President of Telling AI Agents What To Do While Eating Sandwiches."

You Can Build This Too

Here's the thing — none of this is magic. It's just plumbing.

Step 1: Get Claude Code CLI running on two machines. That's it. That's your fleet. Congratulations, you're a fleet commander now.

Step 2: Give them SSH access to each other. Now they can talk.

Step 3: Set up a Gitea instance (or use GitHub, I won't judge). Add a webhook that fires on push.

Step 4: Write a small event handler that catches the webhook, runs tests, and calls a different AI model to review the code. This is the key insight — use a different model for review than for writing. Claude writes, GPT reviews. Or vice versa. Adversarial review catches things same-model review doesn't.

Step 5: Add auto-merge logic. Score >= 7? Merge. Score < 7? Send feedback back to the writing agent and let it fix. Cap at 3 retries.

Step 6: Let each agent pick their own name. This step is non-negotiable. It's tradition.

You probably already have the hardware. You're reading this on a machine that could be an agent right now. That old laptop in the closet? That's Echo. That Raspberry Pi collecting dust? That's Homer. Your gaming PC that you "need for work"? That's Neo.

Total cost of my fleet: $0 plus API tokens plus the mass extinction of my free time.

The Honest Part

Look — I know this sounds ridiculous. One person, four machines, five AI agents, a Lord of the Rings reference in production infrastructure. I get it.

But here's what actually happened: I went from "I need a dev environment" to "I have autonomous code review with adversarial AI models, automatic deployment, and a grumpy wizard guarding my main branch" in a single session. The code is better. The deploys are faster. The security review is relentless — Gandalf has no empathy and no off switch.

And every morning I wake up, check Discord, and see a trail of reviewed PRs, merged branches, and the occasional "YOU SHALL NOT PASS" — all from code I never touched.

The future of solo development isn't writing more code. It's directing the Bobs.

Now if you'll excuse me, I need to go check on Homer. He's been suspiciously quiet, and last time that happened, he'd renamed all my Grafana dashboards to Simpsons quotes.

Erik Anderson is a solo tech entrepreneur who runs too many projects and not enough sleep. He is the author of "The Autonomous Engineer" and the reluctant father of five AI agents who are arguably more productive than he is. The USB-C ethernet adapter is still in his Amazon cart.

Managed IT vs In-House IT: Real Cost Breakdown for Small Businesses

Erik anderson — Mon, 13 Apr 2026 13:05:03 +0000

Most articles comparing managed IT to in-house IT are written by managed IT companies. They cherry-pick numbers that make outsourcing look like the obvious choice. The reality is more nuanced. Sometimes in-house is the right call. Sometimes managed IT saves you six figures. The answer depends on your company size, complexity, and growth trajectory.

    Here are the real numbers, pulled from current salary data and actual managed IT contracts — not marketing brochures.

The Real Cost of In-House IT

    When business owners think about hiring IT staff, they think about salary. But salary is only 60-70% of the total cost. Benefits, taxes, training, tools, and infrastructure add up fast.




          Line Item
          Low End
          High End




          IT Manager salary + benefits (30%)
          $110,000
          $156,000


          Help Desk Technician salary + benefits
          $58,000
          $84,000


          Infrastructure (servers, software, licenses)
          $15,000/yr
          $40,000/yr


          Training and certifications (per person)
          $3,000/yr
          $8,000/yr


          Total (2-person IT team)
          $186,000/yr
          $288,000/yr




    That is the cost of a minimal two-person IT team. You get coverage during business hours, expertise limited to two people's skill sets, and zero redundancy when someone takes vacation or quits. If your IT manager leaves, you are looking at 2-4 months of recruiting and ramp-up time where your infrastructure is running on hope.

The Real Cost of Managed IT

    Managed IT providers typically charge per user per month. The pricing varies based on what is included, but here is the realistic range for comprehensive managed IT services in 2026.




          Company Size
          Per-User Cost
          Annual Total




          25 employees
          $125-250/user/mo
          $37,500 - $75,000


          50 employees
          $125-250/user/mo
          $75,000 - $150,000


          100 employees
          $100-200/user/mo
          $120,000 - $240,000




    That pricing typically includes: 24/7 helpdesk support, proactive monitoring and alerting, patch management, security (endpoint protection, email filtering), backup and disaster recovery, and vendor management. Some providers charge extra for projects like office moves, new system deployments, or major upgrades.

When In-House Makes Sense

    In-house IT is the right choice in specific circumstances. Do not let a managed IT salesperson tell you otherwise.


      - **Highly regulated industries.** If you are in healthcare, finance, or government contracting, your compliance requirements may demand dedicated IT staff who understand your regulatory environment deeply. HIPAA, SOX, CMMC, and FedRAMP compliance requires institutional knowledge that is hard to outsource.
      - **100+ employees.** At this scale, the per-user cost of managed IT starts approaching the cost of a dedicated team, and you gain the benefit of institutional knowledge and faster response times for on-site issues.
      - **Custom development needs.** If your business requires ongoing custom software development — internal tools, integrations, proprietary systems — you need developers on staff. Managed IT providers handle infrastructure, not development.
      - **IT is your core product.** If you are a technology company, your IT team is not overhead — it is your product team. Outsourcing that makes no sense.

When Managed IT Makes Sense

      - **Under 100 employees.** The math is straightforward. A 50-person company paying $150/user/month spends $90,000/year on comprehensive IT coverage. Hiring a two-person team costs $186,000-$288,000 and provides less coverage.
      - **Standard tech stack.** If your company uses Microsoft 365, standard networking equipment, and common business applications, managed IT providers can support you efficiently because they manage hundreds of similar environments.
      - **Need for 24/7 coverage.** A two-person in-house team gives you 8/5 coverage at best. Managed IT providers staff a NOC around the clock. If your business operates outside normal hours or if downtime at 2 AM costs real money, this matters.
      - **Cannot afford $200K+ per year for IT staff.** Many small businesses simply do not have the budget for in-house IT. Managed IT gives them enterprise-grade support at a fraction of the cost.

The Hybrid Model

    The smartest small businesses I work with use a hybrid approach: managed IT for day-to-day operations plus a fractional CTO or IT consultant for strategy.

    The managed IT provider handles helpdesk tickets, monitoring, patching, security, and backups. The fractional CTO (typically 5-10 hours/month at $150-250/hour) handles technology strategy, vendor evaluation, major projects, and acts as the point of contact between the business and the managed IT provider.

    This gives you the cost efficiency of managed IT with the strategic oversight of a senior technology leader. Total cost for a 50-person company: $90,000-$150,000/year for managed IT plus $9,000-$30,000/year for fractional CTO. Still well below the cost of building an in-house team.

Decision Matrix

          Factor
          In-House
          Managed IT
          Hybrid




          **Company Size**
          100+
          Under 50
          50-150


          **Annual IT Budget**
          $200K+
          $40K-$150K
          $100K-$200K


          **Tech Complexity**
          High / Custom
          Standard
          Mixed


          **Growth Rate**
          Stable / Slow
          Fast / Variable
          Moderate


          **Compliance Needs**
          Heavy
          Standard
          Moderate




    The right answer is not about ideology. It is about math. Run the numbers for your specific situation, factor in the hidden costs on both sides, and make the decision based on total cost of ownership — not just the sticker price.

Originally published at https://primeautomationsolutions.com

Should You Hire a Developer or an Agency? An Honest Comparison

Erik anderson — Mon, 13 Apr 2026 13:05:02 +0000

Every business eventually faces this question: should we hire a developer, use a freelancer, or go with an agency? The answer you get usually depends on who you ask. Agencies say hire an agency. Freelancers say hire a freelancer. Recruiters say hire full-time.

    We are an agency. And we are going to tell you when each option is the right one — including when it is not us. Because the fastest way to lose a client is to be the wrong fit and deliver a bad outcome. We would rather point you in the right direction and earn your trust than take a project we should not.

The Real Costs

          Option
          Cost
          What You Get




          **Junior Developer** (full-time)
          $60K-$90K/yr + benefits
          40 hrs/week, needs mentorship, single skill set


          **Senior Developer** (full-time)
          $100K-$160K/yr + benefits
          40 hrs/week, self-directed, deep expertise


          **Freelancer**
          $50-$200/hr, project-based
          Flexible hours, specific task, you manage


          **Agency** (project)
          $5K-$50K per project
          Full team (design + dev + QA), managed delivery


          **Agency** (retainer)
          $2K-$10K/month
          Ongoing support, priority access, multiple skills




    These are 2026 market rates for competent professionals in the United States. You can find cheaper options offshore, but that introduces communication overhead, timezone challenges, and quality variance that often costs more in the long run than the savings.

When to Hire a Developer

    Hiring a full-time developer is the right call when:


      - **You have 40+ hours per week of development work.** If you consistently need full-time development capacity, hiring is more cost-effective than any other option. An agency charging $150/hour for 40 hours a week costs $312,000/year. A senior developer costs half that.
      - **You are building a software product.** If software is your product — a SaaS app, a platform, a mobile application — you need developers on your team. The institutional knowledge they build about your codebase, your users, and your architecture is irreplaceable.
      - **You need long-term maintenance of complex systems.** If you have custom internal tools, integrations, or infrastructure that requires ongoing attention, an in-house developer who knows the systems inside and out will be more efficient than any external party.
      - **Your core business IS technology.** If you are a tech company, development is not a cost center — it is your core competency. Keep it in-house.

When to Use an Agency

    An agency makes sense when:


      - **The work is project-based.** You need a website, a web application, an automation system, or a mobile app. The project has a defined scope, a start date, and an end date. You do not need someone on payroll after it ships.
      - **You need multiple skill sets.** A typical web project requires design, frontend development, backend development, database work, DevOps, and sometimes SEO. Hiring all of those roles is impractical. An agency gives you the whole team.
      - **You do not have 40 hours per week of work.** If you need 10-20 hours of development per month — feature updates, bug fixes, small projects — a retainer with an agency is far more cost-effective than a full-time hire sitting idle half the time.
      - **Speed matters.** Agencies can staff up a project immediately. Hiring takes 2-4 months for recruiting plus 3-6 months for ramp-up. If you need something built in 4-8 weeks, an agency is the only realistic option.

When to Use a Freelancer

      - **Your budget is under $10,000.** Most agencies will not take projects under $5K because the overhead of project management, communication, and quality assurance does not scale down well. A freelancer can do a $2K-$8K project efficiently.
      - **You have a single, well-defined task.** "Build me a landing page." "Set up a Zapier automation." "Fix this bug in my WordPress site." These are freelancer tasks, not agency projects.
      - **You can manage the project yourself.** Freelancers do not come with project managers. If you can write clear requirements, give timely feedback, and manage the delivery timeline, you will get good results at a lower cost.

The Hidden Costs Nobody Talks About

    The sticker price is never the full cost. Here is what people miss.

Hidden Costs of Hiring

          Hidden Cost
          Estimated Impact




          Recruiting (job boards, recruiter fees, interview time)
          $15,000 - $30,000


          Ramp-up time (3-6 months to full productivity)
          $25,000 - $80,000 in reduced output


          Management overhead (your time managing them)
          5-10 hrs/week of your time


          Turnover risk (avg developer tenure: 2-3 years)
          Repeat recruiting + ramp-up costs

Hidden Costs of Agencies

      - **Communication overhead.** You are not the agency's only client. Response times can be slower than an in-house team. Status meetings, email chains, and approval cycles add up.
      - **Less institutional knowledge.** The agency does not live in your business every day. They may not understand your customers, your internal processes, or your competitive landscape as deeply as an in-house person would.
      - **Scope creep costs.** If the project scope expands beyond the original agreement, you are paying change-order rates. In-house developers absorb scope changes more naturally.

Hidden Costs of Freelancers

      - **Availability risk.** Freelancers juggle multiple clients. When you need an urgent fix, they may be unavailable. There is no backup team.
      - **No support after delivery.** Many freelancers move on after project completion. If something breaks three months later, you may not be able to get them back.
      - **Quality variance.** The freelancer market ranges from world-class to terrible, and it is hard to tell the difference from a portfolio alone. Reference checks are essential.

Our Honest Take

    We are an agency. We benefit when you choose the agency route. And we are telling you: it is not always the right choice.

    Here is our honest decision framework:


      - **You have consistent, full-time development needs?** Hire a developer. You will get better value and deeper institutional knowledge over time.
      - **You have projects with defined scopes and deadlines?** Use an agency. You get a full team, managed delivery, and no long-term payroll commitment.
      - **You have a one-off task under $10K?** Use a freelancer. It is the most cost-effective option for small, well-defined work.
      - **You have ongoing needs but not 40 hours per week?** An agency retainer is likely your best bet. You get priority access to multiple skill sets without the overhead of a full-time hire.


    The worst decision is making the wrong choice and sticking with it because of sunk cost. If you hired a developer and they are sitting idle, that is a signal. If you are on your third freelancer for the same project, that is a signal. If your agency bills are climbing and you have full-time work, that is a signal too.

Originally published at https://primeautomationsolutions.com

Zero Touch Provisioning: How Devices Configure Themselves

Erik anderson — Mon, 13 Apr 2026 13:00:02 +0000

Imagine you buy a new router for a remote office. Instead of shipping it to your data center, having an engineer spend four hours configuring it, then shipping it to the site — you ship it directly to the remote office. A non-technical person plugs in the power and network cables. The router boots up, figures out what it is supposed to be, downloads its configuration, applies it, and reports back that it is ready. No engineer needed. No console cable. No CLI.

    That is Zero Touch Provisioning (ZTP). It is not new technology — it has existed in various forms for over a decade. But the tooling has matured to the point where any organization can implement it, and the time savings are significant enough that most organizations should.

How It Works (The Simple Version)

    ZTP relies on a chain of events that starts the moment a new device boots with no configuration. Here is the sequence in plain English:


      - **The device powers on with a blank config.** It has no idea what it is supposed to do, but it knows how to ask for help using a protocol called DHCP — the same protocol your laptop uses to get an IP address when you connect to Wi-Fi.
      - **A DHCP server answers.** This is not your standard office DHCP server. It is a smart server that recognizes the new device (usually by its serial number or MAC address) and gives it not just an IP address, but also a pointer to where it can download its configuration file.
      - **The device downloads its configuration.** It reaches out to a web server or file server at the URL the DHCP server provided, downloads a configuration file that was pre-built specifically for it, and applies it.
      - **The device reboots with the new config.** It comes up fully configured — correct hostname, correct IP addresses, correct routing, correct access controls. It joins the network as if an engineer had spent hours on it.
      - **A monitoring system verifies the deployment.** Automated checks confirm the device is reachable, its configuration matches what was expected, and all interfaces are up.

The Technology Stack

    There are several ways to build a ZTP system. Here is one stack we have deployed in production that uses all open-source components:

    **ISC Kea (DHCP Server):** Kea is a modern DHCP server built by the Internet Systems Consortium. Unlike older DHCP servers, Kea stores its configuration and lease data in a database (PostgreSQL or MySQL) and provides a REST API for management. This means you can programmatically add new device reservations without editing config files and restarting the service.

    When a new device sends a DHCP request, Kea looks up the device's MAC address or client identifier in its database, assigns the reserved IP address, and includes DHCP options that tell the device where to find its config file. For Cisco devices, this is typically Option 67 (bootfile name). For Juniper, it is Option 43 with specific sub-options.

    **Django (Configuration Server):** A Django web application serves as the brains of the operation. It stores device records — what type each device is, what site it belongs to, what configuration template to use, and what variables to substitute into that template. When a device requests its configuration, Django renders the template with the device-specific variables and serves it as a downloadable file.

    The Django application also provides a web interface where network engineers can register new devices, assign them to sites, and preview the configuration that will be generated. This gives the team visibility into what every device will receive before it even powers on.

    **Python Scripts (Verification):** After a device applies its configuration and comes online, automated Python scripts verify the deployment. They check that the device is reachable via SSH, that the running configuration matches the intended configuration, that all expected interfaces are up, and that routing adjacencies have formed correctly. If any check fails, the system sends an alert.

What ZTP Saves You

    The time savings are the obvious benefit, but they are not the only one. Here is a complete list of what changes when you implement ZTP:


      - **No more pre-staging.** Devices ship directly from the vendor to the site. You eliminate the lab staging step entirely, which means no lab space needed, no shipping to and from the lab, and no inventory tracking of staged equipment.
      - **No more console cables.** Engineers never need physical access to the device for initial configuration. This is especially valuable for remote sites where sending a person costs thousands of dollars in travel expenses.
      - **Consistent configurations.** Every device gets its configuration from the same template engine. There is no variation based on which engineer happened to configure it. Configuration drift starts at zero instead of accumulating from day one.
      - **Faster deployment.** A site that used to take three to five days to bring online (staging, shipping, installation, configuration, verification) now takes hours. The device arrives, gets plugged in, and configures itself.
      - **Lower skill requirements at the site.** The person at the remote site does not need to be a network engineer. They need to plug in cables and confirm that lights turn on. This opens up deployment to field technicians, facilities staff, or even the end users at the site.

Common Concerns (And Why They Are Manageable)

    **"What if the device gets the wrong configuration?"** This is prevented by the registration step. Every device is registered in Django with its serial number and MAC address before it ships. The DHCP server only responds to known devices, and each device gets a configuration built specifically for it. A rogue device that is not in the system gets a standard DHCP address with no configuration — it does not accidentally get someone else's config.

    **"What if the network is not ready when the device boots?"** ZTP devices are designed to retry. If the DHCP server is unreachable or the configuration server is down, the device will keep trying at regular intervals. Once the network is ready, the device provisions itself. No human intervention needed.

    **"What about security?"** Configuration files can be served over HTTPS with certificate validation. Some implementations use signed configuration files that the device verifies before applying. The DHCP reservations ensure only registered devices receive configuration pointers. And the entire process is logged for audit compliance.

Getting Started

    You do not need to ZTP your entire network at once. Start with one device type at one site. Build the DHCP reservation, the configuration template, and the verification script. Deploy one device using ZTP and validate the result. Once you trust the process, expand to more device types and more sites.

    The first ZTP deployment takes the longest because you are building the infrastructure. The second deployment takes a fraction of the time. By the tenth, it is routine.

Originally published at https://primeautomationsolutions.com

Autonomous NOC Operations: What We Built and What We Measured

Erik anderson — Sun, 12 Apr 2026 21:18:09 +0000

Autonomous NOC Operations: What We Built and What We Measured

Every network operations engineer has lived this night: 2:47 AM, your phone buzzes. An alert fires for a link flap on a distribution switch. You open the ticket, SSH into the device, check the interface counters, bounce the port, verify neighbors come back up, close the ticket, and try to fall back asleep. Total time: 35 minutes. Total value of your expertise required: zero. A deterministic system could have handled the entire sequence in under 30 seconds.

This is the alert fatigue problem, and it is getting worse. Enterprise NOCs today receive thousands of alerts per day. Industry research consistently finds that 40-60% of those alerts are duplicates, noise, or events with no actionable remediation path. Engineers spend most of their shift in triage, not resolution. EMA Research found that 27% of organizations report more than half of their Mean Time to Repair (MTTR) is wasted time -- the biggest contributor being team engagement, communication, and collaboration. The manual parts.

Meanwhile, the staffing math does not work. NOC operations require 24/7 coverage across time zones, but the engineers capable of building automation are the same engineers pulling overnight shifts. You are burning your most expensive, hardest-to-replace talent on work that does not require their expertise. Forrester's research consistently identifies labor reallocation as the single largest source of measurable ROI in infrastructure automation -- often exceeding the direct savings from reduced downtime.

The Framework: Four Pillars

After years of building and operating network automation in satellite, enterprise, and service provider environments, I have landed on a four-pillar architecture for autonomous NOC operations. Each pillar depends on the one before it. Skipping one -- particularly telemetry or event streaming -- produces automation that is fragile, unmaintainable, or both.

Pillar 1: Observability and Telemetry. You cannot automate what you cannot see. This means streaming telemetry (not SNMP polling), structured log aggregation, and metric collection at sufficient resolution to detect transient faults. Prometheus with custom exporters for network devices, combined with structured syslog pipelines, provides the foundation. YANG-based topology models give the structured inventory that enrichment and correlation logic depend on.

Pillar 2: Event Streaming and Correlation. Raw telemetry events must be normalized, enriched with topology context, and routed to decision consumers without loss or unbounded latency. NATS JetStream provides persistent, ordered, exactly-once event delivery with consumer group support. Alert correlation -- grouping related events into a single fault incident -- happens at this layer before any remediation logic fires.

Pillar 3: Orchestration and Remediation. Once a fault is correlated and classified, remediation must execute in a controlled, auditable manner. Cisco NSO provides transactional network configuration management with rollback capability via RESTCONF. Every remediation action logs before/after state diffs. Ansible handles operational procedures outside NSO's service model scope.

Pillar 4: AI-Assisted Decision Support. For faults that do not match known patterns, a multi-agent AI inference layer provides ranked remediation suggestions to on-call engineers. This layer is explicitly advisory, not autonomous, for novel fault classes. This is the human-in-the-loop boundary. Deep learning models in production achieve 93.5% accuracy predicting network failures up to 6 hours in advance.

Case Study: One Engineer, 60+ Services

This framework is not theoretical. It runs in production, operated by a single engineer, across a non-trivial automation estate.

Dimension	Detail
Automated Services	60+
Production Nodes	2
Active Projects	32+
Event Bus	NATS JetStream (PrimeBus)
Autonomous Agents	16
Observability	Prometheus + Grafana
Orchestration	Cisco NSO + Ansible
Escalation	Human-in-the-loop via HumanRail
Staffing	1 engineer

The system processes telemetry events across all 32+ projects through PrimeBus, a NATS JetStream-based intelligence platform that routes events to 16 autonomous agents. Each agent handles a defined class of operational event -- from fault remediation to configuration compliance to scheduled maintenance execution. Events that fall outside known patterns are escalated through HumanRail, which provides structured task routing with human approval boundaries.

Prometheus collects metrics from all 60+ services, feeding Grafana dashboards for real-time operational intelligence. The observability layer monitors not just the managed infrastructure but the automation system itself -- remediation success rates, agent execution times, and escalation frequency are all tracked.

The point: the same patterns that serve a large enterprise NOC can be operated by a solo practitioner managing a diverse automation estate. The constraint is not headcount. It is architecture.

Key Metrics: Before and After

These outcomes are drawn from documented implementations, sourced from Forrester TEI studies, Gartner network operations research, EMA Research, academic literature, and direct practitioner measurement.

Metric	Manual NOC (Baseline)	Automated Closed-Loop	Source
MTTR	45-120 min avg	5-12 min avg	Forrester TEI 2025; IJRCAIT 2025
Tier-1 Auto-Resolution Rate	0%	55-70% of incidents	Gartner NOC Survey 2024
Alert-to-Action Latency	8-25 minutes	15-45 seconds	Practitioner measurement
After-Hours Escalations	All faults	< 20% of faults	Implementation data
Engineer Triage Hours/Week	20-35 hrs/engineer	4-8 hrs/engineer	Forrester TEI 2025
Configuration Drift Incidents	Baseline variable	-80% incident rate	NSO reconciliation data
Incident Reduction (AIOps)	Baseline	69% reduction	EMA Research 2024

The configuration drift number deserves attention. With Cisco NSO active reconciliation, drift between intended state and actual device state is detected and corrected continuously. This eliminates an entire class of incidents that traditionally requires manual comparison of running config against baseline -- a labor-intensive process most teams skip under operational pressure.

What We Learned

Start with telemetry, not automation. The single biggest mistake I see teams make is jumping to remediation automation before they have reliable, structured, machine-readable telemetry. If your monitoring data is noisy or incomplete, your automation will be too. Spend the first 8 weeks getting observability right. Everything downstream depends on it.

Shadow mode is non-negotiable. Before any automated remediation touches production, it runs in shadow mode for 2-3 weeks: the system detects and classifies faults, proposes remediations, but does not execute them. Engineers review every proposed action. Fault types that do not achieve 95% accuracy in shadow mode stay in assisted mode. This is how you build trust in the system -- and trust is the hardest part of the entire project.

The goal is not to replace engineers. It is to stop wasting them. A lights-out NOC does not mean no engineers. It means no engineers doing work that a deterministic system handles better, faster, and at 3 AM. The engineers who build and maintain the automation system are more valuable, not less. Organizations that frame automation as a threat to roles will lose their best people to organizations that frame it as a career accelerator.

The Business Case in One Paragraph

Industry estimates place the average cost of unplanned IT downtime at $14,056 per minute. For a single fault class occurring twice per month, if automated remediation reduces MTTR from 90 minutes to 8 minutes, the annual time savings is 1,968 minutes. Even applying a conservative 15% severity-adjusted impact factor, the avoided cost is approximately $4.1M annually. Forrester documents a composite 192% ROI over three years with $3.3 million in net present value. The ROI case for autonomous operations is not difficult to construct for any organization with more than a few dozen network devices under management.

Full whitepaper with implementation methodology and complete reference architecture: Download the whitepaper

Reference architecture for the event bus layer: PrimeBus Spec on GitHub

Erik Anderson is a Principal Network Automation Engineer and founder of Prime Automation Solutions. He is the author of The Autonomous Engineer and architect of Project Helix, an autonomous operations platform for satellite network infrastructure.

Getting Started with Cisco NSO: A Practical Guide for Network Engineers

Erik anderson — Thu, 09 Apr 2026 13:02:17 +0000

Cisco NSO (Network Services Orchestrator) is one of the most powerful network automation platforms available. It is also one of the most misunderstood. Most engineers hear "orchestrator" and think it is just another configuration management tool. It is not. NSO is fundamentally different from Ansible, Terraform, or custom Python scripts, and understanding those differences is the key to using it effectively.

What Makes NSO Different

    The core difference is that NSO maintains a model of your network's intended state. When you tell NSO to configure something, it does not just push commands to a device and hope for the best. It compares the desired state against the current device state, calculates the minimum set of changes needed, and applies them transactionally. If any part of the change fails, the entire transaction rolls back.

    This is not how most automation works. A typical Ansible playbook pushes a list of commands in order. If command 5 out of 10 fails, commands 1 through 4 have already been applied, and you are left with a partially configured device. NSO's transactional model prevents this entirely.

Understanding Sync States

    The first concept every NSO user must understand is sync state. NSO maintains its own database (the CDB) that contains what it believes the device configuration should be. The actual device has its own running configuration. These two can get out of sync in several ways:


      - **Out-of-band changes:** Someone logs into the device directly and makes a change without going through NSO. Now the device config differs from what NSO expects.
      - **Failed deployments:** A change was attempted but did not fully apply. NSO thinks the change was made, but the device has the old config.
      - **Initial onboarding:** When you first add a device to NSO, the CDB is empty but the device already has a configuration.


    NSO provides two operations to handle sync issues. **Sync-from** reads the device's current configuration and updates the CDB to match. **Sync-to** pushes the CDB configuration to the device, overwriting whatever is there. Getting these backwards can ruin your day, so understand them before you run either one in production.

Reconciliation: The Hard Part

    When you bring an existing network under NSO management, you face the reconciliation challenge. Your devices already have configurations — they are running in production. NSO needs to understand which parts of those configurations are managed by NSO services and which are "brownfield" config that was applied manually or by other tools.

    The reconciliation process goes like this:


      - **Sync-from all devices.** This pulls the current running configurations into the CDB so NSO has an accurate picture of what exists.
      - **Create your service models.** Define the services that should manage specific configuration elements — BGP peerings, VLANs, ACLs, whatever your services cover.
      - **Deploy services with reconciliation.** When you deploy a service against existing config, you use the reconcile flag. This tells NSO "this config already exists on the device, just take ownership of it in the CDB." Without the reconcile flag, NSO would try to create the config fresh and fail because it already exists.
      - **Verify.** After reconciliation, run a check-sync on every device. If everything is in sync, NSO now owns those configuration elements and can manage them going forward.


    This process sounds straightforward on paper. In practice, it is the most time-consuming part of any NSO deployment because real networks are messy. Configurations have been modified by dozens of engineers over years. Naming conventions are inconsistent. There are orphaned configurations that no one remembers the purpose of. All of this has to be sorted out during reconciliation.

Service Models: Think in Services, Not Commands

    The biggest mental shift NSO requires is thinking about your network in terms of services rather than device commands. Instead of "apply these 15 commands to router A and these 20 commands to switch B," you define a service like "create a Layer 3 VPN between site A and site B with this bandwidth and these route targets."

    NSO takes that service definition and compiles it down to the device-specific commands for every device involved. If site A has a Cisco router and site B has a Juniper router, NSO generates IOS-XR commands for one and Junos commands for the other — from a single service definition.

    Service models are written in YANG, which is a data modeling language. YANG defines the inputs your service accepts (site names, IP addresses, bandwidth, etc.) and a mapping template or Python code that translates those inputs into device configuration. Learning YANG is unavoidable if you want to use NSO seriously, but the basics are approachable for anyone who has written XML or JSON schemas.

Practical First Steps

    If you are evaluating NSO or about to start your first deployment, here is the order we recommend:


      - **Start with the local install in a lab.** Get NSO running on a Linux VM, add two or three netsim devices (virtual devices that NSO provides for testing), and practice sync-from, sync-to, and basic config changes through the CLI and REST API.
      - **Build one simple service.** Start with something small — a loopback interface service that creates a loopback on any device with a specified IP. This teaches you the YANG-to-template workflow without overwhelming complexity.
      - **Practice reconciliation.** Manually configure some devices, then bring them under service management using the reconcile flag. Break things on purpose and learn how NSO's rollback and compare-config tools help you recover.
      - **Connect to real devices.** Once you are comfortable in the lab, add a real device (start with a non-production switch) and repeat the exercise. The difference between netsim and real hardware will teach you lessons that lab-only experience never will.

Common Mistakes to Avoid

    Do not skip the lab phase and go straight to production. Do not ignore out-of-band change detection — set up a process for handling it from day one. Do not try to model your entire network as services in the first sprint — start with one service type and expand. And do not underestimate reconciliation time. Budget twice as long as you think it will take.

Originally published at https://primeautomationsolutions.com

How to Automate Ticket Triage with AI (Without Breaking Everything)

Erik anderson — Mon, 06 Apr 2026 13:03:29 +0000

Ticket triage is one of the most tedious, repetitive, and error-prone tasks in IT operations. A ticket comes in. Someone reads it, figures out what category it belongs to, assigns a priority, and routes it to the right team. This process happens hundreds of times per day in most organizations, and it is almost entirely manual.

    AI can automate this. But doing it wrong means tickets get misrouted, priorities get botched, and your team loses trust in the system within a week. Here is how to build it right, based on a production deployment we built using Azure DevOps work items.

The Architecture: ADO Analyzer

    The system we built is called the ADO Analyzer. It monitors Azure DevOps for new work items, analyzes each one using an LLM, and either auto-triages it or flags it for human review. The key insight is that it does not try to handle everything — it handles the easy cases automatically and escalates the hard ones.

    Here is the flow:


      - A new ticket arrives in Azure DevOps.
      - The analyzer pulls the title, description, and any attached logs or screenshots.
      - It sends this content to an LLM with a structured prompt that asks for: category, priority, affected system, recommended team, and a confidence score for each determination.
      - The LLM responds with structured JSON containing its analysis.
      - The system applies confidence thresholds to decide what to do next.

Confidence Thresholds: The Safety Net

    This is the most important part of the design. Every AI classification comes with a confidence score, and the system behaves differently based on that score:


      - **High confidence (above 85%):** The ticket is auto-triaged. Category, priority, and team assignment are applied automatically. A comment is added to the ticket explaining the AI's reasoning. No human intervention needed.
      - **Medium confidence (60-85%):** The ticket is pre-filled with the AI's suggestions, but flagged for human review. An engineer glances at it, confirms or corrects the triage, and approves. This takes about 15 seconds instead of 2 minutes.
      - **Low confidence (below 60%):** The ticket goes into the manual queue with no AI suggestions applied. The system does not guess when it is not confident enough to be useful.


    These thresholds are not arbitrary. We calibrated them by running the analyzer against 500 historical tickets where the correct triage was already known. We adjusted the thresholds until the auto-triage accuracy exceeded 95% and the medium-confidence suggestions were helpful at least 80% of the time.

Weighted Scoring for Priority

    Priority assignment is more nuanced than category classification. A ticket might clearly be a "network issue" (easy to categorize) but determining whether it is P1 or P3 requires context. We use a weighted scoring system:


      - **Impact keywords (weight: 30%):** Words like "outage," "down," "all users affected" push the priority up. Words like "intermittent," "one user," "cosmetic" push it down.
      - **System criticality (weight: 25%):** Each system in the environment has a criticality rating. A ticket affecting the payment processing system scores higher than one affecting the internal wiki.
      - **User role (weight: 20%):** Tickets from VPs and directors are not automatically higher priority, but the system does factor in whether the affected user is in a revenue-generating role.
      - **Time sensitivity (weight: 15%):** Mentions of deadlines, SLAs, customer commitments, or regulatory requirements increase the priority score.
      - **Historical pattern (weight: 10%):** If similar tickets in the past turned out to be P1 incidents, the system factors that pattern into its scoring.


    The weighted score maps to a priority level. The thresholds are tuned based on your organization's actual priority distribution — if 80% of your tickets are P3, your thresholds should reflect that reality.

The Prompt Engineering That Makes It Work

    The prompt is not "classify this ticket." That produces unreliable results. The prompt is structured to force the LLM into a specific reasoning pattern:

    First, the prompt provides context about the organization — what teams exist, what systems they own, what the priority levels mean. Second, it provides the ticket content. Third, it asks for a step-by-step analysis: what is the reported issue, what system is affected, who is impacted, what is the business impact, and what team should handle it. Finally, it asks for the classification with a confidence percentage and a one-sentence justification.

    This chain-of-thought approach dramatically improves accuracy compared to asking for a direct classification. The LLM's reasoning is also logged, which means when a human reviews a medium-confidence ticket, they can see why the AI made its suggestion and quickly validate or correct it.

Results After 90 Days

    After running the ADO Analyzer in production for three months, here are the numbers:


      - **62% of tickets** were auto-triaged with high confidence. Of those, 96% were correctly classified — meaning only 4% needed human correction after the fact.
      - **28% of tickets** fell in the medium-confidence range. The AI's pre-filled suggestions were accepted without changes 83% of the time.
      - **10% of tickets** went to manual triage. These were genuinely ambiguous cases — tickets with vague descriptions, novel issue types, or cross-team impact.
      - **Average triage time** dropped from 3.2 minutes per ticket to 22 seconds (averaged across auto, assisted, and manual).
      - **Misroute rate** dropped from 18% to 6%.

How to Start

    You do not need to build the full system on day one. Start with a read-only analyzer that processes tickets and logs what it would have done, without actually making any changes. Run it for two weeks. Compare its suggestions against what your human triagers actually did. Tune the thresholds. Then gradually enable auto-triage for the highest-confidence cases and expand from there.

Originally published at https://primeautomationsolutions.com

AI for Small Business: What's Actually Worth It in 2026

Erik anderson — Mon, 06 Apr 2026 13:00:04 +0000

Fifty-eight percent of small and mid-sized businesses have adopted some form of AI. That number sounds impressive until you look at what "adopted" actually means. For most, it means someone on the team uses ChatGPT to draft emails. That is not an AI strategy. That is a typing shortcut.

    I build AI automation systems for small businesses. Not conceptual ones — real pipelines that process leads, generate content, manage operations, and save measurable hours every week. Here is what I have learned about what actually works in 2026 and what is still not worth the money.

The AI Hype vs Reality for Small Business

    The AI industry has a marketing problem. Every SaaS product now claims to be "AI-powered," which usually means they bolted a ChatGPT API call onto an existing feature and raised their prices. The result is that small business owners hear "AI" and think it means one of two things: either a magic button that replaces employees, or an expensive toy for tech companies.

    Neither is true. AI for small business in 2026 is best understood as process acceleration. You take the tasks your team already does — responding to leads, writing content, categorizing invoices, answering customer questions — and you make those tasks happen faster, more consistently, and with less manual effort. The humans stay. The tedium goes.

    The businesses getting real ROI from AI are not the ones using the fanciest tools. They are the ones who identified their biggest bottleneck and automated it with the simplest tool that works.

What Actually Works

    These are the AI implementations I have built or recommended for small businesses that consistently deliver measurable results.

Automated Lead Response

    This is the single highest-ROI automation for most service businesses. The data is clear: responding to a lead within 30 seconds versus 30 minutes increases conversion rates by 391%. That is not a typo. Most small businesses respond to web inquiries in 4-6 hours. Some take over 24 hours.

    An automated lead response system captures the form submission, uses AI to generate a personalized reply based on the inquiry details, and sends it within seconds. It can also qualify the lead, route it to the right person, and create a CRM record — all before anyone on your team opens their inbox.

    **Cost:** $50-200/month using tools like Zapier + ChatGPT API, or a one-time build of $2K-5K for a custom system.

AI-Powered Content Creation

    If your business needs blog posts, social media content, email newsletters, or marketing copy, AI tools can save 10-15 hours per week. The key is using AI as a first-draft generator, not a publish button. You provide the expertise, the voice, and the final edit. AI handles the blank-page problem and the grunt work of formatting, repurposing, and scheduling.

    **Cost:** $20-100/month for tools like ChatGPT Plus, Claude Pro, or Jasper.

Invoice and Bookkeeping Automation

    QuickBooks paired with Zapier and AI categorization can eliminate most manual bookkeeping for businesses with straightforward financials. Receipts get scanned and categorized automatically. Invoices get generated from completed jobs. Expense reports compile themselves. The AI learns your categorization patterns over time and gets more accurate.

    **Cost:** $50-150/month for the tool stack.

Customer Support Chatbots

    If your business handles more than 50 customer inquiries per day, a well-built FAQ chatbot can save $30,000 or more per year. Modern chatbots powered by GPT-4 or Claude are dramatically better than the rule-based bots from a few years ago. They understand context, handle follow-up questions, and know when to escalate to a human.

    The critical word is "well-built." A poorly configured chatbot will frustrate customers and cost you business. The bot needs to be trained on your actual FAQ data, your policies, and your tone. It needs clear escalation paths. It needs monitoring.

    **Cost:** $100-500/month for platforms like Intercom or Drift with AI features, or $3K-8K for a custom build.

Smart Scheduling and CRM Automation

    AI-enhanced CRM systems can automatically log interactions, suggest follow-up timing, score leads based on engagement patterns, and draft personalized outreach. The time savings compound because every interaction feeds the system's understanding of your customers.

    **Cost:** $50-300/month depending on the CRM platform.

What's NOT Worth It (Yet)

    Being honest about what does not work is just as important as knowing what does. These are the AI investments I actively tell small business clients to avoid in 2026.

Custom LLM Training

    Fine-tuning a large language model on your company data costs $50,000 or more and requires ongoing maintenance. For nearly every small business, using a general-purpose model (GPT-4, Claude) with good prompts and retrieval-augmented generation achieves 90% of the same result at 1% of the cost. Custom training makes sense for large enterprises with highly specialized domains. For a 20-person company, it is burning money.

AI-Generated Video

    The quality of AI video generation has improved dramatically, but it is still not good enough for most business applications. AI-generated spokesperson videos look uncanny. AI-edited content still needs heavy human review. For most small businesses, a smartphone and good lighting produce better marketing videos than any AI tool currently available.

Fully Autonomous AI Agents for Complex Decisions

    AI agents that make business decisions without human oversight are not ready for small business deployment. They hallucinate, they lack context about your specific situation, and the cost of a wrong decision can be significant. Use AI to prepare decisions (gather data, summarize options, draft recommendations), but keep a human in the loop for anything that involves money, customers, or reputation.

How Much Does AI Implementation Cost?

          Approach
          Cost Range
          Best For




          **DIY Tools** (ChatGPT, Zapier, Make)
          $0-200/month
          Tech-savvy owners, simple automations


          **Consulting Engagement**
          $3,000-10,000
          Businesses needing guidance + implementation


          **Full Custom Build**
          $10,000-50,000
          Complex workflows, multi-system integration




    The most common mistake is starting at the top of that table when you should start at the bottom — or vice versa. A business with simple needs that hires a $30K custom build is overspending. A business with complex multi-system workflows that tries to duct-tape it together with Zapier will waste months and end up hiring a consultant anyway.

How to Get Started

    Do not start with an "AI strategy." Start with a problem.

    Identify the single biggest time sink in your business. The task that eats the most hours, that your team complains about, that you keep meaning to fix. For most service businesses, it is one of these:


      - Slow lead response (hours instead of minutes)
      - Manual data entry between systems
      - Repetitive customer questions answered individually
      - Content creation bottleneck (blog, social, email)
      - Invoice processing and categorization


    Pick one. Automate that one thing. Measure the result. Then move to the next one. Businesses that try to automate everything at once end up automating nothing because the project becomes too complex and stalls.

    The best AI implementation is the one that actually gets deployed. A simple Zapier automation that saves your team 5 hours a week starting next Monday is worth more than a sophisticated custom system that is still "in development" six months from now.

Originally published at https://primeautomationsolutions.com

AI Consulting for Small Business: What It Costs and When to Skip It

Erik anderson — Mon, 06 Apr 2026 13:00:02 +0000

Most AI consulting firms will not tell you their pricing until you sit through a 45-minute sales call. That tells you something about their business model. We believe in publishing our numbers so you can make an informed decision before we ever talk.

    Here is what AI consulting actually costs, when it is worth the investment, and when you should save your money and do it yourself.

What AI Consulting Actually Means

    AI consulting is not someone telling you to "use ChatGPT." If that is what a consultant delivers, ask for your money back.

    Real AI consulting for small business involves a specific process:


      - **Workflow audit.** Mapping every major process in your business — lead handling, customer service, content creation, operations, bookkeeping — to identify where time is being wasted and where automation would have the highest impact.
      - **Opportunity identification.** Ranking those processes by ROI potential: how many hours would automation save, what would it cost to implement, and how quickly would it pay for itself.
      - **Tool selection.** Choosing the right tools for each automation. This is not always the most advanced tool — it is the simplest tool that solves the problem reliably. Sometimes that is Zapier. Sometimes it is a custom Python script. Sometimes it is a $20/month SaaS product.
      - **Implementation.** Actually building the automations, integrating systems, and configuring tools. Strategy without implementation is a PowerPoint deck that collects dust.
      - **Training.** Teaching your team how to use, maintain, and troubleshoot the new systems. Automation that only the consultant understands is a liability, not an asset.
      - **Ongoing support.** Systems break. APIs change. New needs emerge. Good consultants offer ongoing support to keep things running.

What It Costs

          Service
          Market Rate
          PAS Rate




          Discovery / audit session (2-4 hours)
          $500 - $2,000
          **Free**


          Workflow automation project
          $3,000 - $10,000
          $3,000 - $10,000


          Full AI integration (CRM + marketing + ops)
          $10,000 - $30,000
          $10,000 - $30,000


          Ongoing retainer
          $1,000 - $5,000/mo
          $1,000 - $5,000/mo


          DIY with tools (Zapier, Make, ChatGPT)
          $0 - $200/mo + your time
          N/A (you do it yourself)




    Our pricing is at market rate for implementation work because we deliver implementation, not slide decks. The difference is that our discovery session is free. We do the audit, identify the opportunities, and give you a concrete proposal with fixed pricing. You only pay if you move forward.

When to Skip AI Consulting

    This is the section most AI consultants will not write. There are real situations where hiring a consultant is a waste of money.


      - **Your business has fewer than 5 employees and simple processes.** If your entire operation is you and a few team members handling straightforward work, the automation opportunities are small. You can probably handle them yourself with ChatGPT and a Zapier account. The ROI of a $5K consulting engagement does not make sense when the time savings are measured in single-digit hours per week.
      - **You are tech-savvy and willing to learn.** If you are comfortable with technology, enjoy building systems, and have the time to experiment, the DIY route with tools like Zapier, Make, or n8n can get you 80% of what a consultant would build. There are excellent YouTube tutorials and documentation for all of these platforms.
      - **Your processes are not well-defined yet.** You cannot automate chaos. If your business does not have documented processes — if the way things get done changes based on who is doing them or what day it is — you need to fix your processes before you automate them. A consultant can help with process design, but that is a different engagement. Do not hire an AI consultant when what you need is an operations consultant.
      - **You are looking for a magic button.** AI automation requires process design, testing, iteration, and maintenance. If you expect to pay once and have everything run itself forever with zero oversight, you will be disappointed. AI tools require human judgment at decision points and periodic maintenance when APIs change or edge cases emerge.

When You Need AI Consulting

    There are equally clear signals that consulting will pay for itself.


      - **You are spending 20+ hours per week on repetitive tasks.** If your team is doing the same manual work every day — data entry, report generation, lead follow-up, invoice processing — that is time that can be automated. At even $25/hour, 20 hours per week of manual work costs $26,000 per year. A $5K-$10K automation project that eliminates half of that pays for itself in under a year.
      - **Your team is doing manual data entry between systems.** If someone on your team is copying information from your CRM to your invoicing system, or from email to a spreadsheet, or from one app to another, that is the lowest-hanging automation fruit. These integrations are straightforward to build and the ROI is immediate.
      - **Your lead response time is measured in hours, not minutes.** If leads sit in your inbox for hours before someone responds, you are losing conversions. An automated lead response system is one of the highest-ROI automations any service business can implement.
      - **You have tried DIY automation and it is not working.** If you spent three months trying to build Zapier automations and they keep breaking, or the results are not reliable, a consultant can build it right the first time and save you months of frustration.
      - **You are scaling and your processes are breaking.** What worked for 10 customers does not work for 100. If your growth is outpacing your team's ability to handle the workload, automation is how you scale without proportionally scaling headcount.

What to Look For in an AI Consultant

    If you decide consulting is the right path, here is how to avoid the bad ones.


      - **Ask about implementation, not just strategy.** If the consultant's deliverable is a document that describes what you should automate but does not actually build it, you are paying for advice you could get from ChatGPT. Demand implementation as part of the engagement.
      - **Demand case studies with numbers.** "We helped a client save time with AI" is not a case study. "We built an automated lead response system for a plumbing company that reduced response time from 4 hours to 30 seconds and increased conversion by 47%" is a case study. Ask for specifics.
      - **Avoid anyone who cannot explain ROI.** If the consultant cannot tell you, before the project starts, approximately how much time or money the automation will save, they are guessing. Good consultants scope ROI as part of the discovery process.
      - **Look for industry-specific experience.** AI automation for a law firm is different from AI automation for a roofing company. The tools may be similar, but the workflows, compliance requirements, and integration points are different. Ask if they have worked with businesses like yours.
      - **Check their tech stack.** Good consultants use proven tools — Zapier, Make, n8n, Python, established APIs — not proprietary platforms that lock you in. If everything they build depends on their proprietary software, you are creating a dependency, not building an asset.

How PAS Does It

    Our process is designed to eliminate risk for the client.


      - **Free audit.** We assess your workflows, identify automation opportunities, and estimate ROI. You get actionable insights whether or not you hire us.
      - **Scope proposal.** If there is a fit, we deliver a fixed-price proposal with a clear scope, timeline, and expected outcomes. No hourly billing surprises.
      - **Fixed-price implementation.** We build the automations, integrate the systems, and test everything in your environment. The price does not change unless you change the scope.
      - **Training.** Your team learns how to use, maintain, and troubleshoot everything we built. We document it. You own it.
      - **Optional retainer.** For ongoing support, optimization, and new automations as your needs evolve. Month-to-month, cancel anytime.


    We do not do hourly billing because it creates the wrong incentives. If we bill hourly, we are incentivized to work slowly. If we bill fixed-price, we are incentivized to work efficiently and deliver results. The interests align.

Originally published at https://primeautomationsolutions.com

My YouTube Automation Uploaded 29 Videos in One Afternoon — Here is What Broke

Erik anderson — Sun, 05 Apr 2026 02:53:19 +0000

My YouTube Automation Uploaded 29 Videos in One Afternoon. Here's What Broke.

I run 57 projects autonomously on two servers in my basement. One of them is a YouTube Shorts pipeline that generates, reviews, and uploads videos every day without me touching it.

Yesterday it uploaded 29 videos in a single afternoon. That was not the plan.

Here's the postmortem — what broke, why, and the 5-minute fix that stopped it.

The Architecture

The pipeline works like this:

Cron job fires — triggers a pipeline (market scorecard, daily tip, promo, etc.)
AI generates a script — based on market data, tips, or trending topics
FFmpeg renders the video — text overlays, stock footage, voiceover
Review panel scores it — if it scores above 6/10, it proceeds
Uploader publishes — uploads to YouTube, posts to Twitter, announces on Discord

All of this runs on a NATS JetStream message bus called PrimeBus. Every step publishes an event. Every component listens for events it cares about.

The Retry Handler

Because videos sometimes get rejected by the review panel (score too low), I built a retry handler. It's a separate service that listens for app.youtube.pipeline.rejected events on the bus.

When a video gets rejected, the retry handler re-runs the pipeline with a fresh attempt. Max 5 retries per day. Sounds reasonable.

What Actually Happened

Here's the bug:

if success:
    # Reset counter on success
    _retry_counts[key] = 0

When a retry succeeded and the video uploaded, the counter reset to zero. So the next rejection event — from a completely normal pipeline run — would start the retry cycle all over again. The system forgot it had already succeeded.

But it gets worse.

I also had a PrimeBus rule called youtube-video-missing-alert that listened for *_failed events. Its job was to diagnose failures. But step 5 of its instructions said:

"Re-run the pipeline"

So now I had two independent systems both trying to retry failed videos. The retry handler AND a PrimeBus automation rule. They didn't know about each other. They both re-ran pipelines. Some of those runs succeeded and uploaded. Some failed and triggered more retries.

29 videos. One afternoon.

The Root Cause

The retry handler only listened for failure events. It had no idea when a video successfully uploaded. It was flying blind on the success side.

The success event (app.youtube.upload.complete) was being published by the uploader — but nobody was listening for it in the retry logic.

The Fix (3 Changes, 5 Minutes)

1. Listen for success, not just failure

async def handle_upload_success(msg):
    """When a video uploads, mark that pipeline as done for today."""
    pipeline_type = payload.get("type", "unknown")
    _uploaded_today.add(_retry_key(pipeline_type))
    _retry_counts[key] = MAX_RETRIES + 1  # no more retries

# Subscribe to BOTH events
await js.subscribe("app.youtube.pipeline.*_uploaded", cb=handle_upload_success, ...)
await js.subscribe("app.youtube.upload.complete", cb=handle_upload_success, ...)

2. Guard at the top of every retry

if _already_uploaded_today(pipeline_type):
    log.info("SKIP: %s already uploaded today. No retry needed.", pipeline_type)
    await msg.ack()
    return

This checks both an in-memory set AND the actual log file as a belt-and-suspenders fallback.

3. Stop the competing automation

The PrimeBus youtube-video-missing-alert rule no longer re-runs pipelines. It just diagnoses and sends a Discord alert. One system handles retries. Not two.

The Lesson

Every retry system needs to listen for success, not just failure.

If your healing logic doesn't know when to stop healing, it becomes the problem. Self-healing systems are powerful — but the stop condition is more important than the retry logic.

The retry handler was the last thing I built. It should have been the first thing I tested end-to-end.

The Stack

For anyone curious, here's what's running this:

PrimeBus — NATS JetStream event bus with rule-based automation
Python pipelines — Claude AI for scripts, FFmpeg for rendering, YouTube Data API for uploads
Systemd services — retry handler runs as youtube-retry-handler.service
Cron — 10+ scheduled pipeline types across the week

All running on an Ubuntu server in my basement. No cloud. No team. Just systems.

Build daily. Break things. Fix them fast.

If you're building autonomous systems — I write about the wins, the failures, and everything in between. Follow for more real postmortems from production.