Forem: hagishun

How I Stopped Silent Regressions with GitHub Copilot Coding Agent and ADR

hagishun — Sun, 01 Mar 2026 11:24:30 +0000

TL;DR

While using Copilot coding agent, I ran into regressions where the design doc (file tree) drifted out of sync with the implementation — without anyone noticing
The root cause wasn't the AI. It was that assumptions kept changing, but we had no way to lock them down
I introduced ADR (Architecture Decision Record) to explicitly fix assumptions and have the agent execute steps in order
Result: the same class of regression hasn't happened since, and we got the workflow stabilized in about 2 hours

Background: The "Silent Regression" Problem

I was using Copilot coding agent to automate the flow from design → implementation → testing.

At some point, even though the implementation looked correct, I started noticing:

The design doc and the actual implementation were subtly misaligned
The file structure didn't match the original intent
Follow-up changes broke something elsewhere

This was a "regression that doesn't look like one at first glance" — and that made it hard to catch.

Root Cause: Not the AI — The Assumptions Weren't Locked

My first instinct was to blame:

The agent's understanding being too shallow?
Weak prompts?
Model limitations?

But when I traced the logs, that wasn't it.

What Was Actually Happening

The file tree definition in architect.md had changed mid-way
But the agent kept implementing based on the assumptions it read at the start
And I hadn't explicitly communicated that the assumptions had changed

In short:

The assumptions were changing, but there was no mechanism to lock or update them

That was the real cause.

Before vs. After: Visualized

Before: Assumptions drift

sequenceDiagram
    participant H as Human
    participant I as Issue
    participant A as Agent
    participant C as Code

    H->>I: File issue (assumptions at this point in time)
    Note over I: ⚠️ Assumptions change mid-way
    H->>I: Design changes via another PR
    I-->>A: Old assumptions passed to agent
    A->>C: Implements based on stale assumptions
    Note over C: Regression occurs (hard to notice)

After: Assumptions locked via ADR

sequenceDiagram
    participant H as Human
    participant ADR as ADR
    participant A as Agent
    participant C as Code

    H->>ADR: Document decisions (goal, constraints, verification)
    H->>A: Point Issue to ADR
    A->>ADR: Reads latest assumptions
    A->>C: Implements based on ADR
    A->>A: Runs verification — confirms zero drift
    Note over C: Implementation matches design

Why Agents Are Particularly Vulnerable to This

In a human team, you can share context implicitly — through conversation, memory, shared understanding. You fill in the gaps naturally.

Agents can't do that. They can't carry context across sessions. If the assumptions change but aren't explicitly communicated, the agent will keep treating whatever it last read as ground truth.

ADR has traditionally been framed as a "large team documentation practice." But in agentic development, it plays a different role: a handoff document from humans to the agent. The more you rely on agents, the more critical it becomes to have a clear answer to "where are the decisions written down?"

Reference: The github/gh-aw Design Philosophy

I drew inspiration from github/gh-aw — GitHub's reference repository for Agentic Workflows.

Its core philosophy is:

Humans explicitly write down what was decided
Agents work within those fixed assumptions
When assumptions change, decisions must be updated first

Solution: Lock Assumptions with ADR

The approach I took was ADR-based workflow.

What is an ADR?

Architecture Decision Record — a document that captures "what" was decided, "why," and "to what extent." See adr.github.io for more.

How I Used It

Scoped to Chapter 4 of architect.md: the file tree definition
Explicitly marked what was "already decided" vs. "still open"
Had the agent execute steps sequentially, using the ADR as its ground truth

Repository Structure

.
├── architect.md        # Overall design
├── adr/
│   └── ADR-001.md      # Locked decisions and assumptions
└── .github/
    └── copilot-instructions.md

Example ADR Content

Field	Content
Scope	File tree structure
Decision	Responsibility separation under `pkg/`
Out of scope	Future extension directories
Prohibited	Any structural changes without updating ADR first

How to Drive the Agent (The Important Part)

I wrote the following steps into the ADR and had the agent execute them in order:

Read the ADR
Check the relevant section of architect.md
Implement
Test and verify
Report results

This stopped two recurring failure modes:

Expanding scope beyond what was intended
Continuing to implement based on stale assumptions

Results: Regressions Became a Solvable Problem

Since switching to this workflow:

The same class of regression hasn't happened again
When something does go wrong, it's immediately clear which assumption broke
Review load on the human side has gone down

The AI didn't get smarter. We just locked the assumptions.

Key Takeaways

AI will fill in ambiguous assumptions on its own — that's not a bug, it's by design
If assumptions change, humans need to document the updated decision explicitly
Copilot coding agent is powerful, but it doesn't eliminate the need for design and decision-making

If anything:

"Humans decide. Agents execute."

The cleaner that division is, the more stable your development gets.

Closing Thoughts

Regressions aren't caused by AI. They're caused by workflows that don't record decisions.

ADRs are unglamorous. But in agentic development, they work surprisingly well.

What's Next (Under Consideration)

Publish an ADR template
Build a minimal sample based on the gh-aw structure
Turn this into a team-facing hands-on workshop

References

I Confused Copilot Coding Agent with Agentic Workflows — Turns Out the Guardrails Are the Point

hagishun — Sat, 21 Feb 2026 16:00:22 +0000

I Tried GitHub Agentic Workflows and It Was Nothing Like Copilot Coding Agent

This article is for developers who are already familiar with GitHub Actions and Copilot. The goal is to clarify how Agentic Workflows differs in design philosophy from what you might already know.

Background

In February 2026, GitHub announced "Agentic Workflows" as a technical preview.

https://github.blog/2026-02-16-automate-repository-tasks-with-github-agentic-workflows/

When I heard "AI that automates repository tasks," I jumped in to try it — and immediately got confused. This article documents that confusion along with what I actually learned by getting my hands dirty.

My Initial Mistake: Is Coding Agent = Agentic Workflows?

I asked Copilot Agent Mode in VS Code to walk me through experiencing GitHub Agentic Workflows. It guided me through this flow:

Set up a Python Todo app in a repository
Create a GitHub Issue
Assign Copilot Coding Agent to the Issue
A PR gets created automatically
GitHub Actions runs the tests

It was genuinely useful. Just by assigning Copilot to an Issue, a PR with test code was auto-generated.

But this was "Copilot Coding Agent" — not "Agentic Workflows."

I only realized this after re-reading the official announcement. They're completely different things.

Note: Copilot Coding Agent (assign to Issue → auto-generate PR) is an existing feature. GitHub Agentic Workflows (gh-aw), announced in February 2026, is a separate mechanism entirely.

What GitHub Agentic Workflows Actually Is

In one sentence

A system where you write "what you want done" in Markdown, and an AI agent continuously automates repository tasks based on that intent.

How it differs from GitHub Actions

	GitHub Actions (YAML)	Agentic Workflows (Markdown)
How you write it	Define "steps" in YAML	Write "intent" in Markdown
Who executes	Runs defined steps in order	AI interprets intent and acts autonomously
Decision-making	None (just if/else branching)	AI reads context and judges
Handling unknowns	Only predefined patterns	AI adapts to novel situations
Best for	Build, test, deploy	Repetitive tasks requiring judgment

Actions and Agentic Workflows coexist — one doesn't replace the other. In my own repository, test.yml (Actions) and issue-triage.md (Agentic Workflows) ran side by side without conflict.

How it differs from Copilot Coding Agent

	Copilot Coding Agent	Agentic Workflows
Trigger	Manually assign to an Issue	Auto-triggered by schedule or events
Purpose	One-off coding tasks	Continuous repository automation
Output	Creates a PR	Comments, labels, Issues, PRs, and more
How it's defined	Instructions in Issue body	`.github/workflows/*.md` files

Let's Walk Through It

Setup

# Install the gh-aw CLI extension
gh extension install github/gh-aw

# Initialize the repository
gh aw init

# Create a workflow template
gh aw new issue-triage

Writing the Workflow

I created .github/workflows/issue-triage.md:

---
on:
  issues:
    types: [opened, reopened]

permissions:
  contents: read
  issues: read

safe-outputs:
  add-comment:
    max: 1
  add-labels:

tools:
  github:
---

# Issue Triage

When a new Issue is created, automatically triage it.

## What to do

1. Read the Issue title and body to understand the content
2. Apply the appropriate label from the following:
   - `bug` - Bug report
   - `enhancement` - Feature request
   - `question` - Question
   - `documentation` - Documentation-related
3. Add a comment to the Issue:
   - Briefly summarize the Issue content
   - Mention any relevant source files if applicable
   - Suggest a priority level (high / medium / low)

## Rules

- Respond in English
- Keep comments polite and concise
- If unclear, apply the `question` label and ask for clarification

The key is the two-layer structure: YAML (frontmatter) + Markdown (intent):

Frontmatter: When to run, what it can read, what it's allowed to do
Markdown body: What you want the AI to do (natural language)

Compile and Push

# Generate a lock file (.lock.yml) from the Markdown
gh aw compile

# Push to the repository
git add -A && git commit -m "feat: add Issue triage workflow" && git push

Results

After creating an Issue, the following happened automatically.

Issue created: "I want the list command output to show timestamps"

What the AI did automatically:

✅ Applied the enhancement label (correctly identified as a feature request)
✅ Posted a comment automatically
✅ Identified the relevant source file (todo.py) down to specific line numbers
✅ Suggested an implementation direction
✅ Assessed priority as "medium"

All from just writing "here's what I want" in natural language. The AI read the repository context and made its own judgments.

Guardrails: The Most Important Part of the Design

What stuck with me most after trying this was how strict the guardrails are.

Precisely because you're handing the AI ambiguous intent, a system to prevent unintended behavior is essential. Agentic Workflows implements this through three layers of guardrails.

1. permissions (input constraints)

Controls what the AI can "read."

permissions:
  contents: read    # Can read code, but cannot write
  issues: read      # Can read Issues, but cannot close them

2. safe-outputs (output constraints)

Explicitly grants what the AI is "allowed to do."

safe-outputs:
  add-comment:      # Can add one comment max
    max: 1
  add-labels:       # Can add labels
  # create-pull-request ← Not listed, so PRs are off-limits

Real experience: When I initially wrote issues: write, the compile step rejected it with an error:

strict mode: write permission 'issues: write' is not allowed for security reasons.
Use 'safe-outputs.add-comment', 'safe-outputs.add-labels' to perform write operations safely.

Instead of "write access to do anything," the design enforces individually scoped output permissions like "only allow adding comments."

3. Sandboxed Execution

Network isolation
PRs are never auto-merged (humans must always review)

Why Guardrails Matter

What the AI might otherwise do	Guardrail
Close all Issues without permission	`close-issue` not in `safe-outputs` → blocked
Directly modify code	`permissions: contents: read` → read-only
Send data to external APIs	Network isolation

Allow ambiguity, but limit the blast radius — this is the design philosophy unique to agentic systems, and it's fundamentally different from CI/CD.

Where I Got Stuck

1. Confusing Copilot Coding Agent with Agentic Workflows

The flow that Copilot Agent Mode guided me through was "Actions + Coding Agent" — a completely different thing from Agentic Workflows (gh-aw). Interestingly, the AI assistant itself couldn't accurately distinguish between the two.

2. Secrets configuration is required

gh aw init → gh aw compile → push is not enough. You also need to configure a token for the AI engine.

# Create a Fine-grained PAT (personal account, with Copilot Requests permission)
# ※ Resource owner must be a personal account, not an Org
gh aw secrets set COPILOT_GITHUB_TOKEN --value "<PAT>"

3. A UI trap when creating the PAT

To get the "Copilot Requests" permission to appear on the Fine-grained PAT creation screen:

Set Resource owner to your personal account (it won't appear under an Organization)
Set Repository access to "Public Repositories" (required for the option to show)

This is because Copilot licenses are issued at the individual account level.

6 Use Case Patterns

From the official documentation:

Continuous triage — Auto-classify Issues and apply labels (what I tried today)
Continuous documentation — Auto-update README as code changes
Continuous simplification — Identify refactoring opportunities and create PRs
Continuous test improvement — Evaluate test coverage and auto-add tests
Continuous quality maintenance — Investigate CI failures and suggest fixes
Continuous reporting — Periodically generate repository health reports

Summary

What I learned	Details
Agentic Workflows ≠ Coding Agent	Completely different — it's about writing intent in Markdown for continuous automation
Not a replacement for Actions	Use Actions for CI/CD, Agentic Workflows for tasks requiring judgment
Guardrails are the core design	Three-layer defense: permissions + safe-outputs + sandbox
Setup gotchas	Secrets configuration, UI constraints when creating PATs

GitHub Agentic Workflows is currently in technical preview. The vision it points toward: you open your repository in the morning and Issues are already triaged, CI failures are explained, and documentation is up to date.

For Enterprise adoption, the next questions will be around governance: who is allowed to create Agentic Workflows, and how far should permissions go.

References

I Open-Sourced a 67-file GitHub Enterprise Playbook — Why and How I Built It

hagishun — Mon, 16 Feb 2026 06:20:31 +0000

I published an open-source GitHub Enterprise learning guide (67 files) based on real-world onboarding / enablement work.

I’d love feedback from the community:

Please leave questions or suggestions as GitHub Discussions / Issues
PRs are also welcome

Qiita (JP): https://qiita.com/hagix/items/a64ed1b6fc39454e1bb7
GitHub repo: https://github.com/NRI-Oxalis/Beginner_manual
GItHub Page: https://nri-oxalis.github.io/Beginner_manual/

The content is currently Japanese-first.
I'm planning to expand English coverage,
and community contributions are very welcome.