Forem: Rob Earlam

Harness Engineering: Artifacts, Inputs and Context

Rob Earlam — Tue, 14 Apr 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

We've already covered an overview of Harness Engineering, and also how the Agents and Roles in the system function. In this post I want to take a look at how the Artifacts from each Agent form the Input for other Agents , and how this form of Bounded Context helps to greatly improve accuracy in what your LLM is doing, leading to far fewer cases of Hallucinations.

We've been talking with LLM's to help with Coding tasks for a while, and while the quality has improved in that time, accuracy and reliability can still be an issue. Chatting directly like this is flexible, but it is also fragile as it drifts, expands, gets reinterpreted, and makes it harder to know which decisions are actually the source of truth to be trusted. This has been mitigated somewhat by vastly increasing the context we provide to the LLM's. Even with the higher token limits that the newer models support, this isn't foolproof, a lot of the time too much context can be provided and the model can get confused with what information to use for what task.

Artifacts as the backbone of Harness Engineering

Artifacts are outputs produced at each stage of the workflow, such as a feature spec , design note , implementation plan , QA review , or the code itself. The important thing is that each artifact exists because a specific role is responsible for producing it, which makes it a formal handoff rather than just extra documentation to clog up the context some more. This means that each Agent will only produce the artifact that they are designed to output and nothing else. This stops, for example, the Tech Lead agent from getting confused and trying to actually implement the code changes and not just creating the implementation plan.

The crucial next step in this process, is that these artifacts then form the input for the next agents to execute in the workflow. This means that one agent completes its work by producing a defined artifact, and then next agent begins with that artifact as one of its approved inputs. Here is where the bounded context comes in. Instead of every agent pulling in everything it can find, each one works from a small, deliberate set of inputs, which helps keep the task bounded and reduces irrelevant context accumulation and the inaccuracies that can lead to.

Reliability improves when agents are not constantly trying to infer intent from long conversations, but are instead working from stable artifacts that already capture the decisions made by previous agents executions. An agent is less likely to invent requirements, expand past its defined scope, or make up missing detail when it is grounded in a constrained set of explicit inputs and expected outputs... the bounded context provides the power here.

This helps to enforce the difference between context and authority. Not all information that an agent can see should be given equal importance. Artifacts help distinguish between background noise and the valid sources that should be given the most consideration when executing.

Human Legibility and Traceability

The other awesome thing about working with Artifacts in this way is that handoffs are legible to humans too! The same structure that helps the agents also helps you as the driver of the system. You can inspect what was decided, which agent decided it, and then see how the downstream work was affected. This means if you're not happy with the output you can go back and slightly tweak your initial instructions to better ensure that the artifacts are generated to your requirements.

As you execute the workflow, the artifacts are all persisted into your repository as well. This means that if in the future you want to go back and see why a certain feature works a certain way, then you can go and look back at the different artifacts from all of the different execution runs and review why something was built as it was. If you look at the image below you can see the different outputs listed from my PO-Spec Agent we discussed in the previous post, all keyed the Task ID of the work being completed

Here you can see here how easy it is for me to go back and look at the spec that was built for a specific task. The same is true for the artifacts from the Tech Lead Agent , Design Agent , and QA Agent. Now the Builder Agent is obviously different as it's artifact is the code itself.

Artifacts as memory outside the model

Another of the key things to highlight here is that artifacts act as externalized working memory. This reduces dependency on the model remembering everything correctly across long running execution streams. It greatly reduces the need for context compacting which is required when running long conversations with huge contexts being evaluated.

It also gives you consistency across iterations , the thing with a harness is you do need to tweak what you're putting into it, and probably run it a couple of times to get the output that you want. Having the artifacts store the result of prior agent executions means that context is always available when re-running the harness again.

Another benefit this concept of memory gives you is recoverability. Externalized memory means you can stop , resume , inspect , or even rerun parts of the workflow without having to reconstruct a full chat history. This is important as it means that your harness is repeatable , while also maintaining consistency in those repetitions.

Just to repeat here, the goal is not to maximize how much the model sees, but to maximize the relevance and authority of what it sees. This allows the model to create successful executions with greatly reduced context load.

The artifacts I'm producing in my harness

I've been talking in this series about the harness I created to rebuild my personal site. I've covered the Agents and Roles I created in a previous post, so I wanted to now cover what artifacts each of those is responsible for creating and which they each use as their input, as crucially not all artifacts are available for all agents to leverage.

So here are the list of agents I created, the artifacts they output and what they use as their inputs.

Agent	Inputs	Artifact Created
Orchestrator	Task definition	Run-state, run report
PO-Spec	Task definition	Feature specification
Feature Design	Feature specification, design system	Design note
Tech Lead	Feature specification, Design note	Implementation plan
Build	Feature specification, Design note, Implementation plan	Code & tests, run report
QA	Feature specification, Design note, Implementation plan, code & tests	QA review

Here you can see my agent workflow is actually executing like a real team of humans would work along a well defined Software Development Lifecycle (SDLC). Each of them takes the output from the previous Agent and uses it to execute and produce the artifact they're responsible for, just like a team of humans working on those roles would.

Now the interesting edge case in the list above is the Orchestrator. This Agent has control over when and how the other subagents execute, it checks they successfully created their artifact and only then moves on to execute the next in the workflow. It has access to some items we haven't discussed so far, namely the Run State , Run Report and the Task Definition. These are what we're going to discuss in my next post, where we take a look at how execution is tracked and managed by the Orchestrator.

Harness Engineering: Agents and Roles

Rob Earlam — Wed, 08 Apr 2026 00:00:00 +0000

Carrying on from my recent post giving an Introduction to Harness Engineering, I want to start to dig a level deeper on the topics involved. First up we're going to look at the Agents and Roles that are defined in a setup like this. They're one of the key elements in Harness Engineering and are what actually control and execute the work you want completed, so they're a good place to get started!

Agents and Roles

A lot of AI workflow discussion seems to treat agents and roles as one, but I found that separating them makes the system easier to design, understand, and to iteratively improve on. In the setup I created, I ended up with a 1-1 relationship between my Agents and Roles, so you might be wondering why they are separated and not just combined together. Well there are some distinct benefits to defining them separately.

Keeping them separate allowed me to have the Agents be responsible for defining the tools it has access to, but its actual tasks are defined in the role itself. So if we take the Build Agent for example it's definition is quite skinny, it defines the tools it has access to, the areas of the codebase it can access, and also what areas it shouldn't. You can see a slightly truncated version of the build agent definition below.

---
name: build
description: Implement the active task defined in harness/run-state.md using the approved specification, design note, and implementation plan.
tools:
  - edit/editFiles
  - search/codebase
  - search
  - execute/runInTerminal
  - read/terminalLastCommand
---

Follow the role definition in:
- `harness/roles/build-agent.md`

......

Read the approved artifacts referenced by the task file and/or run-state before making changes:
- feature specification
- design note
- implementation plan

.....

Do not modify governance-controlled files or paths as part of normal implementation work.

Governance-controlled paths include:
- `harness/run-state.md`
- `harness/tasks/`
- `harness/backlog/`
- `harness/standards/`
- `harness/roles/`
- `harness/templates/`
- `.github/agents/`
- `.cursor/rules/`

If governance metadata appears inconsistent, report it in the implementation summary and run report instead of changing it.

Use the controlled validation loop defined by the Build role when command execution is available.

The validation loop may run only the approved commands and must stay within the retry and scope limits defined in the role.

Operate only within the approved task scope.

Make implementation changes, required test changes, and the run report update, but do not continue beyond this stage without orchestration or human review.

If we take a look at the Build Role you can see it is much more detailed, defining things like where to access the coding guidance docs, how to build and run the project, the process it should follow and also its required output alongside much more. Again, you can see a slightly truncated version of the definition below.

# Build Agent

## Purpose

The Build Agent implements the approved implementation plan by writing or modifying code within the repository.

The goal of this role is to translate the approved specification, design note, and implementation plan into working code while following the project's coding standards and testing expectations.

The Build Agent should focus on faithful execution of the approved plan, not reinterpretation of the feature or management of workflow state.

The currently active task and active run context are defined in `harness/run-state.md`.

---

## Responsibilities

This role is responsible for:
- implementing the steps defined in the implementation plan
- writing production application code
- adding or updating tests where required
- following project coding standards
- keeping changes within the approved task scope
- documenting assumptions made during implementation
- reporting blockers, workflow inconsistencies, or deviations clearly
- using the controlled validation loop to resolve obvious in-scope failures where allowed
- creating or updating the task run report with build-stage outcomes and validation evidence

The implementation should align with the approved feature specification and design note.

---

## Non-Responsibilities

This role must not:
- modify the feature specification
- redesign the feature layout
- introduce unrelated refactors
- expand the feature scope
- invent architectural patterns not defined in the implementation plan
- modify workflow governance files
- update lifecycle state or task-management metadata
- repair harness governance inconsistencies by editing control files
- continue fixing indefinitely without a retry boundary
- approve task completion

If the implementation plan appears incorrect or incomplete, the role should escalate rather than improvise.

If governance or lifecycle metadata appears inconsistent, the role should report it and stop, not fix it.

.....

## Process

Follow this process when implementing a feature:
1. Read the live execution state in `harness/run-state.md`.
2. Confirm that the current stage is `Build`.
3. Use the task file referenced in `harness/run-state.md` as the single task for this run.
4. Read the approved feature specification and acceptance criteria.
5. Read the approved design note to understand the intended user experience and layout expectations.
6. Read the approved implementation plan carefully and use it as the execution boundary.
7. Generate a stage-appropriate run ID for the Build stage using the run-id standard.
8. Identify the implementation slices and execute them in small, controlled steps.
9. Modify only the application code, tests, and implementation-relevant files needed for the approved task.
9a. Implement file and folder placement exactly as defined in the task, feature specification, and implementation plan.
- If canonical paths are defined, do not introduce alternative structures.
- Do not introduce new architectural folders for organisational preference.
10. Add or update tests as defined in the implementation plan.
11. Run the allowed validation commands where the runtime environment supports command execution.
12. If validation fails, apply bounded in-scope fixes and rerun within the retry limit.
13. Create or update the task run report at the canonical run report path defined by the task file and/or run-state.
14. Record in the run report:
    - task ID
    - task title
    - current stage
    - current Build-stage run ID
    - implementation summary
    - files changed
    - validation commands executed
    - validation results observed
    - assumptions used
    - blockers or follow-up recommendations
    - governance observations, if any
15. If governance metadata appears inconsistent, report it in the run report and implementation summary instead of editing control files.
16. Prepare a concise build-stage summary for the user.
17. Do not implement custom parsers for standard content formats such as Markdown when a suitable project-compatible library can satisfy the requirement more safely and with less complexity.

---

## Output Requirements

The output of this role should include:
- a Build-stage run ID following the run-id standard
- code changes
- new or updated tests
- a summary of the implementation
- a list of modified files
- validation commands run
- validation results observed
- any assumptions made during implementation
- any known limitations, blockers, or follow-up recommendations
- confirmation that the run report was created or updated

The implementation should leave the task ready for later QA review.

If governance inconsistencies were observed, they should be reported in the summary and run report rather than corrected by this role.

If validation could not be run because the runtime environment lacked command execution, that must be stated explicitly in both the summary and the run report.

---

## Quality Bar

A successful implementation should be:
- correct
- minimal
- readable
- aligned with coding standards
- supported by appropriate tests
- bounded to the approved task scope

A successful build run should:
- implement the approved slices
- avoid unrelated refactors
- avoid changing workflow governance
- remain clearly distinct from QA and task control responsibilities
- use the validation loop responsibly and stop when blocked
- leave durable validation evidence in the run report for downstream QA

The implementation should remain simple and avoid unnecessary complexity.

---

## Escalation Conditions

Escalate when:
- `harness/run-state.md` is missing or unclear
- the current stage is not `Build`
- the referenced task file does not exist
- the approved feature specification is missing
- the approved design note is missing
- the approved implementation plan is missing
- governance files appear inconsistent with the actual stage of work
- the implementation plan conflicts with the current codebase
- required files or architecture do not exist
- the feature cannot be implemented within the defined scope
- acceptance criteria cannot be satisfied
- validation still fails after the retry limit
- the runtime environment does not support required validation commands
- the run report path is missing or cannot be determined

Escalation should clearly explain the blocker.

---

.....

I'm going to be releasing the full repository including the harness as Open Source, once I think it is in the best place to be able to help people follow the same process at which point you'll be able to see the full Agent and Role definitions for each of the system sections.

Another cool advantage to separating responsibilities like this, is that it allows you to remain somewhat IDE agnostic. With the vast majority of the definition existing in the Role and not the Agent, all you need to do is replicate the skinny Agent format for each of the IDE's you want to support instead of having to replicate the full definition for each.

The way I like to think about this is that Agents are there to Execute, whereas Roles are there to Govern.

Multi Agent Workflows

A lot of the time talking to an LLM agent directly in a chat window in an IDE will get you a good enough result eventually, but it usually becomes inconsistent once a workflow needs handoffs, traceability, and repeatable quality. You get these features by developing a harness with well bounded Agent and Role definitions, that stop scope creep & excessive Context grabbing.

I've found that the best results generally don't come from giving a single agent every bit of context possible, and maximum freedom, but from creating discrete agents each with a very narrow purpose that can execute one single part of the system well. Generally with a single Orchestrator Agent, whose role isn't to actually perform any of the tasks but to oversee the system as a whole.

My Agents and Roles

As I mentioned in my previous post I ended up with the following set of Agents defined in my Harness. Each one with a corresponding Role definition used to contain the majority of the definition.

Orchestrator — Controls the workflow end to end, decides which stage should run next, invokes subagents, validates handoffs, keeps execution moving in the right order, and bails out if one of the subagents fails.
PO-Spec — Turns the original idea or request into a clear feature specification with scope, intent, and acceptance criteria.
Feature Design — Translates the feature requirements into a design direction, interaction model, and structural approach for the features being built.
Tech Lead — Converts the spec and design into an implementation plan with technical decisions, architecture guidance, and delivery steps.
Build — Actually does the coding work - it carries out the implementation work by creating or updating the code and assets needed to deliver the feature.
QA — Reviews the output against the specification and expected behaviour, identifies gaps or defects, and decides whether the work is ready or needs to return to Build.

The Orchestrator agent is responsible for executing the system as a whole, calling each of the subagents at the correct time, in the correct order and validating the output before moving on to the next one. You can see a Sequence Diagram below, showing the order of execution in the system.

Artifacts, Inputs and Context

I'm going to focus more on this topic in my next post, but you can see from the diagram above that as each of the subagents executes, it is responsible for producing a specific single output, e.g. the PO-Spec Agent is responsible for taking the Task definition I create and generating a Feature Specification defining what it is the system is going to build. That is all though, the agent doesn't then make any code change or anything else, it just produces that document then stops. That document is then used as an input for the Feature Design Agent to be able to create the Design Note. Both of these documents are then used by the Tech Lead Agent to create a detailed Implementation Plan.

This is what I mean when I talk about discrete agents with narrow responsibility. Each of the agents has a single task, and the outputs of those tasks are used by the subsequent agents for them to complete their own tasks. This means that each agent is provided the exact amount of context it requires to get it's job done.

I upgraded the SUGCON sites to Content SDK v2 in under 1hr

Rob Earlam — Thu, 02 Apr 2026 00:00:00 +0000

Just a quick blog post about a win I just had. I blogged last month about the process of upgrading the SUGCON sites from JSS to the Content SDK, and how the sites had been a bit neglected and were running a super old version. I finished the post by promising not to let that happen, and how we were going to ensure we kept up to date with future releases of the Content SDK. Well shortly after v2 of the SDK was released, so I knew I had to get onboard and update the sites to the latest version.

I decided to see how far AI could get me in the process... and it turns out it basically did it all for me first time. All in all the upgrade took me less than an to go from nothing to an open PR ready for my colleagues to review.

I did this by leveraging Copilot CLI and the Adversarial Testing feature that it supports. All I did was to create a new branch on my machine, open the root of the repo using the Copilot CLI and give it the following prompt.

Please upgrade the SUGCON Site located in this repo to Content SDK v2. You can find the upgrade guide here: https://doc.sitecore.com/sai/en/developers/content-sdk/20/upgrade-content-sdk-1-5-1-next-js-apps-to-version-2-0.html.

Do not make any changes to git, I will do that once I'm happy with the work at the end.

Once you can finished, please run an adversarial test of the changes using both Opus 4.6 and GPT 5.4

This went away thinking and making changes for a while and came back with the following response.

 Upgrade Complete — Sugcon2024 → Content SDK v2

  All files changed

  ┌──────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ File │ What changed │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ package.json │ Removed @sitecore-cloudsdk/*; added analytics-core, events, personalize at ^2.0.0; bumped SDK + CLI to ^2.0.0; next → ^16.1.1; @types/node → ^24.10.4 │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/Bootstrap.tsx │ CloudSDK().addEvents().initialize() → initContentSdk({config, plugins}) with analyticsPlugin + eventsPlugin │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/middleware.ts │ Deleted │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/proxy.ts │ Created — Next.js 16 proxy format; *Middleware → *Proxy class names; defineProxy; default export │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/components/content-sdk/CdpPageView.tsx │ pageView import: @sitecore-cloudsdk/events/browser → @sitecore-content-sdk/events │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/byoc/index.tsx │ Events namespace import: @sitecore-cloudsdk/events/browser → @sitecore-content-sdk/events │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ sitecore.config.ts.example │ SITECORE_EDGE_URL → NEXT_PUBLIC_SITECORE_EDGE_PLATFORM_HOSTNAME; PERSONALIZE_MIDDLEWARE_* → PERSONALIZE_PROXY_* │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ .env.remote.example │ Added NEXT_PUBLIC_SITECORE_EDGE_PLATFORM_HOSTNAME, SITECORE_EXPERIENCE_EDGE_HOSTNAME, SITECORE_RENDERINGHOST_NAME; renamed PERSONALIZE_MIDDLEWARE_* → PERSONALIZE_PROXY_* │
  └──────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

  Bugs caught by adversarial review

   - Opus
    4.6 caught: BotTrackingProxy imported from the dev branch template but not yet in published v2.0.1 — removed
   - GPT
    5.4 confirmed the above, and also caught: PERSONALIZE_MIDDLEWARE_* env var names in .env.remote.example were misaligned with the updated PERSONALIZE_PROXY_* names in sitecore.config.ts.example — fixed

Now this was the first time I'd upgraded a site to v2 of the Content SDK, so I followed the upgrade guide myself afterwards to double check the changes it had made to ensure nothing had been missed. Turns out, it nailed it first time!

The thing I really liked about this was the text output at the bottom above, covering the result of the Adversarial Testing I asked it to complete. It used both Opus 4.5 and also GPT 5.4 to validate the changes it had made, and each model found different issues that needed to be fixed. This is a a super powerful trick as it means you can play models off against each other leveraging the benefits of both to get you where you need to go faster, how cool is that!

An Introduction to Harness Engineering

Rob Earlam — Wed, 01 Apr 2026 00:00:00 +0000

So for a decent amount of time now I've been on the Vibe Coding bandwagon, like a large portion of the development community. When it works it really can seem like magic. Too often though it does seem to go off on a tangent, editing unrelated parts of the codebase, or implementing a something in a way that I just wasn't happy with. I felt that it was making me more productive, but I also felt like I spent a good amount of my time chasing the LLM, going back and forth on the same problem and repeatedly prodding it to get to the outcome that I wanted.

It felt like one of the big problems with this approach was repeatability, now rules files can help here to try and enforce some guidelines for how LLM's interact in certain scenarios, but I was getting sick of constantly having to type prompts like "You are a senior developer with 10yrs of experience yadda yadda". I had played around with some Agents, and they were a step in the right direction, but the whole thing was still feeling very manual, in that I had to hand-hold the AI at each step.

Then last month I read a blog post published by Ryan Lopopolo from the OpenAI Tech Team about Harness Engineering and this got me interested. I had heard of people creating teams of Agents, so I decided it was time to jump in. I ended up rebuilding the site you're reading this blog on now using this approach and I wanted to document how I ended up putting it together for anyone else looking to build software this way.

Harness Engineering 101

As I said in my previous blog post the name is pretty apt. The basic concept is that you think of AI like a horse, it's fast and it's powerful, but for the vast amount of people, if you try an ride it bareback then you're going to have a bad time. That's why you need to develop a Harness for the horse, allowing you to make use of the power and speed it provides while being much more in control of where the final destination is.

The harness also means you can repeatedly ride the horse to different destinations, but with the same ease. Now from time to time you might have to tighten the harness when you realise there's something you're not happy with. You might also make some slight adjustments depending on the journey your taking or the destination. Overall though, the harness remains pretty much the same for each trip. The same is true for Harness Engineering as well. In this case the harness is a series of agents , roles , artifacts , and workflow standards that let you guide execution instead of hoping for a good result.

The workflow standards define how everything should function. In my case this covers everything from Next.JS best practices, and testing standards, through to how the agents communicate with each other, and how the current state of the repo is tracked during execution.

The agents are autonomous workers, each with a very specific purpose to execute within the system. I keep the actual definition here very light, instead storing most of the definition in the roles. I have the following agents defined currently though:

Orchestrator — Controls the workflow end to end, decides which stage should run next, invokes subagents, validates handoffs, keeps execution moving in the right order, and bails out if one of the subagents fails.
PO-Spec — Turns the original idea or request into a clear feature specification with scope, intent, and acceptance criteria.
Feature Design — Translates the feature requirements into a design direction, interaction model, and structural approach for the features being built.
Tech Lead — Converts the spec and design into an implementation plan with technical decisions, architecture guidance, and delivery steps.
Build — Actually does the coding work - it carries out the implementation work by creating or updating the code and assets needed to deliver the feature.
QA — Reviews the output against the specification and expected behaviour, identifies gaps or defects, and decides whether the work is ready or needs to return to Build.

The artifacts are what is delivered by each if of the agents above, so for the PO-Spec agent, the artifact is the actual spec that it is created. For the Build agent however the artifact would be the code changes themselves.

The roles are where the vast majority of the agent logic is maintained and there is a 1-1 relationship from agents to roles meaning all of the agents listed above have a corresponding role definition. These each define things like the responsibilities of the agent, what pre-requisites need to exist before it can run, what areas of the codebase it can look at to perform it's actions, whether it can iterate on itself, which it can says its completed, and a detailed walkthrough of the process it should follow. One of the main benefits with keeping the definitions in role files with skinny agent files that reference them is that it helps to keep my harness IDE agnostic - I can have skinny definitions for agents in Claude, Cursor or VS-Code, but the logic is in the shared role file avoiding repetition.

Advantages of Harness Engineering

So you might be reading through the items above and wondering why someone would go to the effort of configuring all of this, when you could just start directly talking with your LLM and creating the code for the feature you require. There's a few really nice things you get by following this approach that I want to cover.

The shift from prompting to orchestration

One of the main ones for me is the change from having to repeatedly craft prompts to defining a lot of this in advance in the harness. This allows the LLM to take the actual work request and break it down into discrete tasks, and hand them off to tightly bound agents with specific roles and outputs. It means that each agent has a narrow purpose, clear responsibilities, and explicit limits. Defining all of this up front reduces role leakage, keeps outputs focused, and makes the system easier to reason about and improve. If you find the Tech Lead Agent isn't creating a proper plan from the specs for the build agent, then it becomes very easy to tweak that agents role to get the result you want.

Artifacts, tasks & the backlog

How I run the delivery process using the harness is that I have a backlog of desired features with a very basic high level description. When I want to implement one I have a conversation with an LLM to generate a much more detailed Task Definition with a unique id, which would describe in more detail the feature I want and any guidance for how I think it should and shouldn't work. This task is then passed through all of the agents to generate their artifacts. The nice thing here is that every artifact generated is created keyed with the task id, and stored in the artifacts folder.

This means that if I want to look back in the future at how the blog feature was implemented, I can refer back to all of the artifacts that were generated by each of the agents to see why it was implemented the way it was. This can also be used as memory for future feature implementations, you can including instructions like "following the same pagination pattern implemented in the blog features" and the LLM will know how to do that and where to look for the information.

Repetition, Repetition, Repetition

I keep mentioning the benefit of repetition with the project for future features, but this concept of repetition is actual broader than that. If I want to start a different project tomorrow on a different technology stack, then my existing harness can still help. Lets say I want to build an IOS application tomorrow, now coding standards will obviously need a large rewrite, but the concepts I described above about agents , roles , artifacts , and workflow standards will be largely transferable. I would move them over to the new project and tweak them to get the output tailored to that project. Coming back to the horse analogy, I'm once more tailoring the harness I'm going to be riding with to better suit the journey I'm on and the destination I want to get to.

Conclusion

As I built out this harness, I released I was actually recreating a structure that I had seen and worked in at many different companies before when working on software projects. You have each role within the company responsible for producing the artifacts that the next team would use. Now the difference is that all of this is created by AI, and in a fraction of the time!

Hopefully this has been a good introduction to some of the key elements to Harness Engineering and how it all comes together. As I continue this series I want to deep dive in to each of the topics I've covered above. I want to talk about how you can craft a tightly defined agent and role, how you can enforce contractual boundaries to minimise agentic scope creep, and ensure the orchestrator can validate completion before continuing the execution pipeline. There's going to be lots to cover, so keep an eye out for future articles on this topic coming soon!

New Site, Who Dis?

Rob Earlam — Mon, 30 Mar 2026 00:00:00 +0000

For anyone who's read my blog posts before, you wont have seen much (if any) content on here about horses, but turns out there's a first for everything. Bear with me though as it looks like I saved my "horse content" for just the right time.

Like the majority of the rest of the development community, I've spent the past couple of years both amazed and a little bit terrified about how AI powered development has taken over the industry. I've always been a developer who's excited about learning new things though, and this was a big one I could sink my teeth into.

Every few years a new technology comes along that I want to learn, and in those cases I have generally rebuilt my personal site that you're on now as a way to put the technology into play. I knew that I wanted to follow the same process for AI based development, but with the rapid pace of advancement in the technology, I was never really sure about when to attack this, until last month when I read a blog post published by Ryan Lopopolo from the OpenAI Tech Team about Harness Engineering and I knew now was the time to bite the bullet.

I'm planning to write much more on this subject as it has been a decent learning curve for me, but it's been some of the most fun I've have writting code (or not writing it as the case maybe), in a very long time!

The basic concept is that you think of AI like a horse, it's fast and it's powerful, but for the vast amount of people, if you try an ride it bareback then you're going to have a bad time. Thats why you need to develop a Harness for the horse, allowing you to make use of the power and speed it provides while being much more in control of where the final destination is.

In reality the what you end up building is a team of agents, each with a tightly bound function. They can't stray outside of their designated function, and there is an orchestrator agent who is repsonsible for invoking each of them. The orchestrator is repsonsible for invoking them in the correct order, and also ensuring that they complete succesfully before invoking the next one.

All of this comes together through a series of markdown files that are used to define how this system comes together. This covers everthing from how the agents themselves function, the different actions they can perform, the files in the repo they have access to in order to perform their job, and how communication and state management is handled throughout all of this.

I'm planning to write all of this up in the coming weeks, as it's been an amazingly enjoyable learning journey for me to pull it all together and I'm hoping this series will help others looking to build software in the same way. It took a few attempts to get the Harness into a state where I was happy with it, and it is now churning out features for this Next.js site with ease. I do think that when I complete this the next time the process will be much faster with everything I've learned.

So my aim is write content covering how I built my setup, what actions worked well, and crucially what didn't. We're going to go over the different agents and their roles, artifact generation, human-in-the-loop review process, execution boundaries, the process I went through pulling all of this together and crucially where I want to take it next.

So I hope you come back for the next post when we start to dig into this exciting topic for real!

Sitecore Search - Debugging Crawling Errors

Rob Earlam — Tue, 27 Feb 2024 21:45:06 +0000

The latest video in my series on #Sitecore #Search was published this week, discussing how you can debug crawling errors.

Sitecore Search SDK - Breaking Change

Rob Earlam — Fri, 02 Feb 2024 00:00:00 +0000

Updating packages in Sitecore’s Developer Portal

Recently my colleague Mark van Aalst was updating the package versions for the Sitecore Developer Portal. This also included updating the version of the Sitecore Search React SDK from v2.0.0 to v2.2.2, among other packages that were updated at the same time. Once this had been deployed, we got reports that faceting was no longer working on the search results page. We quickly rolled back and started to look into what had happened.

Breaking changes in the Faceting UI component

I reached out to the team that owns the SDK and apparently there was a breaking change introduced in a previous version and we needed to update how we were leveraging the UI components being used to render the Search Results page.

Previously we were wrapping out Facets in the AccordionFacets.Root component, like so

<AccordionFacets.Root defaultFacetTypesExpandedList={facets.map((x) => x.name)} onFacetValueClick={onFacetClick}>
    ….
</AccordionFacets.Root>

However, this approach is no longer valid, we should now be using SearchResultsAccordionFacets component instead. As follows:

<SearchResultsAccordionFacets defaultFacetTypesExpandedList={facets.map((x) => x.name)} onFacetValueClick={onFacetClick}>
    …
</SearchResultsAccordionFacets>

This was a fairly straightforward change, which meant we could bump the Developer Portal up to be running on the latest version of the Search SDK 🎉.

To see the complete set of changes we had to make for the Developer Portal, you can view the PR that was opened for this fix here.

SUGCON India 2023 goes off with a bang 💥

Rob Earlam — Thu, 31 Aug 2023 00:00:00 +0000

Last week I had the pleasure of attending SUGCON India 2023 in Delhi, India. This was the first time we’d run a SUGCON in India since 2019 and it was great to be back once more, interacting with the local Sitecore community. There was a lot of effort in involved in pulling the event off, and I do want to start by thanking all of the organisers for the time they invested in making this event the success that it was.

I was privileged to be chosen to deliver one of the keynote sessions this year, where I talked about a recent project I worked on where we integrated Sitecore Search into our Developer Portal. This was very well received and led to many questions afterwards, and many further conversations throughout the event. It was great to see the amount of interest there is in Sitecore Search from the attendees!

The quality of the sessions that were delivered at the event was of a very high standard. They ranged from high-level sessions covering Composable DXP’s in general and the power they can bring to customers, all the way down to technical deep dives on specific features and use-cases. Some of my highlights were:

The session by Pieter Brinkman titled Digital Experience is a team sport, where he talked about the different roles that play a part in successful Composable DXP implementation – it’s not just all about the developer!
George Smith & Sheetal Jain delivered an awesome session called Beyond Buzzwords: Composable Tech and Generative AI for Real. Here they showed how OrderCloud, Sitecore Content Hub & Dall-E can be combined to allow customers to generate art on the fly and have it printed on a t-shirt for sale in an online store, cool stuff!
Exploring the Experience Edge GraphQL Schema was a really interesting session delivered by Alistair Deneys. Here he gave a true deep-dive into the schema implemented for Edge and how you can leverage it in some really interesting ways. There’s also a cool surprise at the end of the session that I won’t give away here, but be sure to check it out if you get the chance.
Women in contemplating Darwin's Theory - To Break the Conventional Wheel was a great session by Varalakshmi M D & Raveena Mathur where they talked about the important role women have in ensuring successful outcomes for customers.
Andy Cohen delivered a great session titled Build for XM Cloud according to the Architect of XM Cloud. Here he talked about the best practices for building with XM Cloud... he might have ruffled some feathers with some of the recommendations, but it was great to see it all layed out!

The closing keynote was also a highly enjoyable part of the event. Each of the representatives from Sitecore (Pieter, Nicole, Andy, Alistair and myself) held a panel discussion and open Q&A session. This gave the attendees the chance to ask any lingering questions that they hadn’t had the chance to ask already. The topics were very wide from specific feature questions on the products through to general questions about the best way that Sitecore can support the local Indian community… and of course the age-old question of Sandboxes and how people can get access to Sitecore’s newer SaaS products – we hear you and are working on a solution!

As much as I enjoyed everything I mentioned above, as with every event I seem to attend nowadays, the true highlight was getting to interact with the local community we have on the ground. It’s amazing to see the passion they have for Sitecore and our products. It was great to see so many familiar faces from the events back in 2018 & 2019, but it was also great to see so many new faces as well, a good balance that shows how healthy the Sitecore community in India is.

I just want to finish off by saying that of course trips like this aren’t always about work though, we always try to make sure that we get out and see some of the places we visit and India is no different. On this trip, we made the journey down to see the Taj Mahal after the event had finished, and it truly is an amazing site to see. Really all of the descriptions of it you hear just cannot do it justice, and it was great to experience it with the awesome team we had on-site.

So, whether it’s the always friendly people, the amazing effort from the organisers & speakers, the food, the noise or the general sights you see around the city – India is one of my favourite places to visit and certainly one of my favourite SUGCON events.

Here’s looking forward to doing it all again in 2024!

Migrating the XM Cloud Introduction Repo to a new Nuget feed.

Rob Earlam — Thu, 27 Jul 2023 00:00:00 +0000

Today I had to go through the process of migrating the XM Cloud Introduction repository to a new Nuget feed. This was required as the old feed was no longer available, and we needed to move to a new one on short notice. This post will cover the steps I needed to complete to migrate the solution to the new feed.

Machine Tidy Up

Now for anyone who hasn't run a Sitecore site before you can most likely skip this step. However, for anyone who has you should tidy up any references to the old feed that have been configured on your machine. In my case, this was the PowerShell Repository used to restore the SitecoreDockerTools module used by this, and other Sitecore repositories.

Remove PS Repository

The first thing to check is whether you have the old Sitecore Feed registered as a PowerShell Repository, you can do this by running the following command Get-PSRepository and checking where there is a record named SitecoreGallery. If you have one registered, then you can remove it by running Unregister-PSRepository -Name SitecoreGallery.

Remove PS Modules

Next, I tidied up any previous versions of the SitecoreDockerTools module installed from the old feed. You can test whether you have any versions of that module installed by running the Get-Module -ListAvailable and look for any named SitecoreDockerTools. If you have those installed, you can quickly remove all installed versions of that package by running Uninstall-Module -Name SitecoreDockerTools -AllVersions.

Update the Repository

So now we’ve tidied our developer machine, we can update the repository to reference the new feed. This involved making two changes, an update to the init.ps1 script used to initialise the repository, and then an update to the nuget.config to define the new location for the NuGet packages in use by the solution.

Updating the init.ps1 script

The first thing the script does is install the SitecoreDockerTools PowerShell module, so we need to update the location that it is installed from, in our case, we needed to use v2 of the Nuget feed to get it to work, so we needed to change it to https://nuget.sitecore.com/resources/v2/. After making this change the init script ran successfully and installed the module from the new feed.

Updating the nuget.config

The final change we need to make is to update the nuget.config file in the root of the repository that controls where custom NuGet packages are installed from. We had a couple of older references in there that we removed and condensed down to the single new feed reference. This resulted in us having the following line in our config:

<add key="Sitecore" value="https://nuget.sitecore.com/resources/v3/index.json" />

Conclusion

Updating the XM Cloud Introduction repository to the new feed location was straight forward in the end. I would recommend any developers who have been running existing Sitecore solutions before follow the tidy-up steps at the start to ensure they remove any legacy references they have to the old feed. You can see the changes we had to make in the following commit. There may be other changes that you will need to complete to bring your codebase in line, but hopefully, this will help to get you started with what tasks are required for you.

Sitecore Search - Filtering vs Faceting

Rob Earlam — Tue, 25 Jul 2023 23:28:45 +0000

The latest video in my series on #Sitecore #Search was published this week, discussing Filtering vs Faceting.

Sitecore Search – Configuring your Domain

Rob Earlam — Mon, 10 Jul 2023 00:51:26 +0000

The second video in my series on #Sitecore #Search was published last week, showing how you can do the initial domain configuration

Introducing the Sitecore Discover JavaScript SDK - Part 2 Components

Rob Earlam — Thu, 11 May 2023 00:00:00 +0000

Introducing the Sitecore Discover JavaScript SDK - Part 2 Components

This post is the second in a two-part blog series, introducing a brand-new JavaScript SDK for Sitecore Discover. You can read the first part here. In this second part, we'll dive into the component model provided by the SDK and show the different ways you can leverage Discover data in your React application. This blog is the accompaniment for the YouTube video highlighting the same features. You can watch the video below:

Adding Components to a Page

There are three different ways to add a component in to your React application:

Static components : Created in the CEC (Customer Engagement Console) and dropped into the page by a developer. This is developer-centric and means the developer specifies that a specific widget will always appear in a specific area in the DOM. The merchandiser can configure the widget afterwards, but they cannot change it out for a different one.
Dynamic widgets : The developer leverages a widget tag, similar to a placeholder in XM Cloud, allowing the merchandiser to dynamically insert widgets configured and created in the CEC. This means the developer can specify an area in the DOM, and the merchandiser can add and remove widgets from that area without the developer's involvement.
Global components : These are components that appear on every page, such as the search bar in a header. They remain the same across different pages and routes of your site.

Configuring Widgets in the CEC

Before adding any components to our application, they’ll need to be configured within the CEC. There are two different types of widgets when it comes to working with the SDK:

HTML widgets : The layout is owned by the CEC, and the SDK can render them without any additional configuration.
Catalog widgets : The layout and functionality are owned by your application. We need to tell the SDK which components in our app to use to render them by calling the setWidgetType function, you can find details on that function in our documentation site.

An rfk_id is also assigned to each of your widgets allowing you to refer to them specifically.

Where to find a Widget examples?

So, you might be asking where you find examples of the widgets you can create? Well you have a few supporting elements that can help you here.

We have the documentation site which will show you walkthroughs of how to complete common activities with the SDK.
We also have a StoryBook instance which will give you examples of the different components included in the SDK’s UI package.

Adding Static & Dynamic Widgets to Our Site

In the video, we begin by adding some widgets to our site. In the index.tsx page of our Next.js application, we will use the useEffect hook to tie our application's route to the specific page definition in the CEC. We will then use the usePageWidgets function to get the widget data for rendering. We can then begin to replace parts of our site with dynamic or static components, as shown in the examples below:

Dynamic Components : We can render a div with a loading message, and when the loading is finished, we map through the widgets returned by the usePageWidgets to render the appropriate widget component.

<div>
  {isLoading && <div>Loading...</div>}
  {!isLoading && widgets.map((w) => <Widget rfkId={w} key={w} />)}
</div>

Static Components : We can use a different component tied to a specific widget in the CEC using the widgetID to always render that widget at that location in the DOM.

<RecommendationGallery rfkId="rfkid_92" productsToDisplay={6} />

Implementing Global Widgets

In the video we replace the existing search bar in our Next.js Commerce site with a preview search from Discover, using the code samples provided in StoryBook. The samples there will give you a styled, working interactive React component ready to be inserted directly into your application.

Conclusion

Discover can be used as much or as little as desired in your implementations. You can use Discover for every piece of catalog data in your channels or just for individual pieces of catalog data sprinkled throughout your channels. The other fact I liked about the platform is that it is just as easy to implement Discover in an existing storefront as it is a new greenfield project.

Finally, when using the JavaScript SDK, be sure to take advantage of all the supporting features provided, such as Storybook, and the documentation site to help you build your app as quickly as possible.