Forem: Addy Osmani

How to write a good spec for AI agents

Addy Osmani — Wed, 18 Mar 2026 00:00:00 +0000

TL;DR: Aim for a clear spec covering just enough nuance (this may include structure, style, testing, boundaries) to guide the AI without overwhelming it. Break large tasks into smaller ones vs. keeping everything in one large prompt. Plan first in read-only mode, then execute and iterate continuously.

"I've heard a lot about writing good specs for AI agents, but haven't found a solid framework yet. I could write a spec that rivals an RFC, but at some point the context is too large and the model breaks down."

Many developers share this frustration. Simply throwing a massive spec at an AI agent doesn't work - context window limits and the model's "attention budget" get in the way. The key is to write smart specs: documents that guide the agent clearly, stay within practical context sizes, and evolve with the project. This guide distills best practices from my use of coding agents including Claude Code and Gemini CLI into a framework for spec-writing that keeps your AI agents focused and productive.

We'll cover five principles for great AI agent specs, each starting with a bolded takeaway.

1. Start with a high-level vision and let the AI draft the details

Kick off your project with a concise high-level spec, then have the AI expand it into a detailed plan.

Instead of over-engineering upfront, begin with a clear goal statement and a few core requirements. Treat this as a "product brief" and let the agent generate a more elaborate spec from it. This leverages the AI's strength in elaboration while you maintain control of the direction. This works well unless you already feel you have very specific technical requirements that must be met from the start.

Why this works: LLM-based agents excel at fleshing out details when given a solid high-level directive, but they need a clear mission to avoid drifting off course. By providing a short outline or objective description and asking the AI to produce a full specification (e.g. a spec.md), you create a persistent reference for the agent. Planning in advance matters even more with an agent - you can iterate on the plan first, then hand it off to the agent to write the code. The spec becomes the first artifact you and the AI build together.

Practical approach: Start a new coding session by prompting, "You are an AI software engineer. Draft a detailed specification for [project X] covering objectives, features, constraints, and a step-by-step plan." Keep your initial prompt high-level - e.g. "Build a web app where users can track tasks (to-do list), with user accounts, a database, and a simple UI". The agent might respond with a structured draft spec: an overview, feature list, tech stack suggestions, data model, and so on. This spec then becomes the "source of truth" that both you and the agent can refer back to. GitHub's AI team promotes spec-driven development where "specs become the shared source of truth… living, executable artifacts that evolve with the project". Before writing any code, review and refine the AI's spec. Make sure it aligns with your vision and correct any hallucinations or off-target details.

Use Plan Mode to enforce planning-first: Tools like Claude Code offer a Plan Mode that restricts the agent to read-only operations - it can analyze your codebase and create detailed plans but won't write any code until you're ready. This is ideal for the planning phase: start in Plan Mode (Shift+Tab in Claude Code), describe what you want to build, and let the agent draft a spec while exploring your existing code. Ask it to clarify ambiguities by questioning you about the plan. Have it review the plan for architecture, best practices, security risks, and testing strategy. The goal is to refine the plan until there's no room for misinterpretation. Only then do you exit Plan Mode and let the agent execute. This workflow prevents the common trap of jumping straight into code generation before the spec is solid.

Use the spec as context: Once approved, save this spec (e.g. as SPEC.md) and feed relevant sections into the agent as needed. Many developers using a strong model do exactly this - the spec file persists between sessions, anchoring the AI whenever work resumes on the project. This mitigates the forgetfulness that can happen when the conversation history gets too long or when you have to restart an agent. It's akin to how one would use a Product Requirements Document (PRD) in a team: a reference that everyone (human or AI) can consult to stay on track. Experienced folks often "write good documentation first and the model may be able to build the matching implementation from that input alone" as one engineer observed. The spec is that documentation.

Keep it goal-oriented: A high-level spec for an AI agent should focus on what and why, more than the nitty-gritty how (at least initially). Think of it like the user story and acceptance criteria: Who is the user? What do they need? What does success look like? (e.g. "User can add, edit, complete tasks; data is saved persistently; the app is responsive and secure"). This keeps the AI's detailed spec grounded in user needs and outcome, not just technical to-dos. As the GitHub Spec Kit docs put it, provide a high-level description of what you're building and why, and let the coding agent generate a detailed specification focusing on user experience and success criteria. Starting with this big-picture vision prevents the agent from losing sight of the forest for the trees when it later gets into coding.

2. Structure the spec like a professional PRD (or SRS)

Treat your AI spec as a structured document (PRD) with clear sections, not a loose pile of notes.

Many developers treat specs for agents much like traditional Product Requirement Documents (PRDs) or System Design docs - comprehensive, well-organized, and easy for a "literal-minded" AI to parse. This formal approach gives the agent a blueprint to follow and reduces ambiguity.

The six core areas: GitHub's analysis of over 2,500 agent configuration files revealed a clear pattern: the most effective specs cover six areas. Use this as a checklist for completeness:

1. Commands: Put executable commands early - not just tool names, but full commands with flags: npm test, pytest -v, npm run build. The agent will reference these constantly.

2. Testing: How to run tests, what framework you use, where test files live, and what coverage expectations exist.

3. Project structure: Where source code lives, where tests go, where docs belong. Be explicit: "src/ for application code, tests/ for unit tests, docs/ for documentation."

4. Code style: One real code snippet showing your style beats three paragraphs describing it. Include naming conventions, formatting rules, and examples of good output.

5. Git workflow: Branch naming, commit message format, PR requirements. The agent can follow these if you spell them out.

6. Boundaries: What the agent should never touch - secrets, vendor directories, production configs, specific folders. "Never commit secrets" was the single most common helpful constraint in the GitHub study.

Be specific about your stack: Say "React 18 with TypeScript, Vite, and Tailwind CSS" not "React project." Include versions and key dependencies. Vague specs produce vague code.

Use a consistent format: Clarity is king. Many devs use Markdown headings or even XML-like tags in the spec to delineate sections, because AI models handle well-structured text better than free-form prose. For example, you might structure the spec as:

# Project Spec: My team's tasks app

## Objective
- Build a web app for small teams to manage tasks...

## Tech Stack
- React 18+, TypeScript, Vite, Tailwind CSS
- Node.js/Express backend, PostgreSQL, Prisma ORM

## Commands
- Build: `npm run build` (compiles TypeScript, outputs to dist/)
- Test: `npm test` (runs Jest, must pass before commits)
- Lint: `npm run lint --fix` (auto-fixes ESLint errors)

## Project Structure
- `src/` – Application source code
- `tests/` – Unit and integration tests
- `docs/` – Documentation

## Boundaries
- ✅ Always: Run tests before commits, follow naming conventions
- ⚠️ Ask first: Database schema changes, adding dependencies
- 🚫 Never: Commit secrets, edit node_modules/, modify CI config

This level of organization not only helps you think clearly, it helps the AI find information. Anthropic engineers recommend organizing prompts into distinct sections (like <background>, <instructions>, <tools>, <output_format> etc.) for exactly this reason - it gives the model strong cues about which info is which. And remember, "minimal does not necessarily mean short" - don't shy away from detail in the spec if it matters, but keep it focused.

Integrate specs into your toolchain: Treat specs as "executable artifacts" tied to version control and CI/CD. The GitHub Spec Kit uses a four-phase, gated workflow that makes your specification the center of your engineering process. Instead of writing a spec and setting it aside, the spec drives the implementation, checklists, and task breakdowns. Your primary role is to steer; the coding agent does the bulk of the writing. Each phase has a specific job, and you don't move to the next one until the current task is fully validated:

1. Specify: You provide a high-level description of what you're building and why, and the coding agent generates a detailed specification. This isn't about technical stacks or app design - it's about user journeys, experiences, and what success looks like. Who will use this? What problem does it solve? How will they interact with it? Think of it as mapping the user experience you want to create, and letting the coding agent flesh out the details. This becomes a living artifact that evolves as you learn more.

2. Plan: Now you get technical. You provide your desired stack, architecture, and constraints, and the coding agent generates a comprehensive technical plan. If your company standardizes on certain technologies, this is where you say so. If you're integrating with legacy systems or have compliance requirements, all of that goes here. You can ask for multiple plan variations to compare approaches. If you make internal docs available, the agent can integrate your architectural patterns directly into the plan.

3. Tasks: The coding agent takes the spec and plan and breaks them into actual work - small, reviewable chunks that each solve a specific piece of the puzzle. Each task should be something you can implement and test in isolation, almost like test-driven development for your AI agent. Instead of "build authentication," you get concrete tasks like "create a user registration endpoint that validates email format."

4. Implement: Your coding agent tackles tasks one by one (or in parallel). Instead of reviewing thousand-line code dumps, you review focused changes that solve specific problems. The agent knows what to build (specification), how to build it (plan), and what to work on (task). Crucially, your role is to verify at each phase: Does the spec capture what you want? Does the plan account for constraints? Are there edge cases the AI missed? The process builds in checkpoints for you to critique, spot gaps, and course-correct before moving forward.

This gated workflow prevents what Willison calls "house of cards code" - fragile AI outputs that collapse under scrutiny. Anthropic's Skills system offers a similar pattern, letting you define reusable Markdown-based behaviors that agents invoke. By embedding your spec in these workflows, you ensure the agent can't proceed until the spec is validated, and changes propagate automatically to task breakdowns and tests.

Consider agents.md for specialized personas: For tools like GitHub Copilot, you can create agents.md files that define specialized agent personas - a @docs-agent for technical writing, a @test-agent for QA, a @security-agent for code review. Each file acts as a focused spec for that persona's behavior, commands, and boundaries. This is particularly useful when you want different agents for different tasks rather than one general-purpose assistant.

Design for Agent Experience (AX): Just as we design APIs for developer experience (DX), consider designing specs for "Agent Experience." This means clean, parseable formats: OpenAPI schemas for any APIs the agent will consume, llms.txt files that summarize documentation for LLM consumption, and explicit type definitions. The Agentic AI Foundation (AAIF) is standardizing protocols like MCP (Model Context Protocol) for tool integration - specs that follow these patterns are easier for agents to consume and act on reliably.

PRD vs SRS mindset: It helps to borrow from established documentation practices. For AI agent specs, you'll often blend these into one document (as illustrated above), but covering both angles serves you well. Writing it like a PRD ensures you include user-centric context ("the why behind each feature") so the AI doesn't optimize for the wrong thing. Expanding it like an SRS ensures you nail down the specifics the AI will need to actually generate correct code (like what database or API to use). Developers have found that this extra upfront effort pays off by drastically reducing miscommunications with the agent later.

Make the spec a "living document": Don't write it and forget it. Update the spec as you and the agent make decisions or discover new info. If the AI had to change the data model or you decided to cut a feature, reflect that in the spec so it remains the ground truth. Think of it as version-controlled documentation. In spec-driven workflows, the spec drives implementation, tests, and task breakdowns, and you don't move to coding until the spec is validated. This habit keeps the project coherent, especially if you or the agent step away and come back later. Remember, the spec isn't just for the AI - it helps you as the developer maintain oversight and ensure the AI's work meets the real requirements.

3. Break tasks into modular prompts and context, not one big prompt

Divide and conquer: give the AI one focused task at a time rather than a monolithic prompt with everything at once.

Experienced AI engineers have learned that trying to stuff the entire project (all requirements, all code, all instructions) into a single prompt or agent message is a recipe for confusion. Not only do you risk hitting token limits, you also risk the model losing focus due to the "curse of instructions" - too many directives causing it to follow none of them well. The solution is to design your spec and workflow in a modular way, tackling one piece at a time and pulling in only the context needed for that piece.

The curse of too much context/instructions: Research has confirmed what many devs anecdotally saw: as you pile on more instructions or data into the prompt, the model's performance in adhering to each one drops significantly. One study dubbed this the "curse of instructions", showing that even GPT-4 and Claude struggle when asked to satisfy many requirements simultaneously. In practical terms, if you present 10 bullet points of detailed rules, the AI might obey the first few and start overlooking others. The better strategy is iterative focus. Guidelines from industry suggest decomposing complex requirements into sequential, simple instructions as a best practice. Focus the AI on one sub-problem at a time, get that done, then move on. This keeps the quality high and errors manageable.

Divide the spec into phases or components: If your spec document is very long or covers a lot of ground, consider splitting it into parts (either physically separate files or clearly separate sections). For example, you might have a section for "Backend API Spec" and another for "Frontend UI Spec." You don't need to always feed the frontend spec to the AI when it's working on the backend, and vice versa. Many devs using multi-agent setups even create separate agents or sub-processes for each part - e.g. one agent works on database/schema, another on API logic, another on frontend - each with the relevant slice of the spec. Even if you use a single agent, you can emulate this by copying only the relevant spec section into the prompt for that task. Avoid context overload: Don't mix authentication tasks with database schema changes in one go, as the DigitalOcean AI guide warns. Keep each prompt tightly scoped to the current goal.

Extended TOC / Summaries for large specs: One clever technique is to have the agent build an extended Table of Contents with summaries for the spec. This is essentially a "spec summary" that condenses each section into a few key points or keywords, and references where details can be found. For example, if your full spec has a section on "Security Requirements" spanning 500 words, you might have the agent summarize it to: "Security: use HTTPS, protect API keys, implement input validation (see full spec §4.2)". By creating a hierarchical summary in the planning phase, you get a bird's-eye view that can stay in the prompt, while the fine details remain offloaded unless needed. This extended TOC acts as an index: the agent can consult it and say "aha, there's a security section I should look at", and you can then provide that section on demand. It's similar to how a human developer skims an outline and then flips to the relevant page of a spec document when working on a specific part.

To implement this, you can prompt the agent after writing the spec: "Summarize the spec above into a very concise outline with each section's key points and a reference tag." The result might be a list of sections with one or two sentence summaries. That summary can be kept in the system or assistant message to guide the agent's focus without eating up too many tokens. This hierarchical summarization approach is known to help LLMs maintain long-term context by focusing on the high-level structure. The agent carries a "mental map" of the spec.

Utilize sub-agents or "skills" for different spec parts: Another advanced approach is using multiple specialized agents (what Anthropic calls subagents or what you might call "skills"). Each subagent is configured for a specific area of expertise and given the portion of the spec relevant to that area. For instance, you might have a Database Designer subagent that only knows about the data model section of the spec, and an API Coder subagent that knows the API endpoints spec. The main agent (or an orchestrator) can route tasks to the appropriate subagent automatically. The benefit is each agent has a smaller context window to deal with and a more focused role, which can boost accuracy and allow parallel work on independent tasks. Anthropic's Claude Code supports this by letting you define subagents with their own system prompts and tools. "Each subagent has a specific purpose and expertise area, uses its own context window separate from the main conversation, and has a custom system prompt guiding its behavior," as their docs describe. When a task comes up that matches a subagent's domain, Claude can delegate that task to it, with the subagent returning results independently.

Parallel agents for throughput: Running multiple agents simultaneously is emerging as "the next big thing" for developer productivity. Rather than waiting for one agent to finish before starting another task, you can spin up parallel agents for non-overlapping work. Willison describes this as "embracing parallel coding agents" and notes it's "surprisingly effective, if mentally exhausting". The key is scoping tasks so agents don't step on each other - one agent codes a feature while another writes tests, or separate components get built concurrently. Orchestration frameworks like LangGraph or OpenAI Swarm can help coordinate these agents, and shared memory via vector databases (like Chroma) lets them access common context without redundant prompting.

Single vs. multi-agent: when to use each

table {
width: 100%;
border-collapse: collapse;
margin: 20px 0;
border: 1px solid #ddd;
}

tr, td, th {
border: 1px solid #ddd;
padding: 8px;
}

Aspect	Single Agent	Parallel/Multi-Agent
Strengths	Simpler setup; lower overhead; easier to debug and follow	Higher throughput; handles complex interdependencies; specialists per domain
Challenges	Context overload on big projects; slower iteration; single point of failure	Coordination overhead; potential conflicts; needs shared memory (e.g., vector DBs)
Best For	Isolated modules; small-to-medium projects; early prototyping	Large codebases; one codes + one tests + one reviews; independent features
Tips	Use spec summaries; refresh context per task; start fresh sessions often	Limit to 2-3 agents initially; use MCP for tool sharing; define clear boundaries

In practice, using subagents or skill-specific prompts might look like: you maintain multiple spec files (or prompt templates) - e.g. SPEC_backend.md, SPEC_frontend.md - and you tell the AI, "For backend tasks, refer to SPEC_backend; for frontend tasks refer to SPEC_frontend." Or in a tool like Cursor/Claude, you actually spin up a subagent for each. This is certainly more complex to set up than a single-agent loop, but it mimics what human developers do - we mentally compartmentalize a large spec into relevant chunks (you don't keep the whole 50-page spec in your head at once; you recall the part you need for the task at hand, and have a general sense of the overall architecture). The challenge, as noted, is managing interdependencies: the subagents must still coordinate (the frontend needs to know the API contract from the backend spec, etc.). A central overview (or an "architect" agent) can help by referencing the sub-specs and ensuring consistency.

Focus each prompt on one task/section: Even without fancy multi-agent setups, you can manually enforce modularity. For example, after the spec is written, your next move might be: "Step 1: Implement the database schema." You feed the agent the Database section of the spec only, plus any global constraints from the spec (like tech stack). The agent works on that. Then for Step 2, "Now implement the authentication feature", you provide the Auth section of the spec and maybe the relevant parts of the schema if needed. By refreshing the context for each major task, you ensure the model isn't carrying a lot of stale or irrelevant information that could distract it. As one guide suggests: "Start fresh: begin new sessions to clear context when switching between major features". You can always remind the agent of critical global rules (from the spec's Constraints section) each time, but don't shove the entire spec in if it's not all needed.

Use in-line directives and code TODOs: Another modularity trick is to use your code or spec as an active part of the conversation. For instance, scaffold your code with // TODO comments that describe what needs to be done, and have the agent fill them one by one. Each TODO essentially acts as a mini-spec for a small task. This keeps the AI laser-focused ("implement this specific function according to this spec snippet") and you can iterate in a tight loop. It's similar to giving the AI a checklist item to complete rather than the whole checklist at once.

The bottom line: small, focused context beats one giant prompt. This improves quality and keeps the AI from getting "overwhelmed" by too much at once. As one set of best practices sums up, provide "One Task Focus" and "Relevant info only" to the model, and avoid dumping everything everywhere. By structuring the work into modules - and using strategies like spec summaries or sub-spec agents - you'll navigate around context size limits and the AI's short-term memory cap. Remember, a well-fed AI is like a well-fed function: give it only the inputs it needs for the job at hand.

4. Build in self-checks, constraints, and human expertise

Make your spec not just a to-do list for the agent, but also a guide for quality control - and don't be afraid to inject your own expertise.

A good spec for an AI agent anticipates where the AI might go wrong and sets up guardrails. It also takes advantage of what you know (domain knowledge, edge cases, "gotchas") so the AI doesn't operate in a vacuum. Think of the spec as both coach and referee for the AI: it should encourage the right approach and call out fouls.

Use three-tier boundaries: The GitHub analysis of 2,500+ agent files found that the most effective specs use a three-tier boundary system rather than a simple list of don'ts. This gives the agent clearer guidance on when to proceed, when to pause, and when to stop:

✅ Always do: Actions the agent should take without asking. "Always run tests before commits." "Always follow the naming conventions in the style guide." "Always log errors to the monitoring service."

⚠️ Ask first: Actions that require human approval. "Ask before modifying database schemas." "Ask before adding new dependencies." "Ask before changing CI/CD configuration." This tier catches high-impact changes that might be fine but warrant a human check.

🚫 Never do: Hard stops. "Never commit secrets or API keys." "Never edit node_modules/ or vendor/." "Never remove a failing test without explicit approval." "Never commit secrets" was the single most common helpful constraint in the study.

This three-tier approach is more nuanced than a flat list of rules. It acknowledges that some actions are always safe, some need oversight, and some are categorically off-limits. The agent can proceed confidently on "Always" items, flag "Ask first" items for review, and hard-stop on "Never" items.

Encourage self-verification: One powerful pattern is to have the agent verify its work against the spec automatically. If your tooling allows, you can integrate checks like unit tests or linting that the AI can run after generating code. But even at the spec/prompt level, you can instruct the AI to double-check: e.g. "After implementing, compare the result with the spec and confirm all requirements are met. List any spec items that are not addressed." This pushes the LLM to reflect on its output relative to the spec, catching omissions. It's a form of self-audit built into the process.

For instance, you might append to a prompt: "(After writing the function, review the above requirements list and ensure each is satisfied, marking any missing ones)." The model will then (ideally) output the code followed by a short checklist indicating if it met each requirement. This reduces the chance it forgets something before you even run tests. It's not foolproof, but it helps.

LLM-as-a-Judge for subjective checks: For criteria that are hard to test automatically - code style, readability, adherence to architectural patterns - consider using "LLM-as-a-Judge." This means having a second agent (or a separate prompt) review the first agent's output against your spec's quality guidelines. Anthropic and others have found this effective for subjective evaluation. You might prompt: "Review this code for adherence to our style guide. Flag any violations." The judge agent returns feedback that either gets incorporated or triggers a revision. This adds a layer of semantic evaluation beyond syntax checks.

Conformance testing: Willison advocates building conformance suites - language-independent tests (often YAML-based) that any implementation must pass. These act as a contract: if you're building an API, the conformance suite specifies expected inputs/outputs, and the agent's code must satisfy all cases. This is more rigorous than ad-hoc unit tests because it's derived directly from the spec and can be reused across implementations. Include conformance criteria in your spec's Success section (e.g., "Must pass all cases in conformance/api-tests.yaml").

Leverage testing in the spec: If possible, incorporate a test plan or even actual tests in your spec and prompt flow. In traditional development, we use TDD or write test cases to clarify requirements - you can do the same with AI. For example, in the spec's Success Criteria, you might say "These sample inputs should produce these outputs…" or "the following unit tests should pass." The agent can be prompted to run through those cases in its head or actually execute them if it has that capability. Simon Willison noted that having a robust test suite is like giving the agents superpowers - they can validate and iterate quickly when tests fail. In an AI coding context, writing a bit of pseudocode for tests or expected outcomes in the spec can guide the agent's implementation. Additionally, you can use a dedicated "test agent" in a subagent setup that takes the spec's criteria and continuously verifies the "code agent's" output.

Bring your domain knowledge: Your spec should reflect insights that only an experienced developer or someone with context would know. For example, if you're building an e-commerce agent and you know that "products" and "categories" have a many-to-many relationship, state that clearly (don't assume the AI will infer it - it might not). If a certain library is notoriously tricky, mention pitfalls to avoid. Essentially, pour your mentorship into the spec. The spec can contain advice like "If using library X, watch out for memory leak issue in version Y (apply workaround Z)." This level of detail is what turns an average AI output into a truly robust solution, because you've steered the AI away from common traps.

Also, if you have preferences or style guidelines (say, "use functional components over class components in React"), encode that in the spec. The AI will then emulate your style. Many engineers even include small examples in the spec, e.g., "All API responses should be JSON. E.g. {"error": "message"} for errors." By giving a quick example, you anchor the AI to the exact format you want.

Minimalism for simple tasks: While we advocate thorough specs, part of expertise is knowing when to keep it simple. For relatively simple, isolated tasks, an overbearing spec can actually confuse more than help. If you're asking the agent to do something straightforward (like "center a div on the page"), you might just say, "Make sure to keep the solution concise and do not add extraneous markup or styles." No need for a full PRD there. Conversely, for complex tasks (like "implement an OAuth flow with token refresh and error handling"), that's when you break out the detailed spec. A good rule of thumb: adjust spec detail to task complexity. Don't under-spec a hard problem (the agent will flail or go off-track), but don't over-spec a trivial one (the agent might get tangled or use up context on unnecessary instructions).

Maintain the AI's "persona" if needed: Sometimes, part of your spec is defining how the agent should behave or respond, especially if the agent interacts with users. For example, if building a customer support agent, your spec might include guidelines like "Use a friendly and professional tone," "If you don't know the answer, ask for clarification or offer to follow up, rather than guessing." These kind of rules (often included in system prompts) help keep the AI's outputs aligned with expectations. They are essentially spec items for AI behavior. Keep them consistent and remind the model of them if needed in long sessions (LLMs can "drift" in style over time if not kept on a leash).

You remain the exec in the loop: The spec empowers the agent, but you remain the ultimate quality filter. If the agent produces something that technically meets the spec but doesn't feel right, trust your judgement. Either refine the spec or directly adjust the output. The great thing about AI agents is they don't get offended - if they deliver a design that's off, you can say, "Actually, that's not what I intended, let's clarify the spec and redo it." The spec is a living artifact in collaboration with the AI, not a one-time contract you can't change.

Simon Willison humorously likened working with AI agents to "a very weird form of management" and even "getting good results out of a coding agent feels uncomfortably close to managing a human intern". You need to provide clear instructions (the spec), ensure they have the necessary context (the spec and relevant data), and give actionable feedback. The spec sets the stage, but monitoring and feedback during execution are key. If an AI was a "weird digital intern who will absolutely cheat if you give them a chance", the spec and constraints you write are how you prevent that cheating and keep them on task.

Here's the payoff: a good spec doesn't just tell the AI what to build, it also helps it self-correct and stay within safe boundaries. By baking in verification steps, constraints, and your hard-earned knowledge, you drastically increase the odds that the agent's output is correct on the first try (or at least much closer to correct). This reduces iterations and those "why on Earth did it do that?" moments.

5. Test, iterate, and evolve the spec (and use the right tools)

Think of spec-writing and agent-building as an iterative loop: test early, gather feedback, refine the spec, and leverage tools to automate checks.

The initial spec is not the end - it's the beginning of a cycle. The best outcomes come when you continually verify the agent's work against the spec and adjust accordingly. Also, modern AI devs use various tools to support this process (from CI pipelines to context management utilities).

Continuous testing: Don't wait until the end to see if the agent met the spec. After each major milestone or even each function, run tests or at least do quick manual checks. If something fails, update the spec or prompt before proceeding. For example, if the spec said "passwords must be hashed with bcrypt" and you see the agent's code storing plain text - stop and correct it (and remind the spec or prompt about the rule). Automated tests shine here: if you provided tests (or write them as you go), let the agent run them. In many coding agent setups, you can have an agent run npm test or similar after finishing a task. The results (failures) can then feed back into the next prompt, effectively telling the agent "your output didn't meet spec on X, Y, Z - fix it." This kind of agentic loop (code -> test -> fix -> repeat) is extremely powerful and is how tools like Claude Code or Copilot Labs are evolving to handle larger tasks. Always define what "done" means (via tests or criteria) and check for it.

Iterate on the spec itself: If you discover that the spec was incomplete or unclear (maybe the agent misunderstood something or you realized you missed a requirement), update the spec document. Then explicitly re-sync the agent with the new spec: "I have updated the spec as follows... Given the updated spec, adjust the plan or refactor the code accordingly." This way the spec remains the single source of truth. It's similar to how we handle changing requirements in normal dev - but in this case you're also the product manager for your AI agent. Keep version history if possible (even just via commit messages or notes), so you know what changed and why.

Utilize context-management and memory tools: There's a growing ecosystem of tools to help manage AI agent context and knowledge. For instance, retrieval-augmented generation (RAG) is a pattern where the agent can pull in relevant chunks of data from a knowledge base (like a vector database) on the fly. If your spec is huge, you could embed sections of it and let the agent retrieve the most relevant parts when needed, instead of always providing the whole thing. There are also frameworks implementing the Model Context Protocol (MCP), which automates feeding the right context to the model based on the current task. One example is Context7 (context7.com), which can auto-fetch relevant context snippets from docs based on what you're working on. In practice, this might mean the agent notices you're working on "payment processing" and it pulls the "Payments" section of your spec or documentation into the prompt. Consider leveraging such tools or setting up a rudimentary version (even a simple search in your spec document).

Parallelize carefully: Some developers run multiple agent instances in parallel on different tasks (as mentioned earlier with subagents). This can speed up development - e.g., one agent generates code while another simultaneously writes tests, or two features are built concurrently. If you go this route, ensure the tasks are truly independent or clearly separated to avoid conflicts (the spec should note any dependencies). For example, don't have two agents writing to the same file at once. One workflow is to have an agent generate code and another review it in parallel, or to have separate components built that integrate later. This is advanced usage and can be mentally taxing to manage (as Willison admitted, running multiple agents is surprisingly effective, if mentally exhausting!). Start with at most 2-3 agents to keep things manageable.

Version control and spec locks: Use Git or your version control of choice to track what the agent does. Good version control habits matter even more with AI assistance. Commit the spec file itself to the repo. This not only preserves history, but the agent can even use git diff or blame to understand changes (LLMs are quite capable of reading diffs). Some advanced agent setups let the agent query the VCS history to see when something was introduced - surprisingly, models can be "fiercely competent at Git". By keeping your spec in the repo, you allow both you and the AI to track evolution. There are tools (like GitHub Spec Kit mentioned earlier) that integrate spec-driven development into the git workflow - for instance, gating merges on updated specs or generating checklists from spec items. While you don't need those tools to succeed, the takeaway is to treat the spec like code - maintain it diligently.

Cost and speed considerations: Working with large models and long contexts can be slow and expensive. A practical tip is to use model selection and batching smartly. Perhaps use a cheaper/faster model for initial drafts or repetitions, and reserve the most capable (and expensive) model for final outputs or complex reasoning. Some developers use GPT-4 or Claude for planning and critical steps, but offload simpler expansions or refactors to a local model or a smaller API model. If using multiple agents, maybe not all need to be top-tier; a test-running agent or a linter agent could be a smaller model. Also consider throttling context size: don't feed 20k tokens if 5k will do. As we discussed, more tokens can mean diminishing returns.

Monitor and log everything: In complex agent workflows, logging the agent's actions and outputs is essential. Check the logs to see if the agent is deviating or encountering errors. Many frameworks provide trace logs or allow printing the agent's chain-of-thought (especially if you prompt it to think step-by-step). Reviewing these logs can highlight where the spec or instructions might have been misinterpreted. It's not unlike debugging a program - except the "program" is the conversation/prompt chain. If something weird happens, go back to the spec/instructions to see if there was ambiguity.

Learn and improve: Finally, treat each project as a learning opportunity to refine your spec-writing skill. Maybe you'll discover that a certain phrasing consistently confuses the AI, or that organizing spec sections in a certain way yields better adherence. Incorporate those lessons into the next spec. The field of AI agents is rapidly evolving, so new best practices (and tools) emerge constantly. Stay updated via blogs (like the ones by Simon Willison, Andrej Karpathy, etc.), and don't hesitate to experiment.

A spec for an AI agent isn't "write once, done." It's part of a continuous cycle of instructing, verifying, and refining. The payoff for this diligence is substantial: by catching issues early and keeping the agent aligned, you avoid costly rewrites or failures later. As one AI engineer quipped, using these practices can feel like having "an army of interns" working for you, but you have to manage them well. A good spec, continuously maintained, is your management tool.

Avoid common pitfalls

Before wrapping up, it's worth calling out anti-patterns that can derail even well-intentioned spec-driven workflows. The GitHub study of 2,500+ agent files revealed a stark divide: "Most agent files fail because they're too vague." Here are the mistakes to avoid:

Vague prompts: "Build me something cool" or "Make it work better" gives the agent nothing to anchor on. As Baptiste Studer puts it: "Vague prompts mean wrong results." Be specific about inputs, outputs, and constraints. "You are a helpful coding assistant" doesn't work. "You are a test engineer who writes tests for React components, follows these examples, and never modifies source code" does.

Overlong contexts without summarization: Dumping 50 pages of documentation into a prompt and hoping the model figures it out rarely works. Use hierarchical summaries (as discussed in Principle 3) or RAG to surface only what's relevant. Context length is not a substitute for context quality.

Skipping human review: Willison has a personal rule: "I won't commit code I couldn't explain to someone else." Just because the agent produced something that passes tests doesn't mean it's correct, secure, or maintainable. Always review critical code paths. The "house of cards" metaphor applies: AI-generated code can look solid but collapse under edge cases you didn't test.

Conflating vibe coding with production engineering: Rapid prototyping with AI ("vibe coding") is great for exploration and throwaway projects. But shipping that code to production without rigorous specs, tests, and review is asking for trouble. I distinguish "vibe coding" from "AI-assisted engineering" - the latter requires the discipline this guide describes. Know which mode you're in.

Ignoring the "lethal trifecta": Willison warns of three properties that make AI agents dangerous: speed (they work faster than you can review), non-determinism (same input, different outputs), and cost (encouraging corner-cutting on verification). Your spec and review process must account for all three. Don't let speed outpace your ability to verify.

Missing the six core areas: If your spec doesn't cover commands, testing, project structure, code style, git workflow, and boundaries, you're likely missing something the agent needs. Use the six-area checklist from Section 2 as a sanity check before handing off to the agent.

Conclusion

Writing an effective spec for AI coding agents requires solid software engineering principles combined with adaptation to LLM quirks. Start with clarity of purpose and let the AI help expand the plan. Structure the spec like a serious design document - covering the six core areas and integrating it into your toolchain so it becomes an executable artifact, not just prose. Keep the agent's focus tight by feeding it one piece of the puzzle at a time (and consider clever tactics like summary TOCs, subagents, or parallel orchestration to handle big specs). Anticipate pitfalls by including three-tier boundaries (Always/Ask first/Never), self-checks, and conformance tests - essentially, teach the AI how to not fail. And treat the whole process as iterative: use tests and feedback to refine both the spec and the code continuously.

Follow these guidelines and your AI agent will be far less likely to "break down" under large contexts or wander off into nonsense.

Happy spec-writing!

I'm excited to share I've released a new AI-assisted engineering book with O'Reilly. There are a number of free tips on the book site in case interested.

Incremental Code Migrations

Addy Osmani — Sun, 08 Nov 2020 00:00:00 +0000

Incremental migrations are excellent for updating large legacy projects - migrate a small piece at a time with lower risk then gradually do the rest.

Incremental migration strategies for large legacy software systems include breaking down the system into smaller components that can be selectively and individually replaced. This phased migration reduces the expense and risk of a migration compared to doing all of an update or rewrite at once (e.g a phased migration to TypeScript, ES Modules or a newer version of a framework). In some cases, some of the legacy components might be encapsulated in wrappers so they can be used in their original form until new components with a decent degree of reliability can take over their function.

A phased migration from the legacy architecture to the target one can have a lot of benefits. When done well, there can be a fully functioning hybrid legacy/target version of the system running in production while getting key insights into what works and what has rough edges. You may wish to prioritize the order in which phases of your migration take place keeping in mind what the most pressing objectives need to be tackled first. Alternatively, first tackle mission-critical stages to lift the profile of the project and get buy-in to see through the full migration.

Since the system is still operational after each phase of the migration, it is possible to appreciate the benefits of the project faster than if the entire legacy were migrated and implemented at once. Often, bugs in a small component are easier to identify and fix than in a large system. This decreases the risk and inconvenience of the project to users.

There are a number of alternatives to incremental or phased migration, including:

Big bang adoption where a full migration is rolled out on an agreed on date, replacing the old system with a new system. This is effectively an instant changeover.
Parallel adoption where both old and new systems run in parallel (simultaneously) for a time, until the implementation of the new system is considered complete.
Pilot conversion where a very partial migration is rolled out to a group of trusted testers for evaluation before the implementation rolls out to more users (e.g trying out a migration for a specific component or page).

I tend to favor incremental migrations for many reasons, including that lessons gained from phased migration phases can be used to direct the remainder of the project. That way as the implementation progresses, there are ideally fewer issues.

Monitoring Performance with the PageSpeed Insights API

Addy Osmani — Mon, 06 Jan 2020 17:57:42 +0000

The PageSpeed Insights API provides free access to performance monitoring for web pages and returns data with suggestions for how to improve. The V5 API includes lab data from Lighthouse and real-world data from the Chrome User Experience Report (CrUX).



const url = 'https://wordpress.org';
const apiCall = `https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=${url}`;
const response = await fetch(apiCall);
const json = await response.json();

Teams use the API to build dashboards, custom reports and custom integrations with other user-experience measurement tools. The responses from the API could be used to monitor and graph any of the data from the PageSpeed Insights (PSI) tool:

	The PSI score (json.lighthouseResult.categories.performance.score) is determined by running Lighthouse to analyze lab data about the page. A score at or above 90 is considered fast and below 50 is considered to be slow. See the FAQ for the latest on scoring and bit.ly/perf-variance to learn about score variance.
	If available, PSI will report field metric values (json.loadingExperience.metrics) for First Contentful Paint and First Input Delay data from the Chrome User Experience Report for the origin or page URL.
	PSI uses Lighthouse to analyze a URL, generating a performance score that factors in a number of different metrics in a lab setting, like Time to Interactive (json.lighthouseResult.audits['interactive']).
	The Lighthouse report opportunities (e.g json.lighthouseResult.audits['uses-rel-preload'], json.lighthouseResult.audits['offscreen-images'] etc.) provide suggestions how to improve the page’s performance metrics.
	Thumbnail screenshots from the load of your site are available as base64 images via json.lighthouseResult.audits['screenshot-thumbnails']. The last screenshot from pageload is available via json.lighthouseResult.audits['final-screenshot'].

It's possible to build highly customized reports using PSI data. e.g VRBO, a vacation rentals site, graph real-world data from the PSI API to track long term performance trends to ensure their speed remains competitive within the travel industry:

web.dev/measure uses the exact same backend as PageSpeed Insights and uses this data for historical measurement over time:

It's helpful to become familiar with the structure of the PSI API response. There's extensive metrics info available for lab and field/real-world:



const url = 'https://wordpress.org';
const apiCall = `https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=${url}`;
fetch(apiCall)
  .then(response => response.json())
  .then(json => {
      // Real-world metrics
      const cruxMetrics = {
      "First Contentful Paint": json.loadingExperience.metrics.FIRST_CONTENTFUL_PAINT_MS.category,
      "First Input Delay": json.loadingExperience.metrics.FIRST_INPUT_DELAY_MS.category
      };
      // Lab metrics
      const lighthouse = json.lighthouseResult;
      const lighthouseMetrics = {
      'First Contentful Paint': lighthouse.audits['first-contentful-paint'].displayValue,
      'Speed Index': lighthouse.audits['speed-index'].displayValue,
      'Time To Interactive': lighthouse.audits['interactive'].displayValue,
      };
      // ...
});

Responses from the PSI API are focused on performance data. That said, you can supply the ?category argument to specify any additional Lighthouse audit categories you would like data for:



curl -i "https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https://web.dev&category=pwa&category=performance&category=accessibility&category=best-practices&category=seo"

You can also supply a locale or strategy argument (desktop or mobile - which simulates a page load on a median-class device (Moto G4) on a mobile network).

Lighthouse is starting to support Stack Packs - stack-specific recommendations, providing more detailed guidance on how to implement optimizations (e.g WordPress). PSI's API responses also support this so if you're testing a WordPress site, these strings are included (demo URL for wordpress.org):

The PageSpeed Insights Tool also supports fetching PSI API data and rendering it with our official Lighthouse Viewer. Pass ?psiurl as a parameter to see this in action:

One of the tools we built on top of PSI is the psi Node module, offering convenient performance reporting in your build process.



const psi = require('psi');

(async () => {
  // Get the PageSpeed Insights report
  const { data } = await psi('https://web.dev');
  console.log('Speed score:', data.lighthouseResult.categories.performance.score);

  // Output a formatted report to the terminal
  await psi.output('https://theverge.com');
  console.log('Done');
})();

The output of psi looks a little like this when used as a CLI with PageSpeed Insights V5:

To try out the PSI API, check out this Glitch demo using both Lighthouse and CrUX data.

You can also use Google Sheets and a cron job to automate monitoring multiple URLs (e.g competitors) by regularly pinging the PSI API. A handy guide and sheet you can copy are available.

What about third-party resources?

We know third-party scripts are costly. In the top 4 million sites, ~2700 origins are responsible for ~57% of all script execution time. Lighthouse exposes an impact summary for third-party code, available via the API from json.lighthouseResult.audits['third-party-summary'].

PageSpeed Insights API Limits

For infrequent usage, the PSI API functions without an API key.

If you intend to make multiple queries per second however, it is recommended to sign up for a PSI API key. API keys have a daily limit of 25,000 queries a day or 400 queries per 100 seconds. This should be enough for most dashboarding needs. It is also possible to request a higher quota if needed.

What's next for the PSI API?

We hope to bring metrics like Largest Contentful Paint and Cumulative Layout Shift to Lighthouse, CrUX and PageSpeed Insights in 2020. We may also explore better hosted API options for obtaining field-data from CrUX. In the interim, you may be interested in Crux.run, a fast mirror of CrUX reporting data which also has an API.

Additional reading

With thanks to Shane Exterkamp for his review

Loading web pages fast on a $20 feature phone

Addy Osmani — Tue, 17 Dec 2019 18:59:49 +0000

Tip: Building a fast, core foundation for your site gives everyone a good experience; whether they're on a low-cost feature phone or the latest high-end smart phone.

Introduction

Feature phones are affordable (under $20-25), low-end devices enabling 100s of millions of users in developing countries to leverage the web. Think of them as a light version of a smart phone.

Being low-cost, feature phones tend to have slower CPUs (think 6x slower than a high-end smart phone), low RAM (e.g 256MB-512MB, typically under 4GB), low storage (4G) and often don't have a touchscreen. Instead they use a keypad or D-Pad for navigation. Something like this:

These devices can't handle rich JavaScript & media experiences as well as high-end smart phones so special care must be taken with payloads you send down.

Above, we can see Geekbench CPU performance benchmarks for the highest selling smart phones globally in 2019. We can also see (highlighted) the performance of a popular feature phone - the Nokia 3110. JavaScript stresses single-core performance (remember it's inherently more single-threaded than the rest of the Web Platform) and is CPU bound. This means keeping device characteristics in mind when thinking about developing countries is important.

This post attempts to address these pain points so we can build sites that are usable by everyone, regardless of how fast their device is.

Background

You may or may not remember that feature phones were popular until the mid 2000s, before the arrival of smart phones. Small in size with a key-pad instead of a touchscreen, they had pretty basic features e.g calling, texting and simple text-focused web browsing. After the arrival of smart phones, these phones became less prevalent in developed countries.

In developing countries, not everyone can afford a smart phone and an unlimited data-plan on a 4G network. This market has been captured by smart feature phones which combine the hardware and price of a basic phone with some of the functionality found in smart phones.

The smart feature phone market has grown considerably since 2017 and it is expected that 400 million feature phones will be sold globally in 2019.

The growth of feature phones has been powered by Nokia’s revival of its old favourites like the 3110 and 8110 (which Paul Kinlan has a handy debugging guide for). In India, Reliance Jio feature phones provide a cheap, but modern option for accessing the web on the go. Jio have propelled the growth of KaiOS - a Linux-based operating system for feature phones.

This growth in the feature phone market created a need for sites that can run efficiently but there are a few constraints we should probably be aware of.

Above are Google Images Lite and Facebook mBasic, which both load quickly on feature phones, with minimal reliance on client-side script. Proxx being a game does rely on script, but relies on aggressive code-splitting to load fast.

Feature phone constraints

Users in developing countries are restricted by 3 factors:

Few low cost, high quality devices
Unavailability of high quality networks
Affordable mobile data

Keep these constraints in mind when adopting a feature phone mindset:

Hardware: Feature phones usually run on a slow (1.1 GHz) single or dual core CPU and less than 512 MB RAM. Compare this to an iPhone XS with a Hexa-core CPU and 4 GB RAM for some perspective on what this constraint implies.
Data: Data plans are getting cheaper, but are still heavily capped in regions where feature phones will be popular. Avoid large network payloads as much as possible to ensure your pages load fast and aren't costly users money.
Limited screen size: The screen size of a feature phone is typically much smaller than that of a smart phone. It’s in the range of 2.4” and can accommodate only a limited number of interactions. Consider focusing on loading resources needed for in-viewport content as quick as possible.
No Touch: In the absence of touch, each function, action button or link on the screen needs to be easily accessible from the keypad. You don't want to create too many shortcuts.
Keypads: Keypads on feature phones are nothing like the QWERTY keyboards we are used to. They have around 15 buttons where the same button needs to be pressed repeatedly for some characters. Hence, UX should minimize the need for typing.

Even in Japan, limited data plans can impact user experience on the web:

Yoav Weiss

@yoavweiss

In Japan, the web is slower towards the end of the month due to data caps, resulting in reduced engagement

17:51 PM - 14 Nov 2019

1566 3627

Development Guidelines

The following tips can help deliver a fast experience for websites on feature phones. In general, don't make the user wait for anything they didn't ask for. Where possible, keep JavaScript download and execution times low.

Set performance budgets for your initial payloads

Every byte of your page goes through many bottlenecks along the way. These include slow/flaky networks, slow CPUs and the only guaranteed way to improve performance is set budgets and do less.

Performance budgets are an agreed upon set of limits your team follow to ensure good performance. They are constraints for metrics that you commit not to exceed. Defining a budget for quantifiable metrics before development starts helps ensure teams remain within these standards as new features are developed.

Examples of resource metrics that can be budgeted could be JavaScript bundle size, image bytes or number of HTTP requests. User timing budgets could be set for user-experience metrics like First Contentful Paint, Largest Contentful Paint or First Input Delay. Thresholds can be defined for each of these metrics based on the target audience.

Budgets can be set for initial application logic, vendor/commons bundles and so on. Budget can be enforced in a build process, via Lighthouse (LightWallet) and in continuous integration.

PRPL-30 - a JavaScript budget for feature phones

The Chrome team has recommended the PRPL pattern for granularly serving code to quickly get apps interactive on low-end devices on a slow network. PRPL encourages preloading the smallest amount of script to get a page useful then leverages lazy-loading and (optionally) Service Workers to precache these bundles for future navigations.

PRPL-50 took this idea further by setting a 50KB budget for your initial resources. As feature phones are even more CPU-constrained, we need to set an even tighter limit on our JavaScript. I might suggest a PRPL-30 (30KB gzipped + minified initial bundle) if targeting a feature phone.

Under these settings, the first byte from a good edge-caching server after SSL negotiation will usually be around 2s. This leaves us with 3 seconds to get the initial route's payload downloaded, rendered, and ready for user input on the screen. For JavaScript-centric experiences, this means your total initial bundle size for your page or route should be < 30KB minified and gzipped.

Wait. A 30KB initial bundle? That's ridiculous. I can't even fit React and my app code in there!. When building for really constrained devices, you're going to have to make difficult trade-offs in the name of user-experience. For example, you can use React for feature phones if you either (1) limit use of React to the server - a totally fair option or (2) keep your chunks for application logic very, very small and aggressively lazy-load. A (3) would be opting for a solution like Preact, but we'll discuss these trade-offs more later.

An example of an application that works under this 30KB budget is Proxx which has a 25 KB initial size and a Time to Interaction (TTI) of less than 5 seconds. Our perf budgets calculator may be helpful as you’re planning out your own target metrics.

A recommended size for lazy-loaded routes is also less than 35 KB. Route "chunks" between 30 KB and 35 KB would still be large enough to be considered for parallel processing via V8's script streaming.

Be Frugal with JavaScript

tl;dr If you can, opt for static rendering or server rendering with as minimal reliance on script as possible. If client-side or hybrid rendering is a must, only send script a route needs, in as few round trips as possible. Consider techniques like progressive rehydration.

JS is the #1 bottleneck on feature phones

When building interactive experiences for feature phones, be aware that JavaScript is likely your biggest bottleneck. This is important as how you choose to render your pages can delay how soon users can actually use them, even with a D-Pad. If you choose to server-render or static render your pages, ensure interactive payloads are as small as possible.

JavaScript has two key costs: download and execution time. A slow network (e.g effective connection of 3G) can delay how quickly JavaScript gets over the wire while a slow CPU means it can take longer for scripts to execute. Below we can visualize the differences in CPU processing speeds for a popular JavaScript-heavy site - notice it can take 6x times longer for a low-end device compared to a high-end one:

CPU performance comparison between different smart phones

If a page relies heavily on large JS bundles for rendering or interactivity, this can translate to users waiting anywhere between 30-60s before they can even use your UI on a feature phone. Thus to minimize the download and processing time required by JavaScript, developers need to use it frugally and load only the JavaScript for routes and components that may be required by the users, as and when they would need it.

Keep interactive payloads lean

As much as possible, lazy-load routes, components and resources that are offscreen or not critical for above-the-fold content.
Use code-splitting to break up JavaScript so you only serve what users need on the initial route. This would result in faster page load time as amount of script to be downloaded and executed is reduced.
Remove unused code from JavaScript bundles to keep them as lean as possible. This would require you to analyse your bundle and detect the libraries which are not being used at all or not required and may be replaced by custom ones. Libraries which are not required during the initial load may also be lazy loaded.
Consider differential loading so that modern JS will be served to modern browsers, avoiding over-transpilation and excessive polyfilling. Reducing how much legacy code is shipped to modern browsers can help improve page load performance.
If JS is critical for rendering & getting the initial UX interactive, preload those scripts. Preloading the script as shown below tells the browser that it is important and should be fetched as soon as possible.

<link rel="preload" as="script" href="critical.js">‬

Choose your stack wisely

While third party libraries help to speed up development and easily achieve complex tasks, they can also be heavy and should be used with care when developing for feature phones. The following guidance can help to optimally incorporate external dependencies:

As feature phones are very resource constrained, avoid or limit using a JavaScript framework if possible - it's all about giving yourself as much headroom as possible for your application logic. In a situation where JavaScript use should be limited in order to keep the page small and performant, a JavaScript framework would add a lot of overhead. If building with React, consider limiting use to server-rendering or swap it for Preact and Preact compat at build time to shave around 30KB off your bundle. Svelte, lit-html and vanilla JavaScript are all good options for light bundles.
Keep third party dependencies as small as possible to ensure that the size of initial page bundles is within budget- see tools like bundlephobia.com which highlight library costs well. Sanity check your bundles to ensure you’re using lean libraries (E.g. date-fns or luxon instead of Moment.js and its large locale defaults).
Be very cautious when using Redux and inline Redux stores for managing application state. The redux state often inlined in HTML to “hydrate” a page can often bloat up the size of your responses.

Adapt to avoid loading heavy resources on slow connections

Tip: You may be interested in adaptive loading - improving web performance on low-end devices for more on this topic.

Adaptive Loading is a technique which “adapts” resources being served to a user based on their effective connection type (ECT) - available to supported browsers via the Network Information API. Adaptive serving allows sites to ensure users on slow connections can get “a” version of the experience, even if at a lower-fidelity.

console.log(navigator.connection.effectiveType); // 3G

Note: Even on "fast" 4G, a user can experience slow network speeds as you may have experienced on coffee-shop or conference WiFi.

A concrete example where adaptive serving could be used is in a product item component. A user on a slow connection may only be served a compressed version of the product image, while a user on a faster connection may load a high-quality image and enhancements requiring more JavaScript - like the ability to zoom into the product image or view a carousel of different product image perspectives.

With feature phones a slow connection may not necessarily be the biggest obstacle. Slow CPU and low memory are more likely to affect the users experience even when they are on a decent 4G connection. Although we don’t yet have access to CPU reporting, Client Hints provide approximates for Device Memory, Viewport-width, Device Pixel Ratio, network information and other signals which could be used to craft a more granular serving strategy.

Be respectful of users data plans with the Save-Data header

Chrome for Android has a feature called Lite Mode (Data Saver), allowing data-conscious users to opt-in to the browser automatically optimizing resources to improve how quickly pages load. Optimizations include more heavily compressing images, deferring non-critical resources or server-rendering previews of the page. More on Chrome’s efforts can be found in our Lite Pages blog post.

When users enable a data-saving mode (like Lite Mode) in supported browsers, it will append the Save-Data request header to all HTTP and HTTPS requests. Application developers can check if this hint is enabled in JavaScript to deliver an optimized experience to users who have Lite Mode on (e.g conditionally turning off heavy features). The following snippet of code can be used to check for this:

if ("connection" in navigator) {
  if (navigator.connection.saveData === true) {
    // Implement data saving operations here.
  }
}

Note: Although your feature phone may support Chrome, this does not guarantee that Lite Mode (Data Saver) is available. It is best to treat availability of this feature as speculative.

Offload costly app logic and state handling to Web Workers

Tip: Do read Techniques to make a web app load fast, even on a feature phone by Surma. It's excellent.

Besides running JavaScript, the browser’s main thread has other responsibilities like page layout, painting pixels on the screen, and tracking user interactions. Long-running, complex JavaScript, could end up blocking these other tasks.

Web Workers allow JavaScript code to run in the “background” without blocking the main thread. They should be used to keep the main thread free from costly JavaScript overhead such as complex app logic or state management services. The main thread and the worker thread communicate using the postMessage() function and onmessage handler. The postMessage function allows the sender to send a single parameter which can be any value or object. Libraries like Comlink reduce the friction for using workers in your apps.

Surma's case study with and without worker threads for Proxx is a great read - it was observed that without worker threads the app was frozen for 6.6 seconds on a Nokia 2 (1GB RAM/1.3 GHz quad-core processor). With worker threads however the response time was 48ms for the same action/event. Thus If you have CPU intensive logic, it's worth measuring whether moving this to a worker thread is worthwhile for your use-case.

Optimize Images

Images often consume a lot of data. They also take time to decode, especially on lower-end devices, which is a good reason it’s important to ensure images are correctly sized and compressed when being delivered to a feature phone.

Compress images using a tool like Imagemin to generate images which are smaller in size without a perceivable effect on quality.
Replace animated GIF’s with videos which load much faster. Even then, really consider how much you need heavy multimedia on low-end devices.
Lazy load images if possible but ensure JavaScript required for this is not too heavy. The new native loading attribute can help here.
Serve responsive images with correct dimensions by creating multiple versions of images and serving the most appropriate for the user’s viewport
Serve images with the correct dimensions for the screen. Loading low-fidelity, low resolution images on low-end devices can ensure that images are decoded faster.

Detecting screen-size

Many current smart feature phones have a QVGA screen size in which the resolution is 320px horizontally by 240px vertically (320 x 240). If you need to detect screen size on page load (e.g to toggle certain features on/off or use adaptive loading), you could use a snippet like the following:

const isFeaturePhone = Math.min(screen.width, screen.height) <= 240;

This is similar to the approach Proxx uses.

Emulate a feature phone in Chrome DevTools

If building for a feature phone, it's highly recommended to pick up a cheap handset for real-world testing.

Mariko Kosaka

@kosamari

PSA: if you like to test if your website works on KaiOS & if you live in US, a trip to BestBuy and $15 will get you this device for testing.

00:45 AM - 13 May 2019

4 47

If you'd like to emulate a feature phone (e.g a Reliance Jio KaiOS device) in Chrome DevTools, here's how:

Open Chrome DevTools
Toggle the Device Toolbar
Device dropdown > Edit ... > Add a custom device
Name: KaiOS (or customize as needed)
Width: 240, Height: 320
UA: Mozilla/5.0 (Mobile; LYF/F90M/LYF-F90M-000-02-23-181217; Android; rv:48.0) Gecko/48.0 Firefox/48.0 KAIOS/2.5 YouTube/1.66.51.J
Save
(Optionally) customize CPU throttling as needed, however note that this can never be as accurate as testing on a real-device.

Conclusion

It's possible to deliver a delightful experience to all users, regardless of where they are. This does require extra care as not all mobile hardware is equal.

The more affordable a phone handset is, the higher the chance it may contain a slow CPU. Given JavaScript performance depends on download and execution time, do think about how you're delivering your experience.

While this matters for smart phones, it matters even more for feature phones.

Accessibility Tips for Web Developers

Addy Osmani — Wed, 04 Dec 2019 20:28:54 +0000

It's awesome to build sites that are inclusive and accessible to everyone. There are at least six key areas of disability we can optimize for: visual, hearing, mobility, cognition, speech and neural. Many tools and resources can help here, even if you're totally new to web accessibility.

Over 1 billion people live with some form of disability. You might have been in a loud room at some point trying to hear the conversation around you or in a low-lighting condition trying to find something in the dark. Do you remember the frustration you felt with that circumstance? Now, imagine if that temporary condition was permanent. How different would your experience on the web be?

To be accessible, sites need to work across multiple devices with varying screen-sizes and different kinds of input, such as screen readers. Moreover, sites should be usable by the broadest group of users, including those with disabilities. Here are a sample of just a few disabilities your users may have:

Vision	Hearing	Mobility
- Low vision - Blind - Color blind - Cataracts - Sun glare	- Hard of hearing - Deaf - Noise - Ear infection	- Broken arm - Spinal cord injury - Limited dexterity - Hands full
Cognitive	Speech	Neural
- Learning disabilities - Sleepy or distracted - Migraine - Autism - Seizure	- Ambient noise - Sore throat - Speech impediment - Unable to speak	- Depression - PTSD - Bipolar - Anxiety

Visual issues range from an inability to distinguish colors to no vision at all.

Ensure a minimum contrast ratio threshold is met for text content.
Avoid communicating information using solely color and ensure that all text is resizable.
Ensure all user interface components can be used with assistive technologies such as screen readers, magnifiers and braille displays. This entails ensuring that UI components are marked up such that accessibility APIs can programmatically determine the role, state, value and title of any element.

Tip: Inspect element in Chrome, Edge and Firefox DevTools displays a tooltip of CSS properties that includes a quick check for color contrast ratio.

I personally live with low-vision and am embarrassed to say, I'm that person that always zooms in on sites, their DevTools and terminal. While supporting zoom is almost never at the top of anyone's list, optimizing for low-vision users is always appreciated... 🤓

Hearing issues mean a user may have issues hearing sound emitted from a page.

Make the content understandable using text alternatives for all content that is not strictly text.
Ensure you test that your UI components are still functional without sound.

Mobility issues can include the inability to operate a mouse, a keyboard or touch-screen.

Make the content of your UI components functionally accessible from a keyboard for any actions one would otherwise use a mouse for.
Ensure pages are correctly marked up for assistive technologies; these users may use technologies such as voice control software and physical switch controls, which tend to use the same APIs as other assistive technologies like screen readers.

Cognitive issues mean a user may require assistive technologies to help them with reading text, so it’s important to ensure text alternatives exist.

Be mindful when using animations. Avoid a visual presentation that is repetitive or flashing as this can cause some users issues.

The prefers-reduced-motion CSS media query allows you to limit animations and autoplaying videos for users who prefer reduced motion.

/*
If the user expressed a preference for reduced motion, don't use animations on buttons.
*/
@media (prefers-reduced-motion: reduce) {
  button {
    animation: none;
  }
}

Avoid interactions that are timing-based.

This may seem like a lot of bases to cover, but we’ll walk through the process for assessing and then improving the accessibility of your UI components.

Tip: Some great accessibility do's and don'ts digital posters are available by the Gov.uk accessibility team for spreading awareness of best practices in your team:

Are your UI components accessible?

Summary (tl;dr)

When auditing your page's UI components for accessibility, ask yourself:

Can you use your UI component with the keyboard only? Does it manage to focus and avoid focus traps? Can it respond to the appropriate keyboard events?
Can you use your UI component with a screen reader? Have you provided text alternatives for any information which is presented visually? Have you added semantic information using ARIA?
Can your UI component work without sound? Turn off your speakers and go through your use cases.
Can it work without color? Ensure your UI component can be used by someone who cannot see colors. A helpful tool for simulating color blindness is a Chrome extension called SEE, (try all four forms of color blindness simulation available). You may also be interested in the Daltonize extension which is similarly useful.
Can your UI component work with high-contrast mode enabled? All modern operating systems support a high contrast mode. High Contrast is a Chrome extension available that can help here.

Native controls (such as <button> and <select>) have accessibility built-in by the browser. They are focusable using the tab key, respond to keyboard events (like Enter, space and arrow keys), and have semantic roles, states and properties used by accessibility tools. The default styling should also meet the accessibility requirements listed above.

Custom UI components (with the exception of components that extend native elements like <button>) do not have any built-in functionality, including accessibility, so this needs to be provided by you. A good place to start when implementing accessibility is to compare your components to an analogous native element (or a combination of several native elements, depending on how complex your component is).

Tip: Most browser DevTools support inspecting the accessibility tree of a page. In Chrome, this is available via the Accessibility tab in the Elements panel.

Firefox also has an Accessibility panel and Safari exposes this information in the Element's panel Node tab.

The following is a list of questions you can ask yourself when attempting to make your UI components more accessible.

Can your UI component be used with the keyboard alone?

Ideally, ensure that all functionality in your UI component can be reached by a keyboard. During your UX design, think about how you would use your element with the keyboard alone, and figure out a consistent set of keyboard interactions.

Firstly, ensure that you have a sensible focus target for each component. For example, a complex component like a menu may be one focus target within a page, but should then manage focus within itself so that the active menu item always takes focus.

Managing focus within a complex element

Using tabindex

The tabindex attribute allows elements / UI components to be focused using the keyboard. Keyboard-only and assistive technology users both need to be able to place keyboard focus on elements in order to interact with them. Native interactive elements are implicitly focusable, so they don’t need a tabindex attribute unless we wish to change their position in the tab order.

There are three types of tabindex values:

tabindex="0" is the most common, and will place the element in the “natural” tab order (defined by the DOM order).
a tabindex value greater than 0 will place the element in a manual tab order — all elements in the page with a positive tabindex value will be visited in numerical order before elements in the natural tab order.
a tabindex value equal to -1 will cause the element to be programmatically focusable, but not in the tab order.

For custom UI components, always use tabindex values of 0 or -1, as you won’t be able to determine the order of elements on a given page ahead of time — and even if we did, they may be subject to change. A tabindex value of -1 is particularly useful for managing focus within complex components as described above.

Also ensure that focus is always visible, whether by allowing the default focus ring style, or applying a discernible focus style. Remember not to trap the keyboard user — focus should be able to be moved away from an element using only the keyboard.

You may also be interested in the roving tabindex or aria-activedescendant approaches, covered over on MDN.

Using autofocus

The HTML autofocus attribute allows an author to specify that a particular element should automatically take focus when the page is loaded. It is already supported on all web form controls, including . To autofocus elements in your own custom UI components, call the focus() method supported on all HTML elements that can be focused (e.g document.querySelector('myButton').focus()).

Adding keyboard interaction

Once your UI component is focusable, try to provide a good keyboard interaction story when a component is focused, by handling appropriate keyboard events — for example, allow the user to use arrow keys to select menu options, and space or enter to activate buttons. The ARIA design patterns guide provides some guidance here.

Finally, ensure that your keyboard shortcuts are discoverable. For example, a common practice is to have a keyboard shortcut legend (on-screen text) to inform the user that shortcuts exist. For example, "Press ? for keyboard shortcuts". Alternatively a hint such a tooltip could be used to inform the user about the shortcut existing.

The importance of managing focus cannot be understated. One example is a navigation drawer. If adding a UI component to the page you need to direct focus to an element inside of it otherwise users may have to tab through the entire page to get there. This can be a frustrating experience, so be sure to test focus for all keyboard navigable components in your page.

Tip: You can use Puppeteer to automate running keyboard accessibility tests for toggling UI states. WalkMe Engineering have a great guide on this I recommend reading.

// Example for expanding and collapsing a category with the spacebar key
const category = await page.$(`.category`);

// verify tabIndex, role and focus
expect(await page.evaluate(elem => elem.getAttribute(`role`), category)).toEqual(`button`);
expect(await page.evaluate(elem => elem.getAttribute(`tabindex`), category)).toEqual(`0`);
expect(await page.evaluate(elem => window.document.activeElement === elem, category)).toEqual(true);

// verify aria-expanded = false
expect(await page.evaluate(elem => elem.getAttribute(`aria-expanded`), category)).toEqual(`false`);

// toggle category by press space
await page.keyboard.press('Space');

// verify aria-expanded = true
expect(await page.evaluate(elem => elem.getAttribute(`aria-expanded`), category)).toEqual(`true`);

Can you use your UI component with a screen reader?

Around 1–2% of users will be using a screen reader. Can you determine all important information and interact with the component using the screen reader and keyboard alone?

The following questions should help guide you in addressing screen reader accessibility:

Do all components and images have meaningful text alternatives?

Wherever information about the name or purpose of an interactive component is conveyed visually, an accessible text alternative needs to be provided.

For example, if your <fancy-menu> UI component only displays an icon such as a Settings menu icon to indicate that it is a settings menu, it needs an accessible text alternative such as "settings", which conveys the same information. Depending on context, this may use an alt attribute, an aria-label attribute, an aria-labelledby attribute, or plain text in the Shadow DOM. You can find general technical tips in WebAIM Quick Reference.

Any UI component which displays an image should provide a mechanism for providing alternative text for that image, analogous to the alt attribute.

Do your components provide semantic information?

Assistive technology conveys semantic information which is otherwise expressed to sighted users via visual cues such as formatting, cursor style, or position. Native elements have this semantic information built-in by the browser, but for custom components you need to use ARIA to add this information in.

As a rule of thumb, any component which listens to a mouse click or hover event should not only have some kind of keyboard event listener, but also an ARIA role and potentially ARIA states and attributes.

For example, a custom <fancy-slider> UI component might take an ARIA role of slider, which has some related ARIA attributes: aria-valuenow, aria-valuemin and aria-valuemax. By binding these attributes to the relevant properties on your custom component, you can allow users of assistive technology to interact with the element and change its value, and even cause the visual presentation of the element to change accordingly.

A range slider component

<fancy-slider role="slider" aria-valuemin="1" aria-valuemax="5" 
aria-valuenow="2.5"></fancy-slider>

Can users understand everything without relying on color?

Color shouldn’t be used as the only means of conveying information, such as indicating a status, prompting for a response or distinguishing a visual custom component. For example, if you created an <fancy-map> component using color to distinguish between heavy, moderate and light traffic, an alternative means of distinguishing traffic levels should also be made available: one solution might be to hover over an element to display information in a tooltip.

Is there sufficient contrast between the text/images and the background?

Any text content displayed in your component should meet the minimum (AA) contrast bar. Consider providing a high-contrast theme which meets the higher (AAA) bar, and also ensure that user agent style sheets can be applied if users require extreme contrast or different colors. You can use this Color Contrast Checker as an aid when doing design.

Is the moving or flashing content in your components stoppable and safe?

Content that moves, scrolls or blinks that lasts for anything more than 5 seconds should be able to be paused, stopped or hidden. In general, try to flash no more than three times per second.

Accessibility Tooling

A number of tools are available that can assist with debugging the accessibility of your visual components.

Axe provides automated accessibility testing for your framework or browser of choice. Axe Puppeteer can be used for writing automated accessibility tests.
The Lighthouse Accessibility audits provide helpful insights for discovering common accessibility issues. The accessibility score is a weighted average of all accessibility audits, based on Axe user impact assessments. For monitoring accessibility via continuous integration, see Lighthouse CI.

Tenon.io is useful for testing common accessibility problems. Tenon has strong integration support across build tools, browsers (via extensions) and even text editors.
There are many library and framework specific tools for highlighting accessibility issues with components. e.g web.dev covers using eslint-plugin-jsx-a11y to highlight accessibility issues for React components in your editor:

If you use Angular, codelyzer provides in-editor accessibility audits too:

You can examine the way that assistive technologies see web content by using Accessibility Inspector (Mac), or Windows Automation API Testing Tools and AccProbe (Windows). Additionally you can see the full accessibility tree that Chrome creates by navigating to chrome://accessibility.
The best way to test for screen reader support on a Mac is using the VoiceOver utility. You can use ⌘F5 to enable/disable, Ctrl+Option ←→ to move through the page and Ctrl+Shift+Option + ↑↓ to move up/down tree. For more detailed instructions, see the full list of VoiceOver commands and the list of VoiceOver Web commands.
tota11y is a useful visualiser for assistive technology issues built by Khan Academy. It’s a script that adds a button to your document that triggers several plugins for annotating things like insufficient contrast ratio and other a11y violations.
On Windows, NVDA is a free, open source screen reader which is fully featured and rapidly gaining in popularity. However, note that it has a much steeper learning curve for sighted users than VoiceOver.
ChromeLens helps develop for the visually impaired. It’s also got great support for visualising keyboard navigation paths http://chromelens.xyz/

ChromeVox is a screen reader which is available as a Chrome extension, and built in on ChromeOS devices.

Conclusions

We still have a long way to go improving accessibility on the web. Per the Web Almanac:

4 out of every 5 sites have text which easily blends into the background, making it unreadable.
49.91% of pages still fail to provide alt attributes for some of their images
Only 24% of pages that use buttons or links include textual labels with these controls.
Only 22.33% of pages provide labels for all their form inputs.

To learn more about accessibility fundamentals to help us improve the above, I recommend the Accessible to all docs on web.dev. There's much we can do to build experiences that are more inclusive of everyone.

Adaptive Loading - Improving Web Performance on low-end devices

Addy Osmani — Mon, 18 Nov 2019 19:56:47 +0000

Adaptive Loading: Do not just respond based on screen size, adapt based on actual device hardware.

Any user can have a slow experience. In a world with widely varying device capabilities, a "one-size" fits all experience may not always work. Sites that delight users on high-end devices can be unusable on low-end ones, particularly on median mobile and desktop hardware and in emerging markets. What if we could adapt how we deliver pages to better cater for our user's constraints? 🤔

In our Chrome Dev Summit talk, Facebook's Nate Schloss and I talk about Adaptive Loading - this pattern ...

Delivers a fast core experience to all users (including low-end devices)
Progressively adds high-end-only features, if a user's network and hardware can handle it.

This allows users to get an experience best suited to their constraints.

The use-cases adaptive loading unlocks include:

Serving low-quality images & videos on slow networks
Loading non-critical JavaScript for interactivity only on fast CPUs
Throttling the frame-rate of Animations on low-end devices
Avoiding computationally heavy operations on low-end devices
Block 3rd-party scripts on slower devices

There are a number of signals we can use for Adaptive Loading, including:

Network - for fine-tuning data transfer to use less bandwidth (via navigator.connection.effectiveType). We can also leverage the user's Data Saver preferences (via navigator.connection.saveData)
Memory - for reducing memory consumption on low-end devices (via navigator.deviceMemory)
CPU core count - for limiting costly JavaScript execution and reducing CPU intensive logic when a device can't handle it well (via navigator.hardwareConcurrency). This is because JavaScript execution is CPU bound.

In our talk, we cover real-world examples of these ideas being used in sites such as Facebook, eBay, Tinder and others. Check out 24mins in where Nate walks through how Facebook uses some of these ideas in production, via device grouping:

We also released a new (experimental) set of React Hooks & Utilities for adding adaptive-loading techniques to your React apps.

GoogleChromeLabs / react-adaptive-hooks

Deliver experiences best suited to a user's device and network constraints

React Adaptive Loading Hooks & Utilities ·

Deliver experiences best suited to a user's device and network constraints (experimental)

This is a suite of React Hooks and utilities for adaptive loading based on a user's:

It can be used to add patterns for adaptive resource loading, data-fetching, code-splitting and capability toggling.

Objective

Make it easier to target low-end devices while progressively adding high-end-only features on top. Using these hooks and utilities can help you give users a great experience best suited to their device and network constraints.

Installation

npm i react-adaptive-hooks --save or yarn add react-adaptive-hooks

Usage

You can import the hooks you wish to use as follows:

import { useNetworkStatus } from 'react-adaptive-hooks/network';
import { useSaveData } from 'react-adaptive-hooks/save-data';
import { useHardwareConcurrency } from 'react-adaptive-hooks/hardware-concurrency';
import { useMemoryStatus } from

…

View on GitHub

Our hooks/utilities include the useNetworkStatus React hook for adapting based on network status (effective connection type):

import React from 'react';

import { useNetworkStatus } from 'react-adaptive-hooks/network';

const MyComponent = () => {
  const { effectiveConnectionType } = useNetworkStatus();

  let media;
  switch(effectiveConnectionType) {
    case '2g':
      media = <img src='medium-res.jpg'/>;
      break;
    case '3g':
      media = <img src='high-res.jpg'/>;
      break;
    case '4g':
      media = <video muted controls>...</video>;
      break;
    default:
      media = <video muted controls>...</video>;
      break;
  }

  return <div>{media}</div>;
};

The useSaveData utility for adapting based on the user's browser Data Saver preferences:

import React from 'react';

import { useSaveData } from 'react-adaptive-hooks/save-data';

const MyComponent = () => {
  const { saveData } = useSaveData();
  return (
    <div>
      { saveData ? <img src='...' /> : <video muted controls>...</video> }
    </div>
  );
};

...and the useHardwareConcurrency utility for adapting based on the number of logical CPU processor cores on the user's device:

import React from 'react';

import { useHardwareConcurrency } from 'react-adaptive-hooks/hardware-concurrency';

const MyComponent = () => {
  const { numberOfLogicalProcessors } = useHardwareConcurrency();
  return (
    <div>
      { numberOfLogicalProcessors <= 4 ? <img src='...' /> : <video muted controls>...</video> }
    </div>
  );
};

In the future, we hope to see more examples of infrastructure that can automatically deliver the most optimal bundles of code based on a user's network & device constraints. Between Client Hints and the client-side APIs used above, the building blocks to build something compelling in this space may already exist :)

We hope you'll find Adaptive Loading an interesting addition to your Progressive Enhancement toolbox. Check out our talk to learn more :)

React Adaptive Loading hooks and utilities
Sophisticated Adaptive Loading Strategies from Tinder Web engineering
Connection-Aware Components in Angular
Serving Adaptive Components Using the Network Information API with Vue and Web Components
Adaptive serving based on network quality

Web Page Usability Matters

Addy Osmani — Thu, 10 Jan 2019 17:44:23 +0000

Users appreciate pages being usable and interactive soon after they're visually ready. UI interactions (scrolls, taps, clicks) can be delayed by script and other browser work so minimizing their impact can really help your users.

You may have heard there isn't a single metric that fully captures the "loading experience" of a web page. Loading a page is a progressive journey with four key moments to it: is it happening? Is it useful? is it usable? and is it delightful?.

In terms of measurable metrics, these moments breakdown as follows:

Is it happening?: Has the navigation started successfully? has the server started responding? Metric: First Paint
Is it useful?: when you’ve painted text, an image or content that allows the user to derive value from the experience and engage with it. Metrics: First Contentful Paint, First Meaningful Paint, Speed Index (lab)
Is it usable?: when a user can start meaningfully interacting with the experience and have something happen (e.g tapping on a button). This can be critical as users can get disappointed if they try using UI that looks ready but isn't. Metrics: Time to Interactive (lab), First CPU Idle, First Input Delay (field)
Is it delightful?: delightfulness is about ensuring performance of the user experience remains consistent after page load. Can you smoothly scroll without janking? are animations smooth and running at 60fps? do other Long Tasks block any of these from happening?.

Page usability matters. Single-page Apps are unlikely to be usable (able to quickly respond to user input) if they haven't completed loading the JavaScript needed to attach event handlers for the experience or are hogging the browser's main thread (similar to above). This is a reason why monitoring usability metrics can be important.

Do users care about page usability?

In 2018, Akamai conducted a UX study into the impact of interactivity on Rage Clicks using mPulse. Rage Clicks happen when users rapid-click (or tap) on a site out of frustration. Akamai discovered that Rage Click likelihood depends on the latency of page usability:

Rage Clicks are consistent if a user's first interaction is before the page becomes interactive (before interactive or onload). This may be because event handlers are hogging CPU.
In 30%+ of cases, pages were interactive after onload fired and in 15%, users tried interacting sometime between onload and interactive.

What is the optimum time to reduce Rage Clicking?

Akamai observed:

The majority of users who Rage Click attempt to interact with a page between 1.25 and 1.5x the visually ready time.
They suggested making sure your page is interactive and loaded before 1.3x the visually ready time.

For more on this study, see UX and performance metrics that matter by Philip Tellis.

Time to Interactive can have a high correlation with overall conversion rate

In a 2017 study, Akamai and Chrome found that across three real-world sites (in retail, travel and gaming) Time to Interactive had a high correlation with overall conversion rate. Conversions can be tapping a button to complete a purchase flow or any number of other types of responses to an interaction.

They discovered:

Long Tasks directly delayed Time to Interactive.
As first-page Long Task time increased, overall conversion rates decreased.
Mobile devices could see a 12x Long Task time vs Desktop.
Older devices could be spending half of their load-time on Long Tasks.

Note: This is an obviously small sample size and every site can be different.

The phases of loading in more detail

Is it happening? Is it useful?

When the user gets timely feedback that "It is happening" they feel much better, and they perceive the site as faster. At the same time, you don't want a user to render a page which is "useful" but which they cannot interact with because it isn't yet ready. This would leave them feeling it's not usable.

First Paint

First Paint marks when the browser can render anything visually different from what was displayed before navigation. It confirms that rendering has started. This can be an important metric, as the duration of ‘blank screen’ is probably the most significant indicator of page abandonment.

First Contentful Paint

First Contentful Paint is when the browser renders the first content from the DOM - this could be article text, an image or SVG. The hope is this paint communicates a navigation successfully began.

Is it usable?

As I've covered in The Cost Of JavaScript, the web has a problem. Many sites are optimizing for content visibility but ignoring interactivity as their JavaScript for accomplishing this takes time to be processed. This means large numbers of very popular sites have multi-second delays between having painted something useful and being "usable" or interactive. During this time, the web feels slow and unresponsive.

Time to Interactive and First Input Delay

Time to Interactive (TTI) is a metric that measures how long it takes for a web page to become interactive. This is defined as a point when:

The page has displayed useful content
Event handlers are registered for most visible page elements
When a user interacts with the page, the page consistently responds within 50ms - the user does not experience jank.

When a page has a great TTI, a user can tap around the interface with high confidence it will respond to input. This strictly meets the Idle guideline of the RAIL performance model: the page yields control back to the main thread at least once every 50ms. The network is idle. Specifically, there are only two open network requests remaining.

First Input Delay (FID) is TTI's complimentary metric in the field - it measures the time from a user first interacting with a page (e.g. tapping a button) to when the browser can actually respond to the interaction.

Optimizing user-centric metrics

A focus on optimizing user-centric metrics will ultimately improve the user experience. If you would like to reduce...

Time to Interactive or First Input Delay:

Do less work
Split up large JavaScript bundles using code-splitting
Split up long tasks. Consider moving intensive off-main thread to workers.
Postpone non-critical work until after page load

First Paint and First Contentful Paint:

Remove render blocking scripts from head
Identify the critical CSS needed and inline in <head>
App Shell pattern - improve user perception rendering UI skeleton

Monitoring metrics

Performance tools like PageSpeed Insights, WPT and Lighthouse capture user-centric loading metrics:

For CLI users, they are also available via pwmetrics by Paul Irish and Artem Denysov.

For monitoring metrics in the field (via RUM), I recommend the Paint Timing API which provides First Paint and First Contentful Paint. A polyfill for First Input Delay is also available. Paired with an analytics service these enable logging progressive web metrics for your real users.

The Chrome User Experience Report (also in PageSpeed Insights) provides access to some of these metrics (like First Contentful Paint and First Input Delay) for your real users using Chrome:

I also like SpeedCurve or Calibre for tracking metrics like FCP and TTI over time. It allows me to set performance budgets for them too which can help detect regressions:

Debugging metrics

Chrome DevTools now highlights performance metrics in the Performance panel. These can be found under Timings:

In addition to other surfaces these metrics are exposed, this can provide value while attempting to improve times in your iteration workflow.

References and learn more

Speed up next-page navigations with prefetching

Addy Osmani — Thu, 13 Dec 2018 01:40:06 +0000

What is prefetching?

Loading web pages could be made faster by fetching the next page (or set of resources for the next page) a user is likely to visit ahead of time. We call this prefetching. This can make subsequent navigations appear to load instantly in some cases.

Modern prefetching can be achieved in a number of ways. Most commonly this is done using the browser’s <link rel=prefetch> Resource Hint, which works declaratively via HTML..

<!-- HTML -->
<link rel="prefetch" href="/pages/next-page.html">
<link rel="prefetch" href="/js/chat-widget.js">

or a HTTP Header:

Link: </js/chat-widget.js>; rel=prefetch

Prefetching is supported in Webpack too (see prefetch/preload in Webpack).

<!-- Without prefetching -->
import("ChatWidget");

<!-- With prefetching -->
import(/* webpackPrefetch: true */ "ChatWidget");

Webpack's runtime will inject the correct prefetch statements in the page once the parent chunk has finished loading. In the above case, <link rel="prefetch" href="chat-widget.js">.

Per CanIUse, <link rel=prefetch> has decent cross-browser support.

If a resource in Chrome is prefetched with <link rel=prefetch>, it will be fetched at a low priority and kept around for 5 minutes. This lasts until the resource has been consumed, at which point the normal cache-control rules for the resource will apply.

Unsure what pages on your site may be worth considering prefetching? Check out our experimental helper tool for suggested "top next pages" (based on your analytics):

"Prefetching" can also mostly be accomplished using Service Workers [see prefetching during registration]/fetch() or XHR. Note however, not all of these mechanisms have the same "low priority" fetch behavior as <link rel=prefetch>.

Who is using prefetching today?

Several brands you've probably heard of use prefetching in production:

Netflix prefetch JavaScript bundles needed for subsequent navigations ahead of time
IndieGogo prefetch Stripe's JavaScript library for credit-card processing on future pages
Craigslist prefetch their JS bundle for search results pages
Heineken prefetch JS and CSS bundles pages after the date-of-birth verification page may need.

What are the challenges of prefetching?

When a user is on a constrained network connection or a limited data-plan, every last byte counts. While prefetching a number of pages ahead of time can make sense for users on a great WiFi connection, it requires care for those that are not. You don't want to waste anyone's data plan.

You can use the navigator.connection.effectiveType API (part of NetInfo) to only prefetch pages when a user is on a connection that is effectively 4G.

One thing to also be careful about with prefetch is that if requests aren't finished by the time a navigation is initiated, the in-flight prefetch request is cancelled. This can mean wasted bytes so just be sure a user is likely to actually see value from these fetched when using this capability.

Case study

When Japanese publisher Nikkei were confident users would navigate to specific pages, they didn't wait for a navigation to happen.

They leveraged <link rel=prefetch> to prefetch the next page before users tapped on links. Below we can see the impact of waiting for network + server overhead to deliver a page (taking a total of ~880ms). Compare this to prefetching, with almost no request overhead and an immediate page navigation (~37ms).

More on Nikkei can be found in this recent case study.

What is predictive prefetching?

Sites using prefetching typically have a good idea of what top-level pages or resources a user is going to visit and can manually add <link rel=prefetch> tags for them. This manual process can however become harder to achieve for large sites at scale. Once you go several levels deep, it can be difficult to know what the "best" next page is going to be nor how to wire this up without it being a manual process.

Predictive prefetching uses additional data (e.g analytics) to predict what documents users are likely to visit given any URL on a site. This data can then be used to <link rel=prefetch> those pages, improving subsequent page load times. Predictions can be optionally improved using machine learning to ensure only pages with the highest changes of being accessed are fetched ahead of time.

Although there’s nothing here specific to it, Google Analytics has a Reporting API with just enough information to build out predictive prefetching for sites. Minko Gechev, Katie Hempenius and I took advantage of this recently to build a project called Guess.js, which has solutions for 1) predictively prefetching relevant pages and 2) predictively bundling and prefetching the right JavaScript resources that may be needed for those pages (via webpack).

Where can I learn more about predictive prefetching?

Rick Viscomi chatted with Katie Hempenius and I about this topic in depth in case interested in learning more:

We're going to continue exploring opportunities to evolve prefetching on the web. Most recently, we've explored privacy-preserving preserving prefetching using Signed Exchanges, a subset of the emerging web packaging specification.

PS: Keep an eye out for quicklink - a <1KB library I'm releasing that aims to enable faster next-page navigations by prefetching in-viewport links during idle time :)

With thanks to Yoav Weiss for his helpful input on how <link rel=prefetch> works behind the scenes in Chrome

The State Of JavaScript

Addy Osmani — Sat, 27 Oct 2018 02:44:54 +0000

I recently talked about the State Of JavaScript on the State Of The Web show. Yes, I'm a one-trick pony... 😃

On the client, as you pay for things you can't always easily see it becomes important to understand how the cost of JavaScript might impact your site.

How do I tackle these problems?

Start off with Lighthouse 💡🏠
Check how well your site performs on some real phones. There are a number of these setup on WebPageTest. If your team can afford them, a local device lab is always useful too 📱
If JavaScript is impacting your metrics, it's probably time to analyze why. I regularly use source-map-explorer, webpack-bundle-analyzer and bundlephobia to discover opportunities to trim down my bundles.
I use the DevTools code-coverage highlights bundle weight-loss opportunities.
Try to leverage code-splitting patterns. Got a large vendor bundle? Maybe only a few of those dependencies are needed for the initial page load. Large app bundle? Is all that logic really needed right away? Libraries like React are starting to get better at supporting code-splitting out of the box (see React.lazy()).
Consider using babel-preset-env to generate bundles for modern vs legacy browsers. The module/nomodule pattern allows you to serve these in a backwards compatible way.
Leverage compression techniques. I've been suggesting teams try out Brotli for static asset compression. In my own apps, I've seen a 20-30% decrease in bundle size over the wire for JS. If you're a Node shop, shrink-ray or shrink-ray-current are solid middlewares for serving Brotli that I've found work reliably.

Shine a light on JavaScript performance with Lighthouse

Addy Osmani — Thu, 18 Oct 2018 18:41:20 +0000

Unsure if the cost of JavaScript is too high for your user-experience? 🙃 Lighthouse has a JavaScript execution time audit that measures the total impact of JavaScript on your page's load performance:

Try it. It's in the Chrome DevTools Audits panel today. It's also available via WebPageTest.

For the above content site, it takes 51s (oi vey) for the browser to process just the primary bundle for this site on mobile. Including network transfer time, a user could be waiting for up to a minute to interact with this page ⏳😪

That's time spent parsing, compiling and executing script on a median mobile device configuration. dev.to (offering a similar content experience) is able to load their main bundle with a minimal dependency on script execution ❤️

How can we improve the cost of JS in the original site?

By only shipping JavaScript the user really needs upfront. We can lazily load the rest as needed using techniques like code-splitting. I use the DevTools Code Coverage feature to help here.

If I hit record and load up the above experience then interact for a while, we can see about 57% of the code loaded upfront may not be needed. That's a great candidate for something that can be loaded on-demand.

If you haven't checked out Lighthouse before, it's full of useful nuggets like checks for whether you're correctly minifying your scripts or compressing them:

And if you're into automation using headless Chrome, there's also a useful code-coverage example for Puppeteer that can visualize JS code coverage usage across page loads.

Wrapping up.. 🎁

JavaScript can have a large impact on your user-experience; Lighthouse can highlight opportunities to improve here. To keep JavaScript transmission and processing times low:

Only send the code that your users need.
Minify and compress your scripts.
Remove unused code and dependencies.

Adaptive Serving using JavaScript and the Network Information API

Addy Osmani — Mon, 08 Oct 2018 15:46:46 +0000

navigator.connection.effectiveType is useful for delivering different assets based on the quality of the user's network connection.

effectiveType is a property of the Network Information API, exposed to JavaScript via the navigator.connection object. In Chrome, you can drop the following into DevTools to see your effective connection type (ECT):

console.log(navigator.connection.effectiveType); // 4G

Possible values for effectiveType are 'slow-2g', '2g', '3g', or '4g'. On slow connections this capability allows you to improve how quickly pages load by serving lower-quality versions of resources.

Before Chrome 62, we only exposed the theoretical network connection type to developers (via navigator.connection.type) rather than the network quality actually experienced by the client.

Chrome's implementation of effective connection type is now determined using a combination of recently observed round-trip times (rtt) and downlink values.

It summarizes measured network performance as the cellular connection type (e.g. 2G) most similar, even if the actual connection is WiFi. i.e. picture you're on Starbucks WiFi, but your actual effective network type is 2G or 3G.

What about responding to changes in network quality? We can use the connection.onchange event listener to monitor for connection changes:


function onConnectionChange() {
    const { rtt, downlink, effectiveType,  saveData } = navigator.connection;

    console.log(`Effective network connection type: ${effectiveType}`);
    console.log(`Downlink Speed/bandwidth estimate: ${downlink}Mb/s`);
    console.log(`Round-trip time estimate: ${rtt}ms`);
    console.log(`Data-saver mode on/requested: ${saveData}`);
}

navigator.connection.addEventListener('change', onConnectionChange)

Below is a quick test where I emulated a "Low-end mobile" profile in DevTools and was able to switch from "4g" to "2g" conditions:

effectiveType is supported in Chrome, Opera and Firefox on Android. A number of other network quality hints are available on navigator.connection, including rtt, downlink and downlinkMax.

An open-source project I've used effectiveType in was a Vue.js Google Doodles app. Using data-binding, we were able to set a connection property to either fast or slow based on ECT values. Roughly:

if (/\slow-2g|2g|3g/.test(navigator.connection.effectiveType)) {
  this.connection = "slow";
} else {
  this.connection = "fast";
}

This allowed us to conditionally render different output (a video vs. a low-res image) depending on the user's effective connection type.

   <template>
      <div id="home">
        <div v-if="connection === 'fast'">
          <!-- 1.3MB video -->
          <video class="theatre" autoplay muted playsinline control>
            <source src="/static/img/doodle-theatre.webm" type="video/webm">
            <source src="/static/img/doodle-theatre.mp4" type="video/mp4">
          </video>
        </div>
        <!-- 28KB image -->
        <div v-if="connection === 'slow'">
          <img class="theatre" src="/static/img/doodle-theatre-poster.jpg">
        </div>
      </div>
   </template>

Max Böck wrote an interesting article about network-aware components using React. He similarly highlighted how to render different components based on the network speed:

        switch(connectionType) {
            case '4g':
                return <Video src={videoSrc} />

            case '3g':
                return <Image src={imageSrc.hires} alt={alt} />

            default:
                return <Image src={imageSrc.lowres} alt={alt} />
        }

Note: You can pair effectiveType with Service Workers to adapt to when users are offline in addition to slower effective connection types.

For debugging, you can override the network quality estimate using the Chrome flag "force-effective-connection-type" which can be set from chrome://flags. DevTools Network emulation can provide a limited debugging experience for ECT too.

effectiveType values are also exposed via Client Hints allowing developers to convey Chrome's network connection speed to servers.

I'm Addy Osmani, Ask Me Anything!

Addy Osmani — Wed, 11 Jul 2018 16:43:21 +0000

My name is Addy and I’m an engineering lead on the Chrome team at Google leading Developer Experience. We do this through Chrome developer tools, web standards, working with sites & frameworks in the ecosystem and through tools like Lighthouse and Workbox. I give talks about speed and have written books like Learning JavaScript Design Patterns and Essential Image Optimization. I am excited about Developer Experience and how personal software via AI might democratize software development.

To learn more, you can find me on Twitter at @addyosmani