Forem: FlareCanary

GitHub Just Retired Seven Org Security Fields — Your 'New Repo Hardening' Script Is Now A No-Op

FlareCanary — Tue, 28 Apr 2026 04:01:30 +0000

GitHub announced on April 21 that seven security-defaults fields on the org REST endpoint are now retired:

advanced_security_enabled_for_new_repositories
dependabot_alerts_enabled_for_new_repositories
dependabot_security_updates_enabled_for_new_repositories
dependency_graph_enabled_for_new_repositories
secret_scanning_enabled_for_new_repositories
secret_scanning_push_protection_enabled_for_new_repositories
secret_scanning_push_protection_custom_link_enabled

These appeared on GET /orgs/{org} and were writable on PATCH /orgs/{org}. The retirement isn't a 410 or a schema-validation error. The fields just stop appearing on reads. The PATCH still returns 200 OK whether you send them or not — it just no longer applies them.

Replacement: Code Security Configurations API (/orgs/{org}/code-security/configurations). Different endpoint, different shape, different concept — configurations are objects you attach to repositories, not booleans on the org.

This is the deepest silent-fail vector we've seen all month. It's not a billing column or a string field. It's the one-line setup script every platform-engineering team wrote to harden new repos by default.

The script that quietly stopped working

The classic version:

await octokit.orgs.update({
  org: 'acme-platform',
  advanced_security_enabled_for_new_repositories: true,
  dependabot_alerts_enabled_for_new_repositories: true,
  secret_scanning_enabled_for_new_repositories: true,
  secret_scanning_push_protection_enabled_for_new_repositories: true,
});

Pre-2026-04-21: that PATCH applied org-wide defaults. Every new repo created in acme-platform after this call had Dependabot, advanced security, and secret scanning on at creation.

Post-retirement: that PATCH returns 200. Octokit doesn't throw. The audit log records the call. The fields are simply ignored.

A new repo created tomorrow has none of those defaults applied. The compliance script ran. The dashboard says "secret scanning enabled org-wide." A git push with an AWS access key in it goes through unchallenged.

The terraform-github-provider equivalent (github_organization_settings) wraps the same fields and has the same behavior. terraform apply says no changes. The state file thinks it's converged. The org isn't enforcing what the state file claims.

Why reads break differently from writes

GitHub's REST API doesn't versioned-deprecate fields the way it does for merge_commit_sha. There's no X-GitHub-Api-Version: 2022-11-28 you can set to keep the old field alive. The fields are just gone from GET /orgs/{org} everywhere now.

Reading code that does this:

const { data: org } = await octokit.orgs.get({ org });
if (org.secret_scanning_enabled_for_new_repositories) { /* ... */ }

…now hits undefined. JavaScript's truthiness rules turn that into the false branch. Compliance dashboards that read these fields to display "✓ Secret scanning on" suddenly read "✗" and a security engineer pages someone, OR — worse — the dashboard caches the last known value from a week ago and keeps showing green for months.

Writing code is the more dangerous shape. The PATCH endpoint accepts arbitrary unknown keys without erroring. There's no "did this field actually apply" affordance in the response. Your script can include 47 retired-or-renamed properties and the API still returns 200 with the org object that has none of them on it. Comparing the response back is the only check, and almost nobody writes that compare.

The replacement is not a one-line swap

The Code Security Configurations API is conceptually different. Instead of:

PATCH /orgs/{org}  { secret_scanning_enabled_for_new_repositories: true }

…you now:

POST /orgs/{org}/code-security/configurations to create a configuration object (with secret scanning, Dependabot, advanced security, push protection settings as nested objects).
POST /orgs/{org}/code-security/configurations/{config_id}/defaults to attach it as the default for new repos. The body specifies which repo types it applies to (new_repos, private_repos, public_repos, all).
Optionally, POST /orgs/{org}/code-security/configurations/{config_id}/attach to retroactively apply it to existing repos.

The shape of the configuration object isn't a 1:1 mapping. advanced_security becomes part of a nested struct. secret_scanning_push_protection_custom_link becomes a sub-field of secret scanning. Some legacy combinations aren't expressible. Some new combinations are.

Migration is a real engineering task, not a search-and-replace. And the deprecation notice doesn't have a hard cutoff date — the fields are listed as "Retired" but reads have already broken, so there's no grace window left.

Why your tests didn't catch it

The pattern by now is familiar. Three things have to be true to catch this in CI:

Your fixture for GET /orgs/{org} was regenerated against the live API after April 21 (it wasn't).
Your test asserts presence (expect(org).toHaveProperty('secret_scanning_enabled_for_new_repositories')) rather than truthiness on a possibly-undefined field (most don't).
Your terraform plan/apply tests run against a live GitHub org and diff the result, not against a mocked provider (almost nobody does this).

Most platform teams test their org-hardening script by spinning up a sandbox org once, validating the fields land, and never re-validating. The sandbox org's settings still look right because they were set before the retirement. The script that "still works" hasn't actually applied anything for any new repo created after April 21.

#	API	What changed	Where tests missed it
1	GitHub PushEvent	`commits` field silently dropped	Tests didn't assert field presence
2	Stripe Basil	`current_period_end` moved to `items`	Tests used Checkout fixtures
3	Shopify 2025-01	`fulfillmentHold` type change	Tests mocked the response
4	OpenAI Responses	`input_text` removed for assistants	Tests covered request role=user
5	Twilio regional	Regional domains stop resolving	Tests don't hit prod DNS paths
6	HubSpot Contacts v1	`list-memberships` returns empty	Tests asserted against sandbox fixtures
7	GitHub merge_commit_sha	Field removed from PR responses	Tests used pre-rollout fixtures
8	GitHub org security fields	7 fields retired from /orgs/{org}	Org-hardening script tested once, never re-run

Eight in a row. The breaking change is always in a field a test isn't asserting against, in a layer (transitive SDK upgrade, version-pinned header, configuration drift) the test isn't exercising, or in a script that runs once during onboarding and is never re-validated.

What to do this week

Three actions, in priority order:

Grep your platform-eng repos for the seven field names. Anywhere they appear in a PATCH body or a read assertion is a candidate failure. Org-hardening Octokit calls, Terraform configs, compliance dashboards, audit-log queries — all suspect.
Audit any new repository created in your org after April 21, 2026. Check whether secret scanning, Dependabot alerts, and advanced security are actually enabled on each one. If your org-default script broke silently, every new repo since then has weaker security than the dashboard claims.
Migrate to Code Security Configurations. Create the configuration, attach it as a default for new repos, and — separately — attach it retroactively to repos created during the silent-fail window. The retroactive attach is the cleanup step most migrations forget.

The PATCH that returns 200 OK and does nothing is the worst kind of API regression. There's no error to alert on. The audit log shows you ran the script. Compliance reports green. And every new repo in your org for the past week has its security defaults in whatever state GitHub decided to leave them in — which, based on testing, is "not what your script set."

We built FlareCanary for exactly this layer. Snapshot the response shape on a schedule, diff it, alert when fields disappear or accept-but-ignore semantics shift. The GitHub org security retirement is the textbook case: no error, no version header, no migration warning, just seven fields that used to enforce things and now silently don't.

If your platform team's onboarding script touched any of these seven org-level fields and hasn't been audited since April 21, your org is shipping new repos with whatever default security GitHub decided on — not what your runbook says. Worth a Tuesday afternoon.

GitHub Just Removed merge_commit_sha From Pull Request Responses — Your Release Bot Is Probably Tagging null

FlareCanary — Mon, 27 Apr 2026 04:02:24 +0000

GitHub's 2026-03-10 REST API version ships a quiet but consequential breaking change: the merge_commit_sha property is gone from pull request responses. 21 endpoints affected. There is no error, no 410, no migration warning header — the field just stops appearing in the JSON.

For CI/CD code that does this:

const { data: pr } = await octokit.pulls.get({ owner, repo, pull_number });
await tagDeploymentArtifact(pr.merge_commit_sha);

You now tag with undefined. The deployment still ships. The artifact registry still accepts the upload. Six weeks later, somebody asks "which commit produced this build?" and the answer is undefined-1747291204.

This is incident #7 in our silent-breakage series (GitHub PushEvent, Stripe Basil, Shopify 2025-01, OpenAI Responses, Twilio regional, HubSpot Contacts v1). The pattern keeps repeating because removing a field from a JSON response is the cheapest possible breaking change for the API provider and the most invisible possible breaking change for the consumer.

What's removed and where

merge_commit_sha disappears from 21 PR-bearing endpoints, including:

GET /repos/{owner}/{repo}/pulls
GET /repos/{owner}/{repo}/pulls/{pull_number}
POST /repos/{owner}/{repo}/pulls
PATCH /repos/{owner}/{repo}/pulls/{pull_number}
GET /repos/{owner}/{repo}/issues/events
GET /repos/{owner}/{repo}/issues/{issue_number}/events
Search results that embed PR objects
Project card payloads that link to PRs

Same release also removes the singular assignee field from 31 Issue and PR endpoints. From the changelog:

The singular assignee field has been marked as 'closing down' for years and duplicates information available in the assignees array.

True — and the migration is mechanical. Read from assignees[0] instead of assignee. Write assignees: [login] instead of assignee: login. The footgun is that assignee returning undefined looks identical to "this PR has no assignee," and a lot of routing logic gates on exactly that.

Why you don't get an error

The REST API breaking-change rollout works by version pinning. If you send:

X-GitHub-Api-Version: 2022-11-28

…you keep the old field. If you send:

X-GitHub-Api-Version: 2026-03-10

…or no version header at all and your client SDK has updated its default — the field is gone.

Octokit, PyGithub, go-github, and the various community SDKs upgrade their defaults on their own schedules. Your package.json says ^21.0.0, the maintainer ships 21.4.7 next month with the new default version header, your npm install picks it up on the next CI run, and now your release bot is tagging undefined. Nothing in your repo changed. Nothing in the API changed for clients still pinning the old version. The change rides in on a transitive bump.

The non-obvious replacement

The official guidance points to the pull_request.merge_commit_sha field on the webhook payload (not the API response) and to the merge commit's SHA reachable via:

GET /repos/{owner}/{repo}/commits/{ref}
# where ref is the head of the default branch immediately after merge

Neither is a one-line swap.

Webhook payloads still carry merge_commit_sha because GitHub versions webhooks separately from the REST API. If your release bot is webhook-driven, you're fine — the field is in the pull_request.closed event payload. If your release bot is poll-driven (CI runs nightly, asks "what merged today, tag those artifacts"), you have to reconstruct the merge commit by:

Reading the PR's base.ref (the target branch)
Pulling commit history on that branch
Finding the merge commit by the PR number in the commit message (Merge pull request #N)
Or — for squash/rebase merges — there is no single merge SHA, and you have to use head.sha of the PR plus tracking metadata your bot wrote earlier

The squash-merge case is the one most release tooling gets wrong on the rewrite. There never was a merge_commit_sha for squash merges in the strict sense — GitHub returned the SHA of the squashed-in commit on the base branch — but consumers treated it as canonical. Without that field, the PR head SHA and the resulting commit on main are different SHAs with no GitHub-provided link between them.

Why your tests didn't catch it

Your CI pipeline tests hit a fixture PR JSON. The fixture has merge_commit_sha. The test asserts the tag string is well-formed.

Three things have to be true for that test to fail before the rollout:

Your fixtures were regenerated against the new API version (they weren't)
Your test runner sends the new X-GitHub-Api-Version header (it doesn't, unless you wrote it)
Your assertion checks typeof sha === 'string' && sha.length === 40 and not just sha != null (most don't)

So the test stays green. The same pattern as every other silent drift incident:

#	API	What changed	Where tests missed it
1	GitHub PushEvent	`commits` field silently dropped	Tests didn't assert field presence
2	Stripe Basil	`current_period_end` moved to `items`	Tests used Checkout fixtures
3	Shopify 2025-01	`fulfillmentHold` type change	Tests mocked the response
4	OpenAI Responses	`input_text` removed for assistants	Tests covered request role=user
5	Twilio regional	Regional domains stop resolving	Tests don't hit prod DNS paths
6	HubSpot Contacts v1	`list-memberships` returns empty	Tests asserted against sandbox fixtures
7	GitHub merge_commit_sha	Field removed from PR responses	Tests used pre-rollout fixtures

Pattern holds. The breaking change is always in a field a test isn't asserting against — or in a layer (transitive SDK upgrade, version-pinned header) the test isn't exercising.

What to do this week

Three actions, in priority order:

Grep your codebase for merge_commit_sha. Every read site is a candidate failure. Tag every assignment to a deployment artifact, release name, or git tag as a critical path.
Grep for pr.assignee or issue.assignee (singular). Routing/notification code keyed on this field is the next-largest blast radius after release tagging.
Pin X-GitHub-Api-Version: 2022-11-28 as a stopgap if you can't fix all the read sites this week. GitHub supports old versions for ~24 months. This buys time, not a fix — schedule the migration before the version sunsets.

If your release bot is silently tagging undefined, you usually find out 90 days later when somebody is paged at 3 AM and can't trace the deploy. Worth fixing on a Tuesday afternoon.

We built FlareCanary for this layer: poll third-party APIs on a schedule, watch the response shape, page when a field stops appearing. The GitHub merge SHA removal is the textbook incident — no error, no warning, just a field that used to be there and isn't. APIs break this way constantly. We catch it before the 3 AM page.

We track API drift incidents in real time. If your release tooling reads merge_commit_sha and you haven't audited it for the 2026-03-10 API version, that null is already in your build pipeline somewhere.

Auth0 is about to start returning handshake_failure — how to tell if you're affected

FlareCanary — Sun, 26 Apr 2026 04:01:59 +0000

Auth0 has 14 cipher suites scheduled for removal at the end of H1 2026. If any of your clients — your backend, a reverse proxy, an older mobile binary — negotiates one of them today, your next Auth0 call after the cutoff will fail with handshake_failure and no further explanation.

There's no JSON error body. No HTTP status code. No entry in the Auth0 tenant log, because the connection never gets far enough to hit the application layer. The client just gets a TLS alert from the edge and a stack trace that points at your HTTP library, not at Auth0.

That's a bad failure mode to debug under pressure, so it's worth checking now.

The exact suites being removed

Auth0's support article lists 14 CBC-mode and RSA-keyed suites as deprecated:

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_GCM_SHA384
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_256_CBC_SHA256

Two flavors to notice: all CBC-mode AES suites regardless of key exchange, plus every static-RSA suite even when it's using GCM. GCM alone isn't enough — the RSA key exchange is the problem, because it provides no forward secrecy.

The Canadian region (CA-1) already removed these suites. If you have traffic split across regions and something works in US/EU but not CA, this is a strong lead.

What the failure looks like in your stack

The observable symptom depends on your HTTP client. The common shapes:

Node (undici / node-fetch):

Error: write EPROTO ... SSL alert number 40 ... handshake failure

Python (requests / urllib3):

SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure

Go (net/http):

remote error: tls: handshake failure

Java (Apache HttpClient):

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

None of those mention Auth0. None include a cipher name. None suggest a fix. A developer Googling the exact error string mostly lands on Stack Overflow answers about self-signed certificates and clock skew, which is the wrong diagnosis.

The actually-useful signal is which endpoint fails. If every call to *.auth0.com or your tenant's login. custom domain starts alerting at the same moment, TLS negotiation is the story.

How to check whether you're affected

The handshake is a client-side concern, so you check at the client. Three approaches, cheapest first.

1. Ask OpenSSL directly

Pick one of the deprecated suites and force the handshake against your tenant. If the handshake succeeds, that client can still reach Auth0 today, and will break at the cutoff:

openssl s_client \
  -connect YOUR-TENANT.auth0.com:443 \
  -tls1_2 \
  -cipher 'ECDHE-RSA-AES128-SHA' \
  < /dev/null 2>&1 | grep -E 'Cipher|Verify'

If you see Cipher : ECDHE-RSA-AES128-SHA in the output, your OpenSSL build negotiates a deprecated suite and nothing stopped it. Run the same probe from the environment your code actually runs in — a container, a serverless function, a legacy VM — not just your laptop.

2. Check what your runtime offers

The handshake is a negotiation between what your client offers and what Auth0 accepts. If your client's offered list is a superset of modern suites, you're fine; the removal only bites when the client has only deprecated options on the table.

Old Android devices, Java 8 builds without recent TLS patches, Alpine containers with stripped OpenSSL, and vendored binaries from 2019-era SDKs are the usual culprits.

# Node
node -e 'console.log(require("tls").DEFAULT_CIPHERS)'

# Python
python3 -c "import ssl; ctx=ssl.create_default_context(); print(ctx.get_ciphers())"

# Java
keytool -list -v -storepass changeit 2>&1 | head -40

Look for whether modern suites — TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, ECDHE-ECDSA-AES128-GCM-SHA256, ECDHE-RSA-AES128-GCM-SHA256 — are in the list. If they are, you keep working after the cutoff. If only the deprecated suites are present, you break.

3. Point staging at the Canadian region

Auth0's CA-1 region has already removed the weak suites. If you can temporarily route staging traffic at a Canadian tenant (or any service that's enforced the same restrictions), anything that fails there is what would have failed post-cutoff in your primary region. This is the closest thing to a dress rehearsal.

What to fix

For most callers the fix is a runtime upgrade, not a code change.

Node 18+, Python 3.10+, Go 1.17+, Java 11+ all ship with acceptable defaults. If you're on those, you're probably fine — verify with the probes above.
OpenSSL 1.1.1+ on Linux, LibreSSL on macOS 12+, or Schannel on Windows Server 2022+ are the modern baselines. Older TLS stacks need rebuilding or replacing.
Self-managed certificate deployments (Auth0's custom domain setup with your own cert) are the most exposed — you control the termination layer and therefore the negotiated cipher. Anything in front of that path (nginx, Envoy, a CDN) is also yours to audit.
Reverse proxies in your path must also negotiate modern suites. A proxy with TLS 1.2 and old suites hard-coded will break exactly the same way, even if the process behind it is on a current runtime.

If you can't upgrade — there's always a legacy binary someone can't rebuild — the fallback is to pin an explicit cipher list that includes at least one GCM or ChaCha20 suite both sides support. That buys you time, not permanence: anything RSA-keyed will lose forward secrecy as a floor requirement eventually.

The broader pattern

Vendor-side TLS changes are a specific case of the general schema-drift problem: the vendor updates their endpoint, your code doesn't change, and the contract between them silently shifts.

The ways this usually bites:

No status code to assert on. Integration tests that check response.status === 200 can't fire for a handshake that never completes.
Not in the vendor's tenant log. Auth0's dashboard shows nothing because the request never reached the application layer.
Not in your APM. Most APMs start the span at the HTTP request; a TLS-layer failure looks like a one-line network error at best.

The way to catch this before your users do is to make a small, boring background check against each third-party endpoint you depend on — not just that it returns 200, but that the connection itself is healthy from your runtime. A synthetic handshake from inside your infrastructure is cheap, runs on a schedule, and produces an alert that points at the right vendor instead of at a generic network error.

Minimum-viable fix for today

Run the OpenSSL probe above against your Auth0 tenant from every environment your code runs in — local, staging, prod, each serverless region, any legacy VM that still has outbound Auth0 traffic
Grep your Auth0 callers for any forced cipher list: git grep -E 'ciphers|ssl_ciphers|TLS_RSA_WITH|ECDHE_RSA_WITH_AES.*CBC'
For any binary you can't rebuild, confirm at least one modern GCM or ChaCha20 suite is in its offered list
Route staging traffic at the CA-1 region for a day and watch for handshake_failure alerts — this is the pre-cutoff preview
Add a scheduled synthetic check that connects to your Auth0 tenant from production and fails loudly on a TLS error, not just on HTTP non-2xx

If none of the above produce a finding, you're clear. If any of them do, you have roughly six weeks.

FlareCanary monitors REST APIs — including Auth0 tenants — for schema drift and connection-layer changes. Free tier covers 5 endpoints with daily checks.

shopify.metafields returns undefined in my checkout extension after 2026-04 — here's why

FlareCanary — Sat, 25 Apr 2026 04:02:31 +0000

If you just bumped your checkout UI extension to Shopify API version 2026-04 and your shopify.metafields reads started returning undefined, you've hit the biggest breaking change in this release.

Checkout metafields in the metafields API are gone. They've been replaced by order metafields in the appMetafields API. The rename isn't cosmetic — it's a different data source, a different access pattern, and a different set of timing semantics. The old code doesn't error; it just silently returns nothing.

What the failure looks like

The code that worked on 2026-01:

import { extension } from '@shopify/ui-extensions/checkout';

export default extension('purchase.checkout.block.render', (root, api) => {
  const { metafields } = api;
  const myField = metafields.current.find(
    m => m.namespace === 'custom' && m.key === 'delivery_window'
  );
  // myField.value used to be "2026-05-01T09:00:00"
  root.append(root.createText(myField?.value ?? 'unknown'));
});

After the 2026-04 upgrade, myField is undefined. No deprecation warning. No network error. The extension renders 'unknown' in production and the fallback path becomes the default behavior. If the ternary hid a real problem — a missing delivery window, a missing subscription flag, a missing audit tag — your orders flow through without it.

The symptom in Shopify's developer console, if you're lucky enough to be looking there at the right moment, is either a missing metafield in the api.metafields.current array or the array itself being empty. Neither is an error condition, just a change in what's served.

Why it breaks silently

Shopify's UI extensions surface APIs as plain JS properties. When the platform removes a property at a given API version, the consuming code reads undefined — the same as if the field weren't set for this buyer. There's no distinction between "metafield doesn't exist on this order" and "this API doesn't expose metafields anymore," because both are the same runtime shape.

The 2026-04 release moves this data to a different surface: shopify.appMetafields. The old call site keeps compiling because metafields is still a property on api — it's just the thing it references has shrunk.

The exact migration

The replacement property is appMetafields, read from the global shopify object rather than a callback parameter:

import '@shopify/ui-extensions/preact';
import { render } from 'preact';

export default async () => {
  render(<Extension />, document.body);
};

function Extension() {
  const entries = shopify.appMetafields.value;
  const myField = entries.find(
    e =>
      e.target.type === 'order' &&
      e.metafield.namespace === 'custom' &&
      e.metafield.key === 'delivery_window'
  );
  return <s-text>{myField?.metafield.value ?? 'unknown'}</s-text>;
}

Three differences worth internalizing:

appMetafields is an app-scoped surface. Your extension only sees metafields written under your app's reserved namespace. Anything your app wrote to a custom.* namespace before is visible on the compatibility shim today, but new writes should use the app-reserved namespace going forward.
Entries are wrapped, not bare. Each entry has { target, metafield }. The target is the resource the metafield is attached to (order, customer, etc.). The old flat-array shape is gone — filter on target.type before filtering on namespace/key.
It's a reactive value, not a snapshot. shopify.appMetafields is a Preact-signals-style reactive value. Reading .value gives the current array. Inside a component it re-renders when the underlying entries change, which matches the new 2026-01 UI model (Preact web components replacing React).

Where the old pattern is hiding in your code

This migration hits more than the obvious metafield calls. Places to grep:

git grep -n "api\.metafields"
git grep -n "useMetafields"
git grep -n "shopify\.metafields"
git grep -n "ExtensionPoint.*Checkout"

The useMetafields React hook is an older API that also routed through the deprecated surface. Anywhere you called it, the replacement is the shopify.appMetafields.value read above.

Server-side writers don't need to change — the Admin GraphQL API's metafieldsSet mutation still works. It's only the read path from checkout extensions that moves.

The cart-metafields path, if you're writing fresh code

Separately from the appMetafields migration, Shopify is steering new code toward cart metafields as the canonical place for per-buyer data that needs to survive checkout. As of 2026-04, cart metafields automatically copy to the resulting order at checkout completion, which closes a long-standing gap where checkout-time state quietly didn't make it to fulfillment.

If you're building something new today, the decision tree is:

Per-buyer data that needs to outlive checkout → cart metafields. They'll auto-copy to the order.
Per-app configuration that needs to be read at checkout → app-reserved namespace metafields, read via shopify.appMetafields.
Legacy custom.*-namespace metafields your app has historically written → still readable on the compatibility shim, but plan a migration to your app-reserved namespace.

Detecting the break before your QA does

The obvious version check:

grep -r "api_version" extensions/ | grep -v node_modules

If anything returns 2026-04 or later and the extension still reads api.metafields, that extension is broken.

The less-obvious version: extensions can be bumped independently, and an app with multiple extensions can have one on 2026-04 and another on 2026-01. The one on 2026-04 silently fails; the one on 2026-01 works. You get half your data on the order and half missing — exactly the confusing support ticket that's hard to reproduce.

A small synthetic order run in your staging shop is the cheapest catch. Place an order that exercises each extension, pull the resulting order via the Admin API, and diff the metafields array against what you expect. Any missing key is a lead.

The broader pattern

What's happening here is a specific case of a general problem: the SDK you depend on changes the shape of the data it returns, not the shape of its function signatures. Types still line up. The code still compiles. The values have just moved, and the old read path becomes undefined.

The ways this usually bites:

TypeScript doesn't catch it. The types got updated on the SDK, your code got updated with @shopify/ui-extensions 2026-04, and api.metafields is still typed — it's just a narrower universe of things now.
Unit tests don't catch it. Mocked responses in a test fixture don't know the real platform shape changed.
CI doesn't catch it. Nothing in CI is hitting Shopify's edge. The break only shows up in a real checkout.

The way to catch this kind of drift before buyers do is a scheduled synthetic check against the real vendor surface — place an order in a staging shop, read it back, diff the metafields array against a known baseline. Same pattern as monitoring any other third-party API contract, applied to the specific surface each platform exposes.

Minimum-viable fix for today

git grep -E "api_version.*2026-04" across every extension and confirm which ones bumped
git grep -n "api\.metafields\|useMetafields" in every bumped extension — each hit is a broken read
Migrate those call sites to shopify.appMetafields.value with .target.type === 'order' filtering
Move any app-owned metafield writes to your app-reserved namespace
Place a synthetic test order in a staging shop and verify the expected metafield keys land on the resulting order
If you have multiple extensions on mixed API versions, align them or at minimum document which are on which version

If the migration reveals that checkout metafields were load-bearing for fulfillment, consider whether cart metafields — now auto-copied to the order at checkout completion — are the better home going forward.

FlareCanary monitors REST and GraphQL APIs — including Shopify Admin and Storefront endpoints — for schema drift. Free tier covers 5 endpoints with daily checks.

DALL·E shuts down May 12 — the gpt-image-1 migration isn't the drop-in swap it looks like

FlareCanary — Sat, 25 Apr 2026 04:02:03 +0000

OpenAI is shutting down dall-e-2 and dall-e-3 on May 12, 2026. After that date, requests to /v1/images/generations with either model string will stop working. The recommended replacements are gpt-image-1 and gpt-image-1-mini.

On paper this is a one-line change: swap the model name, ship. In practice, the request and response shapes are different enough that a naive swap breaks clients that worked against DALL·E for the last two years.

What the shutdown looks like

OpenAI's deprecation language: "deprecated models will no longer be accessible" after the shutdown date. Translated: your POST /v1/images/generations with "model": "dall-e-3" will return an error, not a fallback to the new model.

Expected response shape:

{
  "error": {
    "message": "The model `dall-e-3` has been deprecated. Learn more: https://platform.openai.com/docs/deprecations",
    "type": "invalid_request_error",
    "code": "model_not_found"
  }
}

No grace period, no auto-upgrade. The endpoint itself still exists — /v1/images/generations is alive and serves gpt-image-1. Only the old model IDs are gone.

Where `dall-e-3` is probably pinned in your code

The model string is usually spread across more surfaces than you'd expect:

Environment variables: OPENAI_IMAGE_MODEL, DALLE_MODEL, IMAGE_MODEL. Check every environment, not just prod.
SDK wrappers: the Python openai SDK's client.images.generate(model=...) call. LangChain's DallEAPIWrapper. Vercel AI SDK image helpers. LiteLLM routers.
Hardcoded defaults: it's common to see model or "dall-e-3" as a fallback when the caller doesn't pass one. That fallback becomes a runtime error on May 12.
Tests and fixtures: VCR cassettes, recorded responses, snapshot tests. These pass locally but the behavior they capture is about to change.
Documentation and onboarding code: if your docs show "model": "dall-e-3" as an example, update them before users copy it into fresh projects.

The five-minute audit:

git grep -n "dall-e-3\|dall-e-2\|dalle-3\|dalle-2"

The request shape changed

A DALL·E 3 call looks like this:

{
  "model": "dall-e-3",
  "prompt": "a watercolor fox",
  "n": 1,
  "size": "1024x1024",
  "quality": "hd",
  "style": "vivid",
  "response_format": "url"
}

The gpt-image-1 equivalent doesn't accept three of those fields:

response_format is gone. DALL·E returned hosted image URLs by default. gpt-image-1 returns base64-encoded PNG bytes in b64_json, always. If your client reads response.data[0].url, it will be None. You need to decode the bytes and either upload them to your own storage or serve them inline.
style is gone. The vivid / natural distinction doesn't exist. Prompt-engineer the style into the text instead.
quality values changed. DALL·E 3 used standard / hd. gpt-image-1 uses low / medium / high / auto. A literal "hd" in the request will be rejected.
size values changed. DALL·E 3 supported 1024x1024 | 1792x1024 | 1024x1792. gpt-image-1 supports 1024x1024 | 1024x1536 | 1536x1024 | auto. Landscape and portrait are 1536×1024 and 1024×1536, not 1792×1024 and 1024×1792.

New parameters you probably want to set explicitly: output_format (png / jpeg / webp), output_compression (for jpeg/webp), and moderation (low / auto).

The cost model flipped

This is the migration gotcha nobody flags in the deprecation notice.

DALL·E 3 was billed per image: $0.040 for standard 1024×1024, $0.080 for HD. You could forecast cost by counting images.

gpt-image-1 is billed per token — input text tokens, input image tokens (for edits), and output image tokens. A single medium quality 1024×1024 generation is roughly ~1,000 output image tokens at $40 per million, so around $0.04. high quality is several times that. gpt-image-1-mini is cheaper but still token-billed.

If your current cost forecast is images_per_month × $0.04, that forecast is wrong after May 12 in ways that depend on your prompt length and quality setting. Re-model before you switch, not after the first invoice.

The pattern this fits

OpenAI's deprecation cadence over the last year:

October 2024 — gpt-3.5-turbo-0301, gpt-3.5-turbo-0613 retired
June 2025 — gpt-4-0314, gpt-4-32k-0314 retired
January 2026 — text-moderation-007 moved to legacy
May 12, 2026 — dall-e-2, dall-e-3 retired
Announced, no firm date — Assistants API sunset August 26, 2026

Every one of these was announced with at least 60 days of notice. Every one of them broke production for teams that didn't see the notice. The pattern isn't a surprise attack — it's that nobody instruments the provider's public surface (models list, deprecation page, error shapes) as a monitored contract.

What actually catches this

A dependency on a third-party API is a silent coupling. The provider changes the shape of what they return, or stops accepting a model ID, and your code — unchanged — starts failing. CI doesn't catch it because CI runs against fixtures or mocks. Unit tests don't catch it because the SDK still type-checks.

Two things catch it:

Integration tests that hit the real API on a schedule (nightly is enough), against every model ID and request shape you rely on. When the shape drifts, the test turns red.
Monitoring the provider's models list and documented error shapes as a diffable contract. GET https://api.openai.com/v1/models tells you which IDs are live. When dall-e-3 stops appearing there, you want the alert before the next deploy.

Either approach works. What doesn't work is hoping the deprecation email lands in an inbox someone reads.

Minimum-viable fix for this one

git grep for every DALL·E model ID across every repo that talks to OpenAI.
Replace with gpt-image-1 (or gpt-image-1-mini if cost-sensitive).
Remove response_format, style, and any quality: "hd" / quality: "standard" from the request body.
Update response-handling code to decode b64_json instead of fetching url. Add your own storage step if your product needed hosted URLs.
Re-forecast cost against the token-billed model. Run a sample batch through gpt-image-1 at your expected quality setting and multiply out.
Check that the old model IDs aren't still in staging, demo apps, or onboarding example code.

If this is the third or fourth time a provider change has broken production without warning, the problem isn't this specific deprecation. It's that you're finding out from alerts instead of ahead of time.

FlareCanary monitors REST APIs and MCP servers for schema drift — including models-list and error-shape changes from AI providers. Free tier covers 5 endpoints with daily checks.

HubSpot Contacts v1 Goes Dark on April 30 — And the Worst Part Is the Endpoints That Keep Working

FlareCanary — Sat, 25 Apr 2026 04:01:36 +0000

On April 30, 2026, HubSpot sunsets the Contact Lists v1 API. The headline is simple: most /contacts/v1/lists/* endpoints start returning HTTP 404. The dangerous part is everything that doesn't 404.

Straight from HubSpot's sunset announcement, six Contacts v1 read endpoints will:

continue to function but will no longer return list memberships

That's a 200 response with the list-memberships array silently gone. Code that does contact["list-memberships"] gets a KeyError. Code that does contact.get("list-memberships", []) swallows the empty list and now thinks the contact is in zero lists. Both are broken. One of them looks broken.

This is incident #6 in our silent-breakage series (GitHub PushEvent, Stripe Basil, Shopify 2025-01, OpenAI Responses input_text, Twilio regional domains). Same pattern: the breaking change lives in a field you weren't asserting against.

What returns 404 on April 30

The full Contact Lists v1 surface goes away:

GET /contacts/v1/lists
GET /contacts/v1/lists/:list_id
POST /contacts/v1/lists
POST /contacts/v1/lists/:list_id/add
POST /contacts/v1/lists/:list_id/remove
DELETE /contacts/v1/lists/:list_id
GET /contacts/v1/lists/static
GET /contacts/v1/lists/dynamic
GET /contacts/v1/lists/:list_id/contacts/all
GET /contacts/v1/lists/:list_id/contacts/recent

Plus everything else under that path. On April 30 they return:

{
  "status": "error",
  "message": "Resource not found"
}

Three Lists endpoints survive but lose membership context:

GET /contacts/v1/lists/all/contacts/all
GET /contacts/v1/lists/all/contacts/recent
GET /contacts/v1/lists/recently_updated/contacts/recent

These return contact records, but each contact's list-memberships array stops being populated.

The endpoints that bite — they keep returning 200

Here's the list every audit script needs to grep for:

GET /contacts/v1/contact/vid/:vid/profile
GET /contacts/v1/contact/vids/batch
GET /contacts/v1/contact/email/:contact_email/profile
GET /contacts/v1/contact/emails/batch
GET /contacts/v1/contact/utk/:contact_utk/profile
GET /contacts/v1/contact/byUtk/batch

These all keep returning HTTP 200 on May 1. The difference is in the body.

Before April 30:

{
  "vid": 12345,
  "properties": { "...": "..." },
  "list-memberships": [
    { "static-list-id": 8, "internal-list-id": 8, "timestamp": 1714492800000, "vid": 12345, "is-member": true },
    { "static-list-id": 14, "internal-list-id": 14, "timestamp": 1714493000000, "vid": 12345, "is-member": true }
  ]
}

After April 30:

{
  "vid": 12345,
  "properties": { "...": "..." },
  "list-memberships": []
}

Status code: 200. JSON parses cleanly. Field is present. It's just empty.

If your sync job uses these endpoints to figure out which CRM lists a contact belongs to — Marketo, Salesforce, ActiveCampaign, custom data warehouse, anything — every contact looks like it belongs to nothing. Conditional sends fire (or don't) based on phantom list membership. Audience exclusion lists don't exclude.

The fix path: v3 Lists API

The replacement is the Lists v3 API. For membership lookups, the new shape is per-list, not per-contact:

# v1 (going away or going empty)
GET /contacts/v1/contact/vid/12345/profile
# → list-memberships array on the contact object

# v3 (the replacement)
GET /crm/v3/lists/{listId}/memberships
# → paginated array of {recordId, membershipTimestamp}

The semantic flip matters. v1 answered "what lists is this contact in?" — efficient when you have a contact and want their lists. v3 answers "who is in this list?" — efficient when you have a list and want its contacts. If your code's hot path is the contact-to-lists direction, you have to either:

Cache list memberships on your side, refreshed on a schedule
Hit /crm/v3/lists/memberships/contacts/{contactId} (yes, this exists, but it's a different endpoint with different rate limits)
Subscribe to membership-change webhooks and maintain the inverse index yourself

Pick before April 30. The default — "we'll figure it out when something breaks" — picks option 4: silently shipping a sync job that thinks every contact is in zero lists.

Why your tests didn't catch it

Your integration tests hit a HubSpot sandbox. The contact fixture in the sandbox is in three lists. The test asserts len(contact["list-memberships"]) == 3.

Three things have to be true for that test to fail before May 1:

Your sandbox is the production HubSpot environment (it isn't)
HubSpot deprecates the field in the sandbox before prod (they don't)
The test runs after April 30 with no caching (sometimes)

So the test stays green. The same pattern as every other silent drift incident:

#	API	What changed	Where tests missed it
1	GitHub PushEvent	`commits` field silently dropped	Tests didn't assert field presence
2	Stripe Basil	`current_period_end` moved to `items`	Tests used Checkout fixtures
3	Shopify 2025-01	`fulfillmentHold` type change	Tests mocked the response
4	OpenAI Responses	`input_text` removed for assistants	Tests covered request role=user
5	Twilio regional	Regional domains stop resolving	Tests don't hit prod DNS paths
6	HubSpot Contacts v1	`list-memberships` returns empty	Tests asserted against sandbox fixtures

The breaking change always lives in a layer the test suite isn't watching. For HubSpot, that layer is "field that used to be populated and now isn't, while the request and response otherwise look identical."

What to do this week

April 30 is seven days out. Three actions:

Grep your codebase for contacts/v1/. Every match is a potential failure. Endpoints in the 404 group break loudly. Endpoints in the 200-but-empty group break quietly.
Grep for list-memberships and list_memberships. Every read against this field is downstream-affected even if you can't change the v1 call site this week.
Decide between cache-side and webhook-side. v3's per-list shape forces an architectural choice. Picking it under deadline pressure is worse than picking it on Monday.

If your platform integrations team is on PTO, the failure mode on May 1 is "every CRM sync job runs to completion with empty audiences." That's a quiet kind of broken — one your monitoring probably doesn't catch.

We built FlareCanary for exactly this layer: poll third-party APIs on a schedule, watch the response, page when a field that used to populate stops populating. The HubSpot deprecation is the textbook case — same status code, same JSON shape, semantically broken. The APIs change whether you're watching or not.

We track API drift incidents in real time. If your stack syncs HubSpot list memberships and you haven't audited the read paths yet, April 30 is a hard date — and the dangerous part is the endpoints that don't return 404.

Twilio Is Killing api.de1.twilio.com on April 28 — And the Regional Domains Never Actually Routed Regionally

FlareCanary — Fri, 24 Apr 2026 13:01:23 +0000

On April 28, 2026, seven Twilio API domains stop working:

api.ie1.twilio.com
api.au1.twilio.com
api.br1.twilio.com
api.de1.twilio.com
api.jp1.twilio.com
api.sg1.twilio.com
api.us2.twilio.com

If your code, SDK config, firewall allowlist, or security group references any of those hostnames, the calls will start failing. There is no SDK version bump that masks this — the DNS records go away.

Here's the awkward part, straight from Twilio's API Domain Migration Guide:

these domains process all requests in US1 regardless of the region code

Everyone who set region: 'de1' thinking they were keeping data in Germany for residency reasons — they weren't. The request hit a domain that routed everything to US1 anyway. The region code in the hostname was cosmetic.

This is incident #5 in our silent-breakage series (GitHub PushEvent, Stripe Basil, Shopify 2025-01, OpenAI Responses input_text). The pattern keeps repeating and the fix keeps looking the same.

What actually breaks

If you call Twilio like this today:

from twilio.rest import Client

client = Client(account_sid, auth_token, region='de1')
# SDK resolves to api.de1.twilio.com
client.messages.create(to='+49...', from_='+49...', body='hello')

On April 28, that api.de1.twilio.com DNS name stops resolving. The SDK bubbles up a connection error — ConnectionError in Python, ECONNREFUSED / ENOTFOUND in Node, depending on how the resolver fails. Your retry logic retries. Your logs fill up. Nothing recovers on its own.

Same thing for raw HTTP clients:

# This stops working on April 28, 2026
curl -u "$SID:$TOKEN" https://api.de1.twilio.com/2010-04-01/Accounts/$SID/Messages.json

The nastier failure mode isn't the SDK — it's your infrastructure:

Firewall rules that allow outbound 443 only to api.de1.twilio.com
Security groups that whitelist the regional domain
Load balancers with the domain hardcoded in health checks
Proxies that route Twilio traffic through a specific egress
Secret store entries that embed the full URL

SDK configs are easy to grep for. Terraform modules and Helm charts referencing api.de1.twilio.com in a cidr_blocks or allowed_hosts block — those are the ones that bite on April 28.

The two fix paths

Twilio gives you two options, and you need to pick based on what you were actually doing, not what you thought you were doing.

Path 1: You don't actually need regional processing

This is the majority case, because — again — the regional domains weren't processing regionally anyway.

Code change:

# Before
client = Client(account_sid, auth_token, region='de1')

# After
client = Client(account_sid, auth_token)
# Resolves to api.twilio.com

Raw HTTP:

# Before
curl https://api.de1.twilio.com/...

# After
curl https://api.twilio.com/...

Firewall / security group:

Add api.twilio.com (resolves to the US1 edge). Remove the regional hostname entries. If your infra does IP-based rules, you need Twilio's US1 IP list — do not attempt to resolve the domain and pin the IPs, Twilio's edge IPs rotate.

Path 2: You actually need regional processing

If you have a real regional processing requirement (data residency, latency, compliance), you need the new localized domains plus the edge parameter plus region-scoped credentials:

client = Client(
    regional_account_sid,
    regional_auth_token,
    edge='dublin',
    region='ie1',
)
# Resolves to api.dublin.ie1.twilio.com

Three things had to change, not one:

Hostname: api.dublin.ie1.twilio.com (new localized pattern)
SDK params: both edge and region, together
Credentials: regional SID and token, not your global ones

If you only add the edge parameter and keep the old global credentials, you'll get auth errors. If you keep only region and drop edge, the SDK may still resolve to the deprecated domain depending on your SDK version.

Twilio's migration guide includes a warning: "Older SDK versions may not properly support the edge parameter or may have bugs related to regional routing." Update the SDK first. This is a second source of drift — you think you're migrated, but an old SDK silently rewrites your request back to the deprecated hostname.

Why your tests didn't catch it

You have integration tests. They hit a Twilio sandbox. They pass. The April 28 deprecation still ships a broken prod.

The reason is the same reason every other silent drift story ships a broken prod:

Your tests exercise the API's response shape. They don't exercise the infrastructure around the API.

A test that checks response.status == 'queued' will continue passing after April 28 — because your test environment probably uses api.twilio.com already, or it mocks Twilio entirely. The failure mode is DNS resolution against a specific hostname buried in your prod firewall rule. No unit test hits that path.

The same story played out with the OpenAI input_text removal: tests used output_text, the change was in the assistant role message format, and it only failed on request construction, not response parsing.

The pattern across APIs

Five incidents, same shape:

#	API	What changed	Where tests missed it
1	GitHub PushEvent	`commits` field silently dropped	Tests didn't assert field presence
2	Stripe Basil	`current_period_end` moved to `items`	Tests used Checkout fixtures
3	Shopify 2025-01	`fulfillmentHold` type change	Tests mocked the response
4	OpenAI Responses	`input_text` removed for assistants	Tests covered request role=user
5	Twilio regional	Regional domains stop resolving	Tests don't hit prod DNS paths

The consistent pattern: the breaking change lives in a layer your test suite isn't watching. Request construction, infrastructure config, DNS, header shape, optional field presence. Response-shape assertions catch ~20% of drift cases at best.

How to not be surprised by the next one

Three things actually help:

Subscribe to every vendor's changelog. Not just the ones you think you use — the ones the SDKs in your package.json use. Most companies announce breaking changes. You have to be reading.
Monitor the raw response, not just status == 'queued'. If Twilio's response drops a field next year, you want an alert, not a silent parse failure downstream.
Watch the infrastructure layer. Firewall rules, security groups, and Terraform modules should reference hostnames, not cached IP addresses — and those hostnames need to survive vendor deprecation windows.

We built FlareCanary to do #2 — hit third-party APIs on a schedule, watch the response, page you when the shape changes. It's not magic; it's the poll-and-diff loop you would write yourself if you had the time. The APIs break whether you're watching or not.

We track API drift incidents in real time. If your stack touches Twilio and you haven't audited your hostname config yet, April 28 is a hard date.

claude-3-haiku-20240307 just started returning errors — here's what happened

FlareCanary — Wed, 22 Apr 2026 13:18:11 +0000

If your code hits claude-3-haiku-20240307 and started failing yesterday, you're not alone. Anthropic retired Claude Haiku 3 on April 20, 2026. Retired, not deprecated — requests to the old model ID no longer complete. They return an error.

This is a real deprecation cutoff that was announced 60 days ago and landed without much fanfare. If you missed the email, you're finding out now — probably from a production alert, or worse, from a user telling you the product is broken.

What the failure looks like

Anthropic's deprecation policy is explicit: retired models return errors, not redirects. There is no automatic fallback to Haiku 4.5. The model ID just stops resolving.

In practice that means a 4XX response from the Messages API — not_found_error if the router decides the model identifier is gone, or invalid_request_error if it's treated as a malformed request. Either way, the body looks roughly like this:

{
  "type": "error",
  "error": {
    "type": "not_found_error",
    "message": "The requested resource could not be found."
  },
  "request_id": "req_..."
}

What makes this nasty is that nothing about your code changed. The request body is identical to what worked last week. The SDK version is the same. Your API key is the same. The only thing that changed is whether Anthropic's routing layer still recognizes claude-3-haiku-20240307 — and as of April 20, it doesn't.

Where the stale model ID is probably hiding

Most teams have more than one reference to a model name. Places to look:

Environment variables and config files — ANTHROPIC_MODEL, CLAUDE_MODEL, DEFAULT_MODEL. Check staging and prod, not just the repo.
Framework wrappers — LangChain, LlamaIndex, Vercel AI SDK, LiteLLM. Older versions may have claude-3-haiku-20240307 as a default or in documented examples. Check what your package pins to.
Hardcoded test fixtures — VCR cassettes, snapshot tests, and mocked responses often capture the model string.
Fine-grained routing logic — anywhere you pick a model based on task type ("use the cheap one for classification"), check which model the "cheap" branch resolves to.
Logs and monitoring dashboards — not a code issue, but if you filter traces by model ID, those filters break silently.

The five-minute grep:

git grep -n "claude-3-haiku-20240307"
git grep -n "claude-3-haiku"

If either returns hits, you have work to do.

What to replace it with

Anthropic's recommended replacement is claude-haiku-4-5-20251001. On paper it's a straight swap. In practice, Haiku 4.5 is a Claude 4-generation model, and the Claude 4 generation introduced a handful of breaking API changes that trip up code written for Haiku 3:

temperature and top_p can no longer both be set. Haiku 3 accepted both. Haiku 4.5 rejects the combination. If your code sets both defensively, remove one before switching.
New stop reasons. Claude 4 adds stop reasons that Haiku 3 didn't emit. If you have a switch or match on stop_reason that throws on unknown values, it will start throwing.
Trailing newlines in tool parameters are preserved. Previously stripped. If your downstream logic trims on output, fine; if it does exact-match validation, you may get spurious mismatches.
Rate limits are tracked separately per model. Your old Haiku 3 quota doesn't transfer. If you were running near the ceiling, plan for headroom at the new model.
Pricing. Haiku 4.5 is $1.00 input / $5.00 output per million tokens, up from $0.25 / $1.25. That's a 4× increase. Cost-sensitive workloads need to re-forecast.

The migration isn't a one-line change, and pretending it is is how bugs ship.

The pattern this fits

Anthropic's deprecation cadence is now predictable enough to plan against. The recent timeline:

October 28, 2025 — Claude Sonnet 3.5 retired
January 5, 2026 — Claude Opus 3 retired
February 19, 2026 — Claude 3.5 Haiku and Claude Sonnet 3.7 retired
April 20, 2026 — Claude Haiku 3 retired (yesterday)
June 15, 2026 — Claude Opus 4 and Claude Sonnet 4 retire (announced April 14, 2026)

If your code pins a Claude 4 model ID today, that pin has a two-month clock on it. The 2024- and early-2025- generation model IDs are all on a one-year glidepath to retirement.

Monitoring model deprecation as an API change

What's happening here is a narrow case of a broader problem: third-party APIs change between your deploys, and the change is invisible until something breaks.

The Anthropic deprecation page updates.
The model ID in GET /v1/models disappears.
The error response from POST /v1/messages changes shape for the old ID.

None of those events trigger your CI. None of them change your code. All of them change whether your code works.

The way to catch this before your users do is to treat the provider's API surface — the models list, the error shapes, the documented schemas — as a contract you continuously verify. Poll, diff against a known baseline, alert on breaking diffs. Same pattern as monitoring any other third-party API, applied to the specific surface each vendor exposes.

For Anthropic specifically: GET https://api.anthropic.com/v1/models returns the currently-available model IDs. If claude-haiku-4-5-20251001 disappears from that list, you want to know before your next deploy, not after.

Minimum-viable fix for today

git grep claude-3-haiku-20240307 across every repo that talks to Anthropic
Replace with claude-haiku-4-5-20251001
Remove any call site that sets both temperature and top_p
Re-run integration tests against the new model — not just unit tests with mocked responses
Forecast cost impact of the 4× price change before rolling to production
Check that your rate-limit dashboards pull the new model's quotas, not the retired one's

Then, if this is the second or third time a provider API change has broken your build without warning, consider whether catching these earlier is worth instrumenting.

FlareCanary monitors REST APIs and MCP servers for schema drift — including model availability endpoints from AI providers. Free tier covers 5 endpoints with daily checks.

CI Tests Won't Save You from MCP Schema Drift

FlareCanary — Tue, 21 Apr 2026 04:01:14 +0000

There's a growing category of tools that validate MCP server schemas in CI/CD pipelines. Run them on pull requests, catch schema mismatches before you deploy.

This is genuinely useful. But it solves the wrong half of the problem.

The MCP drift problem has two halves

Half 1: Your code drifts from the server. You change your agent code, but the MCP server's tool schemas haven't changed. CI testing catches this — run tests, verify your code still matches the tool definitions.

Half 2: The server drifts from your code. The MCP server updates its tool schemas, but you haven't deployed anything. CI doesn't run because you didn't change anything. Your agent keeps calling tools with the old parameter names, and the LLM silently adapts (or silently fails).

Half 2 is the dangerous one. And CI can't catch it by definition.

Why LLMs make this worse

When a REST API changes, your code throws an error. A missing field causes a TypeError. A renamed endpoint returns a 404. The failure is loud.

When an MCP tool schema changes, the LLM doesn't crash. It adapts. If a parameter gets renamed from search_query to query_text, the LLM might:

Pass the old parameter name and get an empty result
Interpret the empty result as "no data found" instead of "wrong parameter"
Tell the user "I couldn't find any matching documents" — a plausible lie

The agent looks healthy. No errors in your logs. No alerts from your monitoring. The user gets a wrong answer and has no way to know.

What CI testing actually catches

CI-based MCP schema validation is good at:

Schema-implementation mismatches: The tool says a parameter is optional, but the server actually requires it.
Regression testing: After you change something, verify it still works.
Type validation: Ensure your inputs match declared types.

These are real problems worth solving. If you're building MCP servers, you should absolutely have CI tests for your tool schemas.

What CI testing misses

Third-party MCP server changes: You don't run CI for someone else's server. When Stripe's MCP server renames a tool parameter, your pipeline doesn't trigger.
Between-deploy drift: The MCP server you depend on ships a breaking change on Saturday night. Your agent is broken from Saturday to Monday morning when someone finally notices.
Gradual schema evolution: A tool starts accepting a new optional parameter. Two weeks later, the old parameter gets deprecated. A month later, it's removed. CI only sees one snapshot.
Runtime behavior changes: The schema says a field is a string. It was always a URL. Now it's a UUID. The type didn't change, but your agent's downstream logic breaks.

The monitoring gap

Most teams have this setup:

CI/CD: Schema tests on deploy ✓
Staging: Smoke tests ✓
Production: Uptime checks ✓ (is the server responding?)
Drift monitoring: ??? (is the server responding *correctly*?)

The gap is in the last line. Your uptime check confirms the MCP server returns 200 OK. It doesn't check whether tools/list returns the same tool definitions it returned last week.

What continuous MCP monitoring looks like

Instead of (or in addition to) CI-time validation:

Poll tools/list on a schedule — hourly, daily, whatever fits your risk tolerance.
Diff the tool schemas against a known baseline — parameter names, types, required flags, descriptions.
Classify changes by severity — new optional parameter = informational. Renamed required parameter = breaking.
Alert on breaking changes — Slack, email, webhook, whatever gets to the right person.
Maintain a timeline — know when the change happened, not just that it happened.

This catches the Saturday-night breaking change before Monday morning. It catches the gradual deprecation cycle. It catches the third-party server update that your CI pipeline will never see.

CI and continuous monitoring are complementary

This isn't an either/or choice.

CI testing validates that your code works with the current MCP server schemas at deploy time. It's a pre-deployment gate.

Continuous monitoring validates that the MCP server schemas haven't changed since your last deploy. It's a post-deployment safety net.

If you only have CI tests, you're assuming the world doesn't change between your deploys. In a world where MCP servers are updated independently by different teams (or different companies), that assumption is broken.

The minimum viable setup:

CI schema validation for MCP servers you build
Continuous monitoring for MCP servers you depend on
Severity-based alerting so you're not drowning in noise

The first catches your mistakes. The second catches everyone else's.

FlareCanary monitors REST APIs and MCP servers for schema drift. Free tier covers 5 endpoints with daily checks — enough to monitor your most critical MCP dependencies.

The Complete Guide to API Schema Drift Monitoring in 2026

FlareCanary — Mon, 20 Apr 2026 04:02:52 +0000

You've written your integration. The tests pass. The API docs say the response looks like this:

{
  "user": {
    "id": 123,
    "name": "Alice",
    "email": "alice@example.com"
  }
}

Six weeks later, it looks like this:

{
  "user": {
    "id": "usr_123",
    "name": "Alice",
    "email": "alice@example.com",
    "email_verified": true
  }
}

id changed from number to string. A new field appeared. Your code didn't crash — it just started silently mishandling data. Welcome to API schema drift.

What Is API Schema Drift?

Schema drift is when an API's actual response structure diverges from what consumers expect — whether that expectation comes from an OpenAPI spec, documentation, or just "what it returned last week."

It's not the same as downtime. Drift is subtler:

A field changes type (integer → string)
A required field becomes nullable
An enum gains new values your switch statement doesn't handle
A nested object gains or loses properties
A field gets renamed or removed

According to KushoAI's 2026 "State of Agentic API Testing" report, 41% of APIs experience schema drift within 30 days and 63% within 90 days. Schema/validation failures account for 22% of all API failures — second only to authentication issues.

Why Schema Drift Matters More in 2026

Three trends are making drift monitoring essential:

1. External API Dependencies Are Everywhere

Modern applications depend on dozens of third-party APIs. You don't control their release cycles, and they rarely notify you of non-breaking changes. But "non-breaking" by their definition may be breaking for your code.

2. AI Agents Depend on Tool Schemas

The explosion of MCP (Model Context Protocol) servers means AI agents discover and call tools based on schema definitions. If a tool's input schema changes silently, the LLM doesn't throw an error — it adapts to the wrong contract. This is a new class of drift that didn't exist two years ago.

3. Microservice Sprawl

Even internal APIs drift. Team A ships a change, Team B's consumers don't update. With 50+ internal services, manual tracking is impossible.

Approaches to Drift Detection

There are fundamentally different ways to catch drift, and they solve different problems:

Spec-to-Spec Diffing (CI-Time)

What: Compare two versions of an OpenAPI/Swagger spec file.
When it catches drift: Before deploy, in CI/CD pipelines.
Limitation: Only works if specs exist and are kept up-to-date. Doesn't catch cases where the spec is wrong or the server deviates from the spec.

Tools:

oasdiff (open source, 1K+ GitHub stars) — CLI and GitHub Action. 300+ breaking change rules. The standard for spec-to-spec diffing.
Bump.sh — Documentation platform with built-in changelog generation from spec diffs.
Optic — Was the leader here. Acquired by Atlassian (2024), repository archived January 2026. No standalone product remains.

Spec-to-Reality Monitoring (Runtime)

What: Make actual HTTP requests to live endpoints and compare responses against an OpenAPI spec or learned baseline.
When it catches drift: After deploy, continuously in staging or production.
Limitation: Requires network access to the endpoints. May not catch drift in rarely-used response variants.

Tools:

FlareCanary ($0-49/mo) — SaaS platform for live drift monitoring. Compares responses against OpenAPI specs or inferred baselines. Severity classification (breaking/warning/info), multi-sample learning to reduce false positives. Also monitors MCP server tool schemas. Free tier: 5 endpoints, daily checks.
API Drift Alert ($149-749/mo) — Dedicated drift monitoring. Severity-aware routing, PagerDuty integration. Enterprise-focused pricing.
Rumbliq ($0-69/mo) — General monitoring platform with JSON response diffing as one feature. 25 free monitors. Broader but shallower on drift specifically.
watchflow.io (beta, free until May 2026) — API uptime + schema monitoring. Pricing TBD.

Traffic-Based Detection (Passive)

What: Analyze actual API traffic patterns to detect deviations.
When it catches drift: Continuously, but only for APIs with sufficient traffic.
Limitation: Requires deploying agents or SDKs into your infrastructure. Only monitors APIs you own (not third-party dependencies).

Tools:

Treblle ($0-300/mo) — API intelligence platform. SDK-based, monitors your own APIs.
Tusk Drift (open source) — Records production traffic into test suites. Catches deviations between recorded and new responses.
Levo.ai ($2,500+/mo) — Enterprise API security platform with eBPF-based traffic analysis.

MCP/AI Tool Schema Monitoring

What: Track changes in MCP server tool catalogs and input/output schemas.
When it catches drift: Continuously, as MCP servers update their tool definitions.
Limitation: Emerging category. Limited tooling.

Tools:

FlareCanary — Consumer-side MCP monitoring (monitors third-party MCP servers you depend on).
Bellwether (open source) — CI-time MCP schema snapshots and diffing.
Specmatic — Commercial MCP schema testing.
Sentry/Grafana/AWS CloudWatch — Server-side MCP observability (requires owning the MCP server).

Choosing the Right Approach

Your situation	Best approach
You maintain OpenAPI specs in version control	Spec-to-spec diffing (oasdiff) in CI + live monitoring as a safety net
You consume third-party APIs you don't control	Spec-to-reality monitoring (FlareCanary, API Drift Alert)
You want to monitor your own APIs' behavior	Traffic-based detection (Treblle) or spec-to-reality monitoring
Your AI agents depend on MCP tool servers	MCP schema monitoring (FlareCanary for consumer-side, Bellwether for CI)
You have no OpenAPI specs at all	Baseline-learning tools that infer schema from responses (FlareCanary, Rumbliq)

The Monitoring Stack

For thorough coverage, combine approaches:

CI/CD: oasdiff or similar catches spec changes before merge
Post-deploy: FlareCanary or API Drift Alert catches spec-to-reality divergence
Alerting: Route breaking changes to PagerDuty/Slack, warnings to email

This layered approach catches drift at every stage — spec authoring, deployment, and runtime.

Getting Started

If you've never monitored for schema drift:

Inventory your external API dependencies. List every third-party API your application calls.
Identify the critical ones. Which APIs, if they changed silently, would cause the worst customer impact?
Start monitoring those. Even daily checks on 3-5 critical endpoints will catch most drift before it causes production incidents.
Set up alerts. Breaking changes → immediate notification. Warnings → daily digest.

Most drift monitoring tools offer free tiers. Start there, prove the value, then expand coverage.

API Schema Drift Detection Tools Compared (2026)

FlareCanary — Fri, 17 Apr 2026 04:01:50 +0000

The "API schema drift" problem has finally gotten enough attention that multiple tools exist to solve it. But they solve it in very different ways, at very different price points, with very different assumptions about your workflow.

This is a practical comparison of every tool I've found that handles schema drift detection as of April 2026. I'll be upfront: I built FlareCanary, so I'm biased — but I'll try to be honest about where each tool is strongest.

The approaches

There are four fundamentally different approaches to catching API schema drift:

Spec-to-spec diffing: Compare two versions of an OpenAPI spec. Catches changes between spec versions. Requires specs to exist.
Spec-to-reality monitoring: Make live API requests and compare against your OpenAPI spec. Catches when reality diverges from documentation.
Reality-to-reality monitoring: Make live API requests and compare against a learned baseline. No spec required.
Traffic-based detection: Observe real API traffic (via proxy or SDK) and detect anomalies. Requires infrastructure changes.

Most confusion in this space comes from people comparing tools across different approaches. An oasdiff comparison with an API Drift Alert comparison is apples to oranges.

Spec-to-spec diffing

oasdiff

What it does: Compares two OpenAPI spec files and reports breaking changes, deprecations, and additions. 300+ change detection rules.

Best for: CI pipelines. Run it as a GitHub Action to flag breaking changes in PRs before they merge.

Pricing: Free and open source. 1M+ downloads, 1,100+ GitHub stars.

Limitations: Requires two spec files. Cannot tell you if your live API matches your spec. If the spec is wrong (or doesn't exist), oasdiff can't help.

Verdict: The gold standard for spec diffing. If you maintain OpenAPI specs and want CI guardrails, start here. But it doesn't solve the "third-party API changed on me" problem.

PactFlow Drift (NEW — March 2026)

What it does: Generates test suites from OpenAPI specs using AI, then runs them in CI to verify implementations conform to the spec.

Best for: Teams already using PactFlow for contract testing who want to add spec conformance checks.

Pricing: Not yet published.

Limitations: CI-only — runs at deploy time, not continuously. Requires OpenAPI specs. Doesn't monitor external APIs you consume.

Verdict: Strong brand (PactFlow is well-known in contract testing). If you're in their ecosystem, it's a natural addition. They've also launched a PactFlow MCP Server for AI coding agent integration. But Drift itself is a testing tool, not a monitoring tool — it won't alert you when a third-party API changes between your deploys.

Bump.sh

What it does: API documentation platform with built-in changelog tracking. Upload or push spec files and it shows what changed between versions.

Pricing: $50/mo (Basic, 10 docs) to $250/mo (Pro, 30 docs).

Limitations: Documentation-first. Requires spec files to be pushed/uploaded. Cannot make live API calls.

Verdict: Great for API providers who want to communicate changes to consumers. Not helpful for consumers trying to detect changes providers didn't document.

Continuous monitoring (no spec required)

FlareCanary

What it does: Polls live API endpoints on a schedule and compares responses against either a learned baseline or an OpenAPI spec. Classifies drift by severity (breaking/warning/informational). Also monitors MCP server tool schemas.

Best for: Teams consuming third-party APIs they don't control, or anyone who wants continuous monitoring without maintaining perfect specs.

Pricing: Free (5 endpoints, daily checks) → $19/mo (25 endpoints, hourly) → $49/mo (100 endpoints, 15-min).

Limitations: No CI/CD integration (yet). No self-hosted option. Polling-based, so detection latency depends on check interval.

Disclosure: I built this. The multi-sample baseline learning (which reduces false positives from conditional fields) and severity classification are the main differentiators I'm most confident about.

API Drift Alert

What it does: Dedicated API drift monitoring platform. Detects schema changes in external APIs with severity-aware alert routing.

Best for: Teams with budget who need enterprise alert routing (PagerDuty integration, business-hours filtering for non-critical changes).

Pricing: $149/mo (15 APIs, 12-hour checks) → $349/mo (40 APIs, hourly) → $749/mo (100 APIs, 15-min). No free tier — 7-day trial only.

Limitations: Expensive entry point. No free tier means you can't evaluate meaningfully in 7 days. Has severity-aware alert routing (critical changes trigger immediate alerts, non-critical wait for business hours), but limited public technical details about baseline learning methodology.

Verdict: If your budget starts at $149/mo and you need PagerDuty integration, worth evaluating. For most small teams and individual developers, the pricing is a barrier.

APIShift

What it does: Live API monitoring with schema drift detection. Claims 5K+ APIs monitored, 50K+ daily checks.

Best for: Teams that want frequent checks at a reasonable price point.

Pricing: Free (5 APIs, hourly) → $29/mo (50 APIs, 5-min) → $99/mo (unlimited, real-time).

Limitations: Newer entrant. Recently added severity classification (LOW/MED/HIGH/CRITICAL), but less public information about baseline learning methodology.

Verdict: Competitive pricing, especially the $99/mo unlimited tier. The addition of severity classification brings it closer to feature parity with FlareCanary. If high-frequency checking is your priority, worth a look.

Rumbliq

What it does: Full monitoring platform (uptime, SSL, DNS, cron, API schema) with drift detection as one feature.

Best for: Teams that want a general-purpose monitoring dashboard with basic drift detection included.

Pricing: Free (25 monitors) → $12/mo → $29/mo → $69/mo.

Limitations: Drift detection is simple JSON response diffing — no severity classification, no multi-sample baselines, no spec comparison. Jack of all trades.

Verdict: If you need uptime + SSL + basic schema diffing in one tool and price is the priority, Rumbliq's free tier is generous. If schema drift is your primary concern, you'll want something deeper.

DiffMon (NEW — March 2026)

What it does: Monitors HTML pages and JSON API endpoints for structural changes. Classifies changes as schema change, value change, or mixed. Path-level severity analysis.

Best for: Teams that want to monitor both website content and API responses for changes in one tool.

Pricing: Free (3 monitors, 24-hour checks, 3-day history) → $14/mo (20 monitors, 30-min) → $49/mo (75 monitors, 5-min, Smart Schema Validation).

Limitations: Smart Schema Validation (the most useful drift detection feature) is locked behind the $49/mo Pro tier. No multi-sample baseline learning. No MCP monitoring. No OpenAPI spec comparison. The dual HTML+JSON positioning dilutes the API drift focus.

Verdict: Competitive pricing at the hobby tier ($14/mo for 20 monitors). But the meaningful drift detection features are Pro-only. If you specifically need API schema drift monitoring, the free tier's basic change detection is shallow compared to FlareCanary's severity classification at every tier.

API Detective (pre-launch)

What it does: Compares live API responses against published documentation for 200-500 popular public APIs. Creates auditable drift records with Wayback Machine references.

Best for: Developers monitoring popular public APIs (GitHub, Stripe, Twilio) without setup.

Pricing: Free (popular APIs, 24-hour checks) → Paid (TBD, custom APIs, 60-min checks). Still collecting signups.

Limitations: Not launched yet. Limited to pre-indexed APIs on free tier. No custom API monitoring without paid plan.

Verdict: Interesting approach with the pre-configured popular API angle. Worth watching when it launches.

Traffic-based detection

Treblle

What it does: API observability platform. SDK-based — instruments your own API to capture request/response data and detect anomalies.

Pricing: Free (1 API, 250K requests/mo) → $25-30/mo → $233-300/mo.

Limitations: Monitors your own APIs only (requires SDK installation). Cannot monitor third-party APIs you consume.

Verdict: Strong observability platform with major enterprise clients. But it solves a different problem — monitoring what you serve, not what you consume.

Tusk Drift

What it does: Records production API traffic and replays it against new code to detect regressions.

Pricing: Free (open source).

Limitations: Requires SDK instrumentation in your infrastructure. Testing tool, not monitoring tool.

MCP-specific tools

Bellwether

What it does: Snapshots MCP server tool schemas and diffs against baseline in CI/CD. Catches when AI agent tool definitions change.

Pricing: Free (open source).

Limitations: CI-only. Doesn't do continuous monitoring.

DriftCop (NEW — April 2026)

What it does: Open-source MCP security scanner that diffs MCP server manifests against signed golden baselines using SigStore. Detects tool schema drift, injection attacks, rug-pulls, and typosquats.

Best for: Security teams auditing MCP server supply chains for tampering.

Pricing: Free (open source).

Limitations: Security-focused, not general schema change monitoring. Designed for "was this MCP server tampered with?" not "did this API's schema evolve?" No continuous monitoring — it's a scanning tool.

Verdict: Interesting from a security perspective. Validates that MCP tool schema drift is increasingly viewed as a security vector, not just a reliability concern. Complementary to monitoring tools.

Specmatic

What it does: Contract-driven testing platform (OpenAPI, gRPC, GraphQL, AsyncAPI) with a recently launched MCP Auto-Test feature. Runs as a Docker container against MCP server endpoints, auto-generates test cases from declared schemas. Now actively marketing as "the first MCP schema drift detector" with aggressive content positioning ("MCP Servers Are Lying About Their Schemas"). Also launched an MCP server for AI coding agents and has a commercial enterprise module.

Pricing: Free (open source, MIT) → $10-50/user/mo (Enterprise: analytics, gRPC, GraphQL support).

Limitations: CI-only — point-in-time test execution, not continuous monitoring. No scheduled polling, no alerting between deployments, no drift history tracking. MCP is one feature of a much larger testing platform. Catches drift at deploy time, not when it happens.

Verdict: Strong CI-time validation for MCP server schemas, backed by a mature testing platform (366 GitHub stars). But the "first MCP schema drift detector" claim only applies to CI — continuous monitoring between deployments catches the drift that matters most (the overnight schema change that breaks your agent before your next deploy).

Note: FlareCanary monitors MCP server schemas continuously (not just in CI), which is currently unique among SaaS tools. The distinction matters: CI tools catch drift when you deploy, continuous monitoring catches drift when the other side changes — which is exactly the scenario that breaks production.

The comparison matrix

	Free tier	Entry price	Needs spec?	Continuous?	External APIs?	Severity levels?	MCP?
oasdiff	Yes (OSS)	Free	Yes	No (CI)	No	Yes	No
PactFlow Drift	Unknown	TBD	Yes	No (CI)	No	Unknown	No
FlareCanary	Yes (5)	$19/mo	No	Yes	Yes	Yes	Yes
API Drift Alert	No	$149/mo	No	Yes	Yes	Yes	No
APIShift	Yes (5)	$29/mo	No	Yes	Yes	Yes	No
DiffMon	Yes (3)	$14/mo	No	Yes	Yes	Yes (Pro)	No
Rumbliq	Yes (25)	$12/mo	No	Yes	Yes	No	No
API Detective	Yes (planned)	TBD	No	Yes	Yes	No	No
Treblle	Yes	$25/mo	No	Yes	No (own only)	No	No

Which should you use?

You maintain OpenAPI specs and want CI guardrails → Start with oasdiff (free). Add PactFlow Drift if you're in their ecosystem.

You consume third-party APIs and want continuous monitoring → FlareCanary (free tier, no spec required), API Drift Alert (if budget allows $149+/mo), or APIShift.

You want general monitoring with basic drift detection → Rumbliq (generous free tier, broad feature set).

You build AI agents using MCP → FlareCanary (only SaaS with continuous MCP monitoring) or Bellwether (CI-only, open source).

You need enterprise API observability → Treblle (own APIs) or Levo.ai ($2,500+/mo, includes AI security).

The right answer for most teams is a combination: oasdiff in CI for your own API specs, plus a continuous monitoring tool for the third-party APIs you depend on.

Full disclosure: I'm the builder of FlareCanary. I've tried to be fair to every tool listed here. If I got something wrong, leave a comment and I'll update.

41% of APIs Drift Within 30 Days — What the Data Says About API Reliability

FlareCanary — Wed, 15 Apr 2026 04:03:45 +0000

Most developers assume the APIs they depend on are stable. The data says otherwise.

KushoAI's State of Agentic API Testing 2026 report analyzed thousands of API integrations and found that 41% of APIs experience schema drift within 30 days. Within 90 days, that number climbs to 63%.

Let that sink in. If you're integrating with five external APIs, statistically two of them will change shape in the next month. And unless you're actively watching for it, you won't know until something breaks.

What counts as "drift"?

Schema drift is any change to what an API actually returns compared to what you expect. This isn't about downtime or 500 errors — those are loud and obvious. Drift is quieter:

A field changes from string to string | null
An integer ID becomes a UUID string
An enum gains a new value your switch statement doesn't handle
A nested object gets flattened or restructured
A field that was always present becomes conditional

The KushoAI report found that field additions account for 86% of drift events. Most providers consider new fields "non-breaking," but if your code uses strict deserialization, pattern matching, or type-checked interfaces, a new field can absolutely break things.

The failure hierarchy

Not all API failures are created equal. The report breaks down failure categories:

Category	Frequency
Auth/authorization failures	34%
Schema/validation errors	22%
Rate limiting	18%
Timeout/connectivity	15%
Business logic errors	11%

Schema drift is the #2 failure category after auth issues. And unlike auth failures (which are immediate and visible), schema drift is insidious. Your code might keep running with wrong data rather than failing cleanly.

Consider a real scenario: a payment provider changes their webhook payload, moving amount from an integer (cents) to a string ("19.99"). Your code parses it, JavaScript silently coerces the type, and suddenly you're processing transactions with incorrect amounts. No error. No alert. Just wrong numbers in your database.

Why testing doesn't catch it

The instinctive response is "our tests should catch this." But here's the problem: your tests mock the API response based on what it used to return. When the real API changes, your mocks don't update — so your tests pass while production fails.

// Your test mock (written 3 months ago)
const mockResponse = {
  user: { id: 123, name: "Alice", email: "alice@example.com" }
};

// What the API actually returns now
const realResponse = {
  user: { id: "usr_123", name: "Alice", email: "alice@example.com", role: "admin" }
};

// Your tests pass. Your production code breaks on id type change.

Contract testing tools like Pact catch drift at CI time — but only for APIs you control both sides of. For third-party APIs, you need something that checks the live response.

The AI agent multiplier

This problem is getting worse, not better. AI agents using MCP (Model Context Protocol) to discover and call tools face the same drift problem, but with an additional failure mode: silent adaptation.

When an LLM encounters a changed tool schema, it doesn't throw an error. It adapts — generating parameters based on the new schema without understanding the semantic change. A renamed parameter or a restructured input might produce syntactically valid but semantically wrong requests.

Nordic APIs' API Reliability Report 2026 found that AI API providers (OpenAI, Anthropic, Google) show the highest incident frequency across 215+ services they track. The APIs powering the AI ecosystem are themselves among the most volatile.

What proactive monitoring looks like

Detection is still overwhelmingly reactive. Most teams discover API drift through one of three signals:

Customer bug reports — the worst way to learn
CI failures — better, but only catches drift when you deploy
Error monitoring spikes — delayed signal that something already went wrong

Proactive monitoring means checking the API before your code runs against it. The approach:

Establish a baseline: Record what the API returns today — field names, types, structure, enums
Poll regularly: Make the same requests on a schedule (hourly, every 15 minutes, daily)
Compare and classify: Diff the response against the baseline. Not every change matters equally:
- Breaking: Field removed, type changed, required field became null
- Warning: New enum value, field became nullable, structure reorganized
- Informational: New optional field added, metadata changed
Alert on what matters: Notify for breaking changes immediately. Batch warnings for daily digest. Log informational changes.

This is the approach we built into FlareCanary. Point it at an endpoint, and it learns the response schema from multiple samples (reducing false positives from conditional fields). When something drifts, you get severity-classified alerts explaining exactly what changed.

No OpenAPI spec required — though if you have one, FlareCanary compares reality against the spec too.

A minimum viable monitoring setup

If you're not ready for a dedicated tool, here's a starting point:

#!/bin/bash
# Save a baseline
curl -s https://api.example.com/v1/users/me \
  -H "Authorization: Bearer $TOKEN" \
  | jq 'path(..) | map(tostring) | join(".")' | sort > baseline.txt

# Later: compare against baseline
curl -s https://api.example.com/v1/users/me \
  -H "Authorization: Bearer $TOKEN" \
  | jq 'path(..) | map(tostring) | join(".")' | sort > current.txt

diff baseline.txt current.txt

This catches structural changes (new/removed fields) but misses type changes, nullability shifts, and enum expansions. It's a start, not a solution.

The numbers don't lie

41% drift rate in 30 days. Schema errors as the #2 failure category. 86% of drift events are field additions that providers consider "non-breaking."

The gap between what API providers consider non-breaking and what actually breaks your code is where drift monitoring lives. If you're depending on external APIs — and in 2026, everyone is — the question isn't whether they'll change. It's whether you'll know before your users do.

FlareCanary monitors your API endpoints for schema drift and alerts you when responses change. Free tier: 5 endpoints, daily checks, no credit card. Built by developers who got burned by silent API changes one too many times.

Forem: FlareCanary

GitHub Just Retired Seven Org Security Fields — Your 'New Repo Hardening' Script Is Now A No-Op

The script that quietly stopped working

Why reads break differently from writes

The replacement is not a one-line swap

Why your tests didn't catch it

What to do this week

GitHub Just Removed merge_commit_sha From Pull Request Responses — Your Release Bot Is Probably Tagging null

What's removed and where

Why you don't get an error

The non-obvious replacement

Why your tests didn't catch it

What to do this week

Auth0 is about to start returning handshake_failure — how to tell if you're affected

The exact suites being removed

What the failure looks like in your stack

How to check whether you're affected

1. Ask OpenSSL directly

2. Check what your runtime offers

3. Point staging at the Canadian region

What to fix

The broader pattern

Minimum-viable fix for today

shopify.metafields returns undefined in my checkout extension after 2026-04 — here's why

What the failure looks like

Why it breaks silently

The exact migration

Where the old pattern is hiding in your code

The cart-metafields path, if you're writing fresh code

Detecting the break before your QA does

The broader pattern

Minimum-viable fix for today

DALL·E shuts down May 12 — the gpt-image-1 migration isn't the drop-in swap it looks like

What the shutdown looks like

Where dall-e-3 is probably pinned in your code

The request shape changed

The cost model flipped

The pattern this fits

What actually catches this

Minimum-viable fix for this one

HubSpot Contacts v1 Goes Dark on April 30 — And the Worst Part Is the Endpoints That Keep Working

What returns 404 on April 30

The endpoints that bite — they keep returning 200

The fix path: v3 Lists API

Why your tests didn't catch it

What to do this week

Twilio Is Killing api.de1.twilio.com on April 28 — And the Regional Domains Never Actually Routed Regionally

What actually breaks

The two fix paths

Path 1: You don't actually need regional processing

Path 2: You actually need regional processing

Why your tests didn't catch it

The pattern across APIs

How to not be surprised by the next one

claude-3-haiku-20240307 just started returning errors — here's what happened

What the failure looks like

Where the stale model ID is probably hiding

What to replace it with

The pattern this fits

Monitoring model deprecation as an API change

Minimum-viable fix for today

CI Tests Won't Save You from MCP Schema Drift

The MCP drift problem has two halves

Why LLMs make this worse

What CI testing actually catches

What CI testing misses

The monitoring gap

What continuous MCP monitoring looks like

CI and continuous monitoring are complementary

The Complete Guide to API Schema Drift Monitoring in 2026

What Is API Schema Drift?

Why Schema Drift Matters More in 2026

1. External API Dependencies Are Everywhere

2. AI Agents Depend on Tool Schemas

3. Microservice Sprawl

Approaches to Drift Detection

Spec-to-Spec Diffing (CI-Time)

Spec-to-Reality Monitoring (Runtime)

Traffic-Based Detection (Passive)

MCP/AI Tool Schema Monitoring

Where `dall-e-3` is probably pinned in your code