The latest articles on Forem by specshield (@specshield_a17bcb1ca84675). https://forem.com/specshield_a17bcb1ca84675 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3883005%2F3801d36c-f381-4627-8633-e340cd886945.png https://forem.com/specshield_a17bcb1ca84675 en specshield Thu, 16 Apr 2026 18:15:42 +0000 https://forem.com/specshield_a17bcb1ca84675/we-broke-production-due-to-an-api-change-heres-how-i-fixed-it-42en https://forem.com/specshield_a17bcb1ca84675/we-broke-production-due-to-an-api-change-heres-how-i-fixed-it-42en <p>In a microservices architecture, one small API change can break multiple downstream systems.</p> <p>I learned this the hard way.</p> <h2> 💥 The Problem </h2> <p>We had a backend service that modified an API:</p> <ul> <li>A field was removed</li> <li>Response structure slightly changed</li> </ul> <p>Everything passed:</p> <ul> <li>Unit tests ✅</li> <li>Integration tests ✅</li> <li>Code review ✅</li> </ul> <p>But in production:<br> 👉 Another service broke.</p> <p>Why?</p> <p>Because we never enforced API contracts.</p> <h2> 🔍 Root Cause </h2> <p>We relied on:</p> <ul> <li>Human review</li> <li>Documentation</li> <li>Assumptions of backward compatibility</li> </ul> <p>But we were missing:</p> <p>👉 <strong>Automated contract validation</strong></p> <h2> 💡 The Solution </h2> <p>I built a simple approach:</p> <ol> <li>Store OpenAPI spec (baseline)</li> <li>Compare new spec during CI</li> <li>Fail build if breaking changes detected</li> </ol> <h2> ⚙️ Implementation </h2> <p>Using a CLI approach:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>npx specshield compare old.yaml new.yaml </code></pre> </div> <p>CI Integration:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight yaml"><code><span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Check API breaking changes</span> <span class="na">run</span><span class="pi">:</span> <span class="s">npx specshield compare</span> </code></pre> </div> <h2> 🚨 What counts as breaking? </h2> <ul> <li>Removing endpoints</li> <li>Removing fields</li> <li>Changing types</li> <li>Making optional → required</li> </ul> <h2> 🧠 Key Learnings </h2> <ul> <li>API contracts should be treated like code</li> <li>CI is the best place to enforce safety</li> <li>Even small changes can have big impact</li> </ul> <h2> 🔮 Future Improvements </h2> <ul> <li>Consumer-driven contracts (Pact-style)</li> <li>Better diff visualization</li> <li>Integration with GitHub PR checks</li> </ul> <h2> 🤔 How do you handle this? </h2> <ul> <li>Do you use OpenAPI validation?</li> <li>Pact?</li> <li>Custom scripts?</li> </ul> <p>Curious to learn how others solve this problem.</p> <p><a href="https://specshield.io/" rel="noopener noreferrer">https://specshield.io/</a></p> api cicd microservices testing