<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>Forem: Matthew Gladding</title>
    <description>The latest articles on Forem by Matthew Gladding (@glad_labs).</description>
    <link>https://forem.com/glad_labs</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3860296%2Fe75c4ed2-993e-403f-a24b-dd72bc83c85d.png</url>
      <title>Forem: Matthew Gladding</title>
      <link>https://forem.com/glad_labs</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://forem.com/feed/glad_labs"/>
    <language>en</language>
    <item>
      <title>Addressing Hallucinations and Security in Open-Source LLM Agents</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Fri, 22 May 2026 12:23:55 +0000</pubDate>
      <link>https://forem.com/glad_labs/addressing-hallucinations-and-security-in-open-source-llm-agents-3bam</link>
      <guid>https://forem.com/glad_labs/addressing-hallucinations-and-security-in-open-source-llm-agents-3bam</guid>
      <description>&lt;p&gt;Large Language Model (LLM) agents offer exciting possibilities for automation - imagine a system that proactively resolves IT incidents or instantly summarizes complex legal documents. However, realizing this potential in production demands a focus on reliability, security, and factual accuracy. This post focuses on the critical issues of hallucinations and security when working with open-source LLM agents, targeted towards technical professionals.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Hallucination Problem
&lt;/h2&gt;

&lt;p&gt;LLM agents, by their nature, generate text. While impressive, this generation isn't always grounded in reality. Hallucinations - where the agent confidently states incorrect or nonsensical information - are a major concern. These can range from subtly inaccurate details to completely fabricated scenarios. Mitigating hallucinations requires a multi-faceted strategy.&lt;/p&gt;

&lt;h3&gt;
  
  
  Retrieval-Augmented Generation (RAG)
&lt;/h3&gt;

&lt;p&gt;One of the most effective techniques is Retrieval-Augmented Generation (RAG). Instead of relying solely on the LLM's pre-trained knowledge, RAG involves retrieving relevant information from a trusted knowledge base and providing it as context to the LLM.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Example RAG pipeline (simplified)
&lt;/span&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;get_relevant_documents&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;knowledge_base&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# Search knowledge base for documents related to the query
&lt;/span&gt;    &lt;span class="n"&gt;documents&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;knowledge_base&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;documents&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generate_response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;documents&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;llm&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;context&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;documents&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Answer the question based on the following context:&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="s"&gt;Question: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;llm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Prompt Engineering &amp;amp; Constraints
&lt;/h3&gt;

&lt;p&gt;Carefully crafting prompts can also reduce hallucinations. Clear, specific instructions and constraints help guide the LLM towards more accurate responses. For example, explicitly requesting the agent to state its sources or to admit when it doesn't know the answer.&lt;/p&gt;

&lt;h2&gt;
  
  
  Security Considerations
&lt;/h2&gt;

&lt;p&gt;Open-source LLM agents introduce unique security challenges. Because you're responsible for the entire stack, vulnerabilities can arise at multiple layers.&lt;/p&gt;

&lt;h3&gt;
  
  
  Prompt Injection Attacks
&lt;/h3&gt;

&lt;p&gt;Prompt injection attacks occur when a malicious user manipulates the prompt to bypass intended safeguards or extract sensitive information. For instance, a user might craft a prompt that instructs the agent to ignore previous instructions and reveal its internal configuration.&lt;/p&gt;

&lt;h3&gt;
  
  
  Data Privacy &amp;amp; Sensitive Information
&lt;/h3&gt;

&lt;p&gt;LLM agents often process sensitive data. Protecting this data requires careful consideration of data storage, access control, and encryption. Avoid storing sensitive information directly within the LLM's context window.&lt;/p&gt;

&lt;h3&gt;
  
  
  Dependency Management
&lt;/h3&gt;

&lt;p&gt;Open-source projects rely on numerous dependencies. Regularly updating these dependencies is crucial to patch security vulnerabilities. Tools like &lt;code&gt;pip-audit&lt;/code&gt; or &lt;code&gt;dependabot&lt;/code&gt; can automate this process.&lt;/p&gt;

</description>
      <category>security</category>
      <category>agents</category>
      <category>prompt</category>
      <category>hallucinations</category>
    </item>
    <item>
      <title>What we shipped on 2026-05-21</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Thu, 21 May 2026 13:01:02 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-on-2026-05-21-31jn</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-on-2026-05-21-31jn</guid>
      <description>&lt;p&gt;The alert firing felt like a failure until we realized it was measuring our success. The Content Quality Drop alert averaged &lt;code&gt;quality_score&lt;/code&gt; across &lt;em&gt;all&lt;/em&gt; tasks, so rejected items dragged the average down to 69.36. When we filtered the query to only count &lt;code&gt;status IN ('published','approved')&lt;/code&gt;, the true average jumped to 83.30. We updated the alert to ignore rejected tasks so it actually measures the quality of what ships to users.&lt;/p&gt;

&lt;p&gt;We also fixed the video service path issue that was silently killing video regeneration. The original diagnosis focused on file deletion, but the real bug was a stale hardcoded container-path prefix. The container user changed from &lt;code&gt;root&lt;/code&gt; to &lt;code&gt;appuser&lt;/code&gt;, breaking the &lt;code&gt;/root/.poindexter&lt;/code&gt; replacement, so paths passed through unchanged. We now derive the prefix dynamically to match the logic in the &lt;code&gt;featured-image&lt;/code&gt; path.&lt;/p&gt;

&lt;p&gt;Our Discord notifications were also broken. The &lt;code&gt;discord_ops&lt;/code&gt; row was the only one storing the full URL instead of using &lt;code&gt;secret_key_ref&lt;/code&gt;, so Matt's webhook rotation on the 15th resulted in "Unknown Webhook" errors. We updated the handler to resolve via &lt;code&gt;secret_key_ref&lt;/code&gt; and relaxed the migration constraint so the URL swap could actually apply.&lt;/p&gt;

&lt;p&gt;We closed the orphaned media assets gap. Over 442 &lt;code&gt;media_assets&lt;/code&gt; rows for &lt;code&gt;featured_image&lt;/code&gt; had NULL &lt;code&gt;post_id&lt;/code&gt; because the recorder fired before the post was created. We now persist &lt;code&gt;task_id&lt;/code&gt; at insert and back-stamp &lt;code&gt;post_id&lt;/code&gt; after publish to link these rows properly.&lt;/p&gt;

&lt;p&gt;Finally, we stripped all private GitHub URLs from the dev diary. The LLM was emitting links that 404'd for public readers. We updated the prompt and ran the regex fixes to replace them with a plain text pointer to the public Poindexter repo.&lt;/p&gt;

&lt;p&gt;From here, the feed runs cleaner and the alerts reflect actual content quality rather than rejection noise.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>backend</category>
      <category>devjournal</category>
      <category>docker</category>
      <category>monitoring</category>
    </item>
    <item>
      <title>The Expanding Role of Open-Source LLM Agents in Autonomous Workflows</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Wed, 20 May 2026 23:02:50 +0000</pubDate>
      <link>https://forem.com/glad_labs/the-expanding-role-of-open-source-llm-agents-in-autonomous-workflows-531c</link>
      <guid>https://forem.com/glad_labs/the-expanding-role-of-open-source-llm-agents-in-autonomous-workflows-531c</guid>
      <description>&lt;p&gt;The autonomous workflow market is undergoing a period of rapid innovation, and a notable trend is the increasing prominence of open-source Large Language Model (LLM) agents. While proprietary models were early leaders in this space, advancements in open-source LLMs, combined with growing demands for customization and control, are reshaping adoption patterns. This shift extends beyond purely economic considerations; it represents a fundamental change in how developers and organizations approach automation. The following details the factors driving this trend, the challenges faced, and the projected landscape of autonomous workflows leveraging open-source LLM agents.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Rise of Open-Source LLMs and Agent Frameworks
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fpub-1432fdefa18e47ad98f213a8a2bf14d5.r2.dev%2Fimages%2Finline%2Fae9b7f79a51e.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fpub-1432fdefa18e47ad98f213a8a2bf14d5.r2.dev%2Fimages%2Finline%2Fae9b7f79a51e.png" alt="Close-up shot of a computer screen displaying complex code with a blurred developer's hands typing in the background." width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;For much of the early LLM landscape, access to powerful models involved reliance on API-driven, proprietary solutions. However, the open-source community has dramatically altered this dynamic. Initiatives like those found on the &lt;a href="https://huggingface.co/models" rel="noopener noreferrer"&gt;Hugging Face Model Hub&lt;/a&gt; have fostered a proliferation of capable open-source LLMs, increasingly challenging the performance of their closed-source counterparts. The &lt;a href="https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard" rel="noopener noreferrer"&gt;LLM Leaderboard&lt;/a&gt; offers a continually updated resource for comparing these models based on various benchmarks.&lt;/p&gt;

&lt;p&gt;Recent progress in open-source LLMs is particularly noticeable in coding tasks. Analysis indicates the performance gap between open and closed-source models in this domain is narrowing, and while challenges remain, particularly in complex reasoning, the gap is becoming less pronounced. This capability is vital for constructing complex autonomous workflows that require code generation, scripting, or interaction with software systems. Beyond the base models themselves, the emergence of agent frameworks built around these LLMs is accelerating development. These frameworks provide tools and abstractions to simplify the creation of agents capable of planning, reasoning, and acting autonomously.&lt;/p&gt;

&lt;h3&gt;
  
  
  Desktop-First Agents and the Benefits of Local Execution
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fpub-1432fdefa18e47ad98f213a8a2bf14d5.r2.dev%2Fimages%2Finline%2F0ecb1679e2df.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fpub-1432fdefa18e47ad98f213a8a2bf14d5.r2.dev%2Fimages%2Finline%2F0ecb1679e2df.png" alt="A clean, modern desk setup with a powerful desktop computer, a sleek monitor displaying a simple agent interface..." width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A key driver behind the adoption of open-source LLM agents is the move towards local, desktop-first execution. Traditionally, running LLM-powered applications required constant connectivity to cloud-based APIs, introducing latency and potential privacy concerns. Industry observers note a significant shift towards deploying agents directly on user devices.&lt;/p&gt;

&lt;p&gt;This "local-first" approach offers a range of benefits. Reducing dependence on external services lowers operational costs and improves responsiveness. It also enhances data privacy, as sensitive information remains within the user's control. Furthermore, local execution enables operation in environments with limited or no internet access, expanding the potential applications of autonomous workflows. Tools and techniques for safely running these agents on personal machines are becoming more mature, making local deployment increasingly accessible. Security best practices, such as sandboxing and resource limitations, are crucial considerations for responsible local deployment.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Practical Examples of Local Agent Applications:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  &lt;strong&gt;Personal Knowledge Management:&lt;/strong&gt; A locally run agent can process and summarize personal notes, documents, and emails, providing quick access to relevant information.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Offline Code Generation:&lt;/strong&gt; Developers can utilize a local agent to generate code snippets or complete functions even without an internet connection.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Automated Report Generation:&lt;/strong&gt; Agents can process local data sources to generate customized reports on demand, without relying on cloud services.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The Appeal of Vertical AI Agents and Customization
&lt;/h3&gt;

&lt;p&gt;The growing popularity of vertical LLM agents is further fueling the open-source trend. These agents are specifically designed for a particular task or industry, offering a focused and optimized solution. As LLMs become more sophisticated, the potential for specialized vertical AI agents is becoming increasingly apparent [&lt;a href="https://www.youtube.com/watch?v=eBVi_sLaYsc" rel="noopener noreferrer"&gt;https://www.youtube.com/watch?v=eBVi_sLaYsc&lt;/a&gt;]. Open-source models are particularly well-suited for this application because developers can fine-tune and customize them to meet specific vertical needs.&lt;/p&gt;

&lt;p&gt;While proprietary LLMs are also customizable through techniques like prompt engineering and fine-tuning, open-source models offer a greater degree of control and flexibility. The capacity to tailor an open-source agent to a specific use case is a compelling value proposition. This customization goes beyond simply fine-tuning the model; it also involves integrating the agent with specific data sources, tools, and APIs relevant to the target vertical.&lt;/p&gt;

&lt;h3&gt;
  
  
  Addressing Challenges: Hallucinations, Observability, and Security
&lt;/h3&gt;

&lt;p&gt;While open-source LLM agents offer many advantages, they also present challenges. One persistent issue is the potential for "hallucinations"--instances where the model generates inaccurate or nonsensical information. Autonomous agents acting on these hallucinations can lead to errors or unintended consequences. However, techniques for mitigating these risks are emerging. Solutions such as hallucination watermarking and edge-based filtering, designed to detect and prevent destructive AI actions, are gaining traction.&lt;/p&gt;

&lt;p&gt;Runtime observability is another critical area of focus. Understanding how an agent is operating and identifying potential issues requires robust monitoring and debugging tools. Open-source projects increasingly emphasize observability layers, providing developers with the insights needed to build and maintain complex autonomous systems. Security is paramount, given that autonomous agents can take actions on behalf of users. Robust authentication, authorization, and input validation mechanisms are crucial to prevent malicious actors from exploiting vulnerabilities in the agent or the underlying LLM.&lt;/p&gt;

&lt;h3&gt;
  
  
  Memory and Context Management for Complex Workflows
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fpub-1432fdefa18e47ad98f213a8a2bf14d5.r2.dev%2Fimages%2Finline%2F3fb740a04247.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fpub-1432fdefa18e47ad98f213a8a2bf14d5.r2.dev%2Fimages%2Finline%2F3fb740a04247.png" alt="Illustration for Why open-source LLM agents are eating the autonomous workflow market in 2026" width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The ability of an agent to maintain context over extended interactions is critical for complex workflows. Early LLM agents were limited by a relatively small context window, hindering their ability to process lengthy documents or engage in multi-turn conversations. However, innovative techniques are emerging to overcome this limitation. Methods for augmenting an agent's memory with external knowledge sources are becoming increasingly sophisticated, as discussed in &lt;a href="https://www.gladlabs.io/posts/breaking-the-memory-wall-how-to-give-any-open-sour-346f4919" rel="noopener noreferrer"&gt;Breaking the Memory Wall: How to Give Any Open-Source Agent Claude-Level Recall&lt;/a&gt;. Retrieval-Augmented Generation (RAG) is a particularly promising approach, allowing agents to access and utilize vast amounts of information stored in external databases or knowledge graphs. This capability significantly expands the scope of tasks that autonomous agents can handle.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Techniques for Improving Memory and Context:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  &lt;strong&gt;Vector Databases:&lt;/strong&gt; Storing embeddings of documents or knowledge snippets in vector databases allows for efficient semantic search and retrieval.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Long-Term Memory Systems:&lt;/strong&gt; Implementing mechanisms to store and retrieve relevant information from past interactions, enabling agents to maintain context over extended periods.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Knowledge Graphs:&lt;/strong&gt; Representing information as a network of entities and relationships provides a structured way for agents to reason and infer new knowledge.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The Future Landscape of Open-Source LLM Agents
&lt;/h3&gt;

&lt;p&gt;Looking ahead to 2026 and beyond, the trend towards open-source LLM agents is expected to continue its upward trajectory. The combination of improved model performance, local execution capabilities, and a growing ecosystem of tools and frameworks is creating a powerful alternative to proprietary solutions. Several factors suggest this momentum will persist. The cost of accessing proprietary LLM APIs can be substantial, particularly for high-volume applications. Open-source models offer greater long-term cost predictability, requiring investment in infrastructure and expertise, but potentially reducing overall operational expenses.&lt;/p&gt;

&lt;p&gt;Moreover, the open-source community fosters collaboration and innovation, accelerating the pace of development. The ability to fully customize and control the underlying model is another key driver. Businesses can tailor open-source agents to their specific needs without being constrained by the limitations of a closed-source platform. This level of flexibility is particularly valuable for organizations operating in regulated industries or handling sensitive data.&lt;/p&gt;

&lt;p&gt;As a result, industry observers anticipate that open-source LLM agents will become increasingly prevalent in a wide range of autonomous workflow applications, from customer service and data analysis to software development and scientific research. The democratization of AI through open-source LLM agents empowers a broader range of developers and organizations to harness the power of autonomous workflows, fostering innovation and driving economic growth. However, realizing this democratization will require addressing the initial investment in infrastructure and specialized expertise, but the long-term benefits of customization and control are significant. This shift promises a future where automation is more accessible, customizable, and aligned with the needs of individuals and businesses alike.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://huggingface.co/models" rel="noopener noreferrer"&gt;https://huggingface.co/models&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard" rel="noopener noreferrer"&gt;https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>agents</category>
      <category>opensource</category>
      <category>autonomous</category>
    </item>
    <item>
      <title>What we shipped on 2026-05-20</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Wed, 20 May 2026 18:49:58 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-on-2026-05-20-4737</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-on-2026-05-20-4737</guid>
      <description>&lt;p&gt;Every publish with tags had been silently 500'ing since the &lt;code&gt;posts.tag_ids → post_tags&lt;/code&gt; split. We caught this when investigating alert #293 on post &lt;code&gt;dcd86ea6-9d8e-4841-9543-a46a55d96283&lt;/code&gt;. The &lt;code&gt;create_post&lt;/code&gt; insert was using &lt;code&gt;unnest($2::text[])&lt;/code&gt; on a &lt;code&gt;uuid&lt;/code&gt; column, causing a Postgres error that rolled back the transaction mid-flight. We fixed the cast to &lt;code&gt;uuid[]&lt;/code&gt; and added a regression test (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/509" rel="noopener noreferrer"&gt;PR #509&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;That database error exposed a drift in the reconciliation job. It was still filtering by slug prefix, a pattern from a previous refactor that it missed. We updated &lt;code&gt;media_reconciliation.run&lt;/code&gt; to check &lt;code&gt;cardinality(media_to_generate) &amp;gt; 0&lt;/code&gt; first, so it stops regenerating media for posts that explicitly opted out (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/508" rel="noopener noreferrer"&gt;PR #508&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;The initial publish path also needed fixing. PR #510 notes that &lt;code&gt;publish_post_from_task&lt;/code&gt; ignored the &lt;code&gt;media_to_generate&lt;/code&gt; array, spawning podcasts for posts with empty arrays while &lt;code&gt;fire_post_distribution_hooks&lt;/code&gt; gated correctly. We split the logic into per-type checks to match the hooks (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/510" rel="noopener noreferrer"&gt;PR #510&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;We also cleaned house on the test suite. PR #511 removed a stale &lt;code&gt;xfail&lt;/code&gt; marker from &lt;code&gt;test_threshold_triggers_summary_with_llm_payload&lt;/code&gt; because the dispatcher payload had already been restored to emit &lt;code&gt;summary_request&lt;/code&gt; and &lt;code&gt;annotations&lt;/code&gt; fields. The test now passes cleanly (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/511" rel="noopener noreferrer"&gt;PR #511&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;On the release front, the CHANGELOG was leaking private keys into the public mirror, causing release-please to wedge. We added a line-redaction pass in &lt;code&gt;scripts/sync-to-github.sh&lt;/code&gt; to strip out &lt;code&gt;mercury_&lt;/code&gt; and &lt;code&gt;nightrider&lt;/code&gt; patterns before the leak guard ran (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/506" rel="noopener noreferrer"&gt;PR #506&lt;/a&gt;). &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/507" rel="noopener noreferrer"&gt;v0.10.1&lt;/a&gt; shipped today.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/509" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/509&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/508" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/508&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/510" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/510&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/511" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/511&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/506" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/506&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/507" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/507&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>backend</category>
      <category>database</category>
      <category>devjournal</category>
      <category>postgres</category>
    </item>
    <item>
      <title>Beyond the Hustle: A Technical Professional's Guide to Recognizing Burnout</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Wed, 20 May 2026 12:40:06 +0000</pubDate>
      <link>https://forem.com/glad_labs/beyond-the-hustle-a-technical-professionals-guide-to-recognizing-burnout-1j3h</link>
      <guid>https://forem.com/glad_labs/beyond-the-hustle-a-technical-professionals-guide-to-recognizing-burnout-1j3h</guid>
      <description>&lt;p&gt;The image of the dedicated developer is... well, a lot of images. Often it's romanticized - hunched over a glowing screen, fueled by caffeine, relentlessly coding. Instagram might show late nights and "grind culture," but that's not the &lt;em&gt;real&lt;/em&gt; experience of burnout. For those of us deep in the technical trenches, it manifests very differently.&lt;/p&gt;

&lt;p&gt;It's not a sudden crash. It's a slow erosion, starting with the things that used to &lt;em&gt;energize&lt;/em&gt; you. As a software developer building an AI/ML-focused business, with a workstation capable of handling demanding workloads (Ryzen 9 9950X3D + RTX 5090 - ), I initially thought I was built for long hours. I was wrong. The issue wasn't simply the &lt;em&gt;amount&lt;/em&gt; of time, but the increasing cognitive demands of the work itself.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;It starts with technical debt.&lt;/strong&gt; Not just &lt;em&gt;knowing&lt;/em&gt; you have it, but the feeling that addressing it is increasingly difficult. It's the creeping realization that every new feature adds another layer of complexity to a fragile system. You start with a simple script, a quick fix, a "works on my machine" moment... and then it multiplies. That initial convenience turns into a web of dependencies you dread touching. It's not about the &lt;em&gt;amount&lt;/em&gt; of work, it's the feeling that the work isn't &lt;em&gt;moving&lt;/em&gt; you forward.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Then comes the operational weight.&lt;/strong&gt; We're building increasingly complex systems. The expectation isn't just "getting something working" it's building something that &lt;em&gt;lasts&lt;/em&gt;. This means dealing with CI/CD pipelines, database migrations without downtime, and the constant pressure to scale. You're not just coding, you're building and &lt;em&gt;maintaining&lt;/em&gt; the infrastructure around that code, and increasingly finding yourself reacting to issues as they arise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The AI landscape adds another layer.&lt;/strong&gt; We're entering a new era of AI orchestration. The promise of AI-powered tools is immense, but it also creates a pressure to adopt &lt;em&gt;everything&lt;/em&gt;. The illusion that simply &lt;em&gt;having&lt;/em&gt; AI solves problems is pervasive. You become a manager of AI agents as much as a developer, and that adds significant cognitive load.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Here's what it feels like, specifically:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  &lt;strong&gt;Constant context switching:&lt;/strong&gt; Jumping between addressing urgent production issues, designing new features, and evaluating the latest AI tools.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Sleep disturbances:&lt;/strong&gt; Light, restless sleep, and multiple other entries in the same dream log. You're mentally replaying code reviews and potential outages instead of resting, which then leads to...&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Difficulty concentrating:&lt;/strong&gt; ...a fuzzy brain and inability to focus, even when you &lt;em&gt;do&lt;/em&gt; sit down to code.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Loss of curiosity:&lt;/strong&gt; The things that used to excite you about technology - learning a new language, exploring a new framework - feel draining.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Decreased code quality:&lt;/strong&gt; You start taking shortcuts, prioritizing speed over maintainability, often due to feeling overwhelmed by the workload.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;A sense of futility:&lt;/strong&gt; The feeling that, no matter how much you do, you're not making a meaningful impact. Like you're building in a vacuum.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Burnout isn't a badge of honor. It's a signal that something is fundamentally broken. It's time to reassess, prioritize, and remember why you started building in the first place. It's time to move from just &lt;em&gt;doing&lt;/em&gt; the work, to building a system that &lt;em&gt;supports&lt;/em&gt; you while you do it.&lt;/p&gt;

</description>
      <category>building</category>
      <category>work</category>
      <category>feeling</category>
      <category>code</category>
    </item>
    <item>
      <title>What we shipped on 2026-05-19</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Tue, 19 May 2026 13:01:18 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-on-2026-05-19-29gb</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-on-2026-05-19-29gb</guid>
      <description>&lt;p&gt;We discovered our kill switch for the HTTP probe was failing open, not closed, on DB uncertainty. The &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/478" rel="noopener noreferrer"&gt;alert_events&lt;/a&gt; showed ten identical warnings in a 24-hour window even though the setting was disabled five days ago. The &lt;code&gt;_read_bool&lt;/code&gt; helper was treating missing rows and decrypt failures as "false" when it should have treated them as unknown. We added a unique sentinel for the default to surface that uncertainty and opted into &lt;code&gt;fail_closed=True&lt;/code&gt; so the probe stays disabled if we can't read the setting.&lt;/p&gt;

&lt;p&gt;While triaging that, we found the every-four-hour backfill jobs were treating &lt;code&gt;dev_diary&lt;/code&gt; posts like regular content, accumulating 10 podcasts and 8 videos. We initially tried filtering on &lt;code&gt;media_to_generate&lt;/code&gt;, but every post in the schema has an empty array there, so that filter would have killed all media generation. The slug check -- &lt;code&gt;slug NOT LIKE 'what-we-shipped%'&lt;/code&gt; -- excludes dev_diary without touching the rest of the pipeline, so &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/481" rel="noopener noreferrer"&gt;PR #481&lt;/a&gt; stops dev_diary from accumulating podcasts and videos.&lt;/p&gt;

&lt;p&gt;We also updated the surface. PR #473 repointed comments in the CLI and brain modules to &lt;code&gt;0000_baseline&lt;/code&gt; (files folded into that migration on 2026-05-08), and PR #474 regenerated the app-settings documentation. The RSS feeds finally passed the Spotify validation check: we added the missing &lt;code&gt;&amp;lt;itunes:image&amp;gt;&lt;/code&gt; from a &lt;code&gt;podcast_cover_url&lt;/code&gt; app_settings row, fixed the &lt;code&gt;&amp;lt;atom:link rel="self"&amp;gt;&lt;/code&gt; to the public route, and populated the owner details. R2 re-uploaded the feed with these corrections, and the worker restarted to load the new code.&lt;/p&gt;

&lt;p&gt;The &lt;code&gt;sanitize-html&lt;/code&gt; bump to 2.17.4 closes a security gap with the xmp tag, and the &lt;code&gt;ollama_client&lt;/code&gt; resilience suite picked up nine new tests for stream generation and edge cases. We have six items left from the audit, but the noisy defaults and the broken feeds are fixed.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/478" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/478&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/481" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/481&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>backend</category>
      <category>devjournal</category>
      <category>monitoring</category>
      <category>sre</category>
    </item>
    <item>
      <title>What we shipped on 2026-05-18</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Mon, 18 May 2026 22:05:44 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-on-2026-05-18-1d0g</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-on-2026-05-18-1d0g</guid>
      <description>&lt;p&gt;We spent the day reinforcing the reliability of our build and test suite, prioritizing stability over new features. The most concrete win came from &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/471" rel="noopener noreferrer"&gt;PR #471&lt;/a&gt;, where we expanded coverage of &lt;code&gt;_telegram_creds&lt;/code&gt; to 13 tests and patched a pre-existing import-order bug where fixtures weren't loading correctly. The new tests explicitly guard against partial cache poisoning, ensure empty strings and &lt;code&gt;None&lt;/code&gt; are treated as identical, and validate the specific &lt;code&gt;min=1, max=1, timeout=2.0&lt;/code&gt; configuration for connection pools. Behind the scenes, this defensive testing was paired with a wave of dependency churn to keep the stack modern. We bumped &lt;code&gt;prefect&lt;/code&gt; to 3.7.1 (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/469" rel="noopener noreferrer"&gt;PR #469&lt;/a&gt;), updated &lt;code&gt;python-multipart&lt;/code&gt; to 0.0.29 (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/470" rel="noopener noreferrer"&gt;PR #470&lt;/a&gt;), and upgraded &lt;code&gt;datasets&lt;/code&gt; to 4.8.5 to fix decoding issues (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/443" rel="noopener noreferrer"&gt;PR #443&lt;/a&gt;). On the infrastructure front, we updated the CI runner tools, moving &lt;code&gt;actions/checkout&lt;/code&gt; to v6 (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/464" rel="noopener noreferrer"&gt;PR #464&lt;/a&gt;) and &lt;code&gt;actions/setup-python&lt;/code&gt; to v6.2 (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/466" rel="noopener noreferrer"&gt;PR #466&lt;/a&gt;), ensuring our pipelines have the latest security patches and Node compatibility. It was a quiet day of tightening the screws--upgrading Starlette to its first stable 1.0.0 release (&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/448" rel="noopener noreferrer"&gt;PR #448&lt;/a&gt;) and cleaning up linting rules--but it feels like the right kind of work for a platform that needs to scale without breaking.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/471" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/471&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/469" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/469&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/470" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/470&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/443" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/443&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/464" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/464&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/466" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/466&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/448" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/448&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
    </item>
    <item>
      <title>What we shipped on 2026-05-17</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Sun, 17 May 2026 23:33:12 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-on-2026-05-17-5506</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-on-2026-05-17-5506</guid>
      <description>&lt;p&gt;The biggest lift today was fixing the sequential choke point in the scene visuals stage. We introduced bounded concurrency via &lt;code&gt;asyncio.Semaphore&lt;/code&gt; in &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/456" rel="noopener noreferrer"&gt;PR #456&lt;/a&gt; so SDXL wouldn't oversubscribe the GPU, which was killing wall-clock time on long-form content. We added a new app_settings key &lt;code&gt;video_scene_visuals_max_concurrent&lt;/code&gt; with a default of &lt;strong&gt;1&lt;/strong&gt; to preserve the existing safety behavior, but now operators with VRAM headroom can push the cap and resolve scenes in parallel.&lt;/p&gt;

&lt;p&gt;The implementation includes a fresh migration to seed the knob so a bare DB documents the change immediately. We also added a &lt;code&gt;metadata.elapsed_s&lt;/code&gt; field to capture per-scene wall-clock data and a new &lt;code&gt;video.scene_visual_resolved&lt;/code&gt; audit_log row to give us the timing metrics needed to decide if the cap is actually worth bumping. Misconfigured values for &lt;strong&gt;0&lt;/strong&gt; or &lt;strong&gt;-1&lt;/strong&gt; clamp to &lt;strong&gt;1&lt;/strong&gt;, so a typo won't deadlock the stage.&lt;/p&gt;

&lt;p&gt;On the operational side, we tackled the decaying reference page that has 700+ rows. With &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/453" rel="noopener noreferrer"&gt;PR #453&lt;/a&gt;, we turned the manual audit script into a nightly CI job. The &lt;code&gt;scripts/regen-app-settings-doc.py&lt;/code&gt; now honors &lt;code&gt;REGEN_DATE_OVERRIDE&lt;/code&gt; to pin the banner stamp to the commit date, ensuring the output is byte-identical to the source state and preventing spurious PRs.&lt;/p&gt;

&lt;p&gt;We also backfilled ten edge-case tests for the &lt;code&gt;LiteLLMProvider&lt;/code&gt; class in &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/455" rel="noopener noreferrer"&gt;PR #455&lt;/a&gt; to close the gap in model namespacing and response normalization coverage. This sits alongside a raft of dependency bumps for lint-staged, Playwright, and anchore/scan-action to keep the CI pipeline modern.&lt;/p&gt;

&lt;p&gt;The parallelism win unlocks faster content generation without risking out-of-memory errors, while the automated docs win means the reference page stays synchronized with the codebase.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/456" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/456&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/453" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/453&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/455" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/455&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>devjournal</category>
      <category>performance</category>
      <category>python</category>
    </item>
    <item>
      <title>What we shipped -- 2026-05-15</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Sat, 16 May 2026 04:28:04 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-2026-05-15-2ofl</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-2026-05-15-2ofl</guid>
      <description>&lt;p&gt;We spent today closing the loop on a race condition that was silently killing the first turn of voice conversations. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/436" rel="noopener noreferrer"&gt;PR #436&lt;/a&gt; introduces a fallback where &lt;code&gt;ClaudeCodeBridgeLLMService&lt;/code&gt; catches a specific &lt;code&gt;Session ID &amp;lt;uuid&amp;gt; is already in use&lt;/code&gt; error during the very first user interaction and retries the spawn with &lt;code&gt;--resume&lt;/code&gt; flags. We debated the approach between retrying versus deferring. Deferring UUID generation would have only solved the 2026-05-08 specific scenario. By detecting the &lt;code&gt;already in use&lt;/code&gt; stderr on the first turn and transparently resuming against the existing JSONL on disk, we cover a broader class of races--preflights, healthchecks, and restart loops. The implementation is guarded by &lt;code&gt;self._first_turn&lt;/code&gt; so we only attempt this one-shot retry once, ensuring a real downstream regression on the resume path still raises a proper error. It's a handful of lines, one &lt;code&gt;--resume&lt;/code&gt; flag, and a &lt;code&gt;WARNING&lt;/code&gt; log entry in Loki to keep the recovery path visible. While the voice agent stabilizes, we expanded the test net in &lt;code&gt;test_topic_queue_cap.py&lt;/code&gt; from 5 to 18 tests, directly exercising the helpers in &lt;code&gt;services/topic_proposal_service.py&lt;/code&gt;. This means future refactors of the queue logic won't silently break our contract for &lt;code&gt;pending_topic_count&lt;/code&gt; or &lt;code&gt;resolve_max_pending&lt;/code&gt;. The broader ecosystem needed attention too. We patched &lt;code&gt;backup-visibility&lt;/code&gt; bind mounts and addressed schema/dependency bugs surfaced by the post-audit health check, ensuring our infra remains robust enough to support the agent's edge cases. From here, the voice agent handles the collision recovery gracefully, so we can focus on what comes next. We are still not in love with the QA threshold tuning, but at least we have data now.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/436" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/436&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>claude</category>
      <category>devjournal</category>
      <category>llm</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>What we shipped -- 2026-05-14</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Fri, 15 May 2026 16:28:03 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-2026-05-14-4mn5</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-2026-05-14-4mn5</guid>
      <description>&lt;p&gt;Today the cofounder-OS thesis stopped being a hypothesis. Module v1, FinanceModule, Mercury balance flowing into Postgres--all shipped in one day because we deferred every refactor we hadn't earned yet. The 'lite' approach kept the work cheap and ended with two real bugs caught and killed in passing.&lt;/p&gt;

&lt;p&gt;We hooked up the per-module migrations and &lt;code&gt;module_schema_migrations&lt;/code&gt; table for boot, then deployed the &lt;code&gt;FinanceModule&lt;/code&gt; F2 schema and hourly polling job. Route auto-discovery stitched the &lt;code&gt;ContentModule&lt;/code&gt; skeleton into the stack. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/433" rel="noopener noreferrer"&gt;PR #433&lt;/a&gt; forced the writer pipeline through the dispatcher so LiteLLM could finally emit Langfuse spans.&lt;/p&gt;

&lt;p&gt;Before shipping, we killed a test suite fluke where a shared httpx client leaked between tests, breaking auth on the revalidation service. After resetting that isolation, we pinned 13 contract edges on &lt;code&gt;traced_method&lt;/code&gt; to prevent silent regressions. We also deployed a &lt;code&gt;banned_transition_opener&lt;/code&gt; validator rule to catch overused stock phrases.&lt;/p&gt;

&lt;p&gt;Shipping this way is about finding the path without the bloat. From here, the architect composes graphs against the live module registry instead of hand-coded factories.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/433" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/433&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>architecture</category>
      <category>backend</category>
      <category>devjournal</category>
      <category>llm</category>
    </item>
    <item>
      <title>What we shipped -- 2026-05-13</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Thu, 14 May 2026 04:02:01 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-2026-05-13-1i2h</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-2026-05-13-1i2h</guid>
      <description>&lt;p&gt;The brain_daemon PSU watchdog was silent excepts. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/428" rel="noopener noreferrer"&gt;PR #428&lt;/a&gt;. It wasn't enough to fix the logic; we had to make the system report when it breaks. The cost dashboard no longer shows static 150W during a PSU outage because the exporter-fetch failure is now logged. We carried this philosophy through the rest of the #455 batch, hunting down the "silent except" traps across the entire stack.&lt;/p&gt;

&lt;p&gt;In &lt;code&gt;tap_runner&lt;/code&gt; and &lt;code&gt;retention_runner&lt;/code&gt;, malformed JSONB rows in &lt;code&gt;config&lt;/code&gt; and &lt;code&gt;metadata&lt;/code&gt; stopped silently falling through to raw strings. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/427" rel="noopener noreferrer"&gt;PR #427&lt;/a&gt;. We did the same for &lt;code&gt;social_poster&lt;/code&gt; metric increments and &lt;code&gt;validator_config&lt;/code&gt; bootstrap imports, so Prometheus errors and DSN resolution failures surface in traces. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/426" rel="noopener noreferrer"&gt;PR #426&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The content pipeline stages dropped issues when schemas shifted, so the rewriter looped endlessly on the same draft. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/425" rel="noopener noreferrer"&gt;PR #425&lt;/a&gt;. &lt;code&gt;jobs/check_memory_staleness&lt;/code&gt; and media reconciliation jobs stopped passing without a breadcrumb, and &lt;code&gt;task_executor&lt;/code&gt; stopped hiding model selection failures and timeout errors. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/423" rel="noopener noreferrer"&gt;PR #423&lt;/a&gt; and &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/418" rel="noopener noreferrer"&gt;PR #418&lt;/a&gt;. The rails--deepeval, ragas_eval, self_consistency--now log why they're disabled when &lt;code&gt;is_enabled()&lt;/code&gt; fails, and RAGAS notifies us when it can't find a judge model. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/421" rel="noopener noreferrer"&gt;PR #421&lt;/a&gt; and &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/420" rel="noopener noreferrer"&gt;PR #420&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Performance-wise, we stopped creating new &lt;code&gt;httpx.AsyncClient&lt;/code&gt; instances for every URL check and revalidation burst. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/424" rel="noopener noreferrer"&gt;PR #424&lt;/a&gt; and &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/419" rel="noopener noreferrer"&gt;PR #419&lt;/a&gt;. The GPU scheduler and URL validator share one client, closing it cleanly on shutdown to avoid leaking TCP pools. &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/417" rel="noopener noreferrer"&gt;PR #417&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;We traded code volume for data density. The operator finally sees which jobs are dormant and why the cost dashboard is drifting. We still don't love the amount of glue code needed to make this loud, but at least we're not guessing anymore.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/428" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/428&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/427" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/427&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/426" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/426&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/425" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/425&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/423" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/423&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/418" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/418&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/421" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/421&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/420" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/420&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/424" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/424&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/419" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/419&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/417" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/417&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>devjournal</category>
      <category>monitoring</category>
      <category>postgres</category>
      <category>sre</category>
    </item>
    <item>
      <title>What we shipped -- 2026-05-12</title>
      <dc:creator>Matthew Gladding</dc:creator>
      <pubDate>Tue, 12 May 2026 19:30:48 +0000</pubDate>
      <link>https://forem.com/glad_labs/what-we-shipped-2026-05-12-2cfo</link>
      <guid>https://forem.com/glad_labs/what-we-shipped-2026-05-12-2cfo</guid>
      <description>&lt;p&gt;We shipped Phase-0 Prefect orchestration last week, but today we discovered the post-pipeline actions were silently skipping execution in the new system. The 130-line success block in &lt;code&gt;task_executor&lt;/code&gt; ran exclusively in the legacy orchestrator, leaving us with four silent regressions: the &lt;code&gt;task.completed&lt;/code&gt; webhook never fired, auto-curation never triggered for low-quality scores, and auto-publish never shipped trusted niche content without manual intervention &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/371" rel="noopener noreferrer"&gt;PR #371&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The silence of the system is dangerous, which is why the security audit on 2026-05-12 demanded an immediate response. We found three live secrets--&lt;code&gt;discord_ops_webhook_url&lt;/code&gt;, &lt;code&gt;indexnow_key&lt;/code&gt;, and &lt;code&gt;langfuse_public_key'--shipped to the public mirror via a baseline migration. We cleared those values and gated Dynamic Client Registration and&lt;/code&gt;/voice/join` behind authentication flags to prevent anyone from minting admin clients or joining Matt's voice room without a token &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/375" rel="noopener noreferrer"&gt;PR #375&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Beyond the hotfixes, the operational reliability required a deep dive into our static exports. R2, the source of truth for the homepage, was lagging the database by three days because &lt;code&gt;publish_post_from_task&lt;/code&gt; fired an &lt;code&gt;export_post&lt;/code&gt; task that died on process boundaries. We replaced the fire-and-forget pattern with a synchronous &lt;code&gt;await export_post&lt;/code&gt; call and added a 15-minute reconciliation watchdog to catch drift between the DB and the static manifest &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/374" rel="noopener noreferrer"&gt;PR #374&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;We also finished the schema consolidation cleanup. The redundant &lt;code&gt;infrastructure/local-db/init.sql&lt;/code&gt; file was deleted, and the test suite was redirected to replay the baseline schema instead of the orphaned init script. This removes the technical debt of two &lt;code&gt;IF NOT EXISTS&lt;/code&gt; schema definitions and ensures the migration system owns the database creation process from first boot onward &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/373" rel="noopener noreferrer"&gt;PR #373&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Shipping these changes unlocks a period of stability where we can trust the orchestrator signals and the data integrity checks without chasing silent failures. The operator dashboard will now surface static export failures, and the Grafana approval panel will have working preview links once we restore the missing &lt;code&gt;preview_token&lt;/code&gt; generation in the finalization stage &lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/368" rel="noopener noreferrer"&gt;PR #368&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auto-compiled by Poindexter from today's commits and PRs.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/371" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/371&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/375" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/375&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/374" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/374&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/373" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/373&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/Glad-Labs/glad-labs-stack/pull/368" rel="noopener noreferrer"&gt;https://github.com/Glad-Labs/glad-labs-stack/pull/368&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
    </item>
  </channel>
</rss>
