Forem: Alvin Lee

The Messaging Challenges No One Talks About in Regulated, Air-Gapped, and Hybrid Environments

Alvin Lee — Thu, 08 Jan 2026 01:42:47 +0000

The modern platform engineering mandate is clear: adopt Kubernetes, embrace microservices, and accelerate velocity.

In theory, this leads to efficiency; in practice, if you operate within highly regulated sectors — Finance, Utilities, Defense, Healthcare, etc. — the journey often slows down due to significant networking and compliance requirements.

While the wider developer community utilizes fully managed queues and streaming services (like AWS SQS or Confluent Cloud), enterprise architects in regulated spaces are confronted with a fundamental modernization challenge:

How do you leverage the agility of cloud-native architecture when your security policy strictly forbids external data egress, necessitates air-gapped deployments, and mandates immutable audit trails for every transaction?

The standard answers — legacy middleware and vanilla open-source solutions — often fall short, creating a gap between operational security requirements and modernization goals.

The Modernization Dilemma

For regulated enterprises, the attempt to modernize messaging infrastructure typically forces architects to navigate two difficult options. Both introduce complexity and can delay migration projects.

1. The Constraints of Legacy Middleware

Platforms like IBM MQ or TIBCO have served the enterprise well for decades. They are trusted and proven. However, their architecture is often at odds with the dynamic, ephemeral nature of Kubernetes.

Architectural Differences: Legacy middleware was designed for static environments where IP addresses rarely change and servers run for years. Kubernetes is dynamic; pods are created and destroyed in seconds. Using a static, heavy-weight message broker to track thousands of ephemeral microservices creates an architecture that requires significant manual configuration.
The “Integration Overhead”: Modernizing with legacy tools often shifts engineering effort from innovation to integration. Developers forced to use older protocols or heavy client libraries in modern languages (like Go, Rust, or Python) spend considerable time writing custom wrappers just to maintain basic connectivity.
Scaling Costs: In a containerized world, the goal is to scale horizontally — adding lightweight instances as load increases. Legacy licensing models, often based on CPU cores or host counts, can make this scaling strategy cost-prohibitive.

2. The Complexity of Self-Managed Open Source

The alternative is often vanilla open-source solutions like Kafka or RabbitMQ. While technically capable, these tools assume an operational environment that is often unavailable inside a secure perimeter.

“Day 2” Operational Complexity: Cloud providers simplify these systems with managed control planes. When you deploy them on-premise without that automation, you inherit the full operational responsibility. Managing dependencies, rebalancing partitions, handling upgrades, and recovering from node failures in an air-gapped environment — where you cannot simply pull the latest Helm chart — requires a dedicated team.
Security Configuration: Most open-source projects prioritize features over enterprise governance. To make them compliant, teams must manually configure security mechanisms — setting up authentication, authorization, and audit logging. This often results in a complex platform that is difficult to upgrade and maintain over time.
The “No Egress” Constraint: Many “Cloud-Native” tools inadvertently rely on external connectivity — whether for pulling dependencies or sending telemetry. In a strictly air-gapped network with “No Egress” policies, these tools may require complex workarounds (like proxy tunnels) to function correctly.

The Result: Architects face a difficult trade-off. Staying on legacy systems limits velocity, but moving to standard open-source tools increases operational overhead and compliance complexity. A purpose-built solution is required.

Kubernetes-Native Messaging for Trust and Control

A third option is to use a Kubernetes-native message broker. This type of message broker is engineered specifically to resolve this trade-off by delivering a Kubernetes-native messaging backbone that is security-first and operationally self-sufficient.

Let’s look at the advantages of a Kubernetes-native messaging platform using as an example a product I’ve been using lately, KubeMQ.

1. One Platform, All Messaging Patterns

Eliminate the complexity of maintaining multiple message brokers for different needs. A Kubernetes-native message broker like KubeMQ unifies all major messaging patterns into a single cluster.

Consolidated Infrastructure: Instead of running Kafka for streaming, RabbitMQ for queuing, and gRPC for request/reply, you run one broker that handles Pub/Sub, Queues, Streams, and RPC in one lightweight platform. This reduces the infrastructure footprint and simplifies the architecture for your development teams.

2. Operational Simplicity (Easy to Use and Manage)

Designed for low operational overhead.

No Dedicated “Messaging Team” Required: Unlike complex open-source products that might require a dedicated team of engineers to keep running, KubeMQ is designed to be easily deployed and managed by a single DevOps engineer or developer.

3. True Air-Gapped Capability and Zero Egress

KubeMQ is designed to run disconnected. There is no requirement for external connectivity for licensing, metrics, or management. You can deploy the container in a high-security data center, and it functions independently.

Zero External Dependencies: You do not need to open firewall ports for a vendor’s control plane. All management and monitoring tools are included and run inside your perimeter, ensuring total data sovereignty.

4. Security & Audit: Deep Policy Enforcement

Compliance requires not just encryption, but verifiable control over access and activity.

Integrated RBAC and SSO: KubeMQ enforces Role-Based Access Control that integrates with your enterprise SSO/LDAP services. This ensures that only authenticated microservices with specific cluster roles can access designated channels or topics.
Immutable Audit and Retention: The platform provides built-in mechanisms for retaining message history and action logs. This gives auditors a clear trail of every action taken within the message bus — a requirement for regulated compliance frameworks like PCI-DSS or HIPAA.

5. Architecting for Hybrid and Edge Resilience

Modern infrastructure is rarely consolidated. It is distributed across headquarters, remote data centers, and field edge devices.

KubeMQ’s Bridges and Connectors allow for secure message replication across segregated environments. This allows you to synchronize data between On-Prem and Cloud without exposing the core network, and manage Day 2 operations declaratively via GitOps, reducing operational risk.

Real-Life Use Case: Unifying Critical Electricity Infrastructure

Let’s look at a real-world example: a major electricity transmission system operator in Europe. This operator manages critical national infrastructure, meaning their systems must be 100% reliable, secure, and operate strictly within a private, air-gapped environment.

The Challenge: Bridging Legacy and Innovation The organization operated a diverse messaging environment, with critical data flowing through legacy systems based on RabbitMQ and ActiveMQ. While robust, these systems were difficult to integrate with their new initiative: building modern, Kubernetes-based microservices to improve grid efficiency. They needed a way to allow new applications to consume data from legacy mainframes without engaging in a high-risk project to rewrite the core legacy code.

The Solution: A Kubernetes-native messaging solution as a Non-Intrusive Bridge.

Rather than replacing their legacy systems immediately, they used their new messaging solution to wrap and extend them. Using unconnected Sources and Targets, they built a bi-directional integration layer:

Inbound: Sources connect to the legacy RabbitMQ queues, consuming AMQP messages and converting them into KubeMQ events.
Outbound: The modern microservices process this data and publish results. Targets then translate these results back into AMQP and push them to the legacy queues.

The Value Delivered: This integration provided three distinct strategic advantages:

Risk-Free Modernization: They achieved a modernization of their architecture without changing any code in their mission-critical legacy systems. The old systems operate exactly as before, ensuring stability for the national grid.
Accelerated Development: The digital team was able to start building advanced microservices immediately. By consuming normalized data from the message broker, they were decoupled from the complexities of the legacy environment.
Future-Proof Foundation: They have effectively abstracted the underlying protocol. This gives the organization the flexibility to decommission the old brokers at their own pace, moving fully to a modern infrastructure without disrupting business logic.

Modernize Without Compromise

In the regulated sector, control is synonymous with security. Relying on external services or adapting incompatible tools is not always a sustainable strategy.

A Kubernetes-native messaging platform provides your platform engineering team with the agility they need, while providing the security and compliance team with the control and visibility they require.

Why Your UEBA Isn’t Working (and how to fix it)

Alvin Lee — Sat, 13 Dec 2025 00:51:23 +0000

UEBA (User Entity Behavior Analysis) is a security layer that uses machine learning and analytics to detect threats by analyzing patterns in user and entity behavior.

Here’s an oversimplified example of UEBA: suppose you live in Chicago. You’ve lived there for several years and rarely travel. But suddenly there’s a charge to your credit card from a restaurant in Italy. Someone is using your card to pay for their lasagna! Luckily, your credit card company recognizes the behavior as suspicious, flags the transaction, and stops it from settling. This is easy for your credit card company to flag: they have plenty of historical information on your habits and have created a set of logical rules and analytics for when to flag your transactions.

But most threats are not this easy to detect. Attackers are continuously becoming more sophisticated and learning to work around established rules.

As a result, traditional UEBA that relies primarily on static, rigid rules is no longer enough to protect your systems.

The End of Traditional UEBA–or, Why Your UEBA No Longer Works

Many UEBA tools were built around static rules and predefined behavioral thresholds. Those approaches were useful for catching predictable, well-understood behavior patterns, but are not great in modern environments where user activity, applications, and attacker behavior change constantly. Soon, AI agents will act on behalf of users and introduce even more diversity.

Here are the main limitations of traditional, static-rule-driven UEBA:

Static behavioral thresholds don’t adapt to real user behavior over time. They rely on fixed assumptions (e.g., “alert if X happens more than Y times”), which quickly become outdated as user behavior and environments evolve.
Rules require continuous manual tuning. Security teams spend time chasing false positives or rewriting rules as user behavior changes, which slows response and increases operational overhead.
Isolated detection logic lacks context. Legacy UEBA often analyzes events in silos, instead of correlating activity across identity, endpoint, network, and application layers. This limits the ability to detect subtle behavioral anomalies.

As a result, certain types of threats that blend into normal activity can go unnoticed despite the presence of rules.

So if legacy UEBA is not effective …what’s the solution?

What Modern Enterprises Actually Need from UEBA

Modern enterprises need UEBA systems that can do three things:

Immediately detect attacks. When attackers can morph in an instant, you need a security layer that moves just as fast.
Recognize attacks that are highly sophisticated and complex. Attacks are no longer simple enough to be caught by a set of rules — even advanced ones backed with behavioral analytics.
Integrate seamlessly with existing security operations.

Let’s look at each in more detail and how it can be achieved.

Immediately detect attacks (without a long training period)

Traditional UEBA training periods leave organizations vulnerable for months and chronically behind on detecting the latest threats. A typical three to six month learning period creates a huge security gap.

Day-one detection capabilities for behavioral threats and compromised accounts require first-seen and outlier rules that can spot anomalous behavior immediately without waiting for machine learning models to mature.

You need near-instant behavioral baselines. How?

Luckily, most organizations already have the data they need: years of historical log information sitting in their Security Information and Event Management (SIEM) systems. Modern UEBA systems use this information to create behavioral baselines instantly.

For example, companies like Sumo Logic — who advocate for the “log everything” approach, have tools that use the data you already have to create powerful baselines in just minutes.

Detect highly sophisticated attacks

Today’s attacks blend in with normal business operations. Correlation rules miss behavioral threats that show only subtle anomalies; they can’t identify compromised accounts or insider threats that are performing normal-looking activities.

Modern UEBA solutions must be able to detect first-seen activities, such as unusual file sharing via OneDrive. They need to catch access to new proxy categories and suspicious cloud service usage that don’t match historical user behavior.

This comes down to using the right tools. For example, Microsoft Sentinel can identify unusual Azure identity behaviors such as abnormal cloud service access patterns that could indicate account compromise or data exfiltration. And Sumo Logic has first-seen and outlier rules that can detect when an attacker is trying to use a network sniffing tool. They catch endpoint enumeration and suspicious email forwarding rules that deviate from established patterns.

Integration with existing security operations

UEBA delivers meaningful value when it fits naturally into existing security workflows. Security teams rely on SIEM, SOAR, identity systems, and endpoint tools to build a complete picture of activity across their environment. UEBA works best when its behavioral insights are delivered in the same place analysts already investigate and respond to alerts.

Effective UEBA solutions therefore integrate directly with the broader security platform, allowing behavioral anomalies to be correlated with logs, identity events, and threat intelligence. This unified context helps analysts make faster, more accurate decisions without switching tools or managing separate consoles.

Flexibility is also important. Organizations must be able to adjust detection logic and behavioral thresholds to match their environment, risk tolerance, and operational needs. When UEBA is tightly integrated and adaptable, it becomes an extension of the security workflow rather than an additional system to maintain*.*

UEBA as the Foundation for AI Security Agents

UEBA hasn’t been replaced by AI. Instead, UEBA has become the way to train AI. AI-powered detection and response solutions perform best when they ingest clean, comprehensive behavior baselines, and that’s exactly what mature UEBA can provide.

AI agents need quality behavioral baselines

AI security agents aren’t magic. They follow the GIGO (garbage in, garbage out) principle just like any other data-intensive system. Feed an AI agent high-quality behavioral data, and it will thrive. But if you feed it insufficient or poor-quality data, then you’ll become part of the 95% statistic of AI pilots that fail to deliver real business value.

Structured UEBA rules also give the agents specialist knowledge, such as who should log in where, how often a service account connects to S3, and typical overnight file volumes. AI agents can learn (and extend) these rulesets.

AI detects, then AI responds

Security teams often drown in alerts. Teams can’t keep up. But when UEBA feeds AI:

First-seen rules become automatic triggers. Instead of waiting for an analyst, an agent can begin gathering data and context within seconds.
AI can rank threats, helping to make sure human attention is given to the events with the biggest deviation or highest blast radius.
Entity relationship maps derived from UEBA help agents model lateral-movement risk and choose containment tactics (for example: quarantine the host, revoke credentials, etc.).

Once the system can reliably detect threats, you can take it to the next level and allow your AI agents to take action, too.

From UEBA rules to autonomous security operations

Manual security operations have a scaling problem. They can’t keep pace with modern threat volumes and complexity. As a result, organizations miss threats or burn out security analysts with overwhelming alert fatigue.

But with UEBA first-seen rules, AI agents can immediately begin collecting evidence and correlating events when anomalous behavior is detected. Outlier detection can feed AI-driven risk scoring and prioritization algorithms. And behavioral baselines can ensure that automated responses are based on a solid understanding of what constitutes legitimate versus suspicious activity within the specific organizational context.

You can still have a human in the loop, authorizing each action recommended by the AI system. Eventually, you may delegate action to the AI system as well, with humans merely being notified after the fact.

Building AI-Ready Behavioral Foundations Now

Modern UEBA platforms are already generating AI-compatible behavioral data. These platforms structure their outputs in ways that AI agents can easily consume and act upon. For example:

Ongoing discovery of the best ways to format and organize data to fit optimally into the context of LLMs and how to provide them with effective tools
Signal-clustering algorithms to reduce the noise that might confuse an AI agent’s decision-making. This ensures that only meaningful behavioral anomalies reach automated systems for action.
Rule customization and match lists provide structured data that AI agents can interpret and act upon. This creates clear decision trees for autonomous responses.
Historical baseline capabilities create rich training datasets without waiting months for AI deployment. Organizations can leverage years of existing log data. AI agents can begin operating with sophisticated behavioral understanding from day one rather than starting with blank behavioral models.

With these capabilities already in place, organizations can transition seamlessly from manual to automated security operations.

The Bottom Line

When implementing UEBA, focus on true principles and actionable strategies:

1. Ensure comprehensive, high‑quality data integration

Use all relevant data sources: existing logs, new telemetry, identity systems, endpoints, and cloud apps to build complete behavioral profiles. If critical data is missing you should collect it and add it to the UEBA’s scope. For example, a user’s calendar showing a business trip to Tokyo is very relevant when the system detects login attempts from Japan.

2. Accelerate meaningful baselines using historical data and rapid observation periods

Leverage historical log data to establish baselines quickly, but expect this to take a couple days to a few weeks. Supplement with fresh data as needed to ensure the baseline reflects current behaviors. For example, if an employee moves to a different team the system should expect a change in behavior.

3. Integrate UEBA insights with your current security workflows

UEBA should capitalize on SIEM and other security tools to deliver high-impact alerts and operational value. Avoid requiring extensive new infrastructure unless necessary, and always align the solution to your organization’s needs.

These approaches deliver immediate value and adapt to changes to maximize the coverage and accuracy of behavioral analytics.

Your success metrics matter just as much as your implementation. Track the following:

How many sophisticated threats UEBA catches that your traditional systems miss
The reduction in dwell time for compromised accounts
Coverage improvements for lateral movement and unknown attack patterns
Analyst efficiency gains from richer contextual alerts.

These metrics prove value to stakeholders and help you continuously refine your approach.

While classic rule-based UEBA relied on manual configuration, today’s UEBA platforms enhance these foundations with autonomous analytics using statistical models, adaptive baselines, and AI-driven outlier detection.

Functions like first-seen and outlier detection do leverage rules, but they operate as part of a dynamic, context-aware system rather than a set of static match expressions. AI agents continuously monitor and analyze vast streams of behavioral data, learning what constitutes normal activity and identifying subtle anomalies that may indicate emerging threats. By correlating signals across users, devices, and time, these agents enable real-time, adaptive detection and response. This elevates security operations from manually maintained static rule matching to intelligent and proactive protection.

Log Spikes? No Sweat: How Top DevOps Teams Tame Bursty Workloads

Alvin Lee — Fri, 11 Jul 2025 16:05:53 +0000

Taylor Swift ticket sales brought the entire platform to its knees… a crypto exchange saw 10x its regular traffic during a price swing… holiday deals dropped at midnight, and retail sites scrambled to keep up. These weren’t just high-traffic moments. They were log storms.

For DevOps teams in bursty verticals like media, fintech, gaming, and retail, moments like these are make-or-break. Whether the spike is planned (like a product drop) or unpredictable (like an influencer mention), what happens behind the scenes is intense: logging pipelines can flood, ingestion costs can surge, and dashboards often freeze when you need them the most.

We’re going to look at how top-performing teams prepare for these log surges before they hit. You’ll learn how modern observability platforms offer scalable architectures, dynamic ingestion, and pricing models that flex with demand, not against it. We’ll also unpack real-world scenarios where things didn’t go to plan, and show how a different approach could have mitigated all the firefighting in the dark.

What Makes a “Bursty” Vertical So Challenging

Troubles with scaling: Ticketmaster’s meltdown

First, let’s look at the challenge of scaling. The hardest part about operating in a bursty vertical isn’t just scaling, it’s doing it fast, without losing visibility or blowing up your budget.

Ticketmaster’s meltdown during the Taylor Swift Eras Tour pre-sale became a masterclass in what can go wrong when systems are unprepared for simultaneous, global demand spikes. The platform saw over 3.5 billion system requests in a single day, four times the previous peak. And it wasn’t just the front-end that struggled. Back-end observability pipelines were reportedly overwhelmed, which slowed root cause analysis and delayed recovery efforts. This was a case where Ticketmaster knew there was going to be significant demand, but they just didn’t expect it to be this large.

For teams in e-commerce, media, fintech, and gaming, these bursts can arrive with little to no warning. Flash sales, viral moments, breaking news, or crypto market moves create sudden demand that can outpace even the best predictive scaling models. Even well-orchestrated campaigns like a product drop or a limited-edition NFT mint can spark volumes that dwarf normal baselines.

Again, the challenge here isn’t only application scale, it’s also observability scale. Log volumes don’t just grow linearly with traffic; they often spike exponentially. API calls increase. Errors multiply. Security events balloon. Suddenly, what was a manageable logging setup turns into a firehose of data, one that traditional logging tools aren’t built to handle.

Why? Most legacy log management systems depend on rigid ingestion pipelines and fixed retention pricing. When volume surges, either logs are dropped, throttled, or stored at unsustainable costs. Worse, engineering teams often choose to ingest only specific logs, filtering out data that might later prove critical for debugging or forensics.

That decision, under pressure, can backfire.

Losing trust: Robinhood goes dark

Our second challenge is reliable triage and finding a root cause.

Consider the Robinhood outages during a surge in Dogecoin trading. While the company cited “unprecedented volumes,” users and analysts alike noted that the company essentially went dark while they scrambled to figure out what was happening. Transparency was limited for hours.

Without reliable observability during peak stress, diagnosing failures became guesswork, and trust took a hit.

In bursty environments, DevOps teams need tools that scale as fast as their demand does — and just as importantly, pricing models that won’t punish them for succeeding. That’s where next-gen observability platforms come in, offering elastic ingestion and smart tiering that keep logs flowing, insights visible, and costs predictable — even when everything else is spiking.

Let’s now look at some solutions.

Modern Observability Platforms That Are “Built for Bursty”

When traffic surges hit, your users expect the same seamless experience they experienced before the surge. And for that, your DevOps team needs observability tools that don’t melt down under pressure.

That’s why modern observability platforms have embraced architectures built for bursty. They are designed from the ground up to scale dynamically, preserve full-fidelity logs, and surface insights even when volumes spike unpredictably.

They use schema-less ingestion, ingest everything models, and AI for triage. Let’s look at each.

Schema-less ingestion

Unlike legacy systems that rely on manual log filtering or hard-coded schemas, today’s observability leaders support schema-less ingestion, meaning you can pump in both structured and unstructured data. This means everything from JSON logs to raw error messages and Slack alerts with no need to reconfigure pipelines.

Modern platforms such as Sumo Logic are built to handle sudden data surges without missing a beat. Its architecture automatically scales ingestion pipelines and performs real-time indexing to keep dashboards responsive and queries fast, even under duress. According to engineers, during a major production event where log ingestion volumes more than doubled, the platform maintained performance while increasing cost by only 10% thanks to its elastic scaling design. This kind of efficiency is critical during high-pressure events, like election nights or viral product launches, where teams can’t afford blind spots in their observability stack.

Ingest everything

Traditional platforms force teams to overprovision for peak load. However, modern platforms offer pay-as-you-go pricing and innovative models, such as flex credits, which enable teams to temporarily “burst” without incurring higher ongoing costs. The best platforms also offer “ingest everything” plans where you can log everything, but only pay for what you actually use.

More modern pricing plans, like the ones above, keep budgets in check and eliminate the guesswork from capacity planning.

AI for triage

The real game-changer? Built-in machine learning. When log volume doubles or triples, humans can’t sift through everything.

Tools like Sumo Logic’s anomaly detection and LogReduce automatically cluster repetitive log lines, highlight deviations from baseline patterns, and tee up root causes before customers notice. That’s how teams keep downtime short and postmortems informative. If your observability platform can’t scale, index, and surface insights in real time during your most critical hours, it’s not ready for bursty workloads.

How Top Teams Stay Calm During the Storm

Not only do you need the right tools to handle bursts, you also need the right mindset and training. When the logs start flying and dashboards light up like a Christmas tree, panic is easy — but the best teams stay calm because they are ready for chaos.

Stress-test your log pipelines

Leading DevOps teams practice what’s essentially chaos engineering for observability: they don’t just test app resilience under load — they deliberately stress-test their log pipelines.

At Netflix, for example, engineers regularly simulate outages and surges as part of their “Failure Injection Testing” framework, which includes observability components to ensure monitoring tools perform under pressure.

But you don’t have to operate at Netflix’s scale to benefit from the same mindset. Effective teams simulate log floods during load tests, which push traffic through staging environments while tracking how ingestion, indexing, and alerting respond to the increased load. Tools like Grafana’s k6 and Locust can simulate thousands of requests per second, while synthetic log generators mimic bursty error scenarios.

Key metrics to watch during these tests include:

Ingestion throughput: Are logs being dropped, delayed, or backed up?
Alert latency: Are critical alerts still firing on time?
Storage tier transitions: Are logs routed to cold or cheaper storage as designed?

Use verbose logs

Also, teams can apply intelligent partitioning, i.e., routing verbose debug logs to lower-cost tiers while maintaining high-value security or performance logs in hot storage. Dynamic sampling and routing rules ensure you’re not overwhelmed, and more importantly, that you don’t lose signal during the noise.

Non-verbose logs (high value):

Keep these in hot storage; they contain immediately useful information.

{
  "timestamp": "2025-06-17T13:02:11Z",
  "level": "ERROR",
  "service": "auth-api",
  "message": "Failed login attempt",
  "userId": "923188",
  "ip": "203.0.113.42",
  "error": "Invalid password"
}

Verbose logs (low value):

These logs may happen millions of times a day, while they aren’t often useful for daily metrics, during bursty periods, they could be a leading indicator of a problem.

{
  "timestamp": "2025-06-17T13:02:12Z",
  "level": "DEBUG",
  "service": "auth-api",
  "message": "Parsed user agent",
  "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
  "ip": "203.0.113.42"
}

Take advantage of runbooks

Operational readiness also means people, not just tools. Top teams develop runbooks, specifically tailored documentation for burst scenarios, including volume-based alerting that adapts thresholds based on time windows or historical norms. Clear escalation paths and role assignments reduce confusion when seconds count.

The difference between chaos and control? Preparation. The best observability platforms support that prep — and the best teams treat bursty events as drills they’ve already rehearsed.

Conclusion

Bursty workloads are no longer rare exceptions — they’re the new normal in high-velocity industries like e-commerce, media, fintech, and gaming. From viral product drops to trading frenzies, these moments create not only traffic spikes but also observability crises.

Traditional log management tools often fail under pressure, either throttling data or overwhelming teams with noise. That’s why top DevOps teams rely on observability platforms purpose-built for scale, speed, and flexibility. With schema-less ingestion, elastic scalability, and usage-based pricing models like flex credits, these platforms don’t just keep logs flowing; they keep insight accessible when it matters most. The best teams don’t wait for a spike to test their resilience: they rehearse chaos, simulate bursts, and fine-tune alerting strategies so they can act with confidence, not confusion. Because in a world where digital performance is tied directly to business success, the ability to weather log storms isn’t a luxury — it’s a competitive edge.

The Human Side of Logs: What Unstructured Data Is Trying to Tell You

Alvin Lee — Mon, 12 May 2025 16:39:17 +0000

It’s Friday afternoon, and your dashboards look great. Charts are green. CPU usage is stable. Database query times are within your SLA. You’re feeling great and ready for the weekend.

But little do you know, there’s a significant issue being overlooked by all your metrics—and it’s about to ruin your weekend.

Unfortunately, you don’t know about the problem yet. That’s because there’s a disparity between your metrics and the actual user experience. Your dashboards might look great, but your users are telling a different story.

It happens to even the best of us. On April 17, 2025, for example, Walmart's website experienced a significant outage. Users were unable to add items to their carts or access certain pages. Revenue was lost, user complaints surged on X and Downdetector, and undoubtedly someone was woken up from a deep sleep to log on and help get it fixed.

Critical signals don’t always show up in CPU graphs or 5xx error charts. They surface in chat threads, Downdetector complaints, and even failed login attempt logs. The first signals often come from people, not probes.

Why? Because traditional monitoring tools focus on structured data, such as CPU usage, memory consumption, database usage, and network throughput. And while these metrics are essential, they can miss the nuances of user behavior and experience.

But unstructured data—such as error messages, user feedback, and logs—can tell a more thorough story. It can provide critical insights into system issues that structured data can overlook and that you otherwise wouldn’t know about—until it’s too late.

The signs are there—if you're listening.

In this article, I’ll explore why unstructured data matters, where you should look for unstructured data, what signals to watch for, and how observability platforms can help you tap into unstructured data without drowning you in noise.

Why Unstructured Data Matters

Structured data is the data that comes in the formats you expect—rows, columns, numbers, stats—and tells you all about what logically happened. It’s the duration of an API call, the response status code, or the CPU load on a node.

Unstructured data, on the other hand, is the messy data. And it’s everywhere. It can be found in support tickets, bug reports, chat threads, error messages, and offhand complaints. It’s the data that often arrives in natural language, not numeric values. It’s messy, not always clear in meaning, and as the name suggests, it’s unstructured.

But unstructured data is critical. It tells you where confusion lives, where intent breaks down, and where the user’s mental model clashes with what the software does. It captures the emotions, intents, and frustrations that users feel when systems misbehave.

And when ingested, interpreted, and aggregated, important patterns can emerge. For example, unstructured data can start to paint a picture if your app is seeing:

A surge in password reset attempts
Rage clicks after a UI release
A sudden drop in engagement
Support tickets clustered around a broken journey

Sometimes your best clues aren’t in the metrics—they’re in this unstructured data. Structured observability gives you the dashboard. Unstructured data gives you the story. And if you're not reading both, you're missing half the plot.

What Signals Should You Watch For?

So where should you look for unstructured data—and what should you watch for? There are many sources. Here are a few to start:

Pay attention to session logs that show users repeatedly attempting the same action. That’s not just a retry—it’s friction.
Watch for freeform error messages that never get piped into your dashboard. That’s often where the real context behind a failure lives.
Don’t ignore the chatter on Slack, in Jira, or even on social media. When three engineers complain about the same “sluggish page,” chances are there’s a performance regression in your latency graph smoothed over.
Even vague user feedback can be invaluable. A spike in “can’t log in” support tickets may be attributed to session expiry handling, rather than infrastructure failure. You’ll only catch it if you’re collecting and analyzing the whole narrative—system logs, yes, but also what people say and do when something doesn’t work the way they expect.
Watch for security anomalies. Failed login attempts, credential stuffing, token mismatches—these may not trigger alarms if thresholds aren’t breached, but patterns buried in raw logs can signal a threat weeks before your SIEM lights up.

How Observability Platforms Can Wrangle Unstructured Data

One of the biggest misconceptions about unstructured data is that it has to be cleaned, labeled, and modeled before it’s useful.

Yes, that was true in the past. Teams often spent hours writing regular expressions or building brittle parsers just to extract a few fields from a messy log line.

But that’s no longer the case.

Modern observability platforms are designed to ingest unstructured data at scale without requiring perfect formatting or predefined schemas. You can pump in raw error messages, user reviews, support tickets, and Slack threads—and the platform handles the rest. Machine learning, natural language processing, and pattern recognition do the bulk of the work.

That means you don’t need a data wrangler to find value. A modern observability platform can:

Automatically surface spikes in login failures by IP address block or geographic location.
Cluster similar feedback into themes using sentiment analysis, even if the wording varies.
Correlate failed transactions to specific deployments, even when the logs don’t follow strict naming conventions.

To be effective, you need an observability platform that can ingest both structured and unstructured data. One that provides a comprehensive view of system health. One that, by analyzing unstructured data, helps you to identify and address issues proactively—before your weekend is ruined.

For example, here’s a screenshot from Sumo Logic showing a parse statement from a section of unstructured log data, and how it helps you make sense of the data.

With a modern platform like this, you can use a “schema-on-read” approach. You just store the data as is, then analyze it when needed. And if you can get friendly pricing, you won’t have to worry about the amount of ingest; you can ingest everything—system logs, application traces, and behavioral data—and explore it later without gaps.

Example Use Cases

For example, let’s say you work at an e-commerce company that experiences a sudden surge in negative comments on social media and customer reviews, all of which mention difficulties with the checkout process. Traditional monitoring tools, focused on structured data like transaction success rates, show no anomalies.

But by employing sentiment analysis on unstructured data sources, you identify a pattern: customers are frustrated with a recent update to the checkout interface. This insight means you can promptly address the issue, improve customer satisfaction, and stop any potential revenue loss.

When observability platforms process behavioral signals at scale, the value isn’t just technical—it’s operational and financial.

E-commerce teams can identify and resolve friction in the checkout flow before it tanks conversion rates.
SaaS platforms can correlate a rise in support volume with a regression in a recent release and take action before churn increases.
SRE and platform teams can detect misconfigurations or silent failures earlier, which means reduced incident duration and lower downtime costs.

This kind of pattern recognition turns what used to be support overhead into strategic insight.

Conclusion

You’re already tracking the numbers—latency, error rates, CPU usage. But that’s only half the story. The other half lives in the messy commotion: Slack rants, support tickets, log messages, and user reviews that don’t fit into a tidy schema.

Unstructured data is where behavioral signals live. It can show you why things are breaking, not just what is broken. It captures confusion, intent, and frustration long before structured telemetry raises a red flag.

If you're responsible for user experience, reliability, or security, you can’t afford to ignore what people are saying—or how they're interacting—with your product. The tools exist to make unstructured data useful. Now it’s a matter of putting them to work. The human side of logs is talking. Start listening.

From Concept to Cloud: Building with Cursor and the Heroku MCP Server

Alvin Lee — Tue, 22 Apr 2025 21:33:55 +0000

I’ve been experimenting with Cursor as a development tool, and it’s been surprisingly helpful in my day-to-day workflow. It’s not just that it writes code — it understands context, offers suggestions in the right moments, and even anticipates what I’m about to do next.

When I saw the announcement about the Heroku MCP Server, I got curious. Could I use Cursor to go beyond just writing code — and actually build and deploy an app to Heroku — primarily via chat prompts and responses? I decided to try it out.

In this post, I’ll walk through how I used Cursor to build a simple SvelteKit app and deploy it to Heroku, powered by the new MCP integration. Would it work? Was it smooth? I thought I would test drive it to see. Come along for the ride.

What is the Model Context Protocol (MCP)?

MCP is an open standard that lets LLMs interact with external tools in a structured, programmatic way. Instead of just generating code or text based on context, MCP support lets your AI system take real actions — like make API calls or execute commands — based on what you ask.

In practice, this turns Cursor into more than just an AI-enhanced code editor. With MCP support, Cursor becomes a command center. I could potentially spin up cloud infrastructure, query databases, or scaffold new projects — all through prompts in the chat.

The Heroku MCP Server introduces this capability for the Heroku platform. That means I can ask Cursor to do things like create a Heroku app, scale dynos, or attach add-ons — without ever leaving my editor or opening a terminal.

Using Cursor to build my SvelteKit app

I wanted Cursor to take a shot at building a to-do list as a single-page application using SvelteKit, backed by PostgreSQL. This should be simple enough for Cursor, and it would save me time, since I have little experience with Svelte. These are the steps I took.

Open Cursor to a new project folder

I started with an empty folder in ~/project. I had a blank canvas and was ready (for Cursor) to get to work.

Describe the task

I explained to Cursor what I wanted to do. To give it a little help with Svelte, I pointed it to Svelte’s LLM-friendly documentation.

I would like to build a “Todo List” single page application using Svelte/SvelteKit. The documentation for Svelte can be found here: https://svelte.dev/llms-full.txt

My application needs to handle listing to-dos, adding to-dos, marking to-dos as complete/incomplete, and deleting to-dos.

It should be backed by a Postgres database.

Eventually, I will deploy this application to Heroku with a Postgres add-on. But I also want to test it locally. I have a local instance of Postgres running. So, I can either give you local db credentials, or you can use the connection string from Heroku once the add-on is running.

Please create my application in the current folder.

This was the response, so I gave Cursor a little guidance as to my preferences.

Cursor worked through creating the app and adding the necessary dependencies. When it hit speed bumps (like command line options that were outdated), it worked through them.

Let it code!

It was time for Cursor to begin implementing the application. At this point, it had only been about two minutes since I opened Cursor to get started.

In a matter of seconds, Cursor generated the code and saved the files necessary for my application.

Set up local database

Next, Cursor gave me next steps related to setting up my database.

I told Cursor that I would want some help with this.

Please give me the commands to create that database and run the schema. And also, create the .env file for me. My local postgres database runs on localhost, port 5432. The username is postgres and the password is postgres.

I probably need source control too. Please initialize a git repo for my project too.

Cursor set up my .env file and gave me the psql commands I needed.

Initialize a git repository

Cursor continued on, setting up source control for my project.

Test application locally

Next, Cursor told me it was ready for me to test on a local development server.

Really? All the coding was done and I was ready to test? I clicked Run command.

Then, I opened my browser to http://localhost:5173 to test out the app. It worked. Listing, adding, marking as complete, and deleting.

So, Cursor had finished building my app. It had been about five minutes since I started. We’re ready to deploy to Heroku. And now, we get to use Cursor with Heroku’s MCP Server.

Deploying to Heroku with help from Cursor

Getting up and running with Heroku is already pretty easy. I was ready to see how Heroku task automation with Cursor could enhance my productivity even more.

Set up Cursor to use Heroku’s MCP Server

To connect Cursor with Heroku’s MCP Server, I needed to go through a few simple steps…

Step 1: Obtain a Heroku authorization token

From a terminal, I made sure I was authenticated with the Heroku CLI. Then I ran the following command:

$ heroku authorizations:create

Creating OAuth Authorization... done
Client:      <none>
ID:          03aff7da-87a9-4f9b-9400-5387164390e9
Description: Long-lived user authorization
Scope:       global
Token:       HRKU-1a2b3c4d-5e6f-7890-abcd-abc123def456
Updated at:  Thu Apr 17 2025 12:04:14 GMT-0700 (Mountain Standard Time) (less than a minute ago)

This would be the only time that I needed to do this in a terminal. I could use this authorization token for all future Cursor projects.

Step 2: Create a `mcp.json` file

In my project folder, I created a subfolder called .cursor. In that folder, I created a file called mcp.json with the following contents:

{
  "mcpServers": {
    "heroku": {
      "command": "npx -y @heroku/mcp-server",
      "env": {
        "HEROKU_API_KEY": "HRKU-1a2b3c4d-5e6f-7890-abcd-abc123def456"
      }
    }
  }
}

Step 3: Enable the new MCP server

As soon as I saved .cursor/mcp.json, Cursor detected it.

I clicked Enable.

Use Cursor to build and deploy app

With Cursor configured, I just needed to tell it to get to work.

Please deploy my app the Heroku. You can call it my-svelte-todo-list\.

That’s a promising start. I clicked Run tool.

Next, Cursor would add the Postgres add-on. There was a bit of a hiccup here. Cursor didn’t have the right name for the lowest-tier add-on plan, but it quickly solved its problem and continued moving forward.

With the Postgres add-on up and running, I nudged Cursor to move ahead.

You’ve created the app and the Postgres database. Help me initialize the database and then deploy the code.

Cursor used the MCP tool to connect to my Postgres add-on — in the Heroku CLI, that would have been heroku pg:psql—and to create a table.

Next, Cursor checked that everything was set up to deploy my app.

Then, Cursor noticed that there was no Heroku remote set up for my git repository. So, it helped with that.

Notice that Cursor needed to run a bash command to set up the remote. It looks like heroku git:remote isn’t one of the available tools in the Heroku MCP Server (yet).

I was almost ready for Cursor to deploy. But if Cursor was going to use an MCP tool (deploy_to_heroku) instead of have me issue a git push heroku main command, then my project would need an app.json file. I reminded Cursor of this:

I think I also need an app.json file so that I can deploy to Heroku (with the Heroku MCP tool).

Cursor created the file.

Now, Cursor was ready to deploy to Heroku with the MCP tool.

After a few seconds, this is what Cursor told me:

Successful deployment? Just like that? I went to my Heroku app URL to be sure.

Yes! Successful deployment!

Other things that Cursor can do with the Heroku MCP Server

Cursor said the deployment was successful, but asked if there were other things I wanted it to do.

Hmm, check application status or view the logs? Ok, sure.

Cursor used the MCP tools to check on my dyno and summarize recent logs. Nice!

Final thoughts: A new way to build

Cursor didn’t just help me scaffold a project or write a frontend for a framework I have little experience with. It carried my entire project from concept to cloud. That was pretty sweet. Heroku already makes deployment straightforward, but using Cursor and the MCP Server made the whole experience feel nearly automated.

This wasn’t just about saving time. It was about shifting how I interact with tools I already know and trust. The idea that I can go from “I want to build…” to “It’s live!” without switching contexts is exciting — and something worth exploring further on more ambitious projects.

Happy coding!

Deploying a Scala Play Application to Heroku

Alvin Lee — Tue, 08 Apr 2025 15:39:00 +0000

I’ve been a web developer for years, but I haven’t touched Java in a long time — like, late-90s long. Back then, Java development felt cumbersome: lots of boilerplate and complex configurations. It was not exactly a pleasant experience for building simple web apps. So when I recently started exploring Scala and the Play Framework, I was curious more than anything. Has the Java developer experience gotten better? Is it actually something I’d want to use today?

Scala runs on the Java Virtual Machine, but it brings a more expressive, modern syntax to the table. It’s often used in backend development, especially when performance and scalability matter. Play is a web framework built for Scala, designed to be fast, reactive, and developer-friendly. And you use sbt, which is Scala’s build tool — roughly comparable to Maven or Gradle in the Java world.

In this post, I’ll walk through setting up a basic Scala Play app, running it locally, and then deploying it to Heroku. My hope is to show you how to get your app running smoothly — without needing to know much about the JVM or how Play works under the hood.

Introducing the Example App

To keep things simple, I’m starting with an existing sample project: the Play Scala REST API example. It’s a small application that exposes a few endpoints for creating and retrieving blog posts. All the data is stored in memory, so there’s no database to configure — perfect for testing and deployment experiments.

To make things easier for this walkthrough, I’ve cloned that sample repo and made a few tweaks to prep it for Heroku. You can follow along using my GitHub repo This isn’t a production-ready app, and that’s the point. It’s just robust enough to explore how Play works and see what running and deploying a Scala app actually feels like.

Taking a quick look at the code

Before we deploy anything, let’s take a quick tour of the app itself. It’s a small codebase, but there are a few things worth pointing out — especially if you’re new to Scala or curious about how it compares to more traditional Java. I learned a lot about the ins and outs of the Play Framework for building an API by reading this basic explanation page. Here are some key points:

Case class to model resources

For this REST API, the basic resource is the blog post, which has an id, link, title, and body. The simplest way to model this is with a Scala case class that looks like this:

case class PostResource(
  id: String,
  link: String,
  title: String,
  body: String
)

This happens at the top of app/v1/post/PostResourceHandler.scala.

Routing is clean and centralized

The conf/routes file maps HTTP requests to a router in a format that’s easy to read and change. It feels closer to something like Express or Flask than an XML-based Java config. In our example, the file is just one line:

->         /v1/posts               v1.post.PostRouter

From there, the router at app/v1/post/PostRouter.scala defines how different requests within this path map to controller methods.

override def routes: Routes = {
  case GET(p"/") =>
    controller.index

  case POST(p"/") =>
    controller.process

  case GET(p"/$id") =>
    controller.show(id)
}

This is pretty clear. A GET request to the path root takes us to the controller’s index method, while a POST request takes us to its process method. Meanwhile, a GET request with an included blog post id will take us to the show method and pass along the given id.

Controllers are concise

That brings us to our controller, at app/v1/post/PostController.scala. Each endpoint is a method that returns an Action, which works with the JSON result using Play’s built-in helpers. For example:

def show(id: String): Action[AnyContent] = PostAction.async {
  implicit request =>
    logger.trace(s"show: id = $id")
    postResourceHandler.lookup(id).map { post =>
      Ok(Json.toJson(post))
    }
}

There’s very little boilerplate — no need for separate interface declarations or verbose annotations.

JSON is handled with implicit values

Play uses implicit values to handle JSON serialization and deserialization. You define an implicit Format[PostResource] using Play JSON’s macros, and then Play just knows how to turn your objects into JSON and back again. No manual parsing or verbose configuration needed. We see this in app/v1/post/PostResourceHandler.scala:

object PostResource {
  /**
    * Mapping to read/write a PostResource out as a JSON value.
    */
    implicit val format: Format[PostResource] = Json.format
}

Modern, expressive syntax

As I dig around through the code, I see some of Scala’s more expressive features in action — things like map operations and pattern matching with match. The syntax looks new at first, but it quickly feels like a streamlined blend of Java and JavaScript.

Running the App Locally

Before deploying to the cloud, it’s always a good idea to make sure the app runs locally. This helps us catch any obvious issues and lets us poke around the API.

On my machine, I make sure to have Java and sbt installed.

~/project$ java --version
openjdk 17.0.14 2025-01-21
OpenJDK Runtime Environment (build 17.0.14+7-Ubuntu-120.04)
OpenJDK 64-Bit Server VM (build 17.0.14+7-Ubuntu-120.04, mixed mode, sharing)

~/project$ sbt --version
sbt version in this project: 1.10.6
sbt script version: 1.10.6

Then, I run the following from my project root:

~/project$ sbt run

[info] welcome to sbt 1.10.6 (Ubuntu Java 17.0.14)
[info] loading settings for project scala-api-heroku-build from plugins.sbt...
[info] loading project definition from /home/alvin/repositories/devspotlight/heroku/scala/scala-api-heroku/project
[info] loading settings for project root from build.sbt...
[info] loading settings for project docs from build.sbt...
[info]   __              __
[info]   \ \     ____   / /____ _ __  __
[info]    \ \   / __ \ / // __ `// / / /
[info]    / /  / /_/ // // /_/ // /_/ /
[info]   /_/  / .___//_/ \__,_/ \__, /
[info]       /_/               /____/
[info] 
[info] Version 3.0.6 running Java 17.0.14
[info] 
[info] Play is run entirely by the community. Please consider contributing and/or donating:
[info] https://www.playframework.com/sponsors
[info] 

--- (Running the application, auto-reloading is enabled) ---

INFO  p.c.s.PekkoHttpServer - Listening for HTTP on /[0:0:0:0:0:0:0:0]:9000

(Server started, use Enter to stop and go back to the console...)

With my local server up and listening on port 9000, I open a new terminal and test the API by sending a request:

$ curl http://localhost:9000/v1/posts | jq

[
  {
    "id": "1",
    "link": "/v1/posts/1",
    "title": "title 1",
    "body": "blog post 1"
  },
  {
    "id": "2",
    "link": "/v1/posts/2",
    "title": "title 2",
    "body": "blog post 2"
  },
  {
    "id": "3",
    "link": "/v1/posts/3",
    "title": "title 3",
    "body": "blog post 3"
  },
  {
    "id": "4",
    "link": "/v1/posts/4",
    "title": "title 4",
    "body": "blog post 4"
  },
  {
    "id": "5",
    "link": "/v1/posts/5",
    "title": "title 5",
    "body": "blog post 5"
  }
]

Nice. That was fast. Next, I want to play around with a few requests — retrieving and creating a blog post.

$ curl http://localhost:9000/v1/posts/3 | jq

{
  "id": "3",
  "link": "/v1/posts/3",
  "title": "title 3",
  "body": "blog post 3"
}


$ curl -X POST \
    --header "Content-type:application/json" \
    --data '{ "title": "Just Another Blog Post", "body": "This is my blog post body." }' \
    http://localhost:9000/v1/posts | jq


{
  "id": "999",
  "link": "/v1/posts/999",
  "title": "Just Another Blog Post",
  "body": "This is my blog post body."
}

Preparing for Deployment to Heroku

Ok, we’re ready to deploy our Scala Play app to Heroku. However, being new to Scala and Play, I’m predisposed to hitting a few speed bumps. I want to cover those so that you can steer clear of them in your development.

Understanding the `AllowedHostsFilter`

When I run my app locally with sbt run, I have no problems sending curl requests and receiving responses. But as soon as I’m in the cloud, I’m using a Heroku app URL, not localhost. For security, Play has an AllowedHostsFilter enabled, which means you need to specify explicitly which hosts can access the app.

I modify conf/application.conf to include the following block:

play.filters.hosts {
  allowed = [
    "localhost:9000",
    ${?PLAY_ALLOWED_HOSTS}
  ]
}

This way, I can set the PLAY_ALLOWED_HOSTS config variable for my Heroku app, adding my Heroku app URL to the list of allowed hosts.

Setting a Play secret

Play requires an application secret for security. It’s used for signing and encryption. You could set the application secret in conf/application.conf, but that hardcodes it into your repo, which isn’t a good practice. Instead, let’s set it at runtime based on an environment config variable. We add the following lines to conf/application.conf:

play.http.secret.key="changeme"
play.http.secret.key=${?PLAY_SECRET}

The first line sets a default secret key, while the second line sets the secret to come from our config variable, PLAY_SECRET, if it is set.

sbt run versus running the compiled binary

Lastly, I want to talk a little bit about sbt run. But first, let’s rewind: on my first attempt at deploying to Heroku, I thought I would just spin up my server by having Heroku execute sbt run. When I did that, my dyno kept crashing. I didn’t realize how memory-intensive this mode was. It’s meant for development, not production.

During that first attempt, I turned on log-runtime-metrics for my Heroku app. My app was using over 800M in memory — way too much for my little Eco Dyno which was only 512M. If I wanted to use sbt run, I would have needed a Performance-M dyno. That didn’t seem right.

As I read Play’s documentation on deploying an application, I realized that I should run my app through the compiled binary that Play creates via another command, sbt stage. This version of my app is precompiled and much more memory efficient — down to around 180MB. Assuming sbt stage would run first, I just needed to modify my startup command to run the binary.

Why I went with Heroku

I chose Heroku for running my Scala app because it removes a lot of the setup friction. I don’t need to manually configure a server or install dependencies. Heroku knows that this is a Scala app (by detecting the presence of project/build.properties and build.sbt files) and applies its buildpack for Scala. This means automatic handling of things like dependency resolution and compilation by running sbt stage.

For someone just experimenting with Scala and Play, this kind of zero-config deployment is ideal. I can focus on understanding the framework and the codebase without getting sidetracked by infrastructure. Once I sorted out a few gotchas from early on — like the AllowedHostsFilter, Play secret, and startup command — deployment was quick and repeatable.

Alright, let’s go!

Deploying the app to Heroku

For deployment, I have the Heroku CLI installed, and I authenticate with heroku login. My first step is to create a new Heroku app.

~/project$ heroku apps:create scala-blog-rest-api


Creating ⬢ scala-blog-rest-api... done
https://scala-blog-rest-api-5d26d52bd1e4.herokuapp.com/ | https://git.heroku.com/scala-blog-rest-api.git

Create the Procfile

Next, I need to create the Procfile that tells Heroku how to spin up my app. In my project root, I create the file, which contains this one line:

web: target/universal/stage/bin/play-scala-rest-api-example -Dhttp.port=${PORT}

Notice that the command for starting up this Heroku web process is not sbt run. Instead, we execute the binary found in target/universal/stage/bin/play-scala-rest-api-example. That’s the binary compiled after running sbt stage. Where does the play-scala-rest-api-example file name come from? This is set in build.sbt. Make sure that name is consistent between build.sbt and your Procfile.

Also, we set the port dynamically at runtime to the value of the environment variable PORT. Heroku sets the PORT environment variable when it starts up our app, so we just need to let our app know what that port is.

Specify the JDK version

Next, I create a one-line file called system.properties, specifying the JDK version I want to use for my app. Since my local machine uses openjdk 17, my system.properties file looks like this:

java.runtime.version=17

Set Heroku app config variables

To make sure everything is in order, we need to set a few config variables for our app:

PLAY_SECRET: A string of our choosing, required to be at least 256 bits.
PLAY_ALLOWED_HOSTS: Our Heroku app URL, which is then included in the AllowedHostsFilter used in conf/application.conf.

We set our config variables like this:

~/project$ heroku config:set \
  PLAY_SECRET='ga87Dd*A7$^SFsrpywMWiyyskeEb9&D$hG!ctWxrp^47HCYI' \
  PLAY_ALLOWED_HOSTS='scala-blog-rest-api-5d26d52bd1e4.herokuapp.com'

Setting PLAY_SECRET, PLAY_ALLOWED_HOSTS and restarting ⬢ scala-blog-rest-api... done, v2
PLAY_ALLOWED_HOSTS: scala-blog-rest-api-5d26d52bd1e4.herokuapp.com
PLAY_SECRET:        ga87Dd*A7$^SFsrpywMWiyyskeEb9&D$hG!ctWxrp^47HCYI

Push code to Heroku

With our Procfile created and our config variables set, we’re ready to push our code to Heroku.

~/project$ git push heroku main


remote: Resolving deltas: 100% (14/14), done.
remote: Updated 95 paths from 4dc4853
remote: Compressing source files... done.
remote: Building source:
remote: 
remote: -----> Building on the Heroku-24 stack
remote: -----> Determining which buildpack to use for this app
remote: -----> Play 2.x - Scala app detected
remote: -----> Installing Azul Zulu OpenJDK 17.0.14
remote: -----> Priming Ivy cache... done
remote: -----> Running: sbt compile stage
remote: -----> Collecting dependency information
remote: -----> Dropping ivy cache from the slug
remote: -----> Dropping sbt boot dir from the slug
remote: -----> Dropping sbt cache dir from the slug
remote: -----> Dropping compilation artifacts from the slug
remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote: 
remote: -----> Compressing...
remote:        Done: 128.4M
remote: -----> Launching...
remote:        Released v5
remote:        https://scala-blog-rest-api-5d26d52bd1e4.herokuapp.com/ deployed to Heroku
remote: 
remote: Verifying deploy... done.
To https://git.heroku.com/scala-blog-rest-api.git
 * [new branch]      main -> main

I love how I only need to create a Procfile and set some config variables, and then Heroku takes care of the rest of it for me.

Time to test

All that’s left to do is test my Scala Play app with a few curl requests:

$ curl https://scala-blog-rest-api-5d26d52bd1e4.herokuapp.com/v1/posts \
       | jq

[
  {
    "id": "1",
    "link": "/v1/posts/1",
    "title": "title 1",
    "body": "blog post 1"
  },
  {
    "id": "2",
    "link": "/v1/posts/2",
    "title": "title 2",
    "body": "blog post 2"
  },
  {
    "id": "3",
    "link": "/v1/posts/3",
    "title": "title 3",
    "body": "blog post 3"
  },
  {
    "id": "4",
    "link": "/v1/posts/4",
    "title": "title 4",
    "body": "blog post 4"
  },
  {
    "id": "5",
    "link": "/v1/posts/5",
    "title": "title 5",
    "body": "blog post 5"
  }
]

$ curl https://scala-blog-rest-api-5d26d52bd1e4.herokuapp.com/v1/posts/4 \
       | jq

{
  "id": "4",
  "link": "/v1/posts/4",
  "title": "title 4",
  "body": "blog post 4"
}

$ curl -X POST \
    --header "Content-type:application/json" \
    --data '{"title":"My blog title","body":"this is my blog post"}' \
    https://scala-blog-rest-api-5d26d52bd1e4.herokuapp.com/v1/posts \
    | jq

{
  "id": "999",
  "link": "/v1/posts/999",
  "title": "My blog title",
  "body": "this is my blog post"
}

The API server works. Deployment was smooth and simple. And I learned a lot about Scala and Play along the way. All in all … it’s been a good day.

Wrapping Up

Jumping into Scala after decades away from Java — and being more used to building web apps with JavaScript — wasn’t as jarring as I expected. Scala’s syntax felt concise and modern, and working with case classes and async controllers reminded me a bit of patterns I already use in Node.js. There’s still a learning curve with the tooling and how things are structured in Play — but overall, it felt approachable, not overwhelming.

Deployment to the cloud had a few gotchas, especially around how Play handles allowed hosts, secrets, and memory usage in production. But once I understood how those pieces worked, getting the app live on Heroku was straightforward. With just a few config changes and a proper startup command, the process was clean and repeatable.

For a first-time Scala build and deployment, I couldn’t have asked for a smoother experience. Are you ready to give it a try?

Happy coding!

Have You Heard About Cloud Native Buildpacks?

Alvin Lee — Tue, 11 Mar 2025 15:11:02 +0000

Do you ever get tired of fiddling with a Dockerfile? Dockerfiles and Docker images are a great way to package your app for reusable, containerized deployments. However, writing and maintaining a Dockerfile is not always intuitive, and it takes up time that could otherwise be used for adding features to your app. Enter Cloud Native Buildpacks. Buildpacks exist to pull together everything your app needs to run and put it into an Open Container Initiative (OCI) image — no Dockerfile required.

For all the developers out there who need a container build process that’s easy to use and will save them time and headaches, Cloud Native Buildpacks might be the solution they’re looking for. Interested? I’ll tell you more.

What Are Cloud Native Buildpacks?

Broadly speaking, a buildpack takes application code and makes it runnable through a build process. So then, Cloud Native Buildpacks take your application source code and turn it into runnable, reproducible OCI images, implementing your requirements for image security, performance optimization, and container build order. It’s like having the exact Dockerfile you need — only you don’t need to write one.

While most developers can write a Dockerfile, few are experts in either Docker or infrastructure. Too many apps have Dockerfiles that are cobbled together from code snippets found across the web — often a mash-up of Copilot, Stack Overflow, and ChatGPT. Dockerfile errors can lead to insecure and poorly performing applications.

Cloud Native Buildpacks take on this burden, automatically applying best practices for each language or framework. A builder can then utilize any number of buildpacks, automatically detecting which buildpacks are needed and applying them to build an application. Here are the buildpacks that Heroku’s builder currently supports:

$ pack builder inspect heroku/builder:24
Inspecting builder: heroku/builder:24

REMOTE:

Description: Ubuntu 24.04 AMD64+ARM64 base image with buildpacks for .NET, Go, Java, Node.js, PHP, Python, Ruby & Scala.

...

Buildpacks:
  ID                           NAME                         VERSION
  heroku/deb-packages          Heroku .deb Packages         0.0.3  
  heroku/dotnet                Heroku .NET                  0.1.10 
  heroku/go                    Heroku Go                    0.5.2  
  heroku/gradle                Heroku Gradle                6.0.4  
  heroku/java                  Heroku Java                  6.0.4  
  heroku/jvm                   Heroku OpenJDK               6.0.4  
  heroku/maven                 Heroku Maven                 6.0.4  
  heroku/nodejs                Heroku Node.js               3.4.5  
  heroku/nodejs-corepack       Heroku Node.js Corepack      3.4.5  
  heroku/nodejs-engine         Heroku Node.js Engine        3.4.5  
  heroku/nodejs-npm-engine     Heroku Node.js npm Engine    3.4.5  
  heroku/nodejs-npm-install    Heroku Node.js npm Install   3.4.5  
  heroku/nodejs-pnpm-engine    Heroku Node.js pnpm Engine   3.4.5  
  heroku/nodejs-pnpm-install   Heroku Node.js pnpm install  3.4.5  
  heroku/nodejs-yarn           Heroku Node.js Yarn          3.4.5  
  heroku/php                   Heroku PHP                   0.2.0  
  heroku/procfile              Heroku Procfile              4.0.0  
  heroku/python                Heroku Python                0.23.0 
  heroku/ruby                  Heroku Ruby                  5.0.1  
  heroku/sbt                   Heroku sbt                   6.0.4  
  heroku/scala                 Heroku Scala                 6.0.4

Other builders, like the ones from Paketo or Google Cloud, also bring an array of buildpacks. All in all, the Cloud Native Buildpacks ecosystem is growing and maturing, which is exciting for developers!

For those of you familiar with Heroku, you’ve already been enjoying the buildpack experience. With git push heroku main, you’ve been able to deploy directly to Heroku, with no Dockerfile required. Cloud Native Buildpacks build on the Heroku buildpack experience, taking what was once a vendor-specific implementation and turning it into a CNCF standard that’s usable on any cloud platform.

In short, Cloud Native Buildpacks allow developers to:

Deploy applications more easily than ever
… in a standard-based fashion without lock-in
… all while applying container best practices
… and without making developers tinker with Dockerfiles.

Use Cases

Sounds great, right? With all these benefits, let’s look at some specific cases where you could benefit from using Cloud Native Buildpacks.

Any place where you would ordinarily need a Dockerfile is an opportunity to use a buildpack. Examples include:

A Node.js web application
A Python microservice
A heterogeneous application that uses multiple languages or frameworks
Building applications for deployment on cloud platforms such as AWS, Azure, and Heroku

One thing to note is this: While buildpacks are declarative, Dockerfiles are procedural. With a buildpack, you simply declare that you want a given application built with a given builder or buildpack. In contrast, a Dockerfile requires you to define the commands and the order in which those commands are run to build your application. As such, buildpacks don’t currently offer the level of configurability that’s available within a Dockerfile, so it might not meet the needs of some more advanced use cases.

That said, there is no vendor lock-in with Cloud Native Buildpacks. They simply build an OCI image. Need more customization and options than are available in the buildpack? Simply replace the builder in your build pipeline with your Dockerfile and a standard OCI image build, and you are good to go.

A Simple Walkthrough

Let’s do a quick walkthrough of how to use Cloud Native Buildpacks.

To get started with buildpacks as an app developer, your first step should be to install the Pack CLI tool. This tool allows you to build an application with buildpacks. Follow the installation instructions for your operating system.

Additionally, if you don’t have it already, you’ll need a Docker daemon for the builder to build your app, and for you to run your image. With these two tools installed, you’re ready to begin.

Build a sample app

With access to the pack tool, you’re ready to try it out by building a sample application. I’ll be running this inside a Next.js application. Need a sample application to test out the buildpack on? Here is a full directory of Next.js sample applications. You can also try out any application you have on hand.

Once you have your application ready, start by seeing what builder the pack tool suggests. In your shell, navigate to your app directory and run this command:

$ pack builder suggest

On my Ubuntu installation, for my Next.js application, the pack tool suggests the following builders:

Let’s try the suggested Heroku buildpack (heroku/builder:24). To use this one, run the following command:

$ pack build my-app --builder heroku/builder:24

Build time will vary depending on the size of your application; for me, building the app took 30 seconds. With that, my image was ready to go. We can run the image with the following:

$ docker run -p 3000:3000 my-app

The result looks like this:

And that’s it! We’ve successfully built an OCI image of our Next.js application — without using a Dockerfile.

Additional configurations

What if you need to configure something inside the buildpack? For this, you would reference the buildpack(s) that were selected by your builder. For example, for my Next.js app, I can see in the logs that the builder selected two buildpacks: nodejs-engine and nodejs-yarn.

Let’s say that I want to specify the yarn version used by the buildpack. First, I would go to the nodejs-yarn buildpack Readme, where I see that I can specify the yarn version in my package.json file with a packageManager key. I would modify my file to look like this:

{
  "packageManager": "yarn@1.22.22"
}

From there, all I would need to do is run pack build my-app --builder heroku/builder:24 again.

Conclusion

Cloud Native Buildpacks are an exciting new way to build container images for our applications. By removing the need for a Dockerfile, they make it faster than ever to get our application packaged and deployed. Plus, as they build standard container images, there is no vendor lock-in.

Cloud Native Buildpacks are in preview on many platforms, which means that the feature set is light but fast growing. Heroku, which has open sourced their Cloud Native Buildpacks, is bringing them to their next generation platform, too. I’m looking forward to seeing how Cloud Native Buildpacks enable secure, speedy application deployment across the cloud platform community.

FastHTML and Heroku

Alvin Lee — Mon, 13 Jan 2025 18:38:28 +0000

When creating a new app or service, what begins as learning just one new tool can quickly turn into needing a whole set of tools and frameworks. For Python devs, jumping into HTML, CSS, and JavaScript to build a usable app can be daunting. For web devs, many Python-first backend tools work in JavaScript but are often outdated. You’re left with a choice: Stick with JavaScript or switch to Python for access to the latest features.

FastHTML bridges the gap between these two groups. For Python devs, it makes creating a web app straightforward—no JavaScript required! For web devs, it makes creating a Python app quick and easy, with the option to extend using JavaScript—you’re not locked in.

As a web developer, I’m always looking for ways to make Python dev more accessible. So, let’s see how quickly we can build and deploy a FastHTML app. I’ll follow the image generation tutorial and then deploy it to Heroku. Let’s go!

Intro to FastHTML

Never heard of FastHTML before? Here’s how FastHTML describes itself:

FastHTML is a new next-generation web framework for fast, scalable web applications with minimal, compact code. It’s designed to be:

Powerful and expressive enough to build the most advanced, interactive web apps you can imagine.
Fast and lightweight, so you can write less code and get more done.
Easy to learn and use, with a simple, intuitive syntax that makes it easy to build complex apps quickly.

FastHTML promises to enable you to generate usable, lightweight apps quickly. Too many web apps are bloated and heavy, requiring a lot of processing and bandwidth for simple tasks. Most web apps just need something simple, beautiful, and easy to use. FastHTML aims to make that task easy.

You may have heard of FastAPI, designed to make creating APIs with Python a breeze. FastHTML is inspired by FastAPI’s philosophy, seeking to do the same for frontend applications.

Opinionated about simplicity and ease of use

Part of the FastHTML vision is to “make it the easiest way to create quick prototypes, and also the easiest way to create scalable, powerful, rich applications.” As a developer tool, FastHTML seems to be opinionated about the right things—simplicity and ease of use without limiting you in the future.

FastHTML gets you up and running quickly while also making it easy for your users. It does this by selecting key core technologies such as ASGI and HTMX. The foundations page from FastHTML introduces these technologies and gives the basics (though you don’t need to know about these to get started).

Get Up and Running Quickly

The tutorials from FastHTML offer several examples of different apps, each with its own use case. I was curious about the Image Generation App tutorial and wanted to see how quickly I could get a text-to-image model into a real, working app. The verdict? It was fast. Really fast.

In less than 60 lines of code, I created a fully functioning web app where a user can type in a prompt and receive an image from the free Pollinations text-to-image model.

Here’s a short demo of the tutorial app:

In this tutorial app, I got a brief glimpse of the power of FastHTML. I learned how to:

Submit data through a form
Interact with external APIs
Display some loading text while waiting

What’s impressive is that it only took one tiny Python file to complete this, and the final app is lightweight and looks good. Here’s the file I ended up with:

from fastcore.parallel import threaded
from fasthtml.common import *
import os, uvicorn, requests, replicate
from PIL import Image

app = FastHTML(hdrs=(picolink,))

# Store our generations
generations = []
folder = f"gens/"
os.makedirs(folder, exist_ok=True)

# Main page
@app.get("/")
def home():
    inp = Input(id="new-prompt", name="prompt", placeholder="Enter a prompt")
    add = Form(Group(inp, Button("Generate")), hx_post="/", target_id='gen-list', hx_swap="afterbegin")
    gen_list = Div(id='gen-list')
    return Title('Image Generation Demo'), Main(H1('Magic Image Generation'), add, gen_list, cls='container')

# A pending preview keeps polling this route until we return the image preview
def generation_preview(id):
    if os.path.exists(f"gens/{id}.png"):
        return Div(Img(src=f"/gens/{id}.png"), id=f'gen-{id}')
    else:
        return Div("Generating...", id=f'gen-{id}', 
                   hx_post=f"/generations/{id}",
                   hx_trigger='every 1s', hx_swap='outerHTML')

@app.post("/generations/{id}")
def get(id:int): return generation_preview(id)


# For images, CSS, etc.
@app.get("/{fname:path}.{ext:static}")
def static(fname:str, ext:str): return FileResponse(f'{fname}.{ext}')

# Generation route
@app.post("/")
def post(prompt:str):
    id = len(generations)
    generate_and_save(prompt, id)
    generations.append(prompt)
    clear_input =  Input(id="new-prompt", name="prompt", placeholder="Enter a prompt", hx_swap_oob='true')
    return generation_preview(id), clear_input

# URL (for image generation)  
def get_url(prompt): 
    return f"https://image.pollinations.ai/prompt/{prompt.replace(' ', '%20')}?model=flux&width=1024&height=1024&seed=42&nologo=true&enhance=true"

@threaded
def generate_and_save(prompt, id):
    full_url = get_url(prompt)
    Image.open(requests.get(full_url, stream=True).raw).save(f"{folder}/{id}.png")
    return True

if __name__ == '__main__': uvicorn.run("app:app", host='0.0.0.0', port=int(os.getenv("PORT", default=5000)))

Looking for more functionality? The tutorial continues, adding some CSS styling, user sessions, and even payment tracking with Stripe. While I didn’t go through it all the way, the potential is clear: lots of functionality and usability without a lot of boilerplate or using both Python and JavaScript.

Deploy Quickly to Heroku

Okay, so now that I have a pure Python app running locally, what do I need to do to deploy it? Heroku makes this easy. I added a single file called Procfile with just one line in it:

web: python app.py

This simple text file tells Heroku how to run the app. With the Procfile in place, I can use the Heroku CLI to create and deploy my app. And it’s fast…from zero to done in less than 45 seconds.

With two commands, I created my project, built it, and deployed it to Heroku. And let’s just do a quick check. Did it actually work?

And it’s up for the world to see!

Conclusion

When I find a new tool that makes it easier and quicker to build an app, my mind starts spinning with the possibilities. If it’s that easy, then maybe next time I need to spin up something, I can do it this way and integrate it with this tool, and that other thing… So much of programming is assembling the right tools for the job.

FastHTML has opened the door to a whole set of Python-based applications for me, and Heroku makes it easy to get those apps off my local machine and into the world. That said, several of the foundations of FastHTML are new to me, and I look forward to understanding them more deeply as I use it more.

I hope you have fun with FastHTML and Heroku! Happy coding!

Enhancing GenAI Applications With KubeMQ: Efficiently Scaling Retrieval-Augmented Generation (RAG)

Alvin Lee — Thu, 12 Dec 2024 15:41:03 +0000

As the adoption of Generative AI (GenAI) surges across industries, organizations are increasingly leveraging Retrieval-Augmented Generation (RAG) techniques to bolster their AI models with real-time, context-rich data. Managing the complex flow of information in such applications poses significant challenges, particularly when dealing with continuously generated data at scale. KubeMQ, a robust message broker, emerges as a solution to streamline the routing of multiple RAG processes, ensuring efficient data handling in GenAI applications.

To further enhance the efficiency and scalability of RAG workflows, integrating a high-performance database like FalkorDB is essential. FalkorDB provides a reliable and scalable storage solution for the dynamic knowledge bases that RAG systems depend on, ensuring rapid data retrieval and seamless integration with messaging systems like KubeMQ.

Understanding RAG in GenAI Workflows

RAG is a paradigm that enhances generative AI models by integrating a retrieval mechanism, allowing models to access external knowledge bases during inference. This approach significantly improves the accuracy, relevance, and timeliness of generated responses by grounding them in the most recent and pertinent information available.

In typical GenAI workflows employing RAG, the process involves multiple steps:

Query processing: Interpreting the user's input to understand intent and context
Retrieval: Fetching relevant documents or data from a dynamic knowledge base, such as FalkorDB, which ensures quick and efficient access to the most recent and pertinent information.
Generation: Producing a response using both the input and the retrieved data
Response delivery: Providing the final, enriched output back to the user

Scaling these steps, especially in environments where data is continuously generated and updated, necessitates an efficient and reliable mechanism for data flow between the various components of the RAG pipeline.

The Critical Role of KubeMQ in RAG Processing

Handling Continuous Data Streams at Scale

In scenarios such as IoT networks, social media platforms, or real-time analytics systems, new data is incessantly produced, and AI models must adapt swiftly to incorporate this information. Traditional request-response architectures can become bottlenecks under high-throughput conditions, leading to latency issues and degraded performance.

KubeMQ manages high-throughput messaging scenarios by providing a scalable and robust infrastructure for efficient data routing between services. By integrating KubeMQ into the RAG pipeline, each new data point is published to a message queue or stream, ensuring that retrieval components have immediate access to the latest information without overwhelming the system. This real-time data handling capability is crucial for maintaining the relevance and accuracy of GenAI outputs.

Serving as the Optimal Router

KubeMQ offers a variety of messaging patterns — including queues, streams, publish-subscribe (pub/sub), and Remote Procedure Calls (RPC) — making it a versatile and powerful router within a RAG pipeline. Its low latency and high-performance characteristics ensure prompt message delivery, which is essential for real-time GenAI applications where delays can significantly impact user experience and system efficacy.

Moreover, KubeMQ's ability to handle complex routing logic allows for sophisticated data distribution strategies. This ensures that different components of the AI system receive precisely the data they need, when they need it, without unnecessary duplication or delays.

Integrating FalkorDB for Enhanced Data Management

While KubeMQ efficiently routes messages between services, FalkorDB complements this by providing a scalable and high-performance graph database solution for storing and retrieving the vast amounts of data required by RAG processes. This integration ensures that as new data flows through KubeMQ, it is seamlessly stored in FalkorDB, making it readily available for retrieval operations without introducing latency or bottlenecks.

Enhancing Scalability and Reliability

As GenAI applications grow in both user base and data volume, scalability becomes a paramount concern. KubeMQ is scalable, supporting horizontal scaling to accommodate increased load seamlessly. It ensures that as the number of RAG processes increases or as data generation accelerates, the messaging infrastructure remains robust and responsive.

Additionally, KubeMQ provides message persistence and fault tolerance. In the event of system failures or network disruptions, KubeMQ ensures that messages are not lost and that the system can recover gracefully. This reliability is critical in maintaining the integrity of AI applications that users depend on for timely and accurate information.

Eliminating the Need for Dedicated Routing Services

Implementing custom routing services for data handling in RAG pipelines can be resource-intensive and complex. It often requires significant development effort to build, maintain, and scale these services, diverting focus from core AI application development.

By adopting KubeMQ, organizations eliminate the need to create bespoke routing solutions. KubeMQ provides out-of-the-box functionality that addresses the routing needs of RAG processes, including complex routing patterns, message filtering, and priority handling. This not only reduces development and maintenance overhead but also accelerates time-to-market for GenAI solutions.

Unified Access via REST and SDK

KubeMQ offers multiple interfaces for interacting with its message broker capabilities:

REST API: Enables language-agnostic integration, allowing services written in any programming language to send and receive messages over HTTP
SDKs: Provides client libraries for various programming languages (such as Python, Java, Go, and .NET), facilitating more efficient communication patterns and better performance through native integrations

This flexibility allows developers to choose the most appropriate method for their specific use case, simplifying the architecture and accelerating development cycles. A single touchpoint for data routing streamlines communication between different components of the RAG pipeline, enhancing overall system coherence.

Implementing KubeMQ in a RAG Pipeline: A Detailed Example

The code example showcases how to build a movie information retrieval system by integrating KubeMQ into a RAG pipeline. It sets up a server that ingests movie URLs from Rotten Tomatoes to build a knowledge graph using GPT-4. Users can interact with this system through a chat client, sending movie-related queries and receiving AI-generated responses. This use case demonstrates how to handle continuous data ingestion and real-time query processing in a practical application, utilizing KubeMQ for efficient message handling and inter-service communication within the context of movies.

Architecture Overview

Data ingestion service: Captures and publishes new data to KubeMQ streams as it becomes available
Retrieval service: Subscribe to the KubeMQ stream to receive updates and refresh the knowledge base
Generation service: Listens for query requests, interacts with the AI model, and generates responses
Response service: Sends the generated responses back to users through appropriate channels

Setting Up KubeMQ

Ensure that KubeMQ is operational, which can be achieved by deploying it using Docker:

docker run -d --rm \
  -p 8080:8080 \
  -p 50000:50000 \
  -p 9090:9090 \
  -e KUBEMQ_TOKEN="your token"

This command starts KubeMQ with the necessary ports exposed for REST and gRPC communications.

RAG Server Side

This code (GitHub repo) implements a RAG server that processes chat queries and manages knowledge sources using KubeMQ for message handling.

# server.py

import json
import threading
from typing import List

from dotenv import load_dotenv
load_dotenv()
import time
from kubemq.common import CancellationToken
from kubemq.cq import Client as CQClient, QueryMessageReceived, QueryResponseMessage, QueriesSubscription
from kubemq.queues import Client as QueuesClient
from graphrag_sdk.models.openai import OpenAiGenerativeModel
from graphrag_sdk.model_config import KnowledgeGraphModelConfig
from graphrag_sdk import KnowledgeGraph, Ontology
from graphrag_sdk.source import URL

class RAGServer:
   def __init__(self):
       self.cq_client = CQClient(address="localhost:50000")
       self.queues_client = QueuesClient(address="localhost:50000")
       model = OpenAiGenerativeModel(model_name="gpt-4o")
       with open("ontology.json", "r") as f:
           ontology = json.load(f)
       ontology = Ontology.from_json(ontology)
       self.kg = KnowledgeGraph(
           name="movies",
           model_config=KnowledgeGraphModelConfig.with_model(model),
           ontology=ontology)
       self.chat = self.kg.chat_session()
       self.shutdown_event = threading.Event()
       self.threads: List[threading.Thread] = []

   def handle_chat(self, request: QueryMessageReceived):
       try:
           message = request.body.decode('utf-8')
           print(f"Received chat message: {message}")
           result= self.chat.send_message(message)
           answer = result.get("response","No answer")
           print(f"Chat response: {answer}")
           response = QueryResponseMessage(
               query_received=request,
               is_executed=True,
               body=answer.encode('utf-8')
           )
           self.cq_client.send_response_message(response)
       except Exception as e:
           print(f"Error processing chat message: {str(e)}")
           self.cq_client.send_response_message(QueryResponseMessage(
               query_received=request,
               is_executed=False,
               error=str(e)
           ))

   def pull_from_queue(self):
       while not self.shutdown_event.is_set():
           try:
               result = self.queues_client.pull("rag-sources-queue", 10, 1)
               if result.is_error:
                   print(f"Error pulling message from queue: {result.error}")
                   continue
               sources = []
               for message in result.messages:
                   source = message.body.decode('utf-8')
                   print(f"Received source: {source}, adding to knowledge graph")
                   sources.append(URL(message.body.decode('utf-8')))
               if sources:
                   self.kg.process_sources(sources)
           except Exception as e:
               if not self.shutdown_event.is_set():  # Only log if not shutting down
                   print(f"Error processing sources: {str(e)}")

   def subscribe_to_chat_queries(self):
       def on_error(err: str):
           if not self.shutdown_event.is_set():  # Only log if not shutting down
               print(f"Error: {err}")

       cancellation_token = CancellationToken()

       try:
           self.cq_client.subscribe_to_queries(
               subscription=QueriesSubscription(
                   channel="rag-chat-query",
                   on_receive_query_callback=self.handle_chat,
                   on_error_callback=on_error,
               ),
               cancel=cancellation_token
           )

           # Wait for shutdown signal
           while not self.shutdown_event.is_set():
               time.sleep(0.1)


           # Cancel subscription when shutdown is requested
           cancellation_token.cancel()

       except Exception as e:
           if not self.shutdown_event.is_set():
               print(f"Error in subscription thread: {str(e)}")
   def run(self):

       chat_thread = threading.Thread(target=self.subscribe_to_chat_queries)
       queue_thread = threading.Thread(target=self.pull_from_queue)

       self.threads.extend([chat_thread, queue_thread])

       for thread in self.threads:
           thread.daemon = True  # Make threads daemon so they exit when main thread exits
           thread.start()

       print("RAG server started")
       try:
           while True:
               time.sleep(1)
       except KeyboardInterrupt:
           print("\nShutting down gracefully...")
           self.shutdown()
           self.cq_client.close()
           self.queues_client.close()

   def shutdown(self):

       print("Initiating shutdown sequence...")
       self.shutdown_event.set()  # Signal all threads to stop

       for thread in self.threads:
           thread.join(timeout=5.0)  # Wait up to 5 seconds for each thread
           if thread.is_alive():
               print(f"Warning: Thread {thread.name} did not shutdown cleanly")

       print("Shutdown complete")
if __name__ == "__main__":
   rag_server = RAGServer()
   rag_server.run()

The server runs two main threads: one that subscribes to chat queries through a channel called "rag-chat-query" and processes them using a knowledge graph with GPT-4, and another that continuously pulls from a queue called "rag-sources-queue" to add new sources to the knowledge graph. The knowledge graph is initialized with a custom ontology loaded from a JSON file and uses OpenAI's GPT-4 model for processing. The server implements graceful shutdown handling and error management, ensuring that all threads are properly terminated when the server is stopped.

Sending Source Data to Ingest Into RAG Knowledge Graph

# sources_client.py

from kubemq.queues import *

class SourceClient:
   def __init__(self, address="localhost:50000"):
       self.client = Client(address=address)

   def send_source(self, message: str) :
       send_result = self.client.send_queues_message(
           QueueMessage(
               channel="rag-sources-queue",
               body=message.encode("utf-8"),
           )
       )
       if send_result.is_error:
           print(f"message send error, error:{send_result.error}")

if __name__ == "__main__":
   client = SourceClient()
   urls = ["https://www.rottentomatoes.com/m/side_by_side_2012",
       "https://www.rottentomatoes.com/m/matrix",
       "https://www.rottentomatoes.com/m/matrix_revolutions",
       "https://www.rottentomatoes.com/m/matrix_reloaded",
       "https://www.rottentomatoes.com/m/speed_1994",
       "https://www.rottentomatoes.com/m/john_wick_chapter_4"]
   for url in urls:
       client.send_source(url)
   print("done")

This code implements a simple client that sends movie URLs to the RAG server through KubeMQ's queue system. Specifically, it creates a SourceClient class that connects to KubeMQ and sends messages to the "rag-sources-queue" channel, which is the same queue that the RAG server monitors. When run as a main program, it sends a list of Rotten Tomatoes movie URLs (including Matrix movies, John Wick, and Speed) to be processed and added to the knowledge graph by the RAG server.

Send and Receive Questions and Answers

#chat_client.py

from kubemq.cq import *

class ChatClient:
   def __init__(self, address="localhost:50000"):
       self.client = Client(address=address)

   def send_message(self, message: str) -> str:
       response = self.client.send_query_request(QueryMessage(
           channel="rag-chat-query",
           body=message.encode('utf-8'),
           timeout_in_seconds=30
       ))
       return response.body.decode('utf-8')

if __name__ == "__main__":
   client = ChatClient()
   print("Sending first question:  Who is the director of the movie The Matrix?")
   response = client.send_message("Who is the director of the movie The Matrix?")
   print(f"Response: {response}")
   print("Sending second question:  How this director connected to Keanu Reeves?")
   response = client.send_message("How this director connected to Keanu Reeves?")
   print(f"Response: {response}")

This code implements a chat client that communicates with the RAG server through KubeMQ's query system. The ChatClient class sends messages to the "rag-chat-query" channel and waits for responses, with a 30-second timeout for each query. When run as a main program, it demonstrates the client's functionality by sending two related questions about The Matrix's director and their connection to Keanu Reeves, printing each response as it receives them.

Code Repository

All code examples can be found in my fork of the original GitHub repository.

Conclusion

Integrating KubeMQ into RAG pipelines for GenAI applications provides a scalable, reliable, and efficient mechanism for handling continuous data streams and complex inter-process communications. By serving as a unified router with versatile messaging patterns, KubeMQ simplifies the overall architecture, reduces the need for custom routing solutions, and accelerates development cycles.

Furthermore, incorporating FalkorDB enhances data management by offering a high-performance knowledge base that seamlessly integrates with KubeMQ. This combination ensures optimized data retrieval and storage, supporting the dynamic requirements of RAG processes.

The ability to handle high-throughput scenarios, combined with features like persistence and fault tolerance, ensures that GenAI applications remain responsive and reliable, even under heavy loads or in the face of system disruptions.

By leveraging KubeMQ and FalkorDB, organizations can focus on enhancing their AI models and delivering valuable insights and services, confident that their data routing infrastructure is robust and capable of meeting the demands of modern AI workflows.

Playwright and Chrome Browser Testing in Heroku

Alvin Lee — Tue, 03 Dec 2024 15:59:09 +0000

I’ve always loved watching my unit tests run (and pass). They’re fast, and passing tests give me the assurance that my individual pieces behave like they’re supposed to. Conversely, I often struggled to prioritize end-to-end tests for the browser because writing and running them was gruelingly slow.

Fortunately, the tools for end-to-end in-browser testing have gotten much better and faster over the years. And with a headless browser setup, I can run my browser tests as part of my CI.

Recently, I came across this Heroku blog post talking about automating in-browser testing with headless Chrome within Heroku CI. Heroku has a buildpack that installs headless Chrome, which you can invoke for your tests in the CI pipeline.

The example setup from the blog post was a React app tested with Puppeteer and Jest. That’s a great start … but what if I use Playwright instead of Puppeteer? Is it possible?

I decided to investigate. As it turns out — yes, you can do this with Playwright too! So, I captured the steps you would need to get Playwright tests running on the headless Chrome browser used in Heroku CI. In this post, I’ll walk you through the steps to get set up.

A Quick Word on Browser Automation for End-to-End Testing

End-to-end testing captures how users actually interact with your app in a browser, validating complete workflows. Playwright makes this process pretty seamless with testing in Chrome, Firefox, and Safari. Of course, running a full slate of browser tests in CI is pretty heavy, which is why headless mode helps.

The Chrome for Testing buildpack from Heroku installs Chrome on a Heroku app, so you can run your Playwright tests in Heroku CI with a really lightweight setup.

Introduction to the Application for Testing

Since I was just trying this out, I forked the GitHub repo that was originally referenced in the Heroku blog post. The application was a simple React app with a link, a text input, and a submit button. There were three tests:

Verify that the link works and redirects to the right location.
Verify that the text input properly displays the user input.
Verify that submitting the form updates the text displayed on the page.

Pretty simple. Now, I just needed to change the code to use Playwright instead of Puppeteer and Jest. Oh, and I also wanted to use pnpm instead of npm. Here’s a link to my forked GitHub repo.

Modify the Code to Use Playwright

Let’s walk through the steps I took to modify the code. I started with my forked repo, identical to the heroku-examples repo.

Use pnpm

I wanted to use pnpm instead of npm. (Personal preference.) So, here’s what I did first:

~/project$ corepack enable pnpm


~/project$ corepack use pnpm@latest

Installing pnpm@9.12.3 in the project…
…
Progress: resolved 1444, reused 1441, downloaded 2, added 1444, done
…
Done in 14.4s

~/project$ rm package-lock.json


~/project$ pnpm install # just to show everything's good


Lockfile is up to date, resolution step is skipped
Already up to date
Done in 1.3s

Add Playwright to the project

Next, I removed Puppeteer and Jest, and I added Playwright.

~/project$ pnpm remove \
           babel-jest jest jest-puppeteer @testing-library/jest-dom

~/project$ $ pnpm create playwright
Getting started with writing end-to-end tests with Playwright:
Initializing project in '.'
✔ Do you want to use TypeScript or JavaScript? · JavaScript
✔ Where to put your end-to-end tests? · tests
✔ Add a GitHub Actions workflow? (y/N) · false
✔ Install Playwright browsers (can be done manually via 'pnpm exec playwright install')? (Y/n) · false
✔ Install Playwright operating system dependencies (requires sudo / root - can be done manually via 'sudo pnpm exec playwright install-deps')? (y/N) · false

Installing Playwright Test (pnpm add --save-dev @playwright/test)…
…
Installing Types (pnpm add --save-dev @types/node)…
…
Done in 2.7s
Writing playwright.config.js.
Writing tests/example.spec.js.
Writing tests-examples/demo-todo-app.spec.js.
Writing package.json.

I also removed the Jest configuration section from package.json.

Configure Playwright to use Chromium only

You can run your Playwright tests in Chrome, Firefox, and Safari. Since I was focused on Chrome, I removed the other browsers from the projects section of the generated playwright.config.js file:

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

//    {
//      name: 'firefox',
//      use: { ...devices['Desktop Firefox'] },
//    },
//
//    {
//      name: 'webkit',
//      use: { ...devices['Desktop Safari'] },
//    },


  ],
…

Exchange the Puppeteer test code for Playwright test code

The original code had a Puppeteer test file at src/tests/puppeteer.test.js. I moved that file to tests/playwright.spec.js. Then, I updated the test to use Playwright’s conventions, which mapped over quite cleanly. The new test file looked like this:

const ROOT_URL = 'http://localhost:8080';
const { test, expect } = require('@playwright/test');

const inputSelector = 'input[name="name"]';
const submitButtonSelector = 'button[type="submit"]';
const greetingSelector = 'h5#greeting';
const name = 'John Doe';

test.beforeEach(async ({ page }) => {
  await page.goto(ROOT_URL);
});

test.describe('Playwright link', () => {
  test('should navigate to Playwright documentation page', async ({ page }) => {
    await page.click('a[href="https://playwright.dev/"]');
    await expect(page.title()).resolves.toMatch('| Playwright');
  });
});

test.describe('Text input', () => {
  test('should display the entered text in the text input', async ({ page }) => {
    await page.fill(inputSelector, name);

    // Verify the input value
    const inputValue = await page.inputValue(inputSelector);
    expect(inputValue).toBe(name);
  });
});

test.describe('Form submission', () => {
  test('should display the "Hello, X" message after form submission', async ({ page }) => {
    const expectedGreeting = `Hello, ${name}.`;
    await page.fill(inputSelector, name);
    await page.click(submitButtonSelector);

    await page.waitForSelector(greetingSelector);
    const greetingText = await page.textContent(greetingSelector);
    expect(greetingText).toBe(expectedGreeting);
  });
});

Remove `start-server-and-test`, using Playwright’s webServer instead

To test my React app, I needed to spin it up (at http://localhost:8080) in a separate process first, and then I could run my tests. This would be the case whether I used Puppeteer or Playwright. With Puppeteer, the Heroku example used the start-server-and-test package. However, you can configure Playwright to spin up the app before running tests. This is pretty convenient!

I removed start-server-and-test from my project.

~/project$ pnpm remove start-server-and-test

In playwright.config.js, I uncommented the webServer section at the bottom, modifying it to look like this:

  /* Run your local dev server before starting the tests */
  webServer: {
     command: 'pnpm start',
     url: 'http://127.0.0.1:8080',
     reuseExistingServer: !process.env.CI,
  },

Then, I removed the test:ci script from the original package.json file. Instead, my test script looked like this:

  "scripts": {
    …
    "test": "playwright test --project=chromium --reporter list"
  },

Install Playwright browser on my local machine

Playwright installs the latest browser binaries to use for its tests. So, on my local machine, I needed Playwright to install its version of Chromium.

~/project$ pnpm playwright install chromium


Downloading Chromium 130.0.6723.31 (playwright build v1140)
from https://playwright.azureedge.net/builds/chromium/1140/chromium-linux.zip
164.5 MiB [====================] 100%

Note: The Chrome for Testing buildpack on Heroku installs the browser we’ll use for testing. We’ll set up our CI so that Playwright uses that browser instead of spending the time and resources installing its own.

Run tests locally

With that, I was all set. It was time to try out my tests locally.

~/project$ pnpm test

> playwright test --project=chromium --reporter list

Running 3 tests using 3 workers

  ✓  1 [chromium] > playwright.spec.js:21:3 > Text input > should display the entered text in the text input (911ms)
  ✘  2 [chromium] > playwright.spec.js:14:3 > Playwright link > should navigate to Playwright documentation page (5.2s)
  ✓  3 [chromium] > playwright.spec.js:31:3 > Form submission > should display the "Hello, X" message after form submission (959ms)

...
      - waiting for locator('a[href="https://playwright.dev/"]')


      13 | test.describe('Playwright link', () => {
      14 |   test('should navigate to Playwright documentation page', async ({ page }) => {
    > 15 |     await page.click('a[href="https://playwright.dev/"]');
         |                ^
      16 |     await expect(page.title()).resolves.toMatch('| Playwright');
      17 |   });
      18 | });

Oh! That’s right. I modified my test to expect the link in the app to take me to Playwright’s documentation instead of Puppeteer’s. I needed to update src/App.js at line 19:

<Link href="https://playwright.dev/" rel="noopener">
  Playwright Documentation
</Link>

Now, it was time to run the tests again…

~/project$ pnpm test

> playwright test --project=chromium --reporter list

Running 3 tests using 3 workers

  ✓  1 [chromium] > playwright.spec.js:21:3 > Text input > should display the entered text in the text input (1.1s)
  ✓  2 [chromium] > playwright.spec.js:14:3 > Playwright link > should navigate to Playwright documentation page (1.1s)
  ✓  3 [chromium] > playwright.spec.js:31:3 > Form submission > should display the "Hello, X" message after form submission (1.1s)

  3 passed (5.7s)

The tests passed! Next, it was time to get us onto Heroku CI.

Deploy to Heroku to Use CI Pipeline

I followed the instructions in the Heroku blog post to get my app set up in a Heroku CI pipeline.

Create a Heroku pipeline

In Heroku, I created a new pipeline and connected it to my forked GitHub repo.

Next, I added my app to staging.

Then, I went to the Tests tab and clicked Enable Heroku CI.

Finally, I modified the app.json file to remove the test script which was set to call npm test:ci. I had already removed the test:ci script from my package.json file. The test script in package.json was now the one to use, and Heroku CI would look for that one by default.

My app.json file, which made sure to use the Chrome for Testing buildpack, looked like this:

{
  "environments": {
    "test": {
      "buildpacks": [
        { "url": "heroku-community/chrome-for-testing" },
        { "url": "heroku/nodejs" }
      ]
    }
  }
}

Initial test run

I pushed my code to GitHub, and this triggered a test run in Heroku CI.

The test run failed, but I wasn’t worried. I knew there would be some Playwright configuration to do.

Digging around in the test log, I found this:

Error: browserType.launch: Executable doesn't exist at 
/app/.cache/ms-playwright/chromium-1140/chrome-linux/chrome

Playwright was looking for the Chrome browser instance. I could install it with the playwright install chromium command as part of my CI test setup. But that would defeat the whole purpose of having the Chrome for Testing buildpack. Chrome was already installed; I just needed to point to it properly.

Looking back in my test setup log for Heroku, I found these lines:

Installed Chrome dependencies for heroku-24
Adding executables to PATH
/app/.chrome-for-testing/chrome-linux64/chrome
/app/.chrome-for-testing/chromedriver-linux64/chromedriver
Installed Chrome for Testing STABLE version 130.0.6723.91

So, the browser I wanted to use was at /app/.chrome-for-testing/chrome-linux64/chrome. I would just need Playwright to look there for it.

Helping Playwright find the installed Chrome browser

Note: If you’re not interested in the nitty-gritty details here, you can skip this section and simply copy the full app.json lower down. This should give you what you need to get up and running with Playwright on Heroku CI.

In Playwright’s documentation, I found that you can set an environment variable that tells Playwright if you used a custom location for all of its browser installs. That env variable is PLAYWRIGHT_BROWSERS_PATH. I decided to start there.

In app.json, I set an env variable like this:

{
  "environments": {
    "test": {
      "env": { 
        "PLAYWRIGHT_BROWSERS_PATH": "/app/.chrome-for-testing"
      },
...

I pushed my code to GitHub to see what would happen with my tests in CI.

As expected, it failed again. However, the log error showed this:

Error: browserType.launch: Executable doesn't exist at
/app/.chrome-for-testing/chromium-1140/chrome-linux/chrome

That got me pretty close. I decided that I would do this:

Create the folders needed for where Playwright expects the Chrome browser to be. That would be a command like:

mkdir -p "$PLAYWRIGHT_BROWSERS_PATH/chromium-1140/chrome-linux"

Create a symlink in this folder to point to the Chrome binary installed by the Heroku buildpack. That would look something like this:

ln -s \
  $PLAYWRIGHT_BROWSERS_PATH/chrome-linux64/chrome \
  $PLAYWRIGHT_BROWSERS_PATH/chromium-1140/chrome-linux/chrome

However, I was concerned about whether this would be future-proof. Eventually, Playwright would use a new version of Chromium, and it won’t look in a chromium-1140 folder anymore. How could I figure out where Playwright would look?

That’s when I discovered you can do a browser installation dry run.

~/project$ pnpm playwright install chromium --dry-run


browser: chromium version 130.0.6723.31
  Install location:    /home/alvin/.cache/ms-playwright/chromium-1140
  Download url:        https://playwright.azureedge.net/builds/chromium/1140/chromium-linux.zip
  Download fallback 1: https://playwright-akamai.azureedge.net/builds/chromium/1140/chromium-linux.zip
  Download fallback 2: https://playwright-verizon.azureedge.net/builds/chromium/1140/chromium-linux.zip

That “Install location” line was crucial. And, if we set PLAYWRIGHT_BROWSERS_PATH, here is what we would see:

~/project$ PLAYWRIGHT_BROWSERS_PATH=/app/.chrome-for-testing \
           pnpm playwright install chromium --dry-run


browser: chromium version 130.0.6723.31
  Install location:    /app/.chrome-for-testing/chromium-1140
...

That’s what I want. With a little awk magic, I did this:

~/project$ CHROMIUM_PATH=$( \
             PLAYWRIGHT_BROWSERS_PATH=/app/.chrome-for-testing \
               pnpm playwright install --dry-run chromium \
               | awk '/Install location/ {print $3}'
           )

~/project$ echo $CHROMIUM_PATH


/app/.chrome-for-testing/chromium-1140

With all that figured out, I simply needed to add a test-setup script to app.json. Because PLAYWRIGHT_BROWSERS_PATH is already set in env, my script would be a little simpler. This was my final app.json file:

{
  "environments": {
    "test": {
      "env": { 
        "PLAYWRIGHT_BROWSERS_PATH": "/app/.chrome-for-testing"
      },
      "buildpacks": [
        { "url": "heroku-community/chrome-for-testing" },
        { "url": "heroku/nodejs" }
      ],
      "scripts": {
        "test-setup": "CHROMIUM_PATH=$(pnpm playwright install --dry-run chromium | awk '/Install location/ {print $3}'); mkdir -p \"$CHROMIUM_PATH/chrome-linux\"; ln -s $PLAYWRIGHT_BROWSERS_PATH/chrome-lin
ux64/chrome $CHROMIUM_PATH/chrome-linux/chrome"
      }
    }
  }
}

I’ll briefly walk through what test-setup does:

Accounting for PLAYWRIGHT_BROWSERS_PATH, uses playwright install -- dry-run with awk to determine the root folder where Playwright will look for the Chrome browser. Sets this as the value for the CHROMIUM_PATH variable.
Creates a new folder (and any necessary parent folders) to CHROMIUM_PATH/chrome-linux, which is the actual folder where Playwright will look for the chrome binary.
Creates a symlink in that folder, for chrome to point to the Heroku buildpack installation of Chrome (/app/.chrome-for-testing/chrome-linux64/chrome).

Run tests again

With my updated app.json file, Playwright should be able to use the Chrome installation from the buildpack. It was time to run the tests once again.

Success!

The test-setup script ran as expected.

Playwright was able to access the chrome binary and run the tests, which passed.

Conclusion

End-to-end testing for my web applications is becoming less cumbersome, so I’m prioritizing it more and more. In recent days, that has meant using Playwright more too. It’s flexible and fast. And now that I’ve done the work (for me and for you!) to get it up and running with the Chrome for Testing buildpack in Heroku CI, I can start building up my browser automation test suites once again.

The code for this walkthrough is available in my GitHub repository.

Happy coding!

Wow, pnpm, You’re Really Fast

Alvin Lee — Wed, 06 Nov 2024 16:25:10 +0000

If you’re a Node.js developer, then you’re familiar with npm and Yarn. You might even have a strong opinion about using one over the other. For years, developers have been struggling with the bloat — in disk storage and build time — when working with Node.js package managers, especially npm.

Then, along came pnpm, a package manager that handles package storage differently, saving users space and reducing build time. Here’s how pnpm describes the difference:

“When you install a package, we keep it in a global store on your machine, then we create a hard link from it instead of copying. For each version of a module, there is only ever one copy kept on disk. When using npm or yarn for example, if you have 100 packages using lodash, you will have 100 copies of lodash on disk. pnpm allows you to save gigabytes of disk space!”

It’s no surprise that pnpm is gaining traction, with more and more developers making it their package manager of choice. Along with that growing adoption rate, many developers who run their apps on Heroku (like I do) wanted to see pnpm supported.

Fortunately, pnpm is available via Corepack, which is distributed with Node.js. So, as of May 2024, pnpm is now available in Heroku!

In this post, we’ll cover what it takes to get started with pnpm on Heroku. And, we’ll also show off some of the storage and build-time benefits you get from using it.

A Quick Primer on pnpm

pnpm was created to solve the longstanding Node.js package manager issue of redundant storage and inefficiencies in dependency handling. npm and Yarn copy dependencies into each project’s node_modules. In contrast, pnpm keeps all the packages for all projects in a single global store, then creates hard links to these packages rather than copying them. What does this mean?

Let’s assume we have a Node.js project that uses lodash. Naturally, the project will have a node_modules folder, along with a subfolder called lodash, filled with files. To be exact, lodash@4.17.21 has 639 files and another subfolder called fp, with another 415 files.

That’s over a thousand files for lodash alone!

I created six Node.js projects: two with pnpm, two with npm, and two with Yarn. Each of them uses lodash. Let’s take a look at information for just one of the files in the lodash dependency folder.

~/six-projects$ ls -i npm-foo/node_modules/lodash/lodash.js
14754214 -rw-rw-r-- 544098 npm-foo/node_modules/lodash/lodash.js

~/six-projects$ ls -i npm-bar/node_modules/lodash/lodash.js
14757384 -rw-rw-r-- 544098 npm-bar/node_modules/lodash/lodash.js

~/six-projects$ ls -i yarn-foo/node_modules/lodash/lodash.js
14760047 -rw-r--r-- 544098 yarn-foo/node_modules/lodash/lodash.js

~/six-projects$ ls -i yarn-bar/node_modules/lodash/lodash.js
14762739 -rw-r--r-- 544098 yarn-bar/node_modules/lodash/lodash.js

~/six-projects$ ls -i pnpm-foo/node_modules/lodash/lodash.js
15922696 -rw-rw-r-- 544098 pnpm-foo/node_modules/lodash/lodash.js

~/six-projects$ ls -i pnpm-bar/node_modules/lodash/lodash.js
15922696 -rw-rw-r-- 544098 pnpm-bar/node_modules/lodash/lodash.js

The lodash.js file is a little over half a megabyte in size. We’re not seeing soft links, so at first glance, it really looks like each project has its own copy of this file. However, that’s not actually the case.

I used ls with the -i flag to display the inode of lodash.js file. You can see in the pnpm-foo and pnpm-bar projects, both files have the same inode (15922696). They’re pointing to the same file! That’s not the case for npm or Yarn.

So, if you have a dozen projects that use npm or Yarn, and those projects use lodash, then you’ll have a dozen different copies of lodash, along with copies from other dependencies in those projects which themselves use lodash. In pnpm, every project and dependency that requires this specific version of lodash points to the same, single, global copy.

The code for lodash@4.17.21 is just under 5 MB in size. Would you rather have 100 redundant copies of it on your machine, or just one global copy?

At the end of the day, dependency installation with pnpm is significantly faster, requiring less disk space and fewer resources. For developers working across multiple projects or managing dependencies on cloud platforms, pnpm offers a leaner, faster way to manage packages. This makes pnpm ideal for a streamlined deployment environment like Heroku.

Are you ready to start using it? Let’s walk through how.

Getting Started With pnpm

Here’s the version of Node.js we’re working with on our machine:

$ node --version
v20.18.0

Enable and Use pnpm

As we mentioned above, Corepack comes with Node.js, so we simply need to use corepack to enable and use pnpm. We create a folder for our project. Then, we run these commands:

~/project-pnpm$ corepack enable pnpm

~/project-pnpm$ corepack use pnpm@latest
Installing pnpm@9.12.2 in the project...
Already up to date
Done in 494ms

This generates a package.json file that looks like this:

{
"packageManager": "pnpm@9.12.2+sha512.22721b3a11f81661ae1ec68ce1a7b879425a1ca5b991c975b074ac220b187ce56c708fe5db69f4c962c989452eee76c82877f4ee80f474cebd61ee13461b6228"
}

This also generates a pnpm-lock.yaml file.

Next, we add dependencies to our project. For demonstration purposes, we’re copying the list of dependencies and devDependencies found in this benchmarking package.json file on GitHub. Now, our package.json file looks like this:

{
  "version": "0.0.1",
  "dependencies": {
    "animate.less": "^2.2.0",
    "autoprefixer": "^10.4.17",
    "babel-core": "^6.26.3",
    "babel-eslint": "^10.1.0",
    ...
    "webpack-split-by-path": "^2.0.0",
    "whatwg-fetch": "^3.6.20"
  },
  "devDependencies": {
    "nan-as": "^1.6.1"
  },
  "packageManager": "pnpm@9.12.2+sha512.22721b3a11f81661ae1ec68ce1a7b879425a1ca5b991c975b074ac220b187ce56c708fe5db69f4c962c989452eee76c82877f4ee80f474cebd61ee13461b6228"
}

Then, we install the packages.

~/project-pnpm$ pnpm install

Comparing Common Commands

The usage for pnpm is fairly similar to npm or yarn, and so it should be intuitive. Below is a table that compares the different usages for common commands (taken from this post).

Heroku Build Speed Comparison

Now that we’ve shown how to get a project up and running with pnpm (it’s pretty simple, right?), we wanted to compare the build times for different package managers when running on Heroku. We set up three projects with identical dependencies — using npm, Yarn, and pnpm.

First, we log in to the Heroku CLI (heroku login).

Then, we create an app for a project. We’ll show the steps for the npm project.

~/project-npm$ heroku apps:create --stack heroku-24 npm-timing

Creating ⬢ npm-timing... done, stack is heroku-24
https://npm-timing-5d4e30a1c656.herokuapp.com/ | https://git.heroku.com/npm-timing.git

We found a buildpack that adds timestamps to the build steps in the Heroku log, so that we can calculate the actual build times for our projects. We want to add that buildpack to our project, and have it run before the standard buildpack for Node.js. We do that with the following two commands:

~/project-npm$ heroku buildpacks:add \
                 --index=1 \
                 https://github.com/edmorley/heroku-buildpack-timestamps.git \
                 --app pnpm-timing

~/project-npm$ heroku buildpacks:add \
                 --index=2 heroku/nodejs \
                 --app npm-timing

Buildpack added. Next release on npm-timing will use:
1. https://github.com/edmorley/heroku-buildpack-timestamps.git
2. heroku/nodejs
Run git push heroku main to create a new release using these buildpacks.

That's it! Then, we push up the code for our npm-managed project.

~/project-npm$ git push heroku main

...
remote: Updated 4 paths from 5af8e67
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Building on the Heroku-24 stack
remote: -----> Using buildpacks:
remote: 1. https://github.com/edmorley/heroku-buildpack-timestamps.git
remote: 2. heroku/nodejs
remote: -----> Timestamp app detected
remote: -----> Node.js app detected
...
remote: 2024-10-22 22:31:29 -----> Installing dependencies
remote: 2024-10-22 22:31:29 Installing node modules
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 added 1435 packages, and audited 1436 packages in 11s
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 184 packages are looking for funding
remote: 2024-10-22 22:31:41 run `npm fund` for details
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 96 vulnerabilities (1 low, 38 moderate, 21 high, 36 critical)
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 To address issues that do not require attention, run:
remote: 2024-10-22 22:31:41 npm audit fix
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 To address all issues possible (including breaking changes), run:
remote: 2024-10-22 22:31:41 npm audit fix --force
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 Some issues need review, and may require choosing
remote: 2024-10-22 22:31:41 a different dependency.
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 Run `npm audit` for details.
remote: 2024-10-22 22:31:41 npm notice
remote: 2024-10-22 22:31:41 npm notice New minor version of npm available! 10.8.2 -> 10.9.0
remote: 2024-10-22 22:31:41 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.0
remote: 2024-10-22 22:31:41 npm notice To update run: npm install -g npm@10.9.0
remote: 2024-10-22 22:31:41 npm notice
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 -----> Build
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 -----> Caching build
remote: 2024-10-22 22:31:41 - npm cache
remote: 2024-10-22 22:31:41
remote: 2024-10-22 22:31:41 -----> Pruning devDependencies
remote: 2024-10-22 22:31:44
remote: 2024-10-22 22:31:44 up to date, audited 1435 packages in 4s
remote: 2024-10-22 22:31:44
remote: 2024-10-22 22:31:44 184 packages are looking for funding
remote: 2024-10-22 22:31:44 run `npm fund` for details
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 96 vulnerabilities (1 low, 38 moderate, 21 high, 36 critical)
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 To address issues that do not require attention, run:
remote: 2024-10-22 22:31:45 npm audit fix
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 To address all issues possible (including breaking changes), run:
remote: 2024-10-22 22:31:45 npm audit fix --force
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 Some issues need review, and may require choosing
remote: 2024-10-22 22:31:45 a different dependency.
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 Run `npm audit` for details.
remote: 2024-10-22 22:31:45 npm notice
remote: 2024-10-22 22:31:45 npm notice New minor version of npm available! 10.8.2 -> 10.9.0
remote: 2024-10-22 22:31:45 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.0
remote: 2024-10-22 22:31:45 npm notice To update run: npm install -g npm@10.9.0
remote: 2024-10-22 22:31:45 npm notice
remote: 2024-10-22 22:31:45
remote: 2024-10-22 22:31:45 -----> Build succeeded!
...

We looked at the timing for the following steps, up until the Build succeeded message near the end:

Installing dependencies
Build
Pruning devDependencies
Caching build

In total, with npm, this build took 16 seconds.

We ran the same setup for the pnpm-managed project, also using the timings buildpack.

~/project-pnpm$ heroku apps:create --stack heroku-24 pnpm-timing

~/project-pnpm$ heroku buildpacks:add \
                  --index=1 \
                  https://github.com/edmorley/heroku-buildpack-timestamps.git \
                  --app pnpm-timing

~/project-pnpm$ heroku buildpacks:add \
                  --index=2 heroku/nodejs \
                  --app pnpm-timing

~/project-pnpm$ git push heroku main

…
remote: 2024-10-22 22:38:34 -----> Installing dependencies
remote: 2024-10-22 22:38:34        Running 'pnpm install' with pnpm-lock.yaml
…
remote: 2024-10-22 22:38:49        
remote: 2024-10-22 22:38:49        dependencies:
remote: 2024-10-22 22:38:49        + animate.less 2.2.0
remote: 2024-10-22 22:38:49        + autoprefixer 10.4.20
remote: 2024-10-22 22:38:49        + babel-core 6.26.3
…
remote: 2024-10-22 22:38:51 -----> Build succeeded!

For the same build with pnpm, it took only 7 seconds.

The time savings, we found, isn’t just for that initial installation. Subsequent builds, which use the dependency cache, are also faster with pnpm.

Conclusion

When I first started Node.js development, I used npm. Several years ago, I switched to Yarn, and that’s what I had been using. . . until recently. Now, I’ve made the switch to pnpm. On my local machine, I’m able to free up substantial disk space. Builds are faster too. And now, with Heroku support for pnpm, this closes the loop so that I can use it exclusively from local development all the way to deployment in the cloud.

Happy coding!

Ending Microservices Chaos: How Architecture Governance Keeps Your Microservices on Track

Alvin Lee — Tue, 22 Oct 2024 16:06:13 +0000

A microservices architecture is the gold standard for building scalable web applications. Gartner estimates that 74% of organizations use microservices for their web applications, with another 23% planning to use them soon.

If you’re an IT leader, architect, or developer, you might have experienced the faster deployments, better fault isolation, and easier scaling that come with microservices.

However, the old maxim, “there’s no such thing as a free lunch,” rings true here. Because microservices can also lead to significant operational overhead and complexity. Microservices sprawl, unclear component dependencies, service duplication, and anti-patterns like circular dependencies are all part of the chaos that comes hand-in-hand with microservices.

Perhaps you’ve seen some of the common symptoms of this microservices chaos:

Engineering velocity drastically slows down
Onboarding new developers grows more difficult
Mean time to recovery (MTTR) for outages or performance issues increases
Overall system resiliency diminishes, putting business outcomes at risk

While it’s still best practice to modernize monolithic applications by breaking them into microservices, it’s also best practice to proactively guard against the issues that microservices introduce.

Architecture Governance to Control the Chaos

The best way to proactively guard against these problems is by using architecture governance—a collection of rules, processes, and practices that manage and control your software architecture.

With proper software architecture governance, you can reduce microservices complexity, ramp up developers faster, reduce MTTR, and improve the resiliency of your system, all while building a culture of intentionality.

(Note that we’re talking about software architecture governance in this article, not enterprise architecture governance which has been around for some time and is concerned with the policies, processes, and procedures of enterprise architecture strategies.)

But exactly how do you build architecture governance?

Governance, whether we’re talking about access, data, or architecture, is often just establishing a set of rules. But for architecture governance, you need more than just rules. You must start with architectural observability.

Architectural Observability: The Foundation of Good Governance

To implement effective governance, you first need comprehensive visibility into your software architecture.

Architectural observability gives you a continuous, real-time view of your architecture. It provides your team with the necessary insights to manage the architecture and interconnectedness of distributed applications.

Architectural observability provides documentation from a working system, it shows you systems flows, dependencies, what’s changing, and how your systems interact. It gives your team a thorough understanding of how the architecture works, how it drifts from release to release, and how its changes impact dependent services and resources.

Without this visibility, you’re in trouble. If you’ve built a microservices project, you’ve probably encountered the results of a lack of visibility:

microservices that are difficult to manage
unclear component dependencies
service duplication
anti-patterns such as circular dependencies that increase the risk of outages
performance issues.

Your overall application resiliency, scalability, and engineering velocity are at high risk. And it only gets worse—the longer you let this lack of visibility continue, the more complex and unmanageable your architecture becomes.

Good Observability Equals Good Governance

But with good architectural observability, you can visualize your architecture layer in real time, gaining an understanding of how your system works and monitoring changes as your architecture evolves.

With observability, your team can proactively address issues, such as service sprawl and dependency mismanagement—before they lead to technical debt or system failures.

And once you have that observability in place—once you know what’s happening and what’s changing—you can now create and enforce your rules of governance.

You can implement a governance program that sits over your architecture, monitors and controls changes, and allows you to build and manage microservices effectively by proactively applying standards and rules to your architecture and codebase as it evolves.

Sounds great! But how do you actually build this? Let’s look at an example of how to build this in the real world.

Implementing Architectural Observability and Governance

First, you need to implement your architectural observability.

The domain of architectural observability is new and rapidly evolving. For our example, we’ll use one of the platforms at the forefront—vFunction. vFunction is an AI-driven platform that provides visualizations of software architecture.

vFunction leverages AI and OpenTelemetry tracing to enable teams to track every aspect of their architecture and monitor changes between releases. It can quickly identify complex flows, architectural technical debt, and architectural drift.

For example, vFunction allows you to better understand your distributed applications:

Visualize the architecture and automatically create exportable sequence diagrams of all system and data flows.
Identify drift, such as new services or dependencies.
Identify resource exclusivity changes (such as multiple new services accessing the same database table) versus previous releases.
Identify circular dependencies between services that could impact resiliency.
Identify duplicate functionality/services that may be merged to remove complexity.
Maintain an up-to-date, real-time view of the global microservices ecosystem.

vFunction uses the same data that APM tools use (so it’s straightforward to install) but uses a different layer of intelligence to address the architectural root cause of issues. This is especially important for managing the sprawl and complexity that result from the rapid and frequent deployment of microservices.

Example of exportable sequence flow diagram in vFunction

Implementing Rules to Maintain Standards

After implementing your observability, you’re ready to create and enforce your rules of governance. vFunction allows you to create architecture rules and tags and then monitor your architecture in real time against those rules, enforcing your critical governance policies. Any time a deployment or merge request breaks these rules, you can block that action.

What type of rules might your governance enforce? You could enforce rules such as:

Dependency requirements. For example, certain services should or should not communicate with other services.
Restricted resources.
Multiple services should not depend on the same database table.
Changes should not increase service interdependencies.
Restrictions on new multi-hop flows that could impact performance and resiliency.

vFunction can watch and send alerts for all of the above.

By implementing a focused set of rules and tags, you can create adaptive governance that guides developers in maintaining an optimal, clean, and efficient architecture. You can build architecture governance into your code and your culture.

Conclusion

Microservices chaos can threaten to derail development velocity, system resiliency, and operational efficiency. Architecture governance offers a way out of this chaos—a way to regain control over your architecture and steer it toward greater scalability and stability.

Whether your organization is just starting to scale with microservices or is already in the depths of microservices sprawl, it’s the right time to introduce architecture governance to ensure that your architecture is efficient and resilient—and ready to support your next phase of growth.

Forem: Alvin Lee

The Messaging Challenges No One Talks About in Regulated, Air-Gapped, and Hybrid Environments

The Modernization Dilemma

1. The Constraints of Legacy Middleware

2. The Complexity of Self-Managed Open Source

Kubernetes-Native Messaging for Trust and Control

1. One Platform, All Messaging Patterns

2. Operational Simplicity (Easy to Use and Manage)

3. True Air-Gapped Capability and Zero Egress

4. Security & Audit: Deep Policy Enforcement

5. Architecting for Hybrid and Edge Resilience

Real-Life Use Case: Unifying Critical Electricity Infrastructure

Modernize Without Compromise

Why Your UEBA Isn’t Working (and how to fix it)

The End of Traditional UEBA–or, Why Your UEBA No Longer Works

What Modern Enterprises Actually Need from UEBA

Immediately detect attacks (without a long training period)

Detect highly sophisticated attacks

Integration with existing security operations

UEBA as the Foundation for AI Security Agents

AI agents need quality behavioral baselines

AI detects, then AI responds

From UEBA rules to autonomous security operations

Building AI-Ready Behavioral Foundations Now

The Bottom Line

Log Spikes? No Sweat: How Top DevOps Teams Tame Bursty Workloads

What Makes a “Bursty” Vertical So Challenging

Troubles with scaling: Ticketmaster’s meltdown

Losing trust: Robinhood goes dark

Modern Observability Platforms That Are “Built for Bursty”

Schema-less ingestion

Ingest everything

AI for triage

How Top Teams Stay Calm During the Storm

Stress-test your log pipelines

Use verbose logs

Non-verbose logs (high value):

Verbose logs (low value):

Take advantage of runbooks

Conclusion

The Human Side of Logs: What Unstructured Data Is Trying to Tell You

Why Unstructured Data Matters

What Signals Should You Watch For?

How Observability Platforms Can Wrangle Unstructured Data

Example Use Cases

Conclusion

From Concept to Cloud: Building with Cursor and the Heroku MCP Server

What is the Model Context Protocol (MCP)?

Using Cursor to build my SvelteKit app

Open Cursor to a new project folder

Describe the task

Let it code!

Set up local database

Initialize a git repository

Test application locally

Deploying to Heroku with help from Cursor

Set up Cursor to use Heroku’s MCP Server

Step 1: Obtain a Heroku authorization token

Step 2: Create a mcp.json file

Step 3: Enable the new MCP server

Use Cursor to build and deploy app

Other things that Cursor can do with the Heroku MCP Server

Final thoughts: A new way to build

Deploying a Scala Play Application to Heroku

Introducing the Example App

Taking a quick look at the code

Case class to model resources

Routing is clean and centralized

Controllers are concise

JSON is handled with implicit values

Modern, expressive syntax

Running the App Locally

Preparing for Deployment to Heroku

Understanding the AllowedHostsFilter

Setting a Play secret

sbt run versus running the compiled binary

Why I went with Heroku

Deploying the app to Heroku

Create the Procfile

Specify the JDK version

Step 2: Create a `mcp.json` file

Understanding the `AllowedHostsFilter`

Remove `start-server-and-test`, using Playwright’s webServer instead