Forem: Pulumi

CDKTF is deprecated: What's next for your team?

Christian Nunciato — Thu, 18 Dec 2025 20:36:56 +0000

In July, 2020, CDK for Terraform (CDKTF) was introduced, and last week, on December 10, it was officially deprecated. Support for CDKTF has stopped, the organization and repository have been archived, and HashiCorp/IBM will no longer be updating or maintaining it, leaving a lot of teams out there without a clear path forward.

For most teams, that means it's time to start looking for a replacement.

It's an unfortunate situation to suddenly find yourself in as a user of CDKTF, but you do have options, and Pulumi is one of them. In this post, we'll help you understand what those options are, how Pulumi fits into them, and what it'd look like to migrate your CDKTF projects to Pulumi.

What are the options?

Teams migrating away from CDKTF generally have three options:

Option 1: Fall back to HCL

HashiCorp's official recommendation is to export your projects to HashiCorp Configuration Language (HCL) and manage them with Terraform. CDKTF even has a command that makes this fairly simple:

cdktf synth --hcl

Of course, if you're using CDKTF, you probably chose it specifically to avoid HCL. So while possible, this probably isn't the choice most teams would make unless they had to.

Option 2: Migrate to AWS CDK

If your team is all-in on AWS, another option would be to migrate to AWS CDK. It's widely used, officially supported, the programming model is similar to CDKTF's, and both CDK and CDKTF transpile to an intermediate format (CloudFormation YAML and Terraform JSON, respectively) that gets passed on to their underlying tools for deployment.

But while their programming and deployment models are conceptually similar, their resource models and APIs are entirely different. Here's the code for an S3 bucket written in AWS CDK, for example:

import * as s3 from 'aws-cdk-lib/aws-s3';

const bucket = new s3.Bucket(this, 'my-bucket', {
    bucketName: 'my-example-bucket',
    versioned: true,
    publicReadAccess: false,
});

And here's the code for a similarly configured bucket in CDKTF:

import { S3Bucket } from '@cdktf/provider-aws/lib/s3-bucket';

const bucket = new S3Bucket(this, 'my-bucket', {
    bucket: 'my-example-bucket',
    versioning: {
        enabled: true,
    },
    acl: 'private',
});

Notice how different these APIs are — and this is just one simple resource with only a few properties; imagine having to rewrite dozens or hundreds of them. Beyond that, there's also the problem of state: How would you go about translating the contents of a Terraform state file containing hundreds of resources into the equivalent CloudFormation YAML or JSON?

Despite their surface similarities, CDKTF and AWS CDK have little in common. Migration would essentially mean a ground-up rewrite that'd also leave you without the multi-cloud support you already have with CDKTF. For most teams, that makes this option a practical non-starter.

Option 3: Migrate to Pulumi

This is where we should acknowledge our obvious bias — but we genuinely believe that for most users of CDKTF, Pulumi really is the simplest and most broadly compatible way forward.

Like CDKTF, Pulumi lets you build and manage your infrastructure with general-purpose languages like TypeScript, Python, Go, C#, and Java, and it supports organizing your code into higher-level abstractions called components, which you can think of like CDKTF constructs. Both organize cloud resources into stacks (think dev, prod), and both track deployment state similarly, with local, remote, and cloud-hosted options available.

Many of Pulumi's most popular providers (e.g., the AWS provider) are also built from open-source Terraform schemas, which means their resource models will be nearly identical to what you're used to with CDKTF. Here's what an S3 bucket looks like in Pulumi, for example:

import * as aws from '@pulumi/aws';

const bucket = new aws.s3.Bucket('my-bucket', {
    bucket: 'my-example-bucket',
    versioning: {
        enabled: true,
    },
    acl: 'private',
});

You can also use any Terraform provider with Pulumi, and you can even reference Terraform modules directly from within your Pulumi code.

Pulumi is also different from CDKTF in several ways. One is that rather than transpile your source code to a format like JSON as CDKTF does (and then deploying it separately later), Pulumi uses its own declarative deployment engine that resolves the resource graph at runtime and provisions cloud resources directly, which is much faster and more flexible. You can learn more about the deployment model in How Pulumi Works.

Given the API similarities, the support for all Terraform providers and modules, the ability to coexist alongside Terraform-managed projects, and the built-in support for conversion (which we'll cover next), we think Pulumi is the best option for most teams looking to migrate.

What migrating to Pulumi looks like

Migrating a CDKTF project to Pulumi generally happens in three steps:

Conversion, which translates your CDKTF code into a new Pulumi program
Import, which reads the contents of your CDKTF state into a new Pulumi stack
Refactoring, which brings the code in the new program into alignment with the stack's currently deployed resources

Conversion and import

Migration starts with exporting your CDKTF project to HCL with cdktf synth. From there, Pulumi's built-in convert and import commands handle creating the new program and importing your state:

# Export your project to HCL.
cdktf synth --hcl

# Convert the HCL into a new Pulumi project.
pulumi convert --from terraform --language typescript

# Create a new Pulumi stack.
pulumi stack init dev

# Import your CDKTF stack's resources into your new Pulumi stack.
pulumi import --from terraform ./terraform.dev.tfstate

The converter automatically translates Terraform input variables, data sources, resources, and outputs into their Pulumi equivalents. You can read more about how this works in Converting Terraform HCL to Pulumi.

Refactoring

Once you've imported your state, you'll often have to make some adjustments to the code to bring it in line with the new Pulumi stack. For instance, pulumi import marks new resources protected by default, to prevent them from being accidentally deleted — but since the code produced by pulumi convert doesn't include the protect resource option, you'll need to add it yourself. Fortunately the import step also emits code that you can copy into your program to make this process a little easier.

Refactoring can get a bit more complicated when custom logic and higher-level abstractions are involved, as fidelity to the original CDKTF code is often lost in the translation to HCL. In these situations, having the help of an LLM to recapture that original logic or translate your CDKTF constructs into Pulumi components can be a big time-saver.

An end-to-end example

The best way to get a feel for how this works, though, is to try it yourself.

The pulumi/cdktf-to-pulumi-example repository on GitHub contains a CDKTF project with multiple stacks written in TypeScript, along with a guide that walks you through the process of migrating that project to Pulumi. The guide covers everything we've discussed here so far, including:

Converting the CDKTF project into a new Pulumi project
Importing its actively running resources into Pulumi stacks
Modifying the generated code to align with imported state
Performing an initial deployment with Pulumi to complete the migration process

The walkthrough takes only a few minutes to complete, and it's a great way to stand up an example of your own to get more familiar with Pulumi.

What's next?

If you’re moving on from CDKTF, there are a few possible paths forward. For teams that want to keep using real languages and avoid a ground-up rewrite, Pulumi offers the clearest way forward.

To learn more about how Pulumi works, how it differs from CDKTF and from Terraform, how to handle additional conversion scenarios, and more, we recommend:

Diving into the Pulumi docs to get familiar with core concepts and features of the platform
Reading Migrating from Terraform or CDKTF to Pulumi for more detailed, Terraform-specific migration guidance
Joining us in the Pulumi Community Slack to ask questions and learn from others who've successfully made the leap from Terraform and CDKTF to Pulumi

And of course, feel free to reach out! We'd love to help in any way we can.

Superintelligence Infrastructure: Managing AI Workloads with General-Purpose Programming Languages

Pulumi Team — Wed, 17 Dec 2025 17:42:13 +0000

AI Infrastructure Is Outgrowing Static Configuration

AI systems no longer resemble traditional application stacks. Training large models, running distributed inference, and scaling GPU-backed services introduce infrastructure patterns that change constantly. Capacity shifts. Regions rebalance. New services appear and disappear based on demand.

Static configuration tools struggle in this environment. They assume infrastructure is declared once and applied repeatedly. AI workloads require infrastructure that behaves like software: adaptive, testable, and designed to evolve.

Superintelligence Infrastructure is built for that reality.

Infrastructure Defined with General-Purpose Programming Languages

Pulumi allows teams to define cloud infrastructure using general-purpose programming languages such as Python, TypeScript, and Go. For AI platforms, this unlocks capabilities that are difficult or impractical to express in declarative templates:

Conditional resource creation based on model type or environment
Loops for provisioning large, dynamic GPU fleets
Shared abstractions for training, tuning, and inference pipelines
Unit tests and previews before deploying infrastructure changes

Infrastructure becomes part of the application lifecycle instead of a separate, static artifact.

Designed for Large-Scale AI Environments

Superintelligence Infrastructure supports AI workloads operating at massive scale, including environments with tens of thousands of resources across regions and cloud providers.

Common use cases include:

Distributed training clusters with elastic GPU capacity
Multi-region inference services with low-latency routing
Automated teardown and rebuild of experimental environments
Policy-enforced deployments for security, cost, and compliance

These systems are defined, reviewed, and deployed using the same engineering workflows teams already use for application development.

AI-Native Operations with Pulumi

Pulumi integrates AI-assisted workflows directly into infrastructure management. Platform teams can use AI to explore infrastructure state, detect drift, generate updates, and apply changes safely under policy control.

This approach reduces manual intervention while keeping humans in the loop through previews, approvals, and audit trails.

Superintelligence Infrastructure combines automation with governance instead of trading one for the other.

A Practical Path to Production AI Infrastructure

For teams building AI platforms, the challenge is not experimentation. The challenge is turning prototypes into durable, repeatable production systems.

Pulumi provides a foundation that supports:
Rapid iteration during early model development
Controlled promotion into production environments
Ongoing evolution as models, data, and usage change

Learn how Superintelligence Infrastructure is used to manage AI systems.

What AWS’s 2025 AI and Cloud Updates Mean for Engineers Building Large Systems

Pulumi Team — Thu, 11 Dec 2025 15:58:12 +0000

AWS announced major changes this year that affect how teams design, train, and operate large-scale systems. If you work in AI infrastructure, MLOps, or cloud engineering, several releases stand out for their long-term impact.

Pulumi put together a full breakdown, but here is the technical signal behind the announcements.

Nova Forge and the new model training workflow

AWS is shifting model training toward a platform-based workflow. Nova Forge abstracts much of the orchestration, data movement, and scaling logic that teams previously had to build themselves. This is important because training pipelines are becoming too large and too complex for ad-hoc solutions.

For teams building specialized LLMs or domain-specific models, this lowers both time and operational overhead.

Trainium 3 and the next stage of high-throughput compute

Trainium 3 extends AWS’s hardware roadmap with higher throughput and deeper integration into the broader AI stack. The takeaway is that large-scale training is becoming more predictable and better supported by AWS-native services.

This matters for iteration speed and total cost at scale.

AgentCore’s move toward structured automation

AgentCore is evolving from helper APIs into a system that understands user intent and applies guardrails across environments. This is a step toward infrastructure that can participate in operations rather than simply expose endpoints.

For teams wrestling with multi-environment complexity, this points toward more automated operational patterns.

A shift in what cloud engineering workflows look like

The common pattern across these launches is integration. Compute, orchestration, training workflows, and automation are becoming parts of a single continuum rather than separate tools stitched together.

Pulumi’s analysis places these changes in the bigger shift toward adaptive infrastructure. Tools like Neo align with this direction by introducing AI-supported workflows into everyday development.

Read the full analysis

For a deeper look at how AWS is shaping AI and cloud infrastructure going into the next decade, read "AWS built an integrated AI Agent training pipeline and they want you to rent it".

Zero to Production with AI Agentic Workflows – Free Hands-On Course

Pulumi Team — Fri, 03 Oct 2025 22:08:38 +0000

Manual infrastructure tasks slow teams down. AI agents and agentic workflows offer a new way forward—reducing repetitive work, improving governance, and unlocking more time for strategy and innovation.

We’re launching a free, hands-on virtual course:
Zero to Production with AI Agentic Workflows
Starts October 14

Over the course of five interactive workshops, you’ll learn how to:

Apply AI agents to accelerate platform engineering
Streamline infrastructure provisioning with automation
Balance developer autonomy with governance at scale
Create reusable patterns and templates to boost productivity
Handle surges in infrastructure demand with intelligent automation

Each session includes live instruction, hands-on labs, and Q&A.
Recordings will be available only to registered participants. Participants who attend all the workshops will earn a completion badge to showcase their skills. Register for the course.

Meet Neo: Your Newest Platform Engineer

Pulumi Team — Thu, 11 Sep 2025 17:27:32 +0000

Infrastructure teams are drowning in demands. While your organization races to adopt AI, platform teams are stretched thin managing the infrastructure demands. What if there was a better way?

Meet Neo, your newest platform engineer teammate.

See what's possible when intelligence meets infrastructure.

Join us September 16th at 10:00 AM PT (UTC-7) for the exclusive introduction to Neo. Save the date now, don't miss out!

Backstage vs Pulumi IDP: Which One Is the Platform Engineering Winner?

Engin Diri — Thu, 14 Aug 2025 12:20:30 +0000

Introduction: Why the IDP Approach Matters

Internal developer platforms (IDPs) are becoming essential for scaling modern software delivery. They promise faster deployments, stronger governance, and consistent environments. Yet, many fail before they deliver value—often because they start with the portal and leave infrastructure for later.

This article compares Backstage (portal-first) and Pulumi IDP (infrastructure-first) so you can choose an approach that avoids common platform engineering pitfalls. Our in-depth analysis reveals why this architectural choice matters more than you might think.

Key Differences at a Glance

Backstage (Portal-First):

Aspect	Details
Setup Time	12-18 months
Maintenance	3-5 FTEs ongoing
Focus	Service catalogs, documentation, templates
Infrastructure	Requires separate tooling (Terraform, Crossplane, etc.)
Best For	Teams with existing infrastructure automation

Pulumi IDP (Infrastructure-First):

Aspect	Details
Setup Time	Hours to days (Get started here)
Maintenance	Minimal ongoing overhead
Focus	Governance, security, cost control built-in
Infrastructure	Native infrastructure-as-code included
Best For	Teams needing fast, governed infrastructure delivery

Why Infrastructure-First Matters for IDPs

With Gartner predicting that 80% of enterprises will deploy IDPs by 2026, the foundation you choose will impact cost, security, and delivery speed. Infrastructure-first platforms like Pulumi IDP embed:

Policy enforcement from day one (no retroactive compliance scrambles)
Cost awareness for efficient scaling (prevent cloud bill surprises)
Security and compliance without bolt-on tools
Cloud-native readiness for Kubernetes and multi-cloud environments

The Hidden Costs of Portal-First Platforms 💸

Backstage excels at developer portals, but its portal-first model can lead to:

Long implementation timelines before developers see value
High maintenance costs and staffing requirements
Infrastructure governance added late, creating compliance debt
Scaling challenges in Kubernetes-native environments
Tool sprawl: Often requires 6+ additional tools for complete functionality

Real Developer Experience:

Portal-First Journey:

Month 1-6: Set up Backstage portal
Month 6-12: Integrate with existing tools
Month 12+: Realize you need infrastructure automation
Month 18+: Finally delivering value (maybe)

Infrastructure-First Journey:

Day 1: Deploy platform with built-in governance
Week 1: Developers provisioning compliant infrastructure using reusable components
Month 1: Add portal features as needed with developer portals
Continuous value delivery from day one

Community Insights 🗣️

Based on discussions in platform engineering communities:

"We spent 18 months on Backstage only to realize we still needed to build all the infrastructure automation separately." - Platform Engineer at Fortune 500

"Pulumi IDP got us from zero to production-ready platform in 2 weeks. The built-in governance saved us from compliance nightmares." - DevOps Lead at Scale-up

Learn more about real-world platform engineering experiences and how teams made their transitions.

Choosing the Right IDP for Your Organization

Select Backstage if you:

Already have robust infrastructure provisioning and governance tools
Primarily need a service catalog and documentation portal
Have budget and staffing for long-term maintenance
Want maximum customization flexibility
Prefer open-source, community-driven solutions

Select Pulumi IDP if you:

Need infrastructure governance from day one with CrossGuard policies
Want to minimize setup time and operational overhead using the Automation API
Require scalable, secure, multi-cloud infrastructure without starting from scratch
Need built-in cost controls and policy enforcement
Want to deliver value to developers immediately with organization templates

The Architecture That Scales

The fundamental difference isn't about features—it's about architecture:

Portal-First: Portal → Templates → (Eventually) Infrastructure → (Maybe) Governance

Infrastructure-First: Infrastructure + Governance → APIs → Portal → Enhanced DevEx

Dive deep into the architectural implications and see how real teams made their choice. You can also explore Pulumi's approach to platform engineering and how it aligns with the CNCF Platform Maturity Model.

Real-World Implementation Examples

Want to see how this works in practice? Check out these resources:

For AWS users: Building an IDP on AWS with Pulumi
For Kubernetes teams: Deploy production-ready clusters with Pulumi's EKS components
Policy examples: Implement compliance-ready policies for your organization
Workshop series: Join the Platform Engineering workshop series to get hands-on experience

Conclusion: It's About Your Foundation

The choice between Backstage and Pulumi IDP is more than a tool decision—it's an architectural decision that will impact your platform engineering success for years. Portal-first platforms can be powerful but risky without a strong infrastructure layer. Infrastructure-first platforms like Pulumi IDP ensure governance, security, and scalability from the start.

Your platform's foundation determines its future. Choose wisely.

💬 Discussion Time!

What's your experience with IDPs? Have you tried portal-first or infrastructure-first approaches? Share your platform engineering journey in the comments!

Want the complete analysis? Read our comprehensive comparison with real-world examples, cost breakdowns, and migration strategies.

Ready to get started? Explore the Pulumi IDP documentation or try the free Platform Engineering guide.

Follow me for more platform engineering insights!

🚀 Pulumi AWS Provider 7.0: Multi-Region AWS IaC & IAM Role Chaining

Pulumi Team — Tue, 12 Aug 2025 15:41:29 +0000

Pulumi AWS Provider 7.0 is here, bringing major enhancements to AWS infrastructure as code workflows.

What’s New Highlights

Multi-region support from a single provider: Define resources in different AWS regions without duplicating provider configurations.
IAM role chaining: Securely assume multiple roles for complex cross-account deployments.
S3 resource simplification: Unified bucket resources for easier migration and management.

Why It Matters

These updates make it easier to manage scalable, secure, multi-region AWS deployments in general programming languages such as TypeScript, Python, Go, C#, or Java.

📖 Full release notes, migration guide, and code examples at Pulumi AWS Provider 7.0.

Kubernetes Best Practices I Wish I Had Known Before

Pulumi Team — Thu, 03 Apr 2025 18:23:36 +0000

Getting started with Kubernetes can feel like climbing Mount Everest in flip-flops. On the surface, it seems straightforward—deploy your containerized app and go. But then the iceberg hits: YAML sprawl, mysterious Pod evictions, and networking tangles that make you question everything.

After years wrangling Kubernetes in production, I’ve pulled together a shortlist of lessons I wish someone had handed me on Day 1. These aren't just tips—they're sanity savers.

🚨 Don't Skimp on Resource Requests & Limits
Skipping these? Say hello to throttling and unexpected Pod evictions. Use tools like Prometheus to fine-tune based on real usage.

📛 Namespace Like Your Life Depends on It
Namespaces are more than folders—they’re your key to organizing, securing, and scaling your workloads. Pair them with RBAC for a tighter ship.

🎯 Use Probes for Container Health
Liveness, readiness, and startup probes let Kubernetes know if your app is working. Without them, your cluster is flying blind.

🔐 Security: Non-Negotiable
Set RBAC from Day 1. Use Pod Security Standards. Store secrets securely (hint: not in environment variables—use tools like Pulumi ESC or external secrets stores).

📦 Don’t Handcraft YAML Forever
Use Helm, Kustomize, or tools like Pulumi to manage infra with code. Your future self (and teammates) will thank you.

📊 Monitor Everything
From metrics (Prometheus) to logs (Loki, EFK) and traces (Jaeger), observability is your safety net. Set alerts before things break.

⚙️ Automate Deployments
Embrace CI/CD and GitOps to avoid the “it works on my machine” dilemma. Tools like Argo CD or GitHub Actions are your friends.

This is just the tip of the iceberg. Want to avoid common Kubernetes pitfalls and keep your clusters running smoothly

👉 Read the full article: Kubernetes Best Practices I Wish I Had Known Before

The Cloud Container Iceberg - At least 105 ways to run a container in the cloud

Pulumi Team — Fri, 28 Feb 2025 16:45:57 +0000

If it connects to the internet, someone has tried running a container on it. From AWS racks packed with phones to quantum computers, the world of container deployment extends far beyond the usual cloud providers. You might think you know where containers run—but that’s just the tip of the iceberg.

At the surface, there are the well-known cloud platforms: AWS, Azure, and Google Cloud, with services like Kubernetes, Fargate, and Cloud Run. But below the waterline? That’s where things get interesting.

Beyond the Surface: The Hidden Depths of Container Hosting

Just below the mainstream, you’ll find alternative cloud platforms and regional providers—some offering K8s with local flair, others designed for hyper-specific workloads. Venture deeper, and you’ll discover serverless platforms that use containers in unexpected ways, experimental multi-cloud orchestration, and hybrid deployments that let you run containers across data centers, IoT devices, and even mail-delivered hardware.

And if you're willing to go all the way down? Containers at the edge—running on factory equipment, Raspberry Pis, and, yes, even smart toasters.

How Deep Does the Iceberg Go?

The Usual Suspects – The mainstream cloud container services you already know.
The Lesser-Known Options – Specialized K8s providers, regional cloud giants, and alternative deployment platforms.
The Experimental Zone – Serverless containerized workloads, hybrid-cloud solutions, and multi-cloud orchestration nightmares.
The Abyss – Containers in places they should never be: quantum computers, edge devices, and mobile test farms.

The world of container hosting is far bigger than it seems. What starts with Kubernetes in the cloud quickly spirals into a rabbit hole of specialized services, hybrid solutions, and experimental edge cases.

Want to see the full breakdown?

🔗 Dive into the complete article here: The Cloud Container Iceberg

Infrastructure as Code: Comparing Pulumi and HCL Approaches

Engin Diri — Mon, 05 Aug 2024 13:37:26 +0000

The Java Language Architect at Oracle, Brian Goetz, author of Java Concurrency in Practice, has commented how declarative languages can be a double-edged sword:

In the realm of Infrastructure as Code (IaC), two prominent solutions offer different approaches to resource declaration. Terraform, developed by HashiCorp, employs a proprietary domain-specific language (DSL) for cloud resource specification. In contrast, Pulumi embraces versatility, allowing developers to utilize popular programming languages such as C#, Java, JavaScript, Go, Python, or TypeScript, as well as the widely-adopted YAML markup language.
While both Terraform and Pulumi operate on declarative IaC principles, their divergent approaches to expression languages lead to notable differences in implementation and user experience. This distinction in language choice has far-reaching implications for developers and organizations alike.

Real-World Consequences

We’ve worked with 1,000s of companies and 100,000+ developers and infrastructure experts alike to adopt infrastructure as code using Pulumi and modern languages, and we see several significant recurring themes as they migrate away from Terraform and HCL.

Tapping Into Millions of Developers

Pulumi allows developers to use widely-adopted general-purpose languages for infrastructure as code. This approach taps into vast existing developer communities, with JavaScript boasting over 17 million developers, Python 15 million, and Go 3 million worldwide. By utilizing these languages, Pulumi enables teams to leverage their current programming expertise and access extensive resources, libraries, and tools. Meanwhile, HCL maintains a dedicated user base of several hundred thousand developers who appreciate its specialized syntax for infrastructure description.

Pulumi Leverages Rich Ecosystems of General-Purpose Languages

Pulumi harnesses the extensive ecosystems surrounding popular programming languages for infrastructure as code. While domain-specific languages like HCL have their strengths, Pulumi's approach offers developers access to:

Comprehensive IDE support, including features like autocompletion, syntax highlighting, and real-time error detection, which may be more advanced than those available for specialized IaC languages
Wide array of established linting tools to enforce coding standards and catch potential issues early in the development process
Robust testing frameworks that allow for thorough verification and validation of infrastructure configurations, often more mature than those specific to IaC languages
Extensive libraries and packages that can be seamlessly integrated into infrastructure code, expanding possibilities beyond traditional IaC toolsets

This approach allows teams to utilize familiar tools and workflows, potentially increasing productivity and code quality. Developers can leverage their existing programming expertise and the vast resources available in these language communities when working on infrastructure projects, which may require less additional learning compared to adopting a specialized syntax

Pulumi Embraces Open-Source Languages and Principles

Languages like Python, JavaScript, and Go are developed and maintained by diverse global communities, ensuring transparency and collective innovation. These languages have remained open-source throughout their history, with their core components freely available for use, modification, and distribution.

In contrast, some domain-specific IaC languages have different ownership and licensing models. For example, HCL has always been under the control of a single company. Recently, there have been changes in its licensing, with parts of its execution engine moving to a "source available" license that is no longer considered fully open-source.

By using mainstream open-source languages, Pulumi offers teams the benefits of well-established ecosystems, community-driven development, and the assurance of continued open availability. This approach can provide additional confidence for organizations concerned about long-term access and evolution of their IaC tooling.

Here's a list of languages Pulumi commonly uses, along with their licenses:

Language	License
C#	MIT (.NET Core)
Java	GNU v2.0 (OpenJDK)
JavaScript	MIT (ECMAScript), BSD-style 3-Clause (V8)
Go	BSD-style 3-Clause
Python	BSD-style Python Software Foundation
TypeScript	Apache 2.0
YAML	Creative Commons Attribution

Developers Onboard Quicker with Pulumi

While HCL serves its purpose in infrastructure as code, many developers find Pulumi's approach more appealing. Pulumi allows engineers to use familiar programming languages, making it easier to leverage existing skills and tools.

This familiarity often leads to faster onboarding and more efficient debugging processes. Pulumi's flexibility enables developers to create custom solutions with ease, tapping into the full power of general-purpose languages. Additionally,

Pulumi's open architecture provides developers with the freedom to integrate various tools and services, fostering a diverse and adaptable ecosystem for infrastructure management.

Enhancing Team Collaboration With Pulumi

Pulumi offers several advantages for fostering better collaboration across teams compared to HCL. While HCL is a powerful tool primarily utilized by operations and DevOps teams, its specialized nature for infrastructure tasks might limit broader engagement with development teams who use other programming languages.

Pulumi, on the other hand, supports multiple languages, allowing development teams to use familiar tools and languages which can enhance collaboration and streamline processes. Additionally, Pulumi's flexibility in integrating with a variety of tools can help bridge gaps between different teams, ensuring smoother communication and more effectiveteamwork.

This inclusive approach not only accelerates progress but also democratizes access to infrastructure management, avoiding bottlenecks and enhancing overall team efficiency.

Reducing Redundancy and Copy-Paste Challenges

HCL often encounters a significant challenge with "copy-pasta," where developers frequently copy and paste code. This practice can lead to redundant configurations across environments, bloating the codebase and complicating management.

Additionally, this increases the risk of errors as small mistakes can propagate across multiple instances, and each one must be manually updated when changes are needed, heightening the chance for inconsistencies. As projects evolve, maintaining a codebase with extensive copy-pasta becomes burdensome, slowing development and elevating the likelihood of bugs.

In contrast, Pulumi reduces the need for such redundancies by enabling more dynamic code reuse and better resource management, streamlining updates and fostering more robust and error-resistant infrastructure development.

Reduce Need for Custom Tooling

As projects expand, managing HCL configurations becomes increasingly complex, often necessitating custom tools for efficient large-scale management. While HCL is adept at defining infrastructure as code, it lacks built-in features to orchestrate, validate, and deploy configurations across extensive networks smoothly. Teams usually need to develop bespoke tools or integrate external solutions like Terragrunt, Atlantis, and Sentinel to automate tasks, ensure consistency, and facilitate integration with other systems. This process can demand significant resources and expertise.

In contrast, Pulumi is designed to scale seamlessly with projects, reducing the need for external tools or extensive custom solutions. Pulumi provides robust capabilities for managing large-scale projects efficiently, streamlining the deployment process, and ensuring consistency across various environments without the overhead of additional tooling. This makes Pulumi a more straightforward and resource-efficient choice for teams looking to manage their infrastructure effectively as they grow.

Future-Proofing Your Infrastructure

HCL, while robust, lacks the future-proof qualities of Pulumi SDKs due to its fixed syntax and usage constraints. In contrast, Pulumi's support for multiple programming languages like Python, TypeScript, and Go, coupled with the potential to adopt new languages over time, offers unparalleled flexibility and adaptability.

This allows developers to write infrastructure as code in familiar languages, facilitating easier integration with existing projects and workflows. As technology progresses, the capacity to embrace new programming languages and paradigms becomes essential. Pulumi's adaptable model supports this continuous evolution, ensuring that infrastructure as code practices can keep pace with the dynamic nature of development environments and preferences, thereby offering a more sustainable and scalable solution for long-term projects.

Making Infrastructure Code Simpler

As HCL continues to evolve, its fixed syntax and structured approach are likely to lead to greater complexity. This evolution could make the language increasingly specialized and challenging for newcomers, particularly those already proficient in more widely-used programming languages. As HCL adapts more bespoke features to meet specific needs, it may become less intuitive and harder to handle effectively, potentially resulting in a language that is cumbersome to use and maintain, especially in long-term projects.

In contrast, Pulumi offers a more streamlined and adaptable alternative. Supporting multiple mainstream programming languages like Python, TypeScript, and Go, Pulumi enables developers to work with tools and languages they are already familiar with, greatly reducing the learning curve and enhancing productivity. This flexibility allows Pulumi to integrate seamlessly into existing projects and workflows, making it a more sustainable and scalable choice for modern infrastructure needs.

HCL in the Wild: Practical Comparisons With Pulumi

In this section, we'll explore real-life examples to compare and contrast HCL with Pulumi programs. By examining how each approach handles similar infrastructure tasks, you'll quickly appreciate the advantages of leveraging a familiar programming language through Pulumi.

This comparison aims to highlight the practical benefits and increased efficiency that Pulumi offers over traditional HCL configurations.

`for_each` Loop With Conditional Expression

First, let's examine an HCL example that includes a for_each loop with a conditional expression. On the surface, it appears straightforward; however, the setup in HCL reveals a unique approach to defining loops.

Initially, you need to choose between using count or for_each. Subsequently, you must comply with a rigid format to implement conditional statements.

This approach is in stark contrast to how such constructs are typically handled in conventional programming languages.

variable "storage_account_names" {
  type = list(string)
  default = ["mickey", "donald", "goofy"]
}

resource "azurerm_resource_group" "example" {
  name     = "storage-rg"
  location = "West Europe"
}

resource "azurerm_storage_account" "my_storage" {
  for_each = {
    for name in var.storage_account_names :
    name => name
    if name != "goofy"
  }
  name                     = each.value
  resource_group_name      = azurerm_resource_group.example.name
  location                 = azurerm_resource_group.example.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
}

Now let’s look at how we can approach the same example using Pulumi.

Typescript:

const storageAccountNames: string[] = [
    "mickey",
    "donald",
    "goofy",
];
const example = new azure.core.ResourceGroup("example", {
    name: "storage-rg",
    location: "West Europe",
});

storageAccountNames.filter((name: string) => name !== "goofy").forEach((name: string) => {
    new azure.storage.Account(`my_storage-${name}`, {
        name: name,
        resourceGroupName: example.name,
        location: example.location,
        accountTier: "Standard",
        accountReplicationType: "GRS",
    });
});

Python:

import pulumi
import pulumi_azure as azure

storage_account_names = ["mickey", "donald", "goofy"]

example = azure.core.ResourceGroup(
    "example",
    name="storage-rg",
    location="West Europe"
)

for name in filter(lambda n: n != "goofy", storage_account_names):
    azure.storage.Account(
        f"my_storage-{name}",
        name=name,
        resource_group_name=example.name,
        location=example.location,
        account_tier="Standard",
        account_replication_type="GRS"
    )

Go:

package main

import (
    "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/core"
    "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/storage"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        storageAccountNames := []string{"mickey", "donald", "goofy"}

        example, err := core.NewResourceGroup(ctx, "example", &core.ResourceGroupArgs{
            Name:     pulumi.String("storage-rg"),
            Location: pulumi.String("West Europe"),
        })
        if err != nil {
            return err
        }

        for _, name := range storageAccountNames {
            if name != "goofy" {
                _, err := storage.NewAccount(ctx, "my_storage-"+name, &storage.AccountArgs{
                    Name:                   pulumi.String(name),
                    ResourceGroupName:      example.Name,
                    Location:               example.Location,
                    AccountTier:            pulumi.String("Standard"),
                    AccountReplicationType: pulumi.String("GRS"),
                })
                if err != nil {
                    return err
                }
            }
        }

        return nil
    })
}

C#:

using Pulumi;
using Pulumi.Azure.Core;
using Pulumi.Azure.Storage;
using System.Collections.Generic;

class Program
{
    static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
    public MyStack()
    {
        var storageAccountNames = new List<string> { "mickey", "donald", "goofy" };

        var resourceGroup = new ResourceGroup("example", new ResourceGroupArgs
        {
            Name = "storage-rg",
            Location = "West Europe",
        });

        foreach (var name in storageAccountNames)
        {
            if (name != "goofy")
            {
                var storageAccount = new Account($"my_storage-{name}", new AccountArgs
                {
                    Name = name,
                    ResourceGroupName = resourceGroup.Name,
                    Location = resourceGroup.Location,
                    AccountTier = "Standard",
                    AccountReplicationType = "GRS",
                });
            }
        }
    }
}

Java:

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azurenative.core.ResourceGroup;
import com.pulumi.azurenative.core.ResourceGroupArgs;
import com.pulumi.azurenative.storage.Account;
import com.pulumi.azurenative.storage.AccountArgs;

import java.util.List;

public class Main {
    public static void main(String[] args) {
        Pulumi.run(Main::stack);
    }

    public static void stack(Context ctx) {
        var storageAccountNames = List.of("mickey", "donald", "goofy");

        var resourceGroup = new ResourceGroup("example", ResourceGroupArgs.builder()
                .name("storage-rg")
                .location("West Europe")
                .build());

        for (var name : storageAccountNames) {
            if (!name.equals("goofy")) {
                var storageAccount = new Account("my_storage-" + name, AccountArgs.builder()
                        .name(name)
                        .resourceGroupName(resourceGroup.name())
                        .location(resourceGroup.location())
                        .kind("StorageV2")
                        .sku(AccountSkuArgs.builder()
                                .name("Standard_GRS")
                                .tier("Standard")
                                .build())
                        .build());
            }
        }
    }
}

We can harness the full capabilities of a generic programming language by using the filter function to streamline our array without needing conditional statements, then iterating through the filtered array with forEach. This method is less prone to errors, faster, and simpler to debug.

However, implementing a for_each loop in HCL can be even more complex.

`for_each` Loop With Multi-Level Nested Dynamic Blocks

To illustrate how HCL manages complex nested dynamic blocks, consider dynamic providers, which are designed to dynamically construct repeatable nested structures. This added complexity can render the code challenging to read and maintain. We will look at a specific resource from this GitHub repository to provide a more detailed example:

variable "enable_auto_scaling" {
  type        = bool
  default     = false
  description = "Enable node pool autoscaling"
}

variable "agents_pool_kubelet_configs" {
  type = list(object({
    cpu_manager_policy = optional(string)
    cpu_cfs_quota_enabled = optional(bool, true)
    cpu_cfs_quota_period = optional(string)
    image_gc_high_threshold = optional(number)
    image_gc_low_threshold = optional(number)
    topology_manager_policy = optional(string)
    allowed_unsafe_sysctls = optional(set(string))
    container_log_max_size_mb = optional(number)
    container_log_max_line = optional(number)
    pod_max_pid = optional(number)
  }))
  default = [
    {
      cpu_manager_policy        = "none"
      cpu_cfs_quota_enabled     = true
      cpu_cfs_quota_period      = "100ms"
      image_gc_high_threshold   = 85
      image_gc_low_threshold    = 65
      topology_manager_policy   = "none"
      allowed_unsafe_sysctls = ["net.*", "kernel.*"]
      container_log_max_size_mb = 10
      container_log_max_line    = 10
      pod_max_pid               = 100
    }
  ]
  nullable = false
}

resource "azurerm_resource_group" "example" {
  name     = "storage-rg"
  location = "West Europe"
}

resource "azurerm_kubernetes_cluster" "main" {
  name                = "example-aks"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  dns_prefix          = "exampleaks"

  dynamic "default_node_pool" {
    for_each = var.enable_auto_scaling == true ? [] : ["default_node_pool_manually_scaled"]
    content {
      name                = "default"
      vm_size             = "Standard_DS2_v2"
      zones = [1, 2, 3]
      enable_auto_scaling = true
      min_count           = 1
      max_count           = 3
      dynamic "kubelet_config" {
        for_each = var.agents_pool_kubelet_configs
        content {
          allowed_unsafe_sysctls = kubelet_config.value.allowed_unsafe_sysctls
        }
      }
    }
  }
  identity {
    type = "SystemAssigned"
  }
}

As demonstrated, incorporating even a single nested dynamic provider block can significantly complicate the code, making it challenging to read. We recommend using dynamic blocks sparingly to preserve readability and simplify maintenance, especially when creating abstractions with the aid of a module. Excessive use of these blocks, particularly when defining a resource's arguments and nested blocks directly from input variables, may lead to abstractions that aren't very practical.

Let's see how this scenario would be handled using Pulumi:

Typescript:

const config = new pulumi.Config();
const enableAutoScaling = config.getBoolean("enableAutoScaling") || false;

const agentsPoolKubeletConfigs = config.getObject("agentsPoolKubeletConfigs") || {
    allowedUnsafeSysctls: [
        "kernel.*",
        "net.*",
    ],
    containerLogMaxLine: 10,
    containerLogMaxSizeMb: 10,
    cpuCfsQuotaEnabled: true,
    cpuCfsQuotaPeriod: "100ms",
    cpuManagerPolicy: "none",
    imageGcHighThreshold: 85,
    imageGcLowThreshold: 65,
    podMaxPid: 100,
    topologyManagerPolicy: "none",
};
const example = new azure.core.ResourceGroup("example", {
    name: "storage-rg",
    location: "West Europe",
});

let kubeletConfigs: azure.types.input.containerservice.KubernetesClusterDefaultNodePoolKubeletConfig = agentsPoolKubeletConfigs;

let defaultNodePool: azure.types.input.containerservice.KubernetesClusterDefaultNodePool = {
    name: "default",
    vmSize: "Standard_DS2_v2",
    zones: [
        "1",
        "2",
        "3",
    ],
    enableAutoScaling: enableAutoScaling,
    minCount: 1,
    maxCount: 3,
    kubeletConfig: kubeletConfigs,
};

Python:

import pulumi
import pulumi_azure as azure

config = pulumi.Config()
enable_auto_scaling = config.get_bool("enableAutoScaling") or False

agents_pool_kubelet_configs = config.get_object("agentsPoolKubeletConfigs") or {
    "allowedUnsafeSysctls": [
        "kernel.*",
        "net.*",
    ],
    "containerLogMaxLine": 10,
    "containerLogMaxSizeMb": 10,
    "cpuCfsQuotaEnabled": True,
    "cpuCfsQuotaPeriod": "100ms",
    "cpuManagerPolicy": "none",
    "imageGcHighThreshold": 85,
    "imageGcLowThreshold": 65,
    "podMaxPid": 100,
    "topologyManagerPolicy": "none",
}

example = azure.core.ResourceGroup("example",
                                   name="storage-rg",
                                   location="West Europe")

kubelet_configs = azure.containerservice.KubernetesClusterDefaultNodePoolKubeletConfigArgs(
    **agents_pool_kubelet_configs
)

default_node_pool = azure.containerservice.KubernetesClusterDefaultNodePoolArgs(
    name="default",
    vm_size="Standard_DS2_v2",
    zones=["1", "2", "3"],
    enable_auto_scaling=enable_auto_scaling,
    min_count=1,
    max_count=3,
    kubelet_config=kubelet_configs,
)

Go:

package main

import (
    "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/core"
    "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/containerservice"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        enableAutoScaling := false

        agentsPoolKubeletConfigs := map[string]interface{}{
            "allowedUnsafeSysctls": []string{
                "kernel.*",
                "net.*",
            },
            "containerLogMaxLine":   10,
            "containerLogMaxSizeMb": 10,
            "cpuCfsQuotaEnabled":    true,
            "cpuCfsQuotaPeriod":     "100ms",
            "cpuManagerPolicy":      "none",
            "imageGcHighThreshold":  85,
            "imageGcLowThreshold":   65,
            "podMaxPid":             100,
            "topologyManagerPolicy": "none",
        }

        example, err := core.NewResourceGroup(ctx, "example", &core.ResourceGroupArgs{
            Name:     pulumi.String("storage-rg"),
            Location: pulumi.String("West Europe"),
        })
        if err != nil {
            return err
        }

        kubeletConfigs := &containerservice.KubernetesClusterDefaultNodePoolKubeletConfigArgs{
            AllowedUnsafeSysctls: pulumi.StringArray{
                pulumi.String("kernel.*"),
                pulumi.String("net.*"),
            },
            ContainerLogMaxLine:   pulumi.Int(agentsPoolKubeletConfigs["containerLogMaxLine"].(int)),
            ContainerLogMaxSizeMb: pulumi.Int(agentsPoolKubeletConfigs["containerLogMaxSizeMb"].(int)),
            CpuCfsQuotaEnabled:    pulumi.Bool(agentsPoolKubeletConfigs["cpuCfsQuotaEnabled"].(bool)),
            CpuCfsQuotaPeriod:     pulumi.String(agentsPoolKubeletConfigs["cpuCfsQuotaPeriod"].(string)),
            CpuManagerPolicy:      pulumi.String(agentsPoolKubeletConfigs["cpuManagerPolicy"].(string)),
            ImageGcHighThreshold:  pulumi.Int(agentsPoolKubeletConfigs["imageGcHighThreshold"].(int)),
            ImageGcLowThreshold:   pulumi.Int(agentsPoolKubeletConfigs["imageGcLowThreshold"].(int)),
            PodMaxPid:             pulumi.Int(agentsPoolKubeletConfigs["podMaxPid"].(int)),
            TopologyManagerPolicy: pulumi.String(agentsPoolKubeletConfigs["topologyManagerPolicy"].(string)),
        }

        _, err = containerservice.NewKubernetesClusterDefaultNodePool(ctx, "defaultNodePool", &containerservice.KubernetesClusterDefaultNodePoolArgs{
            Name:              pulumi.String("default"),
            VmSize:            pulumi.String("Standard_DS2_v2"),
            Zones:             pulumi.StringArray{pulumi.String("1"), pulumi.String("2"), pulumi.String("3")},
            EnableAutoScaling: pulumi.Bool(enableAutoScaling),
            MinCount:          pulumi.Int(1),
            MaxCount:          pulumi.Int(3),
            KubeletConfig:     kubeletConfigs,
        }, pulumi.Parent(example))
        if err != nil {
            return err
        }

        return nil
    })
}

C#:

using Pulumi;
using Pulumi.AzureNative.Resources;
using Pulumi.AzureNative.ContainerService;
using System.Collections.Generic;

class MyStack : Stack
{
    public MyStack()
    {
        var enableAutoScaling = false;

        var agentsPoolKubeletConfigs = new Dictionary<string, object>
        {
            { "allowedUnsafeSysctls", new string[] { "kernel.*", "net.*" } },
            { "containerLogMaxLine", 10 },
            { "containerLogMaxSizeMb", 10 },
            { "cpuCfsQuotaEnabled", true },
            { "cpuCfsQuotaPeriod", "100ms" },
            { "cpuManagerPolicy", "none" },
            { "imageGcHighThreshold", 85 },
            { "imageGcLowThreshold", 65 },
            { "podMaxPid", 100 },
            { "topologyManagerPolicy", "none" }
        };

        var resourceGroup = new ResourceGroup("example", new ResourceGroupArgs
        {
            Name = "storage-rg",
            Location = "West Europe"
        });

        var kubeletConfigArgs = new KubernetesClusterDefaultNodePoolKubeletConfigArgs
        {
            AllowedUnsafeSysctls = agentsPoolKubeletConfigs["allowedUnsafeSysctls"] as string[],
            ContainerLogMaxLine = agentsPoolKubeletConfigs["containerLogMaxLine"] as int?,
            ContainerLogMaxSizeMb = agentsPoolKubeletConfigs["containerLogMaxSizeMb"] as int?,
            CpuCfsQuotaEnabled = agentsPoolKubeletConfigs["cpuCfsQuotaEnabled"] as bool?,
            CpuCfsQuotaPeriod = agentsPoolKubeletConfigs["cpuCfsQuotaPeriod"] as string,
            CpuManagerPolicy = agentsPoolKubeletConfigs["cpuManagerPolicy"] as string,
            ImageGcHighThreshold = agentsPoolKubeletConfigs["imageGcHighThreshold"] as int?,
            ImageGcLowThreshold = agentsPoolKubeletConfigs["imageGcLowThreshold"] as int?,
            PodMaxPid = agentsPoolKubeletConfigs["podMaxPid"] as int?,
            TopologyManagerPolicy = agentsPoolKubeletConfigs["topologyManagerPolicy"] as string
        };

        var defaultNodePool = new KubernetesClusterDefaultNodePool("defaultNodePool", new KubernetesClusterDefaultNodePoolArgs
        {
            Name = "default",
            VmSize = "Standard_DS2_v2",
            Zones = new[] { "1", "2", "3" },
            EnableAutoScaling = enableAutoScaling,
            MinCount = 1,
            MaxCount = 3,
            KubeletConfig = kubeletConfigArgs
        }, new CustomResourceOptions { Parent = resourceGroup });
    }
}

class Program
{
    static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

Java:

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azure.core.ResourceGroup;
import com.pulumi.azure.core.inputs.ResourceGroupArgs;
import com.pulumi.azure.containerservice.KubernetesClusterDefaultNodePool;
import com.pulumi.azure.containerservice.inputs.KubernetesClusterDefaultNodePoolArgs;
import com.pulumi.azure.containerservice.inputs.KubernetesClusterDefaultNodePoolKubeletConfigArgs;

import java.util.Map;
import java.util.HashMap;

public class MyStack {
    public static void main(String[] args) {
        Pulumi.run(ctx -> {
            boolean enableAutoScaling = false;

            Map<String, Object> agentsPoolKubeletConfigs = new HashMap<>();
            agentsPoolKubeletConfigs.put("allowedUnsafeSysctls", new String[]{"kernel.*", "net.*"});
            agentsPoolKubeletConfigs.put("containerLogMaxLine", 10);
            agentsPoolKubeletConfigs.put("containerLogMaxSizeMb", 10);
            agentsPoolKubeletConfigs.put("cpuCfsQuotaEnabled", true);
            agentsPoolKubeletConfigs.put("cpuCfsQuotaPeriod", "100ms");
            agentsPoolKubeletConfigs.put("cpuManagerPolicy", "none");
            agentsPoolKubeletConfigs.put("imageGcHighThreshold", 85);
            agentsPoolKubeletConfigs.put("imageGcLowThreshold", 65);
            agentsPoolKubeletConfigs.put("podMaxPid", 100);
            agentsPoolKubeletConfigs.put("topologyManagerPolicy", "none");

            var resourceGroup = new ResourceGroup("example", ResourceGroupArgs.builder()
                    .name("storage-rg")
                    .location("West Europe")
                    .build());

            var kubeletConfigArgs = KubernetesClusterDefaultNodePoolKubeletConfigArgs.builder()
                    .allowedUnsafeSysctls((String[]) agentsPoolKubeletConfigs.get("allowedUnsafeSysctls"))
                    .containerLogMaxLine((Integer) agentsPoolKubeletConfigs.get("containerLogMaxLine"))
                    .containerLogMaxSizeMb((Integer) agentsPoolKubeletConfigs.get("containerLogMaxSizeMb"))
                    .cpuCfsQuotaEnabled((Boolean) agentsPoolKubeletConfigs.get("cpuCfsQuotaEnabled"))
                    .cpuCfsQuotaPeriod((String) agentsPoolKubeletConfigs.get("cpuCfsQuotaPeriod"))
                    .cpuManagerPolicy((String) agentsPoolKubeletConfigs.get("cpuManagerPolicy"))
                    .imageGcHighThreshold((Integer) agentsPoolKubeletConfigs.get("imageGcHighThreshold"))
                    .imageGcLowThreshold((Integer) agentsPoolKubeletConfigs.get("imageGcLowThreshold"))
                    .podMaxPid((Integer) agentsPoolKubeletConfigs.get("podMaxPid"))
                    .topologyManagerPolicy((String) agentsPoolKubeletConfigs.get("topologyManagerPolicy"))
                    .build();

            var defaultNodePool = new KubernetesClusterDefaultNodePool("defaultNodePool", KubernetesClusterDefaultNodePoolArgs.builder()
                    .name("default")
                    .vmSize("Standard_DS2_v2")
                    .zones(new String[]{"1", "2", "3"})
                    .enableAutoScaling(enableAutoScaling)
                    .minCount(1)
                    .maxCount(3)
                    .kubeletConfig(kubeletConfigArgs)
                    .build(), new CustomResourceOptions().parent(resourceGroup));
        });
    }
}

In Pulumi, particularly with TypeScript, we can straightforwardly create new objects of a specific type and fill them with values from our configuration. When conditions are necessary, we can effortlessly integrate conventional if statements. This method significantly simplifies the code, enhancing readability and maintainability for future modifications.

Execute Code Programming Code With `null_resource`, `local_file`, and `provisioners`

Occasionally, there's a substantial need to execute scripts written in a programming language. A typical scenario involves packaging the source code for an AWS Lambda function. Let's examine an example from the Terraform AWS Lambda module:

locals {
  python = (substr(pathexpand("~"), 0, 1) == "/") ? "python3" : "python.exe"
}

# Generates a filename for the zip archive based on the content of the files
# in source_path. The filename will change when the source code changes.
data "external" "archive_prepare" {
  count = var.create && var.create_package ? 1 : 0
  program = [local.python, "${path.module}/package.py", "prepare"]
  query = {
    paths = jsonencode({
      module = path.module
      root   = path.root
      cwd    = path.cwd
    })
    docker = var.build_in_docker ? jsonencode({
      docker_pip_cache          = var.docker_pip_cache
      docker_build_root         = var.docker_build_root
      docker_file               = var.docker_file
      docker_image              = var.docker_image
      with_ssh_agent            = var.docker_with_ssh_agent
      docker_additional_options = var.docker_additional_options
      docker_entrypoint         = var.docker_entrypoint
    }) : null
    artifacts_dir = var.artifacts_dir
    runtime       = var.runtime
    source_path = jsonencode(var.source_path)
    hash_extra    = var.hash_extra
    hash_extra_paths = jsonencode(
      [
        # Temporary fix when building from multiple locations
        # We should take into account content of package.py when counting hash
        # Related issue: https://github.com/terraform-aws-modules/terraform-aws-lambda/issues/63
        # "${path.module}/package.py"
      ]
    )
    recreate_missing_package = var.recreate_missing_package
  }
}

# This transitive resource used as a bridge between a state stored
# in a Terraform plan and a call of a build command on the apply stage
# to transfer a noticeable amount of data
resource "local_file" "archive_plan" {
  count                = var.create && var.create_package ? 1 : 0
  content              = data.external.archive_prepare[0].result.build_plan
  filename             = data.external.archive_prepare[0].result.build_plan_filename
  directory_permission = "0755"
  file_permission      = "0644"
}

# Build the zip archive whenever the filename changes.
resource "null_resource" "archive" {
  count = var.create && var.create_package ? 1 : 0
  triggers = {
    filename  = data.external.archive_prepare[0].result.filename
    timestamp = var.trigger_on_package_timestamp ? data.external.archive_prepare[0].result.timestamp : null
  }
  provisioner "local-exec" {
    interpreter = [
      local.python, "${path.module}/package.py", "build",
      "--timestamp", data.external.archive_prepare[0].result.timestamp
    ]
    command = data.external.archive_prepare[0].result.build_plan_filename
  }
  depends_on = [local_file.archive_plan]
}

We observe a complex mixture of different providers (local and null_resource) and the use of provisioner in this setup. This complexity makes the code difficult to read and maintain, leading to a split-brain scenario where some logic resides in the code and the rest in HCL. It also complicates debugging, particularly if the team is not well-versed in Python.

In Pulumi, the emphasis is on simplicity: simply write your Python code and integrate it directly with Pulumi SDK calls. No additional steps are required, nor is there a need to run the Python binary separately to execute the file. This approach is all about streamlining the process to simplify your operations. For example, consider this scenario where we read the name of an Azure Resource Group from a JSON file (config.json), demonstrating how seamlessly Python can be integrated with Pulumi’s infrastructure management capabilities.

Python:

import json
import pulumi
import pulumi_azure_native as azure_native

# Load the JSON file
with open('config.json') as json_file:
    config = json.load(json_file)

# Extract values from the JSON file
resource_group_name = config['resourceGroupName']
location = config['location']

# Create an Azure Resource Group
resource_group = azure_native.resources.ResourceGroup(
    "resourceGroup",
    resource_group_name=resource_group_name,
    location=location
)

# Export the name of the resource group
pulumi.export('resourceGroupName', resource_group.name)

If Statements Constructed With `count`

variable "mode" {
  default = "not-spoke"
}

variable "restricted_net_hub_project_id" {
  default = "xxx"
}

data "google_compute_network" "vpc_restricted_net_hub" {
  count   = var.mode == "spoke" ? 1 : 0
  name    = "vpc-c-shared-restricted-hub"
  project = var.restricted_net_hub_project_id
}

In this HCL example, the count parameter is employed in a manner that extends beyond its initial design—effectively using it to conditionally create or omit resources based on the value of var.mode. Although this method is operational, it complicates the code, making it more challenging to understand and maintain, as it repurposes Terraform constructs in ways not originally intended.

In contrast, Pulumi allows the use of native if statements from the programming language, such as TypeScript or Python. This native approach significantly improves readability and maintainability by aligning with standard programming practices, thereby making it simpler for developers to comprehend and troubleshoot. The utilization of familiar programming constructs in Pulumi means that you can develop more intuitive and straightforward infrastructure code without having to manipulate the tool to accommodate unconventional patterns.

Typescript:

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const config = new pulumi.Config();
const mode = config.get("mode") || "";
const restrictedNetHubProjectId = config.get("restrictedNetHubProjectId") || "";
if (mode == "spoke") {
    const vpcSharedSpoke = gcp.compute.getNetworkOutput({
        name: "vpc-c-shared-spoke",
        project: restrictedNetHubProjectId,
    });
}

Python:

import pulumi
import pulumi_gcp as gcp

config = pulumi.Config()
mode = config.get("mode") or ""
restricted_net_hub_project_id = config.get("restrictedNetHubProjectId") or ""

if mode == "spoke":
    vpc_shared_spoke = gcp.compute.get_network_output(
        name="vpc-c-shared-spoke",
        project=restricted_net_hub_project_id,
    )

Go:

package main

import (
    "github.com/pulumi/pulumi-gcp/sdk/v7/go/gcp/compute"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        mode := ""

        if mode == "spoke" {
            _, err := compute.LookupNetwork(ctx, &compute.LookupNetworkArgs{
                Name:    "vpc-c-shared-spoke",
                Project: "your-restrictedNetHubProjectId",
            })

            if err != nil {
                return err
            }
        }

        return nil
    })
}

C#:

using Pulumi;
using Pulumi.Gcp.Compute;

class Program
{
    static Task<int> Main() => Deployment.RunAsync<ComputeStack>();

    class ComputeStack : Stack
    {
        public ComputeStack()
        {
            var mode = "";

            if (mode == "spoke")
            {
                var network = Output.Tuple(Output.Create("vpc-c-shared-spoke"), Output.Create("your-restrictedNetHubProjectId"))
                    .Apply(tuple => GetNetwork.InvokeAsync(new GetNetworkArgs
                    {
                        Name = tuple.Item1,
                        Project = tuple.Item2
                    }));

                // Handle potential errors if necessary
                network.Apply(n =>
                {
                    if (n == null)
                    {
                        throw new Exception("Network not found");
                    }
                });
            }
        }
    }
}

Java:

import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.gcp.compute.ComputeFunctions;
import com.pulumi.gcp.compute.inputs.GetNetworkArgs;

public class Main {
    public static void main(String[] args) {
        Pulumi.run(ctx -> {
            String mode = "";

            if ("spoke".equals(mode)) {
                Output<Tuple<String, String>> networkData = Output.tuple(Output.of("vpc-c-shared-spoke"), Output.of("your-restrictedNetHubProjectId"));

                Output<ComputedNetwork> network = networkData.applyValue(tuple ->
                        ComputeFunctions.getNetwork(GetNetworkArgs.builder()
                                .name(tuple.t1)
                                .project(tuple.t2)
                                .build()
                        )
                );

                network.applyValue(n -> {
                    if (n == null) {
                        throw new RuntimeException("Network not found");
                    }
                    return null;
                });
            }
        });
    }
}

HCL Function Frenzy

Another significant source of confusion and ongoing learning challenges is the intensive use of built-in functions within HCL expressions. Although these functions initially appear beneficial for executing complex operations swiftly, they often result in overly complicated expressions.

Here's an example of highly complex expressions that utilize extensive function usage from the Google Network module:

locals {
  local_network_name = element(reverse(split("/", var.local_network)), 0)
  peer_network_name = element(reverse(split("/", var.peer_network)), 0)

  local_network_peering      = "${var.prefix}-${local.local_network_name}-${local.peer_network_name}"
  local_network_peering_name = length(local.local_network_peering) < 63 ? local.local_network_peering :
    "${substr(local.local_network_peering, 0, min(58, length(local.local_network_peering)))}-${random_string.network_peering_suffix.result}"
  peer_network_peering      = "${var.prefix}-${local.peer_network_name}-${local.local_network_name}"
  peer_network_peering_name = length(local.peer_network_peering) < 63 ? local.peer_network_peering :
    "${substr(local.peer_network_peering, 0, min(58, length(local.peer_network_peering)))}-${random_string.network_peering_suffix.result}"
}

In Pulumi, we can utilize the same functions from the programming language selected for our infrastructure projects. This approach simplifies the development process, as there's no need to learn new, specialized functions as required in HCL. Additionally, we can incorporate extra libraries as needed. Unlike HCL, which necessitates learning unique functions, Pulumi allows you to work with familiar tools, thereby streamlining the entire process and enhancing flexibility.

Typescript:

import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const localNetwork = config.require("localNetwork");
const peerNetwork = config.require("peerNetwork");
const prefix = config.require("prefix");

const localNetworkName = localNetwork.split("/").reverse()[0];
const peerNetworkName = peerNetwork.split("/").reverse()[0];

const localNetworkPeering = `${prefix}-${localNetworkName}-${peerNetworkName}`;
const peerNetworkPeering = `${prefix}-${peerNetworkName}-${localNetworkName}`;
const networkPeeringSuffix = "peering";
const localNetworkPeeringName = localNetworkPeering.length < 63
    ? localNetworkPeering
    : `${localNetworkPeering.substring(0, Math.min(58, localNetworkPeering.length))}-${networkPeeringSuffix}`;

const peerNetworkPeeringName = peerNetworkPeering.length < 63
    ? peerNetworkPeering
    : `${peerNetworkPeering.substring(0, Math.min(58, peerNetworkPeering.length))}-${networkPeeringSuffix}`;

export const outputs = {
    localNetworkName,
    peerNetworkName,
    localNetworkPeeringName,
    peerNetworkPeeringName,
};

Python:

import pulumi

config = pulumi.Config()
local_network = config.require("localNetwork")
peer_network = config.require("peerNetwork")
prefix = config.require("prefix")

local_network_name = local_network.split("/")[::-1][0]
peer_network_name = peer_network.split("/")[::-1][0]

local_network_peering = f"{prefix}-{local_network_name}-{peer_network_name}"
peer_network_peering = f"{prefix}-{peer_network_name}-{local_network_name}"
network_peering_suffix = "peering"

local_network_peering_name = (
    local_network_peering if len(local_network_peering) < 63
    else f"{local_network_peering[:min(58, len(local_network_peering))]}-{network_peering_suffix}"
)

peer_network_peering_name = (
    peer_network_peering if len(peer_network_peering) < 63
    else f"{peer_network_peering[:min(58, len(peer_network_peering))]}-{network_peering_suffix}"
)

pulumi.export("localNetworkName", local_network_name)
pulumi.export("peerNetworkName", peer_network_name)
pulumi.export("localNetworkPeeringName", local_network_peering_name)
pulumi.export("peerNetworkPeeringName", peer_network_peering_name)

Go:

package main

import (
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        localNetwork := "your-local-network"
        peerNetwork := "your-peer-network"
        prefix := "your-prefix"

        localNetworkName := reverseString(getNetworkName(localNetwork))
        peerNetworkName := reverseString(getNetworkName(peerNetwork))

        localNetworkPeering := prefix + "-" + localNetworkName + "-" + peerNetworkName
        peerNetworkPeering := prefix + "-" + peerNetworkName + "-" + localNetworkName
        networkPeeringSuffix := "peering"

        localNetworkPeeringName := condenseName(localNetworkPeering, networkPeeringSuffix)
        peerNetworkPeeringName := condenseName(peerNetworkPeering, networkPeeringSuffix)

        ctx.Export("localNetworkName", pulumi.String(localNetworkName))
        ctx.Export("peerNetworkName", pulumi.String(peerNetworkName))
        ctx.Export("localNetworkPeeringName", pulumi.String(localNetworkPeeringName))
        ctx.Export("peerNetworkPeeringName", pulumi.String(peerNetworkPeeringName))

        return nil
    })
}

func getNetworkName(network string) string {
    parts := strings.Split(network, "/")
    return parts[len(parts)-1]
}

func reverseString(s string) string {
    runes := []rune(s)
    for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
        runes[i], runes[j] = runes[j], runes[i]
    }
    return string(runes)
}

func condenseName(name string, suffix string) string {
    if len(name) < 63 {
        return name
    }
    return name[:min(58, len(name))] + "-" + suffix
}

func min(a, b int) int {
    if a < b {
        return a
    }
    return b
}

C#:

using System;
using System.Linq;
using Pulumi;

class Program
{
    static Task<int> Main() => Deployment.RunAsync<MyStack>();

    class MyStack : Stack
    {
        public MyStack()
        {
            var localNetwork = "your-local-network";
            var peerNetwork = "your-peer-network";
            var prefix = "your-prefix";

            var localNetworkName = ReverseString(GetNetworkName(localNetwork));
            var peerNetworkName = ReverseString(GetNetworkName(peerNetwork));

            var localNetworkPeering = $"{prefix}-{localNetworkName}-{peerNetworkName}";
            var peerNetworkPeering = $"{prefix}-{peerNetworkName}-{localNetworkName}";
            var networkPeeringSuffix = "peering";

            var localNetworkPeeringName = CondenseName(localNetworkPeering, networkPeeringSuffix);
            var peerNetworkPeeringName = CondenseName(peerNetworkPeering, networkPeeringSuffix);

            this.Export("localNetworkName", localNetworkName);
            this.Export("peerNetworkName", peerNetworkName);
            this.Export("localNetworkPeeringName", localNetworkPeeringName);
            this.Export("peerNetworkPeeringName", peerNetworkPeeringName);
        }

        private string GetNetworkName(string network)
        {
            var parts = network.Split('/');
            return parts[parts.Length - 1];
        }

        private string ReverseString(string s)
        {
            var chars = s.ToCharArray();
            Array.Reverse(chars);
            return new string(chars);
        }

        private string CondenseName(string name, string suffix)
        {
            if (name.Length < 63) return name;
            return name.Substring(0, Math.Min(58, name.Length)) + "-" + suffix;
        }
    }
}

Java:

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.resources.Stack;

import java.util.Arrays;

public class Program {
    public static void main(String[] args) {
        Pulumi.run(Program::stack);
    }

    public static void stack(Context ctx) {
        var localNetwork = "your-local-network";
        var peerNetwork = "your-peer-network";
        var prefix = "your-prefix";

        var localNetworkName = reverseString(getNetworkName(localNetwork));
        var peerNetworkName = reverseString(getNetworkName(peerNetwork));

        var localNetworkPeering = String.format("%s-%s-%s", prefix, localNetworkName, peerNetworkName);
        var peerNetworkPeering = String.format("%s-%s-%s", prefix, peerNetworkName, localNetworkName);
        var networkPeeringSuffix = "peering";

        var localNetworkPeeringName = condenseName(localNetworkPeering, networkPeeringSuffix);
        var peerNetworkPeeringName = condenseName(peerNetworkPeering, networkPeeringSuffix);

        ctx.export("localNetworkName", Output.of(localNetworkName));
        ctx.export("peerNetworkName", Output.of(peerNetworkName));
        ctx.export("localNetworkPeeringName", Output.of(localNetworkPeeringName));
        ctx.export("peerNetworkPeeringName", Output.of(peerNetworkPeeringName));
    }

    private static String getNetworkName(String network) {
        var parts = network.split("/");
        return parts[parts.length - 1];
    }

    private static String reverseString(String s) {
        var chars = s.toCharArray();
        for (int i = 0, j = chars.length - 1; i < j; i++, j--) {
            char temp = chars[i];
            chars[i] = chars[j];
            chars[j] = temp;
        }
        return new String(chars);
    }

    private static String condenseName(String name, String suffix) {
        if (name.length() < 63) return name;
        return name.substring(0, Math.min(58, name.length())) + "-" + suffix;
    }
}

Pitfalls of HCL's `try` Function

HCL’s try function can be problematic as it evaluates each argument until one does not produce an error. While this may seem useful for navigating complex data structures, it often obscures underlying issues by ignoring errors instead of resolving them.

This behavior can complicate debugging by concealing the true causes of failures, resulting in brittle and error-prone configurations. Here is an example illustrating the overuse of the try function:

platform = try(each.value.platform, var.eks_managed_node_group_defaults.platform, "linux")
cluster_endpoint = try(time_sleep.this[0].triggers["cluster_endpoint"], "")
cluster_auth_base64 = try(time_sleep.this[0].triggers["cluster_certificate_authority_data"], "")
cluster_service_ipv4_cidr = var.cluster_service_ipv4_cidr
cluster_ip_family         = var.cluster_ip_family
cluster_service_cidr = try(time_sleep.this[0].triggers["cluster_service_cidr"], "")
enable_bootstrap_user_data = try(each.value.enable_bootstrap_user_data, var.eks_managed_node_group_defaults.enable_bootstrap_user_data, false)
pre_bootstrap_user_data = try(each.value.pre_bootstrap_user_data, var.eks_managed_node_group_defaults.pre_bootstrap_user_data, "")
post_bootstrap_user_data = try(each.value.post_bootstrap_user_data, var.eks_managed_node_group_defaults.post_bootstrap_user_data, "")
bootstrap_extra_args = try(each.value.bootstrap_extra_args, var.eks_managed_node_group_defaults.bootstrap_extra_args, "")
user_data_template_path = try(each.value.user_data_template_path, var.eks_managed_node_group_defaults.user_data_template_path, "")
cloudinit_pre_nodeadm = try(each.value.cloudinit_pre_nodeadm, var.eks_managed_node_group_defaults.cloudinit_pre_nodeadm, [])
cloudinit_post_nodeadm = try(each.value.cloudinit_post_nodeadm, var.eks_managed_node_group_defaults.cloudinit_post_nodeadm, [])

Switching to Pulumi provides a clearer, more maintainable approach:

Typescript:

import * as pulumi from "@pulumi/pulumi";

// Example configuration using TypeScript for better error handling
const config = new pulumi.Config();
const platform = config.get("platform") || "linux"; // Default to "linux" if undefined

// Output using straightforward TypeScript logic
pulumi.output({ platform });

Python:

import pulumi

# Example configuration using Python
config = pulumi.Config()
platform = config.get("platform") or "linux"  # Default to "linux" if undefined

# Output using straightforward Python logic
pulumi.export("platform", platform)

Go:

package main

import (
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        platform, ok := ctx.GetConfig("platform")
        if !ok {
            platform = "linux" // Default to "linux" if undefined
        }

        ctx.Export("platform", pulumi.String(platform))
        return nil
    })
}

C#:

using Pulumi;

class Program
{
    static Task<int> Main() => Pulumi.Deployment.RunAsync(() => 
    {
        var config = new Config();
        var platform = config.Get("platform") ?? "linux"; // Default to "linux" if undefined

        return new Dictionary<string, object?>
        {
            { "platform", platform }
        };
    });
}

Java:

import com.pulumi.Config;
import com.pulumi.Pulumi;

import java.util.HashMap;
import java.util.Map;

public class Main {
    public static void main(String[] args) {
        Pulumi.run(ctx -> {
            Config config = Config.auto();
            String platform = config.get("platform").orElse("linux"); // Default to "linux" if undefined

            Map<String, Object> result = new HashMap<>();
            result.put("platform", platform);

            ctx.export("result", result);
        });
    }
}

This Pulumi example demonstrates how to manage configuration values more directly, circumventing the need for complex try functions that might obscure underlying errors. By employing standard programming constructs, you can enhance both the readability and maintainability of your infrastructure code, ensuring that potential issues are addressed rather than hidden.

In Conclusion

In conclusion, Pulumi's approach to infrastructure as code provides substantial benefits over HCL, establishing it as a more developer-friendly and future-ready solution. By utilizing general-purpose programming languages, Pulumi effectively addresses many of the limitations and complexities associated with HCL.

Pulumi's adoption of languages like TypeScript, Python, and Go offers several significant advantages:

Access to a larger ecosystem and community support, which enhances tooling, debugging capabilities, and available resources.
Familiar syntax and constructs that ease the learning curve for developers and improve code readability.
Increased flexibility and expressiveness, allowing developers to devise more sophisticated solutions to complex infrastructure challenges.
Enhanced integration with existing development workflows and practices.
Improved collaboration between development and operations teams, aiding in breaking down traditional silos.

The examples highlighted throughout this comparison illustrate how Pulumi's methodology results in cleaner, more intuitive code. Operations that would typically require intricate HCL expressions or multiple custom resources can often be streamlined using simple programming constructs in Pulumi.

Additionally, Pulumi's framework is inherently adaptable to future changes in cloud technologies and programming paradigms. As new languages or cloud services become available, Pulumi can seamlessly integrate them, ensuring that your infrastructure-as-code practices stay relevant and effective.

For organizations aiming to modernize their infrastructure management, streamline DevOps processes, and empower their development teams, Pulumi stands out as the premier alternative to HCL. Its capability to unify application and infrastructure code within a single programming model paves the way for more efficient, maintainable, and scalable cloud engineering practices.

By opting for Pulumi, teams can leverage the true potential of programming languages for infrastructure, leading to more robust, flexible, and developer-friendly cloud deployments.

We always welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is straightforward and free.
Get started today!

Interested in migrating? Our cloud experts are ready to assist.
Learn more about Pulumi's Migration Services.

The Promise of the Cloud, Unfulfilled

aaronkao — Fri, 02 Aug 2024 15:57:21 +0000

When your company moved to the cloud, it was supposed to supercharge innovation while slashing costs. Unlimited scalability and agility were within reach. But for many, that potential remains frustratingly untapped.

Instead of accelerating development, managing infrastructure has become a bottleneck. Your DevOps and platform engineering teams are mired in:

Clunky, manual processes - spinning up a simple test environment takes days
Painfully slow rollouts - deploying updates across regions drags on for weeks
Overwhelming complexity - the combinations of cloud services seem infinite

There has to be a better way to truly harness the cloud and turn it into your secret weapon. That's where Pulumi comes in.

Pulumi: The 10x Better Approach to Infrastructure as Code

Pulumi is an open source infrastructure as code (IaC) platform that enables you to provision and manage your cloud with the same languages and tools you use to build applications. This unlocks powerful benefits:

Familiar languages and tools: Use Python, TypeScript, Go, or any language you love. Tap into AI copilots, IDEs, testing frameworks, and CI/CD.

Built for modern architectures: Easily automate and scale microservices, containers, serverless, and more. Apply software engineering to tame the complexity.

AI-powered user experience: Semantic understanding of your cloud combined with powerful language models provides unique insights and automations.

Pulumi takes a fundamentally different approach that delivers 10x better productivity and time-to-value compared to other IaC tools. But what does that mean for you?

Compelling Reasons to Make the Switch

Adopting Pulumi, trusted by hundreds of thousands of developers, can be game-changing:

Accelerate deployments. Provision environments in minutes instead of days. Roll out updates in hours instead of weeks. Deliver value faster.
Increase productivity. Use best-in-class languages and tooling to manage infrastructure. No more fighting clunky domain-specific languages.
Automate at infinite scale. Use software engineering to tame the mind-boggling combinations of cloud services. 10x your infrastructure-to-engineer ratio.
Open source stability. Avoid the uncertainty of tools altering licenses and tying open source to commercial services. Get dependable open source.

Pulumi customers routinely see their productivity double, provisioning times compress 99%, and infrastructure management accelerate 10x. But Pulumi shines for specific use cases too.

Why Pulumi for Internal Developer Platforms?

Building an Internal Developer Platform (IDP) to empower your developers requires:

Developer Control Plane - Curated experiences and self-service interfaces
Integration & Delivery - Automation to provision, test, and deploy all aspects of infrastructure
Monitoring & Logging - Observability to maintain operational control and optimize costs
Security & Identity - Guardrails to enforce policies and control access
Diverse Resources - Support for Kubernetes, serverless, AI/ML, hybrid cloud, and more

Pulumi provides the building blocks for each layer, enabling you to craft an IDP that bridges the gap between your developers and the cloud. Abstract away complexity while maintaining security and compliance.

Why Pulumi for AI Workloads?

Deploying AI applications requires provisioning and gluing together many layers:

Model training infrastructure
Data transformation pipelines
Backend services
Web frontends

Pulumi allows you to define your entire AI stack, from machine learning frameworks down to cloud services, as a simple Python library. Go from local development to cloud production environments with ease.

Why Pulumi Over Alternatives?

There are many infrastructure tools out there, but Pulumi delivers an unmatched combination of productivity and power:

Pulumi vs Clickops

Manually provisioning prod infra = bad idea. No repeatability or consistency. Errors lead to downtime or worse. Automate with IaC.

Pulumi vs Terraform

Both are IaC, but Pulumi enables far greater productivity, scalability, and collaboration:

Use real languages + tools, not proprietary DSLs
Apply software engineering to tame complexity
Empower self-service with IaC building blocks

Pulumi vs AWS CDK

Both use code for IaC, but Pulumi provides:

Multi-cloud with no lock-in, not just AWS
Direct execution, not transpiling to CloudFormation
Versatile testing, including fast in-memory mocking

Pulumi vs IDP Services

3rd party IDPs provide simple portals but lack Pulumi's:

Infrastructure as code foundation
Powerful customization through software engineering
Enterprise-grade secrets management and policy enforcement

Making the Switch is Easy

Adopting Pulumi doesn't have to be daunting. We've helped thousands of customers do it:

Ask questions on our Community Slack
Use our self-serve tools to import existing infrastructure
Get expert help from our team to plan and execute your migration

The cloud's promise is within your grasp. You just need the right tools to seize it.

Pulumi can be the bridge between your teams and the cloud. The key to turning infrastructure from a bottleneck into a launchpad.
So why wait? Join the thousands of companies achieving 10x greater velocity in shipping products.

Header image credit: ESO/M. Kornmesser

Building High-scale AI Backends

aaronkao — Tue, 05 Dec 2023 22:53:44 +0000

What does it take to go from an idea in a notebook to an application handling real-world traffic?

The Pulumi and Pinecone teams worked together to build a reference architecture that teaches you how to scale AI apps in production. We tested it with batches of 10k to 1M records. The reference architecture demonstrates microservices scaling, data processing pipelines, infrastructure segmentation through networking and security groups, and UI and database synchronizations.

Architecture Details

On the infrastructure side, the reference architecture uses a Pinecone index as a vector store, a queue to fan out work, networking and security groups to separate the infrastructure, ECS services for the frontend and backend microservices, and autoscaling configured to expand the worker pool up and down elastically in response to system load.

On the application side, the reference architecture launches a semantic search interface over a Postgres database of product records, leveraging Pinecone’s vector database for queries and instant index updates.

The reference architecture is built with the best practices for how to use AWS and Pinecone. It is designed for production, and it is easily modifiable to fit most use cases.

Try it out: https://github.com/pinecone-io/aws-reference-architecture-pulumi

Forem: Pulumi

CDKTF is deprecated: What's next for your team?

What are the options?

Option 1: Fall back to HCL

Option 2: Migrate to AWS CDK

Option 3: Migrate to Pulumi

What migrating to Pulumi looks like

Conversion and import

Refactoring

An end-to-end example

What's next?

Superintelligence Infrastructure: Managing AI Workloads with General-Purpose Programming Languages

AI Infrastructure Is Outgrowing Static Configuration

Infrastructure Defined with General-Purpose Programming Languages

Designed for Large-Scale AI Environments

AI-Native Operations with Pulumi

A Practical Path to Production AI Infrastructure

What AWS’s 2025 AI and Cloud Updates Mean for Engineers Building Large Systems

Nova Forge and the new model training workflow

Trainium 3 and the next stage of high-throughput compute

AgentCore’s move toward structured automation

A shift in what cloud engineering workflows look like

Read the full analysis

Zero to Production with AI Agentic Workflows – Free Hands-On Course

Meet Neo: Your Newest Platform Engineer

Backstage vs Pulumi IDP: Which One Is the Platform Engineering Winner?

Introduction: Why the IDP Approach Matters

Key Differences at a Glance

Backstage (Portal-First):

Pulumi IDP (Infrastructure-First):

Why Infrastructure-First Matters for IDPs

The Hidden Costs of Portal-First Platforms 💸

Real Developer Experience:

Community Insights 🗣️

Choosing the Right IDP for Your Organization

Select Backstage if you:

Select Pulumi IDP if you:

The Architecture That Scales

Real-World Implementation Examples

Conclusion: It's About Your Foundation

💬 Discussion Time!

🚀 Pulumi AWS Provider 7.0: Multi-Region AWS IaC & IAM Role Chaining

What’s New Highlights

Why It Matters

Kubernetes Best Practices I Wish I Had Known Before

The Cloud Container Iceberg - At least 105 ways to run a container in the cloud

Beyond the Surface: The Hidden Depths of Container Hosting

How Deep Does the Iceberg Go?

Infrastructure as Code: Comparing Pulumi and HCL Approaches

Real-World Consequences

Tapping Into Millions of Developers

Pulumi Leverages Rich Ecosystems of General-Purpose Languages

Pulumi Embraces Open-Source Languages and Principles

Developers Onboard Quicker with Pulumi

Enhancing Team Collaboration With Pulumi

Reducing Redundancy and Copy-Paste Challenges

Reduce Need for Custom Tooling

Future-Proofing Your Infrastructure

Making Infrastructure Code Simpler

HCL in the Wild: Practical Comparisons With Pulumi

for_each Loop With Conditional Expression

for_each Loop With Multi-Level Nested Dynamic Blocks

Execute Code Programming Code With null_resource, local_file, and provisioners

If Statements Constructed With count

HCL Function Frenzy

Pitfalls of HCL's try Function

In Conclusion

The Promise of the Cloud, Unfulfilled

Pulumi: The 10x Better Approach to Infrastructure as Code

Compelling Reasons to Make the Switch

Why Pulumi for Internal Developer Platforms?

Why Pulumi for AI Workloads?

Why Pulumi Over Alternatives?

Pulumi vs Clickops

Pulumi vs Terraform

Pulumi vs AWS CDK

Pulumi vs IDP Services

Making the Switch is Easy

Building High-scale AI Backends

Architecture Details

`for_each` Loop With Conditional Expression

`for_each` Loop With Multi-Level Nested Dynamic Blocks

Execute Code Programming Code With `null_resource`, `local_file`, and `provisioners`

If Statements Constructed With `count`

Pitfalls of HCL's `try` Function