Forem: Sudarshan Thakur

How I Built a Terraform Plan JSON Parser in Python

Sudarshan Thakur — Sat, 09 May 2026 20:45:05 +0000

Most DevOps engineers have run terraform plan thousands of times. Very few have looked at what comes out when you add the -json flag.

I spent a couple of weeks inside that JSON output while building tfdrift, an open-source drift detection tool. What I found was a surprisingly well-structured format — and a handful of gotchas that aren't documented anywhere obvious.

This post is the technical deep-dive I wish I'd had when I started. If you're building any kind of tooling on top of Terraform — drift detection, policy enforcement, cost estimation, change analysis — this is the stuff you'll need to know.

Getting the JSON output

First, the basics. You can't just pipe terraform plan to get JSON. The text output and the JSON output are completely different codepaths. You need to do it in two steps:

# Step 1: Generate a plan file
terraform plan -out=tfplan -input=false -no-color

# Step 2: Convert to JSON
terraform show -json tfplan

The -out flag saves the plan as a binary file (not human-readable), and terraform show -json converts that binary into structured JSON. You can't skip step 1 — there's no terraform plan -json flag that gives you JSON directly to stdout. This tripped me up initially.

In Python, that looks like:

import subprocess
import json

def get_plan_json(workspace_path: str) -> dict:
    # Generate the plan file
    subprocess.run(
        ["terraform", "plan", "-out=tfplan", 
         "-input=false", "-no-color"],
        cwd=workspace_path,
        capture_output=True,
        text=True,
        timeout=600,
    )

    # Convert to JSON
    result = subprocess.run(
        ["terraform", "show", "-json", "tfplan"],
        cwd=workspace_path,
        capture_output=True,
        text=True,
        timeout=120,
    )

    return json.loads(result.stdout)

A few things to notice here. The timeout on plan is 10 minutes — for large workspaces with many resources, plan can take a while because it's refreshing state against the cloud provider APIs. The timeout on show is shorter because it's just reading a local file. And we always set -input=false because we're running non-interactively — without it, Terraform might hang waiting for user input.

The structure of plan JSON

The JSON output has a lot of fields, but for drift detection, you really only care about one: resource_changes. Here's what a simplified version looks like:

{
  "format_version": "1.2",
  "terraform_version": "1.7.0",
  "resource_changes": [
    {
      "address": "aws_security_group.api_sg",
      "mode": "managed",
      "type": "aws_security_group",
      "name": "api_sg",
      "provider_name": "registry.terraform.io/hashicorp/aws",
      "change": {
        "actions": ["update"],
        "before": {
          "ingress": [
            {
              "from_port": 443,
              "to_port": 443,
              "cidr_blocks": ["10.0.0.0/8"]
            }
          ],
          "tags": {"Name": "api-sg"}
        },
        "after": {
          "ingress": [
            {
              "from_port": 443,
              "to_port": 443,
              "cidr_blocks": ["10.0.0.0/8"]
            },
            {
              "from_port": 22,
              "to_port": 22,
              "cidr_blocks": ["0.0.0.0/0"]
            }
          ],
          "tags": {"Name": "api-sg"}
        },
        "after_sensitive": {},
        "after_unknown": {}
      }
    }
  ]
}

Each entry in resource_changes represents one resource that Terraform wants to modify. The change object gives you a before snapshot (current state in the cloud) and an after snapshot (what Terraform wants it to look like). The actions array tells you what kind of change it is.

Parsing the actions field

The actions field is where it gets interesting. It's an array, not a string, and the possible values are:

# Single actions
["no-op"]     # Nothing changed
["create"]    # Resource exists in code but not in cloud
["delete"]    # Resource exists in cloud but not in code
["update"]    # Resource exists in both but attributes differ
["read"]      # Data source refresh

# Compound actions (two-element arrays)
["delete", "create"]  # Must destroy and recreate
["create", "delete"]  # Create new before destroying old

That last distinction matters. ["delete", "create"] means Terraform will delete the resource first, then create the replacement — there will be downtime. ["create", "delete"] means Terraform creates the new one first, then removes the old one — no downtime. The create_before_destroy lifecycle setting in your Terraform code controls which one you get.

Here's how I mapped these to a simpler enum:

from enum import Enum

class ChangeAction(str, Enum):
    CREATE = "create"
    UPDATE = "update"
    DELETE = "delete"
    REPLACE = "replace"
    NO_OP = "no-op"

ACTION_MAP = {
    "create": ChangeAction.CREATE,
    "update": ChangeAction.UPDATE,
    "delete": ChangeAction.DELETE,
    "create,delete": ChangeAction.REPLACE,
    "delete,create": ChangeAction.REPLACE,
    "no-op": ChangeAction.NO_OP,
    "read": ChangeAction.NO_OP,
}

def parse_action(actions: list[str]) -> ChangeAction:
    key = ",".join(actions)
    return ACTION_MAP.get(key, ChangeAction.UPDATE)

I collapsed both compound actions into a single REPLACE because for drift detection, the direction doesn't matter — either way, the resource is being recreated.

Extracting attribute-level diffs

This is the core of what makes severity classification possible. The before and after objects are full resource representations, so you can diff them attribute by attribute:

def extract_changes(before: dict, after: dict) -> list[dict]:
    """Find which specific attributes changed."""
    changes = []
    all_keys = set(before.keys()) | set(after.keys())

    for key in sorted(all_keys):
        old_val = before.get(key)
        new_val = after.get(key)

        if old_val != new_val:
            changes.append({
                "attribute": key,
                "old_value": old_val,
                "new_value": new_val,
            })

    return changes

Simple, right? Mostly. There are a few gotchas.

Gotcha 1: Sensitive values

Some attributes are marked as sensitive — passwords, API keys, things you don't want showing up in logs. Terraform handles this through the after_sensitive field:

{
  "after": {
    "password": "newsecret123",
    "username": "admin"
  },
  "after_sensitive": {
    "password": true,
    "username": false
  }
}

You need to check after_sensitive before logging or displaying values:

def extract_changes(before, after, after_sensitive=None):
    changes = []
    after_sensitive = after_sensitive or {}
    all_keys = set(before.keys()) | set(after.keys())

    for key in sorted(all_keys):
        old_val = before.get(key)
        new_val = after.get(key)

        if old_val != new_val:
            is_sensitive = (
                isinstance(after_sensitive, dict) 
                and after_sensitive.get(key, False)
            )
            changes.append({
                "attribute": key,
                "old_value": "(sensitive)" if is_sensitive else old_val,
                "new_value": "(sensitive)" if is_sensitive else new_val,
            })

    return changes

I learned this one the hard way when a database password showed up in a Slack notification during testing. Don't be me.

Gotcha 2: Nested objects and lists

Some attributes are deeply nested. A security group's ingress attribute isn't a simple value — it's a list of rule objects, each with from_port, to_port, protocol, and cidr_blocks. When you compare before["ingress"] to after["ingress"], Python's != operator works because it does deep comparison on lists and dicts. But the diff you get is "ingress changed" — you lose the granularity of which rule was added or removed.

For tfdrift, I made the decision to report at the attribute level, not the nested value level. So you get "ingress changed" rather than "a new rule was added allowing port 22 from 0.0.0.0/0." This is a tradeoff — you lose some detail, but you gain simplicity and the ability to do pattern-based severity matching on the attribute name.

If you need deeper diffing, you'd want something like deepdiff:

from deepdiff import DeepDiff

diff = DeepDiff(before["ingress"], after["ingress"])

But for severity classification, knowing that ingress changed is enough to classify it as Critical.

Gotcha 3: Null vs. absent

Terraform's JSON output isn't always consistent about null values. Sometimes a missing attribute is null, sometimes it's absent from the object entirely. Your comparison code needs to handle both:

old_val = before.get(key)  # Returns None if key doesn't exist
new_val = after.get(key)

# Both None/absent means no change
if old_val is None and new_val is None:
    continue

Gotcha 4: Data sources show up in resource_changes

This surprised me. Data sources (data.aws_ami.latest, data.aws_vpc.main, etc.) appear in the resource_changes array with a mode of "data". They're not real resources — they're just lookups. You need to filter them out:

for change in plan_json.get("resource_changes", []):
    if change.get("mode") == "data":
        continue  # Skip data sources

If you don't, you'll get false drift alerts every time a data source refreshes with slightly different results.

Gotcha 5: Module addresses

Resources inside modules have a module_address field:

{
  "address": "aws_instance.web",
  "module_address": "module.compute.module.web_tier",
  "type": "aws_instance",
  "name": "web"
}

The address field alone is aws_instance.web, which isn't unique if you have the same resource name in multiple modules. The full address is module.compute.module.web_tier.aws_instance.web. In tfdrift I handle this like:

@property
def full_address(self) -> str:
    if self.module:
        return f"{self.module}.{self.address}"
    return self.address

Handling variable files

Here's something I discovered only when testing against real infrastructure: a huge number of workspaces require variable files to plan successfully. Without the right .tfvars file, terraform plan fails immediately with "No value for required variable."

My first scan of a real environment found 150+ workspaces. About 80% of the subdirectory workspaces failed because they expected variables to be passed in. The main workspace had a terraform.tfvars file and worked fine.

The fix was auto-detection:

def find_var_files(workspace_path: Path) -> list[str]:
    """Find .tfvars files in a workspace directory."""
    var_files = []

    # terraform.tfvars is the default — always include if present
    default = workspace_path / "terraform.tfvars"
    if default.exists():
        var_files.append(str(default))

    # Also check for other .tfvars files
    for f in sorted(workspace_path.glob("*.tfvars")):
        if f.name == "terraform.tfvars":
            continue  # Already added
        if f.name.endswith(".auto.tfvars"):
            continue  # Terraform loads these automatically
        var_files.append(str(f))

    return var_files

Then when building the plan command:

plan_cmd = ["terraform", "plan", "-out=tfplan", 
            "-input=false", "-no-color"]

for vf in find_var_files(workspace_path):
    plan_cmd.extend(["-var-file", vf])

This single change took my scan success rate from about 20% to 85% on that same codebase.

The -detailed-exitcode flag

Here's a useful trick: terraform plan supports a -detailed-exitcode flag that changes the exit code behavior:

Exit code 0: Plan succeeded, no changes
Exit code 1: Error
Exit code 2: Plan succeeded, changes detected

This means you can quickly check for drift without parsing any JSON at all:

result = subprocess.run(
    ["terraform", "plan", "-detailed-exitcode",
     "-input=false", "-no-color", "-out=tfplan"],
    cwd=workspace_path,
    capture_output=True,
    text=True,
)

if result.returncode == 0:
    # No drift — skip JSON parsing entirely
    return []
elif result.returncode == 1:
    # Error — log and move on
    return handle_error(result.stderr)
elif result.returncode == 2:
    # Drift detected — now parse the JSON
    return parse_plan_json(workspace_path)

This is a nice optimization because it means you only run terraform show -json (which can be slow for large state files) when there's actually something to look at.

Putting it all together

Here's the complete flow in simplified form:

def scan_workspace(workspace_path: str) -> list[DriftedResource]:
    """Scan a single workspace for drift."""

    # 1. Run terraform init
    subprocess.run(
        ["terraform", "init", "-backend=true", 
         "-input=false", "-no-color"],
        cwd=workspace_path,
        capture_output=True,
    )

    # 2. Build plan command with var files
    plan_cmd = [
        "terraform", "plan", "-detailed-exitcode",
        "-input=false", "-no-color", "-out=tfplan",
    ]
    for vf in find_var_files(Path(workspace_path)):
        plan_cmd.extend(["-var-file", vf])

    # 3. Run plan
    result = subprocess.run(
        plan_cmd, cwd=workspace_path,
        capture_output=True, text=True,
    )

    if result.returncode != 2:
        return []  # No drift or error

    # 4. Get JSON output
    show = subprocess.run(
        ["terraform", "show", "-json", "tfplan"],
        cwd=workspace_path,
        capture_output=True, text=True,
    )
    plan_json = json.loads(show.stdout)

    # 5. Parse changes
    drifted = []
    for change in plan_json.get("resource_changes", []):
        if change.get("mode") == "data":
            continue

        actions = change["change"]["actions"]
        action = parse_action(actions)

        if action == ChangeAction.NO_OP:
            continue

        before = change["change"].get("before") or {}
        after = change["change"].get("after") or {}
        sensitive = change["change"].get("after_sensitive") or {}

        attr_changes = extract_changes(before, after, sensitive)

        drifted.append(DriftedResource(
            address=change["address"],
            resource_type=change["type"],
            action=action,
            changes=attr_changes,
            module=change.get("module_address"),
        ))

    # 6. Clean up plan file
    Path(workspace_path, "tfplan").unlink(missing_ok=True)

    return drifted

That's roughly 60 lines to go from a Terraform workspace directory to a structured list of drifted resources with attribute-level detail. The actual tfdrift implementation adds severity classification, ignore rules, error handling, and reporting on top of this, but the core parsing logic is what you see here.

Lessons learned

Shell out, don't replicate. I briefly considered using the Terraform provider SDKs directly (like boto3 for AWS) to compare state. That would've been months of work and broken with every provider update. Shelling out to terraform plan -json gives you free compatibility with every provider, every backend, and every Terraform version since 0.12. The tradeoff is requiring a Terraform installation, but anyone who needs drift detection already has one.

The JSON format is stable but undocumented. HashiCorp has documentation for the JSON output format, but it's sparse. The format has been stable since Terraform 0.12 (2019), and I haven't encountered breaking changes across versions. But it's not formally versioned, so in theory a future release could change things. Supporting OpenTofu via a --binary flag is a hedge against that.

Test against real infrastructure early. I built the parser against synthetic test data first and thought it was done. Then I pointed it at a real codebase with 150+ workspaces and discovered the variable file problem, the module address problem, and several edge cases with nested attributes. Synthetic tests are necessary but not sufficient.

Clean up after yourself. The tfplan binary file that terraform plan -out creates will accumulate if you don't delete it. In a scan across 150 workspaces, that's 150 binary files sitting in various directories. Always clean up in a finally block.

Try it yourself

All of this code is in the tfdrift source:

Plan execution: src/tfdrift/detectors/drift.py
Data models: src/tfdrift/models.py
Severity classification: src/tfdrift/severity.py

Or just install it:

pip install tfdrift
tfdrift scan --path ./your-terraform-dir --verbose

If you're building your own Terraform tooling and run into edge cases I didn't cover, I'd love to hear about them. Open an issue or drop a comment.

This is part 3 of a series on infrastructure drift detection. Part 1: I Built a Free Terraform Drift Detector — Here's Why. Part 2: Why Severity Classification Changes Everything About Drift Detection. Coming next: Setting Up Continuous Drift Monitoring With GitHub Actions and Slack.

How I Built a Terraform Plan JSON Parser in Python

Sudarshan Thakur — Sat, 09 May 2026 20:45:05 +0000

Most DevOps engineers have run terraform plan thousands of times. Very few have looked at what comes out when you add the -json flag.

Getting the JSON output

First, the basics. You can't just pipe terraform plan to get JSON. The text output and the JSON output are completely different codepaths. You need to do it in two steps:

# Step 1: Generate a plan file
terraform plan -out=tfplan -input=false -no-color

# Step 2: Convert to JSON
terraform show -json tfplan

In Python, that looks like:

import subprocess
import json

def get_plan_json(workspace_path: str) -> dict:
    # Generate the plan file
    subprocess.run(
        ["terraform", "plan", "-out=tfplan", 
         "-input=false", "-no-color"],
        cwd=workspace_path,
        capture_output=True,
        text=True,
        timeout=600,
    )

    # Convert to JSON
    result = subprocess.run(
        ["terraform", "show", "-json", "tfplan"],
        cwd=workspace_path,
        capture_output=True,
        text=True,
        timeout=120,
    )

    return json.loads(result.stdout)

The structure of plan JSON

The JSON output has a lot of fields, but for drift detection, you really only care about one: resource_changes. Here's what a simplified version looks like:

{
  "format_version": "1.2",
  "terraform_version": "1.7.0",
  "resource_changes": [
    {
      "address": "aws_security_group.api_sg",
      "mode": "managed",
      "type": "aws_security_group",
      "name": "api_sg",
      "provider_name": "registry.terraform.io/hashicorp/aws",
      "change": {
        "actions": ["update"],
        "before": {
          "ingress": [
            {
              "from_port": 443,
              "to_port": 443,
              "cidr_blocks": ["10.0.0.0/8"]
            }
          ],
          "tags": {"Name": "api-sg"}
        },
        "after": {
          "ingress": [
            {
              "from_port": 443,
              "to_port": 443,
              "cidr_blocks": ["10.0.0.0/8"]
            },
            {
              "from_port": 22,
              "to_port": 22,
              "cidr_blocks": ["0.0.0.0/0"]
            }
          ],
          "tags": {"Name": "api-sg"}
        },
        "after_sensitive": {},
        "after_unknown": {}
      }
    }
  ]
}

Parsing the actions field

The actions field is where it gets interesting. It's an array, not a string, and the possible values are:

# Single actions
["no-op"]     # Nothing changed
["create"]    # Resource exists in code but not in cloud
["delete"]    # Resource exists in cloud but not in code
["update"]    # Resource exists in both but attributes differ
["read"]      # Data source refresh

# Compound actions (two-element arrays)
["delete", "create"]  # Must destroy and recreate
["create", "delete"]  # Create new before destroying old

Here's how I mapped these to a simpler enum:

from enum import Enum

class ChangeAction(str, Enum):
    CREATE = "create"
    UPDATE = "update"
    DELETE = "delete"
    REPLACE = "replace"
    NO_OP = "no-op"

ACTION_MAP = {
    "create": ChangeAction.CREATE,
    "update": ChangeAction.UPDATE,
    "delete": ChangeAction.DELETE,
    "create,delete": ChangeAction.REPLACE,
    "delete,create": ChangeAction.REPLACE,
    "no-op": ChangeAction.NO_OP,
    "read": ChangeAction.NO_OP,
}

def parse_action(actions: list[str]) -> ChangeAction:
    key = ",".join(actions)
    return ACTION_MAP.get(key, ChangeAction.UPDATE)

I collapsed both compound actions into a single REPLACE because for drift detection, the direction doesn't matter — either way, the resource is being recreated.

Extracting attribute-level diffs

This is the core of what makes severity classification possible. The before and after objects are full resource representations, so you can diff them attribute by attribute:

def extract_changes(before: dict, after: dict) -> list[dict]:
    """Find which specific attributes changed."""
    changes = []
    all_keys = set(before.keys()) | set(after.keys())

    for key in sorted(all_keys):
        old_val = before.get(key)
        new_val = after.get(key)

        if old_val != new_val:
            changes.append({
                "attribute": key,
                "old_value": old_val,
                "new_value": new_val,
            })

    return changes

Simple, right? Mostly. There are a few gotchas.

Gotcha 1: Sensitive values

Some attributes are marked as sensitive — passwords, API keys, things you don't want showing up in logs. Terraform handles this through the after_sensitive field:

{
  "after": {
    "password": "newsecret123",
    "username": "admin"
  },
  "after_sensitive": {
    "password": true,
    "username": false
  }
}

You need to check after_sensitive before logging or displaying values:

def extract_changes(before, after, after_sensitive=None):
    changes = []
    after_sensitive = after_sensitive or {}
    all_keys = set(before.keys()) | set(after.keys())

    for key in sorted(all_keys):
        old_val = before.get(key)
        new_val = after.get(key)

        if old_val != new_val:
            is_sensitive = (
                isinstance(after_sensitive, dict) 
                and after_sensitive.get(key, False)
            )
            changes.append({
                "attribute": key,
                "old_value": "(sensitive)" if is_sensitive else old_val,
                "new_value": "(sensitive)" if is_sensitive else new_val,
            })

    return changes

I learned this one the hard way when a database password showed up in a Slack notification during testing. Don't be me.

Gotcha 2: Nested objects and lists

If you need deeper diffing, you'd want something like deepdiff:

from deepdiff import DeepDiff

diff = DeepDiff(before["ingress"], after["ingress"])

But for severity classification, knowing that ingress changed is enough to classify it as Critical.

Gotcha 3: Null vs. absent

Terraform's JSON output isn't always consistent about null values. Sometimes a missing attribute is null, sometimes it's absent from the object entirely. Your comparison code needs to handle both:

old_val = before.get(key)  # Returns None if key doesn't exist
new_val = after.get(key)

# Both None/absent means no change
if old_val is None and new_val is None:
    continue

Gotcha 4: Data sources show up in resource_changes

for change in plan_json.get("resource_changes", []):
    if change.get("mode") == "data":
        continue  # Skip data sources

If you don't, you'll get false drift alerts every time a data source refreshes with slightly different results.

Gotcha 5: Module addresses

Resources inside modules have a module_address field:

{
  "address": "aws_instance.web",
  "module_address": "module.compute.module.web_tier",
  "type": "aws_instance",
  "name": "web"
}

@property
def full_address(self) -> str:
    if self.module:
        return f"{self.module}.{self.address}"
    return self.address

Handling variable files

The fix was auto-detection:

def find_var_files(workspace_path: Path) -> list[str]:
    """Find .tfvars files in a workspace directory."""
    var_files = []

    # terraform.tfvars is the default — always include if present
    default = workspace_path / "terraform.tfvars"
    if default.exists():
        var_files.append(str(default))

    # Also check for other .tfvars files
    for f in sorted(workspace_path.glob("*.tfvars")):
        if f.name == "terraform.tfvars":
            continue  # Already added
        if f.name.endswith(".auto.tfvars"):
            continue  # Terraform loads these automatically
        var_files.append(str(f))

    return var_files

Then when building the plan command:

plan_cmd = ["terraform", "plan", "-out=tfplan", 
            "-input=false", "-no-color"]

for vf in find_var_files(workspace_path):
    plan_cmd.extend(["-var-file", vf])

This single change took my scan success rate from about 20% to 85% on that same codebase.

The -detailed-exitcode flag

Here's a useful trick: terraform plan supports a -detailed-exitcode flag that changes the exit code behavior:

Exit code 0: Plan succeeded, no changes
Exit code 1: Error
Exit code 2: Plan succeeded, changes detected

This means you can quickly check for drift without parsing any JSON at all:

result = subprocess.run(
    ["terraform", "plan", "-detailed-exitcode",
     "-input=false", "-no-color", "-out=tfplan"],
    cwd=workspace_path,
    capture_output=True,
    text=True,
)

if result.returncode == 0:
    # No drift — skip JSON parsing entirely
    return []
elif result.returncode == 1:
    # Error — log and move on
    return handle_error(result.stderr)
elif result.returncode == 2:
    # Drift detected — now parse the JSON
    return parse_plan_json(workspace_path)

This is a nice optimization because it means you only run terraform show -json (which can be slow for large state files) when there's actually something to look at.

Putting it all together

Here's the complete flow in simplified form:

def scan_workspace(workspace_path: str) -> list[DriftedResource]:
    """Scan a single workspace for drift."""

    # 1. Run terraform init
    subprocess.run(
        ["terraform", "init", "-backend=true", 
         "-input=false", "-no-color"],
        cwd=workspace_path,
        capture_output=True,
    )

    # 2. Build plan command with var files
    plan_cmd = [
        "terraform", "plan", "-detailed-exitcode",
        "-input=false", "-no-color", "-out=tfplan",
    ]
    for vf in find_var_files(Path(workspace_path)):
        plan_cmd.extend(["-var-file", vf])

    # 3. Run plan
    result = subprocess.run(
        plan_cmd, cwd=workspace_path,
        capture_output=True, text=True,
    )

    if result.returncode != 2:
        return []  # No drift or error

    # 4. Get JSON output
    show = subprocess.run(
        ["terraform", "show", "-json", "tfplan"],
        cwd=workspace_path,
        capture_output=True, text=True,
    )
    plan_json = json.loads(show.stdout)

    # 5. Parse changes
    drifted = []
    for change in plan_json.get("resource_changes", []):
        if change.get("mode") == "data":
            continue

        actions = change["change"]["actions"]
        action = parse_action(actions)

        if action == ChangeAction.NO_OP:
            continue

        before = change["change"].get("before") or {}
        after = change["change"].get("after") or {}
        sensitive = change["change"].get("after_sensitive") or {}

        attr_changes = extract_changes(before, after, sensitive)

        drifted.append(DriftedResource(
            address=change["address"],
            resource_type=change["type"],
            action=action,
            changes=attr_changes,
            module=change.get("module_address"),
        ))

    # 6. Clean up plan file
    Path(workspace_path, "tfplan").unlink(missing_ok=True)

    return drifted

Lessons learned

Try it yourself

All of this code is in the tfdrift source:

Plan execution: src/tfdrift/detectors/drift.py
Data models: src/tfdrift/models.py
Severity classification: src/tfdrift/severity.py

Or just install it:

pip install tfdrift
tfdrift scan --path ./your-terraform-dir --verbose

If you're building your own Terraform tooling and run into edge cases I didn't cover, I'd love to hear about them. Open an issue or drop a comment.

Why Severity Classification Changes Everything About Drift Detection

Sudarshan Thakur — Sat, 02 May 2026 17:53:38 +0000

Here's a scenario every DevOps engineer has lived through at least once.

You set up drift detection for your Terraform infrastructure. Maybe it's a cron job running terraform plan, maybe it's a fancier tool. Either way, it works. It finds drift. It sends alerts.

The problem is it finds everything. Tag changes. Description updates. Auto-scaling group adjustments. Security group modifications. Instance type changes. All of it lands in the same Slack channel with the same urgency.

For the first week, your team looks at every alert. By week three, people start skimming. By month two, that Slack channel is muted. I know this because it happened to us.

And buried in that noise? A security group change that opened SSH to the internet. Nobody caught it for eleven days.

The core problem isn't detection. It's prioritization.

terraform plan is perfectly capable of detecting drift. It's been doing it since 2014. The detection part is solved. What isn't solved is the question that comes after detection: so what?

When plan tells you that 47 resources have changed across your infrastructure, the natural human response is "I don't have time for this." And honestly, that's rational. Most of those 47 changes probably don't matter. Tags get updated by cost allocation scripts. Auto-scaling groups adjust desired capacity constantly — that's literally their job. Someone updated a description field three weeks ago.

But somewhere in those 47 changes, there might be an IAM policy that was broadened to allow s3:* on all buckets. Or a KMS key policy that added an unknown account. Or a database that got its publicly_accessible flag flipped to true.

The problem is you can't tell which is which without reading through every single diff. Every. Single. Time.

Research on alert fatigue in operations backs this up. When teams receive more than about 50 alerts per day, response quality drops measurably. Critical alert response times degrade by up to 40%. The monitoring industry learned this lesson a decade ago with tools like PagerDuty adding severity levels and intelligent routing. Infrastructure drift detection hasn't caught up.

Not all drift is created equal

Once you actually think about it, drift falls into pretty natural categories based on security and operational impact.

Some drift is a fire. Someone modified a security group's ingress rules. Someone changed an IAM policy. Someone disabled encryption on a storage bucket. Someone opened up public access on a database. These changes directly affect your security perimeter. If you miss them, you might end up in a breach disclosure filing. I'd call these Critical.

Some drift is a problem. Someone changed an instance type in production — your capacity planning just went out the window. Someone modified a database engine version — your upgrade runbooks are now wrong. Someone changed a Lambda runtime — hope nothing depends on that specific Python version. These won't get you breached, but they can cause outages. That's High.

Some drift is worth knowing about. Most attribute changes that don't fall into the above categories. A CloudFront distribution's cache behavior was modified. A load balancer's idle timeout was changed. Not urgent, but you should probably investigate when you get a chance. That's Medium.

Some drift is noise. Tags. Descriptions. Labels. Metadata that external systems update constantly. Flagging these as drift is technically correct and operationally useless. That's Low.

The moment you apply these categories, your 47 alerts become 2 Critical, 3 High, 8 Medium, and 34 Low. Suddenly you know exactly where to look.

How we implemented this

When I built tfdrift, severity classification was the feature I cared about most. The technical approach is straightforward but the design decisions were interesting.

Pattern matching on resource type + attribute. Each severity rule is a pattern like aws_security_group.*.ingress → Critical. The * matches any resource name. When terraform plan reports that aws_security_group.api_gateway.ingress changed, it matches the Critical pattern.

Why resource type + attribute and not just resource type? Because a security group changing its tags is Low, but a security group changing its ingress rules is Critical. The attribute matters as much as the resource type.

Maximum severity wins. When a single resource has multiple changed attributes at different severity levels — say, both a tag change and an ingress change — we classify it at the highest level. A security group with a Low tag change and a Critical ingress change gets reported as Critical. You don't want someone to dismiss a Critical change because it also had some Low-severity noise attached.

60+ built-in rules. We ship default patterns for AWS (26 rules), Azure (13 rules), and GCP (10 rules), plus cloud-agnostic Low rules for tags, labels, and descriptions. These are opinionated defaults based on what we think matters most from a security perspective. You can override everything in a YAML config file.

Here's what the default Critical rules look like for AWS:

aws_security_group.*.ingress        # network access
aws_security_group.*.egress         # network access
aws_iam_policy.*.policy             # identity & access
aws_iam_role.*.assume_role_policy   # identity & access
aws_s3_bucket_public_access_block.* # data exposure
aws_s3_bucket_policy.*.policy       # data exposure
aws_kms_key.*.key_policy            # encryption
aws_network_acl_rule.*              # network access

And for Azure:

azurerm_network_security_group.*.security_rule  # network access
azurerm_role_assignment.*                        # identity & access
azurerm_key_vault_access_policy.*                # secrets management
azurerm_storage_account.*.network_rules          # data exposure

And GCP:

google_compute_firewall.*.allow         # network access
google_compute_firewall.*.source_ranges # network access
google_project_iam_binding.*            # identity & access
google_project_iam_member.*             # identity & access

Ignore rules for expected drift. Beyond severity, we also support a .tfdriftignore file for drift you never want to see. The classic example:

aws_autoscaling_group.*.desired_capacity
aws_ecs_service.*.desired_count

Auto-scaling groups change desired_capacity every few minutes. That's not drift — that's the system working correctly. Filtering this out before severity classification reduces noise even further.

The numbers

I tested this approach against a sandbox AWS environment with 150+ Terraform workspaces — EC2, IAM, S3, RDS, VPC, the usual mix. I introduced drift through the AWS console and CLI to simulate the kind of changes that happen in real environments: security group tweaks, tag updates, instance type changes, IAM policy modifications.

The results:

Approach	Alert volume	Security changes caught
Binary detection (terraform plan)	100%	100% (buried in noise)
Severity ≥ High only	27%	94%
Severity ≥ High + ignore rules	19%	97%

Filtering to High and Critical severity cut the alert volume by 73% while still catching 94% of the changes that actually mattered from a security perspective. Adding ignore rules brought noise down to just 19% of the original volume at 97% precision.

The 6% of security-relevant changes that got missed at the High threshold were Medium-severity changes that a human might consider security-adjacent — things like Lambda runtime version changes. Whether those matter depends on your risk tolerance, and you can always adjust the severity rules to promote specific attributes.

The .tfdrift.yml as a living document

Here's something I didn't fully appreciate until I started getting feedback from other engineers: the severity configuration file becomes a living document of your team's operational values.

When you configure which changes are Critical and which are Low, you're encoding institutional knowledge about what matters to your organization. A fintech company might classify aws_rds_instance.*.storage_encrypted as Critical. A dev agency might not care about it at all.

The .tfdrift.yml file captures these decisions in version-controlled, reviewable code. When a new engineer joins the team and asks "do we care about CloudFront distribution changes?", the answer is in the config file. When an incident happens because of undetected drift, the post-mortem action item is "add this resource type to our Critical patterns" — a one-line YAML change.

severity:
  critical:
    - aws_security_group.*.ingress
    - aws_iam_policy.*.policy
    # Added after the March 15 incident — ticket INC-4521
    - aws_cloudfront_distribution.*.origin
  high:
    - aws_instance.*.instance_type
    - aws_rds_instance.*.publicly_accessible

That comment in the config file? That's tribal knowledge being captured instead of lost.

What severity classification doesn't solve

I want to be honest about the limitations.

It can't see values. The classification is based on which attribute changed, not what it changed to. A security group rule opening port 443 to a known CIDR range is very different from one opening port 22 to 0.0.0.0/0. Both get classified as Critical because the attribute is ingress. Value-aware classification is on the roadmap, but it adds significant complexity.

It doesn't understand context. The same change might be Critical in production and Low in dev. Right now you handle this by maintaining separate config files per environment. Native environment-aware severity is something I want to build.

It's still just detection. Even with severity classification, the loop is: drift happens → tool detects it → alert goes out → human decides what to do. One of the more interesting pieces of feedback I've gotten is about moving from detection to governance — where Critical drift can automatically block deployments or trigger remediation. That's the next frontier, but it's also the scariest one.

Try it

If this resonates with your experience managing Terraform infrastructure, give tfdrift a try:

pip install tfdrift
tfdrift scan --path ./your-terraform-dir

The severity rules are opinionated out of the box, and you can customize everything. If you have thoughts on what the default rules should cover — especially for Azure and GCP where our coverage is thinner — I'd genuinely love to hear about it.

GitHub: github.com/sudarshan8417/tfdrift

This is part of a series on infrastructure drift detection. Previously: I Built a Free Terraform Drift Detector — Here's Why. Next up: How I Built a Terraform Plan JSON Parser in Python.

I Built tfdrift Free Terraform Drift Detection With Severity Alerts

Sudarshan Thakur — Tue, 21 Apr 2026 19:31:10 +0000

I Built a Free Terraform Drift Detector — Here's Why

If you manage Terraform infrastructure, you've probably experienced this: someone tweaks a security group in the AWS console "just for testing," forgets about it, and three months later your terraform apply blows up — or worse, that change silently creates a security hole that nobody catches.

This is infrastructure drift. And there's no good free tool to detect it properly.

So I built tfdrift — a free, open-source CLI that detects Terraform drift, classifies it by severity, and alerts your team.

The problem with "just run terraform plan"

Yes, terraform plan detects drift. But in practice, it falls short for teams managing real infrastructure:

No severity awareness. Someone changed a tag? Same alert as someone opening port 22 to the world. terraform plan treats all changes equally — but they're not. An IAM policy change is a security incident. A tag change is noise.

No multi-workspace scanning. If you have 20 Terraform workspaces across environments, you need to cd into each one and run plan manually. Nobody does this consistently.

No notifications. You only discover drift when you happen to run plan. By then, the damage may already be done.

No ignore rules. Auto-scaling groups constantly change desired_capacity. ECS services change desired_count. These are expected — but plan flags them every time, creating alert fatigue.

What tfdrift does

pip install tfdrift
tfdrift scan --path ./infrastructure

It recursively discovers all Terraform workspaces, runs terraform plan -json on each, parses the output, and gives you a structured report with severity classification:

CRITICAL — Security group ingress/egress changes, IAM policy modifications, S3 public access changes
HIGH — Instance type changes, RDS public accessibility, encryption config changes
MEDIUM — Most other attribute changes
LOW — Tags, descriptions

Here's what the output looks like:

⚠️  Drift detected: 7 resource(s) across 2/4 workspace(s)

🔴 critical: 2  🟠 high: 2  🟡 medium: 2  🔵 low: 1

📂 infrastructure/production (5 drifted, 3.2s)
┌──────────┬───────────────────────────────────┬────────┬─────────────────────┐
│ Severity │ Resource                          │ Action │ Changed attributes  │
├──────────┼───────────────────────────────────┼────────┼─────────────────────┤
│ CRITICAL │ aws_security_group.api_sg         │ update │ ingress             │
│ CRITICAL │ aws_iam_role_policy.lambda_exec   │ update │ policy              │
│ HIGH     │ aws_instance.web_server           │ update │ instance_type, ami  │
│ HIGH     │ aws_rds_instance.primary          │ update │ publicly_accessible │
│ LOW      │ aws_s3_bucket.assets              │ update │ tags                │
└──────────┴───────────────────────────────────┴────────┴─────────────────────┘

Now you immediately know what matters.

I tested it against real AWS infrastructure

To validate the tool beyond unit tests, I set up a sample AWS environment with EC2 instances managed by Terraform. I ran tfdrift against it:

tfdrift scan --path ./dev-terraform/development/ec2 --verbose

Within 4 seconds, it detected a drifted EC2 instance — an aws_instance resource that had been modified outside of Terraform. The tool correctly classified it as HIGH severity because instance_type was among the changed attributes:

⚠️  Drift detected: 1 resource(s) across 1/4 workspace(s)

🟠 high: 1

📂 development/ec2/dev-machines (1 drifted, 4.2s)
┌──────────┬──────────────────────┬────────┬────────────────────────────────────┐
│ Severity │ Resource             │ Action │ Changed attributes                 │
├──────────┼──────────────────────┼────────┼────────────────────────────────────┤
│ HIGH     │ aws_instance.this[0] │ create │ instance_type, tags, monitoring,  │
│          │                      │        │ metadata_options, volume_tags      │
└──────────┴──────────────────────┴────────┴────────────────────────────────────┘

That drift had been sitting there undetected. A simple terraform plan would have found it too — but nobody was running plan against that workspace regularly. With tfdrift's watch mode, this could have been caught automatically and sent to Slack within minutes.

Key features

Watch mode — continuous monitoring with Slack alerts:

tfdrift watch --interval 30m --slack-webhook https://hooks.slack.com/services/XXX

Auto-remediation — with safety guards that block destructive changes on critical resources:

tfdrift scan --auto-fix --confirm --env dev

Ignore rules — a .tfdriftignore file for expected drift:

aws_autoscaling_group.*.desired_capacity
aws_ecs_service.*.desired_count
*.tags.LastModified

CI/CD integration — exit codes designed for pipelines (0 = clean, 1 = drift, 2 = error, 3 = remediated), plus a ready-made GitHub Actions workflow.

JSON and Markdown output — for programmatic use or generating reports:

tfdrift scan --format json --output drift-report.json
tfdrift scan --format markdown --output drift-report.md

How it compares

Feature	tfdrift	terraform plan	Terraform Enterprise
Actively maintained	✅	✅	✅
Severity classification	✅	❌	❌
Multi-workspace scan	✅	❌	✅
Auto-remediation	✅	❌	✅
Watch mode + alerts	✅	❌	✅
Ignore rules	✅	❌	❌
Cost	Free	Free	$15K+/yr

The architecture

tfdrift is written in Python and built with a modular architecture:

tfdrift/
├── detectors/       # Drift detection engine (terraform plan parser)
├── reporters/       # Output formatters (JSON, Markdown, table, Slack)
├── remediators/     # Auto-fix logic with safety guards
├── severity.py      # 30+ built-in AWS severity rules
├── config.py        # .tfdrift.yml and .tfdriftignore parser
└── cli.py           # CLI (Click)

The severity engine uses pattern matching to classify drift. For example, any change to aws_security_group.*.ingress is automatically classified as CRITICAL, while *.tags changes are classified as LOW. You can extend these rules in your .tfdrift.yml:

severity:
  critical:
    - aws_security_group.*.ingress
    - aws_iam_policy.*.policy
  high:
    - aws_instance.*.instance_type
    - aws_rds_instance.*.publicly_accessible

Why I built this

I'm a DevOps engineer working with AWS, Terraform, and Kubernetes daily. Infrastructure drift was a recurring problem — security-critical changes mixed in with harmless noise, and no good free tool to sort through it.

The existing options were either expensive (Terraform Enterprise at $15K+/year) or too basic (raw terraform plan in a cron job).

I wanted something that could:

Tell me what drifted and how bad it is
Scan all my workspaces in one command
Send me a Slack alert at 2 AM when someone modifies a security group
Let me ignore the noise (auto-scaling changes, tag updates)
Run in CI/CD and fail the pipeline if critical drift is found

So I built it.

Try it

pip install tfdrift

# Scan your infrastructure
tfdrift scan --path ./your-terraform-dir

# Set up continuous monitoring
tfdrift watch --interval 1h --slack-webhook $SLACK_WEBHOOK

# Generate a config file
tfdrift init

The code is fully open source: github.com/sudarshan8417/tfdrift

I'd love feedback, bug reports, and contributions. If you've dealt with Terraform drift at your organization, I'd especially love to hear what features you'd want next — Azure/GCP support, a web dashboard, PagerDuty integration?

Drop a comment or open an issue on GitHub.

If you found this useful, consider giving the repo a ⭐ on GitHub — it helps other engineers find the tool.