Forem: Suhas Mallesh

Azure ML Workspace with Terraform: Your ML Platform on Azure 🔬

Suhas Mallesh — Fri, 03 Apr 2026 07:00:00 +0000

Azure Machine Learning workspace is the hub for all ML activities - experiments, models, endpoints, pipelines. It requires four dependent services. Here's how to provision the entire platform with Terraform including compute instances and clusters.

In Series 1-3, we worked with managed AI services - AI Foundry for models, AI Search for RAG, Agent Service for orchestration. Series 5 shifts to custom ML - training your own models, deploying endpoints, managing features, and building CI/CD pipelines.

It starts with an Azure Machine Learning workspace. The workspace is the top-level resource for all ML activities: experiments, datasets, models, compute targets, endpoints, and pipelines live here. Unlike a simple resource, the workspace requires four dependent services before it can be created: Storage Account, Key Vault, Application Insights, and Container Registry. Terraform provisions the entire stack. 🎯

🏗️ Workspace Architecture

Component	What It Does
Workspace	Central hub for ML experiments, models, and pipelines
Storage Account	Default datastore for datasets, model artifacts, logs
Key Vault	Secrets, connection strings, API keys
Application Insights	Experiment tracking, endpoint monitoring
Container Registry	Custom training images, model deployment containers
Compute Instance	Per-user notebook/IDE (JupyterLab, VS Code)
Compute Cluster	Auto-scaling training cluster (CPU or GPU)

All four dependent services must exist before the workspace. Terraform handles the dependency ordering automatically.

🔧 Terraform: The Full Workspace Setup

Dependent Services

# ml/dependencies.tf

resource "azurerm_storage_account" "ml" {
  name                     = "${var.environment}ml${random_string.suffix.result}"
  location                 = azurerm_resource_group.ml.location
  resource_group_name      = azurerm_resource_group.ml.name
  account_tier             = "Standard"
  account_replication_type = var.storage_replication
  min_tls_version          = "TLS1_2"
  allow_nested_items_to_be_public = false

  tags = var.tags
}

resource "azurerm_key_vault" "ml" {
  name                       = "${var.environment}ml${random_string.suffix.result}kv"
  location                   = azurerm_resource_group.ml.location
  resource_group_name        = azurerm_resource_group.ml.name
  tenant_id                  = data.azurerm_client_config.current.tenant_id
  sku_name                   = "standard"
  purge_protection_enabled   = true
  enable_rbac_authorization  = true

  tags = var.tags
}

resource "azurerm_application_insights" "ml" {
  name                = "${var.environment}-ml-insights"
  location            = azurerm_resource_group.ml.location
  resource_group_name = azurerm_resource_group.ml.name
  application_type    = "web"

  tags = var.tags
}

resource "azurerm_container_registry" "ml" {
  name                = "${var.environment}ml${random_string.suffix.result}acr"
  location            = azurerm_resource_group.ml.location
  resource_group_name = azurerm_resource_group.ml.name
  sku                 = var.acr_sku
  admin_enabled       = false

  tags = var.tags
}

Container Registry is optional but recommended. Without it, the workspace uses Azure-managed image building. With it, you control custom training images and model serving containers. Set admin_enabled = false and use managed identity instead.

The Workspace

# ml/workspace.tf

resource "azurerm_machine_learning_workspace" "this" {
  name                          = "${var.environment}-ml-workspace"
  location                      = azurerm_resource_group.ml.location
  resource_group_name           = azurerm_resource_group.ml.name
  application_insights_id       = azurerm_application_insights.ml.id
  key_vault_id                  = azurerm_key_vault.ml.id
  storage_account_id            = azurerm_storage_account.ml.id
  container_registry_id         = azurerm_container_registry.ml.id
  public_network_access_enabled = var.public_network_access

  identity {
    type = "SystemAssigned"
  }

  tags = var.tags
}

The workspace creates a system-assigned managed identity that accesses the dependent services. No keys or connection strings to manage.

Compute Instance (Per-User IDE)

# ml/compute_instance.tf

resource "azurerm_machine_learning_compute_instance" "this" {
  for_each = var.compute_instances

  name                          = each.key
  machine_learning_workspace_id = azurerm_machine_learning_workspace.this.id
  location                      = azurerm_resource_group.ml.location
  virtual_machine_size          = each.value.vm_size
  node_public_ip_enabled        = var.public_network_access

  identity {
    type = "SystemAssigned"
  }

  tags = merge(var.tags, {
    Team = each.value.team
    User = each.key
  })
}

Each data scientist gets their own compute instance with JupyterLab and VS Code access. Instances stop and start independently - you only pay when they're running.

Compute Cluster (Training)

# ml/compute_cluster.tf

resource "azurerm_machine_learning_compute_cluster" "training" {
  name                          = "${var.environment}-training"
  machine_learning_workspace_id = azurerm_machine_learning_workspace.this.id
  location                      = azurerm_resource_group.ml.location
  vm_size                       = var.training_vm_size
  vm_priority                   = var.training_vm_priority

  identity {
    type = "SystemAssigned"
  }

  scale_settings {
    min_node_count                       = 0
    max_node_count                       = var.training_max_nodes
    scale_down_nodes_after_idle_duration  = "PT${var.scale_down_minutes}M"
  }

  tags = var.tags
}

min_node_count = 0 is the cost control key. The cluster scales to zero when no training jobs are running. You pay nothing for idle compute. scale_down_nodes_after_idle_duration controls how quickly nodes are released after a job finishes.

vm_priority = "LowPriority" saves up to 80% on training costs. Low-priority VMs can be evicted, so use them for fault-tolerant training jobs with checkpointing.

📐 Environment Configuration

# environments/dev.tfvars
environment            = "dev"
public_network_access  = true
storage_replication    = "LRS"
acr_sku                = "Basic"
training_vm_size       = "Standard_DS3_v2"
training_vm_priority   = "Dedicated"
training_max_nodes     = 2
scale_down_minutes     = 15

compute_instances = {
  "ds-dev-1" = {
    vm_size = "Standard_DS3_v2"
    team    = "ml-team"
  }
}

# environments/prod.tfvars
environment            = "prod"
public_network_access  = false
storage_replication    = "GRS"
acr_sku                = "Standard"
training_vm_size       = "Standard_NC6s_v3"   # GPU
training_vm_priority   = "LowPriority"
training_max_nodes     = 8
scale_down_minutes     = 30

compute_instances = {
  "ds-lead" = {
    vm_size = "Standard_DS4_v2"
    team    = "ml-team"
  }
  "ds-engineer-1" = {
    vm_size = "Standard_DS3_v2"
    team    = "ml-team"
  }
  "ds-engineer-2" = {
    vm_size = "Standard_DS3_v2"
    team    = "ml-team"
  }
}

Dev: Public access, LRS storage, small dedicated cluster for quick iteration.
Prod: Private access, GRS storage, GPU cluster with low-priority VMs for cost-efficient training.

🔧 RBAC for Team Access

# ml/rbac.tf

# Data scientists get Contributor on the workspace
resource "azurerm_role_assignment" "ds_contributor" {
  for_each = var.data_scientist_principals

  scope                = azurerm_machine_learning_workspace.this.id
  role_definition_name = "AzureML Compute Operator"
  principal_id         = each.value
}

# ML engineers get full workspace access
resource "azurerm_role_assignment" "mle_contributor" {
  for_each = var.ml_engineer_principals

  scope                = azurerm_machine_learning_workspace.this.id
  role_definition_name = "Contributor"
  principal_id         = each.value
}

Use AzureML Compute Operator for data scientists who need to run experiments but shouldn't modify workspace settings. Use Contributor for ML engineers who manage the full lifecycle.

🔧 Security Hardening

Control	Dev	Prod
Network access	Public	Private (managed VNet)
Storage	LRS, TLS 1.2	GRS, TLS 1.2, no public blob
Key Vault	RBAC auth	RBAC auth, purge protection
Container Registry	Basic, no admin	Standard, managed identity
Compute	Public IP	No public IP, subnet attached
Identity	System-assigned	System-assigned + RBAC

For production, set public_network_access_enabled = false on the workspace and use managed virtual network isolation. All compute instances and clusters run inside a managed VNet with outbound rules controlled by the workspace.

⚠️ Gotchas and Tips

Globally unique names required. Storage account and ACR names must be globally unique. Use random_string suffixes to avoid conflicts across environments.

Container Registry costs. Basic SKU is $5/month. Standard is $20/month. If you're not building custom training images yet, you can skip the ACR initially and add it later.

Compute instance auto-stop. Unlike clusters, compute instances don't auto-stop by default. Set up an idle shutdown schedule through the workspace settings or Azure Policy to prevent overnight charges.

Workspace deletion is complex. Deleting a workspace doesn't automatically delete dependent resources (storage, KV, ACR). Terraform handles this correctly with depends_on, but manual deletion requires cleaning up each resource individually.

SDK v1 is deprecated. Azure ML SDK v1 reached end-of-support in March 2025. Use SDK v2 (azure-ai-ml) for all new development. Terraform provisions the infrastructure; SDK v2 handles experiments and models.

⏭️ What's Next

This is Post 1 of the Azure ML Pipelines & MLOps with Terraform series.

Post 1: Azure ML Workspace (you are here) 🔬
Post 2: Azure ML Online Endpoints - Deploy to Prod
Post 3: Azure ML Feature Store
Post 4: Azure ML Pipelines + Azure DevOps

Your ML platform is provisioned. Workspace, storage, key vault, ACR, compute instances for notebooks, auto-scaling clusters for training - all in Terraform. The foundation for model development, deployment, and production ML pipelines. 🔬

Found this helpful? Follow for the full ML Pipelines & MLOps with Terraform series! 💬

Vertex AI Workbench with Terraform: Your ML Workspace on GCP 🔬

Suhas Mallesh — Thu, 02 Apr 2026 07:00:00 +0000

Vertex AI Workbench is the JupyterLab IDE for ML on GCP - pre-installed ML frameworks, Vertex AI integration, and GPU support out of the box. Here's how to provision instances with Terraform including networking, IAM, auto-shutdown, and custom containers.

In Series 1-3, we worked with managed AI services - Vertex AI for models, RAG Engine for retrieval, ADK for agents. Series 5 shifts to custom ML - training your own models, deploying endpoints, managing features, and building ML pipelines.

It starts with a development environment. Vertex AI Workbench provides JupyterLab instances backed by Compute Engine VMs, pre-loaded with ML frameworks (TensorFlow, PyTorch, JAX, scikit-learn), the Vertex AI SDK, and direct integration with GCS, BigQuery, and Vertex AI services. Each instance is a full ML workspace. Terraform provisions them consistently across your team. 🎯

🏗️ Workbench Architecture

Component	What It Does
Workbench Instance	JupyterLab on a Compute Engine VM
VM Image	Pre-built with ML frameworks and Vertex SDK
Custom Container	Your own Docker image with specific dependencies
Data Disk	Persistent storage for notebooks and datasets
Service Account	Controls access to GCS, BigQuery, Vertex AI
Network	VPC, subnet, firewall rules for isolation

Unlike SageMaker's domain-based approach, Workbench instances are standalone VMs. Each data scientist gets their own instance with full control over machine type, GPU, and installed packages. Terraform standardizes the configuration across the team.

🔧 Terraform: The Full Workbench Setup

APIs and Networking

# workbench/apis.tf

resource "google_project_service" "required" {
  for_each = toset([
    "notebooks.googleapis.com",
    "aiplatform.googleapis.com",
    "compute.googleapis.com",
    "storage.googleapis.com",
    "bigquery.googleapis.com",
  ])
  project = var.project_id
  service = each.value
}

# VPC for Workbench instances
resource "google_compute_network" "ml" {
  name                    = "${var.environment}-ml-network"
  auto_create_subnetworks = false
  project                 = var.project_id
}

resource "google_compute_subnetwork" "ml" {
  name                     = "${var.environment}-ml-subnet"
  ip_cidr_range            = var.subnet_cidr
  region                   = var.region
  network                  = google_compute_network.ml.id
  private_ip_google_access = true  # Access Google APIs without public IP
  project                  = var.project_id
}

# Allow internal traffic for JupyterLab proxy
resource "google_compute_firewall" "internal" {
  name    = "${var.environment}-ml-internal"
  network = google_compute_network.ml.id
  project = var.project_id

  allow {
    protocol = "tcp"
    ports    = ["8080", "443"]
  }

  source_ranges = [var.subnet_cidr]
}

private_ip_google_access = true is essential. It lets Workbench instances access Google APIs (GCS, BigQuery, Vertex AI) without a public IP address, keeping your ML environment network-isolated.

Service Account

# workbench/iam.tf

resource "google_service_account" "workbench" {
  account_id   = "${var.environment}-workbench-sa"
  display_name = "Workbench ML Service Account"
  project      = var.project_id
}

# Access to training data in GCS
resource "google_project_iam_member" "storage_user" {
  project = var.project_id
  role    = "roles/storage.objectUser"
  member  = "serviceAccount:${google_service_account.workbench.email}"
}

# Access to Vertex AI for training and prediction
resource "google_project_iam_member" "vertex_ai_user" {
  project = var.project_id
  role    = "roles/aiplatform.user"
  member  = "serviceAccount:${google_service_account.workbench.email}"
}

# Access to BigQuery for data exploration
resource "google_project_iam_member" "bigquery_user" {
  project = var.project_id
  role    = "roles/bigquery.user"
  member  = "serviceAccount:${google_service_account.workbench.email}"
}

# Read Artifact Registry for custom containers
resource "google_project_iam_member" "artifact_reader" {
  project = var.project_id
  role    = "roles/artifactregistry.reader"
  member  = "serviceAccount:${google_service_account.workbench.email}"
}

Workbench Instance

# workbench/instance.tf

resource "google_workbench_instance" "this" {
  for_each = var.workbench_instances

  name     = "${var.environment}-${each.key}"
  location = var.zone
  project  = var.project_id

  gce_setup {
    machine_type = each.value.machine_type

    vm_image {
      project = "cloud-notebooks-managed"
      family  = "workbench-instances"
    }

    # Optional GPU
    dynamic "accelerator_configs" {
      for_each = each.value.gpu_type != null ? [1] : []
      content {
        type       = each.value.gpu_type
        core_count = each.value.gpu_count
      }
    }

    boot_disk {
      disk_type    = "PD_SSD"
      disk_size_gb = each.value.boot_disk_gb
    }

    data_disks {
      disk_type    = "PD_SSD"
      disk_size_gb = each.value.data_disk_gb
    }

    network_interfaces {
      network  = google_compute_network.ml.id
      subnet   = google_compute_subnetwork.ml.id
      nic_type = "GVNIC"
    }

    service_accounts {
      email = google_service_account.workbench.email
    }

    metadata = {
      idle-timeout-seconds   = each.value.idle_timeout
      install-nvidia-driver  = each.value.gpu_type != null ? "true" : "false"
      post-startup-script    = each.value.post_startup_script
    }

    enable_ip_forwarding = false
  }

  disable_proxy_access = false  # Enable JupyterLab proxy access

  labels = {
    environment = var.environment
    team        = each.value.team
    managed_by  = "terraform"
  }
}

Key configuration points:

vm_image.family = "workbench-instances" uses Google's pre-built image with ML frameworks
data_disks provides persistent storage separate from the boot disk
idle-timeout-seconds in metadata auto-stops the instance after inactivity
install-nvidia-driver auto-installs GPU drivers when GPU is attached
disable_proxy_access = false enables browser access to JupyterLab via IAP

Custom Container Alternative

For teams that need specific package versions or private packages, use a custom container:

resource "google_workbench_instance" "custom" {
  name     = "${var.environment}-custom-ml"
  location = var.zone
  project  = var.project_id

  gce_setup {
    machine_type = "n1-standard-8"

    container_image {
      repository = "${var.region}-docker.pkg.dev/${var.project_id}/ml-images/custom-workbench"
      tag        = "latest"
    }

    boot_disk {
      disk_type    = "PD_SSD"
      disk_size_gb = 150
    }

    network_interfaces {
      network  = google_compute_network.ml.id
      subnet   = google_compute_subnetwork.ml.id
    }

    service_accounts {
      email = google_service_account.workbench.email
    }
  }
}

Build your container from Google's base image, add your packages, push to Artifact Registry, and reference it in Terraform. Updates are a new tag and terraform apply.

📐 Environment Configuration

# environments/dev.tfvars
environment = "dev"
zone        = "us-central1-a"

workbench_instances = {
  "data-scientist-1" = {
    machine_type       = "n1-standard-4"
    gpu_type           = null          # No GPU for dev
    gpu_count          = 0
    boot_disk_gb       = 100
    data_disk_gb       = 200
    idle_timeout        = "3600"       # 1 hour auto-stop
    team               = "ml-team"
    post_startup_script = null
  }
}

# environments/prod.tfvars
environment = "prod"
zone        = "us-central1-a"

workbench_instances = {
  "ml-lead" = {
    machine_type       = "n1-standard-8"
    gpu_type           = "NVIDIA_TESLA_T4"
    gpu_count          = 1
    boot_disk_gb       = 150
    data_disk_gb       = 500
    idle_timeout        = "7200"       # 2 hour auto-stop
    team               = "ml-team"
    post_startup_script = "gs://my-bucket/scripts/setup.sh"
  }
  "ml-engineer-1" = {
    machine_type       = "n1-standard-8"
    gpu_type           = "NVIDIA_TESLA_T4"
    gpu_count          = 1
    boot_disk_gb       = 150
    data_disk_gb       = 500
    idle_timeout        = "7200"
    team               = "ml-team"
    post_startup_script = "gs://my-bucket/scripts/setup.sh"
  }
}

Cost control through idle_timeout. GPU instances cost $1-3/hour. A T4 instance left running overnight costs $24-72 in wasted spend. The idle-timeout-seconds metadata auto-stops the VM when JupyterLab kernels are idle.

🔧 Post-Startup Script for Team Standardization

Standardize packages and configs across all instances:

#!/bin/bash
# scripts/setup.sh - uploaded to GCS

# Install team-standard packages
pip install --upgrade \
  google-cloud-aiplatform \
  google-cloud-bigquery \
  pandas \
  scikit-learn \
  xgboost

# Clone team shared repos
git clone https://github.com/your-org/ml-utils.git /home/jupyter/ml-utils

# Set environment variables
echo 'export PROJECT_ID=your-project' >> /home/jupyter/.bashrc
echo 'export REGION=us-central1' >> /home/jupyter/.bashrc

Reference it in the instance metadata via post_startup_script. Runs once when the instance starts for the first time.

⚠️ Gotchas and Tips

User-managed notebooks are deprecated. As of January 2025, only Workbench Instances (google_workbench_instance) are supported. Don't use google_notebooks_instance - it's legacy.

GPU quota must be requested. T4, V100, A100 GPUs require quota increases in your GCP project. Request early - approval can take 1-2 business days.

Instance stops don't lose data. Stopping a Workbench instance preserves the boot and data disks. You only pay for disk storage when stopped. Starting it again restores your full environment.

Use data disks for large datasets. Boot disks are limited and get recreated on image updates. Store notebooks and datasets on the data disk (/home/jupyter) which persists independently.

Private Google Access vs public IP. With private_ip_google_access = true on the subnet, instances can reach GCS, BigQuery, and Vertex AI without a public IP. Combine with IAP for secure browser access to JupyterLab.

⏭️ What's Next

This is Post 1 of the GCP ML Pipelines & MLOps with Terraform series.

Post 1: Vertex AI Workbench (you are here) 🔬
Post 2: Vertex AI Endpoints - Deploy to Prod
Post 3: Vertex AI Feature Store
Post 4: Vertex AI Pipelines + Cloud Build

Your ML workspace is provisioned. Network-isolated, GPU-ready, auto-shutdown enabled, with standardized packages across your team. The foundation for model training, deployment, and production ML pipelines. 🔬

Found this helpful? Follow for the full ML Pipelines & MLOps with Terraform series! 💬

SageMaker Studio Domain with Terraform: Your ML Workspace on AWS 🔬

Suhas Mallesh — Wed, 01 Apr 2026 07:00:00 +0000

SageMaker Studio is the IDE for ML on AWS - notebooks, training, deployment, all in one place. Here's how to provision the entire domain with Terraform including VPC, IAM, user profiles, and security hardening.

In Series 1-3, we worked with managed AI services - Bedrock for models, Knowledge Bases for RAG, Agents for orchestration. Series 5 shifts to custom ML - training your own models, deploying them to endpoints, managing features, and building CI/CD pipelines.

It all starts with a SageMaker Studio Domain. The domain is the foundation for everything in SageMaker - it provides the IDE (JupyterLab, Code Editor), manages user profiles, attaches shared storage (EFS), and controls network access. Think of it as the workspace where your ML team lives. This post provisions the entire setup with Terraform. 🎯

🏗️ What a SageMaker Domain Contains

Component	What It Does
Domain	Top-level resource: VPC config, auth mode, encryption
User Profile	Per-user config: execution role, instance types, apps
Shared Space	Team collaboration spaces with shared notebooks
EFS Volume	Persistent storage for notebooks and data files
Apps	JupyterLab, Code Editor, Canvas, Studio Classic

One domain per team or environment. User profiles within the domain give each data scientist their own isolated workspace with shared access to the same data and models.

🔧 Terraform: The Full Domain Setup

VPC and Networking

SageMaker Studio requires a VPC. For production, use VPC-only mode with private subnets and VPC endpoints:

# network/main.tf

resource "aws_vpc" "ml" {
  cidr_block           = var.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = { Name = "${var.environment}-ml-vpc" }
}

resource "aws_subnet" "private" {
  count             = 2
  vpc_id            = aws_vpc.ml.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = { Name = "${var.environment}-ml-private-${count.index}" }
}

# S3 Gateway endpoint (required for SageMaker in VPC-only mode)
resource "aws_vpc_endpoint" "s3" {
  vpc_id       = aws_vpc.ml.id
  service_name = "com.amazonaws.${var.region}.s3"

  route_table_ids = [aws_route_table.private.id]
}

# SageMaker API interface endpoint
resource "aws_vpc_endpoint" "sagemaker_api" {
  vpc_id              = aws_vpc.ml.id
  service_name        = "com.amazonaws.${var.region}.sagemaker.api"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = aws_subnet.private[*].id
  security_group_ids  = [aws_security_group.vpc_endpoints.id]
  private_dns_enabled = true
}

# SageMaker Runtime interface endpoint
resource "aws_vpc_endpoint" "sagemaker_runtime" {
  vpc_id              = aws_vpc.ml.id
  service_name        = "com.amazonaws.${var.region}.sagemaker.runtime"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = aws_subnet.private[*].id
  security_group_ids  = [aws_security_group.vpc_endpoints.id]
  private_dns_enabled = true
}

resource "aws_security_group" "vpc_endpoints" {
  name_prefix = "${var.environment}-vpce-"
  vpc_id      = aws_vpc.ml.id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [var.vpc_cidr]
  }
}

VPC-only mode blocks all internet access. You need VPC endpoints for every AWS service SageMaker uses: S3 (gateway), SageMaker API, SageMaker Runtime, and optionally STS, CloudWatch Logs, and ECR for training jobs.

IAM Execution Role

# iam/main.tf

resource "aws_iam_role" "sagemaker_execution" {
  name = "${var.environment}-sagemaker-execution"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action    = "sts:AssumeRole"
      Effect    = "Allow"
      Principal = { Service = "sagemaker.amazonaws.com" }
    }]
  })
}

# Scoped permissions - not AmazonSageMakerFullAccess
resource "aws_iam_role_policy" "sagemaker_core" {
  name = "sagemaker-core"
  role = aws_iam_role.sagemaker_execution.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:PutObject",
          "s3:ListBucket"
        ]
        Resource = [
          "${var.data_bucket_arn}",
          "${var.data_bucket_arn}/*",
          "${var.artifacts_bucket_arn}",
          "${var.artifacts_bucket_arn}/*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "ecr:GetAuthorizationToken",
          "ecr:BatchGetImage",
          "ecr:GetDownloadUrlForLayer"
        ]
        Resource = "*"
      },
      {
        Effect = "Allow"
        Action = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ]
        Resource = "arn:aws:logs:${var.region}:${data.aws_caller_identity.current.account_id}:*"
      }
    ]
  })
}

Avoid AmazonSageMakerFullAccess in production. It grants broad permissions across S3, ECR, Lambda, and more. Scope your policies to the specific buckets, registries, and services your team needs.

KMS Key for Encryption

# security/kms.tf

resource "aws_kms_key" "sagemaker" {
  description             = "SageMaker Studio EFS and EBS encryption"
  deletion_window_in_days = 7
  enable_key_rotation     = true
}

resource "aws_kms_alias" "sagemaker" {
  name          = "alias/${var.environment}-sagemaker"
  target_key_id = aws_kms_key.sagemaker.key_id
}

The Domain

# sagemaker/domain.tf

resource "aws_sagemaker_domain" "this" {
  domain_name = "${var.environment}-ml-studio"
  auth_mode   = "IAM"
  vpc_id      = aws_vpc.ml.id
  subnet_ids  = aws_subnet.private[*].id

  # VPC-only mode blocks internet access
  app_network_access_type = var.network_access_type

  # Encrypt EFS and EBS volumes
  kms_key_id = aws_kms_key.sagemaker.arn

  default_user_settings {
    execution_role = aws_iam_role.sagemaker_execution.arn

    security_groups = [aws_security_group.sagemaker_studio.id]

    jupyter_lab_app_settings {
      default_resource_spec {
        instance_type       = var.default_instance_type
        sagemaker_image_arn = var.jupyter_image_arn
      }
    }

    sharing_settings {
      notebook_output_option = "Allowed"
      s3_output_path         = "s3://${var.artifacts_bucket}/sharing"
    }
  }

  tags = {
    Environment = var.environment
    ManagedBy   = "terraform"
  }
}

User Profiles

# sagemaker/users.tf

resource "aws_sagemaker_user_profile" "this" {
  for_each = var.user_profiles

  domain_id         = aws_sagemaker_domain.this.id
  user_profile_name = each.key

  user_settings {
    execution_role = coalesce(
      each.value.execution_role_arn,
      aws_iam_role.sagemaker_execution.arn
    )
  }

  tags = {
    Team        = each.value.team
    Environment = var.environment
  }
}

📐 Environment Configuration

# environments/dev.tfvars
environment          = "dev"
network_access_type  = "PublicInternetOnly"  # Simpler for dev
default_instance_type = "ml.t3.medium"

user_profiles = {
  "data-scientist-1" = {
    team               = "ml-team"
    execution_role_arn = null  # Uses domain default
  }
}

# environments/prod.tfvars
environment          = "prod"
network_access_type  = "VpcOnly"  # Locked down
default_instance_type = "ml.t3.medium"

user_profiles = {
  "ds-lead" = {
    team               = "ml-team"
    execution_role_arn = null
  }
  "ds-engineer-1" = {
    team               = "ml-team"
    execution_role_arn = null
  }
  "ds-engineer-2" = {
    team               = "ml-team"
    execution_role_arn = null
  }
}

Dev uses PublicInternetOnly for easier setup - no VPC endpoints needed, pip install works out of the box. Prod uses VpcOnly for network isolation - all traffic stays within your VPC, requires VPC endpoints for AWS services.

🔧 Security Hardening Checklist

Control	Dev	Prod
Network access	PublicInternetOnly	VpcOnly
EFS encryption	KMS key	KMS key
IAM scope	Scoped to buckets	Least-privilege per team
Instance types	ml.t3.medium	Restricted via Service Quotas
Notebook sharing	Allowed (S3 output)	Allowed (encrypted S3)
Auto-shutdown	Optional	Lifecycle config enforced

🔧 Auto-Shutdown Lifecycle Config (Cost Control)

Idle notebooks burn money. Attach a lifecycle config to auto-shutdown:

resource "aws_sagemaker_studio_lifecycle_config" "auto_shutdown" {
  studio_lifecycle_config_name     = "${var.environment}-auto-shutdown"
  studio_lifecycle_config_app_type = "JupyterServer"
  studio_lifecycle_config_content  = base64encode(file("${path.module}/scripts/auto-shutdown.sh"))
}

The script checks for idle kernels and shuts down the notebook instance after a configurable timeout (typically 60-120 minutes of inactivity).

⚠️ Gotchas and Tips

One domain per region per account (soft limit). You can request an increase, but the default is one. Use user profiles to separate team members within a single domain.

EFS is created automatically. The domain creates an EFS volume and a home directory per user profile. You don't manage this directly, but you should encrypt it with KMS.

VPC endpoint costs add up. Each interface VPC endpoint costs roughly $7/month plus data processing charges. In VPC-only mode, you may need 5-10 endpoints. Budget $50-100/month for networking in prod.

Domain deletion is slow and complex. Deleting a domain requires deleting all apps, user profiles, and spaces first. Terraform handles this, but terraform destroy can take 15-20 minutes.

Instance type quotas. GPU instances (ml.g5, ml.p4d) require Service Quota increases. Request these early - approval can take days.

⏭️ What's Next

This is Post 1 of the ML Pipelines & MLOps with Terraform series.

Post 1: SageMaker Studio Domain (you are here) 🔬
Post 2: SageMaker Endpoints - Deploy to Prod
Post 3: SageMaker Feature Store
Post 4: SageMaker Pipelines - CI/CD for ML

Your ML workspace is provisioned. VPC-isolated, KMS-encrypted, IAM-scoped, with user profiles for every team member. The foundation for everything that follows: model training, deployment, feature engineering, and ML pipelines. 🔬

Found this helpful? Follow for the full ML Pipelines & MLOps with Terraform series! 💬

Azure AI Agent + Bing Grounding: Real-Time Web Search with Terraform 📚

Suhas Mallesh — Sat, 28 Mar 2026 07:00:00 +0000

An agent without grounding hallucinates about current events. Bing Grounding gives your Azure AI agent real-time web search with source citations. Here's how to provision the Bing resource, connect it to your Foundry project, and attach it to your agent with Terraform.

Through this series, we've built Azure AI agents that can reason, call functions, and coordinate multi-agent workflows. But ask one about today's news, current stock prices, or this week's weather, and it either hallucinates or admits its training data has a cutoff.

Bing Grounding connects your agent to real-time web search. When the agent determines it needs current information, it searches Bing, retrieves relevant results, and generates a response grounded in those results with URL citations. The agent decides when to search. Bing provides the facts. Your users get accurate, sourced answers. 🎯

🏗️ How Bing Grounding Works

User: "What's the weather in Seattle today?"
    ↓
Agent analyzes request → determines it needs current data
    ↓
Agent invokes Bing Grounding tool
    ↓
Bing searches the web, returns relevant results
    ↓
Agent generates response grounded in Bing results
    ↓
User: "It's 55°F and partly cloudy in Seattle today."
       [Citation: weather.gov/seattle]

The agent doesn't call the Bing API directly. The Foundry Agent Service handles the Bing integration through a connection resource. You provision the Bing resource, create the connection, and attach the tool to your agent.

🔧 Two Bing Grounding Options

Tool	Scope	Best For
Grounding with Bing Search	Full public web	General current events, weather, news, live data
Grounding with Bing Custom Search	Specific domains you define	Restricted searches (e.g., only your docs site, partner sites)

Bing Search searches the full web. Bing Custom Search lets you restrict results to specific domains, useful when you want grounding but only from trusted sources.

🔧 Terraform: Provision Bing Grounding

Step 1: Create the Bing Grounding Resource

# grounding/bing.tf

resource "azapi_resource" "bing_grounding" {
  type      = "Microsoft.Bing/accounts@2025-05-01-preview"
  name      = "${var.environment}-bing-grounding"
  location  = "global"
  parent_id = azurerm_resource_group.this.id

  body = {
    kind = "Grounding"
    sku = {
      name = "G1"
    }
  }
}

Why azapi? The Bing Grounding resource isn't available in the azurerm provider yet. The azapi provider gives access to the latest Azure resource types directly.

Step 2: Connect Bing to Your Foundry Project

# grounding/connection.tf

resource "azapi_resource" "bing_connection" {
  type      = "Microsoft.CognitiveServices/accounts/connections@2025-06-01"
  name      = "${var.environment}-bing-connection"
  parent_id = azurerm_cognitive_account.ai_foundry.id

  schema_validation_enabled = false

  body = {
    properties = {
      category  = "ApiKey"
      target    = azapi_resource.bing_grounding.output.properties.endpoint
      authType  = "ApiKey"
      isSharedToAll = true
      credentials = {
        key = azapi_resource.bing_grounding.output.properties.accessKeys.key1
      }
      metadata = {
        ApiType    = "Bing"
        ResourceId = azapi_resource.bing_grounding.id
      }
    }
  }
}

This creates a connection between your Foundry project and the Bing resource. The connection ID is what your agent code references when attaching the Bing tool.

Step 3: Output the Connection ID

# grounding/outputs.tf

resource "local_file" "agent_config" {
  filename = "${path.module}/agent_code/agent_config.json"
  content = jsonencode({
    foundry_endpoint  = azurerm_cognitive_account.ai_foundry.endpoint
    deployment_name   = azurerm_cognitive_deployment.agent_model.name
    agent_name        = "${var.environment}-grounded-agent"
    instruction       = var.agent_instruction
    bing_connection_id = azapi_resource.bing_connection.id
  })
}

🐍 Create Agent with Bing Grounding

# agent_code/create_agent.py

from azure.ai.agents import AgentsClient
from azure.ai.agents.models import BingGroundingTool
from azure.identity import DefaultAzureCredential
import json

with open("agent_config.json") as f:
    config = json.load(f)

client = AgentsClient(
    endpoint=config["foundry_endpoint"],
    credential=DefaultAzureCredential(),
)

# Create Bing grounding tool from connection
bing_tool = BingGroundingTool(
    connection_id=config["bing_connection_id"]
)

# Create agent with Bing grounding
agent = client.create_agent(
    model=config["deployment_name"],
    name=config["agent_name"],
    instructions="""You are a helpful assistant with access to real-time 
    web search. When answering questions about current events, weather, 
    news, stock prices, or any time-sensitive information, use Bing 
    Search to ground your response. Always include source citations.""",
    tools=bing_tool.definitions,
)

print(f"Agent created: {agent.id}")

🧪 Query the Grounded Agent

# Create thread and send message
thread = client.threads.create()
client.messages.create(
    thread_id=thread.id,
    role="user",
    content="What are the top tech headlines today?"
)

# Run the agent
run = client.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent.id,
)

# Get response with citations
if run.status == "completed":
    messages = client.messages.list(thread_id=thread.id)
    for msg in messages:
        if msg.role == "assistant":
            # Print the response text
            print(msg.content[0].text.value)

            # Print URL citations
            if hasattr(msg.content[0].text, "annotations"):
                for annotation in msg.content[0].text.annotations:
                    if hasattr(annotation, "url_citation"):
                        print(f"  Source: {annotation.url_citation.url}")

The response includes inline URL citations. When Bing contributes to the answer, annotations link back to the source websites. These citations must be displayed to end users per Microsoft's Use and Display Requirements.

📐 Combining Bing Grounding with Other Tools

An agent can have Bing Grounding alongside function calling tools:

from azure.ai.agents.models import BingGroundingTool, FunctionTool

bing_tool = BingGroundingTool(connection_id=config["bing_connection_id"])

exchange_rate_tool = FunctionTool(
    name="get_exchange_rate",
    description="Get live exchange rates between currencies.",
    parameters={
        "type": "object",
        "properties": {
            "currency_from": {"type": "string", "description": "Source currency code"},
            "currency_to": {"type": "string", "description": "Target currency code"},
        },
        "required": ["currency_from", "currency_to"]
    }
)

# Agent with both grounding and function tools
agent = client.create_agent(
    model=config["deployment_name"],
    name="full-agent",
    instructions="""You are a helpful assistant.
    Use Bing Search for current events, news, and general web queries.
    Use get_exchange_rate for live currency conversion rates.
    Choose the right tool based on the question.""",
    tools=bing_tool.definitions + [exchange_rate_tool],
)

The agent routes between Bing for general web queries and your custom functions for specific operations. No conflicts between tool types.

📐 Adding Azure AI Search (Private Docs)

For grounding against both public web and private documents, combine Bing with Azure AI Search:

from azure.ai.agents.models import (
    BingGroundingTool,
    AzureAISearchTool,
)

bing_tool = BingGroundingTool(connection_id=config["bing_connection_id"])

search_tool = AzureAISearchTool(
    index_connection_id=config["search_connection_id"],
    index_name=config["search_index_name"],
)

agent = client.create_agent(
    model=config["deployment_name"],
    name="fully-grounded-agent",
    instructions="""You are a company assistant.
    Use Bing for current events and public information.
    Use Azure AI Search for company policies and internal documents.
    Answer general knowledge questions directly.""",
    tools=bing_tool.definitions + search_tool.definitions,
)

The agent now has three paths: Bing for public web, AI Search for private docs, and direct answers for general knowledge.

⚠️ Gotchas and Tips

Citation display is required. When Bing Grounding contributes to a response, you must display the URL citations to end users. This is a requirement from Microsoft's Use and Display Requirements, not optional.

Data crosses compliance boundary. Bing queries are sent outside the Azure compliance boundary to the Bing service. Assess whether this meets your compliance requirements, especially in regulated industries.

Bing Grounding has per-transaction pricing. Each time the agent invokes the Bing tool, it counts as a grounding transaction. Check the Bing Grounding pricing page for current rates.

Agent decides when to search. The agent uses Bing only when it determines web search is needed. Clear instructions about when to use Bing help prevent unnecessary searches on questions the model can answer from general knowledge.

Connection-based architecture. The Bing resource connects to your Foundry project via a connection resource. If you move to a different project, recreate the connection. The Bing resource itself is reusable across projects.

⏭️ Series Complete!

This is Post 4 of the Azure AI Agents with Terraform series, and the final post of the entire multi-cloud blog series.

Post 1: Deploy First Azure AI Agent 🤖
Post 2: Function Calling - Connect to APIs 🔌
Post 3: Multi-Agent Orchestration 🧠
Post 4: Agent + Bing Grounding (you are here) 📚

Your agent is fully grounded. Bing for real-time web data. AI Search for private documents. Function calling for live APIs. From a simple model endpoint to a production-ready, multi-agent, grounded AI system - all defined in Terraform and Python. 📚

Vertex AI Agent + Google Search Grounding: Real-Time Facts with ADK and Terraform 📚

Suhas Mallesh — Fri, 27 Mar 2026 07:00:00 +0000

An agent without grounding hallucinates about current events. Google Search Grounding and Vertex AI Search give your ADK agent real-time web data and private document access with source citations. Here's how to wire both up with Terraform.

Through this series, we've built ADK agents that can reason, call tools, and coordinate with other agents. But ask one about today's stock prices or your company's internal policies, and it either hallucinates or admits it doesn't know. The model's training data has a cutoff, and it has no access to your documents.

Grounding fixes this with two built-in ADK tools: Google Search Grounding connects your agent to real-time web data - current events, live prices, recent news. Vertex AI Search Grounding connects your agent to your private documents - company policies, product specs, internal knowledge bases. Both return source-attributed responses with citations. 🎯

🏗️ Two Grounding Approaches

Tool	Data Source	Best For
Google Search	Public web (Google Search index)	Current events, live data, general facts
Vertex AI Search	Your private documents (indexed datastore)	Company policies, internal docs, product info

You can use one or both. Google Search handles what's public and current. Vertex AI Search handles what's private and specific to your organization.

🔧 Google Search Grounding

The simplest grounding option. Add the built-in google_search tool to your agent:

# agent_source/agent.py

from google.adk.agents import Agent
from google.adk.models.vertexai import VertexAi

model = VertexAi(model=config["model_id"])

search_agent = Agent(
    name="search-grounded-agent",
    model=model,
    instruction="""You are a helpful assistant that provides accurate,
    current information. When answering questions about recent events, 
    current data, or anything that requires up-to-date information,
    use Google Search to ground your response. Always cite your sources.""",
    tools=["google_search"],
)

That's it. One line: tools=["google_search"]. The agent automatically searches the web when it determines the question needs current information.

What the Response Looks Like

When the agent uses Google Search, the response includes groundingMetadata with source URLs, search queries executed, and confidence scores per segment:

async for event in runner.run_async(
    user_id="user-123",
    session_id=session.id,
    new_message=types.Content(
        role="user",
        parts=[types.Part(text="What were today's major tech news?")]
    )
):
    if event.content and event.content.parts:
        for part in event.content.parts:
            if hasattr(part, "text") and part.text:
                print(part.text)

    # Access grounding metadata for citations
    if hasattr(event, "grounding_metadata"):
        for chunk in event.grounding_metadata.grounding_chunks:
            print(f"Source: {chunk.web.uri} - {chunk.web.title}")

Important: When using Google Search Grounding in production, you must display the Search Suggestions provided in the response metadata. This is a requirement from Google's terms of service.

🔧 Vertex AI Search Grounding (Private Documents)

For grounding against your own documents, use the VertexAiSearchTool:

from google.adk.agents import Agent
from google.adk.tools import VertexAiSearchTool

# Your Vertex AI Search datastore path
DATASTORE_PATH = (
    f"projects/{config['project_id']}/locations/global"
    f"/collections/default_collection"
    f"/dataStores/{config['datastore_id']}"
)

doc_agent = Agent(
    name="doc-grounded-agent",
    model=model,
    instruction="""You are a company policy expert. Answer questions 
    about internal policies, procedures, and guidelines using the 
    provided document store. Always cite the source document.""",
    tools=[VertexAiSearchTool(data_store_id=DATASTORE_PATH)],
)

The agent queries your indexed documents, retrieves relevant chunks with semantic search, and generates a response grounded in your actual content. Source attributions include document titles and URIs.

Tool Limitation

Both google_search and VertexAiSearchTool are exclusive tools - they cannot be combined with other tools in the same agent instance. To use grounding alongside custom function tools, wrap the grounded agent as a sub-agent or use AgentTool:

from google.adk.tools import AgentTool

# Grounded agent as a tool for the main agent
search_assistant = Agent(
    name="searcher",
    model=model,
    instruction="Search the web for current information when asked.",
    tools=["google_search"],
)

doc_assistant = Agent(
    name="doc_lookup",
    model=model,
    instruction="Look up company policies and internal documents.",
    tools=[VertexAiSearchTool(data_store_id=DATASTORE_PATH)],
)

# Main agent uses grounded agents as tools alongside custom tools
main_agent = Agent(
    name="assistant",
    model=model,
    instruction="""You are a helpful assistant.
    Use the searcher for current events and live data.
    Use doc_lookup for company policies and internal info.
    Use get_exchange_rate for currency conversions.""",
    tools=[
        AgentTool(search_assistant),
        AgentTool(doc_assistant),
        get_exchange_rate,
    ],
)

This is the production pattern: dedicated grounding agents wrapped as tools for a main orchestrator agent.

🔧 Terraform: Infrastructure for Grounding

APIs and Vertex AI Search Datastore

# grounding/apis.tf

resource "google_project_service" "grounding" {
  for_each = toset([
    "aiplatform.googleapis.com",
    "discoveryengine.googleapis.com",  # For Vertex AI Search
    "storage.googleapis.com",
  ])
  project = var.project_id
  service = each.value
}

Vertex AI Search Datastore (for Private Docs)

# grounding/datastore.tf

resource "google_discovery_engine_data_store" "docs" {
  location                    = "global"
  data_store_id               = "${var.environment}-company-docs"
  display_name                = "Company Documents"
  industry_vertical           = "GENERIC"
  content_config              = "CONTENT_REQUIRED"
  solution_types              = ["SOLUTION_TYPE_SEARCH"]
  create_advanced_site_search = false
  project                     = var.project_id
}

IAM for Grounding

# grounding/iam.tf

# Agent SA needs Discovery Engine access for Vertex AI Search
resource "google_project_iam_member" "discovery_viewer" {
  project = var.project_id
  role    = "roles/discoveryengine.viewer"
  member  = "serviceAccount:${google_service_account.agent.email}"
}

Config with Datastore ID

# grounding/config.tf

resource "local_file" "agent_config" {
  filename = "${path.module}/agent_source/config.json"
  content = jsonencode({
    project_id   = var.project_id
    location     = var.region
    model_id     = var.agent_model.id
    agent_name   = "${var.environment}-grounded-agent"
    datastore_id = google_discovery_engine_data_store.docs.data_store_id
    instruction  = var.agent_instruction
  })
}

📐 Combining Google Search + Vertex AI Search

The production pattern uses both grounding sources through the sub-agent approach:

# Public data: current events, live information
web_agent = Agent(
    name="web_search",
    model=model,
    instruction="Search the public web for current, real-time information.",
    tools=["google_search"],
)

# Private data: company documents, internal policies
docs_agent = Agent(
    name="doc_search",
    model=model,
    instruction="Search company documents for internal policies and procedures.",
    tools=[VertexAiSearchTool(data_store_id=DATASTORE_PATH)],
)

# Orchestrator decides which source to use
grounded_assistant = Agent(
    name="grounded_assistant",
    model=model,
    instruction="""You are a knowledgeable assistant.
    For current events or public information, use web_search.
    For company policies or internal documents, use doc_search.
    For general knowledge questions, answer directly.""",
    tools=[
        AgentTool(web_agent),
        AgentTool(docs_agent),
    ],
)

The orchestrator routes to the right grounding source based on the question type. Public questions go to Google Search. Internal questions go to Vertex AI Search. General knowledge questions get answered directly without grounding.

⚠️ Gotchas and Tips

Exclusive tool limitation. google_search and VertexAiSearchTool can't be mixed with other tools in the same agent. Use the AgentTool wrapper pattern shown above.

Search Suggestions display requirement. When using Google Search Grounding in production, you must display the searchEntryPoint HTML/CSS from the response metadata. This is not optional.

Datastore indexing time. Vertex AI Search datastores need time to index your documents after upload. Don't test grounding immediately after creating the datastore - wait for indexing to complete.

Grounding adds latency. Each grounded response involves a search query plus retrieval. Expect 1-3 seconds of additional latency compared to non-grounded responses.

Confidence scores. The groundingMetadata includes confidence scores per response segment. Use these to flag low-confidence answers for human review in production.

⏭️ Series Complete!

This is Post 4 of the GCP AI Agents with Terraform series, and the final GCP post of the entire multi-cloud series.

Post 1: Deploy First Vertex AI Agent 🤖
Post 2: Agent Tools - Connect to APIs 🔌
Post 3: Multi-Agent Systems 🧠
Post 4: Agent + Grounding (you are here) 📚

Your agent is now grounded in reality. Google Search for real-time web data. Vertex AI Search for your private documents. Source citations on every response. No hallucinations, just facts. 📚

Thanks for following the full series! Follow for what comes next. 💬

Bedrock Agent + Knowledge Base: Ground Your Agent in Real Data with Terraform 📚

Suhas Mallesh — Thu, 26 Mar 2026 07:00:00 +0000

An agent without grounding makes things up. Associate a Knowledge Base with your Bedrock Agent and every response is backed by your actual documents - with citations. Here's how to wire it up with Terraform.

Through this series, we've built Bedrock Agents that can reason, call APIs via action groups, and coordinate with other agents. But ask one a question about your company's policies, and it either hallucinates or admits it doesn't know. The model has no access to your data.

Knowledge Base association changes that. When you attach a Knowledge Base to your agent, the agent automatically queries your vector store when it needs domain-specific information. It retrieves relevant chunks, uses them as context, and generates a grounded response with citations back to your source documents. No hallucination. No guessing. Real answers from real data. 🎯

🏗️ How Agent + Knowledge Base Works

User: "What is our return policy for electronics?"
    ↓
Agent analyzes request → determines it needs domain knowledge
    ↓
Agent queries Knowledge Base (vector search)
    ↓
Knowledge Base returns relevant chunks from policies.pdf
    ↓
Agent generates response grounded in retrieved context
    ↓
User: "Electronics can be returned within 30 days with original 
       receipt. Items must be in original packaging. [Source: 
       return-policy-2024.pdf, page 3]"

The agent decides when to query the Knowledge Base based on the KB instruction you provide. If the question can be answered without domain data (general knowledge, action group results), the agent skips the KB query.

🔧 What You Need

This post assumes you have:

A Bedrock Agent deployed (from Day 9)
A Knowledge Base with synced data (from Series 2, RAG Posts)

We're connecting the two. If you haven't built either, refer back to the earlier posts for the full setup.

🔧 Terraform: Associate Knowledge Base with Agent

The association is a single Terraform resource:

# agent/kb_association.tf

resource "aws_bedrockagent_agent_knowledge_base_association" "this" {
  agent_id             = aws_bedrockagent_agent.this.agent_id
  description          = var.kb_instruction
  knowledge_base_id    = var.knowledge_base_id
  knowledge_base_state = "ENABLED"
}

That's it. One resource. The description field is the most important part - it tells the agent when and how to use the Knowledge Base.

The KB Instruction

# variables.tf

variable "kb_instruction" {
  description = "Instruction telling the agent when to use the Knowledge Base"
  type        = string
  default     = <<-EOT
    Use this knowledge base to answer questions about company policies, 
    product information, and internal procedures. Query the knowledge 
    base when the user asks about policies, returns, warranties, 
    pricing, or any company-specific information. Do not answer these 
    questions from general knowledge.
  EOT
}

This instruction drives retrieval quality. A vague instruction like "use this for questions" means the agent queries the KB for everything, wasting tokens on general knowledge questions. A specific instruction like the one above targets retrieval to domain-specific queries only.

Re-Prepare After Association

The agent must be re-prepared after associating a Knowledge Base:

resource "null_resource" "prepare_after_kb" {
  triggers = {
    kb_association = aws_bedrockagent_agent_knowledge_base_association.this.id
    kb_instruction = var.kb_instruction
  }

  provisioner "local-exec" {
    command = "aws bedrock-agent prepare-agent --agent-id ${aws_bedrockagent_agent.this.agent_id} --region ${var.region}"
  }

  depends_on = [aws_bedrockagent_agent_knowledge_base_association.this]
}

IAM: Agent Needs KB Access

The agent's service role needs permission to retrieve from the Knowledge Base:

# agent/iam.tf

resource "aws_iam_role_policy" "agent_kb_retrieve" {
  name = "kb-retrieve"
  role = aws_iam_role.agent.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = [
        "bedrock:Retrieve",
        "bedrock:RetrieveAndGenerate"
      ]
      Resource = "arn:aws:bedrock:${var.region}:${data.aws_caller_identity.current.account_id}:knowledge-base/${var.knowledge_base_id}"
    }]
  })
}

📐 Full Agent with Action Groups + Knowledge Base

Here's the complete picture - an agent with both tools and grounding:

# agent/main.tf

resource "aws_bedrockagent_agent" "this" {
  agent_name              = "${var.environment}-${var.agent_name}"
  agent_resource_role_arn = aws_iam_role.agent.arn
  foundation_model        = var.agent_model.id
  instruction             = var.agent_instruction
  prepare_agent           = false  # Prepare manually after KB + action groups
  idle_session_ttl_in_seconds = var.idle_session_ttl

  tags = {
    Environment = var.environment
    Model       = var.agent_model.display
  }
}

# Action group for real-time API calls
resource "aws_bedrockagent_agent_action_group" "api_tools" {
  action_group_name = "APITools"
  agent_id          = aws_bedrockagent_agent.this.agent_id
  agent_version     = "DRAFT"
  description       = "Real-time API operations"

  action_group_executor {
    lambda = aws_lambda_function.api_tools.arn
  }

  api_schema {
    payload = file("${path.module}/schemas/api_tools.json")
  }
}

# Knowledge Base for document grounding
resource "aws_bedrockagent_agent_knowledge_base_association" "docs" {
  agent_id             = aws_bedrockagent_agent.this.agent_id
  description          = var.kb_instruction
  knowledge_base_id    = var.knowledge_base_id
  knowledge_base_state = "ENABLED"
}

# Prepare after everything is wired up
resource "null_resource" "prepare_agent" {
  triggers = {
    action_group = aws_bedrockagent_agent_action_group.api_tools.id
    kb           = aws_bedrockagent_agent_knowledge_base_association.docs.id
  }

  provisioner "local-exec" {
    command = "aws bedrock-agent prepare-agent --agent-id ${aws_bedrockagent_agent.this.agent_id} --region ${var.region}"
  }

  depends_on = [
    aws_bedrockagent_agent_action_group.api_tools,
    aws_bedrockagent_agent_knowledge_base_association.docs,
  ]
}

resource "time_sleep" "wait_for_preparation" {
  depends_on      = [null_resource.prepare_agent]
  create_duration = "15s"
}

resource "aws_bedrockagent_agent_alias" "live" {
  agent_alias_name = "${var.environment}-live"
  agent_id         = aws_bedrockagent_agent.this.agent_id
  depends_on       = [time_sleep.wait_for_preparation]
}

The agent now has two capabilities: action groups for real-time API calls (live exchange rates, order lookups) and Knowledge Base for document-grounded answers (policies, product info). The agent decides which to use based on the user's question.

🧪 How the Agent Decides

When a user asks a question, the agent follows this decision flow:

Can I answer from general knowledge? If yes, respond directly.
Does this need a tool call? If the question requires live data (exchange rates, weather), invoke the action group.
Does this need domain knowledge? If the question is about company-specific information, query the Knowledge Base.
Do I need both? For complex questions, the agent can combine action group results with KB context.

The KB instruction and action group descriptions guide these decisions. Clear, non-overlapping descriptions produce the best routing.

📐 Multiple Knowledge Bases

An agent can have multiple Knowledge Bases, each for a different domain:

resource "aws_bedrockagent_agent_knowledge_base_association" "policies" {
  agent_id          = aws_bedrockagent_agent.this.agent_id
  description       = "Use for questions about company policies, returns, and warranties."
  knowledge_base_id = var.policies_kb_id
  knowledge_base_state = "ENABLED"
}

resource "aws_bedrockagent_agent_knowledge_base_association" "products" {
  agent_id          = aws_bedrockagent_agent.this.agent_id
  description       = "Use for questions about product specifications, features, and compatibility."
  knowledge_base_id = var.products_kb_id
  knowledge_base_state = "ENABLED"
}

Each KB instruction should clearly define its domain so the agent routes to the right KB. Overlapping instructions lead to incorrect retrieval.

⚠️ Gotchas and Tips

KB instruction quality is retrieval quality. A vague instruction means the agent queries the KB for everything, adding latency and cost to questions it could answer from general knowledge. Be specific about what types of questions should trigger a KB query.

Citations come free. When the agent uses KB context, the response includes source attributions with document names and page numbers. You don't need to build citation logic.

KB + action groups don't conflict. The agent can use both in a single conversation turn. For example: "Check my order status and tell me the return policy" might trigger an action group for order status and a KB query for the return policy.

Sync your KB before testing. The association works immediately, but the agent retrieves from whatever is currently indexed. If you haven't synced your data source recently, you'll get stale or empty results.

KB instruction updates require re-preparation. Changing the description field on the association triggers a Terraform update, but the agent must be re-prepared for the change to take effect.

⏭️ Series Complete!

This is Post 4 of the AWS AI Agents with Terraform series, and the final post of the entire multi-cloud series.

Post 1: Deploy First Bedrock Agent 🤖
Post 2: Action Groups - Connect to APIs 🔌
Post 3: Multi-Agent Orchestration 🧠
Post 4: Agent + Knowledge Base Grounding (you are here) 📚

Your agent is now grounded. It reasons through complex requests, calls APIs for live data, coordinates with specialist agents, and answers domain questions from your actual documents with citations. From a simple endpoint to a fully autonomous, grounded AI agent - all in Terraform. 📚

Thanks for following the full series! Follow for what comes next. 💬

Multi-Agent Orchestration on Azure: Workflow Patterns with Agent Framework and Terraform 🧠

Suhas Mallesh — Mon, 23 Mar 2026 07:00:00 +0000

Single agents hit their limits on complex workflows. Microsoft Agent Framework gives you sequential, concurrent, handoff, and group chat orchestration patterns for multi-agent systems. Here's how to build them with Terraform provisioning the infrastructure.

In the previous posts, we deployed a single Azure AI agent with function calling. That handles focused tasks. But real business processes span multiple domains: a customer onboarding flow needs compliance checking, document processing, account creation, and welcome messaging. One agent with all those responsibilities performs poorly.

Microsoft Agent Framework is the open-source SDK (successor to Semantic Kernel and AutoGen) that provides orchestration patterns for multi-agent systems. You define specialized agents, then compose them into workflows - sequential pipelines, concurrent execution, handoff chains, and group chat patterns. Terraform provisions the Azure infrastructure; Python defines the agent team. 🎯

🏗️ Three Agent Types in Foundry

Before diving into orchestration, understand the three agent types Foundry Agent Service supports:

Agent Type	Definition	Orchestration	Best For
Prompt agents	Configuration only (portal/API)	Managed by Foundry	Rapid prototyping, simple tasks
Workflow agents	Visual/YAML definitions	Sequential, branching, group chat	Multi-step business processes
Hosted agents	Your code in a container	Full custom control	Complex multi-agent systems

This post focuses on building multi-agent systems with Agent Framework's orchestration patterns, deployable as hosted agents or run locally.

🔧 Four Orchestration Patterns

Pattern 1: Sequential Pipeline

Agents run one after another. The output of one feeds the next:

from agent_framework.azure import AzureOpenAIChatClient
from agent_framework.orchestrations import SequentialBuilder
from azure.identity import AzureCliCredential

client = AzureOpenAIChatClient(credential=AzureCliCredential())

researcher = client.as_agent(
    name="researcher",
    instructions="Research the given topic thoroughly. Provide detailed findings.",
)

writer = client.as_agent(
    name="writer",
    instructions="Based on the research provided, write a clear blog post draft.",
)

editor = client.as_agent(
    name="editor",
    instructions="Review the draft for clarity, grammar, and tone. Output the final version.",
)

# Build sequential pipeline
pipeline = (
    SequentialBuilder()
    .add(researcher)
    .add(writer)
    .add(editor)
    .build()
)

result = await pipeline.invoke("Write about the impact of AI on healthcare")
print(result)

Each agent receives the previous agent's output as context. The pipeline produces a final result after all agents have processed.

Pattern 2: Concurrent Execution

Independent tasks run simultaneously for speed:

from agent_framework.orchestrations import ConcurrentBuilder

energy_analyst = client.as_agent(
    name="energy_analyst",
    instructions="Analyze renewable energy market trends.",
)

ev_analyst = client.as_agent(
    name="ev_analyst",
    instructions="Analyze electric vehicle adoption trends.",
)

# Run both analysts concurrently
concurrent = (
    ConcurrentBuilder()
    .add(energy_analyst)
    .add(ev_analyst)
    .build()
)

results = await concurrent.invoke("Analyze 2025 sustainability trends")
# results contains outputs from both analysts

Use concurrent execution when tasks are independent. Both agents run at the same time, reducing total latency.

Pattern 3: Handoff Chain

One agent transfers control to another based on the conversation:

from agent_framework.orchestrations import HandoffBuilder

order_agent = client.as_agent(
    name="order_agent",
    instructions="""You handle order lookups and cancellations.
    If the customer asks about payments or refunds,
    transfer to payments_agent.""",
)

payments_agent = client.as_agent(
    name="payments_agent",
    instructions="""You handle refunds and billing inquiries.
    If the customer asks about order status,
    transfer to order_agent.""",
)

# Build handoff chain
handoff = (
    HandoffBuilder()
    .add(order_agent)
    .add(payments_agent)
    .build()
)

result = await handoff.invoke(
    "I need to cancel order #1234 and get a refund"
)

The handoff pattern is ideal for customer support where the conversation naturally moves between domains. Each agent decides when to transfer based on its instructions.

Pattern 4: Group Chat

Multiple agents collaborate on a shared conversation:

from agent_framework.orchestrations import GroupChatBuilder

analyst = client.as_agent(
    name="analyst",
    instructions="You analyze data and provide insights. Contribute your analysis to the discussion.",
)

strategist = client.as_agent(
    name="strategist",
    instructions="You develop strategies based on analysis. Build on the analyst's insights.",
)

critic = client.as_agent(
    name="critic",
    instructions="You challenge assumptions and identify risks. Push back on weak reasoning.",
)

# Build group chat
group = (
    GroupChatBuilder()
    .add(analyst)
    .add(strategist)
    .add(critic)
    .set_max_rounds(3)
    .build()
)

result = await group.invoke("Should we expand into the European market?")

Group chat creates a collaborative discussion where agents build on each other's contributions. max_rounds limits the conversation length. The result is the synthesized output after all rounds complete.

🔧 Terraform: Infrastructure for Multi-Agent

Terraform provisions the Foundry resource, model deployments, and supporting infrastructure. The multi-agent logic lives in Python:

# agents/deployments.tf

# Powerful model for the supervisor/orchestrator
resource "azurerm_cognitive_deployment" "orchestrator_model" {
  name                 = "${var.environment}-orchestrator"
  cognitive_account_id = azurerm_cognitive_account.ai_foundry.id

  sku {
    name     = var.orchestrator_model.sku
    capacity = var.orchestrator_model.tpm
  }

  model {
    format  = "OpenAI"
    name    = var.orchestrator_model.name
    version = var.orchestrator_model.version
  }
}

# Cost-effective model for specialist agents
resource "azurerm_cognitive_deployment" "specialist_model" {
  name                 = "${var.environment}-specialist"
  cognitive_account_id = azurerm_cognitive_account.ai_foundry.id

  sku {
    name     = var.specialist_model.sku
    capacity = var.specialist_model.tpm
  }

  model {
    format  = "OpenAI"
    name    = var.specialist_model.name
    version = var.specialist_model.version
  }
}

Environment Configuration

# environments/prod.tfvars

orchestrator_model = {
  name    = "gpt-4o"
  version = "2024-11-20"
  sku     = "GlobalStandard"
  tpm     = 80
}

specialist_model = {
  name    = "gpt-4o-mini"
  version = "2024-07-18"
  sku     = "GlobalStandard"
  tpm     = 120
}

Use the powerful model for orchestration decisions and complex reasoning. Use the cheaper model for specialist agents that execute focused tasks. The cost difference adds up fast when multiple agents process every request.

📐 Combining Patterns

Real systems nest patterns. A customer support pipeline might use sequential processing with concurrent data gathering inside:

# Concurrent: gather order + payment data simultaneously
data_gathering = (
    ConcurrentBuilder()
    .add(order_lookup_agent)
    .add(payment_lookup_agent)
    .build()
)

# Sequential: gather data, then draft response, then review
full_pipeline = (
    SequentialBuilder()
    .add_orchestration(data_gathering)
    .add(response_drafter)
    .add(quality_reviewer)
    .build()
)

result = await full_pipeline.invoke("Customer asking about order #1234 refund status")

🔧 Workflow Agents in Foundry Portal

For teams that prefer visual orchestration over code, Foundry Agent Service supports workflow agents defined through the portal or YAML. These are declarative definitions that support:

Sequential and branching execution paths
Human-in-the-loop approval steps
Agent-to-agent coordination with context sharing
Built-in error handling and retries

Workflow agents are managed by Foundry with no custom hosting needed. Use them when the orchestration logic is straightforward and you want zero infrastructure management.

⚠️ Gotchas and Tips

Agent instructions drive routing quality. In handoff patterns, agents decide when to transfer based on their instructions. Vague instructions lead to agents trying to handle everything themselves. Be explicit about boundaries: "If the customer asks about X, transfer to Y_agent."

Cost multiplier. Every agent in a pipeline or group chat consumes tokens independently. A 4-agent sequential pipeline uses roughly 4x the tokens of a single agent. Use gpt-4o-mini for specialist agents to keep costs manageable.

Group chat rounds. Always set max_rounds in group chat patterns. Without it, agents can loop indefinitely, debating each other and burning tokens.

State management. Agent Framework provides session-based state management for long-running workflows. For workflows that span minutes or hours, leverage the built-in state persistence rather than building your own.

Streaming support. All orchestration patterns support streaming. For user-facing applications, stream the final agent's output while the pipeline processes internally.

⏭️ What's Next

This is Post 3 of the Azure AI Agents with Terraform series.

Post 1: Deploy First Azure AI Agent 🤖
Post 2: Function Calling - Connect to APIs 🔌
Post 3: Multi-Agent Orchestration (you are here) 🧠
Post 4: Agent + Bing Grounding

Your single agent is now a team. Sequential pipelines, concurrent fan-out, handoff chains, and group chat debates - Agent Framework gives you the patterns. Terraform provisions the infrastructure. Python defines the workflow. 🧠

Found this helpful? Follow for the full AI Agents with Terraform series! 💬

Free AI Courses

Suhas Mallesh — Mon, 23 Mar 2026 01:11:45 +0000

Found this on LinkedIn and found these be really useful. So sharing this to help you all on your AI learning journey!

The best AI courses from industry leaders are completely free. Here is the list of top-tier courses:

Top 10 Free AI Courses:

Anthropic: http://anthropic.skilljar.com

Google: http://grow.google/ai

Meta: http://ai.meta.com/resources

NVIDIA: http://developer.nvidia.com/training

Microsoft: http://learn.microsoft.com/training

OpenAI: http://academy.openai.com

IBM: http://skillsbuild.org

AWS: http://skillbuilder.aws

DeepLearningAI: http://deeplearning.ai

Hugging Face: http://huggingface.co/learn

Don't break the bank on AI education. The best ones are from the industry leaders and they are free.

Original board by Andreas Horn.

Multi-Agent Orchestration on AWS: Supervisor Pattern with Terraform 🧠

Suhas Mallesh — Sun, 22 Mar 2026 07:00:00 +0000

One agent handles simple tasks. Complex workflows need a team. Bedrock's multi-agent collaboration lets a supervisor agent break down problems, delegate to specialists, and combine results. Here's how to build it with Terraform.

In the previous posts, we deployed a single Bedrock agent with action groups. That works for focused tasks. But real workflows are multi-domain: a customer support request might need to check an order, look up a policy, and escalate to a specialist. A single agent with 15 action groups performs poorly because the model struggles to select the right tool from too many options.

Multi-agent collaboration solves this. You create specialized agents, each focused on one domain, and a supervisor agent that routes requests, delegates tasks, and combines results. The supervisor reads the user's intent and decides which specialist to call. Each specialist stays focused and performs better than one overloaded generalist. Now GA on Amazon Bedrock. 🎯

🏗️ How Multi-Agent Collaboration Works

User: "Cancel my order #1234 and refund to my original payment method"
    ↓
Supervisor Agent (analyzes intent, plans execution)
    ↓ delegates to
┌──────────────────────────────────────────┐
│ Order Agent          │ Payments Agent    │
│ (cancels order #1234)│ (processes refund)│
└──────────────────────────────────────────┘
    ↓ results flow back to
Supervisor Agent (combines results)
    ↓
User: "Order #1234 has been cancelled. A refund of $89.99 
       will be processed to your Visa ending in 4242 
       within 3-5 business days."

The supervisor doesn't do the work itself. It plans, delegates, and synthesizes.

🔧 Two Collaboration Modes

Mode	How It Works	Best For
Supervisor	Analyzes every request, breaks down complex tasks, coordinates multiple agents	Complex multi-step workflows
Supervisor with Routing	Routes simple requests directly to a specialist; falls back to full supervisor mode for complex queries	Mixed simple/complex traffic

Supervisor with routing is more cost-effective for production traffic where most requests are straightforward. The routing mode skips full orchestration for simple queries, reducing latency and token usage.

🔧 Terraform: Build the Team

Step 1: Create Specialist (Collaborator) Agents

Each collaborator is a standalone agent with its own model, instructions, and tools:

# agents/collaborators.tf

variable "collaborators" {
  description = "Map of specialist agents"
  type = map(object({
    instruction          = string
    collaboration_instruction = string
    model_id             = string
  }))
}

resource "aws_bedrockagent_agent" "collaborator" {
  for_each = var.collaborators

  agent_name              = "${var.environment}-${each.key}"
  agent_resource_role_arn = aws_iam_role.agent.arn
  foundation_model        = each.value.model_id
  instruction             = each.value.instruction
  agent_collaboration     = "COLLABORATOR"
  prepare_agent           = true
  idle_session_ttl_in_seconds = var.idle_session_ttl
}

resource "aws_bedrockagent_agent_alias" "collaborator" {
  for_each = var.collaborators

  agent_alias_name = "${var.environment}-live"
  agent_id         = aws_bedrockagent_agent.collaborator[each.key].agent_id
}

agent_collaboration = "COLLABORATOR" marks these agents as available for a supervisor to call. Each collaborator can have its own action groups, knowledge bases, and guardrails.

Step 2: Create the Supervisor Agent

# agents/supervisor.tf

resource "aws_bedrockagent_agent" "supervisor" {
  agent_name              = "${var.environment}-supervisor"
  agent_resource_role_arn = aws_iam_role.supervisor.arn
  foundation_model        = var.supervisor_model.id
  instruction             = var.supervisor_instruction
  agent_collaboration     = "SUPERVISOR"
  prepare_agent           = false  # Prepare AFTER collaborators are associated
  idle_session_ttl_in_seconds = var.idle_session_ttl
}

prepare_agent = false is critical for supervisors. A supervisor agent cannot be prepared until collaborators are associated. If you set this to true, Terraform fails because the supervisor has no team members yet.

Step 3: Associate Collaborators with Supervisor

# agents/associations.tf

resource "aws_bedrockagent_agent_collaborator" "this" {
  for_each = var.collaborators

  agent_id                  = aws_bedrockagent_agent.supervisor.agent_id
  collaboration_instruction = each.value.collaboration_instruction
  collaborator_name         = each.key
  relay_conversation_history = "TO_COLLABORATOR"

  agent_descriptor {
    alias_arn = aws_bedrockagent_agent_alias.collaborator[each.key].agent_alias_arn
  }

  depends_on = [aws_bedrockagent_agent.supervisor]
}

collaboration_instruction tells the supervisor when to use each collaborator. This is the routing logic. Be specific: "Handle all questions about order status, cancellations, and shipping. Do not handle payment or refund questions."

relay_conversation_history = "TO_COLLABORATOR" shares the conversation context so collaborators have full context when handling their part.

Step 4: Prepare the Supervisor

After all collaborators are associated, prepare the supervisor:

# agents/prepare.tf

resource "null_resource" "prepare_supervisor" {
  triggers = {
    collaborators = jsonencode(keys(var.collaborators))
    supervisor    = aws_bedrockagent_agent.supervisor.agent_id
  }

  provisioner "local-exec" {
    command = <<EOF
aws bedrock-agent prepare-agent \
  --agent-id ${aws_bedrockagent_agent.supervisor.agent_id} \
  --region ${var.region}
EOF
  }

  depends_on = [aws_bedrockagent_agent_collaborator.this]
}

resource "time_sleep" "wait_for_preparation" {
  depends_on      = [null_resource.prepare_supervisor]
  create_duration = "15s"
}

resource "aws_bedrockagent_agent_alias" "supervisor" {
  agent_alias_name = "${var.environment}-live"
  agent_id         = aws_bedrockagent_agent.supervisor.agent_id

  depends_on = [time_sleep.wait_for_preparation]
}

The time_sleep gives the supervisor time to reach the PREPARED state before creating the alias. In production, consider polling the agent status instead of a fixed delay.

📐 Environment Configuration

# environments/dev.tfvars

supervisor_model = {
  id      = "anthropic.claude-sonnet-4-20250514-v1:0"
  display = "Claude Sonnet 4"
}

supervisor_instruction = <<-EOT
  You are a customer support supervisor. Analyze each request
  and delegate to the appropriate specialist. For complex requests
  that span multiple domains, coordinate between specialists and
  combine their results into a clear response.
EOT

collaborators = {
  order-agent = {
    model_id    = "anthropic.claude-sonnet-4-20250514-v1:0"
    instruction = "You handle order lookups, cancellations, and shipping status. You have access to the orders database via action groups."
    collaboration_instruction = "Handle all questions about order status, cancellations, returns, and shipping. Do not handle payment or billing questions."
  }
  payments-agent = {
    model_id    = "anthropic.claude-haiku-4-20250414-v1:0"
    instruction = "You handle payment processing, refunds, and billing inquiries. You have access to the payments API via action groups."
    collaboration_instruction = "Handle all questions about payments, refunds, billing, and payment methods. Do not handle order status questions."
  }
  policy-agent = {
    model_id    = "anthropic.claude-haiku-4-20250414-v1:0"
    instruction = "You answer questions about company policies, warranty terms, and return policies based on the knowledge base."
    collaboration_instruction = "Handle all questions about company policies, warranty, and return rules. Refer to the knowledge base for accurate answers."
  }
}

Model selection per agent. Use a powerful model (Sonnet) for the supervisor that needs to plan and coordinate. Use a faster, cheaper model (Haiku) for simple specialist agents that execute focused tasks. This optimizes cost without sacrificing quality.

🧪 Invoke the Multi-Agent System

From your application, you invoke the supervisor. It handles delegation internally:

import boto3

client = boto3.client("bedrock-agent-runtime")

response = client.invoke_agent(
    agentId=supervisor_agent_id,
    agentAliasId=supervisor_alias_id,
    sessionId="user-session-456",
    inputText="Cancel my order #1234 and refund to my original payment method"
)

for event in response["completion"]:
    if "chunk" in event:
        print(event["chunk"]["bytes"].decode(), end="")

Your application calls one endpoint. The supervisor decomposes the request, delegates to the order agent and payments agent, gathers results, and returns a unified response. Zero orchestration code in your application.

⚠️ Gotchas and Tips

Prompt quality is everything. The most common failure mode is the supervisor trying to handle everything itself instead of delegating. Write clear supervisor instructions that explicitly say "delegate to specialists" and clear collaboration instructions that define non-overlapping responsibilities.

Sequential collaborator association. When adding multiple collaborators, each association triggers a preparation step. Adding them too fast can fail because the agent is in a PREPARING state. The depends_on chain in the Terraform handles this, but be aware of it for manual operations.

Mixed-model teams. Use expensive models for complex reasoning (supervisor, planning agents) and cheaper models for straightforward tasks (lookup agents, policy agents). The cost difference is significant at scale.

Conversation history sharing. Enable relay_conversation_history for collaborators that need context from previous turns. Disable it for stateless lookup agents where the context adds unnecessary tokens.

Adding new specialists. To add a new collaborator, add an entry to the collaborators variable and run terraform apply. The supervisor is re-prepared automatically and can immediately delegate to the new agent.

⏭️ What's Next

This is Post 3 of the AWS AI Agents with Terraform series.

Post 1: Deploy First Bedrock Agent 🤖
Post 2: Action Groups - Connect to APIs 🔌
Post 3: Multi-Agent Orchestration (you are here) 🧠
Post 4: Agent + Knowledge Base Grounding

Your single agent is now a team. A supervisor that plans, specialists that execute, and Terraform that makes the whole team a variable change away from a new specialist. Scale your AI by scaling your team. 🧠

Found this helpful? Follow for the full AI Agents with Terraform series! 💬

Multi-Agent Systems on GCP: Workflow Patterns with ADK and Terraform 🧠

Suhas Mallesh — Sun, 22 Mar 2026 07:00:00 +0000

ADK gives you four ways to orchestrate multi-agent systems - hierarchical delegation, sequential pipelines, parallel fan-out, and iterative loops. Here's how to build each pattern with Terraform provisioning the infrastructure.

In the previous posts, we deployed a single Vertex AI agent with tools. That handles focused tasks well. But complex workflows need multiple agents: one to research, one to write, one to review. Or one to handle orders while another handles payments.

ADK provides four orchestration primitives for building multi-agent systems. Unlike managed supervisor patterns, ADK gives you code-level control over how agents interact - sequential pipelines, parallel fan-out, iterative refinement loops, and LLM-driven delegation. Terraform provisions the infrastructure; Python defines the agent team. 🎯

🏗️ Four Multi-Agent Patterns

Pattern	Agent Type	How It Works	Best For
Hierarchy	`LlmAgent` with `sub_agents`	Parent delegates to children based on LLM reasoning	Dynamic routing, customer support
Pipeline	`SequentialAgent`	Agents run one after another, passing state	fetch → clean → analyze → summarize
Fan-out	`ParallelAgent`	Agents run concurrently, results gathered after	Independent research, multi-API calls
Refinement	`LoopAgent`	Agents repeat until quality threshold met	Draft → critique → revise cycles

Workflow agents (SequentialAgent, ParallelAgent, LoopAgent) are deterministic - they don't use an LLM for flow control. The execution pattern is fixed in code. LlmAgent with sub_agents uses the model to decide which child to invoke, making it dynamic but less predictable.

🔧 Pattern 1: Hierarchical Delegation (LLM-Driven)

A parent agent decides which specialist to delegate to based on the user's request:

from google.adk.agents import Agent

order_agent = Agent(
    name="order_agent",
    model=model,
    instruction="You handle order lookups, cancellations, and shipping status.",
    tools=[get_order_status, cancel_order],
)

payments_agent = Agent(
    name="payments_agent",
    model=model,
    instruction="You handle refunds, billing inquiries, and payment methods.",
    tools=[process_refund, get_billing_info],
)

supervisor = Agent(
    name="supervisor",
    model=model,
    instruction="""You are a customer support supervisor.
    Route order questions to order_agent.
    Route payment questions to payments_agent.
    For requests spanning both domains, coordinate between them.""",
    sub_agents=[order_agent, payments_agent],
)

The supervisor uses the model to analyze intent and delegate. sub_agents makes children permanent team members. The supervisor can transfer control to a child, and the child can escalate back.

Alternative: AgentTool (On-Demand Calling)

For agents you want to call explicitly like a function rather than delegating to:

from google.adk.tools import AgentTool

research_assistant = Agent(
    name="researcher",
    model=model,
    instruction="Research the given topic and return findings.",
    tools=[google_search],
)

main_agent = Agent(
    name="analyst",
    model=model,
    instruction="Use the research assistant to gather data, then analyze it.",
    tools=[AgentTool(research_assistant)],
)

Sub-agent vs AgentTool: A sub-agent is a permanent team member - the parent transfers control to it. An AgentTool is an on-demand consultant - the parent calls it like a function and gets results back without giving up control.

🔧 Pattern 2: Sequential Pipeline

Agents run in a fixed order, passing state between steps:

from google.adk.agents import LlmAgent
from google.adk.agents.sequential_agent import SequentialAgent

researcher = LlmAgent(
    name="researcher",
    model="gemini-2.5-flash",
    instruction="Research the given topic. Write findings clearly.",
    output_key="research_findings",
)

writer = LlmAgent(
    name="writer",
    model="gemini-2.5-flash",
    instruction="Based on {research_findings}, write a blog post draft.",
    output_key="draft",
)

editor = LlmAgent(
    name="editor",
    model="gemini-2.5-flash",
    instruction="Review {draft} for clarity and grammar. Output the final version.",
    output_key="final_post",
)

pipeline = SequentialAgent(
    name="content_pipeline",
    sub_agents=[researcher, writer, editor],
)

output_key is the communication mechanism. Each agent writes to a shared session state key. The next agent reads it via {key_name} in its instructions. No explicit data passing code needed.

🔧 Pattern 3: Parallel Fan-Out

Independent tasks run concurrently for speed:

from google.adk.agents import LlmAgent
from google.adk.agents.parallel_agent import ParallelAgent
from google.adk.agents.sequential_agent import SequentialAgent

energy_researcher = LlmAgent(
    name="energy_researcher",
    model="gemini-2.5-flash",
    instruction="Research recent developments in renewable energy.",
    output_key="energy_findings",
)

ev_researcher = LlmAgent(
    name="ev_researcher",
    model="gemini-2.5-flash",
    instruction="Research recent developments in electric vehicles.",
    output_key="ev_findings",
)

parallel_research = ParallelAgent(
    name="parallel_research",
    sub_agents=[energy_researcher, ev_researcher],
)

synthesizer = LlmAgent(
    name="synthesizer",
    model="gemini-2.5-flash",
    instruction="Combine {energy_findings} and {ev_findings} into a report.",
    output_key="final_report",
)

# Fan-out then gather
full_pipeline = SequentialAgent(
    name="research_pipeline",
    sub_agents=[parallel_research, synthesizer],
)

Each parallel agent must write to a unique output_key. They share session state but run in separate threads. Writing to the same key causes race conditions.

🔧 Pattern 4: Iterative Refinement Loop

A writer and critic repeat until quality is met:

from google.adk.agents import LlmAgent
from google.adk.agents.loop_agent import LoopAgent
from google.adk.agents.sequential_agent import SequentialAgent

writer = LlmAgent(
    name="writer",
    model="gemini-2.5-flash",
    instruction="Write or revise content based on {critic_feedback}. Output the draft.",
    output_key="current_draft",
)

critic = LlmAgent(
    name="critic",
    model="gemini-2.5-flash",
    instruction="""Review {current_draft} against these criteria:
    - Factual accuracy
    - Clear structure
    - Professional tone
    If all criteria are met, call the exit_loop tool.
    Otherwise, provide specific feedback for revision.""",
    output_key="critic_feedback",
    tools=["exit_loop"],  # Built-in tool to break the loop
)

refinement_loop = LoopAgent(
    name="refinement_loop",
    sub_agents=[writer, critic],
    max_iterations=3,
)

exit_loop is ADK's built-in escape hatch. The critic calls it when quality is acceptable. Without it, the loop runs until max_iterations. Always set max_iterations as a safety limit.

🔧 Terraform: Infrastructure for Multi-Agent

The infrastructure is the same as Day 9 - Terraform provisions APIs, service accounts, and config. The multi-agent logic lives entirely in Python:

# agents/config.tf

resource "local_file" "agent_config" {
  filename = "${path.module}/agent_source/config.json"
  content = jsonencode({
    project_id  = var.project_id
    location    = var.region
    models = {
      supervisor = var.supervisor_model.id
      specialist = var.specialist_model.id
    }
    agent_name  = "${var.environment}-multi-agent"
  })
}

Use different models for different roles:

# environments/prod.tfvars
supervisor_model = {
  id      = "gemini-2.5-pro"
  display = "Gemini 2.5 Pro"
}
specialist_model = {
  id      = "gemini-2.5-flash"
  display = "Gemini 2.5 Flash"
}

Pro for the supervisor that needs complex reasoning. Flash for specialists that execute focused tasks. Cost optimization without sacrificing quality.

📐 Combining Patterns

Real systems combine multiple patterns. A customer support pipeline might use all four:

# Parallel: gather order + payment data simultaneously
data_gathering = ParallelAgent(
    name="data_gathering",
    sub_agents=[order_lookup, payment_lookup],
)

# Sequential: gather data, then draft response
response_pipeline = SequentialAgent(
    name="response_pipeline",
    sub_agents=[data_gathering, response_drafter],
)

# Loop: draft and review until quality passes
quality_loop = LoopAgent(
    name="quality_check",
    sub_agents=[response_pipeline, quality_reviewer],
    max_iterations=2,
)

# Root: LLM-driven routing to the right workflow
root_agent = Agent(
    name="support_root",
    model=model,
    instruction="Route customer requests to the appropriate workflow.",
    sub_agents=[quality_loop, escalation_agent],
)

⚠️ Gotchas and Tips

State key collisions. Parallel agents sharing the same output_key create race conditions. Always use unique keys per agent.

Loop safety. Always set max_iterations on LoopAgent. Without it, a critic that never approves runs forever (or until your token budget runs out).

Model cost in loops. Each iteration of a LoopAgent consumes tokens for every sub-agent. Three iterations with two agents means six model calls. Use max_iterations conservatively and cheap models for critics.

Workflow agents are deterministic. SequentialAgent, ParallelAgent, and LoopAgent don't use an LLM for flow control. The execution order is fixed in code. Only LlmAgent with sub_agents uses the model to decide routing.

Test locally first. ADK's CLI (adk run) and dev UI let you visualize multi-agent execution flows, inspect state at each step, and debug routing decisions before deploying to Agent Engine.

⏭️ What's Next

This is Post 3 of the GCP AI Agents with Terraform series.

Post 1: Deploy First Vertex AI Agent 🤖
Post 2: Agent Tools - Connect to APIs 🔌
Post 3: Multi-Agent Systems (you are here) 🧠
Post 4: Agent + Google Search Grounding

Your single agent is now a team. Sequential pipelines, parallel fan-out, iterative refinement, and LLM-driven delegation - ADK gives you the building blocks. Terraform provisions the infrastructure. Python defines the workflow. 🧠

Found this helpful? Follow for the full AI Agents with Terraform series! 💬

Azure AI Agent Function Calling: Connect Your Agent to APIs with Terraform 🔌

Suhas Mallesh — Sat, 21 Mar 2026 07:00:00 +0000

An Azure AI agent without tools is just a chatbot. Function calling gives your agent the ability to invoke your code when it needs real data or actions. Here's how to define functions, handle tool calls, and wire up the infrastructure with Terraform.

In the previous post, we deployed an Azure AI agent that can reason and hold multi-turn conversations. But it can only generate text from what the model already knows. Ask it for a live exchange rate or your account balance, and it either hallucinates or admits it doesn't know.

Function calling changes that. You define functions with descriptions and parameters. When the agent determines it needs data or an action, it returns a structured function call request. Your code executes the function and sends the result back. The agent then uses that real data to generate its response. The model decides what to call; your code does the work. 🎯

🏗️ How Function Calling Works

User: "What's the exchange rate from USD to EUR?"
    ↓
Agent reads function definitions → selects get_exchange_rate
    ↓
Agent returns: call get_exchange_rate(currency_from="USD", currency_to="EUR")
    ↓
Your code executes the function, calls external API
    ↓
Your code sends result back to the agent
    ↓
Agent: "The current rate is 1 USD = 0.92 EUR"

Unlike AWS where a Lambda function executes remotely, Azure function calling is a round-trip: the agent proposes the call, your application code executes it locally, and you submit the output back. This gives you full control over execution, error handling, and security.

🔧 Azure Tool Types

The Foundry Agent Service offers several tool categories:

Tool Type	What It Does	Best For
Function calling	Your code executes locally	Custom logic, internal APIs
Azure Functions	Hosted serverless execution	Stateful, long-running tools
OpenAPI Spec tool	Auto-generated from spec	Existing documented APIs
Code Interpreter	Agent writes and runs code	Math, data analysis, charts
File Search	Searches uploaded documents	RAG over attached files
Bing Grounding	Real-time web search	Current events, live data

This post focuses on function calling, the most flexible and common pattern.

🔧 Terraform: Provision the Infrastructure

The Foundry resource, project, and model deployment from Day 9 remain the same. Add Secret Manager for API keys your tools need:

# agent/secrets.tf

resource "azurerm_key_vault" "agent" {
  name                       = "${var.environment}-agent-kv"
  location                   = azurerm_resource_group.this.location
  resource_group_name        = azurerm_resource_group.this.name
  tenant_id                  = data.azurerm_client_config.current.tenant_id
  sku_name                   = "standard"
  enable_rbac_authorization  = true
  purge_protection_enabled   = false
}

resource "azurerm_key_vault_secret" "tool_api_keys" {
  for_each     = var.tool_api_keys
  name         = each.key
  value        = each.value
  key_vault_id = azurerm_key_vault.agent.id
}

# Grant your app access to read secrets
resource "azurerm_role_assignment" "kv_reader" {
  scope                = azurerm_key_vault.agent.id
  role_definition_name = "Key Vault Secrets User"
  principal_id         = data.azurerm_client_config.current.object_id
}

Config Output with Tool Definitions

# agent/config.tf

resource "local_file" "agent_config" {
  filename = "${path.module}/agent_code/agent_config.json"
  content = jsonencode({
    foundry_endpoint = azurerm_cognitive_account.ai_foundry.endpoint
    deployment_name  = azurerm_cognitive_deployment.agent_model.name
    agent_name       = "${var.environment}-${var.agent_name}"
    instruction      = var.agent_instruction
    key_vault_name   = azurerm_key_vault.agent.name
  })
}

🐍 Define Functions

Define your functions as plain Python, then describe them as tool definitions for the agent:

# agent_code/tools.py

import json
import urllib.request

def get_exchange_rate(currency_from: str, currency_to: str) -> str:
    """Get the current exchange rate between two currencies."""
    url = f"https://api.frankfurter.dev/v1/latest?base={currency_from}&symbols={currency_to}"
    req = urllib.request.Request(url)
    with urllib.request.urlopen(req) as resp:
        data = json.loads(resp.read())
    rate = data["rates"].get(currency_to, "N/A")
    return json.dumps({"rate": rate, "from": currency_from, "to": currency_to})


def get_weather(city: str, units: str = "celsius") -> str:
    """Get current weather for a city."""
    # In production, call a real weather API
    return json.dumps({"city": city, "temp": 22, "units": units, "condition": "sunny"})


# Map function names to implementations
TOOL_FUNCTIONS = {
    "get_exchange_rate": get_exchange_rate,
    "get_weather": get_weather,
}

🐍 Create Agent with Function Tools

# agent_code/create_agent.py

from azure.ai.agents import AgentsClient
from azure.ai.agents.models import FunctionTool
from azure.identity import DefaultAzureCredential
import json

with open("agent_config.json") as f:
    config = json.load(f)

client = AgentsClient(
    endpoint=config["foundry_endpoint"],
    credential=DefaultAzureCredential(),
)

# Define function tool schemas
exchange_rate_tool = FunctionTool(
    name="get_exchange_rate",
    description="Get the current exchange rate between two currencies. Use when the user asks about currency conversion.",
    parameters={
        "type": "object",
        "properties": {
            "currency_from": {
                "type": "string",
                "description": "Source currency code (e.g., USD, EUR, GBP)"
            },
            "currency_to": {
                "type": "string",
                "description": "Target currency code (e.g., USD, EUR, GBP)"
            }
        },
        "required": ["currency_from", "currency_to"]
    }
)

weather_tool = FunctionTool(
    name="get_weather",
    description="Get current weather for a city. Use when the user asks about weather conditions.",
    parameters={
        "type": "object",
        "properties": {
            "city": {
                "type": "string",
                "description": "City name (e.g., Seattle, London, Tokyo)"
            },
            "units": {
                "type": "string",
                "description": "Temperature units: celsius or fahrenheit",
                "default": "celsius"
            }
        },
        "required": ["city"]
    }
)

# Create agent with tools
agent = client.create_agent(
    model=config["deployment_name"],
    name=config["agent_name"],
    instructions=config["instruction"],
    tools=[exchange_rate_tool, weather_tool],
)

print(f"Agent created: {agent.id}")

The description fields drive tool selection. The agent reads these to decide which function to call. Be specific about when each function should and should not be used.

🐍 Handle the Function Call Loop

The key difference from simple chat: you need a loop to handle tool calls:

# agent_code/run_agent.py

from azure.ai.agents import AgentsClient
from azure.identity import DefaultAzureCredential
from tools import TOOL_FUNCTIONS
import json

client = AgentsClient(
    endpoint=config["foundry_endpoint"],
    credential=DefaultAzureCredential(),
)

# Create a thread and add user message
thread = client.threads.create()
client.messages.create(
    thread_id=thread.id,
    role="user",
    content="What's the exchange rate from USD to JPY?"
)

# Create a run
run = client.runs.create(thread_id=thread.id, agent_id=agent.id)

# Poll for completion, handling tool calls
while run.status in ("queued", "in_progress", "requires_action"):
    if run.status == "requires_action":
        tool_outputs = []

        for tool_call in run.required_action.submit_tool_outputs.tool_calls:
            func_name = tool_call.function.name
            func_args = json.loads(tool_call.function.arguments)

            # Execute the function locally
            if func_name in TOOL_FUNCTIONS:
                result = TOOL_FUNCTIONS[func_name](**func_args)
            else:
                result = json.dumps({"error": f"Unknown function: {func_name}"})

            tool_outputs.append({
                "tool_call_id": tool_call.id,
                "output": result
            })

        # Submit results back to the agent
        run = client.runs.submit_tool_outputs(
            thread_id=thread.id,
            run_id=run.id,
            tool_outputs=tool_outputs
        )
    else:
        import time
        time.sleep(1)
        run = client.runs.retrieve(thread_id=thread.id, run_id=run.id)

# Get the final response
messages = client.messages.list(thread_id=thread.id)
for msg in messages:
    if msg.role == "assistant":
        print(msg.content[0].text.value)

The requires_action status is the key. When the agent decides to call a function, the run pauses with this status. Your code executes the function, submits the output, and the run continues. The agent can chain multiple function calls in a single run.

⚠️ Gotchas and Tips

10-minute run expiration. Runs expire 10 minutes after creation. If your functions take longer, consider Azure Functions (hosted execution) instead of local function calling.

Return JSON strings. Function outputs must be strings. Always json.dumps() your return values, even for simple responses.

Multiple tool calls per run. The agent can request several function calls in a single requires_action step. Always iterate over all tool_calls and submit outputs for each.

Error handling. If a function fails, return an error message as the output rather than raising an exception. The agent can reason about the error and try a different approach.

OpenAPI Spec tool alternative. If you have existing APIs with OpenAPI documentation, use the OpenAPI Spec tool instead of manually defining function schemas. The agent auto-generates tool definitions from the spec, similar to ADK's OpenAPIToolset.

Azure Functions for stateful tools. For tools that need database connections, queue access, or long execution times, use Azure Functions as the execution backend. The agent calls the hosted function directly without your application code in the loop.

⏭️ What's Next

This is Post 2 of the Azure AI Agents with Terraform series.

Post 1: Deploy First Azure AI Agent 🤖
Post 2: Function Calling - Connect to APIs (you are here) 🔌
Post 3: Multi-Agent with AI Agent Service
Post 4: Agent + Bing Grounding

Your agent can now take action. Define functions, describe their parameters, and the agent decides when to call them. The model reasons; your code executes. Real data, real actions, real results. 🔌

Found this helpful? Follow for the full AI Agents with Terraform series! 💬

Vertex AI Agent Tools: Connect Your Agent to APIs with ADK and Terraform 🔌

Suhas Mallesh — Fri, 20 Mar 2026 07:00:00 +0000

An agent without tools is just a chatbot. ADK gives your Vertex AI agent five ways to connect to APIs - function tools, OpenAPI tools, MCP tools, built-in tools, and agent-as-tool. Here's how to wire them up with Terraform provisioning the infrastructure.

In the previous post, we deployed a Vertex AI agent with ADK and Agent Engine. It can reason and hold conversations, but without tools it can only generate text. Ask it to check a stock price or create a ticket, and it describes what it would do instead of doing it.

ADK's tool ecosystem is where GCP agents shine. Unlike AWS where action groups require Lambda functions and OpenAPI schemas as separate resources, ADK lets you define tools as Python functions, OpenAPI specs, MCP servers, or even other agents - all in the same codebase. The agent decides which tool to use based on the user's request, and ADK handles the execution. Terraform provisions the infrastructure; Python defines the capabilities. 🎯

🔧 Five Tool Types in ADK

Tool Type	What It Does	Best For
Function Tool	Plain Python function	Custom logic, calculations, internal APIs
OpenAPI Tool	Auto-generated from OpenAPI spec	Existing REST APIs with documentation
MCP Tool	Connects to MCP servers	Third-party services (GitHub, Slack, DBs)
Built-in Tool	Google Search, Code Execution	Web search, running generated code
Agent Tool	Uses another agent as a tool	Delegation, specialized sub-tasks

This post focuses on function tools and OpenAPI tools - the two patterns that connect your agent to your own APIs.

🔧 Function Tools: Python Functions as Agent Capabilities

The simplest way to give an agent a capability. Write a Python function with a docstring, and ADK makes it available as a tool:

# agent_source/tools.py

import urllib.request
import json

def get_exchange_rate(
    currency_from: str,
    currency_to: str,
) -> dict:
    """Get the current exchange rate between two currencies.

    Use this when the user asks about currency conversion or exchange rates.

    Args:
        currency_from: The source currency code (e.g., USD, EUR, GBP).
        currency_to: The target currency code (e.g., USD, EUR, GBP).

    Returns:
        A dictionary with the exchange rate.
    """
    url = f"https://api.frankfurter.dev/v1/latest?base={currency_from}&symbols={currency_to}"
    req = urllib.request.Request(url)
    with urllib.request.urlopen(req) as resp:
        data = json.loads(resp.read())

    rate = data["rates"].get(currency_to, "N/A")
    return {"rate": rate, "from": currency_from, "to": currency_to}

The docstring is the schema. ADK uses type hints for parameter types and the docstring for descriptions. The agent reads these to decide when and how to call the function. Clear docstrings drive better tool selection, just like OpenAPI descriptions in AWS action groups.

Wire Tools to the Agent

# agent_source/agent.py

import json
from google.adk.agents import Agent
from google.adk.models.vertexai import VertexAi
from tools import get_exchange_rate

with open("config.json") as f:
    config = json.load(f)

model = VertexAi(model=config["model_id"])

agent = Agent(
    name=config["agent_name"],
    model=model,
    instruction=config["instruction"],
    tools=[get_exchange_rate],  # Register tools here
)

That's it. No Lambda function, no OpenAPI schema file, no IAM permissions for tool invocation. The function runs in the same process as the agent.

Multiple Function Tools

Add as many tools as needed. The agent decides which to call:

from tools import get_exchange_rate, get_weather, create_ticket

agent = Agent(
    name=config["agent_name"],
    model=model,
    instruction="""You are a helpful assistant.
    Use get_exchange_rate for currency questions.
    Use get_weather for weather queries.
    Use create_ticket for support requests.""",
    tools=[get_exchange_rate, get_weather, create_ticket],
)

Instruction quality matters. Tell the agent when to use each tool. Without guidance, the model guesses based on docstrings alone - explicit instructions improve tool selection accuracy.

🔧 OpenAPI Tools: Auto-Generate from API Specs

For existing REST APIs with OpenAPI documentation, ADK can generate tools automatically:

# agent_source/agent_with_openapi.py

from google.adk.agents import Agent
from google.adk.models.vertexai import VertexAi
from google.adk.tools.openapi_tool.openapi_spec_parser.openapi_toolset import OpenAPIToolset

model = VertexAi(model=config["model_id"])

# Load OpenAPI spec and generate tools
petstore_toolset = OpenAPIToolset(
    spec_str=open("petstore_openapi.yaml").read(),
    spec_str_type="yaml",
)

agent = Agent(
    name="api-agent",
    model=model,
    instruction="You manage the pet store. Use the API tools to list, create, and look up pets.",
    tools=[petstore_toolset],
)

OpenAPIToolset parses the spec and creates one RestApiTool per operation. The agent sees list_pets, create_pet, get_pet_by_id as separate callable tools. When the agent decides to call one, ADK handles the HTTP request automatically.

OpenAPI Tool with Authentication

For APIs requiring auth, pass credentials to the toolset:

from google.adk.tools.openapi_tool.auth.auth_helpers import token_to_scheme_credential

toolset = OpenAPIToolset(
    spec_str=open("my_api.yaml").read(),
    spec_str_type="yaml",
    auth_scheme_credential=token_to_scheme_credential(
        "bearer", "my-api-token"
    ),
)

🔧 Terraform: Infrastructure for Tool-Connected Agents

Terraform provisions the APIs, IAM, and any GCP services your tools need. The tool logic stays in Python.

APIs for External Connectivity

# agent/apis.tf

resource "google_project_service" "agent_tools" {
  for_each = toset([
    "aiplatform.googleapis.com",
    "cloudfunctions.googleapis.com",
    "secretmanager.googleapis.com",  # For API keys
    "run.googleapis.com",            # For Agent Engine
  ])
  project = var.project_id
  service = each.value
}

Store API Keys in Secret Manager

# agent/secrets.tf

resource "google_secret_manager_secret" "api_keys" {
  for_each  = var.tool_api_keys
  secret_id = "${var.environment}-${each.key}-api-key"
  project   = var.project_id

  replication {
    auto {}
  }
}

resource "google_secret_manager_secret_version" "api_keys" {
  for_each    = var.tool_api_keys
  secret      = google_secret_manager_secret.api_keys[each.key].id
  secret_data = each.value
}

# Grant agent SA access to secrets
resource "google_secret_manager_secret_iam_member" "agent_access" {
  for_each  = var.tool_api_keys
  secret_id = google_secret_manager_secret.api_keys[each.key].id
  role      = "roles/secretmanager.secretAccessor"
  member    = "serviceAccount:${google_service_account.agent.email}"
}

Config with Tool Settings

# agent/config.tf

resource "local_file" "agent_config" {
  filename = "${path.module}/agent_source/config.json"
  content = jsonencode({
    project_id  = var.project_id
    location    = var.region
    model_id    = var.agent_model.id
    agent_name  = "${var.environment}-${var.agent_name}"
    instruction = var.agent_instruction
    tool_config = {
      api_key_secret = "${var.environment}-exchange-rate-api-key"
    }
  })
}

📐 Sequential Tool Use

ADK supports chaining tools where one tool's output feeds another. Describe the sequence in the agent's instructions:

agent = Agent(
    name="research-agent",
    model=model,
    instruction="""You are a research assistant.
    When the user asks for analysis:
    1. First use search_documents to find relevant data
    2. Then use summarize_data to create a summary
    3. Finally use create_report to format the output""",
    tools=[search_documents, summarize_data, create_report],
)

The agent follows the sequence, passing results between tools automatically. No orchestration code needed.

⚠️ Gotchas and Tips

Docstrings are your schema. ADK infers tool descriptions from Python docstrings and type hints. Missing or vague docstrings mean the agent can't properly select or parameterize the tool.

Tool limitations. Some built-in tools (Google Search, Code Execution) can't be combined with other tools in the same agent. Check ADK documentation for compatibility.

Long-running tools. For operations that take more than a few seconds, use LongRunningFunctionTool which supports async execution with polling.

Testing locally. ADK's CLI (adk run) and dev UI let you test tool interactions locally before deploying to Agent Engine. Always test tool selection with varied prompts.

⏭️ What's Next

This is Post 2 of the GCP AI Agents with Terraform series.

Post 1: Deploy First Vertex AI Agent 🤖
Post 2: Agent Tools - Connect to APIs (you are here) 🔌
Post 3: Multi-Agent with Agent Engine
Post 4: Agent + Google Search Grounding

Your agent can now take action. Function tools, OpenAPI tools, MCP integrations - ADK makes your Python functions into agent capabilities. The model reasons; your code executes. 🔌

Found this helpful? Follow for the full AI Agents with Terraform series! 💬