Forem: Rosemary Wang

Using AI for Terraform: flows, prompts, and agents with LangFlow & Docling

Rosemary Wang — Mon, 02 Feb 2026 19:38:35 +0000

In part 1, I learned to deploy a local stack for running an AI agent. Originally, I thought I could generate "good" Terraform configuration based on all my content for secure and scalable infrastructure as code.

Shortly after, I received a message encouraging me to think about the second edition of my book. I have procrastinated on this for a while now since it takes time to revise material and generate new examples. One of the biggest challenges with my book was writing examples. Initial feedback suggested I write everything in Python for greater accessibility and Google Cloud for lower cost, which I did. In retrospect, I should have just written everything in Terraform to run on AWS.

As I reflected on this further, I realized something important. Isn't book writing the perfect use case for an AI agent? If I had agent who knew my writing style help me write new examples in Terraform for my book, maybe I could expedite the process of creating a second edition.

With my regrets in mind, I decided to try to create a "book writing" agent that helps me generate examples to match my text. After all, I had the chapters of the book written. I wanted new examples to reframe some of the principles and practices. This sent me on a major exploration of prompts, agent instructions, and flows in LangFlow.

Process PDFs with Docling

I had editable drafts of the chapters, but only the final PDF version of the book had the proper code annotations and figures. I needed to process the PDF book chapters into text and images before chunking and storing them in my OpenSearch vector database. Enter Docling, a document processing tool for unstructured data.

Fortunately, LangFlow has a Docling component for processing a set of files and chunking them. You do have to install it before you run LangFlow.

uv pip install langflow[docling]

When you start creating a flow in LangFlow, drag-and-drop the Docling component.

There are a few attributes you need to consider with Docling:

Pipeline - I opted for standard just to process the text. If I wanted to also process the figures, I could select vlm (Visual Language Model).
OCR Engine - None for now. I wanted to test if I had sufficient resources to run Docling.

My laptop constrains the amount of memory Docling can use to process the documents, which is why I did not upload the entire book or use VLM or OCR.

Next, I need to chunk the text into my vector database. Rather than do fixed-size chunking, I decided to try hybrid chunking which combines fixed-size chunking with semantic chunking. This ensures that the various chapters of my book have chunks with proper context. After chunking three chapters of my book, I stored the chunks in the OpenSearch vector database using Granite embeddings hosted by Ollama.

Besides the PDF chapters of the book, I had a few blogs on best practices for writing Terraform. These had some text and examples that I wanted to include as part of the agent's response. Using the URL component, I added the set of URLs for the blog posts and passed it to the vector database.

Now that I had my expert-level content on infrastructure as code and Terraform practices into my vector database, I could use an agent to reference that context.

Create the Terraform coding agent

I started with what I thought was the easier agent to build - a coding agent that generates "good" Terraform. This agent needed to generate Terraform configuration with the following:

Proper resource and module declarations - no hallucinations please
Correct formatting
All variables and outputs defined

In addition to writing proper working Terraform, I wanted the agent to include a good example. I had a demo repository that I constantly copied and pasted into other repositories, so I wanted the agent to reference that configuration when the prompt matched.

With all these requirements, I realized I need to use two MCP servers:

Terraform MCP server for the latest resource and module documentation
GitHub MCP server for getting files in my reference repository

I did not need all the tools available on these MCP servers. From an access control perspective, I used the following for each MCP server:

Terraform MCP server - get_latest_module_version, get_latest_provider_version, get_module_details, get_provider_details, get_provider_capabilities, search_modules, and search_providers
GitHub MCP server - get_file_contents and search_code

My agent could retrieve the latest Terraform modules and providers or search GitHub for reference code. I connected the GitHub and Terraform MCP servers with the URL component to my agent using LangFlow.

Finally, the coding agent needed instructions. I started with the official agent skills for Terraform and refined the instructions to better suite Granite and my use case. It took quite a bit of trial-and-error. The full set of instructions are on a GitHub repository. I had two main observations.

First, I had be very specific about which repository and commit the agent should reference for a "good" Terraform example. It turns out the MCP component in LangFlow cannot handle optional parameters at the time of this post so I had to put the exact commit hash and branch the agent should reference in the agent instructions.

Second, the prompt had to include the specific module and resource I wanted the example to include (e.g., Create an example with the aws_opensearchserverless_collection resource.) If I did not include the exact module or resource, Granite would search for the wrong module or resource with the Terraform MCP server. After adjusting my expectations on the prompts, I moved onto the book writing agent.

Create the book writing agent

Why did I separate the book writing task into its own agent? I discovered that combining both Terraform generation and an explanation based on context from OpenSearch led to very poor results. The agent was tasked with doing too much ("generate the explanation AND the example"), which led to some garbled responses that made little sense.

I decided to split the book writing into its own agent so it could properly draft a response that sounds like I wrote the paragraph and not just reiterating the Terraform configuration. This worked much better overall. I also moved the book writing agent first, since it could generate the explanation and the Terraform coding agent could adjust the example.

I connected the output of the writer agent to the input of the coding agent. This ensures that the explanation includes references to the expected resources and examples from user input.

The full set of instructions are on the demo GitHub repository. I wanted it to include the sources so I could check exactly where it found the information.

With both agents, I am going to do more work to refine the prompts. They do a decent job of producing a semi-coherent explanation and example but I think I can improve them with more review.

The result

I passed in a few prompts to test out my agents. While the results needed some edits, they turned out more usable than I expected. For example, I asked the following:

Explain the singleton pattern using the aws_opensearchserverless_collection resource. Include ideas on when to refactor from a singleton to a composite module.

The agent returned a very lengthy explanation, which I'll include a few excerpts below. The first paragraph wasn't completely incorrect but needed to remove the mention of Google project resources since I wanted to use AWS for examples.

The singleton pattern is commonly used in Terraform configurations to manage resources that should exist only once within an environment, such as Google project configurations. In the context of AWS OpenSearch Serverless collections, this pattern ensures that there is a single instance of the collection resource, which typically does not change frequently.

As for the example, it did use the resource I requested but it did not use the right arguments. For example, domain_id doesn't exist for the aws_opensearchserverless_collection.

resource "aws_opensearchserverless_collection" "main" {
  name        = var.collection_name
  domain_id   = aws_opensearch_serverless_domain.main.id
  description = "Primary collection for ${var.environment}"

  tags = merge(
    local.common_tags,
    {
      Environment = var.environment
      Project     = var.project_name
    }
  )
}

When I asked the agent to explain when to refactor from a singleton to composite module, it provided a correct but very generic explanation.

Refactoring from a singleton pattern to a composite module becomes necessary when:

Resource Evolution: Resources evolve beyond single instances, requiring multiple configurations or variations.

Shared Configurations: Multiple resources need shared configurations that are not strictly unique but require common parameters.

Complex Workflows: The infrastructure management involves complex workflows where different components interact and share state.

I asked the book writing agent to apply the other principles when possible, which it did try in later paragraphs. In general, the agent provided a good start for me to edit and iterate on the explanation. I would not use the response as-is, since it has some incorrect points and the explanation is far too generic, but it does write it in the style and tone of voice that I would write in my book.

Conclusion

I learned quite a bit about prompt engineering while trying to build a book writing agent for myself. In some situations, I had to be very specific about how and where an agent should refer to certain tools or data sources. In some of the first iterations, the agent kept bringing up other principles like simplicity, which I do not mention in my book. I had to ask the agent for the source of the principle, which was another book entirely.

In general, the agent did improve over time. The more I asked of it and provided feedback, the better the response it generated. However, I still wouldn't use the response in the book without some editing. I could use the examples, for the most part, but I had to check them correctness and clarity.

Next, I plan on moving these components off my local machine into some cloud infrastructure.

Using AI for Terraform: running locally with Langflow, OpenSearch, & Ollama

Rosemary Wang — Tue, 13 Jan 2026 14:53:29 +0000

I'm a pragmatist at heart. While I don't fully believe in using AI for everything, I did find myself getting very frustrated with my copy and paste process for "good" Terraform configuration. I already wrote Terraform configuration that ran with many resources and was mostly secure by default anyway. Why did I have to go back two or three years to an example and then update it? Could I really use AI to write some new demo code?

I realized I had a lot of content I could reference and get myself out of the copy-paste whirlpool. Most of the time, I looked up:

Slides of old talks with accurate diagrams
Some old code from two or three specific repositories on GitHub
My book
Terraform modules in the registry

The problem? I know how to build infrastructure with Terraform but I know nothing about AI. So I decided to learn.

When I started blogging and trying to learn technology for myself, I ran everything locally and avoided paying for resources. That meant using the free credits for most cloud or managed offerings and working within a resource-constrained system. For this series, I decided on the following tools:

Ollama for models
Langflow for no-code/low-code agentic development
OpenSearch for vector search
Docling to process my PDF documents

As for the model, I was willing to try some of the "open" models like Granite through Ollama. If they didn't work, I would try others.

Building something to run models

As a starting point, I ran everything in containers. If I need more resources, I would move it to some cloud deployment later. With a Docker Compose file, I deployed Ollama, Langflow, and OpenSearch.

Ollama runs models on your local machine. Since I get impatient waiting for Ollama to start and pull the models, I built a Docker container with Ollama and pre-pulled models and embeddings.

FROM ollama/ollama:latest

COPY ./init-ollama.sh /tmp/init-ollama.sh

WORKDIR /tmp

RUN chmod +x init-ollama.sh \
   && ./init-ollama.sh

EXPOSE 11434

In this example, I used granite4:tiny-h since I am running it locally on my laptop.

#!/usr/bin/env bash

ollama serve &
ollama list
ollama pull granite4:tiny-h
ollama pull granite-embedding:30m

Deploying an agent toolchain

I do not know how to write an AI agent. I also didn't feel like coding a whole agent toolchain to achieve my goal of writing infrastructure for my purposes. Luckily, I found Langflow, which offers a no-code/low-code way to deploy AI agents and MCP servers. I created a Dockerfile for Langflow.

FROM langflowai/langflow:1.7.2

USER root

RUN apt update && apt install -y libgl1 libglib2.0-0 && uv pip install langflow[docling]

CMD ["python", "-m", "langflow", "run", "--host", "0.0.0.0", "--port", "7860"]

Initially, I did use the Langflow image without creating a custom Dockerfile. Unfortunately, the Docling component I wanted to use for processing PDF chapters of my book needed a dependency installed. I built that into my own Langflow image so I didn't have to run the install separately.

Building a RAG stack for context

It turns out that retrieval augmented generation (RAG) is an important part of getting my use case working. I have context that I want my agent to use, so that information needs to be processed and stored in a vector database.

I chose Opensearch because I deployed it before and it was something I could run locally. Unfortunately, it turns out that using Opensearch as a vector database for Docling requires some additional configuration. Supposedly, Opensearch creates the index if it doesn't already exist. For some reason, it is unclear if auto-creation works for a simple index or it also works for a vector index. I kept getting errors from Langflow that the index did not exist.

As a workaround, I reverse engineered the index and manually requested the Opensearch API to create an empty index. At this point, I was tired of writing scripts and resorted to asking Project Bob, an AI software agent, for help. I think I asked it to generate me a Dockerfile for Opensearch with a step to create a vector index named "langflow" with "ef_search" of 512 and a property named "chunk_embeddings" of "knn_vector" type with 384 dimensions. It gave me a pretty good script as a response, complete with the proper API call.

#!/bin/bash

# Wait for OpenSearch to be ready
echo "Waiting for OpenSearch to start..."
until curl -s http://localhost:9200/_cluster/health > /dev/null; do
   sleep 2
done

echo "OpenSearch is ready. Creating 'langflow' index..."

# Create the langflow index with vector search configuration
curl -X PUT "http://localhost:9200/langflow" -H 'Content-Type: application/json' -d'
{
 "settings": {
   "index": {
     "knn": true,
     "knn.algo_param.ef_search": 512
   }
 },
 "mappings": {
   "properties": {
     "chunk_embedding": {
       "type": "knn_vector",
       "dimension": 384
     }
   }
 }
}
'

echo ""
echo "Index 'langflow' created successfully!"

Next, Bob created a Dockerfile out of the script. Bob was a bit verbose but the script did work with some modifications.

FROM opensearchproject/opensearch:3

# Copy the initialization script
COPY ./init-opensearch.sh /usr/share/opensearch/init-opensearch.sh

# Create a wrapper script to run both OpenSearch and the init script
RUN echo '#!/bin/bash' > /usr/share/opensearch/entrypoint-wrapper.sh && \
   echo '/usr/share/opensearch/opensearch-docker-entrypoint.sh opensearch &' >> /usr/share/opensearch/entrypoint-wrapper.sh && \
   echo 'sleep 5' >> /usr/share/opensearch/entrypoint-wrapper.sh && \
   echo '/usr/share/opensearch/init-opensearch.sh' >> /usr/share/opensearch/entrypoint-wrapper.sh && \
   echo 'wait' >> /usr/share/opensearch/entrypoint-wrapper.sh && \
   chmod +x /usr/share/opensearch/entrypoint-wrapper.sh

# Use the wrapper as the entrypoint
ENTRYPOINT ["/usr/share/opensearch/entrypoint-wrapper.sh"]

As I said before, I am pragmatic about my use of AI for coding. I didn't want to use something like Bob to speed things up but I got tired and thought, "Why not?" I think the Dockerfile and script were generated in about two minutes compared to the hour that it took me to write and test the Ollama one. The result was functional but I wouldn't use AI to generate anything I didn't have the confidence to verify or test myself.

Putting it all together

I created the set of containers using Docker Compose on my local machine, including the Terraform MCP server. By using the MCP server for the Terraform registry, I could search the public modules and providers available to expedite new examples and versions of modules I used before.

Each of the containers includes a set of environment variables to enable it to run locally. Some variables, like those for Opensearch, disable security plugins and enable demo configurations for ease of use.

services:

 terraform-mcp-server:
   image: hashicorp/terraform-mcp-server:0.3.3
   container_name: terraform-mcp-server
   ports:
     - "8080:8080"
   environment:
     - 'TRANSPORT_MODE=streamable-http'
     - 'TRANSPORT_HOST=0.0.0.0'
     - 'TFE_TOKEN=${TFE_TOKEN}'

 ollama:
   build:
     context: Dockerfiles
     dockerfile: Dockerfile.ollama
   container_name: ollama
   ports:
     - "11434:11434"
   volumes:
     - ollama_data:/root/.ollama
   restart: unless-stopped
   environment:
     - 'OLLAMA_CONTEXT_LENGTH=131072'

 langflow:
   build:
     context: Dockerfiles
     dockerfile: Dockerfile.langflow
   container_name: langflow
   ports:
     - "7860:7860"
   environment:
     - 'LANGFLOW_HOST=0.0.0.0'
     - 'LANGFLOW_OPEN_BROWSER=false'
     - 'LANGFLOW_WORKER_TIMEOUT=1800'
   volumes:
     - langflow_data:/app/langflow
   depends_on:
     - ollama
   restart: unless-stopped

 opensearch:
   build:
     context: Dockerfiles
     dockerfile: Dockerfile.opensearch
   container_name: opensearch
   environment:
     - cluster.name=opensearch-cluster
     - node.name=opensearch-node1
     - discovery.type=single-node
     - bootstrap.memory_lock=true
     - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
     - "DISABLE_INSTALL_DEMO_CONFIG=true"
     - "DISABLE_SECURITY_PLUGIN=true"
   ulimits:
     memlock:
       soft: -1
       hard: -1
     nofile:
       soft: 65536
       hard: 65536
   volumes:
     - opensearch_data:/usr/share/opensearch/data
   ports:
     - "9200:9200"
     - "9600:9600"
   restart: unless-stopped

volumes:
 ollama_data: {}
 langflow_data: {}
 opensearch_data: {}

Each component represents an important part of the AI stack, such as prompts, agents, context, and models. After they came up, I could access Langflow on http://localhost:7860.

The remaining components I could access via API or connect it to a flow in Langflow.

Conclusion

After much trial and error, I managed to figure out how to create a local stack to build out an AI agent to help me update my examples based on knowledge from my book, talks, and other code examples. As I explore more, I will add more tools and context to improve the agent (and maybe even build multiple agents).

I realized I could probably use the AI agent built into my coding IDE to do most of this. Project Bob did end up helping in the process to build this stack and it did make it faster. The downside to using any AI agent built into my coding IDE was the overall cost. I quickly realized that I had to check my usage to ensure I didn't make too many requests.

I was glad that I could run this locally. The small Granite model really helped - I only had to give Ollama a little bit more CPU and memory to run the model. I could mitigate the cost of running against hosted LLMs and maybe achieve a similar result. I found the process of deploying each component valuable as a learning experience.

Next, I plan on building a flow in Langflow to process all of my book chapters, slides, and code examples before passing it to an agent to process.