Forem: Florian Lutz

Take control of your AI. In my recent blog post, I show how to run an inference model locally and fully airgapped - no cloud, no third parties, just open source - so your data never leaves your environment. Set the baseline for your local AI Workloads.

Florian Lutz — Thu, 25 Sep 2025 14:21:26 +0000

Florian Lutz

Jul 11 '25

Setting up an airgapped LLM using Ollama

#ai #airgapped #security #tutorial

Comments

4 min read

Setting up an airgapped LLM using Ollama

Florian Lutz — Fri, 11 Jul 2025 11:38:23 +0000

Introduction

This article will show, how to set up a local or even air-gapped LLM using Ollama and either Podman or Docker. This can be used as a basis to facilitate Gen AI around restricted or secret data. I'm further using this setting to set up a chat bot, that can search trough secret documents, and a coding assistant, that I can use with a VS Code extension to get development support on secret classified code projects.

Info: all the used podman commands, execpt for the import and export of the volumes are working for docker as well, if you just replace "podman" with "docker".

Cache yourself the desired LLMs

The first step, to setting up a air-gapped LLM is to make the LLM itself accessible for the containers lying in network isolation. For that, I am going to export the volume from a non air-gapped container, that is already set with our desired LLMs, and import it to Podman.

Info: I could of course just assign the same volume to the non restricted and the air-gapped containers, but in my specific case, I even did the export of the volume on a separate machine, just to be sure. The transfer of the exported volume file, I'm sure you can manage without explanation.

To cache the desired LLMs, I'm first going to set up a Ollama Container, using the official Ollama Image form Docker Hub:

podman run -d \
-v ollama:/root/.ollama \
-p 11434:11434 \
--name ollama docker.io/ollama/ollama

After the container is set up, I can run this command for each of the LLMs I'd like to have inside of the air-gapped container. To find these LLM options, I just browsed trough the [Ollama Library].(https://ollama.com/library).
podman exec -it ollama ollama pull <your desired LLM>

Having pulled all the LLMs I've ever dreamed of, but not too many, since the export and import of the volume takes quite some time, due to its size, I'm now ready to export the Volume and transfer the file to the air-gapped machine:
podman volume export ollama --output ollama.tar

Create the Volume and restore the cached LLMs

On the air-gapped machine, I can now create a new volume using the following command:
podman volume create ollama

Now all there is left to do, is to bring life (or data) to the newly created volume. For that I'm running:
podman volume import ollama ollama.tar

Exporting and Importing Volumes is working in Docker as well. This however needs an active license for Docker Desktop. Also I only found a way to do this using the UI of Docker Desktop. I did not find any CLI commands to do this.

Configure the Network to be internal

To setup the Network around the container to be air-gapped, there are different options. The easiest way to set this up, is to just use the network option --internal. This keeps the container isolated from the host network, but keeps the option to add other containers within the same network:
podman network create ollama-internal-network --internal

However, often you'd like to still reach your container outside of the Podman console interface. This is the case in my use case as well, since I need the container to be reachable by either the VS Code extension as a coding assistant, or some sort of chatbot fronted, that itself I need to reach as well. Because of that I'm creating the network without the internal option, but assign fine tuned firewall rules to the network by using the kernel level rules of iptables.

So first I create the network:
podman network create ollama-internal-network

Then I retrieve the address range of this specific network:

networkAddressSpace=$(podman network inspect ollama-internal-network \
--format '{{range .Subnets}}{{.Subnet}}{{end}}')

And then I assign the specific firewall rules using iptables. In my case, I only care about the container not reaching the outside world, so I disable any egress from the network (address space of the network to be precise):
iptables -A OUTPUT -s $networkAddressSpace -j DROP

This now prevents my container to reach the outside world, by dropping all outgoing packages.

Create the internal Docker Container

After I've set up the isolated network for my container, and prepared the volume with the LLMs, I now can setup the container for my use:

podman run -d -v ollama:/root/.ollama \ 
-p 11434:11434 \
--network ollama-internal-network \
--name ollama-internal ollama/ollama

With the correct network assigned, the container now can't reach the outside world, so my data should be safe.

Test with a cached LLM

To test, if the volume import and assignment to the container was successful, I send the ollama run <myLLM> command to the container and check, if the LLM is actually available:
podman exec -it ollama-internal ollama run mistral

Conclusion

With my Ollama container now being set up in a secure way, I can now either directly throw prompts at the LLMs, that contain restricted information, or I can use this basis to setup a secure Coding Assistant or even a Chatbot that has knowledge about some secret documents and files.

Azure Machine Learning Workspace secure Networking Setup🛡️🔒🔑

Florian Lutz — Sat, 11 Jan 2025 09:30:00 +0000

TLDR

In this post, I'm covering how to set up an Azure ML Workspace in a secure way. The focus is set on networking and integrating the Service with other Azure Services. If you're not interested in the details of this, and just look for a blueprint or template, you can check out this Github Repo.

Motivation
Objective
Architecture
Deployment
Prospect

Motivation

In a recent project of mine, I've set up an Azure Machine Learning Workspace (AMLW). Upon research, I've realized that there is no easy blueprint combining the setup of a Workspace with secured networking and integration with other Azure services. Additionally, the available documentation was not always 100% clear to me.

Usually, in these cases, I'd just go ahead and set up the resources in the Azure Portal to understand how they interact and can be integrated. However, in the case of the AMLW, I was not able to achieve the target architecture with the options available in the portal.

These difficulties led me to post about the secure setup of this resource here. Perhaps some large language models (LLMs) are processing this blog post and provide answers to desperate Cloud Engineers.

Objective

The idea of this post is to generalize my learnings with the AMLW and create a blueprint from the secure and integrated setup for others to use. The focus will be on the following topics:

Networking integration with other Azure services.
Configuration of a Managed Identity for the networking setup to function properly.

Architecture

The architecture of this blueprint as shown below contains:

Azure Machine Learning Workspace (the point of the post)
Storage Account (essential resource to AMLW)
Container Registry (essential resource to AMLW)
Key Vault (essential resource to AMLW)
Solution Vnet (to secure all Azure Services)
OpenAI Workspace (the Azure Service to integrate with AMLW)
Application Insights

The Storage Account, Container Registry, and Key Vault are essential resources required for the deployment of the AMLW. These resources are integrated into both the Solution Virtual Network (VNet) and the Workspace Managed VNet. The OpenAI Workspace is utilized in this architecture to demonstrate how the AMLW can be integrated with various other Platform as a Service (PaaS) offerings on Azure. Additionally, the Application Insights instance and the Jumphost Virtual Machine (VM) serve as supporting resources to facilitate network access and enhance observability of the solution. The Jumphost VM is accessible via Azure Bastion. Aside from the VM and the AMLW, all networking integrations are configured using Private Endpoints.

Deployment

The Deployment of the resources will be done using a bicep Resource Group deployment. For that, a Resource Group needs to be set up.

Deploy Resource Group

Setting up the Resource Group using the Azure CLI:

$rgName = "ml-secure-blueprint"
$location = "germanywestcentral"
az group create --name $rgName --location $location

The Details of the Azure Machine Learning Workspace

In the bicep module of the AMLW, along with standard properties such as name, location, identity, and the resource IDs for essential services like Key Vault, Storage Account, and Container Registry, the most intriguing component is the managedNetwork object. This object primarily includes configuration for the AMLW's networking options. In this instance, I have set AllowInternetOutbound to ensure that outbound traffic is not restricted.

For integration with other Azure services, you can define custom outbound rule objects. In my case, I named the rule allowOpenAi, but you can choose any name you prefer. The type must be set to PrivateEndpoint, and within the destination object, you can specify your target Resource ID and the subresource target, which you can find on this site.

resource machineLearningWorkspace 'Microsoft.MachineLearningServices/workspaces@2024-07-01-preview' = {
  name: machineLearningWorkspaceName
  location: location
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${managedIdentityId}': {}
    }
  }
  properties: {
    applicationInsights: applicationInsightsId
    storageAccount: storageAccountId
    containerRegistry: containerRegistryId
    keyVault: keyVaultId
    imageBuildCompute: 'cpu-compute'
    primaryUserAssignedIdentity: managedIdentityId
    publicNetworkAccess: 'Disabled'
    managedNetwork: {
      isolationMode: 'AllowInternetOutbound'
      outboundRules: {
        allowOpenAi: {
          type: 'PrivateEndpoint'
          destination: {
            serviceResourceId: openAiWorkspaceId
            sparkEnabled: true
            subresourceTarget: 'account'
          }
        }
      }
    }
  }
  sku: {
    name: 'Basic'
    tier: 'Basic'
  }
}

Configuration of the Managed Identity

The configuration of the Managed Identity of the AMLW to set up the network details correctly is shown in the snippet below. The important part is that the Identity has the Azure AI Enterprise Network Connection Approver Role assigned in a context where it is authorized not only to all the essential resources of the AMLW but also to all resources that you want to integrate the AMLW with. I chose to assign this role to the entire resource group because only these resources are within this resource group.

@description('The managed identity name.')
param managedIdentityName string

@description('The main location.')
param location string

@description('The network connection approver role definition id.')
var networkConnectionApproverRoleDefinitionId = 'b556d68e-0be0-4f35-a333-ad7ee1ce17ea'

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: managedIdentityName
  location: location
}

resource connectionApproverAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, networkConnectionApproverRoleDefinitionId, managedIdentity.name)
  properties: {
    principalId: managedIdentity.properties.principalId
    roleDefinitionId: resourceId('Microsoft.Authorization/roleDefinitions', networkConnectionApproverRoleDefinitionId)
    principalType: 'ServicePrincipal'
  }
}

Deploy Bicep Script Resources

The deployment is being created using the Azure CLI. Besides setting the password for the Jumphost VM, you can configure several parameters in the app-parameters.json file.

$vmPassword = "<your password here>"
az deployment group create `
    --resource-group $rgName `
    --template-file app-infrastructure.bicep `
    --parameters @app-parameters.json `
    --parameters "vmAdminPassword=$vmPassword" `
    --name $rgName

The parameters file might look like this:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "instance": {
            "value": "dev"
        },
        "prefix": {
            "value": "blueprint"
        },
        "location": {
            "value": "westeurope"
        }
    }
}

Deploy the ML Workspace Network

To complete the deployment, you need to initiate the network deployment for the AMLW. This step is necessary because, as explained here, the managed virtual network for an AMLW is not automatically created during its initial deployment, it is provisioned only when required. To ensure that the managed network is deployed, simply execute the script shown below to force its creation.

$subscriptionId = "<your Subscription ID>"
$mlWorkspaceName = "<name of your ML Workspace>"
az ml workspace provision-network `
    --subscription $subscriptionId `
    --resource-group $rgName `
    --name $mlWorkspaceName

After this script finishes, the Private Endpoints will be automatically set up for:

Key Vault
Container Registry
Storage Account
OpenAI Workspace

You can verify their status in the Azure Portal. From the AMLW, the OpenAi Workspace Private Endpoint should appear like this:

And on the OpenAI Workspace you should find the following configuration:

Prospect

With your baseline networking setup established, you can now enhance your project by adding additional capabilities.

One example could be implementing an Ingress solution. Depending on your use case and whether you wish to provide public access, this can be achieved using Azure Front Door or Application Gateway in conjunction with API Management. If you intend to grant access to users within your organization, you may consider setting up an ExpressRoute or a Site-to-Site VPN connection to your Solution VNet.

Another capability you might need is a deployment agent. For more information on this, you can refer to one of my previous posts here.

PS: I've written this post without any AI Tools besides from spell checking and improving my wording.

Use Container App Jobs for scaling Devops Build Agents

Florian Lutz — Wed, 24 May 2023 14:54:00 +0000

Setting up Build Agents in Projects, where from different Reasons (networking, capacity, cost, ...) hosted Agents are no option has been quite unpleasing for some time. But now, with the Container App Jobs, this can be done quite easily and with the scaling that we all wanted.

First, a short explanation of why Container Apps Jobs was needed to achieve this. The standard Container Apps supported scaling up with the KEDA Rule so it was not a problem to provision enough Revisions according to the Build Agent Queue. The Issue came up as soon as the first Builds finished. Scaling down Replicas was some kind of random, as soon as the Queue was empty and not all Replicas had a Job, any Replica could get removed, so often a running Container got shut down and therefore the scaling option was not usable. With that said, Container App Jobs solves this issue.

Preparing your Container

To Prepare your container, you can follow this guide . This article will only cover deploying the Build Agents from an Image that is already in a Container Registry.

Create a Build Agent Pool and a Personal Access Token

To create a Build Agent Pool navigate to Organization Settings (if you don't have the Permissions to do it on Organization Level just jump into the Settings of a Project) in DevOps, there in the Category Pipelines>Agent Pools click on "Add pool" and select "Self-hosted" as Pool Type. You can also just add the Agents to an existing Agent Pool of type "Self-hosted".

If you now open your Agent Pool (this time on Organization Level, it's important) you can see the Agent Pool Id in the URL:
https://dev.azure.com//_settings/agentpools?poolId=<poolId>&view=jobs
Note that Id somewhere.

Next step is to create a Personal Access Token, which the Containers can use to authenticate against the Devops API to register themselves as Build Agents. Therefore click on User Settings (top right in Devops) and jump to Personal Access Tokens. The Token should have permission to Read & Manage Agent Pools. Generate the PAT and store it somewhere save (e.g. Password Manager).

Setup a Container Apps Environment

If you already have one, you can use an existing one. A very handy fact about the Container Apps Jobs is that high-level Infrastructure Configuration mostly concerns the Container Apps Environment, therefore if you have an Environment that is already integrated into your networking infrastructure, just use this one to deploy your Container Apps Jobs, and they will automatically be integrated into the Network.

For this article, we're going to stick to a simple Environment with no specific configuration. Because the Container Apps Jobs are in Public Preview and only available with CLI or ARM I am going to stick with Azure CLI. Depending on if you run the CLI Commands in Powershell or Bash, you might need to replace the ""(powershell) with "\"(bash).

az containerapp env create \
    --name "<name>" \
    --location "westeurope" \
    --resource-group "<rg-name>" \
    --logs-workspace-id "<workspace-id>" \
    --logs-workspace-key "<workspace-key>"

You can leave off the workspace id and key, then it will provision you a Log Analytics workspace automatically. Just specify it if you already got one to avoid duplication. You can get the Key with:

az monitor log-analytics workspace get-shared-keys \
   --name "<law-name>" \
   --resource-group "<rg-name>"

Deploy the Container App Job

Now you can deploy the Container App Job like that:

az containerapp job create \
    --name "<name>" \
    --resource-group "<rg-name>" \
    --environment "<containerappenvironment-name>" \
    --trigger-type "Event" \
    --replica-timeout 7200 \  #timeout in seconds
    --replica-retry-limit 0 \
    --replica-completion-count 1 \
    --parallelism 1 \
    --polling-interval 1 \
    --image "<path to your image>" \
    --registry-server "<your container registry server>" \
    --registry-username "<cr-username>" \
    --registry-password "<cr-password>" \
    --cpu "0.25" --memory "0.5Gi" \
    --scale-rule-name "azure-pipelines" \
    --scale-rule-type "azure-pipelines" \
    --scale-rule-metadata "poolID=<poolId>" \
                          "organizationURLFromEnv=AZP_URL" \
                          "personalAccessTokenFromEnv=AZP_TOKEN" \
                          "poolName=AZP_POOL" \
    --secrets "azp-pool=<yourAgentPoolName>" \
              "azp-token=<yourPatToken>" \
              "azp-url=<yourDevopsOrgUrl>" \
    --env-vars "AZP_POOL=secretref:azp-pool" \
               "AZP_TOKEN=secretref:azp-token" \
               "AZP_URL=secretref:azp-url"

This is first a lot, but we'll go through the important things.

First of all, is the "replica-timeout". This needs to be balanced so you don't overspend on stuck build agents, but your pipeline runs also need to have enough time to finish. Just put a bit longer than your longest expected run in there. One tip, since we now have this awesome option: Create a Pool for short-running tasks and one for the long ones. Since they really scale to zero it might save some money.

The Polling interval might also be interesting, there you also have to find the balance between faster scaling -> shorter Queue Times, or lower cost -> fewer retries.

Then to configure the KEDA scaling you need to create Secrets for azp-pool, azp-token and azp-url. These you need to reference with the secretref: notation in your environment variables and pass those into the scale-rule-metadata. This kind of configuration does not look very pleasing but for a public preview, this will do.

Now all there is left to say is thank god for the option of using Build Agent Containers that don't require yourself to manage a K8s Cluster or draw carts of money to Microsoft because of missing scaling capabilities.

Edit:
You can now also find a template to deploy the Container Apps Jobs Build Agents using Powershell and Bicep. Check out this Link.

Advantages of Azure Powershell over Azure CLI

Florian Lutz — Wed, 21 Dec 2022 07:13:02 +0000

Azure PowerShell and Azure CLI are both powerful tools that can be used to manage resources in the Azure cloud platform. While both tools have their own unique advantages, Azure PowerShell has several key benefits that make it a popular choice among Azure administrators.

One of the main advantages of Azure PowerShell is its ability to provide deep integration with the Azure platform. Azure PowerShell includes a wide range of cmdlets (command-line functions) that can be used to perform almost any task in Azure. This includes everything from creating and managing virtual machines to configuring networking and security.

Another advantage of Azure PowerShell is its support for automation. Azure administrators can use Azure PowerShell to write scripts that automate complex tasks, which can save time and reduce the risk of errors. Azure PowerShell scripts can also be run on a schedule, making it easy to automate routine maintenance tasks.

In addition to automation, Azure PowerShell also offers advanced features such as the ability to work with Azure Resource Manager templates. These templates allow administrators to define and deploy complex Azure environments using a simple JSON-based syntax. This can be particularly useful for organizations that need to deploy and manage multiple Azure resources in a consistent and repeatable way.

Finally, Azure PowerShell has a large and active community of users, which means there is a wealth of documentation, examples, and support available online. This can make it easier for administrators to learn and use Azure PowerShell, and to get help when they run into issues.

While Azure CLI is also a useful tool for managing Azure resources, it does not offer the same level of integration or advanced features as Azure PowerShell. For these reasons, many Azure administrators prefer to use Azure PowerShell for their day-to-day management tasks.

In conclusion, Azure PowerShell is a powerful and versatile tool that offers many advantages for managing Azure resources. Its deep integration with the Azure platform, support for automation, and advanced features make it a popular choice among Azure administrators.

Pulumi - Setting up a Pulumi Stack per Environment 🏗️

Florian Lutz — Tue, 22 Nov 2022 14:14:48 +0000

Introduction

This part of the series is to help you to recreate your project structure in Pulumi. Therefore, you need to create a Pulumi Stack for each of your project environments.

What is a Stack?

A Stack in Pulumi is an independent configurable instance of a Pulumi program. On the initial creation of a Pulumi Project, it will automatically create the first Stack for you. Each Stack has its own Statefile containing the Resources of the Stack and can be deployed independent.

Why should I have a Stack per environment,

With having a Stack per environment, you gain the advantage of having separate Statefiles and separate configuration files. It enabled you to use the configuration to handle differences of your resources between your environments. Examples for this would be the naming convention, different resource locations or Business Continuity settings to save cost in the development environments.

Setting Up the Pulumi Project

If you already have a Pulumi Project set up, just skip this chapter and check out how to add new Stacks in your project.

To set up the Pulumi Project with the Azure C# configuration I am using, after logging into Azure CLI and connecting to the Pulumi Backend type:

pulumi new azure-csharp

As additional information you now have to enter the Project Name, a Stack name, a Azure Location and a Passphrase. The Passphrase is used to decrypt Secrets in the Statefile.

Adding new Stacks to the Pulumi project

To add new Stacks to your Pulumi project, use the following command:

pulumi stack init <stackname>

Repeat this command for each of your project environments. This will create a Pulumi..yaml File for each Stack. The .yaml File is containing the configuration for a stack. You can add specific Key-Value Pairs to the configuration file per stack. The config file has the following format:

encryptionsalt: <secretValue>
config:
  azure-native:location: <azureLocation>
  <pulumiProjectName>:<Key> <Value>

Using Key-Value Pairs from configuration files

On top of your code, define a new Pulumi configuration:

var config = new Pulumi.Config();

This var can later be used to Get or Require Secrets from the configuration file. For Key-Value Pairs that cannot be null, you can use:

var valueOfKey = config.Require("<Key>");

If you are using values, that can be absent on another Stack use:

var valueOfKey = config.Get("<Key>");

Be sure to handle any Null-Exceptions in that case.

With this, you have learned how to create Stacks for your environments and also how to use the configuration files for any differences in between your resources.

Pulumi - Setting up a self-managed Backend 🚀

Florian Lutz — Tue, 22 Nov 2022 14:13:43 +0000

Introduction

This post series is to help you into the world of stateful Infrastructure as Code with Pulumi. We're going to use Azure Native as resource provider to provision our desired infrastructure. For the Azure Cloud, it is also possible to use the Azure Classic package, but Azure Native usually gets updated more quickly on changes in the Azure Cloud.

The aim of this post is to prepare yourself a self-managed backend that will bring you up to speed in your project.

What to know about Pulumi

Pulumi as an Infrastructure as Code tool provides stateful deployments. That means it creates a to-do list for every deployment by comparing what is deployed in the cloud and what is specified in the code. This is handled via a Statefile. In this file Pulumi is documenting deployed and imported cloud resources. The resources are noted down as JSON objects, including some of their configuration but mainly their Resource Id.

The handling of the Statefile for Pulumi can happen in multiple options. The simplest option is to just have the Statefile located in the file system of your device. This is set up very fast and useful for quick tests around cloud infrastructure, but this option does not allow team collaboration.

Alternatives are to use the Pulumi Cloud to handle your Statefile or to create a self-managed backend for your Statefile. This can be done by connecting Pulumi to the storage resource of your Cloud and is what we want to achieve here.

Getting Started

For the self-managed backend, you need to set up the following things:

Install Pulumi
Create a Storage Account
Create a empty directory

Preparing the Storage Account

To use the Storage Account, you need to create a Blob Container, that is later used to store the State File(s) of your Infrastructure. In my case, the Container is just called Pulumi. An additional that is optional but recommended is to encrypt the Storage Account with your own key. Therefore, you have to create a Key Vault and follow the Steps here.

Login with Azure CLI

The next step is to log in with the Azure CLI. This is not necessary for setting up the self-managed Pulumi Backend, but still will be handy, since connecting Pulumi to the Storage Account will require the Key of the Storage account.
Use:

az login --tenant xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

or with MFA required:

az login --tenant xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --use-device-code

And make sure you are using the correct Subscription:

az account set --subscripion xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

One hint for any international Readers, by e.g. the Azure China cloud with the CLI Pulumi will also aim for China.

Setup PowerShell Variables and Environment Variables

Out of just being used to it, I go with a mix of the Azure CLI and PowerShell for the commands, of course you can go with bash as well. For Azure PowerShell fans, I've got bad news here, Pulumi only does support the Azure CLI. But fear not - as soon as you got your CI/CD Pipelines set up, you don't have to touch it anymore.

For the Pulumi Login, you need the following Variables:

$storageAccountName=<storageAccountName>
$containerPath=<containerPath>

These are used for Pulumi to Identify your storage account. Additionally Pulumi needs the following Environment Variables setup to login into your backend:

$env:AZURE_STORAGE_ACCOUNT=$storageAccountName
$env:AZURE_STORAGE_KEY=(az storage account keys list --account-name $storageAccountName | ConvertFrom-Json)[0].value

I am using the Azure CLI to avoid copying the Storage Account Key into my Console. With the Variable setup, we can now login with Pulumi.

Pulumi Login

The Pulumi Login can be done with:

pulumi login azblob://$containerPath

Congratulations, now your Device is connected to a Storage Account and Pulumi is able to either create, or access any Statefiles that are located in the specified Container.

One hint for those of you wondering why you had to log in with the Azure CLI and still provide the Storage Account Key. Accessing the Storage Account via RBAC is not supported for now.

Forem: Florian Lutz

Take control of your AI. In my recent blog post, I show how to run an inference model locally and fully airgapped - no cloud, no third parties, just open source - so your data never leaves your environment. Set the baseline for your local AI Workloads.

Setting up an airgapped LLM using Ollama

Setting up an airgapped LLM using Ollama

Introduction

Cache yourself the desired LLMs

Create the Volume and restore the cached LLMs

Configure the Network to be internal

Create the internal Docker Container

Test with a cached LLM

Conclusion

Azure Machine Learning Workspace secure Networking Setup🛡️🔒🔑

TLDR

Table of contents

Motivation

Objective

Architecture

Deployment

Deploy Resource Group

The Details of the Azure Machine Learning Workspace

Configuration of the Managed Identity

Deploy Bicep Script Resources

Deploy the ML Workspace Network

Prospect

Use Container App Jobs for scaling Devops Build Agents

Preparing your Container

Create a Build Agent Pool and a Personal Access Token

Setup a Container Apps Environment

Deploy the Container App Job

Advantages of Azure Powershell over Azure CLI

Pulumi - Setting up a Pulumi Stack per Environment 🏗️

Introduction

What is a Stack?

Why should I have a Stack per environment,

Setting Up the Pulumi Project

Adding new Stacks to the Pulumi project

Using Key-Value Pairs from configuration files

Pulumi - Setting up a self-managed Backend 🚀

Introduction

What to know about Pulumi

Getting Started

Preparing the Storage Account

Login with Azure CLI

Setup PowerShell Variables and Environment Variables

Pulumi Login