Forem: Karim

AIOps : Investigation par l’IA dans Kubernetes avec HolmesGPT, Ollama et RunPod …

Karim — Wed, 15 Jan 2025 00:52:40 +0000

Dans le monde de l’orchestration de conteneurs, Kubernetes est devenu une norme pour gérer les workloads conteneurisés. Cependant, la gestion et le dépannage des clusters Kubernetes peuvent être complexes et chronophages. Cet article explore comment l’intelligence artificielle (IA) peut être intégrée dans Kubernetes pour améliorer l’investigation et la gestion des incidents. J’avais d’ailleurs évoqué le sujet dans un article précédent :

AIOps : Déboguer son cluster Kubernetes en utilisant l’intelligence artificielle générative via…

Ici je vais m’intéresser à HolmesGPT. HolmesGPT, développé par Robusta, est un agent de dépannage open source qui utilise l’IA pour investiguer les incidents dans les clusters Kubernetes avec ces caractéristiques :

Intégration avec les outils de gestion d’incidents : HolmesGPT se connecte à des outils comme PagerDuty, OpsGenie et Prometheus pour collecter des données et analyser les alertes.
Investigation automatisée : Grâce à l’IA, HolmesGPT peut identifier et résoudre des problèmes tels que l’expiration des certificats SSL, les problèmes de ressources insuffisantes et les problèmes d’affinité des nœuds. Cela réduit significativement le temps et l’effort nécessaires pour le dépannage.
Personnalisation : HolmesGPT permet de créer des livres de recettes (runbooks) personnalisés pour gérer des problèmes spécifiques, en utilisant des API et des outils personnalisés si nécessaire.

GitHub - robusta-dev/holmesgpt: On-Call Assistant for Prometheus Alerts - Get a head start on fixing alerts with AI investigation

Pour cet exercice, je vais d’abord lancer une instance Ubuntu 24.04 LTS de nouveau chez le fournisseur Cloud DigitalOcean :

Je vais y installer Incus, un fork de LXD qui va me servir de base pour la formation d’un cluster Kubernetes avec plusieurs containers :

Linux Containers - Incus - Introduction

Comme pour LXD, je vais procéder à la création de plusieurs profiles. Mais dans un premier temps, installation d’Incus sur l’instance :

root@k0s-incus:~# curl -fsSL https://pkgs.zabbly.com/key.asc | gpg --show-keys --fingerprint
gpg: directory '/root/.gnupg' created
gpg: keybox '/root/.gnupg/pubring.kbx' created
pub rsa3072 2023-08-23 [SC] [expires: 2025-08-22]
      4EFC 5906 96CB 15B8 7C73 A3AD 82CC 8797 C838 DCFD
uid Zabbly Kernel Builds <info@zabbly.com>
sub rsa3072 2023-08-23 [E] [expires: 2025-08-22]

root@k0s-incus:~# mkdir -p /etc/apt/keyrings/

root@k0s-incus:~# curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc

root@k0s-incus:~# sh -c 'cat <<EOF > /etc/apt/sources.list.d/zabbly-incus-stable.sources
Enabled: yes
Types: deb
URIs: https://pkgs.zabbly.com/incus/stable
Suites: $(. /etc/os-release && echo ${VERSION_CODENAME})
Components: main
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/zabbly.asc

EOF'

root@k0s-incus:~# apt-get update

Hit:1 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:2 http://mirrors.digitalocean.com/ubuntu noble InRelease
Hit:3 https://repos-droplet.digitalocean.com/apt/droplet-agent main InRelease
Hit:4 http://mirrors.digitalocean.com/ubuntu noble-updates InRelease
Hit:5 http://mirrors.digitalocean.com/ubuntu noble-backports InRelease
Get:6 https://pkgs.zabbly.com/incus/stable noble InRelease [7358 B]   
Get:7 https://pkgs.zabbly.com/incus/stable noble/main amd64 Packages [3542 B]
Fetched 10.9 kB in 1s (13.3 kB/s)   
Reading package lists... Done

root@k0s-incus:~# apt-get install incus incus-client incus-ui-canonical -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  attr dconf-gsettings-backend dconf-service dns-root-data dnsmasq-base fontconfig genisoimage glib-networking glib-networking-common glib-networking-services gsettings-desktop-schemas
  gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-x incus-base iw libaa1 libasyncns0 libavc1394-0 libboost-iostreams1.83.0 libboost-thread1.83.0 libbtrfs0t64 libcaca0
  libcairo-gobject2 libcairo2 libcdparanoia0 libdatrie1 libdaxctl1 libdconf1 libdv4t64 libflac12t64 libgdk-pixbuf-2.0-0 libgdk-pixbuf2.0-bin libgdk-pixbuf2.0-common libgraphite2-3
  libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0 libharfbuzz0b libiec61883-0 libmp3lame0 libmpg123-0t64 libndctl6 libnet1 libogg0 libopus0 liborc-0.4-0t64 libpango-1.0-0
  libpangocairo-1.0-0 libpangoft2-1.0-0 libpixman-1-0 libpmem1 libpmemobj1 libproxy1v5 libpulse0 librados2 libraw1394-11 librbd1 librdmacm1t64 libshout3 libsndfile1 libsoup-3.0-0
  libsoup-3.0-common libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libthai-data libthai0 libtheora0 libtwolame0 libusbredirparser1t64 libv4l-0t64 libv4lconvert0t64 libvisual-0.4-0
  libvorbis0a libvorbisenc2 libvpx9 libwavpack1 libx11-xcb1 libxcb-render0 libxcb-shm0 libxdamage1 libxfixes3 libxi6 libxrender1 libxtst6 libxv1 session-migration sshfs wireless-regdb
  x11-common xdelta3

root@k0s-incus:~# incus
Description:
  Command line client for Incus

  All of Incus's features can be driven through the various commands below.
  For help with any of those, simply call them with --help.

  Custom commands can be defined through aliases, use "incus alias" to control those.

Usage:
  incus [command]

Available Commands:
  admin Manage incus daemon
  cluster Manage cluster members
  config Manage instance and server configuration options
  console Attach to instance consoles
  copy Copy instances within or in between servers
  create Create instances from images
  delete Delete instances
  exec Execute commands in instances
  export Export instance backups
  file Manage files in instances
  help Help about any command
  image Manage images
  import Import instance backups
  info Show instance or server information
  launch Create and start instances from images
  list List instances
  move Move instances within or in between servers
  network Manage and attach instances to networks
  pause Pause instances
  profile Manage profiles
  project Manage projects
  publish Publish instances as images
  rebuild Rebuild instances
  remote Manage the list of remote servers
  rename Rename instances
  restart Restart instances
  resume Resume instances
  snapshot Manage instance snapshots
  start Start instances
  stop Stop instances
  storage Manage storage pools and volumes
  top Display resource usage info per instance
  version Show local and remote versions
  webui Open the web interface

Flags:
      --all Show less common commands
      --debug Show all debug messages
      --force-local Force using the local unix socket
  -h, --help Print help
      --project Override the source project
  -q, --quiet Don't show progress information
      --sub-commands Use with help or --help to view sub-commands
  -v, --verbose Show all information messages
      --version Print version number

Use "incus [command] --help" for more information about a command.

Initialisation d’Incus en version minimaliste :

root@k0s-incus:~# incus admin init
Would you like to use clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (btrfs, dir, lvm) [default=btrfs]: dir
Where should this storage pool store its data? [default=/var/lib/incus/storage-pools/default]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=incusbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
Would you like the server to be available over the network? (yes/no) [default=no]:    
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: 
Would you like a YAML "init" preseed to be printed? (yes/no) [default=no]: 

root@k0s-incus:~# incus list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

root@k0s-incus:~# incus profile list
+---------+-----------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+-----------------------+---------+
| default | Default Incus profile | 0 |
+---------+-----------------------+---------+

root@k0s-incus:~# incus profile show default
config: {}
description: Default Incus profile
devices:
  eth0:
    name: eth0
    network: incusbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
used_by: []
project: default

root@k0s-incus:~# incus profile create k8s

Incus dispose d’un tableau de bord de contrôle qui peut être actionné temporairement par incus webui.

incus webui

Activation de ce dernier :

root@k0s-incus:~# nohup incus webui &
[1] 4104
root@k0s-incus:~# nohup: ignoring input and appending output to 'nohup.out'

root@k0s-incus:~# cat nohup.out 
Web server running at: http://127.0.0.1:34363/ui?auth_token=3c5f5d4b-f9ed-4bf9-a174-d5ea2366cfbf

Utilisation de pinggy.io pour y accéder :

Pinggy - Simple Localhost Tunnels

root@k0s-incus:~# ssh -p 443 -R0:127.0.0.1:34363 a.pinggy.io

Je récupère le même profile qu’utilise LXD pour MicroK8s :

MicroK8s - MicroK8s in LXD | MicroK8s

roothttps://microk8s.io/docs/install-lxd@k0s-incus:~# wget https://raw.githubusercontent.com/ubuntu/microk8s/master/tests/lxc/microk8s.profile -O k8s.profile
--2025-01-14 20:58:42-- https://raw.githubusercontent.com/ubuntu/microk8s/master/tests/lxc/microk8s.profile
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 816 [text/plain]
Saving to: ‘k8s.profile’

k8s.profile 100%[=====================================================================================================>] 816 --.-KB/s in 0s      

2025-01-14 20:58:42 (33.4 MB/s) - ‘k8s.profile’ saved [816/816]

root@k0s-incus:~# cat k8s.profile | incus profile edit k8s
root@k0s-incus:~# rm k8s.profile

root@k0s-incus:~# incus profile show k8s
config:
  boot.autostart: "true"
  linux.kernel_modules: ip_vs,ip_vs_rr,ip_vs_wrr,ip_vs_sh,ip_tables,ip6_tables,netlink_diag,nf_nat,overlay,br_netfilter
  raw.lxc: |
    lxc.apparmor.profile=unconfined
    lxc.mount.auto=proc:rw sys:rw cgroup:rw
    lxc.cgroup.devices.allow=a
    lxc.cap.drop=
  security.nesting: "true"
  security.privileged: "true"
description: ""
devices:
  aadisable:
    path: /sys/module/nf_conntrack/parameters/hashsize
    source: /sys/module/nf_conntrack/parameters/hashsize
    type: disk
  aadisable2:
    path: /dev/kmsg
    source: /dev/kmsg
    type: unix-char
  aadisable3:
    path: /sys/fs/bpf
    source: /sys/fs/bpf
    type: disk
  aadisable4:
    path: /proc/sys/net/netfilter/nf_conntrack_max
    source: /proc/sys/net/netfilter/nf_conntrack_max
    type: disk
name: k8s
used_by: []
project: default

Comme Incus a la faculté d’utiliser cloud-init, je crée un nouveau profile destiné à cet usage :

root@k0s-incus:~# incus profile show cloud
config:
  cloud-init.user-data: |
    #cloud-config
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - vim
      - wget
      - git
      - curl
      - htop
      - openssh-server
    bootcmd:
      - systemctl enable ssh
      - systemctl start ssh
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCpbsaaVUMa2TM9q8VkeBmbKvJpbreXTcqI5F5N3riGsoZ7Z/IIN7eR6J47UP2bj3IBTdgHmij1uOexm60QBO2PY4abIhsN+xnVS4a0LSyI8v6nYECWbEehL/gFn6uDmSLA4m0hZCF5BSpLxQYzKS28dHIdXsLC4CDd67nAXIhOiVpM0q/AUCuSy+mA0VwFa/JAkFCk8TpQBorgwJIq635imrgxYIpEUA2wHXOhw23mO3zTUlay13LSlA2a1xyTkP8hSDWdRYVxr2DEB/MtmTX2BdWlA5rDRmzXE7R2/csE245WAxG+XfSu4zNqhHzm8Df3zmZn3/UyKLcx4eJF//mVZyrM7RQHRteA/im8I4IavrReGyCUKY+OsSfygYVFyO87rYQ+IOauOnB4LxBohBjSBN3Skk4X7krYFIi8D9R1lmL+VvBfpvy0YMurOahY1VJFzD0dUeK2bDUdeWzfFkcX039d9/RRXRxieNpxwp1BLPi5/DXG8FihzgwVTf6h60J9/fkYzY+BO8CKG2kYTUsy1ykuXLzLY5sTCREiEoEKcJ9IGz8OimZ1AmkgJJCrQnI6mT/KiNDU6YCc75ONKTKX5HKVPhZWT255Aw4f5LBbBrj06cJX3GuunV0I30+BYyHwLbPBoqgd4GUk3YJlr8wS3qre/YUSc2iKNDTOzFCC8Q== root@k0s-incus
description: incus with cloud-init
devices: {}
name: cloud
used_by: []
project: default

Je suis prêt pour la création de trois containers qui me serviront de pivot à la création d’un cluster Kubernetes :

root@k0s-incus:~# for i in {1..3}; do incus launch -p default -p k8s -p cloud images:ubuntu/24.04/cloud k0s-$i; done
Launching k0s-1
Launching k0s-2                                    
Launching k0s-3

root@k0s-incus:~# incus list                       
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| k0s-1 | RUNNING | 10.224.160.99 (eth0) | fd42:4641:b619:c782:216:3eff:fea4:53d3 (eth0) | CONTAINER | 0 |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| k0s-2 | RUNNING | 10.224.160.54 (eth0) | fd42:4641:b619:c782:216:3eff:feee:7af8 (eth0) | CONTAINER | 0 |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| k0s-3 | RUNNING | 10.224.160.215 (eth0) | fd42:4641:b619:c782:216:3eff:fef3:709b (eth0) | CONTAINER | 0 |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+

root@k0s-incus:~# cat .ssh/config 
Host *
   StrictHostKeyChecking no
   UserKnownHostsFile=/dev/null

root@k0s-incus:~# ssh ubuntu@10.224.160.99

Welcome to Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-51-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/pro

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@k0s-1:~$

Récupération de k0sctl pour la création d’un cluster Kubernetes avec k0s :

Using k0sctl - Documentation

root@k0s-incus:~# wget -c https://github.com/k0sproject/k0sctl/releases/download/v0.21.0/k0sctl-linux-amd64 && chmod +x k0sctl-linux-amd64 && mv k0sctl-linux-amd64 /usr/local/bin/k0sctl

Saving to: ‘k0sctl-linux-amd64’

k0sctl-linux-amd64 100%[=====================================================================================================>] 18.21M --.-KB/s in 0.1s    

2025-01-14 21:22:23 (122 MB/s) - ‘k0sctl-linux-amd64’ saved [19091608/19091608]

root@k0s-incus:~# k0sctl 
NAME:
   k0sctl - k0s cluster management tool

USAGE:
   k0sctl [global options] command [command options]

COMMANDS:
   version Output k0sctl version
   apply Apply a k0sctl configuration
   kubeconfig Output the admin kubeconfig of the cluster
   init Create a configuration template
   reset Remove traces of k0s from all of the hosts
   backup Take backup of existing clusters state
   config Configuration related sub-commands
   completion  
   help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d Enable debug logging (default: false) [$DEBUG]
   --trace Enable trace logging (default: false) [$TRACE]
   --no-redact Do not hide sensitive information in the output (default: false)
   --help, -h show help

root@k0s-incus:~# k0sctl init --k0s > k0sctl.yaml

root@k0s-incus:~# cat k0sctl.yaml 
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
  user: admin
spec:
  hosts:
  - ssh:
      address: 10.224.160.99 
      user: ubuntu
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: controller
  - ssh:
      address: 10.224.160.54
      user: ubuntu
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: worker
  - ssh:
      address: 10.224.160.215
      user: ubuntu
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: worker
  k0s:
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: Cluster
      metadata:
        name: k0s
      spec:
        api:
          k0sApiPort: 9443
          port: 6443
        installConfig:
          users:
            etcdUser: etcd
            kineUser: kube-apiserver
            konnectivityUser: konnectivity-server
            kubeAPIserverUser: kube-apiserver
            kubeSchedulerUser: kube-scheduler
        konnectivity:
          adminPort: 8133
          agentPort: 8132
        network:
          kubeProxy:
            disabled: false
            mode: iptables
          kuberouter:
            autoMTU: true
            mtu: 0
            peerRouterASNs: ""
            peerRouterIPs: ""
          podCIDR: 10.244.0.0/16
          provider: kuberouter
          serviceCIDR: 10.96.0.0/12
        podSecurityPolicy:
          defaultPolicy: 00-k0s-privileged
        storage:
          type: etcd
        telemetry:
          enabled: true

Lancement de la création :


root@k0s-incus:~# k0sctl apply --config k0sctl.yaml 

⠀⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███ ███ ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███ ███ ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███ ███ ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████ ███ ██████████
k0sctl v0.21.0 Copyright 2023, k0sctl authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
INFO ==> Running phase: Set k0s version  
INFO Looking up latest stable k0s version         
INFO Using k0s version v1.31.3+k0s.0              
INFO ==> Running phase: Connect to hosts 
INFO [ssh] 10.224.160.215:22: connected           
INFO [ssh] 10.224.160.99:22: connected            
INFO [ssh] 10.224.160.54:22: connected            
INFO ==> Running phase: Detect host operating systems 
INFO [ssh] 10.224.160.215:22: is running Ubuntu 24.04.1 LTS 
INFO [ssh] 10.224.160.99:22: is running Ubuntu 24.04.1 LTS 
INFO [ssh] 10.224.160.54:22: is running Ubuntu 24.04.1 LTS 
INFO ==> Running phase: Acquire exclusive host lock 
INFO ==> Running phase: Prepare hosts    
INFO ==> Running phase: Gather host facts 
INFO [ssh] 10.224.160.215:22: using k0s-3 as hostname 
INFO [ssh] 10.224.160.54:22: using k0s-2 as hostname 
INFO [ssh] 10.224.160.99:22: using k0s-1 as hostname 
INFO [ssh] 10.224.160.215:22: discovered eth0 as private interface 
INFO [ssh] 10.224.160.54:22: discovered eth0 as private interface 
INFO [ssh] 10.224.160.99:22: discovered eth0 as private interface 
INFO ==> Running phase: Validate hosts   
INFO ==> Running phase: Validate facts   
INFO ==> Running phase: Download k0s on hosts 
INFO [ssh] 10.224.160.215:22: downloading k0s v1.31.3+k0s.0 
INFO [ssh] 10.224.160.54:22: downloading k0s v1.31.3+k0s.0 
INFO [ssh] 10.224.160.99:22: downloading k0s v1.31.3+k0s.0 
INFO ==> Running phase: Install k0s binaries on hosts 
INFO [ssh] 10.224.160.99:22: validating configuration 
INFO ==> Running phase: Configure k0s    
INFO [ssh] 10.224.160.99:22: installing new configuration 
INFO ==> Running phase: Initialize the k0s cluster 
INFO [ssh] 10.224.160.99:22: installing k0s controller 
INFO [ssh] 10.224.160.99:22: waiting for the k0s service to start 
INFO [ssh] 10.224.160.99:22: wait for kubernetes to reach ready state 
INFO ==> Running phase: Install workers  
INFO [ssh] 10.224.160.99:22: generating a join token for worker 1 
INFO [ssh] 10.224.160.99:22: generating a join token for worker 2 
INFO [ssh] 10.224.160.215:22: validating api connection to https://10.224.160.99:6443 using join token 
INFO [ssh] 10.224.160.54:22: validating api connection to https://10.224.160.99:6443 using join token 
INFO [ssh] 10.224.160.215:22: writing join token to /etc/k0s/k0stoken 
INFO [ssh] 10.224.160.54:22: writing join token to /etc/k0s/k0stoken 
INFO [ssh] 10.224.160.54:22: installing k0s worker 
INFO [ssh] 10.224.160.215:22: installing k0s worker 
INFO [ssh] 10.224.160.215:22: starting service    
INFO [ssh] 10.224.160.215:22: waiting for node to become ready 
INFO [ssh] 10.224.160.54:22: starting service     
INFO [ssh] 10.224.160.54:22: waiting for node to become ready 
INFO ==> Running phase: Release exclusive host lock 
INFO ==> Running phase: Disconnect from hosts 
INFO ==> Finished in 42s                 
INFO k0s cluster version v1.31.3+k0s.0 is now installed 
INFO Tip: To access the cluster you can now fetch the admin kubeconfig using: 
INFO k0sctl kubeconfig

Le cluster est actif :

root@k0s-incus:~# curl -LO https://dl.k8s.io/release/v1.31.3/bin/linux/amd64/kubectl && chmod +x kubectl && mv kubectl /usr/local/bin/
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 138 100 138 0 0 923 0 --:--:-- --:--:-- --:--:-- 926
100 53.7M 100 53.7M 0 0 476k 0 0:01:55 0:01:55 --:--:-- 1023k

root@k0s-incus:~# mkdir .kube
root@k0s-incus:~# k0sctl kubeconfig --config k0sctl.yaml > .kube/config

root@k0s-incus:~# kubectl cluster-info
Kubernetes control plane is running at https://10.224.160.99:6443
CoreDNS is running at https://10.224.160.99:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

root@k0s-incus:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k0s-2 Ready <none> 5m1s v1.31.3+k0s 10.224.160.54 <none> Ubuntu 24.04.1 LTS 6.8.0-51-generic containerd://1.7.24
k0s-3 Ready <none> 5m1s v1.31.3+k0s 10.224.160.215 <none> Ubuntu 24.04.1 LTS 6.8.0-51-generic containerd://1.7.24

root@k0s-incus:~# kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-645c5d6f5b-kgnsf 1/1 Running 0 5m2s
kube-system pod/coredns-645c5d6f5b-n2rbk 1/1 Running 0 5m2s
kube-system pod/konnectivity-agent-2dg8l 1/1 Running 0 5m4s
kube-system pod/konnectivity-agent-5l5dl 1/1 Running 0 5m4s
kube-system pod/kube-proxy-cx47n 1/1 Running 0 5m7s
kube-system pod/kube-proxy-sp5fd 1/1 Running 0 5m7s
kube-system pod/kube-router-6l4qv 1/1 Running 0 5m7s
kube-system pod/kube-router-b9t89 1/1 Running 0 5m7s
kube-system pod/metrics-server-78c4ccbc7f-jxpzz 1/1 Running 0 5m1s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5m17s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 5m7s
kube-system service/metrics-server ClusterIP 10.109.44.51 <none> 443/TCP 5m1s

Il est alors possible de procéder à l’installation d’HolmesGPT via Pipx :

GitHub - robusta-dev/holmesgpt: On-Call Assistant for Prometheus Alerts - Get a head start on fixing alerts with AI investigation

root@k0s-incus:~# apt install pipx -y

root@k0s-incus:~# pipx ensurepath

Success! Added /root/.local/bin to the PATH environment variable.

Consider adding shell completions for pipx. Run 'pipx completions' for instructions.

You will need to open a new terminal or re-login for the PATH changes to take effect.

Otherwise pipx is ready to go! ✨ 🌟 ✨

root@k0s-incus:~# pipx install "https://github.com/robusta-dev/holmesgpt/archive/refs/heads/master.zip"
  installed package holmesgpt 0.1.0, installed using Python 3.12.3
  These apps are now globally available
    - holmes
done! ✨ 🌟 ✨
root@k0s-incus:~# holmes version
/root/.local/share/pipx/venvs/holmesgpt/lib/python3.12/site-packages/pydantic/_internal/_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'fields' has been removed
  warnings.warn(message, UserWarning)
HEAD -> master-bfafbde3

Pour l’accompagner, je récupère K9s qui fournit une interface utilisateur de terminal pour interagir avec vos clusters Kubernetes. L’objectif de ce projet est de faciliter la navigation, l’observation et la gestion de vos applications dans la nature. K9s surveille continuellement Kubernetes pour les changements et offre des commandes ultérieures pour interagir avec vos ressources observées.

GitHub - derailed/k9s: 🐶 Kubernetes CLI To Manage Your Clusters In Style!

root@k0s-incus:~# wget -c https://github.com/derailed/k9s/releases/download/v0.32.7/k9s_linux_amd64.deb
HTTP request sent, awaiting response... 200 OK
Length: 31832132 (30M) [application/octet-stream]
Saving to: ‘k9s_linux_amd64.deb’

k9s_linux_amd64.deb 100%[=====================================================================================================>] 30.36M --.-KB/s in 0.1s    

2025-01-14 21:40:07 (291 MB/s) - ‘k9s_linux_amd64.deb’ saved [31832132/31832132]

root@k0s-incus:~# apt install -f ./k9s_linux_amd64.deb 
root@k0s-incus:~# k9s --help
K9s is a CLI to view and manage your Kubernetes clusters.

Usage:
  k9s [flags]
  k9s [command]

Available Commands:
  completion Generate the autocompletion script for the specified shell
  help Help about any command
  info List K9s configurations info
  version Print version/build info

Flags:
  -A, --all-namespaces Launch K9s in all namespaces
      --as string Username to impersonate for the operation
      --as-group stringArray Group to impersonate for the operation
      --certificate-authority string Path to a cert file for the certificate authority
      --client-certificate string Path to a client certificate file for TLS
      --client-key string Path to a client key file for TLS
      --cluster string The name of the kubeconfig cluster to use
  -c, --command string Overrides the default resource to load when the application launches
      --context string The name of the kubeconfig context to use
      --crumbsless Turn K9s crumbs off
      --headless Turn K9s header off
  -h, --help help for k9s
      --insecure-skip-tls-verify If true, the server's caCertFile will not be checked for validity
      --kubeconfig string Path to the kubeconfig file to use for CLI requests
      --logFile string Specify the log file (default "/root/.local/state/k9s/k9s.log")
  -l, --logLevel string Specify a log level (error, warn, info, debug, trace) (default "info")
      --logoless Turn K9s logo off
  -n, --namespace string If present, the namespace scope for this CLI request
      --readonly Sets readOnly mode by overriding readOnly configuration setting
  -r, --refresh int Specify the default refresh rate as an integer (sec) (default 2)
      --request-timeout string The length of time to wait before giving up on a single server request
      --screen-dump-dir string Sets a path to a dir for a screen dumps
      --token string Bearer token for authentication to the API server
      --user string The name of the kubeconfig user to use
      --write Sets write mode by overriding the readOnly configuration setting

Use "k9s [command] --help" for more information about a command.

Ollama, une alternative à ChatGPT, peut être déployée pour fournir des capacités de traitement du langage naturel directement dans votre environnement. Cela permet de bénéficier des capacités de traitement du langage naturel de Ollama sans dépendre de services cloud externes.

Ollama

En intégrant Ollama à vos outils de dépannage, vous pouvez générer des réponses et des solutions basées sur l’analyse des logs et des données de votre cluster Kubernetes.

Pour son exécution, je suis amené à utiliser RunPod, une plateforme qui permet d’exécuter des tâches de traitement du langage naturel et d’autres tâches IA. RunPod vous permet en effet de créer des environnements de pod personnalisés pour exécuter des modèles de langage comme Ollama ou d’autres applications IA :

RunPod - The Cloud Built for AI

Création d’un Pod GPU qui me permet donc d’utiliser Ollama …

Set up Ollama on your GPU Pod | RunPod Documentation

Je peux m’y connecter via SSH :

Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 6.5.0-44-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

 ____________ _
(_____\ (_____ \ | |
 _____) ) _ _____ _____) )___ __| |
| __/ | | | || _ \ |____ // _ \ / _ |
| | \ \ | |_| || | | || | | |_| |( (_| |
|_| |_|| ____/ |_| |_||_| \___ / \ ____ |

For detailed documentation and guides, please visit:
https://docs.runpod.io/ and https://blog.runpod.io/

root@5ed8df208cf4:~# nvidia-smi
Tue Jan 14 22:03:15 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07 Driver Version: 550.90.07 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti On | 00000000:81:00.0 Off | N/A |
| 0% 28C P8 11W / 285W | 2MiB / 12282MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+

Exécution d’Ollama :

root@5ed8df208cf4:~# apt update 2> /dev/null && apt install -qq lshw -y 2> /dev/null

root@5ed8df208cf4:~# export OLLAMA_HOST=0.0.0.0:11434
root@5ed8df208cf4:~# (curl -fsSL https://ollama.com/install.sh | sh && ollama serve > ollama.log 2>&1) &
[1] 950
root@5ed8df208cf4:~# >>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
######################################################################## 100.0%
>>> Creating ollama user...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
WARNING: systemd is not running
>>> NVIDIA GPU installed.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.

root@5ed8df208cf4:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name    
tcp 0 0 0.0.0.0:7861 0.0.0.0:* LISTEN 52/nginx: master pr 
tcp 0 0 0.0.0.0:8081 0.0.0.0:* LISTEN 52/nginx: master pr 
tcp 0 0 0.0.0.0:8001 0.0.0.0:* LISTEN 52/nginx: master pr 
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 70/sshd: /usr/sbin/ 
tcp 0 0 0.0.0.0:3001 0.0.0.0:* LISTEN 52/nginx: master pr 
tcp 0 0 0.0.0.0:9091 0.0.0.0:* LISTEN 52/nginx: master pr 
tcp 0 0 127.0.0.11:39145 0.0.0.0:* LISTEN -                   
tcp6 0 0 :::22 :::* LISTEN 70/sshd: /usr/sbin/ 
tcp6 0 0 :::11434 :::* LISTEN 1006/ollama         
udp 0 0 127.0.0.11:33663 0.0.0.0:* -

Récupération d’un LLM avec Llama3.2 :

root@5ed8df208cf4:~# ollama pull llama3.2:3b-instruct-q4_K_S
pulling manifest 
pulling d5e517daeee4... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 1.9 GB                         
pulling 966de95ca8a6... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 7.7 KB                         
pulling a70ff7e570d9... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 6.0 KB                         
pulling 56bb8bd477a5... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 96 B                         
pulling 9c65e8607c0c... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 561 B                         
verifying sha256 digest 
writing manifest 
success 

root@5ed8df208cf4:~# ollama list       
NAME ID SIZE MODIFIED       
llama3.2:3b-instruct-q4_K_S 80f2089878c9 1.9 GB 31 seconds ago

L’endpoint d’Ollama est disponible publiquement via le proxy offert par RunPod :

Modification de K9s pour y intégrer cet endpoint et HolmesGPT sous forme d’un plug-in :

root@k0s-incus:~# cat ~/.config/k9s/plugins.yaml
plugins:
  holmesgpt:
    shortCut: Shift-H
    description: Ask HolmesGPT
    scopes:
      - all
    command: bash
    background: false
    confirm: false
    args:
      - -c
      - |
        holmes ask "why is $NAME of $RESOURCE_NAME in -n $NAMESPACE not working as expected" --model="openai/llama3.2:3b-instruct-q4_K_S"
        echo "Press 'q' to exit"
        while : ; do
        read -n 1 k <&1
        if [[$k = q]] ; then
        break
        fi
        done

root@k0s-incus:~# export OPENAI_API_BASE="https://vsr6spvysc6jly-11434.proxy.runpod.net/v1"
root@k0s-incus:~# export OPENAI_API_KEY=123

Déploiement d’un exemple de Pod problématique dans le cluster Kubernetes via les exemples fournis par Robusta :

root@k0s-incus:~# kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/crashpod/broken.yaml
deployment.apps/payment-processing-worker created

root@k0s-incus:~# kubectl get po
NAME READY STATUS RESTARTS AGE
payment-processing-worker-747ccfb9db-njgmw 0/1 CrashLoopBackOff 1 (4s ago) 9s

Je peux lancer la requête relative au plug-in avec HolmesGPT et la combinaison CTRL+H pour obtenir cette première réponse :

The payment-processing-container container has crashed and is being restarted for the 6th time due to a CrashLoopBackOff. The last state indicates that the container was terminated with an  
exit code of 0, which suggests that the command executed successfully but did not complete as expected.                                                                                       

To investigate further, you can check the logs of the payment-processing-container container to see if there are any error messages or clues about what is causing the issue. You can also    
check the Kubernetes events for any other errors or warnings that may be related to this issue.                                                                                               

Additionally, you can try to debug the command executed by the payment-processing-container container to see if it's correct and working as expected. The command is:                         


 if [[-z "${DEPLOY_ENV}"]]; then echo Environment variable DEPLOY_ENV is undefined ; else while true; do echo hello; sleep 10;done; fi

This command checks if the DEPLOY_ENV environment variable is set, and if it's not, it prints a message. If it is set, it enters an infinite loop that prints "hello" every 10 seconds.       

If you're running this container in a Kubernetes pod, you can try to debug the issue by checking the pod's logs or using a tool like kubectl to inspect the container's state and logs.       
Press 'q' to exit

Modification de la requête et autre réponse :

root@k0s-incus:~# cat ~/.config/k9s/plugins.yaml
plugins:
  holmesgpt:
    shortCut: Shift-H
    description: Ask HolmesGPT
    scopes:
      - all
    command: bash
    background: false
    confirm: false
    args:
      - -c
      - |
        holmes ask "why is $NAME of $RESOURCE_NAME in -n $NAMESPACE not working and why $NAME is crashed?" --model="openai/llama3.2:3b-instruct-q4_K_S"
        echo "Press 'q' to exit"
        while : ; do
        read -n 1 k <&1
        if [[$k = q]] ; then
        break
        fi
        done

The payment-processing-container container has crashed and is being restarted for the 6th time due to a CrashLoopBackOff. The last state indicates that the container was terminated with an  
exit code of 0, which suggests that the command executed successfully but did not produce any output.                                                                                         

To investigate further, you can check the logs of the payment-processing-container container to see if there are any error messages or clues about what is causing the crash:                 


 kubectl logs payment-processing-worker-747ccfb9db-njgmw -c payment-processing-container

Additionally, you can check the configuration of the payment-processing-container container to ensure that it is running with the correct environment variables and settings.                 


 kubectl describe pod payment-processing-worker-747ccfb9db-njgmw -c payment-processing-container

This will provide more detailed information about the container's configuration and any errors that may be occurring.

HolmesGPT peut s’intégrer plus globalement à la plateforme Robusta via une installation dans le cluster Kubernetes et Helm …

AI Analysis - Robusta documentation

Pour cela génération du fichier YAML de ce type en configuration :

root@k0s-incus:~# cat generated_values.yaml 
globalConfig:
  signing_key: 568927d5-6e65-4c13-b3fe-fdc50e616fde
  account_id: a4d7cea6-fba3-4ce6-ba3d-941b55ec83db
sinksConfig:
  - robusta_sink:
      name: robusta_ui_sink
      token: <TOKEN>
enablePrometheusStack: true
kube-prometheus-stack:
  grafana:
    persistence:
      enabled: true
enablePlatformPlaybooks: true
runner:
  sendAdditionalTelemetry: true
enableHolmesGPT: true
holmes:
  additionalEnvVars:
    - name: ROBUSTA_AI
      value: "true"

Utilisation des commandes et du fichier de configuration YAML fournis par la plateforme Robusta :

root@k0s-incus:~# helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
"robusta" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "robusta" chart repository
Update Complete. ⎈Happy Helming!⎈
root@k0s-incus:~# helm install robusta robusta/robusta -f ./generated_values.yaml --set clusterName="k0s-cluster" \
--set isSmallCluster=true \
--set holmes.resources.requests.memory=512Mi \
--set kube-prometheus-stack.prometheus.prometheusSpec.retentionSize=9GB \
--set kube-prometheus-stack.prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage=10Gi \
--set kube-prometheus-stack.prometheus.prometheusSpec.resources.requests.memory=512Mi
NAME: robusta
LAST DEPLOYED: Tue Jan 14 22:59:09 2025
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing Robusta 0.20.0

As an open source project, we collect general usage statistics.
This data is extremely limited and contains only general metadata to help us understand usage patterns.
If you are willing to share additional data, please do so! It really help us improve Robusta.

You can set sendAdditionalTelemetry: true as a Helm value to send exception reports and additional data.
This is disabled by default.

To opt-out of telemetry entirely, set a ENABLE_TELEMETRY=false environment variable on the robusta-runner deployment.
Note that if the Robusta UI is enabled, telemetry cannot be disabled even if ENABLE_TELEMETRY=false is set.

Visit the web UI at: https://platform.robusta.dev/

root@k0s-incus:~# helm ls -A
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
robusta default 2 2025-01-14 23:10:13.935906491 +0000 UTC deployed robusta-0.20.0 0.20.0

root@k0s-incus:~# kubectl get po,svc
NAME READY STATUS RESTARTS AGE
pod/alertmanager-robusta-kube-prometheus-st-alertmanager-0 0/2 Pending 0 2m
pod/payment-processing-worker-747ccfb9db-njgmw 0/1 CrashLoopBackOff 10 (2m33s ago) 28m
pod/prometheus-robusta-kube-prometheus-st-prometheus-0 0/2 Pending 0 2m
pod/robusta-forwarder-cd847ccc-wxc6d 1/1 Running 0 2m5s
pod/robusta-grafana-8588b8fb85-fv5vj 3/3 Running 0 2m5s
pod/robusta-holmes-55dd58ff6d-m4zth 1/1 Running 0 2m5s
pod/robusta-kube-prometheus-st-operator-6885c8f675-szncg 1/1 Running 0 2m5s
pod/robusta-kube-state-metrics-8667fd9775-s49z4 1/1 Running 0 2m5s
pod/robusta-prometheus-node-exporter-c6jvb 1/1 Running 0 2m5s
pod/robusta-prometheus-node-exporter-j6zp5 1/1 Running 0 2m5s
pod/robusta-runner-5d667b7d9c-dm2z7 1/1 Running 0 2m5s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/alertmanager-operated ClusterIP None <none> 9093/TCP,9094/TCP,9094/UDP 2m1s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 94m
service/prometheus-operated ClusterIP None <none> 9090/TCP 2m1s
service/robusta-forwarder ClusterIP 10.102.7.41 <none> 80/TCP 2m5s
service/robusta-grafana ClusterIP 10.106.69.72 <none> 80/TCP 2m5s
service/robusta-holmes ClusterIP 10.110.124.241 <none> 80/TCP 2m5s
service/robusta-kube-prometheus-st-alertmanager ClusterIP 10.105.101.210 <none> 9093/TCP,8080/TCP 2m5s
service/robusta-kube-prometheus-st-operator ClusterIP 10.103.213.208 <none> 443/TCP 2m5s
service/robusta-kube-prometheus-st-prometheus ClusterIP 10.107.13.104 <none> 9090/TCP,8080/TCP 2m5s
service/robusta-kube-state-metrics ClusterIP 10.103.53.30 <none> 8080/TCP 2m5s
service/robusta-prometheus-node-exporter ClusterIP 10.102.243.65 <none> 9104/TCP 2m5s
service/robusta-runner ClusterIP 10.97.82.15 <none> 80/TCP 2m5s

Je peux procéder à l’installation complête via cette formule :

root@k0s-incus:~# helm upgrade robusta robusta/robusta -f ./generated_values.yaml --set clusterName="k0s-cluster"
Release "robusta" has been upgraded. Happy Helming!
NAME: robusta
LAST DEPLOYED: Tue Jan 14 23:14:02 2025
NAMESPACE: default
STATUS: deployed
REVISION: 5
NOTES:
Thank you for installing Robusta 0.20.0

As an open source project, we collect general usage statistics.
This data is extremely limited and contains only general metadata to help us understand usage patterns.
If you are willing to share additional data, please do so! It really help us improve Robusta.

You can set sendAdditionalTelemetry: true as a Helm value to send exception reports and additional data.
This is disabled by default.

To opt-out of telemetry entirely, set a ENABLE_TELEMETRY=false environment variable on the robusta-runner deployment.
Note that if the Robusta UI is enabled, telemetry cannot be disabled even if ENABLE_TELEMETRY=false is set.

Visit the web UI at: https://platform.robusta.dev/

Le cluster apparaît sur Robusta :

Et là également via HolmesGPT, intterogation de la plateforme sur les éventuelles problématiques rencontrées dans le cluster Kubernetes :

Le tout avec une consommation moindre dans le cluster …

L’utilisation de l’IA pour le dépannage et l’analyse des incidents réduit le temps et l’effort humain nécessaire, permettant aux équipes de se concentrer sur des tâches plus stratégiques.

Les outils comme HolmesGPT et Ollama peuvent être mis à l’échelle en fonction de la demande, ce qui est particulièrement utile dans les environnements de production où la charge de travail peut varier significativement.

On peut donc en conclure que l’intégration de l’IA dans les clusters Kubernetes à l’aide d’outils comme HolmesGPT, Ollama et de fournisseur d’instances GPU comme RunPod, offre des avantages significatifs en termes d’efficiacité, de scalabilité et de tolérance aux pannes.

Ces technologies permettent de rationaliser le cycle de vie des applications, de simplifier le dépannage et d’améliorer la gestion des ressources, rendant ainsi les opérations Kubernetes plus robustes et plus performantes …

À suivre !

Implémenter le Retrieval Augmented Generation (RAG) de manière privée sur Kubernetes avec KubeAI et…

Karim — Mon, 30 Dec 2024 18:01:16 +0000

L’utilisation combinée de KubeAI et Weaviate offre une solution puissante pour implémenter le Retrieval Augmented Generation (RAG) de manière privée et évolutive sur Kubernetes. Dans cet article, nous allons explorer comment mettre en place cette architecture pour créer des applications d’IA avancées.

KubeAI est une plateforme qui permet de déployer des modèles d’IA open source sur Kubernetes, offrant une alternative privée aux services d’IA dans le cloud

KubeAI

KubeAI utilise une API HTTP compatible avec OpenAI et peut être considéré comme un opérateur de modèlequi gère les serveurs vLLM et Ollama.

Pour commencer, il faut déployer KubeAI sur un cluster Kubernetes. Voici les étapes principales : je pars d’une instance Ubuntu 24.04 LTS sur DigitalOcean mettant en oeuvre des processeurs CPU dédiés premium d’Intel …

Introducing Premium CPU-Optimized Droplets | DigitalOcean

Et j’y installe de nouveau k3s pour former mon cluster Kubernetes local avec les clients nécessaires (kubectl et helm) :

K3s

root@kubeai:~# snap install kubectl --classic
2024-12-30T13:37:55Z INFO Waiting for automatic snapd restart...
kubectl 1.31.4 from Canonical✓ installed
root@kubeai:~# snap install helm --classic
helm 3.16.4 from Snapcrafters✪ installed
root@kubeai:~# type kubectl && type helm
kubectl is /snap/bin/kubectl
helm is /snap/bin/helm

root@kubeai:~# curl -sfL https://get.k3s.io | sh -
[INFO] Finding release for channel stable
[INFO] Using v1.31.4+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.31.4+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.31.4+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Skipping /usr/local/bin/kubectl symlink to k3s, command exists in PATH at /snap/bin/kubectl
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s

root@kubeai:~# mkdir .kube && cp /etc/rancher/k3s/k3s.yaml ~/.kube/config && chmod 600 ~/.kube/config
root@kubeai:~# helm ls
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
root@kubeai:~# kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root@kubeai:~# kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-ccb96694c-l6mzp 1/1 Running 0 3m51s
kube-system pod/helm-install-traefik-crd-462cg 0/1 Completed 0 3m51s
kube-system pod/helm-install-traefik-jbx28 0/1 Completed 1 3m51s
kube-system pod/local-path-provisioner-5cf85fd84d-hkdkl 1/1 Running 0 3m51s
kube-system pod/metrics-server-5985cbc9d7-vjmrj 1/1 Running 0 3m51s
kube-system pod/svclb-traefik-75ae73e0-5s2fq 2/2 Running 0 3m43s
kube-system pod/traefik-57b79cf995-b662r 1/1 Running 0 3m43s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 3m58s
kube-system service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 3m53s
kube-system service/metrics-server ClusterIP 10.43.49.80 <none> 443/TCP 3m53s
kube-system service/traefik LoadBalancer 10.43.140.130 164.92.248.129 80:32667/TCP,443:32508/TCP 3m43s

KubeAI offre une compatibilité avec certains API d’OpenAI :

# Implemented #
/v1/chat/completions
/v1/completions
/v1/embeddings
/v1/models
/v1/audio/transcriptions

# Planned #
# /v1/assistants/*
# /v1/batches/*
# /v1/fine_tuning/*
# /v1/images/*
# /v1/vector_stores/*

Je procède au déploiement de KubeAI via Helm en définissant les modèles nécessaires, notamment un modèle d’embedding et un modèle génératif (Le modèle d’intégration nomique est utilisé à la place de text-embedding-ada-002 et Google Gemma 2-2B est utilisé à la place de GPT-3.5-turbo) :

root@kubeai:~# cat kubeai-model-values.yaml 
catalog:
  text-embedding-ada-002:
    enabled: true
    minReplicas: 1
    features: ["TextEmbedding"]
    owner: nomic
    url: "ollama://nomic-embed-text"
    engine: OLlama
    resourceProfile: cpu:1
  gpt-3.5-turbo:
    enabled: true
    minReplicas: 1
    features: ["TextGeneration"]
    owner: google
    url: "ollama://gemma2:2b"
    engine: OLlama
    resourceProfile: cpu:4

root@kubeai:~# helm repo add kubeai https://www.kubeai.org && helm repo update
"kubeai" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kubeai" chart repository
Update Complete. ⎈Happy Helming!⎈
root@kubeai:~# helm install kubeai kubeai/kubeai
NAME: kubeai
LAST DEPLOYED: Mon Dec 30 13:47:00 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

root@kubeai:~# helm install kubeai-models kubeai/models \
    -f ./kubeai-model-values.yaml
NAME: kubeai-models
LAST DEPLOYED: Mon Dec 30 13:47:11 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

root@kubeai:~# kubectl get po,svc
NAME READY STATUS RESTARTS AGE
pod/kubeai-6b7b6866fb-26qwq 1/1 Running 0 3m1s
pod/model-gpt-3.5-turbo-545cf68d8d-v97gs 1/1 Running 0 27s
pod/model-text-embedding-ada-002-64dc467cf4-8m64j 1/1 Running 0 30s
pod/openwebui-55d54bd69-k4ttm 1/1 Running 0 3m1s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubeai ClusterIP 10.43.24.35 <none> 80/TCP,8080/TCP 3m1s
service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 9m12s
service/openwebui ClusterIP 10.43.107.76 <none> 80/TCP 3m1s

Ollama est bien présent avec KubeAI :

(weaviate) root@kubeai:~# kubectl logs pod/model-gpt-3.5-turbo-545cf68d8d-v97gs
Couldn't find '/root/.ollama/id_ed25519'. Generating new private key.
Your new public key is: 

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAGHk39koguimeZFYBSjv9LDxjj5vZRjFmwznLdXSWUV

2024/12/30 13:49:36 routes.go:1259: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:8000 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:999999h0m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2024-12-30T13:49:36.768Z level=INFO source=images.go:757 msg="total blobs: 0"
time=2024-12-30T13:49:36.768Z level=INFO source=images.go:764 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env: export GIN_MODE=release
 - using code: gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers)
[GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST /api/embed --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers)
[GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers)
[GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers)
[GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers)
[GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers)
[GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] GET /api/ps --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers)
[GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)

Weaviate est un moteur de recherche vectoriel qui peut s’intégrer de manière transparente aux modèles d’intégration et de génération de KubeAI. Je vais utiliser KubeAI comme point de terminaison OpenAI pour Weaviate :

KubeAI + Weaviate | Weaviate

Création d’un fichier nommé weaviate-values.yaml avec le contenu suivant pour l’installation de Weaviate via Helm (et activer les modules text2vec-openai et generative-openai ; apiKeyest ignoré dans ce cas car on utilise ici KubeAI comme point de terminaison OpenAI) :

root@kubeai:~# cat weaviate-values.yaml 
modules:
  text2vec-openai:
    enabled: true
    apiKey: thisIsIgnored
  generative-openai:
    enabled: true
    apiKey: thisIsIgnored
  default_vectorizer_module: text2vec-openai
service:
  # To prevent Weaviate being exposed publicly
  type: ClusterIP

root@kubeai:~# helm repo add weaviate https://weaviate.github.io/weaviate-helm && helm repo update
"weaviate" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "weaviate" chart repository
...Successfully got an update from the "kubeai" chart repository
Update Complete. ⎈Happy Helming!⎈

root@kubeai:~# helm install \
  "weaviate" \
  weaviate/weaviate \
  -f weaviate-values.yaml
NAME: weaviate
LAST DEPLOYED: Mon Dec 30 13:51:48 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

root@kubeai:~# kubectl get po,svc
NAME READY STATUS RESTARTS AGE
pod/kubeai-6b7b6866fb-26qwq 1/1 Running 0 5m18s
pod/model-gpt-3.5-turbo-545cf68d8d-v97gs 1/1 Running 0 2m44s
pod/model-text-embedding-ada-002-64dc467cf4-8m64j 1/1 Running 0 2m47s
pod/openwebui-55d54bd69-k4ttm 1/1 Running 0 5m18s
pod/weaviate-0 1/1 Running 0 31s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubeai ClusterIP 10.43.24.35 <none> 80/TCP,8080/TCP 5m18s
service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 11m
service/openwebui ClusterIP 10.43.107.76 <none> 80/TCP 5m18s
service/weaviate ClusterIP 10.43.211.85 <none> 80/TCP 31s
service/weaviate-grpc LoadBalancer 10.43.37.63 164.92.248.129 50051:31360/TCP 31s
service/weaviate-headless ClusterIP None <none> 80/TCP 31s

Je vais exécuter des requêtes en Python pour interagir avec Weaviate. Installation de l’environnement nécessaire via uv (gestionnaire de paquets et de projets pour Python extrêmement rapide et écrit en Rust) :

root@kubeai:~# curl -LsSf https://astral.sh/uv/install.sh | sh
downloading uv 0.5.13 x86_64-unknown-linux-gnu
no checksums to verify
installing to /root/.local/bin
  uv
  uvx
everything's installed!

To add $HOME/.local/bin to your PATH, either restart your shell or run:

    source $HOME/.local/bin/env (sh, bash, zsh)
    source $HOME/.local/bin/env.fish (fish)

root@kubeai:~# source $HOME/.local/bin/env

root@kubeai:~# uv
An extremely fast Python package manager.

Usage: uv [OPTIONS] <COMMAND>

Commands:
  run Run a command or script
  init Create a new project
  add Add dependencies to the project
  remove Remove dependencies from the project
  sync Update the project's environment
  lock Update the project's lockfile
  export Export the project's lockfile to an alternate format
  tree Display the project's dependency tree
  tool Run and install commands provided by Python packages
  python Manage Python versions and installations
  pip Manage Python packages with a pip-compatible interface
  venv Create a virtual environment
  build Build Python packages into source distributions and wheels
  publish Upload distributions to an index
  cache Manage uv's cache
  self Manage the uv executable
  version Display uv's version
  help Display documentation for a command

Cache options:
  -n, --no-cache Avoid reading from or writing to the cache, instead using a temporary directory for the duration of the operation [env: UV_NO_CACHE=]
      --cache-dir <CACHE_DIR> Path to the cache directory [env: UV_CACHE_DIR=]

Python options:
      --python-preference <PYTHON_PREFERENCE> Whether to prefer uv-managed or system Python installations [env: UV_PYTHON_PREFERENCE=] [possible values: only-managed, managed, system,
                                               only-system]
      --no-python-downloads Disable automatic downloads of Python. [env: "UV_PYTHON_DOWNLOADS=never"]

Global options:
  -q, --quiet Do not print any output
  -v, --verbose... Use verbose output
      --color <COLOR_CHOICE> Control colors in output [default: auto] [possible values: auto, always, never]
      --native-tls Whether to load TLS certificates from the platform's native certificate store [env: UV_NATIVE_TLS=]
      --offline Disable network access [env: UV_OFFLINE=]
      --allow-insecure-host <ALLOW_INSECURE_HOST> Allow insecure connections to a host [env: UV_INSECURE_HOST=]
      --no-progress Hide all progress outputs [env: UV_NO_PROGRESS=]
      --directory <DIRECTORY> Change to the given directory prior to running the command
      --project <PROJECT> Run the command within the given project directory
      --config-file <CONFIG_FILE> The path to a `uv.toml` file to use for configuration [env: UV_CONFIG_FILE=]
      --no-config Avoid discovering configuration files (`pyproject.toml`, `uv.toml`) [env: UV_NO_CONFIG=]
  -h, --help Display the concise help for this command
  -V, --version Display the uv version

Use `uv help` for more details.

root@kubeai:~# uv self update
info: Checking for updates...
success: You're on the latest version of uv (v0.5.13)
root@kubeai:~# uv venv weaviate
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: weaviate
Activate with: source weaviate/bin/activate

root@kubeai:~# source weaviate/bin/activate

Installation du client Python pour Weaviate à cette étape :

(weaviate) root@kubeai:~# uv pip install -U weaviate-client requests
Using Python 3.12.3 environment at: weaviate
Resolved 25 packages in 382ms
Prepared 25 packages in 252ms
Installed 25 packages in 11ms
 + annotated-types==0.7.0
 + anyio==4.7.0
 + authlib==1.3.1
 + certifi==2024.12.14
 + cffi==1.17.1
 + charset-normalizer==3.4.1
 + cryptography==44.0.0
 + grpcio==1.68.1
 + grpcio-health-checking==1.68.1
 + grpcio-tools==1.68.1
 + h11==0.14.0
 + httpcore==1.0.7
 + httpx==0.28.1
 + idna==3.10
 + protobuf==5.29.2
 + pycparser==2.22
 + pydantic==2.10.4
 + pydantic-core==2.27.2
 + requests==2.32.3
 + setuptools==75.6.0
 + sniffio==1.3.1
 + typing-extensions==4.12.2
 + urllib3==2.3.0
 + validators==0.34.0
 + weaviate-client==4.10.2

Weaviate n’est pas exposé publiquement dans cette configuration. Je configure des redirections de port locales pour accéder aux services Weaviate :

(weaviate) root@kubeai:~# screen -L -S weaviate-http
[detached from 11693.weaviate-http]
(weaviate) root@kubeai:~# screen -L -S weaviate-grpc
[detached from 11996.weaviate-grpc]
(weaviate) root@kubeai:~# tail -f screenlog.0
root@kubeai:~# kubectl port-forward svc/weaviate 8080:80 && kubectl port-forward svc/weaviate-grpc 50051:50051
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
root@kubeai:~# kubectl port-forward svc/weaviate-grpc 50051:50051
Forwarding from 127.0.0.1:50051 -> 50051
Forwarding from [::1]:50051 -> 50051

(weaviate) root@kubeai:~# netstat -tunlp | grep kubectl
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 11710/kubectl       
tcp 0 0 127.0.0.1:50051 0.0.0.0:* LISTEN 12018/kubectl       
tcp6 0 0 ::1:8080 :::* LISTEN 11710/kubectl       
tcp6 0 0 ::1:50051 :::* LISTEN 12018/kubectl

J’aborde le cas d’une recherche sémantique à l’aide d’un modèle d’intégration avec collecte et importation de données. Pour cela création et exécution du fichier Python suivant :

(weaviate) root@kubeai:~# cat create-collection.py 

import json
import weaviate
import requests
from weaviate.classes.config import Configure

# This works due to port forward in previous step
with weaviate.connect_to_local(port=8080, grpc_port=50051) as client:

    client.collections.create(
        "Question",
        vectorizer_config=Configure.Vectorizer.text2vec_openai(
                model="text-embedding-ada-002",
                base_url="http://kubeai/openai",
        ),
        generative_config=Configure.Generative.openai(
            model="gpt-3.5-turbo",
            base_url="http://kubeai/openai",
        ),
    )

    # import data
    resp = requests.get('https://raw.githubusercontent.com/weaviate-tutorials/quickstart/main/data/jeopardy_tiny.json')
    data = json.loads(resp.text) # Load data

    question_objs = list()
    for i, d in enumerate(data):
        question_objs.append({
            "answer": d["Answer"],
            "question": d["Question"],
            "category": d["Category"],
        })

    questions = client.collections.get("Question")
    questions.data.insert_many(question_objs)
    print("Data imported successfully")

(weaviate) root@kubeai:~# python create-collection.py
"Data imported successfully"

On se base sur le petit fichier JSON (questions/réponses sur le jeu Jeopardy).

[{"Category":"SCIENCE","Question":"This organ removes excess glucose from the blood & stores it as glycogen","Answer":"Liver"},{"Category":"ANIMALS","Question":"It's the only living mammal in the order Proboseidea","Answer":"Elephant"},{"Category":"ANIMALS","Question":"The gavial looks very much like a crocodile except for this bodily feature","Answer":"the nose or snout"},{"Category":"ANIMALS","Question":"Weighing around a ton, the eland is the largest species of this animal in Africa","Answer":"Antelope"},{"Category":"ANIMALS","Question":"Heaviest of all poisonous snakes is this North American rattlesnake","Answer":"the diamondback rattler"},{"Category":"SCIENCE","Question":"2000 news: the Gunnison sage grouse isn't just another northern sage grouse, but a new one of this classification","Answer":"species"},{"Category":"SCIENCE","Question":"A metal that is ductile can be pulled into this while cold & under pressure","Answer":"wire"},{"Category":"SCIENCE","Question":"In 1953 Watson & Crick built a model of the molecular structure of this, the gene-carrying substance","Answer":"DNA"},{"Category":"SCIENCE","Question":"Changes in the tropospheric layer of this are what gives us weather","Answer":"the atmosphere"},{"Category":"SCIENCE","Question":"In 70-degree air, a plane traveling at about 1,130 feet per second breaks it","Answer":"Sound barrier"}]

La collection est maintenant créée et les données sont importées. Les vecteurs sont générés par KubeAI et stockés dans Weaviate. Je peux effectuer une recherche sémantique en lien avec le terme “biology” qui utilise les intégrations via la création de ce fichier Python :

(weaviate) root@kubeai:~# cat search.py 
import weaviate
from weaviate.classes.config import Configure

# This works due to port forward in previous step
with weaviate.connect_to_local(port=8080, grpc_port=50051) as client:
    questions = client.collections.get("Question")
    response = questions.query.near_text(
        query="biology",
        limit=2
    )
    print(response.objects[0].properties) # Inspect the first object

avec la réponse suivante :

(weaviate) root@kubeai:~# python search.py
{'answer': 'DNA', 'question': 'In 1953 Watson & Crick built a model of the molecular structure of this, the gene-carrying substance', 'category': 'SCIENCE'}

Je peux modifier ma recherche sémantique avec le terme “poison” par exemple :

(weaviate) root@kubeai:~# cat search.py 
import weaviate
from weaviate.classes.config import Configure

# This works due to port forward in previous step
with weaviate.connect_to_local(port=8080, grpc_port=50051) as client:
    questions = client.collections.get("Question")
    response = questions.query.near_text(
        query="poison",
        limit=2
    )
    print(response.objects[0].properties) # Inspect the first object

(weaviate) root@kubeai:~# python search.py
{'answer': 'the diamondback rattler', 'question': 'Heaviest of all poisonous snakes is this North American rattlesnake', 'category': 'ANIMALS'}

Passons maintenant à la recherche générative, qui utilise le modèle génératif (Génération de texte LLM). Le modèle génératif est exécuté localement et géré par KubeAI. Création d’un fichier nommé generate.pyavec le contenu suivant pour genérer un tweet avec des emojis en lien avec la recherche sémantique précédente (et en utilisant les redirections de port locales) :


(weaviate) root@kubeai:~# cat generate.py 
import weaviate
from weaviate.classes.config import Configure

# This works due to port forward in previous step
with weaviate.connect_to_local(port=8080, grpc_port=50051) as client:
    questions = client.collections.get("Question")

    response = questions.generate.near_text(
        query="biology",
        limit=2,
        grouped_task="Write a tweet with emojis about these facts."
    )

    print(response.generated) # Inspect the generated text

avec cette réponse …

(weaviate) root@kubeai:~# python generate.py
🧬 **Watson & Crick** cracked the code in 1953! 🤯 They built a model of DNA, the blueprint of life. 🧬

🧠 **Liver power!** 💪 This organ keeps your blood sugar balanced by storing glucose as glycogen. 🩸 #ScienceFacts #Biology

L’importation de données a permis d’effectuer des recherches et à générer du contenu à l’aide de modèles gérés par KubeAI. Il est possible de l’utiliser avec LangChain qui facilite la création d’applications basées sur des LLM :

Introduction | 🦜️🔗 LangChain

LangChain peut en effet interagir avec l’API compatible OpenAI de KubeAI. Installation du client Python pour LangChain avec uv :

(weaviate) root@kubeai:~# uv pip install -U langchain_openai
Using Python 3.12.3 environment at: weaviate
Resolved 30 packages in 1.32s
Prepared 16 packages in 93ms
Installed 16 packages in 7ms
 + distro==1.9.0
 + jiter==0.8.2
 + jsonpatch==1.33
 + jsonpointer==3.0.0
 + langchain-core==0.3.28
 + langchain-openai==0.2.14
 + langsmith==0.2.6
 + openai==1.58.1
 + orjson==3.10.13
 + packaging==24.2
 + pyyaml==6.0.2
 + regex==2024.11.6
 + requests-toolbelt==1.0.0
 + tenacity==9.0.0
 + tiktoken==0.8.0
 + tqdm==4.67.1

Accession locale à l’API compatible KubeAI OpenAI pour faciliter la tâche via un transfert de port vers le service KubeAI :

(weaviate) root@kubeai:~# screen -L -S kubeai
[detached from 15501.kubeai]
(weaviate) root@kubeai:~# cat screenlog.0 
root@kubeai:~# kubectl port-forward svc/kubeai 8000:80
Forwarding from 127.0.0.1:8000 -> 8000
Forwarding from [::1]:8000 -> 8000
(weaviate) root@kubeai:~# netstat -tunlp | grep kubectl
tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 15511/kubectl       
tcp6 0 0 ::1:8000 :::* LISTEN 15511/kubectl

Création d’un fichier Python simple qui utilise LangChain et connecté à KubeAI (c’est cette interface compatible avec OpenAI qui est consommée au lieu de l’API publique OpenAI par défaut) :

(weaviate) root@kubeai:~# cat test-langchain.py 

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-3.5-turbo",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    api_key="thisIsIgnored",
    base_url="http://localhost:8000/openai/v1",
)

messages = [
    (
        "system",
        "You are a helpful assistant in cloud native technologies",
    ),
    ("How can i create a kubernetes cluster with k3s ?"),
]
ai_msg = llm.invoke(messages)
print(ai_msg.content)

Je pose une question au LLM embarqué via Ollama sur la création d’un cluster k3s avec cette réponse :

(weaviate) root@kubeai:~# python test-langchain.py

Let's get you started with creating a Kubernetes cluster using K3s! 

**What is K3s?**

K3s is a lightweight, production-ready distribution of Kubernetes designed for simplicity and efficiency. It's ideal for:

* **Small to medium-sized deployments:** It excels in environments where resource constraints are a factor.
* **Edge computing:** Deploying applications on devices with limited resources.
* **DevOps teams:** Its ease of use makes it perfect for rapid prototyping and deployment. 

**Steps to Create a K3s Cluster**

Here's a breakdown of the process, along with explanations:

1. **Prerequisites:**
   * **Hardware:** You'll need a machine (physical or virtual) capable of running Kubernetes. K3s is designed for low-resource environments. 
   * **Networking:** Ensure your machine has network connectivity and can reach the internet.
   * **Basic Linux knowledge:** Familiarity with command-line tools like `curl`, `wget`, and basic terminal navigation will be helpful.

2. **Installation:**
   * **Download K3s:** Visit the official K3s website ([https://k3s.io/](https://k3s.io/)) to download the latest version of K3s for your operating system (e.g., Linux, macOS). 
   * **Installation:** Follow the instructions provided in the K3s documentation to install K3s on your machine. You'll likely need to use a terminal or command-line interface.

3. **Cluster Configuration:**
   * **Networking:** Configure your network settings for your K3s cluster (e.g., IP addresses, subnet masks). 
   * **Storage:** Decide how you want to store data within your cluster (e.g., local disk, persistent volumes).
   * **Security:** Implement security measures like TLS certificates and firewall rules if needed.

4. **Initialization:**
   * **Start the K3s Cluster:** Use the `k3s` command-line tool to start the cluster. 
   * **Access the Dashboard (Optional):** If you're using a dashboard, follow the instructions provided by your chosen dashboard provider.

5. **Deploying Applications:**
   * **kubectl:** Use the `kubectl` command-line tool to interact with your K3s cluster and deploy applications. 
   * **YAML Configuration:** Create Kubernetes YAML files (configuration files) for your desired application deployments. 

**Example Commands:**


# Install K3s on Ubuntu
curl -LO https://get.k3s.io/install.sh
chmod +x install.sh
./install.sh

# Start the cluster
k3s start


**Key Advantages of K3s:**

* **Simplicity:** K3s is designed for ease of use, making it a great choice for beginners and experienced users alike. 
* **Lightweight:** It's incredibly efficient, requiring minimal resources to run. This makes it ideal for edge deployments or environments with limited hardware.
* **Fast Deployment:** K3s offers quick cluster setup times, allowing you to get your applications running faster.

**Additional Resources:**

* **K3s Website:** [https://k3s.io/](https://k3s.io/) 
* **K3s Documentation:** [https://docs.k3s.io/](https://docs.k3s.io/)

Let me know if you have any specific questions or want to dive deeper into a particular aspect of K3s!

Pour plus de visibiliter, j’insère Rancher Server sur ce cluster k3s avec les commandes suivantes :

helm repo add rancher-latest https://releases.rancher.com/server-charts/latest

kubectl create namespace cattle-system

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.crds.yaml

helm repo add jetstack https://charts.jetstack.io

helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace

"rancher-latest" has been added to your repositories
namespace/cattle-system created
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
"jetstack" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "weaviate" chart repository
...Successfully got an update from the "kubeai" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "rancher-latest" chart repository
Update Complete. ⎈Happy Helming!⎈
NAME: cert-manager
LAST DEPLOYED: Mon Dec 30 15:00:45 2024
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
cert-manager v1.16.2 has been deployed successfully!

In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).

More information on the different types of issuers and how to configure them
can be found in our documentation:

https://cert-manager.io/docs/configuration/

For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:

https://cert-manager.io/docs/usage/ingress/


(weaviate) root@kubeai:~# helm install rancher rancher-latest/rancher \
  --namespace cattle-system \
  --set hostname=rancher.164.92.248.129.sslip.io \
  --set replicas=1 \
  --set bootstrapPassword=nochangeme
NAME: rancher
LAST DEPLOYED: Mon Dec 30 15:01:59 2024
NAMESPACE: cattle-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Rancher Server has been installed.

NOTE: Rancher may take several minutes to fully initialize. Please standby while Certificates are being issued, Containers are started and the Ingress rule comes up.

Check out our docs at https://rancher.com/docs/

If you provided your own bootstrap password during installation, browse to https://rancher.164.92.248.129.sslip.io to get started.

If this is the first time you installed Rancher, get started by running this command and clicking the URL it generates:


echo https://rancher.164.92.248.129.sslip.io/dashboard/?setup=$(kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}')


To get just the bootstrap password on its own, run:


kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{ "\n" }}'


Happy Containering!

Je modifie le fichier YAML pour KubeAI pour bénéficier d’un accès à Ollama :

root@kubeai:~# cat <<EOF > models-helm-values.yaml
catalog:
  gemma2-2b-cpu:
    enabled: true
    minReplicas: 1
EOF

root@kubeai:~# helm upgrade kubeai-models kubeai/models \
    -f ./models-helm-values.yaml

Release "kubeai-models" has been upgraded. Happy Helming!
NAME: kubeai-models
LAST DEPLOYED: Mon Dec 30 15:14:49 2024
NAMESPACE: default
STATUS: deployed
REVISION: 2
TEST SUITE: None
root@kubeai:~# helm ls
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
kubeai default 1 2024-12-30 13:47:00.713372779 +0000 UTC deployed kubeai-0.10.0 v0.12.0    
kubeai-models default 2 2024-12-30 15:14:49.154697606 +0000 UTC deployed models-0.10.0 1.16.0     
weaviate default 1 2024-12-30 13:51:48.357464607 +0000 UTC deployed weaviate-17.3.3 1.27.8

Avec de nouveau une redirection locale de port pour accéder à Open WebUI (intégré à KubeAI), une interface utilisateur extensible, riche en fonctionnalités et conviviale, conçue pour fonctionner entièrement hors ligne. Elle prend en charge différents runners LLM, y compris Ollama et les API compatibles avec OpenAI :

(weaviate) root@kubeai:~# kubectl port-forward service/openwebui 10000:80 --address='0.0.0.0'
Forwarding from 0.0.0.0:10000 -> 8080
Handling connection for 10000
Handling connection for 10000
Handling connection for 10000

Accession à l’interface utilisateur d’Open WebUI :

C’est ce petit LLM utilisé précedemment qui est encore mis en oeuvre ici :

J’effectue alors une recherche sur la base d’un document PDF extrait de la documentation officielle de k3s via cette interface …

Dans cet article, j’ai utiliser uniquement des modèles CPU (il devrait donc fonctionner même sur votre ordinateur portable) qui fait que KubeAI peut fonctionner sur votre matériel existant, réduisant ainsi le besoin de payer pour des intégrations et des modèles génératifs.

KubeAI s’exécute localement dans votre cluster Kubernetes, de sorte que vos données ne quittent jamais votre infrastructure. Vous pouvez facilement changer ou mettre à jour les modèles utilisés sans modifier votre code applicatif et KubeAI permet une mise à l’échelle automatique des modèles en fonction de la charge.

En conclusion, on a vu que l’utilisation combinée de KubeAI et Weaviate offre une solution robuste et flexible pour implémenter le RAG de manière privée et évolutive. Cette approche permet aux entreprises de bénéficier des avancées de l’IA tout en gardant le contrôle sur leurs données et leur infrastructure …

À suivre !

AIOps : Déboguer son cluster Kubernetes en utilisant l’intelligence artificielle générative via…

Karim — Sat, 28 Dec 2024 14:57:15 +0000

Les avancées dans l’Intelligence Artificielle Générative et sa mise en oeuvre simplifiée via certains outils, transforment profondément la gestion des clusters Kubernetes via ce concept d’ AIOps (acronyme anglais de Artificial Intelligence for IT Operations) énoncé par Gartner, un processus dans lequel vous utilisez des techniques d’intelligence artificielle (IA) pour maintenir par exemple une infrastructure. L’un des domaines dans lesquels les ingénieurs DevOps et les opérateurs de cluster débutants sont souvent confrontés, est relatif aux défis de l’identification, de la compréhension et de la résolution de problèmes au sein d’un cluster Kubernetes …

Dans cet article, nous allons donc explorer comment configurer et utiliser K8sGPT, un outil open source basé sur l’IA générative, en combinaison avec Ollama et le modèle Falcon3, pour identifier et résoudre les problèmes dans un cluster Kubernetes.

Pour cela, je pars ici d’une instance Ubuntu 24.04 LTS dans DigitalOcean :

En y installant localement le moteur Docker …

(base) root@k8sgpt:~# curl -fsSL https://get.docker.com | sh -
# Executing docker install script, commit: 4c94a56999e10efcf48c5b8e3f6afea464f9108e
+ sh -c apt-get -qq update >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get -y -qq install ca-certificates curl >/dev/null
Scanning processes...                                                                                                                                                                         
Scanning candidates...                                                                                                                                                                        
Scanning linux images...                                                                                                                                                                      
+ sh -c install -m 0755 -d /etc/apt/keyrings
+ sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" -o /etc/apt/keyrings/docker.asc
+ sh -c chmod a+r /etc/apt/keyrings/docker.asc
+ sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu noble stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get -qq update >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get -y -qq install docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
Scanning processes...                                                                                                                                                                         
Scanning candidates...                                                                                                                                                                        
Scanning linux images...                                                                                                                                                                      
+ sh -c docker version
Client: Docker Engine - Community
 Version: 27.4.1
 API version: 1.47
 Go version: go1.22.10
 Git commit: b9d17ea
 Built: Tue Dec 17 15:45:46 2024
 OS/Arch: linux/amd64
 Context: default

Server: Docker Engine - Community
 Engine:
  Version: 27.4.1
  API version: 1.47 (minimum version 1.24)
  Go version: go1.22.10
  Git commit: c710b88
  Built: Tue Dec 17 15:45:46 2024
  OS/Arch: linux/amd64
  Experimental: false
 containerd:
  Version: 1.7.24
  GitCommit: 88bf19b2105c8b17560993bee28a01ddc2f97182
 runc:
  Version: 1.2.2
  GitCommit: v1.2.2-0-g7cb3632
 docker-init:
  Version: 0.19.0
  GitCommit: de40ad0

================================================================================

To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:

    dockerd-rootless-setuptool.sh install

Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.

To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/

WARNING: Access to the remote API on a privileged Docker daemon is equivalent
         to root access on the host. Refer to the 'Docker daemon attack surface'
         documentation for details: https://docs.docker.com/go/attack-surface/

================================================================================

Et je suis capable d’y lancer directmeent Ollama via son image officiellle pour exécuter des modèles de langage de grande taille (LLM) localement en y utilisant les CPU premium d’Intel présents dans l’instance Ubuntu :

Ollama is now available as an official Docker image · Ollama Blog

(base) root@k8sgpt:~# docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:latest
Unable to find image 'ollama/ollama:latest' locally
latest: Pulling from ollama/ollama
6414378b6477: Pull complete 
9423a26b200c: Pull complete 
629da9618c4f: Pull complete 
00b71e3f044c: Pull complete 
Digest: sha256:18bfb1d605604fd53dcad20d0556df4c781e560ebebcd923454d627c994a0e37
Status: Downloaded newer image for ollama/ollama:latest
7b09d9fcdacff4319e553c41f741a15266eb5a5ec745959363e7754c53a203ef

(base) root@k8sgpt:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7b09d9fcdacf ollama/ollama:latest "/bin/ollama serve" About a minute ago Up About a minute 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama


(base) root@k8sgpt:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name    
tcp 0 0 0.0.0.0:11434 0.0.0.0:* LISTEN 46227/docker-proxy  
tcp 0 0 127.0.0.54:53 0.0.0.0:* LISTEN 744/systemd-resolve 
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 744/systemd-resolve 
tcp6 0 0 :::22 :::* LISTEN 1/init              
tcp6 0 0 :::11434 :::* LISTEN 46235/docker-proxy  
udp 0 0 127.0.0.54:53 0.0.0.0:* 744/systemd-resolve 
udp 0 0 127.0.0.53:53 0.0.0.0:* 744/systemd-resolve

Et en y chargeant le grand modèle Falcon 3 développé par le Technology Innovation Institute (TII) d’Abu Dhab et présent dans la bibliothèque de grands modèles d’Ollama :

(base) root@k8sgpt:~# docker exec -it ollama ollama pull falcon3
pulling manifest 
pulling 3717a52b7aea... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 4.6 GB                         
pulling 803b5adc3448... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 218 B                         
pulling 58f83c52a4e3... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 13 KB                         
pulling 35e31ed4c388... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 101 B                         
pulling acb75345e14b... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 487 B                         
verifying sha256 digest 
writing manifest 
success 
(base) root@k8sgpt:~# docker exec -it ollama ollama list
NAME ID SIZE MODIFIED       
falcon3:latest 472ea1c89f64 4.6 GB 11 seconds ago

Je peux à cette étape y charger K8sGPT, un outil pour scanner vos clusters kubernetes, diagnostiquer et trier les problèmes en anglais simple. Il dispose d’une expérience SRE codifiée dans ses analyseurs et aide à extraire les informations les plus pertinentes pour les enrichir avec de l’IA générative.

K8sGPT fonctionne en trois étapes :

Extraction : Récupération des détails de configuration de toutes les charges de travail déployées dans le cluster.
Filtration : Un composant appelé « analyzer » filtre les données nécessaires.
Génération : Les données filtrées sont traitées pour générer des insights et des rapports en anglais simple.

K8sGPT

(base) root@k8sgpt:~# curl -LO https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.48/k8sgpt_amd64.deb
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 34.5M 100 34.5M 0 0 30.5M 0 0:00:01 0:00:01 --:--:-- 30.5M
(base) root@k8sgpt:~# dpkg -i k8sgpt_amd64.deb 
(Reading database ... 74447 files and directories currently installed.)
Preparing to unpack k8sgpt_amd64.deb ...
Unpacking k8sgpt (0.3.48) over (0.3.48) ...
Setting up k8sgpt (0.3.48) ...

(base) root@k8sgpt:~# k8sgpt
Kubernetes debugging powered by AI

Usage:
  k8sgpt [command]

Available Commands:
  analyze This command will find problems within your Kubernetes cluster
  auth Authenticate with your chosen backend
  cache For working with the cache the results of an analysis
  completion Generate the autocompletion script for the specified shell
  custom-analyzer Manage a custom analyzer
  dump Creates a dumpfile for debugging issues with K8sGPT
  filters Manage filters for analyzing Kubernetes resources
  generate Generate Key for your chosen backend (opens browser)
  help Help about any command
  integration Integrate another tool into K8sGPT
  serve Runs k8sgpt as a server
  version Print the version number of k8sgpt

Flags:
      --config string Default config file (/root/.config/k8sgpt/k8sgpt.yaml)
  -h, --help help for k8sgpt
      --kubeconfig string Path to a kubeconfig. Only required if out-of-cluster.
      --kubecontext string Kubernetes context to use. Only required if out-of-cluster.

Use "k8sgpt [command] --help" for more information about a command.

La ligne de commande est alors disponible pour y vérifier les fournisseurs disponibles localement dans cette instance :

Installation


(base) root@k8sgpt:~# k8sgpt auth list
Default: 
> openai
Active: 
Unused: 
> openai
> localai
> ollama
> azureopenai
> cohere
> amazonbedrock
> amazonsagemaker
> google
> noopai
> huggingface
> googlevertexai
> oci
> ibmwatsonxai

Ollama sera utilisé comme fournisseur de backend IA pour K8sGPT au travers de LocalAI (qui agit comme l’API REST de remplacement compatible avec les spécifications de l’API OpenAI pour une inférence locale).

Voici la commande pour configurer K8sGPT avec Ollama et le modèle Falcon3 :

(base) root@k8sgpt:~# k8sgpt auth add --backend localai --model falcon3 --baseurl http://localhost:11434/v1
localai added to the AI backend provider list

Je lance un cluster Kubernetes managé sur DigitalOcean via DigitalOcean Kubernetes (DOKS) :

DigitalOcean Managed Kubernetes | Starting at $12/mo.

Récupération du fichier Kubeconfig depuis ce cluster pour l’insérer localement sur l’instance Ubuntu pour l’utiliser avec le client Kubectl :

(base) root@k8sgpt:~# curl -LO https://dl.k8s.io/release/v1.32.0/bin/linux/amd64/kubectl && chmod +x ./kubectl && mv kubectl /usr/local/bin/ && kubectl
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 138 100 138 0 0 1000 0 --:--:-- --:--:-- --:--:-- 1007
100 54.6M 100 54.6M 0 0 120M 0 --:--:-- --:--:-- --:--:-- 120M
kubectl controls the Kubernetes cluster manager.

 Find more information at: https://kubernetes.io/docs/reference/kubectl/

Basic Commands (Beginner):
  create Create a resource from a file or from stdin
  expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes service
  run Run a particular image on the cluster
  set Set specific features on objects

Basic Commands (Intermediate):
  explain Get documentation for a resource
  get Display one or many resources
  edit Edit a resource on the server
  delete Delete resources by file names, stdin, resources and names, or by resources and label selector

Deploy Commands:
  rollout Manage the rollout of a resource
  scale Set a new size for a deployment, replica set, or replication controller
  autoscale Auto-scale a deployment, replica set, stateful set, or replication controller

Cluster Management Commands:
  certificate Modify certificate resources
  cluster-info Display cluster information
  top Display resource (CPU/memory) usage
  cordon Mark node as unschedulable
  uncordon Mark node as schedulable
  drain Drain node in preparation for maintenance
  taint Update the taints on one or more nodes

Troubleshooting and Debugging Commands:
  describe Show details of a specific resource or group of resources
  logs Print the logs for a container in a pod
  attach Attach to a running container
  exec Execute a command in a container
  port-forward Forward one or more local ports to a pod
  proxy Run a proxy to the Kubernetes API server
  cp Copy files and directories to and from containers
  auth Inspect authorization
  debug Create debugging sessions for troubleshooting workloads and nodes
  events List events

Advanced Commands:
  diff Diff the live version against a would-be applied version
  apply Apply a configuration to a resource by file name or stdin
  patch Update fields of a resource
  replace Replace a resource by file name or stdin
  wait Experimental: Wait for a specific condition on one or many resources
  kustomize Build a kustomization target from a directory or URL

Settings Commands:
  label Update the labels on a resource
  annotate Update the annotations on a resource
  completion Output shell completion code for the specified shell (bash, zsh, fish, or powershell)

Subcommands provided by plugins:

Other Commands:
  api-resources Print the supported API resources on the server
  api-versions Print the supported API versions on the server, in the form of "group/version"
  config Modify kubeconfig files
  plugin Provides utilities for interacting with plugins
  version Print the client and server version information

Usage:
  kubectl [flags] [options]

Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).

(base) root@k8sgpt:~# kubectl cluster-info
Kubernetes control plane is running at https://738af175-32d4-43e9-9e31-b7ae3058be3e.k8s.ondigitalocean.com
CoreDNS is running at https://738af175-32d4-43e9-9e31-b7ae3058be3e.k8s.ondigitalocean.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
(base) root@k8sgpt:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
pool-kuaxj3k47-ejvl3 Ready <none> 6m40s v1.31.1 10.110.0.3 159.65.200.52 Debian GNU/Linux 12 (bookworm) 6.1.0-27-amd64 containerd://1.6.31
pool-kuaxj3k47-ejvl8 Ready <none> 6m38s v1.31.1 10.110.0.2 164.92.147.176 Debian GNU/Linux 12 (bookworm) 6.1.0-27-amd64 containerd://1.6.31
(base) root@k8sgpt:~# kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/cilium-4gxht 1/1 Running 0 6m46s
kube-system pod/cilium-lw8gd 1/1 Running 0 6m48s
kube-system pod/coredns-c5c6457c-bnzfc 0/1 Running 0 36s
kube-system pod/coredns-c5c6457c-nz6gr 0/1 Running 0 36s
kube-system pod/cpc-bridge-proxy-ebpf-7ncbq 1/1 Running 0 55s
kube-system pod/cpc-bridge-proxy-ebpf-qth8w 1/1 Running 0 55s
kube-system pod/hubble-relay-67597fb8-kmlw5 1/1 Running 1 (51s ago) 8m40s
kube-system pod/hubble-ui-79957d9f7b-4n9kj 2/2 Running 0 74s
kube-system pod/konnectivity-agent-7ml7p 1/1 Running 0 61s
kube-system pod/konnectivity-agent-tnf8j 1/1 Running 0 61s
kube-system pod/kube-proxy-ebpf-4gt2z 1/1 Running 0 6m48s
kube-system pod/kube-proxy-ebpf-ztjql 1/1 Running 0 6m46s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.108.32.1 <none> 443/TCP 9m54s
kube-system service/hubble-peer ClusterIP 10.108.54.62 <none> 443/TCP 8m40s
kube-system service/hubble-relay ClusterIP 10.108.41.20 <none> 80/TCP 8m40s
kube-system service/hubble-ui ClusterIP 10.108.52.164 <none> 80/TCP 8m40s
kube-system service/kube-dns ClusterIP 10.108.32.10 <none> 53/UDP,53/TCP,9153/TCP 36s

Installation d’Headlamp, une interface web qui remplace le dashboard traditionnel de Kubernetes, facile à utiliser et extensible.

Headlamp a été créé pour combiner les fonctionnalités traditionnelles des autres interfaces web et tableaux de bord (c’est-à-dire pour lister et visualiser les ressources) avec des fonctionnalités supplémentaires.

(base) root@k8sgpt:~# kubectl apply -f https://raw.githubusercontent.com/kinvolk/headlamp/main/kubernetes-headlamp.yaml
service/headlamp created
deployment.apps/headlamp created
secret/headlamp-admin created
(base) root@k8sgpt:~# kubectl get po,svc -n kube-system
NAME READY STATUS RESTARTS AGE
pod/cilium-4gxht 1/1 Running 0 11m
pod/cilium-lw8gd 1/1 Running 0 11m
pod/coredns-c5c6457c-bnzfc 1/1 Running 0 5m7s
pod/coredns-c5c6457c-nz6gr 1/1 Running 0 5m7s
pod/cpc-bridge-proxy-ebpf-7ncbq 1/1 Running 0 5m26s
pod/cpc-bridge-proxy-ebpf-qth8w 1/1 Running 0 5m26s
pod/csi-do-node-gxlmb 2/2 Running 0 4m24s
pod/csi-do-node-swqfv 2/2 Running 0 4m24s
pod/do-node-agent-7bgsh 1/1 Running 0 4m11s
pod/do-node-agent-hwt6l 1/1 Running 0 4m11s
pod/headlamp-7dfd97b98b-wmn66 1/1 Running 0 48s
pod/hubble-relay-67597fb8-kmlw5 1/1 Running 1 (5m22s ago) 13m
pod/hubble-ui-79957d9f7b-4n9kj 2/2 Running 0 5m45s
pod/konnectivity-agent-7ml7p 1/1 Running 0 5m32s
pod/konnectivity-agent-tnf8j 1/1 Running 0 5m32s
pod/kube-proxy-ebpf-4gt2z 1/1 Running 0 11m
pod/kube-proxy-ebpf-ztjql 1/1 Running 0 11m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/headlamp ClusterIP 10.108.38.247 <none> 80/TCP 48s
service/hubble-peer ClusterIP 10.108.54.62 <none> 443/TCP 13m
service/hubble-relay ClusterIP 10.108.41.20 <none> 80/TCP 13m
service/hubble-ui ClusterIP 10.108.52.164 <none> 80/TCP 13m
service/kube-dns ClusterIP 10.108.32.10 <none> 53/UDP,53/TCP,9153/TCP 5m7s

en l’exposant localement et récupérant la clé nécessaire à l’accès à son interface web :

(base) root@k8sgpt:~# nohup kubectl port-forward -n kube-system service/headlamp 8080:80 &
(base) root@k8sgpt:~# cat nohup.out 
Forwarding from 127.0.0.1:8080 -> 4466
Forwarding from [::1]:8080 -> 4466

(base) root@k8sgpt:~# kubectl -n kube-system create serviceaccount headlamp-admin
serviceaccount/headlamp-admin created

(base) root@k8sgpt:~# kubectl create clusterrolebinding headlamp-admin --serviceaccount=kube-system:headlamp-admin --clusterrole=cluster-admin
clusterrolebinding.rbac.authorization.k8s.io/headlamp-admin created

(base) root@k8sgpt:~# kubectl create token headlamp-admin -n kube-system
eyJhbGciOiJSUzI1NiIsImtpZCI6Ikd0UHltNHV6c1liSkY0VkhNRWFZMXJKYklqY1R6ckZrMDZQS281dEg3dUUifQ.eyJhdWQiOlsic3lzdGVtOmtvbm5lY3Rpdml0eS1zZXJ2ZXIiXSwiZXhwIjoxNzM1MzkyNDEwLCJpYXQiOjE3MzUzODg4MTAsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiNTVkNjI1ZTItNjA2Yi00MTNhLTk2OTgtODFmYjdjZDU4MWY4Iiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJoZWFkbGFtcC1hZG1pbiIsInVpZCI6IjdmOGQwMTU0LWRiYWQtNGU2MS04NTUzLWU1NWI3ZWU0ZjhlOSJ9fSwibmJmIjoxNzM1Mzg4ODEwLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06aGVhZGxhbXAtYWRtaW4ifQ.jbrtuXS7uMP6HfwR3CbIbnpRTq4CDaacq0okwm_4tvmJNNcExi9-Dti3cGj1J3tteszpxVzurWPhrWgFlL4UkEacY9fD1TRH4GAZDCFldJ_jvyeaclzGeymrjEGAZ9TbBdoyuXtLeIVhApdICF1KNM-s8mfr1oOREDwlR9HzzrhoECozYxVS9uM1WIEZpum4FwMEl6cKPqOyNx1Rn5MtKPcc87JyK0FxuXzg9WC-cPSNOxu_rUFrZYHyrVapCDpl_XLymD3pFUUuB8XPVidVXcVOthH1Djwm8TRE6aAD4XlkHTcyTYchvN_CpOI2JQ6DVY60unSU8nq2pxfqLC6G2Q

Avant de lancer une analyse via K8sGPT, introduction d’un problème dans le cluster Kubernetes pour simuler une situation réelle. Vous pouvez utiliser des exemples de déploiements défectueux disponibles sur des répositories comme celui de Robusta :

GitHub - robusta-dev/kubernetes-demos: YAMLs for creating Kubernetes errors and other scenarios

Exemple avec ce déploiement d’un Pod cassé via :

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-processing-worker
spec:
  replicas: 1
  selector:
    matchLabels:
      app: payment-processing-worker
  template:
    metadata:
      labels:
        app: payment-processing-worker
    spec:
      containers:
      - name: payment-processing-container
        image: bash
        command: ["/bin/sh"]
        args: ["-c", "if [[-z \"${DEPLOY_ENV}\"]]; then echo Environment variable DEPLOY_ENV is undefined ; else while true; do echo hello; sleep 10;done; fi"]

base) root@k8sgpt:~# kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/crashpod/broken.yaml
deployment.apps/payment-processing-worker created

(base) root@k8sgpt:~# kubectl get po
NAME READY STATUS RESTARTS AGE
payment-processing-worker-747ccfb9db-dzjqx 0/1 CrashLoopBackOff 1 (11s ago) 17s

(base) root@k8sgpt:~# kubectl logs po/payment-processing-worker-747ccfb9db-dzjqx
Environment variable DEPLOY_ENV is undefined

Il s’est en effet crashé …

Lancement de l’analyse avec une sortie en JSON :

(base) root@k8sgpt:~# k8sgpt analyze -o json --explain --filter=Pod --backend localai | jq .

{
  "provider": "localai",
  "errors": null,
  "status": "ProblemDetected",
  "problems": 1,
  "results": [
    {
      "kind": "Pod",
      "name": "default/payment-processing-worker-747ccfb9db-dzjqx",
      "error": [
        {
          "Text": "the last termination reason is Completed container=payment-processing-container pod=payment-processing-worker-747ccfb9db-dzjqx",
          "KubernetesDoc": "",
          "Sensitive": []
        }
      ],
      "details": "Error: The pod \"payment-processing-worker-747ccfb9db-dzjqx\" has completed its execution with a \"Completed\" termination reason, indicating the container \"payment-processing-container\" has finished successfully.\n\nSolution: Verify the logs for the container to ensure data integrity, then check related services for expected outcomes; if successful, mark the pod as ready in the cluster.",
      "parentObject": "Deployment/payment-processing-worker"
    }
  ]
}

ou en mode texte :

(base) root@k8sgpt:~# k8sgpt analyze --explain --backend localai --with-doc
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| (1/1, 7960 it/s)        
AI Provider: localai

0: Pod default/payment-processing-worker-747ccfb9db-dzjqx(Deployment/payment-processing-worker)
- Error: the last termination reason is Completed container=payment-processing-container pod=payment-processing-worker-747ccfb9db-dzjqx
Error: The pod "payment-processing-worker-747ccfb9db-dzjqx" has completed its execution with a "Completed" termination reason, indicating the container "payment-processing-container" has finished successfully.

Solution: Verify the logs for the container to ensure data integrity, then check related services for expected outcomes; if successful, mark the pod as ready in the cluster.

Comme le montre cette sortie, K8sGPT a identifié et signalé le pod problématique et a également fourni des conseils sur les mesures potentielles à prendre pour comprendre et résoudre le problème.

Autre exemple avec ce Pod Nginx problématique :

apiVersion: v1
kind: Pod
metadata:
  name: inventory-management-api
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
    command:
      - wge
      - "-O"
      - "/work-dir/index.html"
      - https://home.robusta.dev

(base) root@k8sgpt:~# kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/refs/heads/main/crashloop_backoff/create_crashloop_backoff.yaml
pod/inventory-management-api created

(base) root@k8sgpt:~# kubectl get po
NAME READY STATUS RESTARTS AGE
inventory-management-api 0/1 ContainerCreating 0 5s

(base) root@k8sgpt:~# kubectl get po
NAME READY STATUS RESTARTS AGE
inventory-management-api 0/1 RunContainerError 1 (1s ago) 10s

Et une nouvelle analyse met le doigt sur le Pod problématique …

(base) root@k8sgpt:~# k8sgpt analyze --explain --backend localai --with-doc
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| (1/1, 2 it/min)         
AI Provider: localai

0: Pod default/inventory-management-api()
- Error: the last termination reason is StartError container=nginx pod=inventory-management-api
Error: The Kubernetes error indicates that there was a StartError issue with the nginx container for the pod named inventory-management-api.

Solution: 
1. Check the nginx configuration file for syntax errors.
2. Ensure all required resources and permissions are correctly set.
3. Verify network accessibility within the pod.
4. Confirm proper image pull secrets if using Docker images.
5. Review any recent changes to the deployment or service configurations.

Dans cet autre exemple il peut ne pas détecter de problématique avec cette simulation de faux positif via busybox :

apiVersion: batch/v1
kind: Job
metadata:
  name: java-api-checker
spec:
  template:
    spec:
      containers:
      - name: java-beans
        image: busybox
        command: ["/bin/sh", "-c"]
        args: ["echo 'Java Network Exception: \nAll host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256 \nAll host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256 \nAll host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256 \nAll host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256'; sleep 60; exit 1"]
      restartPolicy: Never
  backoffLimit: 1

(base) root@k8sgpt:~# kubectl delete -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/refs/heads/main/crashloop_backoff/create_crashloop_backoff.yaml
pod "inventory-management-api" deleted
(base) root@k8sgpt:~# kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/refs/heads/main/job_failure/job_crash.yaml
job.batch/java-api-checker created

(base) root@k8sgpt:~# kubectl get po
NAME READY STATUS RESTARTS AGE
java-api-checker-5s6dc 1/1 Running 0 7s
(base) root@k8sgpt:~# kubectl logs po/java-api-checker-5s6dc
Java Network Exception: 
All host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256 
All host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256 
All host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256 
All host(s) tried for db query failed (tried: prod-db:3333) - no available connection and the queue has reached its max size 256

(base) root@k8sgpt:~# k8sgpt analyze --explain --backend localai --with-doc
AI Provider: localai

No problems detected

Pour une intégration plus complète, vous pouvez installer l’opérateur K8sGPT dans votre cluster Kubernetes. Cet opérateur surveille en continu les problèmes dans le cluster et génère des insights que vous pouvez consulter en interrogeant la ressource personnalisée (CR) de l'opérateur.

GitHub - k8sgpt-ai/k8sgpt-operator: Automatic SRE Superpowers within your Kubernetes cluster

$ helm repo add k8sgpt https://charts.k8sgpt.ai/
$ helm repo update
$ helm install release k8sgpt/k8sgpt-operator -n k8sgpt-operator-system --create-namespace

# Installer l'opérateur k8sGPT
$ kubectl apply -n k8sgpt-operator-system -f - << EOF
apiVersion: core.k8sgpt.ai/v1alpha1
kind: K8sGPT
metadata:
  name: k8sgpt-ollama
spec:
  ai:
    enabled: true
    model: falcon3
    backend: localai
    baseUrl: http://localhost:11434/v1
  noCache: false
  filters: ["Pod"]
  repository: ghcr.io/k8sgpt-ai/k8sgpt
  version: v0.3.48
EOF

Debugging your Rancher Kubernetes Cluster the GenAI Way w...

Il est également possible d’analyser plusieurs clusters Kubernetes en spécifiant le chemin vers le fichier Kubeconfig concerné :

$ k8sgpt analyze --explain --backend localai --with-doc --kubeconfig <chemin vers le fichier Kubeconfig>

L’opérateur recherchera les problèmes dans le cluster et générera des résultats d’analyse. En fonction de la puissance de votre machine (pour accélerer les temps de réponse d’Ollama, des ressources GPU sont nécessaires), il faut un certain temps à l’opérateur pour appeler le LLM et générer les informations …

Pour conclure, K8sGPT en combinaison avec Ollama offre une solution puissante pour déboguer et gérer les clusters Kubernetes de manière efficace. Cette intégration utilise l'intelligence artificielle pour fournir des insights clairs et des recommandations pour résoudre les problèmes, simplifiant ainsi la vie des opérateurs de cluster. En suivant ces étapes, vous pouvez mettre en place une solution de diagnostic automatisée et basée sur l'IA pour votre environnement Kubernetes …

À suivre !

OpenStack sur Kubernetes en toute simplicité avec Canonical MicroStack et Pulumi …

Karim — Tue, 24 Dec 2024 12:48:32 +0000

MicroStack est une solution développée par Canonical, utilisant Snap, Juju et Kubernetes pour déployer et gérer OpenStack. Cette solution simplifie considérablement la complexité traditionnellement associée à la gestion des clouds OpenStack. Elle a évolué comme je le relatais auparavant dans différents articles en y insérant Canonical Kubernetes :

Avantages de MicroStack

OpenStack on Kubernetes | Ubuntu | Canonical

Déploiement Simplifié : MicroStack offre un processus d’installation rationalisé, capable de démarrer un déploiement cloud en moins de 6 commandes, avec un temps de déploiement moyen de 40 minutes. Cela le rend particulièrement adapté aux organisations qui cherchent à établir ou à étendre rapidement un environnement Cloud sans disposer d’une expertise technique approfondie.
Flexibilité et Customisation : MicroStack permet une grande flexibilité, incluant l’intégration d’un large éventail de plug-ins et d’extensions, ce qui permet aux entreprises de construire un environnement Cloud qui s’aligne précisément sur leurs objectifs opérationnels.

Mise en oeuvre concrète de Microstack sur un noeud unique selon ces préconisations techniques :

Enterprise requirements | Canonical

Je pars donc d’une instance dédiée sur DigitalOcean autorisant la virtualisation imbriquée :

Pour installer MicroStack sur un nœud unique, vous pouvez suivre ces étapes. Pour commencer, création d’un utilisateur non root avec sudo activé :

root@microstack:~# useradd -s /bin/bash -d /home/ubuntu -m ubuntu
root@microstack:~# echo "ubuntu ALL=(ALL) NOPASSWD: ALL" | tee /etc/sudoers.d/ubuntu
ubuntu ALL=(ALL) NOPASSWD: ALL
root@microstack:~# cp -r .ssh/ /home/ubuntu/
root@microstack:~# chown -R ubuntu:ubuntu /home/ubuntu/.ssh/

Puis installation de snapd :

ubuntu@microstack:~$ sudo apt install snapd
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  zenity | kdialog
The following packages will be upgraded:
  snapd
1 upgraded, 0 newly installed, 0 to remove and 195 not upgraded.
Need to get 30.0 MB of archives.
After this operation, 5513 kB of additional disk space will be used.
Get:1 http://mirrors.digitalocean.com/ubuntu noble-updates/main amd64 snapd amd64 2.66.1+24.04 [30.0 MB]
Fetched 30.0 MB in 0s (93.2 MB/s)
(Reading database ... 71895 files and directories currently installed.)
Preparing to unpack .../snapd_2.66.1+24.04_amd64.deb ...
Unpacking snapd (2.66.1+24.04) over (2.63+24.04) ...
Setting up snapd (2.66.1+24.04) ...
Installing new version of config file /etc/apparmor.d/usr.lib.snapd.snap-confine.real ...
snapd.failure.service is a disabled or a static unit not running, not starting it.
snapd.snap-repair.service is a disabled or a static unit not running, not starting it.
Processing triggers for dbus (1.14.10-4ubuntu4) ...
Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...                                                                                                                                                                         
Scanning linux images...                                                                                                                                                                      

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
ubuntu@microstack:~$ sudo systemctl enable --now snapd

Installation de MicroStack via la dernière version avec snapd :

Install Canonical MicroStack on Linux | Snap Store

ubuntu@microstack:~$ sudo snap install openstack --channel 2024.1/beta
2024-12-24T08:58:22Z INFO Waiting for automatic snapd restart...
openstack (2024.1/beta) 2024.1 from Canonical✓ installed
ubuntu@microstack:~$ sudo snap list
Name Version Rev Tracking Publisher Notes
core24 20240920 609 latest/stable canonical✓ base
openstack 2024.1 637 2024.1/beta canonical✓ -
snapd 2.66.1 23258 latest/stable canonical✓ snapd

MicroStack utilise Sunbeam pour générer un script qui s’assure que la machine dispose de toutes les dépendances nécessaires et qu’elle est configurée correctement pour être utilisée dans OpenStack. Lancement de ce dernier de manière directe :

ubuntu@microstack:~$ sunbeam prepare-node-script | bash -x && newgrp snap_daemon
++ lsb_release -sc
+ '[' noble '!=' noble ']'
++ whoami
+ USER=ubuntu
++ id -u
+ '[' 1000 -eq 0 -o ubuntu = root ']'
+ SUDO_ASKPASS=/bin/false
+ sudo -A whoami
+ sudo grep -r ubuntu /etc/sudoers /etc/sudoers.d
+ grep NOPASSWD:ALL
+ echo 'ubuntu ALL=(ALL) NOPASSWD:ALL'
+ sudo install -m 440 /tmp/90-ubuntu-sudo-access /etc/sudoers.d/90-ubuntu-sudo-access
+ rm -f /tmp/90-ubuntu-sudo-access
+ dpkg -s openssh-server
+ dpkg -s curl
+ sudo usermod --append --groups snap_daemon ubuntu
+ '[' -f /home/ubuntu/.ssh/id_rsa ']'
+ ssh-keygen -b 4096 -f /home/ubuntu/.ssh/id_rsa -t rsa -N ''
Generating public/private rsa key pair.
Your identification has been saved in /home/ubuntu/.ssh/id_rsa
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:NTnupee3yat23zAuoYy5U6VoXqK+JaU5R3L36nwrauM ubuntu@microstack
The key's randomart image is:
+---[RSA 4096]----+
| |
| . |
| = |
| o o. |
| . S.oo. |
| B+o++. |
| =+oBo.o.o |
| .=B.+++oo+.|
| .o+E*+++**+o|
+----[SHA256]-----+
+ cat /home/ubuntu/.ssh/id_rsa.pub
++ hostname --all-ip-addresses
+ ssh-keyscan -H 134.209.225.128 10.19.0.5 10.114.0.2
# 134.209.225.128:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 134.209.225.128:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 134.209.225.128:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 134.209.225.128:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 134.209.225.128:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.19.0.5:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.19.0.5:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.19.0.5:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.19.0.5:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.19.0.5:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.114.0.2:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.114.0.2:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.114.0.2:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
# 10.114.0.2:22 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.4
10.114.0.2: Connection closed by remote host
+ grep -E 'HTTPS?_PROXY' /etc/environment
+ curl -s -m 10 -x '' api.charmhub.io
+ sudo snap connect openstack:ssh-keys
+ sudo snap install --channel 3.6/stable juju
juju (3.6/stable) 3.6.1 from Canonical✓ installed
+ mkdir -p /home/ubuntu/.local/share
+ mkdir -p /home/ubuntu/.config/openstack
++ snap list openstack --unicode=never --color=never
++ grep openstack
+ snap_output='openstack 2024.1 637 2024.1/beta canonical** -'
++ awk -v col=4 '{print $col}'
+ track=2024.1/beta
+ [[2024.1/beta =~ edge]]
+ [[2024.1/beta == \-]]
+ [[2024.1/beta =~ beta]]
+ risk=beta
+ [[beta != \s\t\a\b\l\e]]
+ sudo snap set openstack deployment.risk=beta
+ echo 'Snap has been automatically configured to deploy from' 'beta channel.'
Snap has been automatically configured to deploy from beta channel.
+ echo 'Override by passing a custom manifest with -m/--manifest.'
Override by passing a custom manifest with -m/--manifest.

Possibilité à cette étape de déployer le cloud OpenStack à l’aide de cette commande en acceptant les valeurs par défaut (cela prend environ 30 minutes en fonction de la vitesse de connexion) :

ubuntu@microstack:~$ sunbeam cluster bootstrap --accept-defaults
Node has been bootstrapped with roles: compute, control

ubuntu@microstack:~$ sunbeam cluster list
                           controller                           
┏━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┓
┃ Node ┃ Cluster ┃ Machine ┃ Compute ┃ Control ┃ Storage ┃
┡━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━┩
│ microstack │ ONLINE │ running │ active │ active │ │
└────────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

ubuntu@microstack:~$ sudo systemctl status snap.openstack.clusterd.service
● snap.openstack.clusterd.service - Service for snap application openstack.clusterd
     Loaded: loaded (/etc/systemd/system/snap.openstack.clusterd.service; enabled; preset: enabled)
     Active: active (running) since Tue 2024-12-24 08:58:36 UTC; 1h 10min ago
   Main PID: 4497 (sunbeamd)
      Tasks: 18 (limit: 77123)
     Memory: 34.6M (peak: 39.2M)
        CPU: 11.947s
     CGroup: /system.slice/snap.openstack.clusterd.service
             └─4497 sunbeamd --state-dir /var/snap/openstack/common/state --socket-group snap_daemon --verbose

MicroStack est alors déployé et on peut lancer l’environnement de démo qui crée un tenant préconfiguré (avec les accès au dashboard) :

ubuntu@microstack:~$ sunbeam configure --accept-defaults --openrc demo-openrc
⠋ Generating openrc for cloud admin usage ... Writing openrc to demo-openrc ... done
The cloud has been configured for sample usage.
You can start using the OpenStack client or access the OpenStack dashboard at http://172.16.1.204:80/openstack-horizon

Je dispose des identifiants pour ce dernier :

ubuntu@microstack:~$ cat demo-openrc 
# openrc for demo
export OS_AUTH_URL=http://172.16.1.204/openstack-keystone/v3
export OS_USERNAME=demo
export OS_PASSWORD=C0jg0mAgdvD5
export OS_USER_DOMAIN_NAME=users
export OS_PROJECT_DOMAIN_NAME=users
export OS_PROJECT_NAME=demo
export OS_AUTH_VERSION=3
export OS_IDENTITY_API_VERSION=3

Ou pour le compte admin :

ubuntu@microstack:~$ sunbeam openrc > admin-openrc
ubuntu@microstack:~$ cat admin-openrc 
# openrc for access to OpenStack
export OS_USERNAME=admin
export OS_PASSWORD=f7M1ey2dqpHo
export OS_AUTH_URL=http://172.16.1.204/openstack-keystone/v3
export OS_USER_DOMAIN_NAME=admin_domain
export OS_PROJECT_DOMAIN_NAME=admin_domain
export OS_PROJECT_NAME=admin
export OS_AUTH_VERSION=3
export OS_IDENTITY_API_VERSION=3

Lancement rapide d’une instance de test :

ubuntu@microstack:~$ sunbeam launch ubuntu --name instance1 
Launching an OpenStack instance ... 
Access the instance by running the following command:
`ssh -i /home/ubuntu/snap/openstack/637/sunbeam ubuntu@172.16.2.31`

ubuntu@microstack:~$ source demo-openrc 
ubuntu@microstack:~$ openstack server list
+--------------------------------------+-----------+--------+-----------------------------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-----------+--------+-----------------------------------------+--------+---------+
| efe46971-56f4-4da4-9c6e-eebee2795b72 | instance1 | ACTIVE | demo-network=172.16.2.31, 192.168.0.166 | ubuntu | m1.tiny |
+--------------------------------------+-----------+--------+-----------------------------------------+--------+---------+

J’accède ici au dashboard via une redirection ssh :

$ ssh -L 0.0.0.0:8888:172.16.1.204:80 ubuntu@134.209.225.128

Dans MicroStack, Sunbeam permet l’activation de plusieurs extensions intéressantes :

ubuntu@microstack:~$ sunbeam enable --help
Usage: sunbeam enable [OPTIONS] COMMAND [ARGS]...

  Enable features.

Options:
  -m, --manifest FILE Manifest file.
  -h, --help Show this message and exit.

Commands:
  caas Enable Container as a Service feature.
  dns Enable dns service.
  images-sync Enable images-sync service.
  ldap Enable ldap service.
  loadbalancer Enable Loadbalancer service.
  observability Enable Observability service.
  orchestration Enable Orchestration service.
  pro Enable Ubuntu Pro across deployment.
  resource-optimization Enable Resource Optimization service (watcher).
  secrets Enable OpenStack Secrets service.
  telemetry Enable OpenStack Telemetry applications.
  tls Enable tls group.
  validation Enable OpenStack Integration Test Suite (tempest).
  vault Enable Vault.

Activation de plusieures d’entre elles …

ubuntu@microstack:~$ sunbeam enable orchestration
OpenStack orchestration application enabled.
ubuntu@microstack:~$ sunbeam enable telemetry
OpenStack telemetry application enabled.
ubuntu@microstack:~$ sunbeam enable observability embedded
Observability enabled.

Dont celle relative à la stack d’observabilité mettant en oeuvre Grafana dont on obtient ici le lien et les identifiants :

Observability | Canonical


(base) ubuntu@microstack:~$ sunbeam observability dashboard-url
http://172.16.1.205/observability-grafana

(base) ubuntu@microstack:~$ juju run --model observability grafana/leader get-admin-password
Running operation 5 with 1 task
  - task 6 on unit-grafana-0

Waiting for task 6...
admin-password: 0EAJrXNIt3jd
url: http://172.16.1.205/observability-grafana

Sunbeam utilise une série de manifestes dont on retrouve la liste à la suite de tous ces déploiements :

ubuntu@microstack:~$ sunbeam manifest list
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ ID ┃ Applied Date ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ ecd1a516be9a244376923e9f7b8217ce │ 2024-12-24 09:00:10 │
└──────────────────────────────────┴─────────────────────┘

ubuntu@microstack:~$ sunbeam manifest show ecd1a516be9a244376923e9f7b8217ce
core:
  software:
    charms:
      cinder-ceph-k8s:
        channel: 2024.1/beta
      cinder-k8s:
        channel: 2024.1/beta
      glance-k8s:
        channel: 2024.1/beta
      horizon-k8s:
        channel: 2024.1/beta
      keystone-k8s:
        channel: 2024.1/beta
      microceph:
        channel: squid/beta
        config:
          snap-channel: squid/beta
      neutron-k8s:
        channel: 2024.1/beta
      nova-k8s:
        channel: 2024.1/beta
      openstack-hypervisor:
        channel: 2024.1/beta
        config:
          snap-channel: 2024.1/beta
      ovn-central-k8s:
        channel: 24.03/beta
      ovn-relay-k8s:
        channel: 24.03/beta
      placement-k8s:
        channel: 2024.1/beta
      sunbeam-clusterd:
        channel: 2024.1/beta
        config:
          snap-channel: 2024.1/beta
      sunbeam-machine:
        channel: 2024.1/beta
features:
  caas:
    software:
      charms:
        magnum-k8s:
          channel: 2024.1/beta
  dns:
    software:
      charms:
        designate-bind-k8s:
          channel: 9/beta
        designate-k8s:
          channel: 2024.1/beta
  images-sync:
    software:
      charms:
        openstack-images-sync-k8s:
          channel: 2024.1/beta
  instance-recovery:
    software:
      charms:
        consul-client:
          channel: 1.19/beta
        consul-k8s:
          channel: 1.19/beta
        masakari-k8s:
          channel: 2024.1/beta
  ldap:
    software:
      charms:
        keystone-ldap-k8s:
          channel: 2024.1/beta
  loadbalancer:
    software:
      charms:
        octavia-k8s:
          channel: 2024.1/beta
  orchestration:
    software:
      charms:
        heat-k8s:
          channel: 2024.1/beta
  resource-optimization:
    software:
      charms:
        watcher-k8s:
          channel: 2024.1/beta
  secrets:
    software:
      charms:
        barbican-k8s:
          channel: 2024.1/beta
  telemetry:
    software:
      charms:
        aodh-k8s:
          channel: 2024.1/beta
        ceilometer-k8s:
          channel: 2024.1/beta
        gnocchi-k8s:
          channel: 2024.1/beta
        openstack-exporter-k8s:
          channel: 2024.1/beta
  validation:
    software:
      charms:
        tempest-k8s:
          channel: 2024.1/beta

Dont on peut aussi vérifier la présence via Juju :


ubuntu@microstack:~$ juju status -m admin/controller
Model Controller Cloud/Region Version SLA Timestamp
controller sunbeam-controller one-deer/default 3.6.1 unsupported 10:04:08Z

SAAS Status Store URL
ceilometer waiting local microstack/openstack.ceilometer
cert-distributor active local microstack/openstack.cert-distributor
certificate-authority active local microstack/openstack.certificate-authority
cinder-ceph blocked local microstack/openstack.cinder-ceph
grafana-dashboards active local microstack/observability.grafana-dashboards
keystone-credentials active local microstack/openstack.keystone-credentials
keystone-endpoints active local microstack/openstack.keystone-endpoints
loki-logging active local microstack/observability.loki-logging
nova active local microstack/openstack.nova
ovn-relay active local microstack/openstack.ovn-relay
prometheus-receive-remote-write active local microstack/observability.prometheus-receive-remote-write
rabbitmq active local microstack/openstack.rabbitmq

App Version Status Scale Charm Channel Rev Exposed Message
controller active 1 juju-controller 3.6/stable 116 no       
grafana-agent active 1 grafana-agent latest/stable 260 no tracing: off
k8s 1.31.3 active 1 k8s 1.31/candidate 141 no Ready
microceph unknown 0 microceph squid/beta 84 no       
openstack-hypervisor waiting 1 openstack-hypervisor 2024.1/beta 221 no (ceph-access) integration incomplete
sunbeam-machine active 1 sunbeam-machine 2024.1/beta 49 no       

Unit Workload Agent Machine Public address Ports Message
controller/0* active idle 0 134.209.225.128            
k8s/0* active idle 0 134.209.225.128 6443/tcp Ready
openstack-hypervisor/0* waiting idle 0 134.209.225.128 (ceph-access) integration incomplete
  grafana-agent/0* active idle 134.209.225.128 tracing: off
sunbeam-machine/0* active idle 0 134.209.225.128            

Machine State Address Inst id Base AZ Message
0 started 134.209.225.128 manual: ubuntu@24.04 Manually provisioned machine

Offer Application Charm Rev Connected Endpoint Interface Role
microceph microceph microceph 84 0/0 ceph ceph-client provider

ubuntu@microstack:~$ juju status -m openstack
Model Controller Cloud/Region Version SLA Timestamp
openstack sunbeam-controller one-deer-k8s/localhost 3.6.1 unsupported 10:04:48Z

SAAS Status Store URL
grafana-dashboards active local microstack/observability.grafana-dashboards
loki-logging active local microstack/observability.loki-logging
prometheus-receive-remote-write active local microstack/observability.prometheus-receive-remote-write

App Version Status Scale Charm Channel Rev Address Exposed Message
aodh active 1 aodh-k8s 2024.1/beta 62 10.152.183.238 no       
aodh-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.134 no       
aodh-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.200 no       
ceilometer waiting 1 ceilometer-k8s 2024.1/beta 62 10.152.183.88 no (workload) Not all relations are ready
certificate-authority active 1 self-signed-certificates latest/beta 228 10.152.183.220 no       
cinder active 1 cinder-k8s 2024.1/beta 99 10.152.183.160 no       
cinder-ceph blocked 1 cinder-ceph-k8s 2024.1/beta 97 10.152.183.202 no (ceph) integration missing
cinder-ceph-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.244 no       
cinder-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.77 no       
cinder-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.92 no       
glance active 1 glance-k8s 2024.1/beta 120 10.152.183.187 no       
glance-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.163 no       
glance-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.116 no       
gnocchi blocked 1 gnocchi-k8s 2024.1/beta 61 10.152.183.81 no (ceph) integration missing
gnocchi-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.35 no       
gnocchi-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.72 no       
grafana-agent 0.40.4 active 1 grafana-agent-k8s latest/stable 80 10.152.183.169 no       
heat active 1 heat-k8s 2024.1/beta 79 10.152.183.151 no       
heat-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.222 no       
heat-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.249 no       
horizon active 1 horizon-k8s 2024.1/beta 111 10.152.183.234 no http://172.16.1.204/openstack-horizon
horizon-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.131 no       
horizon-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.140 no       
keystone active 1 keystone-k8s 2024.1/beta 213 10.152.183.63 no       
keystone-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.48 no       
keystone-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.108 no       
neutron active 1 neutron-k8s 2024.1/beta 119 10.152.183.212 no       
neutron-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.75 no       
neutron-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.197 no       
nova active 1 nova-k8s 2024.1/beta 109 10.152.183.104 no       
nova-api-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.189 no       
nova-cell-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.178 no       
nova-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.24 no       
nova-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.152 no       
openstack-exporter active 1 openstack-exporter-k8s 2024.1/beta 72 10.152.183.100 no       
ovn-central active 1 ovn-central-k8s 24.03/beta 110 10.152.183.194 no       
ovn-relay active 1 ovn-relay-k8s 24.03/beta 97 172.16.1.201 no       
placement active 1 placement-k8s 2024.1/beta 92 10.152.183.199 no       
placement-mysql 8.0.37-0ubuntu0.22.04.3 active 1 mysql-k8s 8.0/stable 180 10.152.183.83 no       
placement-mysql-router 8.0.37-0ubuntu0.22.04.3 active 1 mysql-router-k8s 8.0/stable 155 10.152.183.248 no       
rabbitmq 3.12.1 active 1 rabbitmq-k8s 3.12/stable 34 172.16.1.202 no       
traefik 2.11.0 active 1 traefik-k8s latest/beta 223 10.152.183.125 no Serving at 172.16.1.203
traefik-public 2.11.0 active 1 traefik-k8s latest/beta 223 10.152.183.54 no Serving at 172.16.1.204
vault blocked 1 vault-k8s 1.16/stable 280 10.152.183.78 no Please initialize Vault or integrate with an auto-unseal provider

Unit Workload Agent Address Ports Message
aodh-mysql-router/0* active idle 10.1.0.57          
aodh-mysql/0* active idle 10.1.0.6 Primary
aodh/0* active idle 10.1.0.90          
ceilometer/0* waiting idle 10.1.0.149 (workload) Not all relations are ready
certificate-authority/0* active idle 10.1.0.5           
cinder-ceph-mysql-router/0* active idle 10.1.0.167         
cinder-ceph/0* blocked idle 10.1.0.108 (ceph) integration missing
cinder-mysql-router/0* active idle 10.1.0.253         
cinder-mysql/0* active idle 10.1.0.145 Primary
cinder/0* active idle 10.1.0.56          
glance-mysql-router/0* active idle 10.1.0.85          
glance-mysql/0* active idle 10.1.0.183 Primary
glance/0* active idle 10.1.0.251         
gnocchi-mysql-router/0* active idle 10.1.0.196         
gnocchi-mysql/0* active idle 10.1.0.213 Primary
gnocchi/0* blocked idle 10.1.0.55 (ceph) integration missing
grafana-agent/0* active idle 10.1.0.2           
heat-mysql-router/0* active idle 10.1.0.54          
heat-mysql/0* active idle 10.1.0.9 Primary
heat/0* active idle 10.1.0.138         
horizon-mysql-router/0* active idle 10.1.0.248         
horizon-mysql/0* active idle 10.1.0.185 Primary
horizon/0* active idle 10.1.0.35          
keystone-mysql-router/0* active idle 10.1.0.243         
keystone-mysql/0* active idle 10.1.0.104 Primary
keystone/0* active idle 10.1.0.223         
neutron-mysql-router/0* active idle 10.1.0.135         
neutron-mysql/0* active idle 10.1.0.79 Primary
neutron/0* active idle 10.1.0.23          
nova-api-mysql-router/0* active idle 10.1.0.93          
nova-cell-mysql-router/0* active idle 10.1.0.165         
nova-mysql-router/0* active idle 10.1.0.143         
nova-mysql/0* active idle 10.1.0.226 Primary
nova/0* active idle 10.1.0.147         
openstack-exporter/0* active idle 10.1.0.100         
ovn-central/0* active idle 10.1.0.222         
ovn-relay/0* active idle 10.1.0.82          
placement-mysql-router/0* active idle 10.1.0.224         
placement-mysql/0* active idle 10.1.0.148 Primary
placement/0* active idle 10.1.0.78          
rabbitmq/0* active idle 10.1.0.238         
traefik-public/0* active idle 10.1.0.96 Serving at 172.16.1.204
traefik/0* active idle 10.1.0.151 Serving at 172.16.1.203
vault/0* blocked idle 10.1.0.178 Please initialize Vault or integrate with an auto-unseal provider

Offer Application Charm Rev Connected Endpoint Interface Role
ceilometer ceilometer ceilometer-k8s 62 1/1 ceilometer-service ceilometer provider
cert-distributor keystone keystone-k8s 213 2/2 send-ca-cert certificate_transfer provider
certificate-authority certificate-authority self-signed-certificates 228 1/1 certificates tls-certificates provider
cinder-ceph cinder-ceph cinder-ceph-k8s 97 1/1 ceph-access cinder-ceph-key provider
keystone-credentials keystone keystone-k8s 213 1/1 identity-credentials keystone-credentials provider
keystone-endpoints keystone keystone-k8s 213 1/1 identity-service keystone provider
nova nova nova-k8s 109 1/1 nova-service nova provider
ovn-relay ovn-relay ovn-relay-k8s 97 1/1 ovsdb-cms-relay ovsdb-cms provider
rabbitmq rabbitmq rabbitmq-k8s 34 1/1 amqp rabbitmq provider

Ou via Canonical Kubernetes (qui supporte OpenStack) :

ubuntu@microstack:~$ sudo k8s kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/coredns:udp-53/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
ubuntu@microstack:~$ sudo k8s kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/cilium-87pxh 1/1 Running 0 62m
kube-system pod/cilium-operator-6f7f8cf67-5vfsx 1/1 Running 0 63m
kube-system pod/ck-storage-rawfile-csi-controller-0 2/2 Running 0 63m
kube-system pod/ck-storage-rawfile-csi-node-5vbjd 4/4 Running 0 63m
kube-system pod/coredns-598bfdf87d-qt2j4 1/1 Running 0 63m
kube-system pod/metrics-server-7ff9f4d4c9-jqb9x 1/1 Running 0 63m
metallb-system pod/metallb-controller-7bb5f6c9b4-pbzdb 1/1 Running 0 63m
metallb-system pod/metallb-speaker-dxg5x 1/1 Running 0 62m
observability pod/alertmanager-0 2/2 Running 0 13m
observability pod/catalogue-0 2/2 Running 0 13m
observability pod/grafana-0 3/3 Running 0 13m
observability pod/loki-0 3/3 Running 0 12m
observability pod/modeloperator-88fc49d74-tjcnm 1/1 Running 0 14m
observability pod/prometheus-0 2/2 Running 0 13m
observability pod/traefik-0 2/2 Running 0 13m
openstack pod/aodh-0 6/6 Running 0 18m
openstack pod/aodh-mysql-0 2/2 Running 0 22m
openstack pod/aodh-mysql-router-0 2/2 Running 0 19m
openstack pod/ceilometer-0 3/3 Running 0 22m
openstack pod/certificate-authority-0 1/1 Running 0 61m
openstack pod/cinder-0 3/3 Running 0 56m
openstack pod/cinder-ceph-0 2/2 Running 0 56m
openstack pod/cinder-ceph-mysql-router-0 2/2 Running 0 56m
openstack pod/cinder-mysql-0 2/2 Running 0 60m
openstack pod/cinder-mysql-router-0 2/2 Running 0 56m
openstack pod/glance-0 2/2 Running 0 56m
openstack pod/glance-mysql-0 2/2 Running 0 61m
openstack pod/glance-mysql-router-0 2/2 Running 0 56m
openstack pod/gnocchi-0 3/3 Running 0 19m
openstack pod/gnocchi-mysql-0 2/2 Running 0 22m
openstack pod/gnocchi-mysql-router-0 2/2 Running 0 19m
openstack pod/grafana-agent-0 2/2 Running 0 10m
openstack pod/heat-0 4/4 Running 0 29m
openstack pod/heat-mysql-0 2/2 Running 0 30m
openstack pod/heat-mysql-router-0 2/2 Running 0 29m
openstack pod/horizon-0 2/2 Running 0 55m
openstack pod/horizon-mysql-0 2/2 Running 0 61m
openstack pod/horizon-mysql-router-0 2/2 Running 0 55m
openstack pod/keystone-0 2/2 Running 0 55m
openstack pod/keystone-mysql-0 2/2 Running 0 61m
openstack pod/keystone-mysql-router-0 2/2 Running 0 55m
openstack pod/modeloperator-56b4d68fb7-tznnv 1/1 Running 0 62m
openstack pod/neutron-0 2/2 Running 0 55m
openstack pod/neutron-mysql-0 2/2 Running 0 60m
openstack pod/neutron-mysql-router-0 2/2 Running 0 55m
openstack pod/nova-0 5/5 Running 0 56m
openstack pod/nova-api-mysql-router-0 2/2 Running 0 56m
openstack pod/nova-cell-mysql-router-0 2/2 Running 0 56m
openstack pod/nova-mysql-0 2/2 Running 0 60m
openstack pod/nova-mysql-router-0 2/2 Running 0 56m
openstack pod/openstack-exporter-0 2/2 Running 0 22m
openstack pod/ovn-central-0 4/4 Running 0 61m
openstack pod/ovn-relay-0 2/2 Running 0 61m
openstack pod/placement-0 2/2 Running 0 55m
openstack pod/placement-mysql-0 2/2 Running 0 61m
openstack pod/placement-mysql-router-0 2/2 Running 0 55m
openstack pod/rabbitmq-0 2/2 Running 0 61m
openstack pod/traefik-0 2/2 Running 0 61m
openstack pod/traefik-public-0 2/2 Running 0 60m
openstack pod/vault-0 2/2 Running 0 27m

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 63m
kube-system service/ck-storage-rawfile-csi-controller ClusterIP None <none> <none> 63m
kube-system service/ck-storage-rawfile-csi-node ClusterIP 10.152.183.237 <none> 9100/TCP 63m
kube-system service/coredns ClusterIP 10.152.183.37 <none> 53/UDP,53/TCP 63m
kube-system service/hubble-peer ClusterIP 10.152.183.23 <none> 443/TCP 63m
kube-system service/metrics-server ClusterIP 10.152.183.119 <none> 443/TCP 63m
metallb-system service/metallb-webhook-service ClusterIP 10.152.183.110 <none> 443/TCP 63m
observability service/alertmanager ClusterIP 10.152.183.36 <none> 9093/TCP,9094/TCP 13m
observability service/alertmanager-endpoints ClusterIP None <none> <none> 13m
observability service/catalogue ClusterIP 10.152.183.190 <none> 80/TCP 14m
observability service/catalogue-endpoints ClusterIP None <none> <none> 14m
observability service/grafana ClusterIP 10.152.183.170 <none> 3000/TCP 13m
observability service/grafana-endpoints ClusterIP None <none> <none> 13m
observability service/loki ClusterIP 10.152.183.201 <none> 3100/TCP 13m
observability service/loki-endpoints ClusterIP None <none> <none> 13m
observability service/modeloperator ClusterIP 10.152.183.59 <none> 17071/TCP 14m
observability service/prometheus ClusterIP 10.152.183.147 <none> 9090/TCP 13m
observability service/prometheus-endpoints ClusterIP None <none> <none> 13m
observability service/traefik ClusterIP 10.152.183.198 <none> 65535/TCP 13m
observability service/traefik-endpoints ClusterIP None <none> <none> 13m
observability service/traefik-lb LoadBalancer 10.152.183.60 172.16.1.205 80:30845/TCP,443:31176/TCP 13m
openstack service/aodh ClusterIP 10.152.183.238 <none> 8042/TCP 20m
openstack service/aodh-endpoints ClusterIP None <none> <none> 20m
openstack service/aodh-mysql ClusterIP 10.152.183.134 <none> 3306/TCP,33060/TCP 23m
openstack service/aodh-mysql-endpoints ClusterIP None <none> <none> 22m
openstack service/aodh-mysql-primary ClusterIP 10.152.183.176 <none> 3306/TCP 19m
openstack service/aodh-mysql-replicas ClusterIP 10.152.183.74 <none> 3306/TCP 19m
openstack service/aodh-mysql-router ClusterIP 10.152.183.200 <none> 6446/TCP,6447/TCP,65535/TCP 20m
openstack service/aodh-mysql-router-endpoints ClusterIP None <none> <none> 20m
openstack service/ceilometer ClusterIP 10.152.183.88 <none> 65535/TCP 23m
openstack service/ceilometer-endpoints ClusterIP None <none> <none> 22m
openstack service/certificate-authority ClusterIP 10.152.183.220 <none> 65535/TCP 61m
openstack service/certificate-authority-endpoints ClusterIP None <none> <none> 61m
openstack service/cinder ClusterIP 10.152.183.160 <none> 8776/TCP 58m
openstack service/cinder-ceph ClusterIP 10.152.183.202 <none> 65535/TCP 58m
openstack service/cinder-ceph-endpoints ClusterIP None <none> <none> 57m
openstack service/cinder-ceph-mysql-router ClusterIP 10.152.183.244 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/cinder-ceph-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/cinder-endpoints ClusterIP None <none> <none> 57m
openstack service/cinder-mysql ClusterIP 10.152.183.77 <none> 3306/TCP,33060/TCP 61m
openstack service/cinder-mysql-endpoints ClusterIP None <none> <none> 61m
openstack service/cinder-mysql-primary ClusterIP 10.152.183.113 <none> 3306/TCP 55m
openstack service/cinder-mysql-replicas ClusterIP 10.152.183.127 <none> 3306/TCP 55m
openstack service/cinder-mysql-router ClusterIP 10.152.183.92 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/cinder-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/glance ClusterIP 10.152.183.187 <none> 9292/TCP 58m
openstack service/glance-endpoints ClusterIP None <none> <none> 57m
openstack service/glance-mysql ClusterIP 10.152.183.163 <none> 3306/TCP,33060/TCP 61m
openstack service/glance-mysql-endpoints ClusterIP None <none> <none> 61m
openstack service/glance-mysql-primary ClusterIP 10.152.183.245 <none> 3306/TCP 55m
openstack service/glance-mysql-replicas ClusterIP 10.152.183.246 <none> 3306/TCP 55m
openstack service/glance-mysql-router ClusterIP 10.152.183.116 <none> 6446/TCP,6447/TCP,65535/TCP 58m
openstack service/glance-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/gnocchi ClusterIP 10.152.183.81 <none> 65535/TCP 20m
openstack service/gnocchi-endpoints ClusterIP None <none> <none> 20m
openstack service/gnocchi-mysql ClusterIP 10.152.183.35 <none> 3306/TCP,33060/TCP 23m
openstack service/gnocchi-mysql-endpoints ClusterIP None <none> <none> 22m
openstack service/gnocchi-mysql-primary ClusterIP 10.152.183.146 <none> 3306/TCP 19m
openstack service/gnocchi-mysql-replicas ClusterIP 10.152.183.153 <none> 3306/TCP 19m
openstack service/gnocchi-mysql-router ClusterIP 10.152.183.72 <none> 6446/TCP,6447/TCP,65535/TCP 20m
openstack service/gnocchi-mysql-router-endpoints ClusterIP None <none> <none> 20m
openstack service/grafana-agent ClusterIP 10.152.183.169 <none> 3500/TCP,3600/TCP 10m
openstack service/grafana-agent-endpoints ClusterIP None <none> <none> 10m
openstack service/heat ClusterIP 10.152.183.151 <none> 8004/TCP 30m
openstack service/heat-endpoints ClusterIP None <none> <none> 29m
openstack service/heat-mysql ClusterIP 10.152.183.222 <none> 3306/TCP,33060/TCP 30m
openstack service/heat-mysql-endpoints ClusterIP None <none> <none> 30m
openstack service/heat-mysql-primary ClusterIP 10.152.183.98 <none> 3306/TCP 29m
openstack service/heat-mysql-replicas ClusterIP 10.152.183.80 <none> 3306/TCP 29m
openstack service/heat-mysql-router ClusterIP 10.152.183.249 <none> 6446/TCP,6447/TCP,65535/TCP 30m
openstack service/heat-mysql-router-endpoints ClusterIP None <none> <none> 29m
openstack service/horizon ClusterIP 10.152.183.234 <none> 65535/TCP 57m
openstack service/horizon-endpoints ClusterIP None <none> <none> 57m
openstack service/horizon-mysql ClusterIP 10.152.183.131 <none> 3306/TCP,33060/TCP 61m
openstack service/horizon-mysql-endpoints ClusterIP None <none> <none> 61m
openstack service/horizon-mysql-primary ClusterIP 10.152.183.126 <none> 3306/TCP 55m
openstack service/horizon-mysql-replicas ClusterIP 10.152.183.145 <none> 3306/TCP 55m
openstack service/horizon-mysql-router ClusterIP 10.152.183.140 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/horizon-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/keystone ClusterIP 10.152.183.63 <none> 5000/TCP 57m
openstack service/keystone-endpoints ClusterIP None <none> <none> 57m
openstack service/keystone-mysql ClusterIP 10.152.183.48 <none> 3306/TCP,33060/TCP 61m
openstack service/keystone-mysql-endpoints ClusterIP None <none> <none> 61m
openstack service/keystone-mysql-primary ClusterIP 10.152.183.159 <none> 3306/TCP 55m
openstack service/keystone-mysql-replicas ClusterIP 10.152.183.114 <none> 3306/TCP 55m
openstack service/keystone-mysql-router ClusterIP 10.152.183.108 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/keystone-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/modeloperator ClusterIP 10.152.183.26 <none> 17071/TCP 62m
openstack service/neutron ClusterIP 10.152.183.212 <none> 9696/TCP 57m
openstack service/neutron-endpoints ClusterIP None <none> <none> 57m
openstack service/neutron-mysql ClusterIP 10.152.183.75 <none> 3306/TCP,33060/TCP 60m
openstack service/neutron-mysql-endpoints ClusterIP None <none> <none> 60m
openstack service/neutron-mysql-primary ClusterIP 10.152.183.55 <none> 3306/TCP 55m
openstack service/neutron-mysql-replicas ClusterIP 10.152.183.34 <none> 3306/TCP 55m
openstack service/neutron-mysql-router ClusterIP 10.152.183.197 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/neutron-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/nova ClusterIP 10.152.183.104 <none> 8774/TCP 58m
openstack service/nova-api-mysql-router ClusterIP 10.152.183.189 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/nova-api-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/nova-cell-mysql-router ClusterIP 10.152.183.178 <none> 6446/TCP,6447/TCP,65535/TCP 58m
openstack service/nova-cell-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/nova-endpoints ClusterIP None <none> <none> 57m
openstack service/nova-mysql ClusterIP 10.152.183.24 <none> 3306/TCP,33060/TCP 61m
openstack service/nova-mysql-endpoints ClusterIP None <none> <none> 61m
openstack service/nova-mysql-primary ClusterIP 10.152.183.130 <none> 3306/TCP 56m
openstack service/nova-mysql-replicas ClusterIP 10.152.183.186 <none> 3306/TCP 56m
openstack service/nova-mysql-router ClusterIP 10.152.183.152 <none> 6446/TCP,6447/TCP,65535/TCP 58m
openstack service/nova-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/openstack-exporter ClusterIP 10.152.183.100 <none> 9180/TCP 23m
openstack service/openstack-exporter-endpoints ClusterIP None <none> <none> 22m
openstack service/ovn-central ClusterIP 10.152.183.194 <none> 6641/TCP,6642/TCP 62m
openstack service/ovn-central-endpoints ClusterIP None <none> <none> 61m
openstack service/ovn-relay LoadBalancer 10.152.183.149 172.16.1.201 6642:32293/TCP 62m
openstack service/ovn-relay-endpoints ClusterIP None <none> <none> 62m
openstack service/placement ClusterIP 10.152.183.199 <none> 8778/TCP 57m
openstack service/placement-endpoints ClusterIP None <none> <none> 57m
openstack service/placement-mysql ClusterIP 10.152.183.83 <none> 3306/TCP,33060/TCP 61m
openstack service/placement-mysql-endpoints ClusterIP None <none> <none> 61m
openstack service/placement-mysql-primary ClusterIP 10.152.183.188 <none> 3306/TCP 55m
openstack service/placement-mysql-replicas ClusterIP 10.152.183.165 <none> 3306/TCP 55m
openstack service/placement-mysql-router ClusterIP 10.152.183.248 <none> 6446/TCP,6447/TCP,65535/TCP 57m
openstack service/placement-mysql-router-endpoints ClusterIP None <none> <none> 57m
openstack service/rabbitmq LoadBalancer 10.152.183.150 172.16.1.202 5672:31615/TCP,15672:31040/TCP 61m
openstack service/rabbitmq-endpoints ClusterIP None <none> <none> 61m
openstack service/traefik ClusterIP 10.152.183.125 <none> 65535/TCP 61m
openstack service/traefik-endpoints ClusterIP None <none> <none> 61m
openstack service/traefik-lb LoadBalancer 10.152.183.221 172.16.1.203 80:32485/TCP,443:31534/TCP 61m
openstack service/traefik-public ClusterIP 10.152.183.54 <none> 65535/TCP 61m
openstack service/traefik-public-endpoints ClusterIP None <none> <none> 61m
openstack service/traefik-public-lb LoadBalancer 10.152.183.115 172.16.1.204 80:30599/TCP,443:30927/TCP 60m
openstack service/vault ClusterIP 10.152.183.78 <none> 8200/TCP 27m
openstack service/vault-endpoints ClusterIP None <none> <none> 27m

Une fois que MicroStack a été déployé, vous avez la possibilité de gérer les charges de travail manuellement (c’est-à-dire via la CLI d’openstack) ou avec Juju :

Manage workloads with Juju | Canonical

Mais je vais utiliser Pulumi. Pulumi est une plateforme moderne d’infrastructure en tant que code (IaC) qui permet aux utilisateurs de gérer et de fournir une infrastructure en nuage à l’aide de langages de programmation (notamment TypeScript, JavaScript, Python, Go, .NET, Java et YAML).

Pulumi - Infrastructure as Code, Secrets Management, and AI

Pulumi adopte une approche déclarative pour définir l’infrastructure. Les utilisateurs spécifient l’état souhaité de leur infrastructure, et Pulumi gère la création, la mise à jour et la suppression des ressources pour atteindre cet état. Cette approche est plus intuitive que la programmation impérative, où chaque étape pour atteindre l’état souhaité doit être explicitement décrite.

L’interface de ligne de commande de Pulumi est le principal outil de gestion et de déploiement de l’infrastructure et je l’installe à cette étape :

(base) ubuntu@microstack:~$ curl -fsSL https://get.pulumi.com | sh
=== Installing Pulumi 3.144.1 ===
+ Downloading https://github.com/pulumi/pulumi/releases/download/v3.144.1/pulumi-v3.144.1-linux-x64.tar.gz...
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 80.1M 100 80.1M 0 0 103M 0 --:--:-- --:--:-- --:--:-- 433M
+ Extracting to /home/ubuntu/.pulumi/bin
+ Adding /home/ubuntu/.pulumi/bin to $PATH in /home/ubuntu/.bashrc

=== Pulumi is now installed! 🍹 ===
+ Please restart your shell or add /home/ubuntu/.pulumi/bin to your $PATH
+ Get started with Pulumi: https://www.pulumi.com/docs/quickstart
(base) ubuntu@microstack:~$ source .bashrc
(base) ubuntu@microstack:~$ pulumi
Pulumi - Modern Infrastructure as Code

To begin working with Pulumi, run the `pulumi new` command:

    $ pulumi new

This will prompt you to create a new project for your cloud and language of choice.

The most common commands from there are:

    - pulumi up : Deploy code and/or resource changes
    - pulumi stack : Manage instances of your project
    - pulumi config : Alter your stack's configuration or secrets
    - pulumi destroy : Tear down your stack's resources entirely

For more information, please visit the project page: https://www.pulumi.com/docs/

Usage:
  pulumi [command]

Stack Management Commands:
  new Create a new Pulumi project
  config Manage configuration
  stack Manage stacks and view stack state
  console Opens the current stack in the Pulumi Console
  import Import resources into an existing stack
  refresh Refresh the resources in a stack
  state Edit the current stack's state
  install Install packages and plugins for the current program or policy pack.

Deployment Commands:
  up Create or update the resources in a stack
  destroy Destroy all existing resources in the stack
  preview Show a preview of updates to a stack's resources
  cancel Cancel a stack's currently running update, if any

Environment Commands:
  env Manage environments

Pulumi Cloud Commands:
  login Log in to the Pulumi Cloud
  logout Log out of the Pulumi Cloud
  whoami Display the current logged-in user
  org Manage Organization configuration

Policy Management Commands:
  policy Manage resource policies

Plugin Commands:
  plugin Manage language and resource provider plugins
  schema Analyze package schemas
  package Work with Pulumi packages

Other Commands:
  version Print Pulumi's version number
  about Print information about the Pulumi environment.
  gen-completion Generate completion scripts for the Pulumi CLI

Experimental Commands:
  convert Convert Pulumi programs from a supported source program into other supported languages
  watch Continuously update the resources in a stack
  logs Show aggregated resource logs for a stack

Flags:
      --color string Colorize output. Choices are: always, never, raw, auto (default "auto")
  -C, --cwd string Run pulumi as if it had been started in another directory
      --disable-integrity-checking Disable integrity checking of checkpoint files
  -e, --emoji Enable emojis in the output
  -Q, --fully-qualify-stack-names Show fully-qualified stack names
  -h, --help help for pulumi
      --logflow Flow log settings to child processes (like plugins)
      --logtostderr Log to stderr instead of to files
      --memprofilerate int Enable more precise (and expensive) memory allocation profiles by setting runtime.MemProfileRate
      --non-interactive Disable interactive mode for all commands
      --profiling string Emit CPU and memory profiles and an execution trace to '[filename].[pid].{cpu,mem,trace}', respectively
      --tracing file: Emit tracing to the specified endpoint. Use the file: scheme to write tracing data to a local file
  -v, --verbose int Enable verbose logging (e.g., v=3); anything >3 is very verbose

Use `pulumi [command] --help` for more information about a command.

J’utilise le système de fichiers de la machine afin de stocker les fichiers d’état localement :

(base) ubuntu@microstack:~$ pulumi login --local
Logged in to microstack as ubuntu (file://~)

J’utilise Miniconda pour rapidement disposer d’un environnement Python prêt à l’emploi :

Miniconda - Anaconda documentation

ubuntu@microstack:~$ mkdir -p ~/miniconda3
ubuntu@microstack:~$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
ubuntu@microstack:~$ bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
ubuntu@microstack:~$ rm ~/miniconda3/miniconda.sh

PREFIX=/home/ubuntu/miniconda3
Unpacking payload ...

Installing base environment...

Preparing transaction: ...working... done
Executing transaction: ...working... done
installation finished.
ubuntu@microstack:~$ source ~/miniconda3/bin/activate
(base) ubuntu@microstack:~$ conda init --all
no change /home/ubuntu/miniconda3/condabin/conda
no change /home/ubuntu/miniconda3/bin/conda
no change /home/ubuntu/miniconda3/bin/conda-env
no change /home/ubuntu/miniconda3/bin/activate
no change /home/ubuntu/miniconda3/bin/deactivate
no change /home/ubuntu/miniconda3/etc/profile.d/conda.sh
no change /home/ubuntu/miniconda3/etc/fish/conf.d/conda.fish
no change /home/ubuntu/miniconda3/shell/condabin/Conda.psm1
no change /home/ubuntu/miniconda3/shell/condabin/conda-hook.ps1
no change /home/ubuntu/miniconda3/lib/python3.12/site-packages/xontrib/conda.xsh
no change /home/ubuntu/miniconda3/etc/profile.d/conda.csh
modified /home/ubuntu/.bashrc
modified /home/ubuntu/.zshrc
modified /home/ubuntu/.config/fish/config.fish
modified /home/ubuntu/.xonshrc
modified /home/ubuntu/.tcshrc

==> For changes to take effect, close and re-open your current shell. <==

(base) ubuntu@microstack:~$ source .bashrc
(base) ubuntu@microstack:~$ type pip
pip is /home/ubuntu/miniconda3/bin/pip

Création d’un environnement avec le fournisseur OpenStack et son template en Python pour Pulumi :

(base) ubuntu@microstack:~$ mkdir test
(base) ubuntu@microstack:~$ cd test
(base) ubuntu@microstack:~/test$ pulumi new openstack-python
This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

Project name (test):  
Project description (A minimal OpenStack Python Pulumi program):  
Created project 'test'

Stack name (dev):  
Enter your passphrase to protect config/secrets:  
Re-enter your passphrase to confirm:  
Created stack 'dev'

The toolchain to use for installing dependencies and running the program pip
Installing dependencies...

Creating virtual environment...
Finished creating virtual environment
Updating pip, setuptools, and wheel in virtual environment...
Requirement already satisfied: pip in ./venv/lib/python3.12/site-packages (24.3.1)
Collecting setuptools
  Downloading setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB)
Collecting wheel
  Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB)
Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 29.2 MB/s eta 0:00:00
Downloading wheel-0.45.1-py3-none-any.whl (72 kB)
Installing collected packages: wheel, setuptools
Successfully installed setuptools-75.6.0 wheel-0.45.1
Finished updating
Installing dependencies in virtual environment...
Collecting pulumi<4.0.0,>=3.0.0 (from -r requirements.txt (line 1))
  Downloading pulumi-3.144.1-py3-none-any.whl.metadata (12 kB)
Collecting pulumi-openstack<4.0.0,>=3.0.0 (from -r requirements.txt (line 2))
  Downloading pulumi_openstack-3.15.2-py3-none-any.whl.metadata (9.2 kB)
Collecting protobuf~=4.21 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading protobuf-4.25.5-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
Collecting grpcio~=1.66.2 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading grpcio-1.66.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.9 kB)
Collecting dill~=0.3 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading dill-0.3.9-py3-none-any.whl.metadata (10 kB)
Collecting six~=1.12 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting semver~=2.13 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading semver-2.13.0-py2.py3-none-any.whl.metadata (5.0 kB)
Collecting pyyaml~=6.0 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting debugpy~=1.8.7 (from pulumi<4.0.0,>=3.0.0->-r requirements.txt (line 1))
  Downloading debugpy-1.8.11-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.1 kB)
Collecting parver>=0.2.1 (from pulumi-openstack<4.0.0,>=3.0.0->-r requirements.txt (line 2))
  Downloading parver-0.5-py3-none-any.whl.metadata (2.7 kB)
Collecting arpeggio>=1.7 (from parver>=0.2.1->pulumi-openstack<4.0.0,>=3.0.0->-r requirements.txt (line 2))
  Downloading Arpeggio-2.0.2-py2.py3-none-any.whl.metadata (2.4 kB)
Collecting attrs>=19.2 (from parver>=0.2.1->pulumi-openstack<4.0.0,>=3.0.0->-r requirements.txt (line 2))
  Downloading attrs-24.3.0-py3-none-any.whl.metadata (11 kB)
Downloading pulumi-3.144.1-py3-none-any.whl (294 kB)
Downloading pulumi_openstack-3.15.2-py3-none-any.whl (551 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 551.5/551.5 kB 19.0 MB/s eta 0:00:00
Downloading debugpy-1.8.11-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 99.4 MB/s eta 0:00:00
Downloading dill-0.3.9-py3-none-any.whl (119 kB)
Downloading grpcio-1.66.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.8/5.8 MB 10.8 MB/s eta 0:00:00
Downloading parver-0.5-py3-none-any.whl (15 kB)
Downloading protobuf-4.25.5-cp37-abi3-manylinux2014_x86_64.whl (294 kB)
Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 767.5/767.5 kB 120.7 MB/s eta 0:00:00
Downloading semver-2.13.0-py2.py3-none-any.whl (12 kB)
Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)
Downloading Arpeggio-2.0.2-py2.py3-none-any.whl (55 kB)
Downloading attrs-24.3.0-py3-none-any.whl (63 kB)
Installing collected packages: arpeggio, six, semver, pyyaml, protobuf, grpcio, dill, debugpy, attrs, pulumi, parver, pulumi-openstack
Successfully installed arpeggio-2.0.2 attrs-24.3.0 debugpy-1.8.11 dill-0.3.9 grpcio-1.66.2 parver-0.5 protobuf-4.25.5 pulumi-3.144.1 pulumi-openstack-3.15.2 pyyaml-6.0.2 semver-2.13.0 six-1.17.0
Finished installing dependencies
Finished installing dependencies

Your new project is ready to go! 

To perform an initial deployment, run `pulumi up`


(base) ubuntu@microstack:~/test$ ls
Pulumi.dev.yaml Pulumi.yaml __main__.py requirements.txt venv
(base) ubuntu@microstack:~/test$ cat Pulumi.yaml 
name: test
description: A minimal OpenStack Python Pulumi program
runtime:
  name: python
  options:
    toolchain: pip
    virtualenv: venv
config:
  pulumi:tags:
    value:
      pulumi:template: openstack-python

Je modifie le principal fichier Python pour déployer une stack avec une nouvelle instance Ubuntu de test :

(base) ubuntu@microstack:~/test$ cat __main__.py 
"""An OpenStack Python Pulumi program"""

import pulumi
from pulumi_openstack import compute

# Create an OpenStack resource (Compute Instance)
instance = compute.Instance('test-pulumi',
        flavor_name='m1.small',
    key_pair="sunbeam",                        
        image_name='ubuntu')

# Export the IP of the instance
pulumi.export('instance_ip', instance.access_ip_v4)

Et je déploie ma stack avec Pulumi en utilisant les variables d’environnement du tenant de démonstration :

(base) ubuntu@microstack:~/test$ source ../demo-openrc 
(base) ubuntu@microstack:~/test$ pulumi up
Enter your passphrase to unlock config/secrets
    (set PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE to remember):  
Enter your passphrase to unlock config/secrets
Previewing update (dev):
     Type Name Plan Info
 + pulumi:pulumi:Stack test-dev create 1 warning
 + └─ openstack:compute:Instance test-pulumi create     

Diagnostics:
  pulumi:pulumi:Stack (test-dev):
    warning: provider config warning: Users not using loadbalancer resources can ignore this message. Support for neutron-lbaas will be removed on next major release. Octavia will be the only supported method for loadbalancer resources. Users using octavia will have to remove 'use_octavia' option from the provider configuration block. Users using neutron-lbaas will have to migrate/upgrade to octavia.

Outputs:
    instance_ip: output<string>

Resources:
    + 2 to create

Do you want to perform this update? yes
Updating (dev):
     Type Name Status Info
 + pulumi:pulumi:Stack test-dev created (15s) 1 warning
 + └─ openstack:compute:Instance test-pulumi created (15s)     

Diagnostics:
  pulumi:pulumi:Stack (test-dev):
    warning: provider config warning: Users not using loadbalancer resources can ignore this message. Support for neutron-lbaas will be removed on next major release. Octavia will be the only supported method for loadbalancer resources. Users using octavia will have to remove 'use_octavia' option from the provider configuration block. Users using neutron-lbaas will have to migrate/upgrade to octavia.

Outputs:
    instance_ip: "192.168.0.227"

Resources:
    + 2 created

Duration: 16s

L’instance est créée …

(base) ubuntu@microstack:~/test$ openstack server list
+--------------------------------------+---------------------+--------+-----------------------------------------+--------+----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------------------+--------+-----------------------------------------+--------+----------+
| 6ec4c753-92c5-4221-a76d-8045638efd32 | test-pulumi-54bdcae | ACTIVE | demo-network=192.168.0.227 | ubuntu | m1.small |
| efe46971-56f4-4da4-9c6e-eebee2795b72 | instance1 | ACTIVE | demo-network=172.16.2.31, 192.168.0.166 | ubuntu | m1.tiny |
+--------------------------------------+---------------------+--------+-----------------------------------------+--------+----------+
(base) ubuntu@microstack:~/test$ openstack server show test-pulumi-54bdcae --fit
+-------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hostname | test-pulumi-54bdcae |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | None |
| OS-EXT-SRV-ATTR:kernel_id | None |
| OS-EXT-SRV-ATTR:launch_index | None |
| OS-EXT-SRV-ATTR:ramdisk_id | None |
| OS-EXT-SRV-ATTR:reservation_id | None |
| OS-EXT-SRV-ATTR:root_device_name | None |
| OS-EXT-SRV-ATTR:user_data | None |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2024-12-24T10:36:58.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | demo-network=192.168.0.227 |
| config_drive | |
| created | 2024-12-24T10:36:53Z |
| description | test-pulumi-54bdcae |
| flavor | description=, disk='30', ephemeral='0', , id='m1.small', is_disabled=, is_public='True', location=, name='m1.small', original_name='m1.small', |
| | ram='2048', rxtx_factor=, swap='0', vcpus='1' |
| hostId | 021ebc639163d77a5eb8018996d0b8aad50066a8552682313f3f293f |
| host_status | None |
| id | 6ec4c753-92c5-4221-a76d-8045638efd32 |
| image | ubuntu (ff3ccb3b-f44f-4b50-a030-20267c302d75) |
| key_name | sunbeam |
| locked | False |
| locked_reason | None |
| name | test-pulumi-54bdcae |
| progress | 0 |
| project_id | 8b373f844efd47c8b38c4f1bcdcfba2a |
| properties | |
| security_groups | name='default' |
| server_groups | [] |
| status | ACTIVE |
| tags | |
| trusted_image_certificates | None |
| updated | 2024-12-24T10:36:59Z |
| user_id | 114709b3342c45f295d116c63c51884a |
| volumes_attached | |
+-------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+

Et on s’y connecte en y adjoignant une adresse IP flottante :

(base) ubuntu@microstack:~$ ssh -i snap/openstack/637/sunbeam ubuntu@172.16.2.124
Welcome to Ubuntu 22.04.5 LTS (GNU/Linux 5.15.0-127-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/pro

 System information as of Tue Dec 24 10:42:48 UTC 2024

  System load: 0.0 Processes: 89
  Usage of /: 5.0% of 28.89GB Users logged in: 0
  Memory usage: 9% IPv4 address for ens3: 192.168.0.227
  Swap usage: 0%

Expanded Security Maintenance for Applications is not enabled.

0 updates can be applied immediately.

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status

The list of available updates is more than a week old.
To check for new updates run: sudo apt update
New release '24.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Tue Dec 24 10:42:48 2024 from 172.16.2.1
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

Une instance facilement supprimée via la stack déployée par Pulumi :

(base) ubuntu@microstack:~/test$ pulumi destroy
Enter your passphrase to unlock config/secrets
    (set PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE to remember):  
Enter your passphrase to unlock config/secrets
Previewing destroy (dev):
     Type Name Plan       
 - pulumi:pulumi:Stack test-dev delete     
 - └─ openstack:compute:Instance test-pulumi delete     

Outputs:
  - instance_ip: "192.168.0.227"

Resources:
    - 2 to delete

Do you want to perform this destroy? yes
Destroying (dev):
     Type Name Status              
 - pulumi:pulumi:Stack test-dev deleted (0.00s)     
 - └─ openstack:compute:Instance test-pulumi deleted (10s)       

Outputs:
  - instance_ip: "192.168.0.227"

Resources:
    - 2 deleted

Duration: 11s

The resources in the stack have been deleted, but the history and configuration associated with the stack are still maintained. 
If you want to remove the stack completely, run `pulumi stack rm dev`.

(base) ubuntu@microstack:~/test$ openstack server list
+--------------------------------------+-----------+--------+-----------------------------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-----------+--------+-----------------------------------------+--------+---------+
| efe46971-56f4-4da4-9c6e-eebee2795b72 | instance1 | ACTIVE | demo-network=172.16.2.31, 192.168.0.166 | ubuntu | m1.tiny |
+--------------------------------------+-----------+--------+-----------------------------------------+--------+---------+

(base) ubuntu@microstack:~/test$ pulumi stack rm dev
This will permanently remove the 'dev' stack!
Please confirm that this is what you'd like to do by typing `dev`: dev
Stack 'dev' has been removed!
(base) ubuntu@microstack:~/test$ pulumi stack ls
NAME LAST UPDATE RESOURCE COUNT

La pile d’observabilité Canonical (COS) a été déployée précedemment. MicroStack propagera automatiquement les métriques et les tableaux de bord par défaut, vous permettant de surveiller sans effort l’état de votre déploiement de Sunbeam à un ou plusieurs nœuds sans avoir besoin d’une configuration supplémentaire via Grafana (dont on a obtenu les identifiants via Juju) :

Et le tout (dans ce cas précis) pour une consommation conséquente …

Pour des déploiements plus avancés, MicroStack supporte également les clusters multi-nœuds. Vous pouvez suivre le tutoriel détaillé sur la documentation officielle de Canonical pour déployer un cluster OpenStack multi-nœuds via Sunbeam et MAAS :

Comme on a pule voir, MicroStack est conçu pour être hautemement personnalisable , permettant l’intégration de divers backends de stockage tels que Cinder (stockage en bloc) et Swift (stockage d’objets).

Object Storage | Canonical

Les entreprises peuvent adapter la plateforme à leurs besoins uniques en intégrant des plug-ins et des extensions spécifiques. MicroStack offre une solution de cloud privée légère, facile à installer et à utiliser, ce qui en fait un choix idéal pour les organisations cherchant à déployer rapidement un environnement cloud sans la complexité traditionnelle associée à OpenStack. Avec sa flexibilité et sa capacité de customisation, MicroStack est une solution robuste et adaptable pour une variété de besoins en infrastructure cloud y compris pour le contexte de l’Edge Computing …

Une autre manière de faire fonctionner OpenStack au-dessus de Kubernetes avec Atmosphere …

Karim — Mon, 03 Jun 2024 15:53:30 +0000

Plusieurs projets de déploiement d’OpenStack qui reposent sur un cluster Kubernetes existent à commencer par OpenStack Helm :

Mais c’est aussi le cas du projet open source Atmosphere du fournisseur Cloud Vexxhost qui fournit un large éventail de fonctionnalités d’infrastructure, notamment des machines virtuelles, Kubernetes, du bare metal, du stockage en bloc et en objet, des équilibreurs de charge en tant que service, et plus encore …

Atmosphere deployment tool | VEXXHOST

Atmosphere fait tourner OpenStack au-dessus de Kubernetes. Cette configuration permet des déploiements, des mises à niveau et des contrôles de santé simples et facile.

VEXXHOST Launches Atmosphere, a New Open Source, OpenStack Deployment Tool

GitHub - vexxhost/atmosphere: Simple & easy private cloud platform featuring VMs, Kubernetes & bare-metal

Je vais simplement suivre le modus operandi proposé par Vexxhost pour un serveur Bare Metal :

Pour cela je pars d’un serveur dans PhoenixNAP sous Ubuntu 22.04 LTS :

Mise à jour de ce dernier et installation des paquets nécessaires au lancement du déploiement d’Atmosphere :

ubuntu@atmosphere:~$ sudo apt-get update && sudo apt-get install git python3-pip -y && sudo pip install poetry

ubuntu@atmosphere:~$ git clone https://github.com/vexxhost/atmosphere.git
Cloning into 'atmosphere'...
remote: Enumerating objects: 25111, done.
remote: Counting objects: 100% (825/825), done.
remote: Compressing objects: 100% (398/398), done.
remote: Total 25111 (delta 377), reused 704 (delta 306), pack-reused 24286
Receiving objects: 100% (25111/25111), 10.78 MiB | 50.17 MiB/s, done.
Resolving deltas: 100% (14299/14299), done.
ubuntu@atmosphere:~$ cd atmosphere/
ubuntu@atmosphere:~/atmosphere$ ls
CHANGELOG.md Jenkinsfile build doc galaxy.yml hack meta playbooks pyproject.toml test-requirements.txt tox.ini
Dockerfile README.md charts flake.lock go.mod images mkdocs.yml plugins release-please-config.json tests zuul.d
Earthfile atmosphere cmd flake.nix go.sum internal molecule poetry.lock roles tools

ubuntu@atmosphere:~/atmosphere$ sudo poetry install --with dev
Creating virtualenv atmosphere-NEvTTHEY-py3.10 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 89 installs, 0 updates, 0 removals

  - Installing attrs (23.2.0)
  - Installing pycparser (2.22)
  - Installing rpds-py (0.18.0)
  - Installing cffi (1.16.0)
  - Installing markupsafe (2.1.5)
  - Installing mdurl (0.1.2)
  - Installing referencing (0.35.0)
  - Installing cryptography (42.0.5)
  - Installing jinja2 (3.1.3)
  - Installing jsonschema-specifications (2023.12.1)
  - Installing markdown-it-py (3.0.0)
  - Installing packaging (24.0)
  - Installing pbr (6.0.0)
  - Installing pygments (2.17.2)
  - Installing pyyaml (6.0.1)
  - Installing resolvelib (1.0.1)
  - Installing wrapt (1.16.0)
  - Installing ansible-core (2.16.6)
  - Installing bracex (2.4)
  - Installing certifi (2024.2.2)
  - Installing charset-normalizer (3.3.2)
  - Installing click (8.1.7)
  - Installing debtcollector (3.0.0)
  - Installing idna (3.7)
  - Installing iso8601 (2.1.0)
  - Installing jsonschema (4.21.1)
  - Installing netaddr (0.8.0)
  - Installing netifaces (0.11.0)
  - Installing oslo-i18n (6.3.0)
  - Installing pyparsing (3.1.2)
  - Installing rich (13.7.1)
  - Installing subprocess-tee (0.4.1)
  - Installing tzdata (2024.1)
  - Installing urllib3 (2.2.1)
  - Installing ansible-compat (4.1.11)
  - Installing click-help-colors (0.9.4)
  - Installing decorator (5.1.1)
  - Installing distro (1.9.0)
  - Installing enrich (1.2.7)
  - Installing exceptiongroup (1.2.1)
  - Installing iniconfig (2.0.0)
  - Installing jsonpointer (2.4)
  - Installing mccabe (0.7.0)
  - Installing msgpack (1.0.8)
  - Installing os-service-types (1.7.0)
  - Installing oslo-utils (7.1.0)
  - Installing pluggy (1.5.0)
  - Installing pycodestyle (2.9.1)
  - Installing pyflakes (2.5.0)
  - Installing requests (2.31.0)
  - Installing rfc3986 (2.0.0)
  - Installing six (1.16.0)
  - Installing stevedore (5.2.0)
  - Installing tomli (2.0.1)
  - Installing typing-extensions (4.11.0)
  - Installing wcmatch (8.5.1)
  - Installing appdirs (1.4.4)
  - Installing coverage (7.5.0)
  - Installing docker (7.0.0)
  - Installing dogpile-cache (1.3.2)
  - Installing execnet (2.1.1)
  - Installing flake8 (5.0.4)
  - Installing isort (5.13.2)
  - Installing jmespath (1.0.1)
  - Installing jsonpatch (1.33)
  - Installing keystoneauth1 (5.6.0)
  - Installing molecule (6.0.3)
  - Installing munch (4.0.0)
  - Installing oslo-config (9.4.0)
  - Installing oslo-context (5.5.0)
  - Installing oslo-serialization (5.4.0)
  - Installing py (1.11.0)
  - Installing pyinotify (0.9.6)
  - Installing pytest (7.4.4)
  - Installing python-dateutil (2.9.0.post0)
  - Installing regex (2024.4.28)
  - Installing requestsexceptions (1.4.0)
  - Installing selinux (0.3.0)
  - Installing docker-image-py (0.1.12)
  - Installing flake8-isort (4.2.0)
  - Installing molecule-plugins (23.5.3)
  - Installing openstacksdk (0.62.0)
  - Installing oslo-log (5.5.1)
  - Installing pytest-cov (3.0.0)
  - Installing pytest-forked (1.6.0)
  - Installing pytest-mock (3.14.0)
  - Installing pytest-xdist (3.6.1)
  - Installing rjsonnet (0.5.4)
  - Installing ruyaml (0.91.0)

Installing the current project: atmosphere (1.10.4.post186.dev0+779cb921)

Très rapidement, je lance le déploiement qui prend un peu moins d’une heure ici :

ubuntu@atmosphere:~/atmosphere$ sudo poetry run molecule converge -s aio
INFO aio scenario test matrix: dependency, create, prepare, converge
INFO Performing prerun with role_name_check=0...
INFO Running aio > dependency
WARNING Skipping, missing the requirements file.
WARNING Skipping, missing the requirements file.
INFO Running aio > create

PLAY [Wait for user to read warning] *******************************************

TASK [Gathering Facts] *********************************************************
Monday 03 June 2024 12:02:39 +0000 (0:00:00.020) 0:00:00.020 ***********
ok: [localhost]
.
.
.
.
.
PLAY [Configure networking] ****************************************************

TASK [Gathering Facts] *********************************************************
Monday 03 June 2024 12:49:26 +0000 (0:00:00.418) 0:45:34.621 ***********
ok: [instance]

TASK [Add IP address to "br-ex"] ***********************************************
Monday 03 June 2024 12:49:28 +0000 (0:00:02.159) 0:45:36.781 ***********
ok: [instance]

TASK [Set "br-ex" interface to "up"] *******************************************
Monday 03 June 2024 12:49:28 +0000 (0:00:00.153) 0:45:36.934 ***********
ok: [instance]

PLAY RECAP *********************************************************************
instance : ok=669 changed=267 unreachable=0 failed=0 skipped=246 rescued=0 ignored=1   

Monday 03 June 2024 12:49:28 +0000 (0:00:00.304) 0:45:37.239 ***********
===============================================================================
vexxhost.atmosphere.percona_xtradb_cluster : Apply Percona XtraDB cluster - 203.51s
vexxhost.atmosphere.cinder : Deploy Helm chart ------------------------ 157.84s
vexxhost.atmosphere.keycloak : Deploy Helm chart ---------------------- 156.65s
vexxhost.atmosphere.heat : Deploy Helm chart -------------------------- 120.67s
vexxhost.atmosphere.manila : Deploy Helm chart ------------------------ 104.84s
vexxhost.atmosphere.nova : Deploy Helm chart -------------------------- 100.67s
vexxhost.ceph.osd : Install OSDs --------------------------------------- 89.67s
vexxhost.atmosphere.glance : Deploy Helm chart ------------------------- 88.95s
vexxhost.atmosphere.magnum : Deploy Helm chart ------------------------- 88.06s
vexxhost.atmosphere.octavia : Deploy Helm chart ------------------------ 83.51s
vexxhost.atmosphere.keystone : Deploy Helm chart ----------------------- 80.67s
vexxhost.atmosphere.neutron : Deploy Helm chart ------------------------ 74.13s
vexxhost.atmosphere.barbican : Deploy Helm chart ----------------------- 67.48s
vexxhost.ceph.mon : Run Bootstrap coomand ------------------------------ 62.21s
vexxhost.atmosphere.placement : Deploy Helm chart ---------------------- 58.82s
vexxhost.kubernetes.cluster_api : Set node selector for Cluster API components -- 57.15s
vexxhost.atmosphere.glance_image : Download image ---------------------- 53.93s
vexxhost.atmosphere.glance_image : Check if image exists --------------- 50.38s
vexxhost.atmosphere.neutron : Create networks -------------------------- 36.62s
vexxhost.atmosphere.rabbitmq : Deploy cluster -------------------------- 35.65s

Le cluster est déployé et je peux vérifier les accès disponibles localement sur ce serveur physique :

root@instance:~# apt install python3-openstackclient -y

root@instance:~# source openrc
root@instance:~# openstack endpoint list
+----------------------------------+-----------+--------------+-----------------+---------+-----------+------------------------------------------------------------------------------+
| ID | Region | Service Name | Service Type | Enabled | Interface | URL |
+----------------------------------+-----------+--------------+-----------------+---------+-----------+------------------------------------------------------------------------------+
| 0584539654844f7f956088e43836ed4c | RegionOne | swift | object-store | True | public | https://object-store.131-153-200-197.nip.io/swift/v1/%(tenant_id)s |
| 069df7eb37584d91a22a95e7d36493d3 | RegionOne | manila | share | True | internal | http://manila-api.openstack.svc.cluster.local:8786/v1 |
| 08b5e9b96ddf4ce79f270f582dc6bf7b | RegionOne | manilav2 | sharev2 | True | public | https://share.131-153-200-197.nip.io/v2 |
| 0b95720b8c0e4de692acf18b769ce8be | RegionOne | heat-cfn | cloudformation | True | admin | http://heat-cfn.openstack.svc.cluster.local:8000/v1 |
| 120958efeb924a0a98bf03ce807c0ace | RegionOne | cinderv3 | volumev3 | True | internal | http://cinder-api.openstack.svc.cluster.local:8776/v3/%(tenant_id)s |
| 1427e413681e4518ac03d768bb44cb60 | RegionOne | placement | placement | True | admin | http://placement-api.openstack.svc.cluster.local:8778/ |
| 1611e222d7b8447596aa44c0965aec1a | RegionOne | cinderv3 | volumev3 | True | public | https://volume.131-153-200-197.nip.io/v3/%(tenant_id)s |
| 19047158ec1446b5b746682bc8d9dd93 | RegionOne | heat-cfn | cloudformation | True | internal | http://heat-cfn.openstack.svc.cluster.local:8000/v1 |
| 1e04bacc8aae44b3818de40e386cf68e | RegionOne | barbican | key-manager | True | admin | http://barbican-api.openstack.svc.cluster.local:9311/ |
| 27d7b0d81a104b54a3dcb8632297707f | RegionOne | keystone | identity | True | admin | http://keystone-api.openstack.svc.cluster.local:5000/ |
| 2c4218a3da784f98890bcb6ac6cb20ae | RegionOne | heat | orchestration | True | admin | http://heat-api.openstack.svc.cluster.local:8004/v1/%(project_id)s |
| 2f19518bd66c421eb6a5e25f7c93c96e | RegionOne | manila | share | True | public | http://manila.openstack.svc.cluster.local/v1 |
| 3578d37453854cfbafdc18f2be4f0a62 | RegionOne | glance | image | True | public | https://image.131-153-200-197.nip.io/ |
| 3a2e106afdbf43689676c85718319f88 | RegionOne | glance | image | True | admin | http://glance-api.openstack.svc.cluster.local:9292/ |
| 3e3240df5df847bc84c204e0b18783f1 | RegionOne | glance | image | True | internal | http://glance-api.openstack.svc.cluster.local:9292/ |
| 3f51c888203c433ea31d2ca67cf3e359 | RegionOne | manilav2 | sharev2 | True | admin | http://manila-api.openstack.svc.cluster.local:8786/v2 |
| 44f08551b541410d8bb28b3a148dca0f | RegionOne | heat | orchestration | True | internal | http://heat-api.openstack.svc.cluster.local:8004/v1/%(project_id)s |
| 6501cca34cd5401d8561522f062ec126 | RegionOne | magnum | container-infra | True | internal | http://magnum-api.openstack.svc.cluster.local:9511/v1 |
| 65c4f687017d4ff4b774a61cd670de52 | RegionOne | barbican | key-manager | True | public | https://key-manager.131-153-200-197.nip.io/ |
| 695342e32bf84ae48cc0c873227ce1ce | RegionOne | heat | orchestration | True | public | https://orchestration.131-153-200-197.nip.io/v1/%(project_id)s |
| 71af00dd937e49c28827dadbe6d55bbe | RegionOne | swift | object-store | True | internal | http://rook-ceph-rgw-ceph.openstack.svc.cluster.local/swift/v1/%(tenant_id)s |
| 7ada3efd05b64af1ae726d495fe61a6e | RegionOne | cinderv3 | volumev3 | True | admin | http://cinder-api.openstack.svc.cluster.local:8776/v3/%(tenant_id)s |
| 87e74af69cbb4577b4dd29d106aed5fa | RegionOne | magnum | container-infra | True | public | https://container-infra.131-153-200-197.nip.io/v1 |
| 8dae31f3eed946eea30025e56fbfd83a | RegionOne | nova | compute | True | admin | http://nova-api.openstack.svc.cluster.local:8774/v2.1 |
| 8e562226dccc4df0af3c8cdfdec95084 | RegionOne | barbican | key-manager | True | internal | http://barbican-api.openstack.svc.cluster.local:9311/ |
| 8f629f7e748d4c3087bf17aaa0ce2e47 | RegionOne | manila | share | True | admin | http://manila-api.openstack.svc.cluster.local:8786/v1 |
| 91149164a0584145b6344ae1d525457b | RegionOne | keystone | identity | True | public | https://identity.131-153-200-197.nip.io/ |
| a936e3ca42a54b6e97672f20e17c095b | RegionOne | neutron | network | True | admin | http://neutron-server.openstack.svc.cluster.local:9696/ |
| b987ba9af66c4a14ba42d1fd8c1b2285 | RegionOne | placement | placement | True | public | https://placement.131-153-200-197.nip.io/ |
| b9e2e43e90d649d5b6089c7474ba30cd | RegionOne | octavia | load-balancer | True | internal | http://octavia-api.openstack.svc.cluster.local:9876/ |
| bf2839c099f946e39d21a0b6bcad0b87 | RegionOne | neutron | network | True | public | https://network.131-153-200-197.nip.io/ |
| bf959da766554e91972fe1da9cff6d8e | RegionOne | heat-cfn | cloudformation | True | public | https://cloudformation.131-153-200-197.nip.io/v1 |
| d027a62b34ca4eaabc508c04ad8f94e7 | RegionOne | magnum | container-infra | True | admin | http://magnum-api.openstack.svc.cluster.local:9511/v1 |
| d3e857109be14bfabf82e962068b54af | RegionOne | nova | compute | True | public | https://compute.131-153-200-197.nip.io/v2.1 |
| d6985c01aca54ec49e0ec1c728fc271b | RegionOne | nova | compute | True | internal | http://nova-api.openstack.svc.cluster.local:8774/v2.1 |
| d74feae56d034bd7ad421e4588f0731e | RegionOne | octavia | load-balancer | True | admin | http://octavia-api.openstack.svc.cluster.local:9876/ |
| e7bcb22c547e41489cc7d5732bca4a84 | RegionOne | manilav2 | sharev2 | True | internal | http://manila-api.openstack.svc.cluster.local:8786/v2 |
| f005b62e7769497eb54b0c0a3fa3c587 | RegionOne | octavia | load-balancer | True | public | https://load-balancer.131-153-200-197.nip.io/ |
| f40bb006a4984428b581042a78539ef8 | RegionOne | neutron | network | True | internal | http://neutron-server.openstack.svc.cluster.local:9696/ |
| f4d8af60368645c7a5f1d9605afb5494 | RegionOne | placement | placement | True | internal | http://placement-api.openstack.svc.cluster.local:8778/ |
| f5aa8f8ba519480199f224800480e8cb | RegionOne | keystone | identity | True | internal | http://keystone-api.openstack.svc.cluster.local:5000/ |
+----------------------------------+-----------+--------------+-----------------+---------+-----------+------------------------------------------------------------------------------+

Ainsi que des accès via l’Ingress Controller du cluster Kubernetes sous-jacent :

root@instance:~# kubectl cluster-info
Kubernetes control plane is running at https://10.96.240.10:6443
CoreDNS is running at https://10.96.240.10:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root@instance:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
instance Ready control-plane 60m v1.28.4 131.153.200.197 <none> Ubuntu 22.04.4 LTS 5.15.0-105-generic containerd://1.7.9
root@instance:~# kubectl get ing -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
auth-system keycloak atmosphere keycloak.131-153-200-197.nip.io 10.98.36.135 80, 443 54m
monitoring kube-prometheus-stack-alertmanager atmosphere alertmanager.131-153-200-197.nip.io 10.98.36.135 80, 443 53m
monitoring kube-prometheus-stack-grafana atmosphere grafana.131-153-200-197.nip.io 10.98.36.135 80, 443 53m
monitoring kube-prometheus-stack-prometheus atmosphere prometheus.131-153-200-197.nip.io 10.98.36.135 80, 443 53m
openstack cloudformation atmosphere cloudformation.131-153-200-197.nip.io 10.98.36.135 80, 443 33m
openstack compute atmosphere compute.131-153-200-197.nip.io 10.98.36.135 80, 443 38m
openstack compute-novnc-proxy atmosphere vnc.131-153-200-197.nip.io 10.98.36.135 80, 443 38m
openstack container-infra atmosphere container-infra.131-153-200-197.nip.io 10.98.36.135 80, 443 27m
openstack container-infra-registry atmosphere container-infra-registry.131-153-200-197.nip.io 10.98.36.135 80, 443 27m
openstack dashboard atmosphere dashboard.131-153-200-197.nip.io 10.98.36.135 80, 443 23m
openstack identity atmosphere identity.131-153-200-197.nip.io 10.98.36.135 80, 443 51m
openstack image atmosphere image.131-153-200-197.nip.io 10.98.36.135 80, 443 46m
openstack key-manager atmosphere key-manager.131-153-200-197.nip.io 10.98.36.135 80, 443 49m
openstack load-balancer atmosphere load-balancer.131-153-200-197.nip.io 10.98.36.135 80, 443 30m
openstack network atmosphere network.131-153-200-197.nip.io 10.98.36.135 80, 443 36m
openstack orchestration atmosphere orchestration.131-153-200-197.nip.io 10.98.36.135 80, 443 33m
openstack placement atmosphere placement.131-153-200-197.nip.io 10.98.36.135 80, 443 41m
openstack rook-ceph-cluster atmosphere object-store.131-153-200-197.nip.io 10.98.36.135 80, 443 48m
openstack sharev2 atmosphere share.131-153-200-197.nip.io 10.98.36.135 80, 443 23m
openstack volumev3 atmosphere volume.131-153-200-197.nip.io 10.98.36.135 80, 443 42m

Le domaine WildCard utilisé ici me donne accès au Dashboard Horizon pour plus de simplicité :

Je charge une image locale d’Ubuntu 24.04 LTS dans le cluster :

root@instance:~# cat openrc 
# Ansible managed: Do NOT edit this file manually!

export OS_IDENTITY_API_VERSION=3

export OS_AUTH_URL="https://identity.131-153-200-197.nip.io/v3"
export OS_AUTH_TYPE=password
export OS_REGION_NAME="RegionOne"
export OS_USER_DOMAIN_NAME=Default
export OS_USERNAME="admin-RegionOne"
export OS_PASSWORD="lzi232PTaHpzoC2HjwSLKepZELQd6ENJ"
export OS_PROJECT_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin

export OS_CACERT=/usr/local/share/ca-certificates/atmosphere.crt

root@instance:~# wget -c https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

Saving to: ‘noble-server-cloudimg-amd64.img’

noble-server-cloudimg-amd64.img 100%[=====================================================================================================>] 454.00M 17.9MB/s in 30s     

2024-06-03 13:22:40 (15.3 MB/s) - ‘noble-server-cloudimg-amd64.img’ saved [476053504/476053504]

root@instance:~# openstack image create --public --container-format bare --disk-format qcow2 --file ~/noble-server-cloudimg-amd64.img ubuntu-24.04
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| container_format | bare |
| created_at | 2024-06-03T13:23:12Z |
| disk_format | qcow2 |
| file | /v2/images/81d8eafa-4054-455c-9640-4e83c0566d21/file |
| id | 81d8eafa-4054-455c-9640-4e83c0566d21 |
| min_disk | 0 |
| min_ram | 0 |
| name | ubuntu-24.04 |
| owner | 43321f42e8434f8aa53531bd104e2809 |
| properties | locations='[]', os_hidden='False', owner_specified.openstack.md5='', owner_specified.openstack.object='images/ubuntu-24.04', owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2024-06-03T13:23:12Z |
| visibility | public |
+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+

root@instance:~# openstack image list
+--------------------------------------+---------------------------+--------+
| ID | Name | Status |
+--------------------------------------+---------------------------+--------+
| 04e2f39a-e4ff-426b-aeee-b82acd3bf611 | amphora-x64-haproxy | active |
| 49a3b09d-c191-4c36-8541-31efaffb404d | cirros | active |
| b7e75cdb-37b0-4f2a-badf-cfdee7bca83d | manila-service-image | active |
| 3c5df46c-7015-411c-8009-afa6695672a6 | ubuntu-2204-kube-v1.27.8s | active |
| 81d8eafa-4054-455c-9640-4e83c0566d21 | ubuntu-24.04 | active |
+--------------------------------------+---------------------------+--------+

Configuration d’une topologie réseau pour un rapide test :

Avec l’image chargée dans Glance, lancement de trois petites instances Ubuntu 24.04 LTS …

root@instance:~# openstack server list
+--------------------------------------+-------+--------+--------------------------------------+--------------+----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-------+--------+--------------------------------------+--------------+----------+
| 07d6bf6e-edff-4729-9133-31849ea6fe87 | k0s-3 | ACTIVE | network1=10.96.250.216, 11.12.13.185 | ubuntu-24.04 | m1.small |
| afd10cda-f99c-4877-9948-a7f25b5e756a | k0s-1 | ACTIVE | network1=10.96.250.211, 11.12.13.99 | ubuntu-24.04 | m1.small |
| d70230a1-9fdf-40d9-959d-c9960479b4a5 | k0s-2 | ACTIVE | network1=10.96.250.208, 11.12.13.80 | ubuntu-24.04 | m1.small |
+--------------------------------------+-------+--------+--------------------------------------+--------------+----------+

Un cluster kubernetes simple sur la base de ces trois instances est alors possible avec k0sctl.

GitHub - k0sproject/k0sctl: A bootstrapping and management tool for k0s clusters.

root@instance:~# wget -c https://github.com/k0sproject/k0sctl/releases/download/v0.17.8/k0sctl-linux-x64
root@instance:~# mv k0sctl-linux-x64 /usr/local/bin/k0sctl && chmod +x /usr/local/bin/k0sctl 
root@instance:~# k0sctl
NAME:
   k0sctl - k0s cluster management tool

USAGE:
   k0sctl [global options] command [command options] 

COMMANDS:
   version Output k0sctl version
   apply Apply a k0sctl configuration
   kubeconfig Output the admin kubeconfig of the cluster
   init Create a configuration template
   reset Remove traces of k0s from all of the hosts
   backup Take backup of existing clusters state
   config Configuration related sub-commands
   completion  
   help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d Enable debug logging (default: false) [$DEBUG]
   --trace Enable trace logging (default: false) [$TRACE]
   --no-redact Do not hide sensitive information in the output (default: false)
   --help, -h show help

root@instance:~# k0sctl init --k0s > k0sctl.yaml

root@instance:~# cat k0sctl.yaml 
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - ssh:
      address: 10.96.250.211
      user: ubuntu
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: controller
  - ssh:
      address: 10.96.250.208
      user: ubuntu
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: worker
  - ssh:
      address: 10.96.250.216
      user: ubuntu
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: worker
  k0s:
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: Cluster
      metadata:
        name: k0s
      spec:
        api:
          k0sApiPort: 9443
          port: 6443
        installConfig:
          users:
            etcdUser: etcd
            kineUser: kube-apiserver
            konnectivityUser: konnectivity-server
            kubeAPIserverUser: kube-apiserver
            kubeSchedulerUser: kube-scheduler
        konnectivity:
          adminPort: 8133
          agentPort: 8132
        network:
          kubeProxy:
            disabled: false
            mode: iptables
          kuberouter:
            autoMTU: true
            mtu: 0
            peerRouterASNs: ""
            peerRouterIPs: ""
          podCIDR: 10.244.0.0/16
          provider: kuberouter
          serviceCIDR: 10.96.0.0/12
        podSecurityPolicy:
          defaultPolicy: 00-k0s-privileged
        storage:
          type: etcd
        telemetry:
          enabled: true

root@instance:~# k0sctl apply --config k0sctl.yaml 

⠀⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███ ███ ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███ ███ ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███ ███ ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████ ███ ██████████
k0sctl v0.17.8 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
INFO ==> Running phase: Set k0s version  
INFO Looking up latest stable k0s version         
INFO Using k0s version v1.30.1+k0s.0              
INFO ==> Running phase: Connect to hosts 
INFO [ssh] 10.96.250.216:22: connected            
INFO [ssh] 10.96.250.208:22: connected            
INFO [ssh] 10.96.250.211:22: connected            
INFO ==> Running phase: Detect host operating systems 
INFO [ssh] 10.96.250.216:22: is running Ubuntu 24.04 LTS 
INFO [ssh] 10.96.250.208:22: is running Ubuntu 24.04 LTS 
INFO [ssh] 10.96.250.211:22: is running Ubuntu 24.04 LTS 
INFO ==> Running phase: Acquire exclusive host lock 
INFO ==> Running phase: Prepare hosts    
INFO ==> Running phase: Gather host facts 
INFO [ssh] 10.96.250.216:22: using k0s-3 as hostname 
INFO [ssh] 10.96.250.211:22: using k0s-1 as hostname 
INFO [ssh] 10.96.250.208:22: using k0s-2 as hostname 
INFO [ssh] 10.96.250.211:22: discovered ens3 as private interface 
INFO [ssh] 10.96.250.208:22: discovered ens3 as private interface 
INFO [ssh] 10.96.250.216:22: discovered ens3 as private interface 
INFO [ssh] 10.96.250.211:22: discovered 11.12.13.99 as private address 
INFO [ssh] 10.96.250.208:22: discovered 11.12.13.80 as private address 
INFO [ssh] 10.96.250.216:22: discovered 11.12.13.185 as private address 
INFO ==> Running phase: Validate hosts   
INFO ==> Running phase: Validate facts   
INFO ==> Running phase: Download k0s on hosts 
INFO [ssh] 10.96.250.216:22: downloading k0s v1.30.1+k0s.0 
INFO [ssh] 10.96.250.211:22: downloading k0s v1.30.1+k0s.0 
INFO [ssh] 10.96.250.208:22: downloading k0s v1.30.1+k0s.0 
INFO ==> Running phase: Install k0s binaries on hosts 
INFO [ssh] 10.96.250.211:22: validating configuration 
INFO ==> Running phase: Configure k0s    
INFO [ssh] 10.96.250.211:22: installing new configuration 
INFO ==> Running phase: Initialize the k0s cluster 
INFO [ssh] 10.96.250.211:22: installing k0s controller 
INFO [ssh] 10.96.250.211:22: waiting for the k0s service to start 
INFO [ssh] 10.96.250.211:22: waiting for kubernetes api to respond 
INFO ==> Running phase: Install workers  
INFO [ssh] 10.96.250.216:22: validating api connection to https://11.12.13.99:6443 
INFO [ssh] 10.96.250.208:22: validating api connection to https://11.12.13.99:6443 
INFO [ssh] 10.96.250.211:22: generating a join token for worker 1 
INFO [ssh] 10.96.250.211:22: generating a join token for worker 2 
INFO [ssh] 10.96.250.216:22: writing join token   
INFO [ssh] 10.96.250.208:22: writing join token   
INFO [ssh] 10.96.250.216:22: installing k0s worker 
INFO [ssh] 10.96.250.208:22: installing k0s worker 
INFO [ssh] 10.96.250.216:22: starting service     
INFO [ssh] 10.96.250.216:22: waiting for node to become ready 
INFO [ssh] 10.96.250.208:22: starting service     
INFO [ssh] 10.96.250.208:22: waiting for node to become ready 
INFO ==> Running phase: Release exclusive host lock 
INFO ==> Running phase: Disconnect from hosts 
INFO ==> Finished in 42s                 
INFO k0s cluster version v1.30.1+k0s.0 is now installed 
INFO Tip: To access the cluster you can now fetch the admin kubeconfig using: 
INFO k0sctl kubeconfig

Kubernetes via k0s est actif et accessible via son fichier kubeconfigspécifique :

root@instance:~# k0sctl kubeconfig > kubeconfig
root@instance:~# kubectl --kubeconfig=kubeconfig cluster-info
Kubernetes control plane is running at https://10.96.250.211:6443
CoreDNS is running at https://10.96.250.211:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root@instance:~# kubectl --kubeconfig=kubeconfig get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k0s-2 Ready <none> 2m41s v1.30.1+k0s 11.12.13.80 <none> Ubuntu 24.04 LTS 6.8.0-31-generic containerd://1.7.17
k0s-3 Ready <none> 2m41s v1.30.1+k0s 11.12.13.185 <none> Ubuntu 24.04 LTS 6.8.0-31-generic containerd://1.7.17

root@instance:~# kubectl --kubeconfig=kubeconfig get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-6997b8f8bd-957bd 1/1 Running 0 2m41s
kube-system pod/coredns-6997b8f8bd-ddr5t 1/1 Running 0 2m41s
kube-system pod/konnectivity-agent-2bxgl 1/1 Running 0 2m51s
kube-system pod/konnectivity-agent-4gfsw 1/1 Running 0 2m51s
kube-system pod/kube-proxy-2cq5w 1/1 Running 0 2m51s
kube-system pod/kube-proxy-m6rnv 1/1 Running 0 2m51s
kube-system pod/kube-router-p9s4t 1/1 Running 0 2m51s
kube-system pod/kube-router-qhcp4 1/1 Running 0 2m51s
kube-system pod/metrics-server-5cd4986bbc-rf4wc 1/1 Running 0 2m57s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3m15s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 3m5s
kube-system service/metrics-server ClusterIP 10.97.91.41 <none> 443/TCP 3m1s

Le démonstrateur FC en une commande (habituelle maintenant) :

root@instance:~# cat test.yaml 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fcdemo3
  labels:
    app: fcdemo3
spec:
  replicas: 4
  selector:
    matchLabels:
      app: fcdemo3
  template:
    metadata:
      labels:
        app: fcdemo3
    spec:
      containers:
      - name: fcdemo3
        image: mcas/franceconnect-demo2:latest
        ports:
        - containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: fcdemo-service
spec:
  type: ClusterIP
  selector:
    app: fcdemo3
  ports:
  - protocol: TCP
    port: 80
    targetPort: 3000

root@instance:~# kubectl --kubeconfig=kubeconfig apply -f test.yaml
deployment.apps/fcdemo3 created
service/fcdemo-service created

root@instance:~# kubectl --kubeconfig=kubeconfig get po,svc
NAME READY STATUS RESTARTS AGE
pod/fcdemo3-85f6bd87c-7jvpk 1/1 Running 0 13s
pod/fcdemo3-85f6bd87c-btv9m 1/1 Running 0 13s
pod/fcdemo3-85f6bd87c-nlw9x 1/1 Running 0 13s
pod/fcdemo3-85f6bd87c-v66bs 1/1 Running 0 13s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/fcdemo-service ClusterIP 10.98.123.24 <none> 80/TCP 13s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8m19s

Redirection de port ici pour accès local à ce dernier …

root@instance:~# kubectl --kubeconfig=kubeconfig port-forward service/fcdemo-service 12222:80 --address='0.0.0.0'
Forwarding from 0.0.0.0:12222 -> 3000

Sachant qu’Atmosphere disposait de Magnum pour déployer un cluster Kubernetes intrinsèque …

Pour conclure, Atmosphere est un projet pour un déploiement simplifié d’un cluster Openstack conteneurisé dans un cluster Kubernetes de base (en reprenant OpenStack Helm) et que l’on retrouve également dans la nouvelle mouture de Canonical MicroStack par exemple …

OpenStack for the edge, micro clouds and developers

À suivre !

Gérer des conteneurs qui ressemblent à des machines virtuelles avec Bootloose (successeur de…

Karim — Sun, 31 Mar 2024 18:02:05 +0000

Dans plusieurs articles précédents, j’avais décrit l’usage de Footloose, un binaire écrit en Go qui permettait de créer des conteneurs qui ressemblent à des machines virtuelles.

Ces conteneurs exécutent systemd en tant que PID 1 et un démon ssh qui peut être utilisé pour se connecter au conteneur. Ces “machines” se comportent comme des VM, il est même possible d’y faire tourner dockerd.

GitHub - weaveworks/footloose: Container Machines - Containers that look like Virtual Machines

Footloose peut être utilisé pour une variété de tâches, partout où vous aimeriez avoir des machines virtuelles mais voulez des temps de démarrage rapides ou avez besoin de plusieurs d’entre elles.

C’est au tour de l’équipe à l’origine de k0s de reprendre le flambeau avec Bootloose

GitHub - k0sproject/bootloose: Manage containers that look like virtual machines

Sur le même principe …

Illustration avec le lancement d’une instance Ubuntu 22.04 LTS en ARM64 Bits dans Hetzner Cloud :

dans laquelle je commence par installer le moteurDocker :

root@ubuntu-16gb-hel1-1:~# curl -fsSL https://get.docker.com | sh -
# Executing docker install script, commit: e5543d473431b782227f8908005543bb4389b8de
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
+ sh -c install -m 0755 -d /etc/apt/keyrings
+ sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
+ sh -c chmod a+r /etc/apt/keyrings/docker.gpg
+ sh -c echo "deb [arch=arm64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
+ sh -c docker version
Client: Docker Engine - Community
 Version: 26.0.0
 API version: 1.45
 Go version: go1.21.8
 Git commit: 2ae903e
 Built: Wed Mar 20 15:18:14 2024
 OS/Arch: linux/arm64
 Context: default

Server: Docker Engine - Community
 Engine:
  Version: 26.0.0
  API version: 1.45 (minimum version 1.24)
  Go version: go1.21.8
  Git commit: 8b79278
  Built: Wed Mar 20 15:18:14 2024
  OS/Arch: linux/arm64
  Experimental: false
 containerd:
  Version: 1.6.28
  GitCommit: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version: 1.1.12
  GitCommit: v1.1.12-0-g51d5e94
 docker-init:
  Version: 0.19.0
  GitCommit: de40ad0

================================================================================

To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:

    dockerd-rootless-setuptool.sh install

Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.

To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/

WARNING: Access to the remote API on a privileged Docker daemon is equivalent
         to root access on the host. Refer to the 'Docker daemon attack surface'
         documentation for details: https://docs.docker.com/go/attack-surface/

================================================================================

root@ubuntu-16gb-hel1-1:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Puis récupération du binaire Bootloose depuis GitHub :

root@ubuntu-16gb-hel1-1:~# wget -c https://github.com/k0sproject/bootloose/releases/download/v0.7.3/bootloose-linux-arm64
HTTP request sent, awaiting response... 200 OK
Length: 6291456 (6.0M) [application/octet-stream]
Saving to: ‘bootloose-linux-arm64’

bootloose-linux-arm64 100%[=====================================================================================================>] 6.00M 21.8MB/s in 0.3s    

(21.8 MB/s) - ‘bootloose-linux-arm64’ saved [6291456/6291456]

root@ubuntu-16gb-hel1-1:~# chmod +x bootloose-linux-arm64 
root@ubuntu-16gb-hel1-1:~# mv bootloose-linux-arm64 /usr/local/bin/bootloose
root@ubuntu-16gb-hel1-1:~# bootloose
bootloose - Container Machines

Usage:
  bootloose [command]

Available Commands:
  completion Generate the autocompletion script for the specified shell
  config Manage cluster configuration
  create Create a cluster
  delete Delete a cluster
  help Help about any command
  show Show all running machines or a single machine with a given hostname.
  ssh SSH into a machine
  start Start cluster machines
  stop Stop cluster machines
  version Print bootloose version

Flags:
  -c, --config string Cluster configuration file (default "bootloose.yaml")
  -h, --help help for bootloose

Use "bootloose [command] --help" for more information about a command.

Releases · k0sproject/bootloose

Et création de de fichier de configuration YAML pour créer mes instances avec cette image :

root@ubuntu-16gb-hel1-1:~# bootloose config create --replicas 3 --image quay.io/k0sproject/bootloose-ubuntu22.04:latest

root@ubuntu-16gb-hel1-1:~# cat bootloose.yaml 
cluster:
  name: cluster
  privateKey: ~/.ssh/id_rsa
machines:
- count: 3
  spec:
    image: quay.io/k0sproject/bootloose-ubuntu22.04:latest
    name: node%d
    portMappings:
    - containerPort: 22
    privileged: true
    volumes:
    - type: volume
      destination: /var/lib/k0s

root@ubuntu-16gb-hel1-1:~# bootloose create
INFO[0000] Docker Image: quay.io/k0sproject/bootloose-ubuntu22.04:latest present locally 
INFO[0000] Creating machine: cluster-node0 ...          
INFO[0000] Creating machine: cluster-node1 ...          
INFO[0000] Creating machine: cluster-node2 ...  

root@ubuntu-16gb-hel1-1:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f452da977dd6 quay.io/k0sproject/bootloose-ubuntu22.04:latest "/sbin/init" About a minute ago Up About a minute 0.0.0.0:32770->22/tcp, :::32770->22/tcp cluster-node2
790a6594d4a3 quay.io/k0sproject/bootloose-ubuntu22.04:latest "/sbin/init" About a minute ago Up About a minute 0.0.0.0:32769->22/tcp, :::32769->22/tcp cluster-node1
c674c7d3551f quay.io/k0sproject/bootloose-ubuntu22.04:latest "/sbin/init" About a minute ago Up About a minute 0.0.0.0:32768->22/tcp, :::32768->22/tcp cluster-node0

root@ubuntu-16gb-hel1-1:~# docker inspect $(docker ps -aq) | grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "172.17.0.4",
                    "IPAddress": "172.17.0.4",
            "SecondaryIPAddresses": null,
            "IPAddress": "172.17.0.3",
                    "IPAddress": "172.17.0.3",
            "SecondaryIPAddresses": null,
            "IPAddress": "172.17.0.2",
                    "IPAddress": "172.17.0.2",

root@ubuntu-16gb-hel1-1:~# for i in {2..4}; do ssh root@172.17.0.$i 'hostname'; done
Warning: Permanently added '172.17.0.2' (ED25519) to the list of known hosts.
node0
Warning: Permanently added '172.17.0.3' (ED25519) to the list of known hosts.
node1
Warning: Permanently added '172.17.0.4' (ED25519) to the list of known hosts.
node2

Les instances sont donc accessibles en SSH et je vais utiliser k0sctl pour y créer un cluster Kubernetes via k0s:

GitHub - k0sproject/k0sctl: A bootstrapping and management tool for k0s clusters.

root@ubuntu-16gb-hel1-1:~# chmod +x k0sctl-linux-arm64 
root@ubuntu-16gb-hel1-1:~# mv k0sctl-linux-arm64 /usr/local/bin/k0sctl
root@ubuntu-16gb-hel1-1:~# k0sctl
NAME:
   k0sctl - k0s cluster management tool

USAGE:
   k0sctl [global options] command [command options] 

COMMANDS:
   version Output k0sctl version
   apply Apply a k0sctl configuration
   kubeconfig Output the admin kubeconfig of the cluster
   init Create a configuration template
   reset Remove traces of k0s from all of the hosts
   backup Take backup of existing clusters state
   config Configuration related sub-commands
   completion  
   help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d Enable debug logging (default: false) [$DEBUG]
   --trace Enable trace logging (default: false) [$TRACE]
   --no-redact Do not hide sensitive information in the output (default: false)
   --help, -h show help

Initialisation du cluster dans Bootloose via ce fichier de configuration YAML :

root@ubuntu-16gb-hel1-1:~# k0sctl init --k0s > k0sctl.yaml

root@ubuntu-16gb-hel1-1:~# cat k0sctl.yaml 
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - ssh:
      address: 172.17.0.2
      user: root
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: controller
  - ssh:
      address: 172.17.0.3
      user: root
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: worker
  - ssh:
      address: 172.17.0.4
      user: root
      port: 22
      keyPath: /root/.ssh/id_rsa
    role: worker
  k0s:
    version: null
    versionChannel: stable
    dynamicConfig: false
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: Cluster
      metadata:
        name: k0s
      spec:
        api:
          k0sApiPort: 9443
          port: 6443
        installConfig:
          users:
            etcdUser: etcd
            kineUser: kube-apiserver
            konnectivityUser: konnectivity-server
            kubeAPIserverUser: kube-apiserver
            kubeSchedulerUser: kube-scheduler
        konnectivity:
          adminPort: 8133
          agentPort: 8132
        network:
          kubeProxy:
            disabled: false
            mode: iptables
          kuberouter:
            autoMTU: true
            mtu: 0
            peerRouterASNs: ""
            peerRouterIPs: ""
          podCIDR: 10.244.0.0/16
          provider: kuberouter
          serviceCIDR: 10.96.0.0/12
        podSecurityPolicy:
          defaultPolicy: 00-k0s-privileged
        storage:
          type: etcd
        telemetry:
          enabled: false

Lancement de la création du cluster k0s avec ce fichier de configuration :

root@ubuntu-16gb-hel1-1:~# k0sctl apply --config k0sctl.yaml 

⠀⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███ ███ ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███ ███ ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███ ███ ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████ ███ ██████████
k0sctl v0.17.5 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
INFO ==> Running phase: Set k0s version  
INFO Looking up latest stable k0s version         
INFO Using k0s version v1.29.2+k0s.0              
INFO ==> Running phase: Connect to hosts 
INFO [ssh] 172.17.0.4:22: connected               
INFO [ssh] 172.17.0.3:22: connected               
INFO [ssh] 172.17.0.2:22: connected               
INFO ==> Running phase: Detect host operating systems 
INFO [ssh] 172.17.0.4:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 172.17.0.2:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 172.17.0.3:22: is running Ubuntu 22.04.3 LTS 
INFO ==> Running phase: Acquire exclusive host lock 
INFO ==> Running phase: Prepare hosts    
INFO [ssh] 172.17.0.2:22: is a container, applying a fix 
INFO [ssh] 172.17.0.3:22: is a container, applying a fix 
INFO [ssh] 172.17.0.4:22: is a container, applying a fix 
INFO ==> Running phase: Gather host facts 
INFO [ssh] 172.17.0.4:22: using node2 as hostname 
INFO [ssh] 172.17.0.3:22: using node1 as hostname 
INFO [ssh] 172.17.0.2:22: using node0 as hostname 
INFO [ssh] 172.17.0.4:22: discovered eth0 as private interface 
INFO [ssh] 172.17.0.3:22: discovered eth0 as private interface 
INFO [ssh] 172.17.0.2:22: discovered eth0 as private interface 
INFO ==> Running phase: Validate hosts   
INFO ==> Running phase: Validate facts   
INFO ==> Running phase: Download k0s on hosts 
INFO [ssh] 172.17.0.4:22: downloading k0s v1.29.2+k0s.0 
INFO [ssh] 172.17.0.3:22: downloading k0s v1.29.2+k0s.0 
INFO [ssh] 172.17.0.2:22: downloading k0s v1.29.2+k0s.0 
INFO ==> Running phase: Install k0s binaries on hosts 
INFO [ssh] 172.17.0.2:22: validating configuration 
INFO ==> Running phase: Configure k0s    
INFO [ssh] 172.17.0.2:22: installing new configuration 
INFO ==> Running phase: Initialize the k0s cluster 
INFO [ssh] 172.17.0.2:22: installing k0s controller 
INFO [ssh] 172.17.0.2:22: waiting for the k0s service to start 
INFO [ssh] 172.17.0.2:22: waiting for kubernetes api to respond 
INFO ==> Running phase: Install workers  
INFO [ssh] 172.17.0.3:22: validating api connection to https://172.17.0.2:6443 
INFO [ssh] 172.17.0.4:22: validating api connection to https://172.17.0.2:6443 
INFO [ssh] 172.17.0.2:22: generating a join token for worker 1 
INFO [ssh] 172.17.0.2:22: generating a join token for worker 2 
INFO [ssh] 172.17.0.4:22: writing join token      
INFO [ssh] 172.17.0.3:22: writing join token      
INFO [ssh] 172.17.0.3:22: installing k0s worker   
INFO [ssh] 172.17.0.4:22: installing k0s worker   
INFO [ssh] 172.17.0.4:22: starting service        
INFO [ssh] 172.17.0.3:22: starting service        
INFO [ssh] 172.17.0.4:22: waiting for node to become ready 
INFO [ssh] 172.17.0.3:22: waiting for node to become ready 
INFO ==> Running phase: Release exclusive host lock 
INFO ==> Running phase: Disconnect from hosts 
INFO ==> Finished in 48s                 
INFO k0s cluster version v1.29.2+k0s.0 is now installed 
INFO Tip: To access the cluster you can now fetch the admin kubeconfig using: 
INFO k0sctl kubeconfig

Je peux alors récupérer le fichier kubeconfig pour l’utilisation locale du client kubectl :

root@ubuntu-16gb-hel1-1:~# mkdir .kube
root@ubuntu-16gb-hel1-1:~# k0sctl kubeconfig > .kube/config
root@ubuntu-16gb-hel1-1:~# cat .kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURBRENDQWVpZ0F3SUJBZ0lVSnhLUU9xRlVzZ0NnY0FiTWNDdEFSTXlmNDFVd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0dERVdNQlFHQTFVRUF4TU5hM1ZpWlhKdVpYUmxjeTFqWVRBZUZ3MHlOREF6TXpFeE5qQXhNREJhRncwegpOREF6TWpreE5qQXhNREJhTUJneEZqQVVCZ05WQkFNVERXdDFZbVZ5Ym1WMFpYTXRZMkV3Z2dFaU1BMEdDU3FHClNJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUURXM3V4cVZtMUdMWWNJWlozMDIrajcwTXQ3Y1Y1Zkw3ZmYKcDNwaURhUVk1d1RGMHVmMEVWcDZqQXV5TVpNaGtWTzlSSmo2TGYyb0xNcVV1V0ZqVzFKdVZUVHY5U0JDc1FhVApudVpxVktyWnJ0Nkt5bm1zaVBuWTlMZW5wWklkdzA4NUZnOHdWdGFlclVmOUlaaUMrSEZPL1grdjlIOUFySlJJCjVNby80dzBaZEJjdXVUcEREVzZpcTNNMjF2b1pCR3Jwbk43TkcvWUpLcGNwM2xlbVV5d1JHL0dEMisyTlRML08KdEx5cmF4TDVWVkVYbENEbTFlTXJZOXBzSjNQcDJySmE0RmxyMGRVVFJRUHcycjhBUUxyTFhJUmFBYXpEV3lBLwpKR1NKZG81YUdrL3NxM2l3clRJb3c0UmlwRGRQWmV2UkdZSlRLaWZWOStRYk14bkVuUEZSQWdNQkFBR2pRakJBCk1BNEdBMVVkRHdFQi93UUVBd0lCQmpBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJScFl0V2MKMERrSlRaTDFxR0tkK284N1ZndTFBakFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBbWtGVTBYam14ZWRPMDFtegozWU9QS3hBeTg2N1NpMlM0WTRMcDNqL3NCcW9TWVlkYldzS3c2Wmx1aTdra2VYUEMvM0pJRTd4YjNoL0p6WWxvCnRZMEliOEpnUnZvVk1sV0ZHNitxMnNnZEo3cmx6blg1QXBQZkRqUXY5RUJHR3VXM21IY1UxbXVCUStrUG9JK2cKaU5OZzFITjRoYllYMVJOMitOK3pKcVNVaWh1WFZDbVhFa2YwQ05WU3VyTEZ6cGNKWWJpNWowVVcwYmRhVWFpTwpJRDJOd3VZcUhDNS96RDM0RDB6T1RVMzRtUTVsZDRUYUx4ZmlwRzZrWDVVSEJtT0U5d3lIbVZuY2VzelJ0alZTCnlFblNJV0g1cmM5Sk84RVBXLzVPRTJsb2E3a240U05iQU5NbUNFaVZtZE9IdU5NSHFGaUQxR2oxOWI5MzVpSnkKUnpGS2hBPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    server: https://172.17.0.2:6443
  name: k0s-cluster
contexts:
- context:
    cluster: k0s-cluster
    user: admin
  name: k0s-cluster
current-context: k0s-cluster
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURXVENDQWtHZ0F3SUJBZ0lVY3lhN3dQUGVIR0c1eUM3WWdUV1FCZnlNM2lJd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0dERVdNQlFHQTFVRUF4TU5hM1ZpWlhKdVpYUmxjeTFqWVRBZUZ3MHlOREF6TXpFeE5qQXhNREJhRncweQpOVEF6TXpFeE5qQXhNREJhTURReEZ6QVZCZ05WQkFvVERuTjVjM1JsYlRwdFlYTjBaWEp6TVJrd0Z3WURWUVFECkV4QnJkV0psY201bGRHVnpMV0ZrYldsdU1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0MKQVFFQTFrbjJONVorWEt0MW50N3A1QSt2YWQ5OU9DQ25DQ1FKN1JDeXUzTlRaZXlPWDlwK2lRNzBqaC9mWTZjagovNElFMDVDQlJNZzU1cGtVSWszc0k5anJqNFhBcmRaSnQ1QTFUbDZFY0tuWEQ2NzJyWFV4bVlsRG9FK3p6SitkClc1VnVkc0JmbXVqRnluTGJyT2xDYzZEMDlOWmo5b24zRVlGK0dhcXFBTG5wOU9mNENkaFdLL3hYcWVabFBPaFMKZ2RLdERYVzRXYjlEVEZpbytXQ2RObmR6NWd5SUF5OTM3akRzUG9ESEl2OXdEMkVkd2JSVzNMUERjRjhFaFpKWQpUcUV1Z2F0bzB6ek1vaVZzdkptNVZSU0ZjS2h6S1NkZW1obnFyaTBLWXFFTG9Oa2QwcnFkbFF2cnhCSUlvd1E1Cnl4TWhnOHhRb3pDSUtVZXY5SVhoVTlLT2VRSURBUUFCbzM4d2ZUQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0hRWUQKVlIwbEJCWXdGQVlJS3dZQkJRVUhBd0VHQ0NzR0FRVUZCd01DTUF3R0ExVWRFd0VCL3dRQ01BQXdIUVlEVlIwTwpCQllFRk5SbjV4ek1qVXZSeFVtRXdmbzBLVm45cFgzME1COEdBMVVkSXdRWU1CYUFGR2xpMVp6UU9RbE5rdldvCllwMzZqenRXQzdVQ01BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQjRqU29XNmpnZmRkMlFGM0FRNWxJckVSMFkKQVJSV1RtNTdtTkZPM041OE1PZXR1UDAzOWxvSnIrbDY4aXVQcEUyd2VoL1hTNGxrM3B6dSszSS9IVDNlNW1PcwpiU0pGMXZWbHlaZnZTd1BveEMxcVBPakR3R28xQW1xUUpYRzhjc1E1WWVPR3lVN3BjdjY0KzRrL2xXQmovSEliCmQ4ZnVkMzh4cUo5SUV5SXdBU0lVenhVbllPZ1ZyOWlzRGtxTlk4czBtc3loQXpFYkNQdDQ2NFhiU0pqWG40aSsKOTFVK2FFZG1BMHN5Mi9jc2h6TFZCYzlOanJER1Yya0ZTL2hQNU9YZ0ZycEpEcnNnRVVOcjRudzNRaEl0RkJvdAp4Q2VXYkxTdXFPWTJjcnVvU1hWdmMrdFkyV0xWZ0RQVDRkZ3E3UGhhZm5oTUJkNmxyME55WFNuVzdOWFQKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMWtuMk41WitYS3QxbnQ3cDVBK3ZhZDk5T0NDbkNDUUo3UkN5dTNOVFpleU9YOXArCmlRNzBqaC9mWTZjai80SUUwNUNCUk1nNTVwa1VJazNzSTlqcmo0WEFyZFpKdDVBMVRsNkVjS25YRDY3MnJYVXgKbVlsRG9FK3p6SitkVzVWdWRzQmZtdWpGeW5MYnJPbENjNkQwOU5aajlvbjNFWUYrR2FxcUFMbnA5T2Y0Q2RoVwpLL3hYcWVabFBPaFNnZEt0RFhXNFdiOURURmlvK1dDZE5uZHo1Z3lJQXk5MzdqRHNQb0RISXY5d0QyRWR3YlJXCjNMUERjRjhFaFpKWVRxRXVnYXRvMHp6TW9pVnN2Sm01VlJTRmNLaHpLU2RlbWhucXJpMEtZcUVMb05rZDBycWQKbFF2cnhCSUlvd1E1eXhNaGc4eFFvekNJS1VldjlJWGhVOUtPZVFJREFRQUJBb0lCQUFhR3dpVDNSR24ySHVMegp6eFBQRm55Vy9lMVRzVUtpTmxzdUF3T0tnNk83REtzR3NJdmtGTGF2YWRKVEtObURVRHBSVUY2VDZvK0hZZ0daCmRmT3hpNXNYYThMZm4rY2pVVHhOektMUnlXY0U0U1p2UjA5eHlzbDdJL0s3ZWNOc1RhejROdkUwM2JGSXhrQUIKNnJBeTJzTUtOSWt4c29DcC9Qa3pKWEpZTnpQcVBtL05Kb3hja1h5KzVXRUxiY25Od2lxUUZzU09zT21ldEZ3YQpYcTZrSkVYRzIwVGpvQjg0SlFsNldNN0M5TFp0TWUrcnpsRGdsZFl1ZU92M1FWSE9FU0d2T0VuTFh3TEVSdkg3CmpPbUs4L040dU53L0NpTXJlUjh0WWR2dVBQNVA2Z1V5OUdmdXc2bkM4SzlTOE9CSVZmOWpxRjlsdG9wZndhNVMKRmdCd1BnRUNnWUVBNEI4T0hVZmp1ajR1amNyU01RYm0xQzRhN0QxckJ3YmxuY2RuNFlSYndpKzhOdmhEcGt5SQowMjBod1czRWpUTGJTSTB5UGJvNnQ4MWJEZk1xcjdudm1CblBZTDN2T1lmY3hDRklMcWJwSHd3M1h3aTBsM1BuCmhKVkZyaGJZckthSlVNcDZXVW9zTGdpVDE3MWx2ZnRRZFd5WGpWaGhLQ25SVTJSTExIOEg5NUVDZ1lFQTlNVGgKQWMyaTVSTnRzWDFFOTBFQjZUK0Rxekd3WkQ5WG1YVFQ4M3hrSzVENWV5UzIyUFpGa0NVU3pTSUZtVkpHSzh1ZwpuL2tQQTVZbWMzQUNiWEptVHlDQStSTkxhMU5TTDhGNXN4RzJ3Ris4OXNLVDBhT3VnNE5Ld1RHYkd6QVFKSjN0CitndmcwYXhJQW04b2RrU1RMbTg3K2N0UDVWK2QyNGNHVmh4VHhHa0NnWUVBaE5uancvZVpSZzBXQzNidW9hRTEKc3hDaFpPZ0RTV2NOTlRtK21pK2JOTUNYRVA2Wkd6ckM0SkVRTVZpZjZoTDdhVVpKUWMzaWdKRjZLQXE4Z3UzMAoySFIwT1NSZGFmemZJR09hSmcxS290emE5YnB6VWxPaUtUVlMySjh5VVNWbXdEMUZ5U2Z1aUZzTlNCVTgraUMrCjBOeE4rYnNwM0dUdGNFRkRUbHorbkFFQ2dZQXBOTS9RZHViNmU2Zlczb2p5dXgzd1A0SVNHdjVnRWczVVJzZTcKMFBBb0tYTG1tVXF6QWRxNkpwT2d0eDZNTGo2ajl0Ym10NDRnZzNHYnMzcWxKRUkzQmZqUWRjQVhwR0pNcWR5cwpHY3BUWG9xNFhBOTRsbjYxb3krOWtIVlZRV1VtVlBRUVNWbWkwc2NZcWMvOUFSUnFGODNZQXJORG9USTVGK1VvCk1BS21LUUtCZ1FDZ0tSOEpYMkpOakR6QjVWd3lwdmo3SDdFNFV4Q1FBVXdZZ1JVZ0VCUThZZElPUitSWTFuREEKUWRLd1NOaWxodWxrTjFERFVCZmZqUHhJWndTNExJZEkyN0lqTHlqMFUvbDFrMkludU94OFdHck1hUnp6WXRGcgpTTjhQV2x6eERZTHNHaVpyR2MzVXk0Znd6Ty9lV2xFUFpod0NRVk5lTFpHcFRCYVpvbUIxQkE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

root@ubuntu-16gb-hel1-1:~# curl -LO https://dl.k8s.io/release/v1.29.2/bin/linux/arm64/kubectl && chmod +x kubectl && mv kubectl /usr/local/bin/

root@ubuntu-16gb-hel1-1:~# chmod 400 .kube/config
root@ubuntu-16gb-hel1-1:~# kubectl cluster-info
Kubernetes control plane is running at https://172.17.0.2:6443
CoreDNS is running at https://172.17.0.2:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Le cluster est alors opérationnel et actif avec deux noeuds Workers :

root@ubuntu-16gb-hel1-1:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
node1 Ready <none> 4m58s v1.29.2+k0s 172.17.0.3 <none> Ubuntu 22.04.3 LTS 5.15.0-100-generic containerd://1.7.13
node2 Ready <none> 4m58s v1.29.2+k0s 172.17.0.4 <none> Ubuntu 22.04.3 LTS 5.15.0-100-generic containerd://1.7.13

root@ubuntu-16gb-hel1-1:~# kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-555d98c87b-7dxxh 1/1 Running 0 5m
kube-system pod/coredns-555d98c87b-9ngzm 1/1 Running 0 5m
kube-system pod/konnectivity-agent-4fb6m 1/1 Running 0 5m5s
kube-system pod/konnectivity-agent-kj99g 1/1 Running 0 5m5s
kube-system pod/kube-proxy-6pbnw 1/1 Running 0 5m5s
kube-system pod/kube-proxy-xg4mh 1/1 Running 0 5m5s
kube-system pod/kube-router-tvzsb 1/1 Running 0 5m5s
kube-system pod/kube-router-wt97k 1/1 Running 0 5m5s
kube-system pod/metrics-server-7556957bb7-qtgd2 1/1 Running 0 5m6s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5m23s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 5m14s
kube-system service/metrics-server ClusterIP 10.100.127.105 <none> 443/TCP 5m10s

Je vais utiliser l’image Docker du projet Kasmweb pour obtenir un Desktop en ligne au sein de ce cluster :

Pour cela conversion de la formule donnée dans le Docker Hub en fichier YAML pour docker-compose via Composerize :

Composerize

Puis conversion en manifests YAML via Kompose:

root@ubuntu-16gb-hel1-1:~# cat docker-compose.yml 
name: ubuntu
services:
    ubuntu-jammy-dind:
        stdin_open: true
        tty: true
        shm_size: 512m
        ports:
            - 6901:6901
        environment:
            - VNC_PW=password
        image: kasmweb/ubuntu-jammy-dind:1.14.0-rolling

root@ubuntu-16gb-hel1-1:~# ./kompose convert
INFO Kubernetes file "ubuntu-jammy-dind-service.yaml" created 
INFO Kubernetes file "ubuntu-jammy-dind-deployment.yaml" created 

root@ubuntu-16gb-hel1-1:~# cat ubuntu-jammy-dind-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.32.0 (765fde254)
  labels:
    io.kompose.service: ubuntu-jammy-dind
  name: ubuntu-jammy-dind
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: ubuntu-jammy-dind
  template:
    metadata:
      annotations:
        kompose.cmd: ./kompose convert
        kompose.version: 1.32.0 (765fde254)
      labels:
        io.kompose.network/ubuntu-default: "true"
        io.kompose.service: ubuntu-jammy-dind
    spec:
      containers:
        - env:
            - name: VNC_PW
              value: password
          image: kasmweb/ubuntu-jammy-dind:1.14.0-rolling
          name: ubuntu-jammy-dind
          ports:
            - containerPort: 6901
              hostPort: 6901
              protocol: TCP
          stdin: true
          tty: true
      restartPolicy: Always

root@ubuntu-16gb-hel1-1:~# cat ubuntu-jammy-dind-service.yaml 
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.32.0 (765fde254)
  labels:
    io.kompose.service: ubuntu-jammy-dind
  name: ubuntu-jammy-dind
spec:
  ports:
    - name: "6901"
      port: 6901
      targetPort: 6901
  selector:
    io.kompose.service: ubuntu-jammy-dind

Déploiement de ces derniers dans le cluster k0s…


root@ubuntu-16gb-hel1-1:~# kubectl apply -f ubuntu-jammy-dind-deployment.yaml 
deployment.apps/ubuntu-jammy-dind created
root@ubuntu-16gb-hel1-1:~# kubectl apply -f ubuntu-jammy-dind-service.yaml 
service/ubuntu-jammy-dind created

root@ubuntu-16gb-hel1-1:~# kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default pod/ubuntu-jammy-dind-5d84c9847f-vhftx 1/1 Running 0 3m37s
kube-system pod/coredns-555d98c87b-7dxxh 1/1 Running 0 24m
kube-system pod/coredns-555d98c87b-9ngzm 1/1 Running 0 24m
kube-system pod/konnectivity-agent-4fb6m 1/1 Running 0 24m
kube-system pod/konnectivity-agent-kj99g 1/1 Running 0 24m
kube-system pod/kube-proxy-6pbnw 1/1 Running 0 24m
kube-system pod/kube-proxy-xg4mh 1/1 Running 0 24m
kube-system pod/kube-router-tvzsb 1/1 Running 0 24m
kube-system pod/kube-router-wt97k 1/1 Running 0 24m
kube-system pod/metrics-server-7556957bb7-qtgd2 1/1 Running 0 24m

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 24m
default service/ubuntu-jammy-dind ClusterIP 10.97.43.198 <none> 6901/TCP 3m30s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 24m
kube-system service/metrics-server ClusterIP 10.100.127.105 <none> 443/TCP 24m

J’expose le service via kubectl pour accéder au Desktop via NoVNC et les identifiants par défaut :


root@ubuntu-16gb-hel1-1:~# kubectl port-forward svc/ubuntu-jammy-dind 6901:6901 --address='0.0.0.0'
Forwarding from 0.0.0.0:6901 -> 6901

Comme dans l’article précédent, récupération d’Ollama en test …

curl -fsSL https://ollama.com/install.sh | sh

avec Llama2 :

D’autres exemples d’utilisation peuvent se retrouver dans le dépôt GitHub avec notamment Ansible ou Ignite …

À suivre !

Installer Canonical Kubernetes dans LXD avec Juju …

Karim — Sun, 31 Mar 2024 14:48:42 +0000

Canonical Kubernetes que l’on a pu aborder dans l’article précédent,

k8s-snap (Canonical Kubernetes) pour un déploiement simple et rapide d’un cluster k8s …

peut également être installé dans LXD. C’est un excellent moyen, par exemple, de tester Canonical Kubernetes en cluster sans avoir besoin de plusieurs hôtes physiques.

LXD: five easy pieces | Ubuntu

Je pars donc d’une simple instance Ubuntu 22.04 LTS dans DigitalOcean :

curl -X POST -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer '$TOKEN'' \
    -d '{"name":"lxd",
        "size":"s-4vcpu-16gb-amd",
        "region":"fra1",
        "image":"ubuntu-22-04-x64",
        "vpc_uuid":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"}' \
    "https://api.digitalocean.com/v2/droplets"

et j’y installe et initie LXD ainsi que de l’orchestrateur Juju :

root@lxd:~# snap remove lxd --purge && snap install lxd --candidate
lxd removed
lxd (candidate) 5.21.0-eb655b9 from Canonical✓ installed
root@lxd:~# snap list
Name Version Rev Tracking Publisher Notes
core20 20230207 1828 latest/stable canonical✓ base
core22 20240111 1122 latest/stable canonical✓ base
lxd 5.21.0-eb655b9 27957 latest/candidate canonical✓ -
snapd 2.58.2 18357 latest/stable canonical✓ snapd

root@lxd:~# lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (dir, lvm, powerflex, zfs, btrfs, ceph) [default=zfs]: dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=lxdbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: none
Would you like the LXD server to be available over the network? (yes/no) [default=no]: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:

The simplest way to deploy and maintain applications in the cloud

root@lxd:~# snap install juju
juju (3/stable) 3.4.0 from Canonical✓ installed

Edition du profil par défaut dans LXD pour lui permettre de prendre en charge Canonical Kubernetes avec ce modèle …

Install Canonical Kubernetes in LXD

name: default
description: Default LXD profile
config:
  boot.autostart: "true"
  linux.kernel_modules: ip_vs,ip_vs_rr,ip_vs_wrr,ip_vs_sh,ip_tables,ip6_tables,iptable_raw,netlink_diag,nf_nat,overlay,br_netfilter,xt_socket
  raw.lxc: |
    lxc.apparmor.profile=unconfined
    lxc.mount.auto=proc:rw sys:rw cgroup:rw
    lxc.cgroup.devices.allow=a
    lxc.cap.drop=
  security.nesting: "true"
  security.privileged: "true"
devices:
  aadisable2:
    path: /dev/kmsg
    source: /dev/kmsg
    type: unix-char
  dev-loop-control:
    major: "10"
    minor: "237"
    path: /dev/loop-control
    type: unix-char
  dev-loop0:
    major: "7"
    minor: "0"
    path: /dev/loop0
    type: unix-block
  dev-loop1:
    major: "7"
    minor: "1"
    path: /dev/loop1
    type: unix-block
  dev-loop2:
    major: "7"
    minor: "2"
    path: /dev/loop2
    type: unix-block
  dev-loop3:
    major: "7"
    minor: "3"
    path: /dev/loop3
    type: unix-block
  dev-loop4:
    major: "7"
    minor: "4"
    path: /dev/loop4
    type: unix-block
  dev-loop5:
    major: "7"
    minor: "5"
    path: /dev/loop5
    type: unix-block
  dev-loop6:
    major: "7"
    minor: "6"
    path: /dev/loop6
    type: unix-block
  dev-loop7:
    major: "7"
    minor: "7"
    path: /dev/loop7
    type: unix-block
  dev-loop8:
    major: "7"
    minor: "8"
    path: /dev/loop8
    type: unix-block
  dev-loop9:
    major: "7"
    minor: "9"
    path: /dev/loop9
    type: unix-block
  dev-loop10:
    major: "7"
    minor: "10"
    path: /dev/loop10
    type: unix-block
  dev-loop11:
    major: "7"
    minor: "11"
    path: /dev/loop11
    type: unix-block
  dev-loop12:
    major: "7"
    minor: "12"
    path: /dev/loop12
    type: unix-block
  dev-loop13:
    major: "7"
    minor: "13"
    path: /dev/loop13
    type: unix-block
  dev-loop14:
    major: "7"
    minor: "14"
    path: /dev/loop14
    type: unix-block
  dev-loop15:
    major: "7"
    minor: "15"
    path: /dev/loop15
    type: unix-block
  dev-loop16:
    major: "7"
    minor: "16"
    path: /dev/loop16
    type: unix-block
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
used_by: []

Application de ce profil par défaut dans LXD :

root@lxd:~# lxc profile edit default < lxd-profile-default.yaml

root@lxd:~# lxc profile show default
name: default
description: Default LXD profile
config:
  boot.autostart: "true"
  linux.kernel_modules: ip_vs,ip_vs_rr,ip_vs_wrr,ip_vs_sh,ip_tables,ip6_tables,iptable_raw,netlink_diag,nf_nat,overlay,br_netfilter,xt_socket
  raw.lxc: |
    lxc.apparmor.profile=unconfined
    lxc.mount.auto=proc:rw sys:rw cgroup:rw
    lxc.cgroup.devices.allow=a
    lxc.cap.drop=
  security.nesting: "true"
  security.privileged: "true"
devices:
  aadisable2:
    path: /dev/kmsg
    source: /dev/kmsg
    type: unix-char
  dev-loop-control:
    major: "10"
    minor: "237"
    path: /dev/loop-control
    type: unix-char
  dev-loop0:
    major: "7"
    minor: "0"
    path: /dev/loop0
    type: unix-block
  dev-loop1:
    major: "7"
    minor: "1"
    path: /dev/loop1
    type: unix-block
  dev-loop2:
    major: "7"
    minor: "2"
    path: /dev/loop2
    type: unix-block
  dev-loop3:
    major: "7"
    minor: "3"
    path: /dev/loop3
    type: unix-block
  dev-loop4:
    major: "7"
    minor: "4"
    path: /dev/loop4
    type: unix-block
  dev-loop5:
    major: "7"
    minor: "5"
    path: /dev/loop5
    type: unix-block
  dev-loop6:
    major: "7"
    minor: "6"
    path: /dev/loop6
    type: unix-block
  dev-loop7:
    major: "7"
    minor: "7"
    path: /dev/loop7
    type: unix-block
  dev-loop8:
    major: "7"
    minor: "8"
    path: /dev/loop8
    type: unix-block
  dev-loop9:
    major: "7"
    minor: "9"
    path: /dev/loop9
    type: unix-block
  dev-loop10:
    major: "7"
    minor: "10"
    path: /dev/loop10
    type: unix-block
  dev-loop11:
    major: "7"
    minor: "11"
    path: /dev/loop11
    type: unix-block
  dev-loop12:
    major: "7"
    minor: "12"
    path: /dev/loop12
    type: unix-block
  dev-loop13:
    major: "7"
    minor: "13"
    path: /dev/loop13
    type: unix-block
  dev-loop14:
    major: "7"
    minor: "14"
    path: /dev/loop14
    type: unix-block
  dev-loop15:
    major: "7"
    minor: "15"
    path: /dev/loop15
    type: unix-block
  dev-loop16:
    major: "7"
    minor: "16"
    path: /dev/loop16
    type: unix-block
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
used_by: []

Suivi du lancement d’un contrôleur local dans LXD pour Juju :

root@lxd:~# mkdir -p ~/.local/share/juju

root@lxd:~# juju bootstrap localhost lxd
Creating Juju controller "lxd" on localhost/localhost
Looking for packaged Juju agent version 3.4.0 for amd64
Located Juju agent version 3.4.0-ubuntu-amd64 at https://streams.canonical.com/juju/tools/agent/3.4.0/juju-3.4.0-linux-amd64.tgz
To configure your system to better support LXD containers, please see: https://documentation.ubuntu.com/lxd/en/latest/explanation/performance_tuning/
Launching controller instance(s) on localhost/localhost...
 - juju-b6d031-0 (arch=amd64)                 
Installing Juju agent on bootstrap instance
Waiting for address
Attempting to connect to 10.11.106.167:22
Connected to 10.11.106.167
Running machine configuration script...
Bootstrap agent now started
Contacting Juju controller at 10.11.106.167 to verify accessibility...

Bootstrap complete, controller "lxd" is now available
Controller machines are in the "controller" model

Now you can run
        juju add-model <model-name>
to create a new model to deploy workloads.

root@lxd:~# juju controllers
Use --refresh option with this command to see the latest information.

Controller Model User Access Cloud/Region Models Nodes HA Version
lxd* - admin superuser localhost/localhost 1 1 none 3.4.0  

root@lxd:~# lxc ls
+---------------+---------+----------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------+---------+----------------------+------+-----------+-----------+
| juju-b6d031-0 | RUNNING | 10.11.106.167 (eth0) | | CONTAINER | 0 |
+---------------+---------+----------------------+------+-----------+-----------+

Visualisation des Charms pour Canonical Kubernetes avant installation :

root@lxd:~# juju info k8s
name: k8s
publisher: Canonical Kubernetes
summary: A machine charm for K8s
description: "A machine charm which operates a complete Kubernetes cluster.\n\nThis
  charm installs and operates a Kubernetes cluster via the k8s snap. It exposes\nrelations
  to co-operate with other kubernetes components such as optional CNIs, \noptional
  cloud-providers, optional schedulers, external backing stores, and external\ncertificate
  storage.\n\nThis charm provides the following running components:\n* kube-apiserver\n*
  kube-scheduler\n* kube-controller-manager\n* kube-proxy\n* kubelet\n* containerd\n\nThis
  charm can optionally disable the following components:\n* A Kubernetes Backing Store\n*
  A Kubernetes CNI\n"
store-url: https://charmhub.io/k8s
charm-id: 5GG4ydqob6yLrcZ3rrscqL0gVJvK53Ij
supports: ubuntu@20.04, ubuntu@22.04
subordinate: false
relations:
  provides:
    cos-agent: cos_agent
    cos-worker-tokens: cos-tokens
    k8s-cluster: k8s-cluster
  requires: {}
channels: |
  latest/stable: –
  latest/candidate: –
  latest/beta: –
  latest/edge: 26 2024-03-29 (26) 3MB amd64 ubuntu@20.04, ubuntu@22.04

root@lxd:~# juju info k8s-worker
name: k8s-worker
publisher: Canonical Kubernetes
summary: A machine charm for a K8s Worker
description: |
  A machine charm which operates a Kubernetes worker.

  This charm installs and operates a Kubernetes worker via the k8s snap. It exposes
  relations to co-operate with other kubernetes components

  This charm provides the following running components:
  * kube-proxy
  * kubelet
  * containerd
store-url: https://charmhub.io/k8s-worker
charm-id: ZL6PvT8yleZOMih0oKRYPgQI4FntLB48
supports: ubuntu@20.04, ubuntu@22.04
subordinate: false
relations:
  provides:
    cos-agent: cos_agent
  requires:
    cluster: k8s-cluster
    cos-tokens: cos-tokens
channels: |
  latest/stable: –
  latest/candidate: –
  latest/beta: –
  latest/edge: 27 2024-03-29 (27) 6MB amd64 ubuntu@20.04, ubuntu@22.04

qui peuvent être installés avec Juju, avec ici le noeud contrôleur du futur cluster Kubernetes …

Install Canonical Kubernetes from a charm

root@lxd:~# juju add-model k8s
Added 'k8s' model on localhost/localhost with credential 'localhost' for user 'admin'
root@lxd:~# juju deploy k8s --channel=latest/edge
Deployed "k8s" from charm-hub charm "k8s", revision 26 in channel latest/edge on ubuntu@22.04/stable

root@lxd:~# juju status
Model Controller Cloud/Region Version SLA Timestamp
k8s lxd localhost/localhost 3.4.0 unsupported 12:37:00Z

App Version Status Scale Charm Channel Rev Exposed Message
k8s waiting 1 k8s latest/edge 26 no Cluster not yet ready

Unit Workload Agent Machine Public address Ports Message
k8s/0* waiting idle 0 10.11.106.116 Cluster not yet ready

Machine State Address Inst id Base AZ Message
0 started 10.11.106.116 juju-85fd7d-0 ubuntu@22.04 Running

Déploiement de deux noeuds Workers :

root@lxd:~# juju deploy k8s-worker --channel=latest/edge -n 2
Deployed "k8s-worker" from charm-hub charm "k8s-worker", revision 27 in channel latest/edge on ubuntu@22.04/stable
root@lxd:~# juju integrate k8s k8s-worker:cluster

Le cluster Kubernetes est alors actif et opérationnel :

root@lxd:~# juju status
Model Controller Cloud/Region Version SLA Timestamp
k8s lxd localhost/localhost 3.4.0 unsupported 12:39:53Z

App Version Status Scale Charm Channel Rev Exposed Message
k8s 1.29.3 active 1 k8s latest/edge 26 no Ready
k8s-worker 1.29.3 active 2 k8s-worker latest/edge 27 no Ready

Unit Workload Agent Machine Public address Ports Message
k8s-worker/0* active idle 1 10.11.106.20 Ready
k8s-worker/1 active idle 2 10.11.106.178 Ready
k8s/0* active idle 0 10.11.106.116 6443/tcp Ready

Machine State Address Inst id Base AZ Message
0 started 10.11.106.116 juju-85fd7d-0 ubuntu@22.04 Running
1 started 10.11.106.20 juju-85fd7d-1 ubuntu@22.04 Running
2 started 10.11.106.178 juju-85fd7d-2 ubuntu@22.04 Running

Connexion au noeud Contrôleur du cluster pour vérifier son statut :

root@lxd:~# juju ssh k8s/0

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@juju-85fd7d-0:~$ sudo k8s status
status: ready
high-availability: no
datastore:
  voter-nodes:
    - 10.11.106.116:6400
  standby-nodes: none
  spare-nodes: none

network:
  enabled: true
dns:
  enabled: true
  cluster-domain: cluster.local
  service-ip: 10.152.183.28
  upstream-nameservers:
  - /etc/resolv.conf
metrics-server:
  enabled: true

ubuntu@juju-85fd7d-0:~$ sudo k8s kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/coredns:udp-53/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
ubuntu@juju-85fd7d-0:~$ sudo k8s kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
juju-85fd7d-0 Ready control-plane,worker 7m10s v1.29.3 10.11.106.116 <none> Ubuntu 22.04.4 LTS 5.15.0-67-generic containerd://1.6.28
juju-85fd7d-1 Ready worker 3m23s v1.29.3 10.11.106.20 <none> Ubuntu 22.04.4 LTS 5.15.0-67-generic containerd://1.6.28
juju-85fd7d-2 Ready worker 3m10s v1.29.3 10.11.106.178 <none> Ubuntu 22.04.4 LTS 5.15.0-67-generic containerd://1.6.28
ubuntu@juju-85fd7d-0:~$ sudo k8s kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/cilium-b5hkt 1/1 Running 0 7m13s
kube-system pod/cilium-gghfz 1/1 Running 0 3m17s
kube-system pod/cilium-kzwpz 1/1 Running 0 3m30s
kube-system pod/cilium-operator-597b55b886-jhqkq 1/1 Running 0 7m13s
kube-system pod/coredns-66579b5b88-x9dv2 1/1 Running 0 7m13s
kube-system pod/metrics-server-57db9dfb7b-lnllv 1/1 Running 0 7m13s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 7m20s
kube-system service/coredns ClusterIP 10.152.183.28 <none> 53/UDP,53/TCP 7m21s
kube-system service/hubble-peer ClusterIP 10.152.183.184 <none> 443/TCP 7m21s
kube-system service/metrics-server ClusterIP 10.152.183.60 <none> 443/TCP 7m20s

Je peux récupérer le fichierkubeconfigavec Juju :

root@lxd:~# juju ssh k8s/0 'sudo k8s config'
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURHekNDQWdPZ0F3SUJBZ0lRQ2RCNysvUWI1Ums4MzdmRGJhWlp2ekFOQmdrcWhraUc5dzBCQVFzRkFEQVkKTVJZd0ZBWURWUVFERXcxcmRXSmxjbTVsZEdWekxXTmhNQjRYRFRJME1ETXpNVEV5TXpVeE1Gb1hEVFEwTURNegpNVEV5TXpVeE1Gb3dHREVXTUJRR0ExVUVBeE1OYTNWaVpYSnVaWFJsY3kxallUQ0NBU0l3RFFZSktvWklodmNOCkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFNK1VRNllEVS9vaEVrWjhGcHlnQ0Y3N1JHaWdEMUdEZUR5ekdCeTIKZjhleVhZMGRUWlFJcmw0amtIL29SS2tiRnB4QnZVSVlzWFV5RlpkOUxKM3VFMWhPQTNDNXYzM0VOUVhrdU5pNQpELy9kRHNyZ0pHWitzTnJKSTIzSWZOV1VXanNLc1hzejFzZXN5cHRYb2dYbytkTkxxRitVTjFnUjVBOTZ6VVB0ClZOZFJLKzUyaTFxcWcxeUFhczcybVhwVkNQZUxxSi9adXhDSFhTcE8rcWt0Q2pSeUVTKzJmVFVTaUhtRlQ5clMKVG9vZndTZ0RkWHJlMmsvN3NFSjg5aGk2NFpQeVQyZDFPYzcxbzEyQUlyZ0VBYlorZnJ0NnFCSVhENkkxSEJzKwphZlh1UmVmTyswUEFVYzgxN25QaGhTWW1OcGJrdk5vdFF4WlRZRGRONEhEbEJUVUNBd0VBQWFOaE1GOHdEZ1lEClZSMFBBUUgvQkFRREFnS0VNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01DQmdnckJnRUZCUWNEQVRBUEJnTlYKSFJNQkFmOEVCVEFEQVFIL01CMEdBMVVkRGdRV0JCUnFobUlNNzlRVnFYMks1YXVuVUdBMENYOTBkVEFOQmdrcQpoa2lHOXcwQkFRc0ZBQU9DQVFFQWNnaUJad0R3MTlVVlFaQVpHRCs5THFSOFBycXJyMlRqVFh0Q1RkbzFGWWZICnlla2IzdmV1YjJUblNOT2RUeEhjWmtlR2d3QXFYV216Z1MrSHl5TFZSTlY2WTNJK0ZKQmJpVkJJOTV0TlRmaG4KMEZvSEVzWUNSY3Q5L1JVQkVOTTFsZDY1bWFVNFc2TWFvdE5MOVpSczNGTDJ3SmRKTjdCVEJIamxYZzhrckkrOQpOdlcvTWZCS0FvcWRleXNBMHJHY3RlczVWZm1YUDRMa1hxNXhQNmV1MWhmSU1ObXM1UTk4QTdsck9TZG5BNzJYClFNKzBBc1ovckhZQ0hnRUFqNjdIK0drTFhQeEFzelZ4UTZpRHRJMmhyWVVKYmNFSTg1REF0RFNhc2xGS2JHLzMKNU53b1U0VTZjVXMxcHZiTjFsVFZvTGVFTHFoYU9EMkdiUnRYb0h4UnFBPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    server: https://10.11.106.116:6443
  name: k8s
contexts:
- context:
    cluster: k8s
    user: k8s-user
  name: k8s
current-context: k8s
kind: Config
preferences: {}
users:
- name: k8s-user
  user:
    token: token::00669e4efab3674984256068943beee135d06d1a

Connection to 10.11.106.116 closed.

Pour bénéficier localement du client kubectl :

root@lxd:~# juju expose k8s
root@lxd:~# snap install kubectl --classic
snap "kubectl" is already installed, see 'snap help refresh'
root@lxd:~# mkdir .kube
root@lxd:~# juju ssh k8s/0 'sudo k8s config' > .kube/config
Connection to 10.11.106.116 closed.
root@lxd:~# chmod 400 .kube/config
root@lxd:~# kubectl cluster-info
Kubernetes control plane is running at https://10.11.106.116:6443
CoreDNS is running at https://10.11.106.116:6443/api/v1/namespaces/kube-system/services/coredns:udp-53/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root@lxd:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
juju-85fd7d-0 Ready control-plane,worker 11m v1.29.3 10.11.106.116 <none> Ubuntu 22.04.4 LTS 5.15.0-67-generic containerd://1.6.28
juju-85fd7d-1 Ready worker 7m47s v1.29.3 10.11.106.20 <none> Ubuntu 22.04.4 LTS 5.15.0-67-generic containerd://1.6.28
juju-85fd7d-2 Ready worker 7m34s v1.29.3 10.11.106.178 <none> Ubuntu 22.04.4 LTS 5.15.0-67-generic containerd://1.6.28

root@lxd:~# lxc ls
+---------------+---------+--------------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------+---------+--------------------------+------+-----------+-----------+
| juju-85fd7d-0 | RUNNING | 10.11.106.116 (eth0) | | CONTAINER | 0 |
| | | 10.1.0.231 (cilium_host) | | | |
+---------------+---------+--------------------------+------+-----------+-----------+
| juju-85fd7d-1 | RUNNING | 10.11.106.20 (eth0) | | CONTAINER | 0 |
| | | 10.1.1.16 (cilium_host) | | | |
+---------------+---------+--------------------------+------+-----------+-----------+
| juju-85fd7d-2 | RUNNING | 10.11.106.178 (eth0) | | CONTAINER | 0 |
| | | 10.1.2.20 (cilium_host) | | | |
+---------------+---------+--------------------------+------+-----------+-----------+
| juju-b6d031-0 | RUNNING | 10.11.106.167 (eth0) | | CONTAINER | 0 |
+---------------+---------+--------------------------+------+-----------+-----------+

Comme dans l’article précédent, je peux bénéficier d’Ollama et des LLM ouverts via Helm :

root@lxd:~# curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
Downloading https://get.helm.sh/helm-v3.14.3-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm

root@lxd:~# helm ls
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION

root@lxd:~# kubectl create namespace ollama
namespace/ollama created

root@lxd:~# helm install ollama ollama-helm/ollama --namespace ollama
NAME: ollama
LAST DEPLOYED:
NAMESPACE: ollama
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace ollama -l "app.kubernetes.io/name=ollama,app.kubernetes.io/instance=ollama" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace ollama $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace ollama port-forward $POD_NAME 8080:$CONTAINER_PORT

root@lxd:~# helm ls --namespace ollama
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
ollama ollama 1 2024-03-31 13:11:43.199405708 +0000 UTC deployed ollama-0.21.1 0.1.30    

root@lxd:~# kubectl get po,svc -n ollama
NAME READY STATUS RESTARTS AGE
pod/ollama-75c7b79658-rnf7h 1/1 Running 0 94s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ollama ClusterIP 10.152.183.78 <none> 11434/TCP 94s

Et je peux exposer localement l’API RESTful d’Ollama :

root@lxd:~# kubectl port-forward service/ollama -n ollama 11434:11434 --address='0.0.0.0'
Forwarding from 0.0.0.0:11434 -> 11434

root@lxd:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name    
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1399/sshd: /usr/sbi 
tcp 0 0 10.11.106.1:53 0.0.0.0:* LISTEN 9929/dnsmasq        
tcp 0 0 0.0.0.0:11434 0.0.0.0:* LISTEN 52634/kubectl       
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 623/systemd-resolve 
tcp6 0 0 :::8443 :::* LISTEN 9784/lxd            
tcp6 0 0 :::22 :::* LISTEN 1399/sshd: /usr/sbi 
udp 0 0 10.11.106.1:53 0.0.0.0:* 9929/dnsmasq        
udp 0 0 127.0.0.53:53 0.0.0.0:* 623/systemd-resolve 
udp 0 0 0.0.0.0:67 0.0.0.0:* 9929/dnsmasq

Récupération du client Ollama via GitHub :

Releases · ollama/ollama

root@lxd:~# wget -c https://github.com/ollama/ollama/releases/download/v0.1.30/ollama-linux-amd64 && chmod +x ollama-linux-amd64 && mv ollama-linux-amd64 /usr/local/bin/ollama
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 297108760 (283M) [application/octet-stream]
Saving to: ‘ollama-linux-amd64’

ollama-linux-amd64 100%[=====================================================================================================>] 283.34M 351MB/s in 0.8s    

(351 MB/s) - ‘ollama-linux-amd64’ saved [297108760/297108760]

Pour exécuter par exemple ici Code Llama, une famille de LLM pour le code basée sur Llama 2 :

root@lxd:~# ollama run codellama
pulling manifest 
pulling 3a43f93b78ec... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 3.8 GB                         
pulling 8c17c2ebb0ea... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 7.0 KB                         
pulling 590d74a5569b... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 4.8 KB                         
pulling 2e0493f67d0c... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 59 B                         
pulling 7f6a57943a88... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 120 B                         
pulling 316526ac7323... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 529 B                         
verifying sha256 digest 
writing manifest 
removing any unused layers 
success

Avec ces petits exemples à parfaire …

>>> How deploy manifest with Juju inside Kubernetes cluster ?

To deploy a manifest file with Juju inside a Kubernetes cluster, you can follow these steps:

1. Install Juju on your local machine and connect it to the Kubernetes cluster that you want to deploy the manifest file to.
2. Create a Juju model for the Kubernetes cluster by running the following command:

juju add-model <cluster-name>

Replace `<cluster-name>` with the name of your Kubernetes cluster.
3. Set the deployment environment to the Kubernetes cluster that you want to deploy the manifest file to by running the following command:

juju set-environment <cluster-name>

4. Create a new Juju application using the `juju add` command, passing in the name of the application and the path to the manifest file as arguments. For example:

juju add my-application manifests/my-application.yaml

Replace `<my-application>` with the name of your application and `<manifests/my-application.yaml>` with the path to your manifest file.
5. Deploy the application by running the following command:

juju deploy my-application

This will deploy the application using the configuration specified in the manifest file.
6. Once the deployment is complete, you can manage the application using standard Juju commands, such as `juju status` to view the status of the application and `juju logs` to view the
logs for the application.
7. To remove the application from the Kubernetes cluster, you can use the `juju destroy` command:

juju destroy my-application

This will destroy the application and all its resources in the Kubernetes cluster.

Note that this is a basic example of how to deploy a manifest file with Juju inside a Kubernetes cluster. There are many other options and features available in Juju, so you can 
explore those as well.

>>> How deploy a Pod with Rust inside Kubernetes ?

To deploy a Pod with Rust inside a Kubernetes cluster, you can follow these steps:

1. Install the `cargo` package on your local machine by running the following command:

sudo apt-get install cargo

2. Create a new Rust project using the `cargo new` command, passing in the name of your project as an argument. For example:

cargo new my-rust-project

3. Change into the directory of your new project by running the following command:

cd my-rust-project

4. Add the `kubernetes` crate to your projects dependencies by adding the following line to your `Cargo.toml` file:

[dependencies]
kubernetes = "0.18"

5. Create a new Rust file for your Kubernetes deployment, such as `deployment.rs`, and add the following code to it:

use kubernetes::api::Api;
use kubernetes::config::Configuration;
use kubernetes::apis::CoreApi;

fn main() {
    let config = Configuration::new("https://<your-kubernetes-cluster>");
    let client = Api::all(config);
    let pod = client.create_namespaced_pod(
        "default",
        &Api::Pod("my-rust-deployment"),
        None,
    );
}

Replace `<your-kubernetes-cluster>` with the URL of your Kubernetes cluster.
6. Compile your Rust code by running the following command:

cargo build --release
7. Run your Rust code by running the following command:

./target/release/deployment

This will deploy a new Pod with Rust inside the `default` namespace of your Kubernetes cluster. You can view the status of your Pod using the `kubectl get pods` command.

Note that this is a basic example of how to deploy a Pod with Rust inside a Kubernetes cluster. There are many other options and features available in Rust, so you can explore those as
well

>>> Send a message (/? for help)

Canonical Kubernetes peut également être installé à l’intérieur d’un conteneur LXD en utilisant le profil énoncé çi-dessus comme le précise cette documentation détaillée …

Install Canonical Kubernetes in LXD

Sans compter qu’Ollama et une interface graphique conviviale avec Open WebUI pourraient être installé via Snap ou Helm :

À suivre !

k8s-snap (Canonical Kubernetes) pour un déploiement simple et rapide d’un cluster k8s …

Karim — Sun, 25 Feb 2024 21:45:59 +0000

Focus sur Canonical Kubernetes, une distribution Kubernetes minimaliste (à l’image de k0s/k3s) livrée en quelques clics via Snap. L’accent est notamment mis sur des opérations simplifiées et une posture de sécurité renforcée sur n’importe quelle infrastructure.

Pour cela, déploiement sur une instance Ubuntu 22.04 LTS dans OVH qui autorise la virtualisation imbriquée :

Je commence par y installer Multipass pour lancer un groupe de trois machines virtuelles :

Multipass orchestrates virtual Ubuntu instances

Dès lors lancement des trois VMs Ubuntu 22.04 LTS pour initier l’ossature du futur cluster k8s :

Dans la première machine virtuelle qui va supporter la plan de contrôle, installation de k8s-snap rapidement via Snap :

Le cluster k8s est initialisé et prêt pour rattacher un noeud Worker :

Pour cela génération d’un token :

Dès lors, il est possible de rattacher le nouveau noeud Worker au cluster via ce token généré précedemment :

Rattachement d’un autre noeud Worker avec ce même token :

Les noeuds Worker sont alors opérationnels dans le cluster :

Il est possible de récupérer le fichier kubeconfig pour pouvoir utiliser le client kubectl :

Lancement d’un Pod Ubuntu très rapidement :

qui va servir ici au lancement d’Ollama :

GitHub - ollama/ollama: Get up and running with Llama 2, Mistral, Gemma, and other large language models.

Ollama est une plateforme qui permet aux utilisateurs d’exécuter de grands modèles de langages (LLM), tels que Mistral, Gemma, Llama 2 de Research Lab et d’autres … , directement sur leurs ordinateurs personnels plutôt que de dépendre uniquement de services en nuage. Il permet de télécharger des modèles pré-entraînés compatibles avec différents systèmes d’exploitation, notamment macOS, Linux et des versions préliminaires pour Windows. Les utilisateurs peuvent également personnaliser et développer leurs propres applications en utilisant ces modèles.

Ici sans l’utilisation d’une carte GPU et en mode CPU uniquement via de petits LLMs comme le très récent Gemma :

avec une petite interrogation :

La même chose avec un autre LLM :

avec une faible consommation de ressources …

Canonical Kubernetes est donc un moyen très rapide et très simple de déployer un cluster Kubernetes entièrement conforme.

Cette distribution ajoute des pièces manquantes (par exemple, Ingress, DNS, Réseau) pour une expérience “Zéro-op”. Pour les clusters à un seul nœud, elle peut être déployée avec deux commandes. Et Ajouter de nouveaux nœuds avec seulement deux autres commandes …

À suivre !

Déployer un cluster Nomad et Consul très rapidement avec hashi-up …

Karim — Sat, 11 Mar 2023 17:16:05 +0000

Hashi-up est un utilitaire léger qui permet d’installer HashiCorp Consul, Nomad ou Vault sur n’importe quel hôte Linux distant.

Load Balancing avec HAProxy, Nomad et Consul …

Tout ce dont vous avez besoin est un accès ssh et le binaire hashi-up pour construire son cluster.

GitHub - jsiebens/hashi-up: bootstrap HashiCorp Consul, Nomad, or Vault over SSH < 1 minute

L’outil est écrit en Go et est compilé pour Linux, Windows, MacOS et même pour Raspberry Pi.

Building a Nomad cluster on Raspberry Pi running Ubuntu server

Ce projet est fortement inspiré du travail d’Alex Ellis qui a créé k3sup, un outil pour passer de zéro à KUBECONFIG avec k3s.

Implémentation avec le lancement d’un serveur Ubuntu 22.04 LTS autorisant la virtualisation imbriquée dans DigitalOcean :

curl -X POST -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer '$TOKEN'' \
    -d '{"name":"minione",
        "size":"m-4vcpu-32gb",
        "region":"fra1",
        "image":"ubuntu-22-04-x64",
        "monitoring":true,
        "vpc_uuid":"8b8c0544-e7b6-4d0a-977d-4406ea518f7a"}' \
    "https://api.digitalocean.com/v2/droplets"

Je vais y installer OpenNebula via le script miniONE :

root@minione:~# wget -c https://raw.githubusercontent.com/OpenNebula/minione/master/minione
--2023-03-11 14:21:46-- https://raw.githubusercontent.com/OpenNebula/minione/master/minione
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 51458 (50K) [text/plain]
Saving to: ‘minione’

minione 100%[=============================================================================================>] 50.25K --.-KB/s in 0s      

2023-03-11 14:21:46 (149 MB/s) - ‘minione’ saved [51458/51458]

Déploiement automatisé très rapidement avec lancement du script :

root@minione:~# bash minione

### Checks & detection
Checking augeas is installed SKIP will try to install
Checking AppArmor SKIP will try to modify
Checking for present ssh key SKIP
Checking (iptables|netfilter)-persistent are installed SKIP will try to install
Checking docker is installed SKIP will try to install
Checking python3-pip is installed SKIP will try to install
Checking ansible SKIP will try to install
Checking terraform SKIP will try to install
Checking unzip is installed SKIP will try to install

### Main deployment steps:
Install OpenNebula frontend version 6.6
Install Terraform
Install Docker
Configure bridge minionebr with IP 172.16.100.1/24
Enable NAT over eth0
Modify AppArmor
Install OpenNebula KVM node
Export appliance and update VM template
Install augeas-tools iptables-persistent netfilter-persistent python3-pip unzip
Install pip 'ansible==2.9.9'

Do you agree? [yes/no]:
yes

### Installation
Updating APT cache OK
Install augeas-tools iptables-persistent netfilter-persistent python3-pip unzip OK
Updating PIP OK
Install from PyPI 'ansible==2.9.9' OK
Creating bridge interface minionebr OK
Bring bridge interfaces up OK
Enabling IPv4 forward OK
Persisting IPv4 forward OK
Configuring NAT using iptables OK
Saving iptables changes OK
Installing DNSMasq OK
Starting DNSMasq OK
Configuring repositories OK
Updating APT cache OK
Installing OpenNebula packages OK
Installing opennebula-provision package OK
Installing TerraForm OK
Create docker packages repository OK
Install docker OK
Start docker service OK
Enable docker service OK
Installing OpenNebula kvm node packages OK
Updating AppArmor OK
Disable default libvirtd networking OK
Restart libvirtd OK

### Configuration
Generating ssh keypair in /root/.ssh-oneprovision/id_rsa OK
Add oneadmin to docker group OK
Update network hooks OK
Switching OneGate endpoint in oned.conf OK
Switching OneGate endpoint in onegate-server.conf OK
Switching keep_empty_bridge on in OpenNebulaNetwork.conf OK
Switching scheduler interval in oned.conf OK
Setting initial password for current user and oneadmin OK
Changing WebUI to listen on port 80 OK
Switching FireEdge public endpoint OK
Starting OpenNebula services OK
Enabling OpenNebula services OK
Add ssh key to oneadmin user OK
Update ssh configs to allow VM addresses reusing OK
Ensure own hostname is resolvable OK
Checking OpenNebula is working OK
Disabling ssh from virtual network OK
Adding localhost ssh key to known_hosts OK
Testing ssh connection to localhost OK
Updating datastores template OK
Creating KVM host OK
Restarting OpenNebula OK
Creating virtual network OK
Exporting [Alpine Linux 3.17] from Marketplace to local datastore OK
Waiting until the image is ready OK
Updating VM template OK

### Report
OpenNebula 6.6 was installed
Sunstone is running on:
  http://164.90.215.231/
FireEdge is running on:
  http://164.90.215.231:2616/
Use following to login:
  user: oneadmin
  password: SyEnJiYuWC

chargement d’une image Ubuntu 22.04 LTS pour création de plusieures machines virtuelles :

Lancement d’une première machine virtuelle sous Ubuntu :

et installation de hashi-up :

root@minione:~# curl -sLS https://get.hashi-up.dev | sh

Downloading package https://github.com/jsiebens/hashi-up/releases/download/v0.16.0/hashi-up as /tmp/hashi-up
Download complete.

Running with sufficient permissions to attempt to move hashi-up to /usr/local/bin
New version of hashi-up installed to /usr/local/bin
Version: 0.16.0
Git Commit: b062f5d

root@minione:~# hashi-up version
Version: 0.16.0
Git Commit: b062f5d

pour la mise en place d’un serveur Consul :

root@minione:~# export IP=172.16.100.2

root@minione:~# hashi-up consul install --ssh-target-addr $IP --ssh-target-user root --server --client-addr 0.0.0.0 --ssh-target-key .ssh/id_rsa --version 1.15.1

[INFO] Uploading generated Consul configuration ...
[INFO] Installing Consul ...
[INFO] -> User 'consul' already exists, will not create again
[INFO] -> Copying configuration files
[INFO] -> Downloading consul_1.15.1_linux_amd64.zip
[INFO] -> Downloading consul_1.15.1_SHA256SUMS
[INFO] -> Verifying downloaded consul_1.15.1_linux_amd64.zip
consul_1.15.1_linux_amd64.zip: OK
[INFO] -> Unpacking consul_1.15.1_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/consul.service
[INFO] -> Enabling systemd service
[INFO] -> Starting systemd service
[INFO] Done.

J’utilise “le VPN du pauvre” avec sshuttle pour me connecter au dashboard du serveur Consul :

Usage - sshuttle 1.1.1 documentation

$ sshuttle --dns -NHr root@164.90.215.231 0.0.0.0/0  ✔  base   15:45:09  
c : Connected to server.

Ce qui peut me permettre d’atteindre le serveur Consul directement …

Je peux y adjoindre un serveur Nomad avec hashi-up :

Install | Nomad | HashiCorp Developer

root@minione:~# hashi-up nomad install --ssh-target-addr $IP --ssh-target-user root --server --ssh-target-key .ssh/id_rsa --version 1.5.0

[INFO] Uploading generated Nomad configuration ...
[INFO] Installing Nomad ...
[INFO] -> Copying configuration files
[INFO] -> Downloading nomad_1.5.0_linux_amd64.zip
[INFO] -> Downloading nomad_1.5.0_SHA256SUMS
[INFO] -> Verifying downloaded nomad_1.5.0_linux_amd64.zip
nomad_1.5.0_linux_amd64.zip: OK
[INFO] -> Unpacking nomad_1.5.0_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/nomad.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/nomad.service → /etc/systemd/system/nomad.service.
[INFO] -> Starting systemd service
[INFO] Done.

Une interface graphique est également accessible pour Nomad à cette étape …

Lancement de la partie cliente de Consul et Nomad avec trois autres machines virtuelles sous Ubuntu 22.04 LTS dans OpenNebula :

Personnalisation de ces trois machines virtuelles avec Pyinfra, une autre alternative sous Python à Ansible avec pipx :

pyinfra

root@minione:~# pipx install pyinfra

  installed package pyinfra 2.6.2, installed using Python 3.10.6
  These apps are now globally available
    - pyinfra
⚠️ Note: '/root/.local/bin' is not on your PATH environment variable. These apps will not be globally accessible until your PATH is updated. Run `pipx ensurepath` to
    automatically add it, or manually modify your PATH in your shell config file (i.e. ~/.bashrc).
done! ✨ 🌟 ✨

root@minione:~# pipx ensurepath

Success! Added /root/.local/bin to the PATH environment variable.

Consider adding shell completions for pipx. Run 'pipx completions' for instructions.

You will need to open a new terminal or re-login for the PATH changes to take effect.

Otherwise pipx is ready to go! ✨ 🌟 ✨

root@minione:~# source .bashrc

root@minione:~# pyinfra --help

Usage: pyinfra [OPTIONS] INVENTORY OPERATIONS...

  pyinfra manages the state of one or more servers. It can be used for
  app/service deployment, config management and ad-hoc command execution.

  Documentation: pyinfra.readthedocs.io

  # INVENTORY

  + a file (inventory.py)
  + hostname (host.net)
  + Comma separated hostnames:
    host-1.net,host-2.net,@local

  # OPERATIONS

  # Run one or more deploys against the inventory
  pyinfra INVENTORY deploy_web.py [deploy_db.py]...

  # Run a single operation against the inventory
  pyinfra INVENTORY server.user pyinfra home=/home/pyinfra

  # Execute an arbitrary command against the inventory
  pyinfra INVENTORY exec -- echo "hello world"

  # Run one or more facts against the inventory
  pyinfra INVENTORY fact server.LinuxName [server.Users]...
  pyinfra INVENTORY fact files.File path=/path/to/file...

  # Debug the inventory hosts and data
  pyinfra INVENTORY debug-inventory

Options:
  -v Print meta (-v), input (-vv) and output
                                  (-vvv).
  --dry Don't execute operations on the target
                                  hosts.
  --limit TEXT Restrict the target hosts by name and group
                                  name.
  --fail-percent INTEGER % of hosts that need to fail before exiting
                                  early.
  --data TEXT Override data values, format key=value.
  --group-data TEXT Paths to load additional group data from
                                  (overrides matching keys).
  --config TEXT Specify config file to use (default:
                                  config.py).
  --chdir TEXT Set the working directory before executing.
  --sudo Whether to execute operations with sudo.
  --sudo-user TEXT Which user to sudo when sudoing.
  --use-sudo-password Whether to use a password with sudo.
  --su-user TEXT Which user to su to.
  --shell-executable TEXT Shell to use (ex: "sh", "cmd", "ps").
  --parallel INTEGER Number of operations to run in parallel.
  --no-wait Don't wait between operations for hosts.
  --serial Run operations in serial, host by host.
  --ssh-user, --user TEXT SSH user to connect as.
  --ssh-port, --port INTEGER SSH port to connect to.
  --ssh-key, --key PATH SSH Private key filename.
  --ssh-key-password, --key-password TEXT
                                  SSH Private key password.
  --ssh-password, --password TEXT
                                  SSH password.
  --winrm-username TEXT WINRM user to connect as.
  --winrm-password TEXT WINRM password.
  --winrm-port TEXT WINRM port to connect to.
  --winrm-transport TEXT WINRM transport for use.
  --support Print useful information for support and
                                  exit.
  --quiet Hide most pyinfra output.
  --debug Print debug info.
  --debug-facts Print facts after generating operations and
                                  exit.
  --debug-operations Print operations after generating and exit.
  --version Show the version and exit.
  --help Show this message and exit.

root@minione:~# pyinfra 172.16.100.4 exec -- hostnamectl set-hostname client0

--> Loading config...

--> Loading inventory...

--> Connecting to hosts...
    No host key for 172.16.100.4 found in known_hosts
    [172.16.100.4] Connected
    [172.16.100.4] Ready: shell

--> Proposed changes:
    Ungrouped:
    [172.16.100.4] Operations: 1 Change: 1 No change: 0   

--> Beginning operation run...
--> Starting operation: Server/Shell (hostnamectl set-hostname client0)
    [172.16.100.4] Success

--> Results:
    Ungrouped:
    [172.16.100.4] Changed: 1 No change: 0 Errors: 0   


root@minione:~# pyinfra 172.16.100.3 exec -- hostnamectl set-hostname client1

--> Loading config...

--> Loading inventory...

--> Connecting to hosts...
    No host key for 172.16.100.3 found in known_hosts
    [172.16.100.3] Connected
    [172.16.100.3] Ready: shell

--> Proposed changes:
    Ungrouped:
    [172.16.100.3] Operations: 1 Change: 1 No change: 0   

--> Beginning operation run...
--> Starting operation: Server/Shell (hostnamectl set-hostname client1)
    [172.16.100.3] Success

--> Results:
    Ungrouped:
    [172.16.100.3] Changed: 1 No change: 0 Errors: 0  

root@minione:~# pyinfra 172.16.100.5 exec -- hostnamectl set-hostname client2

--> Loading config...

--> Loading inventory...

--> Connecting to hosts...
    No host key for 172.16.100.5 found in known_hosts
    [172.16.100.5] Connected
    [172.16.100.5] Ready: shell

--> Proposed changes:
    Ungrouped:
    [172.16.100.5] Operations: 1 Change: 1 No change: 0   

--> Beginning operation run...
--> Starting operation: Server/Shell (hostnamectl set-hostname client2)
    [172.16.100.5] Success

--> Results:
    Ungrouped:
    [172.16.100.5] Changed: 1 No change: 0 Errors: 0

Par ce biais, insertion du moteur Docker dans ces machines virtuelles :

root@minione:~# cat inventory.py 

my_hosts = ["172.16.100.4", "172.16.100.3", "172.16.100.5"]

root@minione:~# pyinfra inventory.py exec -- "curl -fsSL https://get.docker.com | sh -"

--> Loading config...

--> Loading inventory...

--> Connecting to hosts...
    No host key for 172.16.100.5 found in known_hosts
    No host key for 172.16.100.3 found in known_hosts
    [172.16.100.5] Connected
    No host key for 172.16.100.4 found in known_hosts
    [172.16.100.4] Connected
    [172.16.100.3] Connected
    [172.16.100.4] Ready: shell
    [172.16.100.3] Ready: shell
    [172.16.100.5] Ready: shell

--> Proposed changes:
    Groups: inventory / my_hosts
    [172.16.100.4] Operations: 1 Change: 1 No change: 0   
    [172.16.100.3] Operations: 1 Change: 1 No change: 0   
    [172.16.100.5] Operations: 1 Change: 1 No change: 0   

--> Beginning operation run...
--> Starting operation: Server/Shell (curl -fsSL https://get.docker.com | sh -)
[172.16.100.5] # Executing docker install script, commit: 66474034547a96caa0a25be56051ff8b726a1b28
[172.16.100.5] + sh -c apt-get update -qq >/dev/null
[172.16.100.4] # Executing docker install script, commit: 66474034547a96caa0a25be56051ff8b726a1b28
[172.16.100.4] + sh -c apt-get update -qq >/dev/null
[172.16.100.3] # Executing docker install script, commit: 66474034547a96caa0a25be56051ff8b726a1b28
[172.16.100.3] + sh -c apt-get update -qq >/dev/null
[172.16.100.4] + sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
[172.16.100.5] + sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
[172.16.100.3] + sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
[172.16.100.5] + sh -c mkdir -p /etc/apt/keyrings && chmod -R 0755 /etc/apt/keyrings
[172.16.100.5] + sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
[172.16.100.5] + sh -c chmod a+r /etc/apt/keyrings/docker.gpg
[172.16.100.5] + sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable" > /etc/apt/sources.list.d/docker.list
[172.16.100.5] + sh -c apt-get update -qq >/dev/null
[172.16.100.4] + sh -c mkdir -p /etc/apt/keyrings && chmod -R 0755 /etc/apt/keyrings
[172.16.100.3] + sh -c mkdir -p /etc/apt/keyrings && chmod -R 0755 /etc/apt/keyrings
[172.16.100.4] + sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
[172.16.100.3] + sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
[172.16.100.3] + sh -c chmod a+r /etc/apt/keyrings/docker.gpg
[172.16.100.3] + sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable" > /etc/apt/sources.list.d/docker.list
[172.16.100.3] + sh -c apt-get update -qq >/dev/null
[172.16.100.4] + sh -c chmod a+r /etc/apt/keyrings/docker.gpg
[172.16.100.4] + sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable" > /etc/apt/sources.list.d/docker.list
[172.16.100.4] + sh -c apt-get update -qq >/dev/null
[172.16.100.3] + sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-scan-plugin docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
[172.16.100.4] + sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-scan-plugin docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
[172.16.100.5] + sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-scan-plugin docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
[172.16.100.5] + sh -c docker version
[172.16.100.4] + sh -c docker version
[172.16.100.3] + sh -c docker version
[172.16.100.5] Client: Docker Engine - Community
[172.16.100.5] Version: 23.0.1
[172.16.100.5] API version: 1.42
[172.16.100.5] Go version: go1.19.5
[172.16.100.5] Git commit: a5ee5b1
[172.16.100.5] Built: Thu Feb 9 19:47:01 2023
[172.16.100.5] OS/Arch: linux/amd64
[172.16.100.5] Context: default
[172.16.100.5] 
[172.16.100.5] Server: Docker Engine - Community
[172.16.100.5] Engine:
[172.16.100.5] Version: 23.0.1
[172.16.100.5] API version: 1.42 (minimum version 1.12)
[172.16.100.5] Go version: go1.19.5
[172.16.100.5] Git commit: bc3805a
[172.16.100.5] Built: Thu Feb 9 19:47:01 2023
[172.16.100.5] OS/Arch: linux/amd64
[172.16.100.5] Experimental: false
[172.16.100.5] containerd:
[172.16.100.5] Version: 1.6.18
[172.16.100.5] GitCommit: 2456e983eb9e37e47538f59ea18f2043c9a73640
[172.16.100.5] runc:
[172.16.100.5] Version: 1.1.4
[172.16.100.5] GitCommit: v1.1.4-0-g5fd4c4d
[172.16.100.5] docker-init:
[172.16.100.5] Version: 0.19.0
[172.16.100.5] GitCommit: de40ad0
[172.16.100.5] 
[172.16.100.5] ================================================================================
[172.16.100.5] 
[172.16.100.5] To run Docker as a non-privileged user, consider setting up the
[172.16.100.5] Docker daemon in rootless mode for your user:
[172.16.100.5] 
[172.16.100.5] dockerd-rootless-setuptool.sh install
[172.16.100.5] 
[172.16.100.5] Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.
[172.16.100.5] 
[172.16.100.5] 
[172.16.100.5] To run the Docker daemon as a fully privileged service, but granting non-root
[172.16.100.5] users access, refer to https://docs.docker.com/go/daemon-access/
[172.16.100.5] 
[172.16.100.5] WARNING: Access to the remote API on a privileged Docker daemon is equivalent
[172.16.100.5] to root access on the host. Refer to the 'Docker daemon attack surface'
[172.16.100.5] documentation for details: https://docs.docker.com/go/attack-surface/
[172.16.100.5] 
[172.16.100.5] ================================================================================
[172.16.100.5] 
    [172.16.100.5] Success
[172.16.100.4] Client: Docker Engine - Community
[172.16.100.4] Version: 23.0.1
[172.16.100.4] API version: 1.42
[172.16.100.4] Go version: go1.19.5
[172.16.100.4] Git commit: a5ee5b1
[172.16.100.4] Built: Thu Feb 9 19:47:01 2023
[172.16.100.4] OS/Arch: linux/amd64
[172.16.100.4] Context: default
[172.16.100.4] 
[172.16.100.4] Server: Docker Engine - Community
[172.16.100.4] Engine:
[172.16.100.4] Version: 23.0.1
[172.16.100.4] API version: 1.42 (minimum version 1.12)
[172.16.100.4] Go version: go1.19.5
[172.16.100.4] Git commit: bc3805a
[172.16.100.4] Built: Thu Feb 9 19:47:01 2023
[172.16.100.4] OS/Arch: linux/amd64
[172.16.100.4] Experimental: false
[172.16.100.4] containerd:
[172.16.100.4] Version: 1.6.18
[172.16.100.4] GitCommit: 2456e983eb9e37e47538f59ea18f2043c9a73640
[172.16.100.4] runc:
[172.16.100.4] Version: 1.1.4
[172.16.100.4] GitCommit: v1.1.4-0-g5fd4c4d
[172.16.100.4] docker-init:
[172.16.100.4] Version: 0.19.0
[172.16.100.4] GitCommit: de40ad0
[172.16.100.4] 
[172.16.100.4] ================================================================================
[172.16.100.4] 
[172.16.100.4] To run Docker as a non-privileged user, consider setting up the
[172.16.100.4] Docker daemon in rootless mode for your user:
[172.16.100.4] 
[172.16.100.4] dockerd-rootless-setuptool.sh install
[172.16.100.4] 
[172.16.100.4] Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.
[172.16.100.4] 
[172.16.100.4] 
[172.16.100.4] To run the Docker daemon as a fully privileged service, but granting non-root
[172.16.100.4] users access, refer to https://docs.docker.com/go/daemon-access/
[172.16.100.4] 
[172.16.100.4] WARNING: Access to the remote API on a privileged Docker daemon is equivalent
[172.16.100.4] to root access on the host. Refer to the 'Docker daemon attack surface'
[172.16.100.4] documentation for details: https://docs.docker.com/go/attack-surface/
[172.16.100.4] 
[172.16.100.4] ================================================================================
[172.16.100.4] 
    [172.16.100.4] Success
[172.16.100.3] Client: Docker Engine - Community
[172.16.100.3] Version: 23.0.1
[172.16.100.3] API version: 1.42
[172.16.100.3] Go version: go1.19.5
[172.16.100.3] Git commit: a5ee5b1
[172.16.100.3] Built: Thu Feb 9 19:47:01 2023
[172.16.100.3] OS/Arch: linux/amd64
[172.16.100.3] Context: default
[172.16.100.3] 
[172.16.100.3] Server: Docker Engine - Community
[172.16.100.3] Engine:
[172.16.100.3] Version: 23.0.1
[172.16.100.3] API version: 1.42 (minimum version 1.12)
[172.16.100.3] Go version: go1.19.5
[172.16.100.3] Git commit: bc3805a
[172.16.100.3] Built: Thu Feb 9 19:47:01 2023
[172.16.100.3] OS/Arch: linux/amd64
[172.16.100.3] Experimental: false
[172.16.100.3] containerd:
[172.16.100.3] Version: 1.6.18
[172.16.100.3] GitCommit: 2456e983eb9e37e47538f59ea18f2043c9a73640
[172.16.100.3] runc:
[172.16.100.3] Version: 1.1.4
[172.16.100.3] GitCommit: v1.1.4-0-g5fd4c4d
[172.16.100.3] docker-init:
[172.16.100.3] Version: 0.19.0
[172.16.100.3] GitCommit: de40ad0
[172.16.100.3] 
[172.16.100.3] ================================================================================
[172.16.100.3] 
[172.16.100.3] To run Docker as a non-privileged user, consider setting up the
[172.16.100.3] Docker daemon in rootless mode for your user:
[172.16.100.3] 
[172.16.100.3] dockerd-rootless-setuptool.sh install
[172.16.100.3] 
[172.16.100.3] Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.
[172.16.100.3] 
[172.16.100.3] 
[172.16.100.3] To run the Docker daemon as a fully privileged service, but granting non-root
[172.16.100.3] users access, refer to https://docs.docker.com/go/daemon-access/
[172.16.100.3] 
[172.16.100.3] WARNING: Access to the remote API on a privileged Docker daemon is equivalent
[172.16.100.3] to root access on the host. Refer to the 'Docker daemon attack surface'
[172.16.100.3] documentation for details: https://docs.docker.com/go/attack-surface/
[172.16.100.3] 
[172.16.100.3] ================================================================================
[172.16.100.3] 
    [172.16.100.3] Success

--> Results:
    Groups: inventory / my_hosts
    [172.16.100.4] Changed: 1 No change: 0 Errors: 0   
    [172.16.100.3] Changed: 1 No change: 0 Errors: 0   
    [172.16.100.5] Changed: 1 No change: 0 Errors: 0 

root@minione:~# pyinfra inventory.py exec -- "docker ps -a"

--> Loading config...

--> Loading inventory...

--> Connecting to hosts...
    No host key for 172.16.100.5 found in known_hosts
    No host key for 172.16.100.3 found in known_hosts
    No host key for 172.16.100.4 found in known_hosts
    [172.16.100.5] Connected
    [172.16.100.3] Connected
    [172.16.100.4] Connected
    [172.16.100.5] Ready: shell
    [172.16.100.3] Ready: shell
    [172.16.100.4] Ready: shell

--> Proposed changes:
    Groups: inventory / my_hosts
    [172.16.100.5] Operations: 1 Change: 1 No change: 0   
    [172.16.100.3] Operations: 1 Change: 1 No change: 0   
    [172.16.100.4] Operations: 1 Change: 1 No change: 0   

--> Beginning operation run...
--> Starting operation: Server/Shell (docker ps -a)
[172.16.100.5] CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    [172.16.100.5] Success
[172.16.100.4] CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    [172.16.100.4] Success
[172.16.100.3] CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    [172.16.100.3] Success

--> Results:
    Groups: inventory / my_hosts
    [172.16.100.5] Changed: 1 No change: 0 Errors: 0   
    [172.16.100.3] Changed: 1 No change: 0 Errors: 0   
    [172.16.100.4] Changed: 1 No change: 0 Errors: 0

Déploiement ensuite des clients Consul et Nomad toujours avec hashi-up :

root@minione:~# export SERVER_IP=172.16.100.2
root@minione:~# export AGENT_1_IP=172.16.100.4
root@minione:~# export AGENT_2_IP=172.16.100.3
root@minione:~# export AGENT_3_IP=172.16.100.5

root@minione:~# hashi-up consul install --ssh-target-addr $AGENT_1_IP --ssh-target-user root --retry-join $SERVER_IP --ssh-target-key .ssh/id_rsa --version 1.15.1

[INFO] Uploading generated Consul configuration ...
[INFO] Installing Consul ...
[INFO] -> Creating user named 'consul'
[INFO] -> Copying configuration files
[INFO] -> Downloading consul_1.15.1_linux_amd64.zip
[INFO] -> Downloading consul_1.15.1_SHA256SUMS
[INFO] -> Verifying downloaded consul_1.15.1_linux_amd64.zip
consul_1.15.1_linux_amd64.zip: OK
[INFO] -> Unpacking consul_1.15.1_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/consul.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/consul.service → /etc/systemd/system/consul.service.
[INFO] -> Starting systemd service
[INFO] Done.

root@minione:~# hashi-up consul install --ssh-target-addr $AGENT_2_IP --ssh-target-user root --retry-join $SERVER_IP --ssh-target-key .ssh/id_rsa --version 1.15.1

[INFO] Uploading generated Consul configuration ...
[INFO] Installing Consul ...
[INFO] -> Creating user named 'consul'
[INFO] -> Copying configuration files
[INFO] -> Downloading consul_1.15.1_linux_amd64.zip
[INFO] -> Downloading consul_1.15.1_SHA256SUMS
[INFO] -> Verifying downloaded consul_1.15.1_linux_amd64.zip
consul_1.15.1_linux_amd64.zip: OK
[INFO] -> Unpacking consul_1.15.1_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/consul.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/consul.service → /etc/systemd/system/consul.service.
[INFO] -> Starting systemd service
[INFO] Done.

root@minione:~# hashi-up consul install --ssh-target-addr $AGENT_3_IP --ssh-target-user root --retry-join $SERVER_IP --ssh-target-key .ssh/id_rsa --version 1.15.1

[INFO] Uploading generated Consul configuration ...
[INFO] Installing Consul ...
[INFO] -> Creating user named 'consul'
[INFO] -> Copying configuration files
[INFO] -> Downloading consul_1.15.1_linux_amd64.zip
[INFO] -> Downloading consul_1.15.1_SHA256SUMS
[INFO] -> Verifying downloaded consul_1.15.1_linux_amd64.zip
consul_1.15.1_linux_amd64.zip: OK
[INFO] -> Unpacking consul_1.15.1_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/consul.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/consul.service → /etc/systemd/system/consul.service.
[INFO] -> Starting systemd service
[INFO] Done.

root@minione:~# hashi-up nomad install --ssh-target-addr $AGENT_1_IP --ssh-target-user root --client --retry-join $SERVER_IP --ssh-target-key .ssh/id_rsa --version 1.5.0

[INFO] Uploading generated Nomad configuration ...
[INFO] Installing Nomad ...
[INFO] -> Copying configuration files
[INFO] -> Downloading nomad_1.5.0_linux_amd64.zip
[INFO] -> Downloading nomad_1.5.0_SHA256SUMS
[INFO] -> Verifying downloaded nomad_1.5.0_linux_amd64.zip
nomad_1.5.0_linux_amd64.zip: OK
[INFO] -> Unpacking nomad_1.5.0_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/nomad.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/nomad.service → /etc/systemd/system/nomad.service.
[INFO] -> Starting systemd service
[INFO] Done.

root@minione:~# hashi-up nomad install --ssh-target-addr $AGENT_2_IP --ssh-target-user root --client --retry-join $SERVER_IP --ssh-target-key .ssh/id_rsa --version 1.5.0

[INFO] Uploading generated Nomad configuration ...
[INFO] Installing Nomad ...
[INFO] -> Copying configuration files
[INFO] -> Downloading nomad_1.5.0_linux_amd64.zip
[INFO] -> Downloading nomad_1.5.0_SHA256SUMS
[INFO] -> Verifying downloaded nomad_1.5.0_linux_amd64.zip
nomad_1.5.0_linux_amd64.zip: OK
[INFO] -> Unpacking nomad_1.5.0_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/nomad.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/nomad.service → /etc/systemd/system/nomad.service.
[INFO] -> Starting systemd service
[INFO] Done.

root@minione:~# hashi-up nomad install --ssh-target-addr $AGENT_3_IP --ssh-target-user root --client --retry-join $SERVER_IP --ssh-target-key .ssh/id_rsa --version 1.5.0

[INFO] Uploading generated Nomad configuration ...
[INFO] Installing Nomad ...
[INFO] -> Copying configuration files
[INFO] -> Downloading nomad_1.5.0_linux_amd64.zip
[INFO] -> Downloading nomad_1.5.0_SHA256SUMS
[INFO] -> Verifying downloaded nomad_1.5.0_linux_amd64.zip
nomad_1.5.0_linux_amd64.zip: OK
[INFO] -> Unpacking nomad_1.5.0_linux_amd64.zip
[INFO] -> Adding systemd service file /etc/systemd/system/nomad.service
[INFO] -> Enabling systemd service
Created symlink /etc/systemd/system/multi-user.target.wants/nomad.service → /etc/systemd/system/nomad.service.
[INFO] -> Starting systemd service
[INFO] Done.

Les clients Nomad avec Consul sont présents :

Lancement de Fabio qui s’intègre nativement avec Consul et fournit une interface Web optionnelle pour visualiser le routage. Le principal cas d’utilisation de Fabio est la distribution des requêtes HTTP(S) et TCP entrantes depuis Internet vers les services frontaux qui peuvent traiter ces requêtes.

Load Balancing with Fabio | Nomad | HashiCorp Developer

Ici, je lance Fabio en tant que job système afin qu’il puisse acheminer le trafic entrant de manière homogène vers un groupe de serveurs quels que soient les nœuds clients sur lesquels il s’exécute. On peut donc placer tous les nœuds clients derrière un équilibreur de charge pour fournir à l’utilisateur final un point d’accès unique.

job "fabio" {
  datacenters = ["dc1"]
  type = "system"

  group "fabio" {
    network {
      port "lb" {
        static = 9999
      }
      port "ui" {
        static = 9998
      }
    }
    task "fabio" {
      driver = "docker"
      config {
        image = "fabiolb/fabio"
        network_mode = "host"
        ports = ["lb","ui"]
      }

      resources {
        cpu = 200
        memory = 128
      }
    }
  }
}

L’interface graphique de Fabio est disponible sur chacun des trois noeuds clients sur le port TCP 9998 :

Exemple avec ce job et une image de base Nginx avec création de trois serveurs attenants :

job "webserver" {
  datacenters = ["dc1"]
  type = "service"

  group "webserver" {
    count = 3
    network {
      port "http" {
        to = 8080
      }
    }

    service {
      name = "nginx-webserver"
      tags = ["urlprefix-/"]
      port = "http"
      check {
        name = "alive"
        type = "http"
        path = "/"
        interval = "10s"
        timeout = "2s"
      }
    }

    restart {
      attempts = 2
      interval = "30m"
      delay = "15s"
      mode = "fail"
    }

    task "nginx" {
      driver = "docker"
      config {
        image = "bitnami/nginx:latest"
        ports = ["http"]
      }
    }
  }
}

Fabio a détecté le déploiement de ces trois serveurs Nginx :

en fournissant des ports d’accès à ces entités :

Test d’une instance Alpine Linux avec un serveur OpenSSH qui mime le fonctionnement d’une machine virtuelle dans Nomad en utilisant cette image Docker :

Docker

job "ssh" {
  datacenters = ["dc1"]
  type = "service"

  group "ssh" {
    count = 1
    network {
      port "ssh" {
        to = 22
      }
    }

    service {
      name = "sshd"
      port = "ssh"
    }

    task "openssh" {
      driver = "docker"
      config {
        image = "mcas/alpine-sshd:latest"
        ports = ["ssh"]
      }
    }
  }
}

Exécution du job via la dashboard du serveur Nomad :

Consul voit l’ensemble des instances déployées dans Nomad :

en me donnant les informations sur le port d’accès à cette instance Alpine Linux :

Connexion locale à cette instance en SSH avec ce port d’accès :

root@minione:~# ssh -p 26053 alpine@172.16.100.5
Warning: Permanently added '[172.16.100.5]:26053' (ED25519) to the list of known hosts.
alpine@172.16.100.5 password: 
Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <https://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

1900f3e8b78a:~$ ps aux
PID USER TIME COMMAND
    1 root 0:00 sshd: /usr/sbin/sshd -D -e [listener] 0 of 10-100 startups
   10 root 0:00 sshd: alpine [priv]
   12 alpine 0:00 sshd: alpine@pts/0
   13 alpine 0:00 -bash
   14 alpine 0:00 ps aux
1900f3e8b78a:~$ free -m
              total used free shared buff/cache available
Mem: 5934 348 4982 1 604 5346
Swap: 0 0 0
1900f3e8b78a:~$ df -h
Filesystem Size Used Available Use% Mounted on
overlay 19.2G 2.3G 16.9G 12% /
tmpfs 64.0M 0 64.0M 0% /dev
shm 64.0M 0 64.0M 0% /dev/shm
/dev/vda1 19.2G 2.3G 16.9G 12% /alloc
/dev/vda1 19.2G 2.3G 16.9G 12% /local
tmpfs 1.0M 0 1.0M 0% /secrets
/dev/vda1 19.2G 2.3G 16.9G 12% /etc/resolv.conf
/dev/vda1 19.2G 2.3G 16.9G 12% /etc/hostname
/dev/vda1 19.2G 2.3G 16.9G 12% /etc/hosts
tmpfs 2.9G 0 2.9G 0% /proc/acpi
tmpfs 64.0M 0 64.0M 0% /proc/kcore
tmpfs 64.0M 0 64.0M 0% /proc/keys
tmpfs 64.0M 0 64.0M 0% /proc/timer_list
tmpfs 2.9G 0 2.9G 0% /proc/scsi
tmpfs 2.9G 0 2.9G 0% /sys/firmware

Le sempiternel démonstrateur FC s’y exécute localement comme s’il s’agissait d’une machine virtuelle :

GitHub - france-connect/service-provider-example: An implementation example of the FranceConnect button on a service provider's website.

1900f3e8b78a:~/fcdemo3$ pm2 start start.sh 

                        -------------

__/\\\\\\\\\\\\\____ /\\\\ ____________/\\\\____ /\\\\\\\\\ _____
 _\/\\\/////////\\\_\/\\\\\\ ________/\\\\\\__ /\\\///////\\\___
  _\/\\\ _______\/\\\_\/\\\//\\\____ /\\\//\\\_\/// ______\//\\\__
   _\/\\\\\\\\\\\\\/ __\/\\\\///\\\/\\\/_\/\\\___________ /\\\/___
    _\/\\\///////// ____\/\\\__ \///\\\/ ___\/\\\________ /\\\// _____
     _\/\\\ _____________\/\\\____ \/// _____\/\\\_____ /\\\// ________
      _\/\\\ _____________\/\\\_____________ \/\\\ ___/\\\/___________
       _\/\\\ _____________\/\\\_____________ \/\\\__/\\\\\\\\\\\\\\\_
        _\/// ______________\///______________ \/// __\///////////////__

                          Runtime Edition

        PM2 is a Production Process Manager for Node.js applications
                     with a built-in Load Balancer.

                Start and Daemonize any application:
                $ pm2 start app.js

                Load Balance 4 instances of api.js:
                $ pm2 start api.js -i 4

                Monitor in production:
                $ pm2 monitor

                Make pm2 auto-boot at server restart:
                $ pm2 startup

                To go further checkout:
                http://pm2.io/

                        -------------

[PM2] Spawning PM2 daemon with pm2_home=/home/alpine/.pm2
[PM2] PM2 Successfully daemonized
[PM2] Starting /home/alpine/fcdemo3/start.sh in fork_mode (1 instance)
[PM2] Done.
┌─────┬──────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼──────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0 │ start │ default │ 0.0.0 │ fork │ 466 │ 0s │ 0 │ online │ 0% │ 1.7mb │ alpine │ disabled │
└─────┴──────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘

1900f3e8b78a:~/fcdemo3$ pm2 log start

[TAILING] Tailing last 15 lines for [start] process (change the value with --lines option)
/home/alpine/.pm2/logs/start-error.log last 15 lines:
/home/alpine/.pm2/logs/start-out.log last 15 lines:
0|start | 
0|start | > service-provider-mock@0.0.0 start
0|start | > node ./bin/www
0|start | 
0|start | Server listening on http://0.0.0.0:3000

1900f3e8b78a:~$ curl http://localhost:3000
<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport"
          content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.1/css/bulma.min.css" integrity="sha256-zIG416V1ynj3Wgju/scU80KAEWOsO5rRLfVyRDuOv7Q=" crossorigin="anonymous" />
    <title>Démonstrateur Fournisseur de Service</title>
</head>

<body>
<nav class="navbar" role="navigation" aria-label="main navigation">
    <div class="navbar-start">
        <div class="navbar-brand">
            <a class="navbar-item" href="/">
                <img src="/img/fc_logo_v2.png" alt="Démonstrateur Fournisseur de Service" height="28">
            </a>
        </div>
        <a href="/" class="navbar-item">
            Home
        </a>
    </div>
    <div class="navbar-end">
        <div class="navbar-item">

                <div class="buttons">
                    <a class="button is-light" href="/login">Se connecter</a>
                </div>

        </div>
    </div>
</nav>

<section class="hero is-info is-medium">
    <div class="hero-body">
        <div class="container">
            <h1 class="title">
                Bienvenue sur le démonstrateur de fournisseur de service
            </h1>
            <h2 class="subtitle">
                Cliquez sur "se connecter" pour vous connecter via <strong>FranceConnect</strong>
            </h2>
        </div>
    </div>
</section>

Dans cet article, il est décrit la possibilité d’utiliser hashi-up pour provisionner un cluster avec Vault ou en mode HA pour le cluster Nomad et Consul :

Ce cluster commun entre Nomad et Consul aurait pû être utilisé avec Waypoint pour y provisionner ses jobs comme avec cet exemple de fichier HCL …

Waypoint | HashiCorp Developer

Deploy an Application to Nomad | Waypoint | HashiCorp Developer

# Copyright (c) HashiCorp, Inc.
# SPDX-License-Identifier: MPL-2.0

project = "nomad-nodejs"

app "nomad-nodejs-web" {

  build {
    use "pack" {}
    registry {
      use "docker" {
        image = "nomad-nodejs-web"
        tag = "1"
        local = true
      }
    }
  }

  deploy {
    use "nomad" {
      // these options both default to the values shown, but are left here to
      // show they are configurable
      datacenter = "dc1"
      namespace = "default"
    }
  }

}

On en allant beaucoup plus loin avec cet exemple de déploiement Blue/Green …

À suivre !

Effectuer des opérations de base autour des machines virtuelles avec kubevirt-manager …

Karim — Thu, 09 Mar 2023 14:35:11 +0000

Kubevirt permet comme on l’a vu dans plusieurs articles précedemment de fournir une plateforme de développement unifiée où les développeurs peuvent construire, modifier et déployer des applications résidant à la fois dans des conteneurs d’application et dans des machines virtuelles dans un environnement commun et partagé.

https://kubevirt.io/2020/KubeVirt_deep_dive-virtualized_gpu_workloads.html

Même si d’autres solutions alternatives de création et d’utilisations de machines virtuelles existent pour Kubernetes :

Une solution existait pour gérer ses machines virtuelles graphiquement dans Kubernetes et KubeVirt avec OpenShift Web Console :

Ici focus sur KubeVirt Manager, un projet récent qui fournit un Frontend Angular simple pour faire fonctionner KubeVirt.

Cet outil vous permet d’effectuer des opérations de base autour des machines virtuelles, des instances de machines virtuelles et des disques :

GitHub - kubevirt-manager/kubevirt-manager: Kubevirt Web UI / Web Interface Manager

Application rapide avec le déploiement d’un cluster AKS dans Azure avec un Scale Set utilisant des instances Spot et ces paramètres :

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "resourceName": {
            "value": "AKS"
        },
        "location": {
            "value": "francecentral"
        },
        "dnsPrefix": {
            "value": "AKS-dns"
        },
        "kubernetesVersion": {
            "value": "1.26.0"
        },
        "networkPlugin": {
            "value": "kubenet"
        },
        "enableRBAC": {
            "value": true
        },
        "nodeResourceGroup": {
            "value": "MC_RG-AKS_AKS_francecentral"
        },
        "upgradeChannel": {
            "value": "none"
        },
        "adminGroupObjectIDs": {
            "value": []
        },
        "disableLocalAccounts": {
            "value": false
        },
        "azureRbac": {
            "value": false
        },
        "enablePrivateCluster": {
            "value": false
        },
        "enableHttpApplicationRouting": {
            "value": false
        },
        "enableAzurePolicy": {
            "value": false
        },
        "enableSecretStoreCSIDriver": {
            "value": false
        },
        "vmssNodePool": {
            "value": true
        }
    }
}

Mise en oeuvre d’un cluster Kubernetes 1.14 avec des noeuds Windows via AKS-Engine …

Le cluster AKS est déployé avec récupération des identifiants :

$ az aks get-credentials --resource-group RG-AKS --name AKS
Merged "AKS" as current context in /home/cert/.kube/config

$ kubectl cluster-info
Kubernetes control plane is running at https://aks-dns-bn9a54nt.hcp.francecentral.azmk8s.io:443
CoreDNS is running at https://aks-dns-bn9a54nt.hcp.francecentral.azmk8s.io:443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://aks-dns-bn9a54nt.hcp.francecentral.azmk8s.io:443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

$ kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/azure-ip-masq-agent-6g72l 1/1 Running 0 6m18s
kube-system pod/azure-ip-masq-agent-nzbf5 1/1 Running 0 6m20s
kube-system pod/azure-ip-masq-agent-rj6mj 1/1 Running 0 6m6s
kube-system pod/cloud-node-manager-rfpcf 1/1 Running 0 6m18s
kube-system pod/cloud-node-manager-rg5jm 1/1 Running 0 6m6s
kube-system pod/cloud-node-manager-zpsgs 1/1 Running 0 6m20s
kube-system pod/coredns-78669d9946-txcm8 1/1 Running 0 4m26s
kube-system pod/coredns-78669d9946-zt9b5 1/1 Running 0 6m30s
kube-system pod/coredns-autoscaler-5b5c4f5b4f-zl2hq 1/1 Running 0 6m30s
kube-system pod/csi-azuredisk-node-44ffr 3/3 Running 0 6m20s
kube-system pod/csi-azuredisk-node-m2s5r 3/3 Running 0 6m18s
kube-system pod/csi-azuredisk-node-v4dc8 3/3 Running 0 6m6s
kube-system pod/csi-azurefile-node-5pzl4 3/3 Running 0 6m18s
kube-system pod/csi-azurefile-node-mqxw4 3/3 Running 0 6m6s
kube-system pod/csi-azurefile-node-nfvct 3/3 Running 0 6m20s
kube-system pod/konnectivity-agent-f94c65c6f-bjcwq 1/1 Running 0 6m29s
kube-system pod/konnectivity-agent-f94c65c6f-rdbfl 1/1 Running 0 6m29s
kube-system pod/kube-proxy-52svq 1/1 Running 0 6m18s
kube-system pod/kube-proxy-cnplk 1/1 Running 0 6m6s
kube-system pod/kube-proxy-dnjjp 1/1 Running 0 6m20s
kube-system pod/metrics-server-5f6654d4df-nqj2w 2/2 Running 0 4m20s
kube-system pod/metrics-server-5f6654d4df-qk22x 2/2 Running 0 4m20s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 7m4s
kube-system service/kube-dns ClusterIP 10.0.0.10 <none> 53/UDP,53/TCP 6m31s
kube-system service/metrics-server ClusterIP 10.0.139.110 <none> 443/TCP 6m30s

$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
aks-agentpool-16963728-vmss000000 Ready agent 6m15s v1.26.0 10.224.0.5 <none> Ubuntu 22.04.1 LTS 5.15.0-1033-azure containerd://1.6.17+azure-1
aks-userpool-16963728-vmss000000 Ready agent 6m27s v1.26.0 10.224.0.4 20.111.27.134 Ubuntu 22.04.1 LTS 5.15.0-1033-azure containerd://1.6.17+azure-1
aks-userpool-16963728-vmss000001 Ready agent 6m29s v1.26.0 10.224.0.6 20.111.27.202 Ubuntu 22.04.1 LTS 5.15.0-1033-azure containerd://1.6.17+azure-1

J’utilise ici un pool de Noeud Spot avec des instances Esv5 qui autorise la virtualisation imbriquée pour cette expérience :

Ev5 and Esv5-series - Azure Virtual Machines

Ce qui permet le déploiement de KubeVirt via HCO (Hyperconverged Cluster Operator) :

GitHub - kubevirt/hyperconverged-cluster-operator: Operator pattern for managing multi-operator products

avec ce script unifié :

curl https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/deploy.sh | bash

$ curl https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/deploy.sh | bash

+ hco_namespace=kubevirt-hyperconverged
+ IS_OPENSHIFT=false
+ kubectl api-resources
+ grep clusterversions
+ grep config.openshift.io
+ kubectl create ns kubevirt-hyperconverged --dry-run=true -o yaml
+ kubectl apply -f -
W0309 13:38:03.250568 3638 helpers.go:677] --dry-run=true is deprecated (boolean value) and can be replaced with --dry-run=client.
namespace/kubevirt-hyperconverged created
+ namespaces=("openshift")
+ for namespace in ${namespaces[@]}
++ kubectl get ns openshift
Error from server (NotFound): namespaces "openshift" not found
+ [['' == '']]
+ kubectl create ns openshift --dry-run=true -o yaml
+ kubectl apply -f -
W0309 13:38:04.072929 3667 helpers.go:677] --dry-run=true is deprecated (boolean value) and can be replaced with --dry-run=client.
namespace/openshift created
+ LABEL_SELECTOR_ARG=
+ '[' false '!=' true ']'
+ LABEL_SELECTOR_ARG='-l name!=ssp-operator,name!=hyperconverged-cluster-cli-download'
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/cluster-network-addons00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/networkaddonsconfigs.networkaddonsoperator.network.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/containerized-data-importer00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/cdis.cdi.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/hco00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/hyperconvergeds.hco.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/kubevirt00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/kubevirts.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/hostpath-provisioner00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/hostpathprovisioners.hostpathprovisioner.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/scheduling-scale-performance00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/ssps.ssp.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/crds/tekton-tasks-operator00.crd.yaml
customresourcedefinition.apiextensions.k8s.io/tektontasks.tektontasks.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/cert-manager.yaml
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
namespace/cert-manager created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
role.rbac.authorization.k8s.io/cert-manager:leaderelection created
role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
+ kubectl -n cert-manager wait deployment/cert-manager --for=condition=Available --timeout=300s
deployment.apps/cert-manager condition met
+ kubectl -n cert-manager wait deployment/cert-manager-webhook --for=condition=Available --timeout=300s
deployment.apps/cert-manager-webhook condition met
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -n kubevirt-hyperconverged -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/cluster_role.yaml
role.rbac.authorization.k8s.io/cluster-network-addons-operator created
role.rbac.authorization.k8s.io/kubevirt-operator created
role.rbac.authorization.k8s.io/tekton-tasks-operator created
role.rbac.authorization.k8s.io/cdi-operator created
role.rbac.authorization.k8s.io/hostpath-provisioner-operator created
clusterrole.rbac.authorization.k8s.io/hyperconverged-cluster-operator created
clusterrole.rbac.authorization.k8s.io/cluster-network-addons-operator created
clusterrole.rbac.authorization.k8s.io/kubevirt-operator created
clusterrole.rbac.authorization.k8s.io/tekton-tasks-operator created
clusterrole.rbac.authorization.k8s.io/cdi-operator created
clusterrole.rbac.authorization.k8s.io/hostpath-provisioner-operator created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -n kubevirt-hyperconverged -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/service_account.yaml
serviceaccount/cdi-operator created
serviceaccount/cluster-network-addons-operator created
serviceaccount/hostpath-provisioner-operator created
serviceaccount/hyperconverged-cluster-operator created
serviceaccount/kubevirt-operator created
serviceaccount/tekton-tasks-operator created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -n kubevirt-hyperconverged -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/cluster_role_binding.yaml
rolebinding.rbac.authorization.k8s.io/cluster-network-addons-operator created
rolebinding.rbac.authorization.k8s.io/kubevirt-operator created
rolebinding.rbac.authorization.k8s.io/tekton-tasks-operator created
rolebinding.rbac.authorization.k8s.io/cdi-operator created
rolebinding.rbac.authorization.k8s.io/hostpath-provisioner-operator created
clusterrolebinding.rbac.authorization.k8s.io/hyperconverged-cluster-operator created
clusterrolebinding.rbac.authorization.k8s.io/cluster-network-addons-operator created
clusterrolebinding.rbac.authorization.k8s.io/kubevirt-operator created
clusterrolebinding.rbac.authorization.k8s.io/tekton-tasks-operator created
clusterrolebinding.rbac.authorization.k8s.io/cdi-operator created
clusterrolebinding.rbac.authorization.k8s.io/hostpath-provisioner-operator created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -n kubevirt-hyperconverged -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/webhooks.yaml
issuer.cert-manager.io/selfsigned created
certificate.cert-manager.io/hyperconverged-cluster-webhook-service-cert created
validatingwebhookconfiguration.admissionregistration.k8s.io/validate-hco.kubevirt.io created
certificate.cert-manager.io/node-maintenance-operator-service-cert created
validatingwebhookconfiguration.admissionregistration.k8s.io/nodemaintenance-validation.kubevirt.io created
certificate.cert-manager.io/hostpath-provisioner-operator-webhook-service-cert created
validatingwebhookconfiguration.admissionregistration.k8s.io/hostpathprovisioner.kubevirt.io created
mutatingwebhookconfiguration.admissionregistration.k8s.io/mutate-hco.kubevirt.io created
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -n kubevirt-hyperconverged -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/operator.yaml
deployment.apps/hyperconverged-cluster-operator created
deployment.apps/hyperconverged-cluster-webhook created
deployment.apps/cluster-network-addons-operator created
deployment.apps/virt-operator created
deployment.apps/tekton-tasks-operator created
deployment.apps/cdi-operator created
deployment.apps/hostpath-provisioner-operator created
service/hyperconverged-cluster-webhook-service created
service/hostpath-provisioner-operator-service created
+ kubectl -n kubevirt-hyperconverged wait deployment/hyperconverged-cluster-webhook --for=condition=Available --timeout=300s
deployment.apps/hyperconverged-cluster-webhook condition met
+ kubectl apply -l 'name!=ssp-operator,name!=hyperconverged-cluster-cli-download' -n kubevirt-hyperconverged -f https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/main/deploy/hco.cr.yaml

hyperconverged.hco.kubevirt.io/kubevirt-hyperconverged created

KubeVirt est alors déployé dans ce cluster :

$ kubectl get po,svc -n kubevirt-hyperconverged

NAME READY STATUS RESTARTS AGE
pod/bridge-marker-n7dgz 1/1 Running 0 3m14s
pod/bridge-marker-qld66 1/1 Running 0 3m14s
pod/bridge-marker-w9rjv 1/1 Running 0 3m14s
pod/cdi-apiserver-5b79dbb75c-2wjgl 1/1 Running 0 2m41s
pod/cdi-deployment-68787cb469-l5cpl 1/1 Running 0 2m37s
pod/cdi-operator-b5fc58cd7-b9f2s 1/1 Running 0 3m24s
pod/cdi-uploadproxy-56d645d8d5-trt54 1/1 Running 0 2m46s
pod/cluster-network-addons-operator-6f6765958d-rn2fn 2/2 Running 0 3m35s
pod/hostpath-provisioner-operator-7bb46fd5b7-cg2hb 1/1 Running 0 3m24s
pod/hyperconverged-cluster-operator-7df6b469b4-942tx 1/1 Running 0 3m35s
pod/hyperconverged-cluster-webhook-55cfd6cb69-46r8v 1/1 Running 0 3m35s
pod/kube-cni-linux-bridge-plugin-62jrn 1/1 Running 0 3m14s
pod/kube-cni-linux-bridge-plugin-hqpsg 1/1 Running 0 3m14s
pod/kube-cni-linux-bridge-plugin-nzkpp 1/1 Running 0 3m14s
pod/kubemacpool-cert-manager-6dfd4bb587-b29pl 1/1 Running 0 3m14s
pod/kubemacpool-mac-controller-manager-646dd6f884-gxttc 2/2 Running 0 3m14s
pod/multus-ggncz 1/1 Running 0 3m15s
pod/multus-m2xjm 1/1 Running 0 3m15s
pod/multus-zjm8n 1/1 Running 0 3m15s
pod/tekton-tasks-operator-7d98668446-prr4b 1/1 Running 0 3m35s
pod/virt-api-94f856488-k7fcb 1/1 Running 0 2m20s
pod/virt-controller-7bd9bdbccc-rn762 1/1 Running 0 107s
pod/virt-exportproxy-5c8d7c5668-mwfnz 1/1 Running 0 107s
pod/virt-handler-mzjv5 1/1 Running 0 107s
pod/virt-operator-6797b9bb9c-5jqj8 1/1 Running 0 3m24s
pod/virt-operator-6797b9bb9c-rrm4n 1/1 Running 0 3m24s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cdi-api ClusterIP 10.0.59.53 <none> 443/TCP 2m41s
service/cdi-prometheus-metrics ClusterIP 10.0.220.94 <none> 8080/TCP 2m37s
service/cdi-uploadproxy ClusterIP 10.0.37.12 <none> 443/TCP 2m46s
service/hostpath-provisioner-operator-service ClusterIP 10.0.118.100 <none> 9443/TCP 3m35s
service/hyperconverged-cluster-webhook-service ClusterIP 10.0.122.49 <none> 4343/TCP 3m35s
service/kubemacpool-service ClusterIP 10.0.96.143 <none> 443/TCP 3m14s
service/kubevirt-operator-webhook ClusterIP 10.0.145.178 <none> 443/TCP 2m22s
service/kubevirt-prometheus-metrics ClusterIP None <none> 443/TCP 2m22s
service/virt-api ClusterIP 10.0.230.208 <none> 443/TCP 2m22s
service/virt-exportproxy ClusterIP 10.0.218.176 <none> 443/TCP 2m22s

KubeVirt Manager est mise en oeuvre via les manifests YAML fournis dans son dépôt GitHub :

$ git clone https://github.com/kubevirt-manager/kubevirt-manager
Cloning into 'kubevirt-manager'...
remote: Enumerating objects: 4464, done.
remote: Counting objects: 100% (1287/1287), done.
remote: Compressing objects: 100% (446/446), done.
remote: Total 4464 (delta 838), reused 1191 (delta 815), pack-reused 3177
Receiving objects: 100% (4464/4464), 67.53 MiB | 3.41 MiB/s, done.
Resolving deltas: 100% (1461/1461), done.

~/kubevirt-manager$ kubectl apply -f kubernetes/ns.yaml
namespace/kubevirt-manager created

~/kubevirt-manager$ cat kubernetes/ns.yaml 

apiVersion: v1
kind: Namespace
metadata:
  name: kubevirt-manager
  labels:
    app: kubevirt-manager
    kubevirt-manager.io/version: 1.2.0

~/kubevirt-manager$ kubectl apply -f kubernetes/rbac.yaml

serviceaccount/kubevirt-manager created
clusterrole.rbac.authorization.k8s.io/kubevirt-manager created
clusterrolebinding.rbac.authorization.k8s.io/kubevirt-manager created

~/kubevirt-manager$ kubectl apply -f kubernetes/deployment.yaml
deployment.apps/kubevirt-manager created

~/kubevirt-manager$ kubectl apply -f kubernetes/pc.yaml
priorityclass.scheduling.k8s.io/vm-standard created
priorityclass.scheduling.k8s.io/vm-preemptible created

~/kubevirt-manager$ kubectl apply -f kubernetes/service.yaml
service/kubevirt-manager created

KubeVirt Manager est déployé et je change en éditant le service d’exposition de ClusterIP à Load Balancer pour obtenir une exposition en HTTP :

Services, Load Balancing, and Networking

$ kubectl get po,svc -n kubevirt-manager
NAME READY STATUS RESTARTS AGE
pod/kubevirt-manager-5858499887-wfrgn 1/1 Running 0 95s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubevirt-manager ClusterIP 10.0.173.82 <none> 8080/TCP 63s

$ kubectl edit -n kubevirt-manager service/kubevirt-manager
service/kubevirt-manager edited

$ kubectl get po,svc -n kubevirt-manager
NAME READY STATUS RESTARTS AGE
pod/kubevirt-manager-5858499887-wfrgn 1/1 Running 0 4m56s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubevirt-manager LoadBalancer 10.0.173.82 20.74.99.190 8080:30143/TCP 4m24s

Et j’y accède via l’adresse IP publique fournie dans AKS :

Krew est installé localement via Kubectl ainsi que le plugin correspondant pour Kubevirt :

Quickstart

$ kubectl krew update
Updated the local copy of plugin index.

$ kubectl krew search virt
NAME DESCRIPTION INSTALLED
pv-migrate Migrate data across persistent volumes no
pvmigrate Migrates PVs between StorageClasses no
view-cert View certificate information stored in secrets no
view-secret Decode Kubernetes secrets no
view-serviceaccount-kubeconfig Show a kubeconfig setting to access the apiserv... no
virt Control KubeVirt virtual machines using virtctl yes

Ce qui permet de lancer directement une petite machine virtuelle avec Cirros :

GitHub - cirros-dev/cirros

$ kubectl apply -f https://raw.githubusercontent.com/kubevirt/demo/master/manifests/vm.yaml
virtualmachine.kubevirt.io/testvm created

$ kubectl describe vm testvm
Name: testvm
Namespace: default
Labels: <none>
Annotations: kubevirt.io/latest-observed-api-version: v1
              kubevirt.io/storage-observed-api-version: v1alpha3
API Version: kubevirt.io/v1
Kind: VirtualMachine
Metadata:
  Creation Timestamp: 2023-03-09T13:23:32Z
  Finalizers:
    kubevirt.io/virtualMachineControllerFinalize
  Generation: 1
  Managed Fields:
    API Version: kubevirt.io/v1alpha3
    Fields Type: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:kubevirt.io/latest-observed-api-version:
          f:kubevirt.io/storage-observed-api-version:
        f:finalizers:
          .:
          v:"kubevirt.io/virtualMachineControllerFinalize":
    Manager: Go-http-client
    Operation: Update
    Time: 2023-03-09T13:23:32Z
    API Version: kubevirt.io/v1alpha3
    Fields Type: FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:printableStatus:
        f:volumeSnapshotStatuses:
    Manager: Go-http-client
    Operation: Update
    Subresource: status
    Time: 2023-03-09T13:23:32Z
    API Version: kubevirt.io/v1alpha3
    Fields Type: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:running:
        f:template:
          .:
          f:metadata:
            .:
            f:labels:
              .:
              f:kubevirt.io/domain:
              f:kubevirt.io/size:
          f:spec:
            .:
            f:domain:
              .:
              f:devices:
                .:
                f:disks:
                f:interfaces:
              f:resources:
                .:
                f:requests:
                  .:
                  f:memory:
            f:networks:
            f:volumes:
    Manager: kubectl-client-side-apply
    Operation: Update
    Time: 2023-03-09T13:23:32Z
  Resource Version: 39937
  UID: 01de5614-922a-4ae8-9bec-40eeb9a7c500
Spec:
  Running: false
  Template:
    Metadata:
      Creation Timestamp: <nil>
      Labels:
        kubevirt.io/domain: testvm
        kubevirt.io/size: small
    Spec:
      Domain:
        Devices:
          Disks:
            Disk:
              Bus: virtio
            Name: rootfs
            Disk:
              Bus: virtio
            Name: cloudinit
          Interfaces:
            Mac Address: 02:9d:e4:00:00:00
            Masquerade:
            Name: default
        Machine:
          Type: q35
        Resources:
          Requests:
            Memory: 64M
      Networks:
        Name: default
        Pod:
      Volumes:
        Container Disk:
          Image: kubevirt/cirros-registry-disk-demo
        Name: rootfs
        Cloud Init No Cloud:
          userDataBase64: SGkuXG4=
        Name: cloudinit
Status:
  Conditions:
    Last Probe Time: 2023-03-09T13:23:32Z
    Last Transition Time: 2023-03-09T13:23:32Z
    Message: VMI does not exist
    Reason: VMINotExists
    Status: False
    Type: Ready
  Printable Status: Stopped
  Volume Snapshot Statuses:
    Enabled: false
    Name: rootfs
    Reason: Snapshot is not supported for this volumeSource type [rootfs]
    Enabled: false
    Name: cloudinit
    Reason: Snapshot is not supported for this volumeSource type [cloudinit]
Events: <none>

$ kubectl virt start testvm
VM testvm was scheduled to start

$ kubectl describe vmi testvm
Name: testvm
Namespace: default
Labels: kubevirt.io/domain=testvm
              kubevirt.io/size=small
Annotations: kubevirt.io/latest-observed-api-version: v1
              kubevirt.io/storage-observed-api-version: v1alpha3
API Version: kubevirt.io/v1
Kind: VirtualMachineInstance
Metadata:
  Creation Timestamp: 2023-03-09T13:26:14Z
  Finalizers:
    kubevirt.io/virtualMachineControllerFinalize
    foregroundDeleteVirtualMachine
  Generation: 4
  Managed Fields:
    API Version: kubevirt.io/v1alpha3
    Fields Type: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubevirt.io/latest-observed-api-version:
          f:kubevirt.io/storage-observed-api-version:
        f:finalizers:
          .:
          v:"kubevirt.io/virtualMachineControllerFinalize":
        f:labels:
          .:
          f:kubevirt.io/domain:
          f:kubevirt.io/size:
        f:ownerReferences:
          .:
          k:{"uid":"01de5614-922a-4ae8-9bec-40eeb9a7c500"}:
      f:spec:
        .:
        f:domain:
          .:
          f:devices:
            .:
            f:disks:
            f:interfaces:
          f:firmware:
            .:
            f:uuid:
          f:machine:
            .:
            f:type:
          f:resources:
            .:
            f:requests:
              .:
              f:memory:
        f:networks:
        f:volumes:
      f:status:
        .:
        f:activePods:
          .:
          f:f2afda42-6f8f-4bad-b35a-7591c3233cc1:
        f:conditions:
        f:guestOSInfo:
        f:phase:
        f:phaseTransitionTimestamps:
        f:qosClass:
        f:runtimeUser:
        f:virtualMachineRevisionName:
    Manager: Go-http-client
    Operation: Update
    Time: 2023-03-09T13:26:14Z
  Owner References:
    API Version: kubevirt.io/v1
    Block Owner Deletion: true
    Controller: true
    Kind: VirtualMachine
    Name: testvm
    UID: 01de5614-922a-4ae8-9bec-40eeb9a7c500
  Resource Version: 41728
  UID: 745caf89-6e75-47d1-b094-91e6119f3cc4
Spec:
  Domain:
    Cpu:
      Cores: 1
      Model: host-model
      Sockets: 1
      Threads: 1
    Devices:
      Disks:
        Disk:
          Bus: virtio
        Name: rootfs
        Disk:
          Bus: virtio
        Name: cloudinit
      Interfaces:
        Mac Address: 02:9d:e4:00:00:00
        Masquerade:
        Name: default
    Features:
      Acpi:
        Enabled: true
    Firmware:
      Uuid: 5a9fc181-957e-5c32-9e5a-2de5e9673531
    Machine:
      Type: q35
    Resources:
      Requests:
        Memory: 64M
  Networks:
    Name: default
    Pod:
  Volumes:
    Container Disk:
      Image: kubevirt/cirros-registry-disk-demo
      Image Pull Policy: Always
    Name: rootfs
    Cloud Init No Cloud:
      userDataBase64: SGkuXG4=
    Name: cloudinit
Status:
  Active Pods:
    f2afda42-6f8f-4bad-b35a-7591c3233cc1: aks-agentpool-16963728-vmss000000
  Conditions:
    Last Probe Time: 2023-03-09T13:26:14Z
    Last Transition Time: 2023-03-09T13:26:14Z
    Message: Guest VM is not reported as running
    Reason: GuestNotRunning
    Status: False
    Type: Ready
  Guest OS Info:
  Phase: Scheduling
  Phase Transition Timestamps:
    Phase: Pending
    Phase Transition Timestamp: 2023-03-09T13:26:14Z
    Phase: Scheduling
    Phase Transition Timestamp: 2023-03-09T13:26:14Z
  Qos Class: Burstable
  Runtime User: 107
  Virtual Machine Revision Name: revision-start-vm-01de5614-922a-4ae8-9bec-40eeb9a7c500-2
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal SuccessfulCreate 8s virtualmachine-controller Created virtual machine pod virt-launcher-testvm-b77hf

Elle est en exécution dans KubeVirt :

$ kubectl get vmis
NAME AGE PHASE IP NODENAME READY
testvm 61s Running 10.244.2.38 aks-agentpool-16963728-vmss000000 True

$ kubectl get vms
NAME AGE STATUS READY
testvm 3m47s Running True

avec sa console VNC fournie dans KubeVirt Manager :

Bien évidemment, je pouvais également accéder à la console de la machine virtuelle localement :

$ kubectl virt console testvm

testvm login: cirros
Password: gocubsgo

$ ps aux
PID USER COMMAND
    1 root init
    2 root [kthreadd]
    3 root [ksoftirqd/0]
    4 root [kworker/0:0]
    5 root [kworker/0:0H]
    7 root [rcu_sched]
    8 root [rcu_bh]
    9 root [migration/0]
   10 root [watchdog/0]
   11 root [kdevtmpfs]
   12 root [netns]
   13 root [perf]
   14 root [khungtaskd]
   15 root [writeback]
   16 root [ksmd]
   17 root [crypto]
   18 root [kintegrityd]
   19 root [bioset]
   20 root [kblockd]
   21 root [ata_sff]
   22 root [md]
   23 root [devfreq_wq]La sup

L’interruption peut se faire graphiquement ou via la ligne de commande habituelle :

$ kubectl virt stop testvm
VM testvm was scheduled to stop

$ kubectl delete vm testvm
virtualmachine.kubevirt.io "testvm" deleted

Avec les options graphiques, il serait possible de reproduire le schéma de création de machine virtuelle via “Containerized Data Importer (CDI)” …

Experiment with CDI | KubeVirt.io

KubeVirt Manager offre également une intégration avec Prometheus (et donc potentiellement à Grafana).

GitHub - kubevirt-manager/kubevirt-manager: Kubevirt Web UI / Web Interface Manager

Ce projet encore jeune est en toujours développement et des fonctionnalités plus avancées sur le contrôle des machines virtuelles sont à espérer …

À suivre !

Empaqueter JupyterLite pour être déployé sur la plateforme globale d’applications Fly.io …

Karim — Wed, 01 Mar 2023 23:37:04 +0000

Fly.io (que certains présentent comme la principale alternative à Heroku) est, comme beaucoup de plateformes d’infrastructure, un système complexe construit sur un certain nombre de projets open source avec notamment Nomad, Consul et Vault de Hashicorp.

Au cœur de l’exécution des applications se trouve une variante de Firecracker d’Amazon. Et une surveillance avec des tableaux de bord utilisant Prometheus et Grafana.

Load Balancing avec HAProxy, Nomad et Consul …

Fly and Open Source

Fly.io utilise des microVMs Firecracker. Il s’agit de machines virtuelles légères et sécurisées basées sur une virtualisation matérielle forte.

Les machines Fly sont des VM Firecracker dotées d’une API REST rapide qui peuvent démarrer des instances en 300 ms environ, dans toutes les régions prises en charge par Fly.io.

Run User Code on Fly Machines

Une des utilisations des machines est d’exécuter du code utilisateur pour étendre votre service, ou comme un service en soi. On appelle parfois cela “Functions-as-a-Service” (FaaS). L’idée générale est la suivante :

emballer le code des utilisateurs

définir un environnement d’exécution

lancer le code dans une VM

puis éteindre la VM lorsqu’elle est inactive afin d’économiser sur les factures de calcul.

Le tout en facturant les utilisateurs à la demande ! …

Je vais m’interesser ici au projet JupyterLite. JupyterLite est en effet une distribution JupyterLab qui s’exécute entièrement dans le navigateur, construite de A à Z à partir de composants et d’extensions JupyterLab.

JupyterLite - JupyterLite 0.1.0-beta.18 documentation

Bien que JupyterLite soit actuellement développé par les développeurs principaux de Jupyter, le projet est encore non officiel.

How-To Guides - JupyterLite 0.1.0-beta.18 documentation

Pour cela, dans un premier temps, récupération de Mambaforge :

GitHub - conda-forge/miniforge: A conda-forge distribution.

ubuntu@julia50b1000001:~$ wget -c https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
ubuntu@julia50b1000001:~$ chmod +x Mambaforge-Linux-x86_64.sh 
ubuntu@julia50b1000001:~$ ./Mambaforge-Linux-x86_64.sh 

Welcome to Mambaforge 22.11.1-4

In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
>>> 
Miniforge installer code uses BSD-3-Clause license as stated below.
Installing base environment...
Transaction finished
installation finished.
Do you wish the installer to initialize Mambaforge
by running conda init? [yes|no]
[no] >>> yes
no change /home/ubuntu/mambaforge/condabin/conda
no change /home/ubuntu/mambaforge/bin/conda
no change /home/ubuntu/mambaforge/bin/conda-env
no change /home/ubuntu/mambaforge/bin/activate
no change /home/ubuntu/mambaforge/bin/deactivate
no change /home/ubuntu/mambaforge/etc/profile.d/conda.sh
no change /home/ubuntu/mambaforge/etc/fish/conf.d/conda.fish
no change /home/ubuntu/mambaforge/shell/condabin/Conda.psm1
no change /home/ubuntu/mambaforge/shell/condabin/conda-hook.ps1
no change /home/ubuntu/mambaforge/lib/python3.10/site-packages/xontrib/conda.xsh
no change /home/ubuntu/mambaforge/etc/profile.d/conda.csh
modified /home/ubuntu/.bashrc

==> For changes to take effect, close and re-open your current shell. <==

                  ____  ____
                 / \ / \ / \ / \
                / \/ \/ \/ \
███████████████/ /██/ /██/ /██/ /████████████████████████
              / / \ / \ / \ / \ \ ____
             / / \_/ \_/ \_/ \ o \__,
            / _/ \ _____ / `
            |/
        ███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║
        ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝

        mamba (1.1.0) supported by @QuantStack

        GitHub: https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████

no change /home/ubuntu/mambaforge/condabin/conda
no change /home/ubuntu/mambaforge/bin/conda
no change /home/ubuntu/mambaforge/bin/conda-env
no change /home/ubuntu/mambaforge/bin/activate
no change /home/ubuntu/mambaforge/bin/deactivate
no change /home/ubuntu/mambaforge/etc/profile.d/conda.sh
no change /home/ubuntu/mambaforge/etc/fish/conf.d/conda.fish
no change /home/ubuntu/mambaforge/shell/condabin/Conda.psm1
no change /home/ubuntu/mambaforge/shell/condabin/conda-hook.ps1
no change /home/ubuntu/mambaforge/lib/python3.10/site-packages/xontrib/conda.xsh
no change /home/ubuntu/mambaforge/etc/profile.d/conda.csh
no change /home/ubuntu/.bashrc
No action taken.
Added mamba to /home/ubuntu/.bashrc

==> For changes to take effect, close and re-open your current shell. <==

If you'd prefer that conda's base environment not be activated on startup, 
   set the auto_activate_base parameter to false: 

conda config --set auto_activate_base false

Thank you for installing Mambaforge!
ubuntu@julia50b1000001:~$ source .bashrc
(base) ubuntu@julia50b1000001:~$ mamba
usage: mamba [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

Options:

positional arguments:
  command
    clean Remove unused packages and caches.
    compare Compare packages between conda environments.
    config Modify configuration values in .condarc. This is modeled after the git config command. Writes to the user .condarc file (/home/ubuntu/.condarc) by default.
                      Use the --show-sources flag to display all identified configuration locations on your computer.
    create Create a new conda environment from a list of specified packages.
    info Display information about current conda install.
    init Initialize conda for shell interaction.
    install Installs a list of packages into a specified conda environment.
    list List installed packages in a conda environment.
    package Low-level conda package utility. (EXPERIMENTAL)
    remove (uninstall)
                      Remove a list of packages from a specified conda environment.
    rename Renames an existing environment.
    run Run an executable in a conda environment.
    search Search for packages and display associated information.The input is a MatchSpec, a query language for conda packages. See examples below.
    update (upgrade) Updates conda packages to the latest compatible version.
    notices Retrieves latest channel notifications.
    repoquery Query repositories using mamba.

options:
  -h, --help Show this help message and exit.
  -V, --version Show the conda version number and exit.

conda commands available from other packages (legacy):
  env

                                           __
          ________  _______  ________ / /_ ____ _
         / / / / __`__ \/ __`/__ ` __\/__ \/ __ `/
        / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
       / . ___/_/ /_/ /_/\__ ,_/_/ /_/ /_/_. ___/\__ ,_/
      /_/

Transaction

  Prefix: /home/ubuntu/mambaforge

Et installation de JupyterLite avec Pip (il sera bientôt disponible dans conda forge) :

GitHub - jupyterlite/jupyterlite: Wasm powered Jupyter running in the browser 💡

(base) ubuntu@julia50b1000001:~$ python -m pip install --pre jupyterlite

Collecting jupyterlite
  Downloading jupyterlite-0.1.0b18-py3-none-any.whl (7.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.6/7.6 MB 71.3 MB/s eta 0:00:00
Collecting doit<1,>=0.34
  Downloading doit-0.36.0-py3-none-any.whl (85 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.9/85.9 kB 17.4 MB/s eta 0:00:00
Collecting jupyter_core>=4.7
  Downloading jupyter_core-5.2.0-py3-none-any.whl (94 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.3/94.3 kB 19.5 MB/s eta 0:00:00
Collecting cloudpickle
  Downloading cloudpickle-2.2.1-py3-none-any.whl (25 kB)
Collecting importlib-metadata>=4.4
  Downloading importlib_metadata-6.0.0-py3-none-any.whl (21 kB)
Collecting platformdirs>=2.5
  Downloading platformdirs-3.0.0-py3-none-any.whl (14 kB)
Collecting traitlets>=5.3
  Downloading traitlets-5.9.0-py3-none-any.whl (117 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.4/117.4 kB 21.0 MB/s eta 0:00:00
Collecting zipp>=0.5
  Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB)
Installing collected packages: zipp, traitlets, platformdirs, cloudpickle, jupyter_core, importlib-metadata, doit, jupyterlite
Successfully installed cloudpickle-2.2.1 doit-0.36.0 importlib-metadata-6.0.0 jupyter_core-5.2.0 jupyterlite-0.1.0b18 platformdirs-3.0.0 traitlets-5.9.0 zipp-3.15.0

Installation de quelques Kernels en accompagnement :

(base) ubuntu@julia50b1000001:~$ pip install jupyterlite-xeus-python
Collecting jupyterlite-xeus-python
  Downloading jupyterlite_xeus_python-0.6.3-py3-none-any.whl (18.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 4.0 MB/s eta 0:00:00
Requirement already satisfied: jupyterlite in ./mambaforge/lib/python3.10/site-packages (from jupyterlite-xeus-python) (0.1.0b18)
Collecting typer
  Downloading typer-0.7.0-py3-none-any.whl (38 kB)
Collecting empack<3,>=2.0.9
  Downloading empack-2.0.9-py3-none-any.whl (15 kB)
Requirement already satisfied: requests in ./mambaforge/lib/python3.10/site-packages (from jupyterlite-xeus-python) (2.28.2)
Requirement already satisfied: traitlets in ./mambaforge/lib/python3.10/site-packages (from jupyterlite-xeus-python) (5.9.0)
Collecting pyyaml
  Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 kB 67.7 MB/s eta 0:00:00
Collecting networkx
  Downloading networkx-3.0-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 73.2 MB/s eta 0:00:00
Collecting appdirs
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting pydantic
  Downloading pydantic-1.10.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 116.2 MB/s eta 0:00:00
Requirement already satisfied: jupyter_core>=4.7 in ./mambaforge/lib/python3.10/site-packages (from jupyterlite->jupyterlite-xeus-python) (5.2.0)
Requirement already satisfied: doit<1,>=0.34 in ./mambaforge/lib/python3.10/site-packages (from jupyterlite->jupyterlite-xeus-python) (0.36.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./mambaforge/lib/python3.10/site-packages (from requests->jupyterlite-xeus-python) (1.26.14)
Requirement already satisfied: idna<4,>=2.5 in ./mambaforge/lib/python3.10/site-packages (from requests->jupyterlite-xeus-python) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in ./mambaforge/lib/python3.10/site-packages (from requests->jupyterlite-xeus-python) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in ./mambaforge/lib/python3.10/site-packages (from requests->jupyterlite-xeus-python) (2022.12.7)
Collecting click<9.0.0,>=7.1.1
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 19.1 MB/s eta 0:00:00
Requirement already satisfied: cloudpickle in ./mambaforge/lib/python3.10/site-packages (from doit<1,>=0.34->jupyterlite->jupyterlite-xeus-python) (2.2.1)
Requirement already satisfied: importlib-metadata>=4.4 in ./mambaforge/lib/python3.10/site-packages (from doit<1,>=0.34->jupyterlite->jupyterlite-xeus-python) (6.0.0)
Requirement already satisfied: platformdirs>=2.5 in ./mambaforge/lib/python3.10/site-packages (from jupyter_core>=4.7->jupyterlite->jupyterlite-xeus-python) (3.0.0)
Collecting typing-extensions>=4.2.0
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Requirement already satisfied: zipp>=0.5 in ./mambaforge/lib/python3.10/site-packages (from importlib-metadata>=4.4->doit<1,>=0.34->jupyterlite->jupyterlite-xeus-python) (3.15.0)
Installing collected packages: appdirs, typing-extensions, pyyaml, networkx, click, typer, pydantic, empack, jupyterlite-xeus-python
Successfully installed appdirs-1.4.4 click-8.1.3 empack-2.0.9 jupyterlite-xeus-python-0.6.3 networkx-3.0 pydantic-1.10.5 pyyaml-6.0 typer-0.7.0 typing-extensions-4.5.0
(base) ubuntu@julia50b1000001:~$ pip install jupyterlite-p5-kernel
Collecting jupyterlite-p5-kernel
  Downloading jupyterlite_p5_kernel-0.1.1-py3-none-any.whl (21 kB)
Installing collected packages: jupyterlite-p5-kernel
Successfully installed jupyterlite-p5-kernel-0.1.1
(base) ubuntu@julia50b1000001:~$ pip install jupyterlite_xeus_lua
Collecting jupyterlite_xeus_lua
  Downloading jupyterlite_xeus_lua-0.3.3-py3-none-any.whl (2.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 2.6 MB/s eta 0:00:00
Installing collected packages: jupyterlite_xeus_lua
Successfully installed jupyterlite_xeus_lua-0.3.3
(base) ubuntu@julia50b1000001:~$ pip install jupyterlite_xeus_wren
Collecting jupyterlite_xeus_wren
  Downloading jupyterlite_xeus_wren-0.2.1-py3-none-any.whl (4.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 3.9 MB/s eta 0:00:00
Collecting jupyter-wren-syntax
  Downloading jupyter_wren_syntax-0.1.1-py3-none-any.whl (68 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.6/68.6 kB 8.8 MB/s eta 0:00:00
Installing collected packages: jupyter-wren-syntax, jupyterlite_xeus_wren
Successfully installed jupyter-wren-syntax-0.1.1 jupyterlite_xeus_wren-0.2.1

Et génération du contenu statique dans un répertoire contenant JupyterLite et ses extensions via cette commande :

(base) ubuntu@julia50b1000001:~$ jupyter lite build --output-dir dist

(base) ubuntu@julia50b1000001:~$ ls dist/
api config-utils.js icon-120x120.png jupyter-lite.ipynb kernelspecs package.json retro
bootstrap.js doc icon-512x512.png jupyter-lite.json lab piplite.schema.v0.json service-worker-b2fb40a.js
build extensions index.html jupyterlite.schema.v0.json manifest.webmanifest repl tree

Maintenant que les actifs statiques ont été construits, vous pouvez utiliser un serveur HTTP simple pour les servir et accéder à JupyterLite depuis un navigateur web.

La commande jupyter lite serve offre soit un serveur web propulsé par le module http.server intégré de Python, soit Tornado , qui sera probablement disponible si d’autres outils Jupyter sont installés :


(base) ubuntu@julia50b1000001:~$ jupyter lite serve 

Serving JupyterLite Debug Server from:
            /home/ubuntu/dist/_output
        on:
            http://127.0.0.1:8000/index.html

        ***Exit by:***
            - Pressing Ctrl+C

Il ne me reste plus qu’à l’empaqueter pour le déployer dans la plateforme fly.io. Pour cela je m’inspire de la fiche documentaire consacrée au déploiement de contenus statiques :

Run a Static Website

Installation du client flyctl :

(base) ubuntu@julia50b1000001:~$ curl -L https://fly.io/install.sh | sh

  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 1475 0 1475 0 0 6089 0 --:--:-- --:--:-- --:--:-- 6069
######################################################################## 100.0%
set channel to shell
flyctl was installed successfully to /home/ubuntu/.fly/bin/flyctl
Manually add the directory to your $HOME/.bash_profile (or similar)
  export FLYCTL_INSTALL="/home/ubuntu/.fly"
  export PATH="$FLYCTL_INSTALL/bin:$PATH"
Run '/home/ubuntu/.fly/bin/flyctl --help' to get started

(base) ubuntu@julia50b1000001:~$ flyctl auth login

(base) ubuntu@julia50b1000001:~$ flyctl
This is flyctl, the Fly.io command line interface.

It doesn't look like you're logged in. Try "flyctl auth signup" to create an account,
or "flyctl auth login" to log in to an existing account.

flyctl does a lot of stuff! Don't panic, it's easy to get started:

  * fly launch: launch a new application ("fly help launch" for more)

  * fly apps: create and manage apps ("fly help apps")

  * fly machines: create and manage individual Fly.io machines ("fly help machines")

  * fly postgres: create and manage Postgres databases ("fly help postgres")

  * fly redis: create and manage Redis databases ("fly help redis")

  * fly help: for more help, and a complete list of commands.

J’ajoute ce fichier Dockerfile avec ce contenu :

(base) ubuntu@julia50b1000001:~$ ls
Dockerfile dist mambaforge

(base) ubuntu@julia50b1000001:~$ cat Dockerfile 

FROM pierrezemb/gostatic
COPY ./dist/ /srv/http/

Génération du fichier fly.toml qui va servir au déploiement dans la plateforme …

(base) ubuntu@julia50b1000001:~$ flyctl launch

Creating app in /home/ubuntu
Scanning source code
Detected a Dockerfile app
? Choose an app name (leave blank to generate one): 
automatically selected personal organization: Karim
Some regions require a paid plan (fra, maa).
See https://fly.io/plans to set up a plan.

? Choose a region for deployment: Paris, France (cdg)
Created app old-sunset-3102 in organization personal
Admin URL: https://fly.io/apps/old-sunset-3102
Hostname: old-sunset-3102.fly.dev
? Would you like to set up a Postgresql database now? No
? Would you like to set up an Upstash Redis database now? No
? Create .dockerignore from 4 .gitignore files? No
Wrote config file fly.toml
? Would you like to deploy now? No
Your app is ready! Deploy with `flyctl deploy`

Modifié pour prendre en compte le fait que le futur conteneur utilisera le port 8043 par défaut :

(base) ubuntu@julia50b1000001:~$ cat fly.toml 

# fly.toml file generated for old-sunset-3102 on 2023-03-01T22:42:29Z

app = "old-sunset-3102"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]

[experimental]
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8043
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

Lancement du déploiement :

(base) ubuntu@julia50b1000001:~$ flyctl deploy
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-small-water-3057 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 54.5s (0/1)                                                                                                                                                           
[+] Building 84.2s (0/1)                                                                                                                                                           
[+] Building 16.3s (6/6) FINISHED                                                                                                                                                  
 => [internal] load remote build context 0.0s
 => copy /context / 11.6s
 => [internal] load metadata for docker.io/pierrezemb/gostatic:latest 0.7s
 => [1/2] FROM docker.io/pierrezemb/gostatic@sha256:7e5718f98f2172f7c8dffd152ef0b203873ba889c8d838b2e730484fc71f6acd 0.1s
 => => resolve docker.io/pierrezemb/gostatic@sha256:7e5718f98f2172f7c8dffd152ef0b203873ba889c8d838b2e730484fc71f6acd 0.0s
 => => extracting sha256:0cf3c901807f7df57d792cd4a926ac2eb4078eb337750316dbde44bc7e7acd83 0.1s
 => => sha256:0cf3c901807f7df57d792cd4a926ac2eb4078eb337750316dbde44bc7e7acd83 1.88MB / 1.88MB 0.0s
 => => sha256:7e5718f98f2172f7c8dffd152ef0b203873ba889c8d838b2e730484fc71f6acd 2.67kB / 2.67kB 0.0s
 => => sha256:f846dcfe68518bd5a624acb44abee440deedfca894e641b7947ba494f6e0f18a 527B / 527B 0.0s
 => => sha256:37dd3994986381311fdfc59ea190c8e60c6c8c5a38f3cdec3419ee8b333c9fa9 915B / 915B 0.0s
 => [2/2] COPY ./dist/ /srv/http/ 0.3s
 => exporting to image 0.5s
 => => exporting layers 0.5s
 => => writing image sha256:29bf836545e5b679b481077dc137a880d90baa8ca49839b603e5c4a2966133ea 0.0s
 => => naming to registry.fly.io/old-sunset-3102:deployment-01GTFNNV4ZS5S210YX9MPXXCNW 0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/old-sunset-3102]
688585856d25: Pushed 
f347b3d1982a: Pushed 
deployment-01GTFNNV4ZS5S210YX9MPXXCNW: digest: sha256:0c085f048ab642afe7481d6b85c4444b6ac5d3f553717a9527fa53170de9853f size: 740
--> Pushing image done
image: registry.fly.io/old-sunset-3102:deployment-01GTFNNV4ZS5S210YX9MPXXCNW
image size: 91 MB
==> Creating release
--> release v2 created

--> You can detach the terminal anytime without stopping the deployment
==> Monitoring deployment
Logs: https://fly.io/apps/old-sunset-3102/monitoring

 1 desired, 1 placed, 1 healthy, 0 unhealthy [health checks: 1 total, 1 passing]
--> v0 deployed successfully

Cette version v0 est opérationnelle :

Accompagnée de son monitoring :

Et les Kernels sont fonctionnels :

Avec suppression à la fois du volume et du conteneur associé à la fin de cette expérience …

Pour aller plus loin avec JupyterLite, ces prises en charges de Langages et Frameworks :

Language & Framework Guides

Sachant que pour Fly.io, les orchestrateurs relient des grappes de serveurs de travail et offrent une API pour exécuter des tâches sur ces grappes avec HashiCorp Nomad et ces mécanismes exposés dans cet article :

Carving The Scheduler Out Of Our Orchestrator

À Suivre !

Remplacer MetalLB par Cilium BGP pour la mise en œuvre d’un équilibreur de charge dans Kubernetes …

Karim — Wed, 01 Mar 2023 21:32:45 +0000

https://cilium.io/

Cilium est un logiciel open source permettant de sécuriser de manière transparente la connectivité réseau entre les services applicatifs déployés à l’aide de plateformes de gestion de conteneurs Linux comme Docker et Kubernetes.

À la base de Cilium se trouve une nouvelle technologie du noyau Linux appelée eBPF, qui permet l’insertion dynamique d’une puissante logique de visibilité et de contrôle de la sécurité au sein même de Linux. Comme eBPF s’exécute à l’intérieur du noyau Linux, les politiques de sécurité de Cilium peuvent être appliquées et mises à jour sans aucune modification du code de l’application ou de la configuration du conteneur.

Introduction to Cilium & Hubble - Cilium 1.13.0 documentation

"How the Hive Came To Bee" - a story of eBPF and Cilium so far - Isovalent

BGP fournit un moyen d’annoncer des routes à l’aide de protocoles de réseau traditionnels pour permettre aux services gérés par Cilium d’être accessibles en dehors du cluster. Ici on va voir comment configurer la prise en charge native de Cilium pour annoncer les IP des services de l’équilibreur de charge et la plage CIDR de Pod d’un nœud Kubernetes via BGP.

Il s’appuie sur l’implémentation simple et efficace de l’allocation d’IP de MetalLB et sur le support minimal du protocole BGP pour ce faire. La configuration de Cilium est la même que celle de MetalLB. Cette fonctionnalité demeure encore en version beta :

Plus précisément, si un service de type LoadBalancer est créé, Cilium lui allouera une IP à partir d’un pool spécifié. Une fois l’IP allouée, les agents annonceront via BGP en fonction de la ExternalTrafficPolicy du service. Application ici avec le lancement d’une instance dans Linode (maintenant Akamai) avec Ubuntu 22.04 LTS :

Akamai prend le virage du cloud computing - Le Monde Informatique

linode-cli linodes create \
  --image 'linode/ubuntu22.04' \
  --region eu-central \
  --type g7-highmem-1 \
  --label lxdserver\
  --authorized_users $USER\
  --booted true \
  --backups_enabled false \
  --private_ip false

Get Started

Installation de l’hyperviseur LXD sur cette instance Ubuntu 22.04 LTS :

root@localhost:~# snap install lxd

2023-03-01T19:27:40Z INFO Waiting for automatic snapd restart...
lxd 5.11-ad0b61e from Canonical✓ installed

root@localhost:~# lxd init

Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (lvm, zfs, btrfs, ceph, dir) [default=zfs]: dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=lxdbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: none
Would you like the LXD server to be available over the network? (yes/no) [default=no]: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: 
root@localhost:~# lxc ls
To start your first container, try: lxc launch ubuntu:22.04
Or for a virtual machine: lxc launch ubuntu:22.04 --vm

+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

Configuration du profil par défaut dans LXD :

root@localhost:~# ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:Nk0YMZ5JM5i5EMOjZ/YTJ+GiWNTOhv/9jhLtVAz7f8k root@localhost
The key's randomart image is:
+---[RSA 3072]----+
| oo. +B. |
| . =.=o.O |
| . = + o=+. |
| + O = ooo |
| o B o =So. |
|. . . +.o.. |
| . * . . . |
| o o. . E |
| ..oo . |
+----[SHA256]-----+
root@localhost:~# lxc profile show default > lxd-profile-default.yaml

root@localhost:~# cat lxd-profile-default.yaml 
config:
  user.user-data: |
    #cloud-config
    ssh_authorized_keys:
      - @@SSHPUB@@
  environment.http_proxy: ""
  user.network_mode: ""
description: Default LXD profile
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
used_by: []

root@localhost:~# sed -ri "s'@@SSHPUB@@'$(cat ~/.ssh/id_rsa.pub)'" lxd-profile-default.yaml

root@localhost:~# lxc profile edit default < lxd-profile-default.yaml

et d’un profil dédié pour l”installation de Kubernetes :

root@localhost:~# lxc profile create k8s

Profile k8s created
root@localhost:~# wget https://raw.githubusercontent.com/ubuntu/microk8s/master/tests/lxc/microk8s.profile -O microk8s.profile
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 2606:50c0:8000::154, 2606:50c0:8001::154, 2606:50c0:8002::154, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8000::154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 816 [text/plain]
Saving to: ‘microk8s.profile’

microk8s.profile 100%[=============================================================================================>] 816 --.-KB/s in 0s      

root@localhost:~# cat microk8s.profile | lxc profile edit k8s

root@localhost:~# rm microk8s.profile

Lancement de trois instances LXC avec Ubuntu 22.04 LTS et de ce profil k8s :

root@localhost:~# for i in {1..3}; do lxc launch -p default -p k8s ubuntu:22.04 k3s$i; done

Creating k3s1
Starting k3s1                               
Creating k3s2
Starting k3s2                               
Creating k3s3
Starting k3s3    

root@localhost:~# lxc ls

+------+---------+---------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+---------------------+------+-----------+-----------+
| k3s1 | RUNNING | 10.26.23.163 (eth0) | | CONTAINER | 0 |
+------+---------+---------------------+------+-----------+-----------+
| k3s2 | RUNNING | 10.26.23.75 (eth0) | | CONTAINER | 0 |
+------+---------+---------------------+------+-----------+-----------+
| k3s3 | RUNNING | 10.26.23.87 (eth0) | | CONTAINER | 0 |
+------+---------+---------------------+------+-----------+-----------+

Je procède à l’installation d’un cluster k3s conformément à ce que préconise la documentation fournie par Cilium.

La première étape consiste en effet à installer un nœud maître K3s en veillant à désactiver la prise en charge du plugin CNI par défaut et de l’applicateur de politique réseau intégré :

Installation Using K3s - Cilium 1.13.0 documentation

root@localhost:~# ssh ubuntu@10.26.23.163

Warning: Permanently added '10.26.23.163' (ED25519) to the list of known hosts.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@k3s1:~$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--flannel-backend=none --disable-network-policy' sh -

[INFO] Finding release for channel stable
[INFO] Using v1.25.6+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.25.6+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.25.6+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s

Après récupération du node-token , installation des deux workers dans ce cluster k3s :

root@localhost:~# ssh ubuntu@10.26.23.75
Warning: Permanently added '10.26.23.75' (ED25519) to the list of known hosts.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@k3s2:~$ curl -sfL https://get.k3s.io | K3S_URL='https://10.26.23.163:6443' K3S_TOKEN=K100df339c075cfa287710648ed7ba381667496813d04b7603150856c8c673cc07b::server:2300bb645db019e47caaa971f0f8460b sh -

[INFO] Finding release for channel stable
[INFO] Using v1.25.6+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.25.6+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.25.6+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent
ubuntu@k3s2:~$ exit
logout
Connection to 10.26.23.75 closed.

root@localhost:~# ssh ubuntu@10.26.23.87
Warning: Permanently added '10.26.23.87' (ED25519) to the list of known hosts.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@k3s3:~$ curl -sfL https://get.k3s.io | K3S_URL='https://10.26.23.163:6443' K3S_TOKEN=K100df339c075cfa287710648ed7ba381667496813d04b7603150856c8c673cc07b::server:2300bb645db019e47caaa971f0f8460b sh -

[INFO] Finding release for channel stable
[INFO] Using v1.25.6+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.25.6+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.25.6+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent

ubuntu@k3s1:~$ mkdir .kube && sudo cp /etc/rancher/k3s/k3s.yaml .kube/config && sudo chown -R $USER .kube/*

ubuntu@k3s1:~$ curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.25.6/bin/linux/amd64/kubectl && chmod +x kubectl && sudo rm /usr/local/bin/kubectl && sudo mv kubectl /usr/bin

ubuntu@k3s1:~$ kubectl cluster-info

Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

ubuntu@k3s1:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k3s2 NotReady <none> 3m31s v1.25.6+k3s1 10.26.23.75 <none> Ubuntu 22.04.1 LTS 5.15.0-60-generic containerd://1.6.15-k3s1
k3s3 NotReady <none> 3m8s v1.25.6+k3s1 10.26.23.87 <none> Ubuntu 22.04.1 LTS 5.15.0-60-generic containerd://1.6.15-k3s1
k3s1 NotReady control-plane,master 7m19s v1.25.6+k3s1 10.26.23.163 <none> Ubuntu 22.04.1 LTS 5.15.0-60-generic containerd://1.6.15-k3s1

ubuntu@k3s1:~$ kubectl get po,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/metrics-server-5f9f776df5-jzq2f 0/1 Pending 0 7m13s
kube-system pod/local-path-provisioner-79f67d76f8-tg2dj 0/1 Pending 0 7m13s
kube-system pod/coredns-597584b69b-4bg2h 0/1 Pending 0 7m13s
kube-system pod/helm-install-traefik-k9m7j 0/1 Pending 0 7m14s
kube-system pod/helm-install-traefik-crd-4vh8j 0/1 Pending 0 7m14s

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 7m28s
kube-system service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 7m25s
kube-system service/metrics-server ClusterIP 10.43.252.18 <none> 443/TCP 7m24s

Suivi de l’installation de Helm 3 :

Installing Helm

ubuntu@k3s1:~$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
Downloading https://get.helm.sh/helm-v3.11.1-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
ubuntu@k3s1:~$ chmod 400 .kube/config
ubuntu@k3s1:~$ helm ls
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION

suivi de Cilium BGP (beta) …

BGP (beta) - Cilium 1.13.0 documentation

ubuntu@k3s1:~$ helm repo add cilium https://helm.cilium.io/
"cilium" has been added to your repositories

ubuntu@k3s1:~$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "cilium" chart repository
Update Complete. ⎈Happy Helming!⎈

Le support BGP est activé en fournissant la configuration BGP via un ConfigMap et en définissant quelques valeurs avec Helm. Sinon, BGP est désactivé par défaut. Je me base pour cela sur la passerelle fournie dans LXD :

root@localhost:~# ifconfig lxdbr0
lxdbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.26.23.1 netmask 255.255.255.0 broadcast 0.0.0.0
        ether 00:16:3e:a3:ba:e8 txqueuelen 1000 (Ethernet)
        RX packets 8742 bytes 563385 (563.3 KB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 18343 bytes 264191495 (264.1 MB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

root@localhost:~# lxc ls
+------+---------+---------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+---------------------+------+-----------+-----------+
| k3s1 | RUNNING | 10.26.23.163 (eth0) | | CONTAINER | 0 |
+------+---------+---------------------+------+-----------+-----------+
| k3s2 | RUNNING | 10.26.23.75 (eth0) | | CONTAINER | 0 |
+------+---------+---------------------+------+-----------+-----------+
| k3s3 | RUNNING | 10.26.23.87 (eth0) | | CONTAINER | 0 |
+------+---------+---------------------+------+-----------+-----------+

et ce segment d’adresses IP libres :

d’où ce fichier YAML pour la configuration :

ubuntu@k3s1:~$ cat config.yaml 

apiVersion: v1
kind: ConfigMap
metadata:
  name: bgp-config
  namespace: kube-system
data:
  config.yaml: |
    peers:
      - peer-address: 10.26.23.1
        peer-asn: 64512
        my-asn: 64512
    address-pools:
      - name: default
        protocol: bgp
        addresses:
          - 10.26.23.32/27

ubuntu@k3s1:~$ kubectl apply -f config.yaml 
configmap/bgp-config created

Installation du Pod Cilium avec Helm :

ubuntu@k3s1:~$ helm install cilium cilium/cilium --version 1.13.0 \
  --namespace kube-system \
  --set bgp.enabled=true \
  --set bgp.announce.loadbalancerIP=true \
  --set bgp.announce.podCIDR=true
NAME: cilium
LAST DEPLOYED: Wed Mar 1 20:05:18 2023
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.

Your release version is 1.13.0.

For any further help, visit https://docs.cilium.io/en/v1.13/gettinghelp

Mise à jour de cette installation avec l’insertion de Cilium Hubble (avec une interface utilisateur de Hubble pouvant accéder à la carte graphique des services):

ubuntu@k3s1:~$ helm upgrade cilium cilium/cilium --version 1.13.0 \
    --namespace kube-system \
    --set bgp.enabled=true \
    --set bgp.announce.loadbalancerIP=true \
    --set bgp.announce.podCIDR=true \
    --set sctp.enabled=true \
    --set hubble.enabled=true \
    --set hubble.metrics.enabled="{dns,drop,tcp,flow,icmp,http}" \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set hubble.ui.service.type=LoadBalancer \
    --set hubble.relay.service.type=LoadBalancer
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Wed Mar 1 20:09:00 2023
NAMESPACE: kube-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble Relay and Hubble UI.

Your release version is 1.13.0.

For any further help, visit https://docs.cilium.io/en/v1.13/gettinghelp

root@localhost:~# lxc ls
+------+---------+--------------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+--------------------------+------+-----------+-----------+
| k3s1 | RUNNING | 10.26.23.163 (eth0) | | CONTAINER | 0 |
| | | 10.0.2.99 (cilium_host) | | | |
+------+---------+--------------------------+------+-----------+-----------+
| k3s2 | RUNNING | 10.26.23.75 (eth0) | | CONTAINER | 0 |
| | | 10.0.0.117 (cilium_host) | | | |
+------+---------+--------------------------+------+-----------+-----------+
| k3s3 | RUNNING | 10.0.1.16 (cilium_host) | | CONTAINER | 0 |
+------+---------+--------------------------+------+-----------+-----------+

Dès lors, test avec le déploiement encore une fois du sempiternel démonstrateur FC :

ubuntu@k3s1:~$ cat <<EOF | kubectl apply -f -
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fcdemo3
  labels:
    app: fcdemo3
spec:
  replicas: 4
  selector:
    matchLabels:
      app: fcdemo3
  template:
    metadata:
      labels:
        app: fcdemo3
    spec:
      containers:
      - name: fcdemo3
        image: mcas/franceconnect-demo2:latest
        ports:
        - containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: fcdemo-service
spec:
  type: LoadBalancer
  selector:
    app: fcdemo3
  ports:
  - protocol: TCP
    port: 80
    targetPort: 3000
EOF
deployment.apps/fcdemo3 created
service/fcdemo-service created

ubuntu@k3s1:~$ kubectl get po,svc
NAME READY STATUS RESTARTS AGE
pod/fcdemo3-6bff77544b-n8x5j 1/1 Running 0 56s
pod/fcdemo3-6bff77544b-n7shv 1/1 Running 0 56s
pod/fcdemo3-6bff77544b-z6s44 1/1 Running 0 56s
pod/fcdemo3-6bff77544b-wmcrg 1/1 Running 0 56s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 54m
service/fcdemo-service LoadBalancer 10.43.212.198 10.26.23.35 80:30918/TCP 56s

Et il est en effet possible à ce stade d’accéder au démonstrateur avec l’adresse IP fournie avec Cilium BGP :

ubuntu@k3s1:~$ http http://10.26.23.35
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 2740
Content-Type: text/html; charset=utf-8
Date: Wed, 01 Mar 2023 20:39:41 GMT
ETag: W/"ab4-v8wreE8WoQJ/5EqjkvRXgbacjz4"
X-Powered-By: Express
set-cookie: connect.sid=s%3AiC15aGdDdBtHWW7zrIjqNiUw0Kq93-W6.WBQcxUEQIloR1IqQTlWXFRUOijoLNsdvAq4bVcSKOso; Path=/; HttpOnly

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport"
          content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.1/css/bulma.min.css" integrity="sha256-zIG416V1ynj3Wgju/scU80KAEWOsO5rRLfVyRDuOv7Q=" crossorigin="anonymous" />
    <title>Démonstrateur Fournisseur de Service</title>
</head>

<body>
<nav class="navbar" role="navigation" aria-label="main navigation">
    <div class="navbar-start">
        <div class="navbar-brand">
            <a class="navbar-item" href="/">
                <img src="/img/fc_logo_v2.png" alt="Démonstrateur Fournisseur de Service" height="28">
            </a>
        </div>
        <a href="/" class="navbar-item">
            Home
        </a>
    </div>
    <div class="navbar-end">
        <div class="navbar-item">

                <div class="buttons">
                    <a class="button is-light" href="/login">Se connecter</a>
                </div>

        </div>
    </div>
</nav>

<section class="hero is-info is-medium">
    <div class="hero-body">
        <div class="container">
            <h1 class="title">
                Bienvenue sur le démonstrateur de fournisseur de service
            </h1>
            <h2 class="subtitle">
                Cliquez sur "se connecter" pour vous connecter via <strong>FranceConnect</strong>
            </h2>
        </div>
    </div>
</section>

<section class="section is-small">
    <div class="container">
        <h1 class="title">Récupérer vos données via FranceConnect</h1>

        <p>Pour récupérer vos données via <strong>FranceConnect</strong> cliquez sur le bouton ci-dessous</p>
    </div>
</section>
<section class="section is-small">
    <div class="container has-text-centered">
        <!-- FC btn -->
        <a href="/data" class="button is-link">Récupérer mes données via FranceConnect</a>
    </div>
</section>
<footer class="footer custom-content">
    <div class="content has-text-centered">
        <p>
            <a href="https://partenaires.franceconnect.gouv.fr/fcp/fournisseur-service"
               target="_blank"
               alt="lien vers la documentation France Connect">
                <strong>Documentation FranceConnect Partenaires</strong>
            </a>
        </p>
    </div>
</footer>
<!-- This script brings the FranceConnect tools modal which enable "disconnect", "see connection history" and "see FC FAQ" features -->
<script src="https://fcp.integ01.dev-franceconnect.fr/js/franceconnect.js"></script>

</body>
</html>

Pour plus de clarté, exposition via Ngrok :

ngrok - download

ubuntu@k3s1:~$ curl -s https://ngrok-agent.s3.amazonaws.com/ngrok.asc | sudo tee /etc/apt/trusted.gpg.d/ngrok.asc >/dev/null && echo "deb https://ngrok-agent.s3.amazonaws.com buster main" | sudo tee /etc/apt/sources.list.d/ngrok.list && sudo apt update && sudo apt install ngrok

ubuntu@k3s1:~$ ngrok http 10.26.23.35:80

ngrok (Ctrl+C to quit)

Add Okta or Azure to protect your ngrok dashboard with SSO: https://ngrok.com/dashSSO                                                                                              

Session Status online                                                                                                                                               
Account eddi now (Plan: Free)                                                                                                                                
Version 3.1.1                                                                                                                                                
Region Europe (eu)                                                                                                                                          
Latency -                                                                                                                                                    
Web Interface http://127.0.0.1:4040                                                                                                                                
Forwarding https://f1e1-172-105-130-186.eu.ngrok.io -> http://10.26.23.35:80                                                                                    

Connections ttl opn rt1 rt5 p50 p90                                                                                                          
                              0 0 0.00 0.00 0.00 0.00

Un petit regard côté observabilité dans Cilium avec Hubble et son interface graphique fournie également en mode LoadBalancer :

ubuntu@k3s1:~$ kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 72m
kube-system kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 72m
kube-system metrics-server ClusterIP 10.43.252.18 <none> 443/TCP 72m
kube-system hubble-peer ClusterIP 10.43.92.181 <none> 443/TCP 49m
kube-system traefik LoadBalancer 10.43.155.56 10.26.23.32 80:30264/TCP,443:31266/TCP 48m
kube-system hubble-relay LoadBalancer 10.43.128.151 10.26.23.33 80:31405/TCP 45m
kube-system hubble-metrics ClusterIP None <none> 9965/TCP 45m
kube-system hubble-ui LoadBalancer 10.43.66.116 10.26.23.34 80:30183/TCP 45m
default fcdemo-service LoadBalancer 10.43.212.198 10.26.23.35 80:30918/TCP 19m 

root@localhost:~# ssh -L 0.0.0.0:80:10.43.66.116:80 ubuntu@10.26.23.163

Il faut noter qu’il est également offert la possibilité de configurer l’équilibrage de charge par proxy pour les services Kubernetes en utilisant Cilium.

Ce qui est utile pour des cas d’utilisation tels que l’équilibrage de charge gRPC.

Une fois activé, le trafic vers un service Kubernetes sera redirigé vers un proxy Envoy géré par Cilium pour l’équilibrage de charge. Cette fonctionnalité est indépendante de la fonctionnalité Kubernetes Ingress Support (Proxy Load Balancing for Kubernetes Services (beta)) :

Proxy Load Balancing for Kubernetes Services (beta) - Cilium 1.13.0 documentation

Test avec ce déploiement comprenant notamment des charges de travail de test consistant en un client de déploiement et un service echo-service avec deux Pods en backend :

ubuntu@k3s1:~$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes/servicemesh/envoy/test-application-proxy-loadbalancing.yaml
configmap/coredns-configmap created
deployment.apps/client created
deployment.apps/echo-service created
service/echo-service created

ubuntu@k3s1:~$ watch -c kubectl get po,svc -A
ubuntu@k3s1:~$ CLIENT=$(kubectl get pods -l name=client -o jsonpath='{.items[0].metadata.name}')
ubuntu@k3s1:~$ kubectl exec -it $CLIENT -- curl -v echo-service:8080/

* Trying 10.43.199.92:8080...
* Connected to echo-service (10.43.199.92) port 8080 (#0)
> GET / HTTP/1.1
> Host: echo-service:8080
> User-Agent: curl/7.83.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Vary: Origin, Accept-Encoding
< Access-Control-Allow-Credentials: true
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Last-Modified: Wed, 21 Sep 2022 10:25:56 GMT
< ETag: W/"809-1835f952f20"
< Content-Type: text/html; charset=UTF-8
< Content-Length: 2057
< Date: Wed, 01 Mar 2023 21:12:11 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< 
<html>
  <head>
    <link
      rel="stylesheet"
      href="https://use.fontawesome.com/releases/v5.8.2/css/all.css"
      integrity="sha384-oS3vJWv+0UjzBfQzYUhtDYW+Pj2yciDJxpsK1OYPAYjqT085Qq/1cq5FLXAZQ7Ay"
      crossorigin="anonymous"
    />
    <link rel="stylesheet" href="style.css" />
    <title>JSON Server</title>
  </head>

  <body>
    <header>
      <div class="container">
        <nav>
          <ul>
            <li class="title">
              JSON Server
            </li>
            <li>
              <a href="https://github.com/users/typicode/sponsorship">
                <i class="fas fa-heart"></i>GitHub Sponsors
              </a>
            </li>
            <li>
              <a href="https://my-json-server.typicode.com">
                <i class="fas fa-burn"></i>My JSON Server
              </a>
            </li>
            <li>
              <a href="https://thanks.typicode.com">
                <i class="far fa-laugh"></i>Supporters
              </a>
            </li>
          </ul>
        </nav>
      </div>
    </header>
    <main>
      <div class="container">
        <h1>Congrats!</h1>
        <p>
          You're successfully running JSON Server
          <br />
          ✧*｡٩(ˊᗜˋ*)و✧*｡
        </p>

        <div id="resources"></div>

        <p>
          To access and modify resources, you can use any HTTP method:
        </p>
        <p>
          <code>GET</code>
          <code>POST</code>
          <code>PUT</code>
          <code>PATCH</code>
          <code>DELETE</code>
          <code>OPTIONS</code>
        </p>

        <div id="custom-routes"></div>

        <h1>Documentation</h1>
        <p>
          <a href="https://github.com/typicode/json-server">
            README
          </a>
        </p>
      </div>
    </main>

    <footer>
      <div class="container">
        <p>
          To replace this page, create a
          <code>./public/index.html</code> file.
        </p>
      </div>
    </footer>

    <script src="script.js"></script>
  </body>
</html>
* Connection #0 to host echo-service left intact

L’ajout d’une politique en couche 7 introduit le proxy Envoy dans le chemin de ce trafic. La requête passant maintenant par le proxy Envoy et se dirigeant ensuite vers le backend …

ubuntu@k3s1:~$ kubectl annotate service echo-service service.cilium.io/lb-l7=enabled
service/echo-service annotated

ubuntu@k3s1:~$ kubectl exec -it $CLIENT -- curl -v echo-service:8080/
* Trying 10.43.199.92:8080...
* Connected to echo-service (10.43.199.92) port 8080 (#0)
> GET / HTTP/1.1
> Host: echo-service:8080
> User-Agent: curl/7.83.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Vary: Origin, Accept-Encoding
< Access-Control-Allow-Credentials: true
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Last-Modified: Wed, 21 Sep 2022 10:25:56 GMT
< ETag: W/"809-1835f952f20"
< Content-Type: text/html; charset=UTF-8
< Content-Length: 2057
< Date: Wed, 01 Mar 2023 21:12:33 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< 
<html>
  <head>
    <link
      rel="stylesheet"
      href="https://use.fontawesome.com/releases/v5.8.2/css/all.css"
      integrity="sha384-oS3vJWv+0UjzBfQzYUhtDYW+Pj2yciDJxpsK1OYPAYjqT085Qq/1cq5FLXAZQ7Ay"
      crossorigin="anonymous"
    />
    <link rel="stylesheet" href="style.css" />
    <title>JSON Server</title>
  </head>

  <body>
    <header>
      <div class="container">
        <nav>
          <ul>
            <li class="title">
              JSON Server
            </li>
            <li>
              <a href="https://github.com/users/typicode/sponsorship">
                <i class="fas fa-heart"></i>GitHub Sponsors
              </a>
            </li>
            <li>
              <a href="https://my-json-server.typicode.com">
                <i class="fas fa-burn"></i>My JSON Server
              </a>
            </li>
            <li>
              <a href="https://thanks.typicode.com">
                <i class="far fa-laugh"></i>Supporters
              </a>
            </li>
          </ul>
        </nav>
      </div>
    </header>
    <main>
      <div class="container">
        <h1>Congrats!</h1>
        <p>
          You're successfully running JSON Server
          <br />
          ✧*｡٩(ˊᗜˋ*)و✧*｡
        </p>

        <div id="resources"></div>

        <p>
          To access and modify resources, you can use any HTTP method:
        </p>
        <p>
          <code>GET</code>
          <code>POST</code>
          <code>PUT</code>
          <code>PATCH</code>
          <code>DELETE</code>
          <code>OPTIONS</code>
        </p>

        <div id="custom-routes"></div>

        <h1>Documentation</h1>
        <p>
          <a href="https://github.com/typicode/json-server">
            README
          </a>
        </p>
      </div>
    </main>

    <footer>
      <div class="container">
        <p>
          To replace this page, create a
          <code>./public/index.html</code> file.
        </p>
      </div>
    </footer>

    <script src="script.js"></script>
  </body>
</html>
* Connection #0 to host echo-service left intact

Cilium et kube-router coopèrent pour utiliser kube-router pour le peering BGP et la propagation des routes et Cilium pour l’application des politiques et l’équilibrage des charges.

Using Kube-Router to Run BGP - Cilium 1.13.0 documentation

ou de l’utilisation conjointe de BIRD :

Using BIRD to run BGP - Cilium 1.13.0 documentation

Le routage à travers les clusters Kubernetes est offert avec Cluster Mesh :

Topology Aware Routing across Clusters with Cluster Mesh

Récapitulatif des nouveautés induites dans la dernière version de Cilium et quelques liens supplémentaires …

Cilium 1.13 - Gateway API, mTLS datapath, Service Mesh, BIG TCP, SBOM, SNI NetworkPolicy, ... - Isovalent