Forem: Robert Wang

Unleash the Power of RAG - Building Intelligent Apps with Chroma and Gemini Pro

Robert Wang — Sat, 24 Aug 2024 15:22:42 +0000

Introduction

The field of Generative AI (GenAI) is rapidly evolving, offering exciting possibilities for applications that interact with and understand human language. One promising approach is Retriever-Augmented Generation (RAG), which combines information retrieval with powerful language models. This blog delves into building a RAG application using Chroma, a vector database, and Gemini Pro, a state-of-the-art large language model from Google.

Chroma: The Key to Efficient Information Retrieval

Chroma serves as the foundation for our RAG application. It's a vector database designed specifically for storing and retrieving vast amounts of data represented as numerical vectors or embeddings. These embeddings capture the semantic meaning of text, enabling Chroma to perform rapid searches based on similarity, rather than exact keyword matches.

Why Chroma is Crucial for GenAI Applications

GenAI applications, like chatbots, question-answering systems, and recommendation engines, rely heavily on processing and understanding information quickly and accurately. Chroma plays a critical role in this process by offering several advantages:

Efficient Information Retrieval: Chroma's vector-based search allows for rapid retrieval of relevant information based on semantic similarity. This is crucial for providing accurate and contextually relevant responses.
Knowledge Base Management: GenAI models often require access to a large knowledge base. Chroma can efficiently store and manage this knowledge, making it readily available for the model to access.
Real-time Performance: Chroma is optimized for speed, enabling real-time interactions with users. This is essential for applications that require immediate responses, such as chatbots or virtual assistants.
Scalability: As the amount of data grows, Chroma can scale to handle increasing workloads, ensuring the continued performance of the GenAI application.

In essence, Chroma empowers GenAI applications to deliver exceptional performance and user experiences by effectively managing and accessing information.

Building a RAG App with Gemini Pro API

Now, let's get hands-on and build a RAG application using Chroma and Gemini Pro. We'll cover the installation process, explore code examples, and see how these tools work together.

Installation

!pip install -U -q google-generativeai   
!pip install -q chromadb

Import packages

import textwrap
import chromadb
import numpy as np
import pandas as pd

import google.generativeai as genai

from IPython.display import Markdown
from chromadb import Documents, EmbeddingFunction, Embeddings

Setup API key
Put Gemini API key in .env under current folder,

GOOGLE_API_KEY=xxx

import os

from dotenv import load_dotenv

load_dotenv()

genai.configure(api_key=os.environ["GOOGLE_API_KEY"])

list supported embedding models

for m in genai.list_models():
  if 'embedContent' in m.supported_generation_methods:
    print(m.name)

models/embedding-001
models/text-embedding-004

Sample data

We use documents list to hold reference document, each item is a string, represents one short text document or chunk of long text document.

DOCUMENT1 = "Gemini is the result of large-scale collaborative efforts by teams across Google, including our colleagues at Google Research. It was built from the ground up to be multimodal, which means it can generalize and seamlessly understand, operate across and combine different types of information including text, code, audio, image and video."
DOCUMENT2 = "We designed Gemini to be natively multimodal, pre-trained from the start on different modalities. Then we fine-tuned it with additional multimodal data to further refine its effectiveness. This helps Gemini seamlessly understand and reason about all kinds of inputs from the ground up, far better than existing multimodal models — and its capabilities are state of the art in nearly every domain."
DOCUMENT3 = "Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment."
documents = [DOCUMENT1, DOCUMENT2, DOCUMENT3]
documents

    ['Gemini is the result of large-scale collaborative efforts by teams across Google, including our colleagues at Google Research. It was built from the ground up to be multimodal, which means it can generalize and seamlessly understand, operate across and combine different types of information including text, code, audio, image and video.',
     'We designed Gemini to be natively multimodal, pre-trained from the start on different modalities. Then we fine-tuned it with additional multimodal data to further refine its effectiveness. This helps Gemini seamlessly understand and reason about all kinds of inputs from the ground up, far better than existing multimodal models — and its capabilities are state of the art in nearly every domain.',
     'Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment.']

Embeddings with model embedding-001

class GeminiEmbeddingFunction(EmbeddingFunction):
  def __call__(self, input: Documents) -> Embeddings:
    model = 'models/embedding-001'
    title = "Custom query"
    return genai.embed_content(model=model,
                                content=input,
                                task_type="retrieval_document",
                                title=title)["embedding"]

This is a custom class named GeminiEmbeddingFunction that implements the EmbeddingFunction interface. This class is designed to interact with the Gemini language model to generate embeddings for text documents.

Class Definition:

GeminiEmbeddingFunction(EmbeddingFunction): This line declares the class GeminiEmbeddingFunction and indicates that it inherits from the EmbeddingFunction class. This means it must implement the __call__ method.

__call__ Method:

def __call__(self, input: Documents) -> Embeddings: This is the method that is called when an instance of the class is used as a function. It takes an input parameter of type Documents (which is presumably a data structure representing a list of documents) and returns an Embeddings object.

Model Selection:

model = 'models/embedding-001': This line sets the model variable to the string 'models/embedding-001'. This likely refers to a specific embedding model within the Gemini language model framework.

Title Setting:

title = "Custom query": This line sets the title variable to the string "Custom query". This might be used as a metadata field for the embedding.

Embedding Generation:

return genai.embed_content(model=model, content=input, task_type="retrieval_document", title=title)["embedding"]: This line calls the genai.embed_content function from the genai module. It passes the following arguments:
- model: The selected embedding model ('models/embedding-001').
- content: The input documents to be embedded.
- task_type: Specifies the task for which the embeddings are being generated. In this case, it's set to "retrieval_document", indicating that the embeddings will be used for document retrieval.
- title: The custom title for the embedding.
The function returns a dictionary containing the embeddings and other metadata. The code extracts the 'embedding' key from this dictionary and returns it as the result of the call method.

We define a function that can be used to generate embeddings for a list of documents using the Gemini language model. The embeddings can then be used for various tasks, such as semantic search or document similarity calculations.

Create chroma db

def create_chroma_db(documents, name):
  chroma_client = chromadb.Client()
  db = chroma_client.create_collection(name=name, embedding_function=GeminiEmbeddingFunction())

  for i, d in enumerate(documents):
    db.add(
      documents=d,
      ids=str(i)
    )
  return db

Setup vector db

list and clean up old db

client = chromadb.Client()

collections = client.list_collections
client.list_collections()

    [Collection(name=geminidb)]

for collection_name in (collection.name for collection in collections()):
    print(collection_name.strip())

    geminidb

for collection_name in (collection.name for collection in collections()):
    if collection_name.strip() == "geminidb":
        print("Collection 'geminidb' exists, will remove it.")
        client.delete_collection(name="geminidb")
    else:
        print("Collection 'geminidb' does not exist.")

    Collection 'geminidb' exists, will remove it.

client.list_collections()

[]

db = create_chroma_db(documents, "geminidb")

Verify db

You might find embeddings is None from db.get(). When using get or query you can use the include parameter to specify which data you want returned - any of embeddings, documents, metadatas, and for query, distances. By default, Chroma will return the documents, metadatas and in the case of query, the distances of the results. embeddings are excluded by default for performance and the ids are always returned.

db.get(include=['embeddings', 'documents', 'metadatas'])

{'ids': ['0', '1', '2'],
 'embeddings': [[0.04447142407298088,
   -0.052165351808071136,
   -0.061552658677101135,
......
   -0.04504038393497467,
   0.005292229354381561]],
 'metadatas': [None, None, None],
 'documents': ['Gemini is the result of large-scale collaborative efforts by teams across Google, including our colleagues at Google Research. It was built from the ground up to be multimodal, which means it can generalize and seamlessly understand, operate across and combine different types of information including text, code, audio, image and video.',
  'We designed Gemini to be natively multimodal, pre-trained from the start on different modalities. Then we fine-tuned it with additional multimodal data to further refine its effectiveness. This helps Gemini seamlessly understand and reason about all kinds of inputs from the ground up, far better than existing multimodal models — and its capabilities are state of the art in nearly every domain.',
  'Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment.'],
 'uris': None,
 'data': None}

db.get(include=['embeddings', 'documents', 'metadatas'])['embeddings']

pd.DataFrame(db.peek(3))

ids embeddings  metadatas   documents   uris    data
0   0   [0.04447142407298088, -0.052165351808071136, -...   None    Gemini is the result of large-scale collaborat...   None    None
1   1   [0.05980474501848221, -0.05400879308581352, -0...   None    We designed Gemini to be natively multimodal, ...   None    None
2   2   [0.017100121825933456, -0.06025487929582596, -...   None    Gemini has the most comprehensive safety evalu...   None    None

Getting the relevant document

def get_relevant_passage(query, db):
  passage = db.query(query_texts=[query], n_results=1)['documents'][0][0]
  return passage

Perform embedding search

passage = get_relevant_passage("safety", db)
Markdown(passage)

Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment.

Make a prompt

def make_prompt(query, relevant_passage):
  escaped = relevant_passage.replace("'", "").replace('"', "").replace("\n", " ")
  prompt = ("""You are a helpful and informative bot that answers questions using text from the reference passage included below. \
  Be sure to respond in a complete sentence, being comprehensive, including all relevant background information. \
  However, you are talking to a non-technical audience, so be sure to break down complicated concepts and \
  strike a friendly and converstional tone. \
  If the passage is irrelevant to the answer, you may ignore it.
  QUESTION: '{query}'
  PASSAGE: '{relevant_passage}'

    ANSWER:
  """).format(query=query, relevant_passage=escaped)

  return prompt

Pass a query to the prompt

query = "what is safety evaluations for gemini?"
prompt = make_prompt(query, passage)
Markdown(prompt)

You are a helpful and informative bot that answers questions using text from the reference passage included below.   Be sure to respond in a complete sentence, being comprehensive, including all relevant background information.   However, you are talking to a non-technical audience, so be sure to break down complicated concepts and   strike a friendly and converstional tone.   If the passage is irrelevant to the answer, you may ignore it.
  QUESTION: 'what is safety evaluations for gemini?'
  PASSAGE: 'Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment.'

    ANSWER:

Generate a response

model = genai.GenerativeModel('gemini-pro')
answer = model.generate_content(prompt)
Markdown(answer.text)

Gemini, a Google AI model, has undergone rigorous safety evaluations to ensure its responsible use. These evaluations encompass a wide range of potential risks, such as bias, toxicity, and even the possibility of misuse for offensive or manipulative purposes. To mitigate these risks, Google Research has employed advanced adversarial testing techniques to identify and address critical safety issues before Gemini's deployment.

Build RAG chatbot using Chroma - wraping to class for easy integration

GeminiEmbeddingFunction: This class defines a custom function to generate embeddings for the input documents using Google's generative AI model.

RAGChatBot: This class represents the core of the chatbot functionality.

init: Initializes various properties like environment path, database name, list of documents, subject, embedding function, query, passage, model name, model object, chat history, and a ChromaDB client object.
create_chroma_db: Creates a ChromaDB collection named geminidb (or deletes an existing one) and adds the provided documents with their IDs.
get_relevant_passage: Retrieves the most relevant document (passage) from the ChromaDB collection based on the user's query.
make_prompt: Constructs a formatted prompt for the generative model, including the user's question, the retrieved passage, and other information.
log_chat_history: Saves the chat history to a text file with a timestamp.
ragchat: The main function that drives the chatbot interaction:

Main Execution:

Creates an instance of the RAGChatBot class.
Defines sample documents and sets the subject.
Assigns the documents and subject to the chatbot instance.
Calls the ragchat function on the chatbot instance to initiate the conversation.

We implement a Retrieval-Augmented Generation (RAG) based chatbot that utilizes Google's generative AI model to answer user questions in a comprehensive and informative way, considering relevant information from a provided document set

# gemini-rag-chatbot.py
# - Generative AI Gemini Chatbot
# - RAG Chatbot using ChromaDB
# @robertluwang
# Aug 2024

import textwrap
import chromadb
import numpy as np
import pandas as pd
import datetime

import google.generativeai as genai

from chromadb import Documents, EmbeddingFunction, Embeddings

import os

from dotenv import load_dotenv

class GeminiEmbeddingFunction(EmbeddingFunction):
    def __call__(self, input: Documents) -> Embeddings:
        model = 'models/embedding-001'
        title = "Custom query"
        return genai.embed_content(model=model,
                                   content=input,
                                   task_type="retrieval_document",
                                   title=title)["embedding"]
class RAGChatBot:
    def __init__(self):
        self.envpath = '~'
        self.envfile = '.env'

        if self.envpath == '~':
            self.envpath = os.path.expanduser("~")

        load_dotenv(os.path.join(self.envpath, self.envfile))

        genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

        self.dbname = 'geminidb'
        self.db = None
        self.documents = []
        self.subject = ''
        self.embedding_function = GeminiEmbeddingFunction()
        self.query = ''
        self.passage = ''
        self.model_name = 'gemini-1.5-flash'
        self.model = genai.GenerativeModel(self.model_name)
        self.chat_history = []

        self.chroma_client = chromadb.Client() 

    def create_chroma_db(self):
        collections = self.chroma_client.list_collections
        for collection_name in (collection.name for collection in collections()):
            if collection_name.strip() == self.dbname:
                #print(f"Collection '{self.dbname}' exists, will remove it.")
                self.chroma_client.delete_collection(name=self.dbname)

        self.db = self.chroma_client.create_collection(name=self.dbname, embedding_function=self.embedding_function)

        for i, d in enumerate(self.documents):
            self.db.add(
              documents=d,
              ids=str(i)
            )
        return self.db

    def get_relevant_passage(self):
      self.passage = self.db.query(query_texts=[self.query], n_results=1)['documents'][0][0]
      return self.passage

    def make_prompt(self):
      escaped = self.passage.replace("'", "").replace('"', "").replace("\n", " ")
      prompt = ("""You are a helpful and informative bot that answers questions using text from the reference passage included below.\
      Be sure to respond in a complete sentence, being comprehensive, including all relevant background information.\
      However, you are talking to a non-technical audience, so be sure to break down complicated concepts and\
      strike a friendly and converstional tone.\
      If the passage is irrelevant to the answer, you may ignore it.
      QUESTION: '{query}'
      PASSAGE: '{passage}'
      ANSWER:
      """).format(query=self.query, passage=escaped)

      return prompt

    def log_chat_history(self,logpath):
        timestamp = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
        log_filename = f"chat-log-{timestamp}.txt"
        log_path = os.path.join(logpath, log_filename)

        os.makedirs(logpath, exist_ok=True)

        with open(log_path, "w") as f:
            for message in self.chat_history:
                f.write(f"{message}\n")

        print(f"chat log file: {log_path}")

    def ragchat(self):

        self.db = self.create_chroma_db()

        n=1 # input number
        print("Welcome to Gemini RAG Chatbot ! ('/q' to exit)")
        self.chat_history.append(f"Welcome to Gemini RAG Chatbot ! ('/q' to exit)")
        print(f"Based on doc set of subject: '{self.subject}'\n")
        self.chat_history.append(f"Based on doc set of subject: '{self.subject}'\n")
        while True:
            self.query = input(f"{n} You: ")

            if not self.query:  # Check if input is empty (only Enter pressed)
                continue  # Skip processing empty input

            self.chat_history.append(f"{n} You: {self.query}\n")

            if self.query.lower() == "/q":
                self.log_chat_history('./log')
                print("Chat history saved. Exiting.")
                break

            self.passage = self.get_relevant_passage()

            prompt = self.make_prompt()
            print(f"prompt: {prompt}\n")

            response = self.model.generate_content(prompt)
            print(f"{n} RAG Chatbot: {response.text}")
            self.chat_history.append(f"{n} RAG Chatbot: {response.text}")
            n += 1

if __name__ == "__main__":
    ragchatbot = RAGChatBot()

    #ragchatbot.model_name = "gemini-1.5-flash"
    #ragchatbot.dbname = "geminidb"
    DOCUMENT1 = "Gemini is the result of large-scale collaborative efforts by teams across Google, including our colleagues at Google Research. It was built from the ground up to be multimodal, which means it can generalize and seamlessly understand, operate across and combine different types of information including text, code, audio, image and video."
    DOCUMENT2 = "We designed Gemini to be natively multimodal, pre-trained from the start on different modalities. Then we fine-tuned it with additional multimodal data to further refine its effectiveness. This helps Gemini seamlessly understand and reason about all kinds of inputs from the ground up, far better than existing multimodal models — and its capabilities are state of the art in nearly every domain."
    DOCUMENT3 = "Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment."
    ragchatbot.documents = [DOCUMENT1, DOCUMENT2, DOCUMENT3]
    ragchatbot.subject = 'Gemini intro'

    ragchatbot.ragchat()

    Welcome to Gemini RAG Chatbot ! ('/q' to exit)
    Based on doc set of subject: 'Gemini intro'


    1 You:  safety


    prompt: You are a helpful and informative bot that answers questions using text from the reference passage included below.      Be sure to respond in a complete sentence, being comprehensive, including all relevant background information.      However, you are talking to a non-technical audience, so be sure to break down complicated concepts and      strike a friendly and converstional tone.      If the passage is irrelevant to the answer, you may ignore it.
          QUESTION: 'safety'
          PASSAGE: 'Gemini has the most comprehensive safety evaluations of any Google AI model to date, including for bias and toxicity. We’ve conducted novel research into potential risk areas like cyber-offense, persuasion and autonomy, and have applied Google Research’s best-in-class adversarial testing techniques to help identify critical safety issues in advance of Gemini’s deployment.'
          ANSWER:


    1 RAG Chatbot: Gemini has undergone extensive safety evaluations, more than any other Google AI model before it, to ensure its responsible use. 


    2 You:  /q


    chat log file: ./log/chat-log-2024-08-25_00-13-03.txt
    Chat history saved. Exiting.

Conclusion

This blog has demonstrated how Chroma and Gemini Pro can be combined to create a powerful RAG application. By leveraging Chroma's efficient information retrieval capabilities and Gemini Pro's advanced language understanding, we can build intelligent applications that can answer questions, generate creative text formats, and provide informative summaries. As GenAI technology continues to evolve, the potential for building even more sophisticated and user-friendly applications becomes increasingly exciting.

About Me

Hey! I am Robert Wang, live in Montreal.

More simple and more efficient.

Give Your Chatbot a Brain - Contextual AI with Python

Robert Wang — Sun, 18 Aug 2024 17:45:12 +0000

Introduction

Imagine a conversation that feels more like talking to a friend and less like interacting with a machine. That's the power of a chatbot with a memory! In today's digital world, chatbots are everywhere, from answering customer service questions to providing companionship. But what truly sets a great chatbot apart? The ability to remember past conversations and tailor responses accordingly. This is where the magic of contextual chatbots comes in.

The most of popular LLM already supports contextual chatbot by including chat history, such as OpenAI and Gemini Pro, but not all of them. As education purpose, we implment a simple chat history chatbot using Gemini Pro.

This tutorial will guide you through building a secure and contextual Gemini chatbot using Python. We'll leverage the power of Google's generative AI library (google.generativeai) to create chat sessions that chatbot will remember. By incorporating memory into your chatbot, you'll design a more engaging and helpful experience.

Gemini chat history demo

import google.generativeai as genai
import os
import datetime
from dotenv import load_dotenv
from os.path import expanduser


envpath = '~'
envfile = '.env'

if envpath == '~':
    envpath = os.path.expanduser("~")

load_dotenv(os.path.join(envpath, envfile))

genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

modelname = "gemini-1.5-flash"
model = genai.GenerativeModel(modelname)

chat = model.start_chat(history=[])

response = chat.send_message('tell me a joke about AI')
print(response.text) 

response = chat.send_message('tell me another joke about AI')
print(response.text)

    Why did the AI get fired from its job at the library? 

    Because it was always getting lost in the stacks! 📚🤖 

    Why did the AI cross the road? 

    To get to the other *side* of the algorithm! 🤖

print(chat.history)

    [parts {
      text: "tell me a joke about AI"
    }
    role: "user"
    , parts {
      text: "Why did the AI get fired from its job at the library? \n\nBecause it was always getting lost in the stacks! 📚🤖 \n"
    }
    role: "model"
    , parts {
      text: "tell me a joke about AI"
    }
    role: "user"
    , parts {
      text: "Why did the AI cross the road? \n\nTo get to the other *side* of the algorithm! 🤖 \n"
    }
    role: "model"
    ]

From below test, we can see Gemini has memory of previous conversatoin.

response = chat.send_message('what was 1st joke you told me?')

print(response.text)

    You're right! I'm still under development, and I sometimes forget things like past conversations.  

    The first joke I told you was: 

    "Why did the AI get fired from its job at the library? Because it was always getting lost in the stacks!" 📚🤖 

    Let me know if you'd like to hear another one! 😄

for part in (chat.history):
    print(part.parts[0])

    text: "tell me a joke about AI"

    text: "Why did the AI get fired from its job at the library? \n\nBecause it was always getting lost in the stacks! 📚🤖 \n"

    text: "tell me a joke about AI"

    text: "Why did the AI cross the road? \n\nTo get to the other *side* of the algorithm! 🤖 \n"

    text: "what was 1st joke you told me?"

    text: "You\'re right! I\'m still under development, and I sometimes forget things like past conversations.  \n\nThe first joke I told you was: \n\n\"Why did the AI get fired from its job at the library? Because it was always getting lost in the stacks!\" 📚🤖 \n\nLet me know if you\'d like to hear another one! 😄 \n"

Let's create chat history feature without using default Gemini history[], this concept and approach will apply for any LLM.

Building the Chatbot Class

Class Definition and Initialization

import google.generativeai as genai
import os
import datetime
from dotenv import load_dotenv
from os.path import expanduser

class GeminiChatbot:
    def __init__(self):
        # Load the .env file from the home directory
        self.envpath = '~'
        self.envfile = '.env'

        if self.envpath == '~':
            self.envpath = os.path.expanduser("~")

        load_dotenv(os.path.join(self.envpath, self.envfile))

        genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

        self.modelname = "gemini-1.5-flash"
        self.model = genai.GenerativeModel(self.modelname)

        self.chat_history = []

Class Definition: The GeminiChatbot class encapsulates the chatbot's functionality.
.env File Loading: By default, the load_dotenv function loads the .env file from the home directory, securely storing the API key. You can change to any envpath and envfile.
API Key Configuration: The os.getenv("GOOGLE_API_KEY") retrieves the API key from the environment variable and configures the Gemini API.
Model Initialization: By default, a GenerativeModel instance is created using the gemini-1.5-flash model, you can change to any Gemini model.
Chat History Initialization: An empty list is created to store the chat history.

chatbot = GeminiChatbot()
user_input = ("tell a joke about AI")
chatbot.chat_history.append(f"You: {user_input}")
response = chatbot.model.generate_content(user_input)
chatbot.chat_history.append(f"Chatbot: {response.text}")
print(response.text)
print(chatbot.chat_history)

    Why did the AI get fired from the dating app?

    Because it couldn't tell the difference between a "like" and a "dislike," and kept matching people with their worst enemies! 

    ['You: tell a joke about AI', 'Chatbot: Why did the AI get fired from the dating app?\n\nBecause it couldn\'t tell the difference between a "like" and a "dislike," and kept matching people with their worst enemies! \n']

Generating Response

import google.generativeai as genai
import os
import datetime
from dotenv import load_dotenv
from os.path import expanduser

class GeminiChatbot:
    def __init__(self):
        # Load the .env file from the home directory
        self.envpath = '~'
        self.envfile = '.env'

        if self.envpath == '~':
            self.envpath = os.path.expanduser("~")

        load_dotenv(os.path.join(self.envpath, self.envfile))

        genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

        self.modelname = "gemini-1.5-flash"
        self.model = genai.GenerativeModel(self.modelname)

        self.chat_history = []

    def generate_response(self, prompt):
        full_prompt = "Please go through chat history below if user ask question regarding on previous conversation.\nPlease anwser question directly if it is not related to previous conversation\n" + "+++chat history\n" + ''.join(self.chat_history) + "+++\n" + "new prompt: " + prompt
        response = self.model.generate_content(full_prompt)
        return response.text

The generate_response method takes a user prompt as input.
The full combined prompt is concatenated with the chat history lookup rule, entire chat history and the current prompt to provide context for the response.
The Gemini model generates a response based on the combined prompt.

chatbot = GeminiChatbot()
user_input = ("tell a joke about AI")
chatbot.chat_history.append(f"You: {user_input}")
response = chatbot.generate_response(user_input)
chatbot.chat_history.append(f"Chatbot: {response}")
print(response)

    Why did the AI cross the road?  
    To get to the other *side* of the algorithm!

user_input = ("what was joke you told me?")
chatbot.chat_history.append(f"You: {user_input}")
response = chatbot.generate_response(user_input)
chatbot.chat_history.append(f"Chatbot: {response}")
print(response)

    The joke I told you was:
    Why did the AI cross the road? 
    To get to the other *side* of the algorithm!

print(chatbot.chat_history)

    ['You: tell a joke about AI', 'Chatbot: Why did the AI cross the road? \n\nTo get to the other *side* of the algorithm! \n', 'You: what was joke you told me?', 'Chatbot: The joke I told you was:\n\nWhy did the AI cross the road? \n\nTo get to the other *side* of the algorithm! \n']

Logging Chat History

import google.generativeai as genai
import os
import datetime
from dotenv import load_dotenv
from os.path import expanduser

class GeminiChatbot:
    def __init__(self):
        # Load the .env file from the home directory
        self.envpath = '~'
        self.envfile = '.env'

        if self.envpath == '~':
            self.envpath = os.path.expanduser("~")

        load_dotenv(os.path.join(self.envpath, self.envfile))

        genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

        self.modelname = "gemini-1.5-flash"
        self.model = genai.GenerativeModel(self.modelname)

        self.chat_history = []

    def generate_response(self, prompt):
        full_prompt = "Please go through chat history below if user ask question regarding on previous conversation.\nPlease anwser question directly if it is not related to previous conversation\n" + "+++chat history\n" + ''.join(self.chat_history) + "+++\n" + "new prompt: " + prompt
        response = self.model.generate_content(full_prompt)
        return response.text

    def log_chat_history(self,logpath):
        timestamp = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
        log_filename = f"chat-log-{timestamp}.txt"
        log_path = os.path.join(logpath, log_filename)

        os.makedirs(logpath, exist_ok=True)

        with open(log_path, "w") as f:
            for message in self.chat_history:
                f.write(f"{message}\n")
        print(f"chat log file: {log_path}")

This method logs the chat history to a text file.
A timestamp is generated to create a unique filename for the log.
The log directory is created if it doesn't exist.
The chat history is written to the log file, one message per line.

chatbot = GeminiChatbot()
user_input = "tell a joke about AI"
chatbot.chat_history.append(f"You: {user_input}")
response = chatbot.generate_response(user_input)
chatbot.chat_history.append(f"Chatbot: {response}")
print(response)

user_input = "how are you today?"
chatbot.chat_history.append(f"You: {user_input}")
response = chatbot.generate_response(user_input)
chatbot.chat_history.append(f"Chatbot: {response}")
print(response)

user_input = "what was joke you told me?"
chatbot.chat_history.append(f"You: {user_input}")
response = chatbot.generate_response(user_input)
chatbot.chat_history.append(f"Chatbot: {response}")
print(response)

chatbot.log_chat_history('./log')

    Why did the AI cross the road? 
    To get to the other *side* of the algorithm! 
    I'm doing well, thank you for asking! How about you? 😊 
    You asked: "Why did the AI cross the road?" 
    And I answered: "To get to the other *side* of the algorithm!" 
    chat log file: ./log/chat-log-2024-08-18_12-54-43.txt

chatbot.chat_history

    ['You: tell a joke about AI',
     'Chatbot: Why did the AI cross the road? \n\nTo get to the other *side* of the algorithm! \n',
     'You: how are you today?',
     "Chatbot: I'm doing well, thank you for asking! How about you? 😊 \n",
     'You: what was joke you told me?',
     'Chatbot: You asked: "Why did the AI cross the road?" \n\nAnd I answered: "To get to the other *side* of the algorithm!" \n']

Using the Chatbot

    def chat(self):
        n=1 # input number
        print("Welcome to GeminiChatbot! ('/q' to exit)\n")
        self.chat_history.append("Welcome to GeminiChatbot! ('/q' to exit)\n")
        while True:
            user_input = input(f"{n} You: ")
            self.chat_history.append(f"{n} You: {user_input}\n")

            if user_input.lower() == "/q":
                self.log_chat_history('./log')
                print("Chat history saved. Exiting.")
                break
            response = self.generate_response(user_input)
            print(f"{n} Chatbot: {response}")
            self.chat_history.append(f"{n} Chatbot: {response}")
            n += 1

The chat function is main chatbot to end user:

It enters a loop to continuously prompt the user for input.
The chatbot generates responses based on the user's input and the chat history.
Using input number to track conversation.
When the user enters "/q", the chat history is logged and the program exits.

Complete gemini chatbot app

import google.generativeai as genai
import os
import datetime
from dotenv import load_dotenv
from os.path import expanduser

class GeminiChatbot:
    def __init__(self):
        # Load the .env file from the home directory
        self.envpath = '~'
        self.envfile = '.env'

        if self.envpath == '~':
            self.envpath = os.path.expanduser("~")

        load_dotenv(os.path.join(self.envpath, self.envfile))

        genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

        self.modelname = "gemini-1.5-flash"
        self.model = genai.GenerativeModel(self.modelname)

        self.chat_history = []

    def generate_response(self, prompt):
        full_prompt = "Please go through chat history below if user ask question regarding on previous conversation.\nPlease anwser question directly if it is not related to previous conversation\n" + "+++chat history\n" + ''.join(self.chat_history) + "+++\n" + "new prompt: " + prompt
        response = self.model.generate_content(full_prompt)
        return response.text

    def log_chat_history(self,logpath):
        timestamp = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
        log_filename = f"chat-log-{timestamp}.txt"
        log_path = os.path.join(logpath, log_filename)

        os.makedirs(logpath, exist_ok=True)

        with open(log_path, "w") as f:
            for message in self.chat_history:
                f.write(f"{message}\n")

        print(f"chat log file: {log_path}")

    def chat(self):
        n=1 # input number
        print("Welcome to GeminiChatbot! ('/q' to exit)\n")
        self.chat_history.append("Welcome to GeminiChatbot! ('/q' to exit)\n")
        while True:
            user_input = input(f"{n} You: ")
            self.chat_history.append(f"{n} You: {user_input}\n")

            if user_input.lower() == "/q":
                self.log_chat_history('./log')
                print("Chat history saved. Exiting.")
                break
            response = self.generate_response(user_input)
            print(f"{n} Chatbot: {response}")
            self.chat_history.append(f"{n} Chatbot: {response}")
            n += 1

if __name__ == "__main__":
    chatbot = GeminiChatbot()
    chatbot.chat()

    Welcome to GeminiChatbot! ('/q' to exit)
    1 You:  hello
    1 Chatbot: Hello! 👋 How can I help you today? 
    2 You:  how are you?
    2 Chatbot: I'm doing well, thank you for asking! How can I help you today? 😊 
    3 You:  tell me a joke
    3 Chatbot: Why don't scientists trust atoms?
    Because they make up everything! 
    4 You:  tell me another joke
    4 Chatbot: Why don't they play poker in the jungle? 
    Too many cheetahs! 🐆 
    5 You:  what was 1st joke?
    5 Chatbot: The first joke was:  "Why don't scientists trust atoms? Because they make up everything!" 
    6 You:  what was 2nd joke?
    6 Chatbot: The second joke was: "Why don't they play poker in the jungle? Too many cheetahs!" 🐆 
    7 You:  what was 1st word I said?
    7 Chatbot: The first word you said was "hello". 
    8 You:  what was 2nd word I said?
    8 Chatbot: The second word you said was "how". 
    9 You:  what was 2nd question I asked?
    9 Chatbot: The second question you asked was "how are you?" 
    10 You:  /q
    chat log file: ./log/chat-log-2024-08-18_12-41-13.txt
    Chat history saved. Exiting.

chatbot.chat_history

    ["Welcome to GeminiChatbot! ('/q' to exit)\n",
     '1 You: hello\n',
     '1 Chatbot: Hello! 👋 How can I help you today? \n',
     '2 You: how are you?\n',
     "2 Chatbot: I'm doing well, thank you for asking! How can I help you today? 😊 \n",
     '3 You: tell me a joke\n',
     "3 Chatbot: Why don't scientists trust atoms?\n\nBecause they make up everything! \n",
     '4 You: tell me another joke\n',
     "4 Chatbot: Why don't they play poker in the jungle? \n\nToo many cheetahs! 🐆 \n",
     '5 You: what was 1st joke?\n',
     '5 Chatbot: The first joke was:  "Why don\'t scientists trust atoms? Because they make up everything!" \n',
     '6 You: what was 2nd joke?\n',
     '6 Chatbot: The second joke was: "Why don\'t they play poker in the jungle? Too many cheetahs!" 🐆 \n',
     '7 You: what was 1st word I said?\n',
     '7 Chatbot: The first word you said was "hello". \n',
     '8 You: what was 2nd word I said?\n',
     '8 Chatbot: The second word you said was "how". \n',
     '9 You: what was 2nd question I asked?\n',
     '9 Chatbot: The second question you asked was "how are you?" \n',
     '10 You: /q\n']

Change chatbot instance

chatbot = GeminiChatbot()
chatbot.envpath = "/mnt/c/dclab/dev/ai-ml"
chatbot.modelname = "gemini-1.5-pro"
chatbot.chat()

    Welcome to GeminiChatbot! ('/q' to exit)
    1 You:  hello
    1 Chatbot: Hello! 👋 How can I help you today? 😊 
    2 You:  tell me a joke
    2 Chatbot: Why don't scientists trust atoms? 
    Because they make up everything! 
    3 You:  tell me another joke
    3 Chatbot: Why don't they play poker in the jungle? 
    Too many cheetahs! 🐆 
    4 You:  what was 1st joke?
    4 Chatbot: The first joke was: 
    Why don't scientists trust atoms? 
    Because they make up everything! 
    5 You:  what was 2nd joke?
    5 Chatbot: The second joke was:
    Why don't they play poker in the jungle? 
    Too many cheetahs! 🐆 
    6 You:  what was 1st word I said?
    6 Chatbot: The first word you said was "hello". 
    7 You:  /q
    chat log file: ./log/chat-log-2024-08-18_12-42-19.txt
    Chat history saved. Exiting.

chatbot.chat_history

Conclusion

Why is memory so important for chatbots? Just like real conversations, remembering past interactions allows chatbots to provide more relevant and personalized responses. In this blog, we've built a Gemini chatbot using Python that can summarize chat history and use that context to inform its responses. This approach leads to a more natural and engaging conversation for the user.

By following these steps and using the google.generativeai library, you can create a robust and contextual Gemini chatbot that users will enjoy interacting with. Remember to replace 'GOOGLE_API_KEY' with your actual Gemini API key in the .env file. With this foundation, you can extend your chatbot's capabilities and create truly memorable chat experiences.

About Me

Hey! I am Robert Wang, live in Montreal.

More simple and more efficient.

Build k8s Cluster on Hyper-V

Robert Wang — Mon, 22 Apr 2024 13:51:30 +0000

Intro

I used Virtualbox to launch Linux vm using Vagrant for many years, have to switch to Hyper-V due to it by default enabled in win10 host.

The outstanding point comparing to Virtualbox is that it supports nested virtualization, means you can use kvm hypervisor on top of Hyper-V Linux VM, to launch nested vm easily.

The drawback or learning curve with Hyper-V is that network model is different than Virtualbox.

I will address few pain points and go through completely k8s lab deployment on top of Hyper-V on win10 host in this post.

static ip for Hyper-V vm and permanent ip for k8s API server
experiment k8s setup automation with Vagrant

Lab env

Win10 32GB RAM
Hyper-V
vagrant 2.4.1 windows
WSL2 optional as terminal

k8s hyperv tool set

We heavy use handy scripts to make k8s cluster on Hyper-V automatically, you can clone to local or download them manually,

git clone git@github.com:robertluwang/hands-on-nativecloud.git
cd ./hands-on-nativecloud/src/k8s-cri-dockerd

Hyper-V network

3 types:

private - vm<->vm only
internal - vm<->vm, vm<->host, vm->Internet if NAT enabled
external - kind of bridge, vm<->host, vm->Internet

private network
It is isolated network, vm can access each other only.

internal network
It is private network plus host communication due to virtual NIC on host.

It is similar with Host-only in Virtualbox.

It could be further as NAT to allow vm to access Internet by adding internal network in NetNat, which enabling the routing traffic to outside.

The "Default Switch" is default NAT switch, the only issue is switch ip always changed after reboot; also DNS not working well. As remedy you can manually change vm ip and DNS inside vm, to match new change on Default Switch.

The good news is that it is possible to setup own static NAT network, to have stable test lab env, which is host-only + NAT if comparing with Virtualbox.

external switch
External switch will bind with physical NIC on host, sharing same network with host to access outside, ip assigned from DHCP.

In my pc there is only Wi-Fi adaptor, so external switch will bind to Wi-Fi adaptor, it always crashes and lost all Wi-Fi Internet access during creating, have to recover by Network Reset.

Let's forget about external switch for wi-fi at all, since it will directly impact host network access.

So Internal network + NAT is best and stable option for static IP, it allow host and vm talk each other and also access to Internet.

Create own NAT switch

We need to create new Hyper-V switch for example myNAT as type of Internal either from Hyper-V Manager GUI or Powershell as admin role, then create vNIC network like 192.168.120.0/24, and enable as NAT.

You can run PowerShell term as admin on win10 host, then run below script to complete vm switch, vNIC and NAT setup.

Please change your prefer values in script:

hyperv switch name: myNAT
vEthernet ip: 192.168.120.1 it is gateway on win host
NetNAT name: myNAT
NetNAT network: 192.168.120./24

myNAT.ps1

If ("myNAT" -in (Get-VMSwitch | Select-Object -ExpandProperty Name) -eq $FALSE) {
    'Creating Internal-only switch "myNAT" on Windows Hyper-V host...'

    New-VMSwitch -SwitchName "myNAT" -SwitchType Internal

    New-NetIPAddress -IPAddress 192.168.120.1 -PrefixLength 24 -InterfaceAlias "vEthernet (myNAT)"

    New-NetNAT -Name "myNAT" -InternalIPInterfaceAddressPrefix 192.168.120.0/24
}
else {
    'Internal-only switch "myNAT" for static IP configuration already exists; skipping'
}

If ("192.168.120.1" -in (Get-NetIPAddress | Select-Object -ExpandProperty IPAddress) -eq $FALSE) {
    'Registering new IP address 192.168.120.1 on Windows Hyper-V host...'

    New-NetIPAddress -IPAddress 192.168.120.1 -PrefixLength 24 -InterfaceAlias "vEthernet (myNAT)"
}
else {
    'IP address "192.168.120.1" for static IP configuration already registered; skipping'
}

If ("192.168.120.0/24" -in (Get-NetNAT | Select-Object -ExpandProperty InternalIPInterfaceAddressPrefix) -eq $FALSE) {
    'Registering new NAT adapter for 192.168.120.0/24 on Windows Hyper-V host...'

    New-NetNAT -Name "myNAT" -InternalIPInterfaceAddressPrefix 192.168.120.0/24
}
else {
    '"192.168.120.0/24" for static IP configuration already registered; skipping'
}

Here is sample of running result,

.\myNAT.ps1
Creating Internal-only switch named "myNAT" on Windows Hyper-V host...

Name   SwitchType NetAdapterInterfaceDescription
----   ---------- ------------------------------
myNAT Internal                                 

IPv4Address              : 192.168.120.1
IPv6Address              : 
IPVersionSupport         : 
PrefixLength             : 24
SubnetMask               : 
AddressFamily            : IPv4
AddressState             : Tentative
InterfaceAlias           : vEthernet (myNAT)
InterfaceIndex           : 109
IPAddress                : 192.168.120.1
PreferredLifetime        : 10675199.02:48:05.4775807
PrefixOrigin             : Manual
SkipAsSource             : False
Store                    : ActiveStore
SuffixOrigin             : Manual
Type                     : Unicast
ValidLifetime            : 10675199.02:48:05.4775807
PSComputerName           : 
ifIndex                  : 109

IPv4Address              : 192.168.120.1
IPv6Address              : 
IPVersionSupport         : 
PrefixLength             : 24
SubnetMask               : 
AddressFamily            : IPv4
AddressState             : Invalid
InterfaceAlias           : vEthernet (myNAT)
InterfaceIndex           : 109
IPAddress                : 192.168.120.1
PreferredLifetime        : 10675199.02:48:05.4775807
PrefixOrigin             : Manual
SkipAsSource             : False
Store                    : PersistentStore
SuffixOrigin             : Manual
Type                     : Unicast
ValidLifetime            : 10675199.02:48:05.4775807
PSComputerName           : 
ifIndex                  : 109

Caption                          : 
Description                      : 
ElementName                      : 
InstanceID                       : myNAT;0
Active                           : True
ExternalIPInterfaceAddressPrefix : 
IcmpQueryTimeout                 : 30
InternalIPInterfaceAddressPrefix : 192.168.120.0/24
InternalRoutingDomainId          : {00000000-0000-0000-0000-000000000000}
Name                             : myNAT1
Store                            : Local
TcpEstablishedConnectionTimeout  : 1800
TcpFilteringBehavior             : AddressDependentFiltering
TcpTransientConnectionTimeout    : 120
UdpFilteringBehavior             : AddressDependentFiltering
UdpIdleSessionTimeout            : 120
UdpInboundRefresh                : False
PSComputerName                   : 

IP address "192.168.120.1" for static IP configuration already registered; skipping
"192.168.120.0/24" for static IP configuration already registered; skipping

Also there is a handy PowerShell script to delete hyper-v switch myNAT, vNIC and associated NAT, in case tear down NAT switch or reset it.
myNAT-delete.ps1

If (Get-VMSwitch | ? Name -Eq myNAT) {
    'Deleting Internal-only switch named myNAT on Windows Hyper-V host...'
    Remove-VMSwitch -Name myNAT
}
else {
    'VMSwitch "myNAT" not exists; skipping'
}

If (Get-NetIPAddress | ? IPAddress -Eq "192.168.120.1") {
    'Deleting IP address 192.168.120.1 on Windows Hyper-V host...'
    Remove-NetIPAddress -IPAddress "192.168.120.1"
}
else {
    'IP address "192.168.120.1" not existing; skipping'
}

If (Get-NetNAT | ? Name -Eq myNAT) {
    'Deleting NAT adapter myNAT on Windows Hyper-V host...'
    Remove-NetNAT -Name myNAT
}
else {
    'NAT adapter myNAT not existing; skipping'
}

Bring up static host-only ip with Vagrant

Until now the Vagrant not supports well with Hyper-V network configuration like Virtualbox in Vagrantfile.

https://developer.hashicorp.com/vagrant/docs/providers/hyperv/limitations

host-only NIC for Virtualbox,

        vm.network :private_network, ip: "192.168.99.30"

then what should we do in hyper-v Vagrantfile? we only can choice network switcher, not chance to assign static ip.

Good news is that Vagrant can ssh to new launched vm via ipv6 IP, means we have chance to run any provision script, so as remedy we can run a shell inline or offline in provision stage, to setup static ip to eth0 manually, also correct /etc/resolv.conf for DNS setting.

Here is sample to launch Ubuntu 22.04 vm with 192.168.120.20 with Vagrant, to demo how static ip setup works for Ubuntu vm using Vagrantfile.

oldhorse@wsl2:$ cat Vagrantfile
$nic = <<SCRIPT

echo === $(date) Provisioning - nic $1 by $(whoami) start

SUBNET=$(echo $1 | cut -d"." -f1-3)

cat <<EOF | sudo tee /etc/netplan/01-netcfg.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      dhcp4: no
      dhcp6: no
      addresses: [$1/24]
      routes:
      - to: default
        via: ${SUBNET}.1
      nameservers:
        addresses: [8.8.8.8,1.1.1.1]
EOF

sudo unlink /etc/resolv.conf
sudo rm /etc/resolv.conf
cat << EOF | sudo tee /etc/resolv.conf
nameserver 8.8.8.8
nameserver 1.1.1.1
EOF

sudo chattr +i /etc/resolv.conf

cat /etc/netplan/01-netcfg.yaml
cat /etc/resolv.conf

sudo netplan apply
sleep 30 
echo eth0 setting

ip addr
ip route
ping -c 2 google.ca

echo === $(date) Provisioning - nic $1 by $(whoami) end
SCRIPT

Vagrant.configure("2") do |config|
  config.vm.box = "generic/ubuntu2204"
  config.vm.provision "shell", inline: "sudo timedatectl set-timezone America/Montreal", privileged: false, run: "always"
  config.ssh.insert_key = false
  config.vm.box_check_update = false
  config.vm.network "public_network", bridge: "myNAT"

  config.vm.define "master" do |master|
      master.vm.hostname = "ub22hpv"
      master.vm.provider "hyperv" do |v|
          v.vmname = "ub22hpv"
          v.memory = 1024
      end
      master.vm.provision "shell", inline: $nic, args: "192.168.120.20", privileged: false
  end
end

We can see vagrant trying to ssh to vm via ipv6 with private key,

    master: SSH address: fe80::215:5dff:fec9:606c:22
    master: SSH username: vagrant
    master: SSH auth method: private key

Full vagrant up log as below,

oldhorse@wsl2: vagrant.exe up
Bringing machine 'master' up with 'hyperv' provider...
==> master: Verifying Hyper-V is enabled...
==> master: Verifying Hyper-V is accessible...
==> master: Importing a Hyper-V instance
    master: Creating and registering the VM...
    master: Successfully imported VM
    master: Configuring the VM...
    master: Setting VM Enhanced session transport type to disabled/default (VMBus)
==> master: Starting the machine...
==> master: Waiting for the machine to report its IP address...
    master: Timeout: 120 seconds
    master: IP: fe80::215:5dff:fec9:606c
==> master: Waiting for machine to boot. This may take a few minutes...
    master: SSH address: fe80::215:5dff:fec9:606c:22
    master: SSH username: vagrant
    master: SSH auth method: private key
==> master: Machine booted and ready!
==> master: Setting hostname...
==> master: Running provisioner: shell...
    master: Running: inline script
==> master: Running provisioner: shell...
    master: Running: inline script
    master: === Sun Apr 21 05:36:13 PM EDT 2024 Provisioning - nic 192.168.120.20 by vagrant start
    master: network:
    master:   version: 2
    master:   renderer: networkd
    master:   ethernets:
    master:     eth0:
    master:       dhcp4: no
    master:       dhcp6: no
    master:       addresses: [192.168.120.20/24]
    master:       routes:
    master:       - to: default
    master:         via: 192.168.120.1
    master:       nameservers:
    master:         addresses: [8.8.8.8,1.1.1.1]
    master: rm: cannot remove '/etc/resolv.conf': No such file or directory
    master: nameserver 8.8.8.8
    master: nameserver 1.1.1.1
    master: network:
    master:   version: 2
    master:   renderer: networkd
    master:   ethernets:
    master:     eth0:
    master:       dhcp4: no
    master:       dhcp6: no
    master:       addresses: [192.168.120.20/24]
    master:       routes:
    master:       - to: default
    master:         via: 192.168.120.1
    master:       nameservers:
    master:         addresses: [8.8.8.8,1.1.1.1]
    master: nameserver 8.8.8.8
    master: nameserver 1.1.1.1
    master: 
    master: ** (generate:3029): WARNING **: 17:36:14.124: Permissions for /etc/netplan/00-installer-config.yaml are too open. Netplan configuration should NOT be accessible by others.

    master: eth0 setting
    master: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    master:     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    master:     inet 127.0.0.1/8 scope host lo
    master:        valid_lft forever preferred_lft forever
    master: 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    master:     link/ether 00:15:5d:c9:60:6c brd ff:ff:ff:ff:ff:ff
    master:     inet 192.168.120.20/24 brd 192.168.120.255 scope global eth0
    master:        valid_lft forever preferred_lft forever
    master:     inet6 fe80::215:5dff:fec9:606c/64 scope link
    master:        valid_lft forever preferred_lft forever
    master: default via 192.168.120.1 dev eth0 proto static
    master: 192.168.120.0/24 dev eth0 proto kernel scope link src 192.168.120.20
    master: PING google.ca (142.250.65.163) 56(84) bytes of data.
    master: 64 bytes from lga25s71-in-f3.1e100.net (142.250.65.163): icmp_seq=1 ttl=105 time=30.8 ms
    master: 64 bytes from lga25s71-in-f3.1e100.net (142.250.65.163): icmp_seq=2 ttl=105 time=69.3 ms
    master:
    master: --- google.ca ping statistics ---
    master: 2 packets transmitted, 2 received, 0% packet loss, time 1004ms
    master: rtt min/avg/max/mdev = 30.791/50.024/69.258/19.233 ms
    master: === Sun Apr 21 05:36:45 PM EDT 2024 Provisioning - nic 192.168.120.20 by vagrant end

Build up k8s cluster with Hyper-V using Vagrant

After we make hyper-v vm with static ip works, then it is possible to install Docker, k8s package and build up k8s cluster via provision inline script.

We assume it is 2 nodes k8s cluster:

Ubuntu 22.04
master: 2GB RAM 1 cpu 192.168.120.20
worker: 1GB RAM 1 cpu 192.168.120.30
Docker: latest version
k8s: latest version , it is 1.30 here

It spent around 26 mins to build up k8s 2 nodes cluster very smoothly,

oldhorse@wsl2: date; vagrant.exe up ; date 
Sun Apr 21 18:05:06 EDT 2024
Bringing machine 'master' up with 'hyperv' provider...
Bringing machine 'worker' up with 'hyperv' provider...
...
master: kubeadm join 192.168.120.20:6443 --token y10r7h.1vycfilas9kpovzp \
    master:     --discovery-token-ca-cert-hash sha256:64e627afc4ed15beb607adf4073ae856400097fb4fed248094fb679e235cc597

...
worker: This node has joined the cluster:
worker: * Certificate signing request was sent to apiserver and a response was received.
...
Sun Apr 21 18:31:33 EDT 2024

check k8s cluster, it is running as 2 nodes cluster.

We can ssh to master vm using vagrant, or directly ssh to 192.168.120.20 master and 192.168.120.30 worker.

vagrant ssh master

It looks good,

vagrant@master:~$ k get node
NAME     STATUS   ROLES           AGE   VERSION
master   Ready    control-plane   31m   v1.30.0
worker   Ready    <none>          19m   v1.30.0

vagrant@master:~$ k get pod -A
NAMESPACE          NAME                                       READY   STATUS    RESTARTS   AGE
calico-apiserver   calico-apiserver-7868f6b748-glqdx          1/1     Running   0          11m
calico-apiserver   calico-apiserver-7868f6b748-kc4gq          1/1     Running   0          11m
calico-system      calico-kube-controllers-78788579b8-vvrdh   1/1     Running   0          30m
calico-system      calico-node-dlrs8                          1/1     Running   0          19m
calico-system      calico-node-vvxm6                          1/1     Running   0          30m
calico-system      calico-typha-65bf4f686-mlpbx               1/1     Running   0          30m
kube-system        coredns-76f75df574-65mk8                   1/1     Running   0          31m
kube-system        coredns-76f75df574-z982z                   1/1     Running   0          31m
kube-system        etcd-master                                1/1     Running   0          31m
kube-system        kube-apiserver-master                      1/1     Running   0          31m
kube-system        kube-controller-manager-master             1/1     Running   0          31m
kube-system        kube-proxy-dcfcc                           1/1     Running   0          19m
kube-system        kube-proxy-x7qtl                           1/1     Running   0          31m
kube-system        kube-scheduler-master                      1/1     Running   0          31m
tigera-operator    tigera-operator-6fbc4f6f8d-h5wcm           1/1     Running   0          

vagrant@master:~$ k run test --image=nginx $do 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: test
  name: test
spec:
  containers:
  - image: nginx
    name: test
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

vagrant@master:~$ k run test --image=nginx 
pod/test created
vagrant@master:~$ k get pod 
NAME   READY   STATUS              RESTARTS   AGE
test   0/1     ContainerCreating   0          4s

vagrant@master:~$ k get pod -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE     NOMINATED NODE   READINESS GATES
test   1/1     Running   0          42s   192.168.171.67   worker   <none>           <none>

vagrant@master:~$ curl 192.168.171.67
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
</html>

k8s setup tool set

In above example, we demo the quick way to launch k8s cluster using Vagrant, which uses inline scripts to install docker, setup cri-dockerd, install k8s packages, init k8s cluster or join to cluster on worker node.

In fact we can use scripts for handy setup k8s cluster when you already have running Linux node, no matter it is physical Linux, Linux vm, or WSL2.

Just run script as sudo user,

master node

docker-server.sh
k8s-install.sh
cri-dockerd.sh
k8s-init.sh "192.168.120.20"

worker node

docker-server.sh
k8s-install.sh
cri-dockerd.sh
k8s-join-worker.sh "192.168.120.20"

Conclusion

We make the k8s 2 nodes cluster setup automation smoothly using Vagrant:

it is possible to make k8s cluster setup on Hyper-V full automatically
it is ideal local k8s lab for production like testing and education

About Me

Hey! I am Robert Wang, live in Montreal.

More simple and more efficient.

Build k8s Cluster on WSL2

Robert Wang — Sun, 14 Apr 2024 21:50:44 +0000

The main difference between WSL1 and WSL2 that later is hyper-v VM based Microsoft Linux version, it sounds like excited news at least it is running full Linux kernel but there are too many issues if using as devops box.

Here are some outstanding pain points:

default systemd is not enabled, which needed to install packages
default network is NAT, always assign dynamic ip after reboot, will give trouble for app like k8s api server

Lab env

win10 host 32GB RAM
WSL2 Ubuntu 22.04
Docker 25.0.4 on WSL2
k8s 1.29.3 on WSL2

Install WSL2 Ubuntu

Install WSL2 from CMD or PS on win host, following official doc,

https://docs.microsoft.com/en-us/windows/wsl/install

wsl -l -o
The following is a list of valid distributions that can be installed.
Install using 'wsl.exe --install <Distro>'.

NAME                                   FRIENDLY NAME
Ubuntu                                 Ubuntu
Debian                                 Debian GNU/Linux
kali-linux                             Kali Linux Rolling
Ubuntu-18.04                           Ubuntu 18.04 LTS
Ubuntu-20.04                           Ubuntu 20.04 LTS
Ubuntu-22.04                           Ubuntu 22.04 LTS
OracleLinux_7_9                        Oracle Linux 7.9
OracleLinux_8_7                        Oracle Linux 8.7
OracleLinux_9_1                        Oracle Linux 9.1
openSUSE-Leap-15.5                     openSUSE Leap 15.5
SUSE-Linux-Enterprise-Server-15-SP4    SUSE Linux Enterprise Server 15 SP4
SUSE-Linux-Enterprise-15-SP5           SUSE Linux Enterprise 15 SP5
openSUSE-Tumbleweed                    openSUSE Tumbleweed

wsl --install Ubuntu-22.04

Launch WSL2 instance from CMD or PS shell,

wsl -d Ubuntu-22.04

Launch WSL2 instance from SecureCRT client

create wsl2.bat under C:\tools, including below line

wsl -d Ubuntu-22.04

SecureCRT->Session->Connection/Protocol:'Local Shell', Connection->Local Shell/Shell path: C:\tools\wsl2.bat

WSL2 config

There are two kind of config for WSL2:

global config for all WSL2 instance on win host, C:\Users\oldhorse\.wslconfig

[wsl2]
memory=16GB
processors=4
swap=0

config per one WSL2 instance inside vm, /etc/wsl.conf

[boot]
systemd=true
[network]
hostname = wsl2
generateHosts = false
generateResolvConf = false

This will enable systemd, setup hostname and don't generate /etc/resolv.conf.

Remedy for "static" ip

After WSL2 instance launched, will see NAT ip as below,

oldhorse@wsl2:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1350 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:ea:e8:22 brd ff:ff:ff:ff:ff:ff
    inet 172.30.73.22/20 brd 172.30.79.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:feea:e822/64 scope link 
       valid_lft forever preferred_lft forever

Assume we want to keep static ip 192.168.80.2 for WSL2 Ubuntu, then need to manually assign ip to WSL2 eth0 inside WSL2 VM, and recreate NAT network adaptor for WSL on win host end, it is in fact the network type of Host-only + NAT.

Run below batch script as Administrator on windows host, you can type wsl2ip.bat from CMD, PS or place shortcut icon on desktop.

wsl2ip.bat

date /t
time /t

wsl -d Ubuntu-22.04 -u root ip addr del $(ip addr show eth0 ^| grep 'inet\b' ^| awk '{print $2}' ^| head -n 1) dev eth0
wsl -d Ubuntu-22.04 -u root ip addr add 192.168.80.2/24 broadcast 192.168.80.255 dev eth0
wsl -d Ubuntu-22.04 -u root ip route add 0.0.0.0/0 via 192.168.80.1 dev eth0
wsl -d Ubuntu-22.04 -u root echo nameserver 8.8.8.8 ^> /etc/resolv.conf
wsl -d Ubuntu-22.04 -u root chattr +i /etc/resolv.conf

powershell -c "Get-NetAdapter 'vEthernet (WSL)' | Get-NetIPAddress | Remove-NetIPAddress -Confirm:$False; New-NetIPAddress -IPAddress 192.168.80.1 -PrefixLength 24 -InterfaceAlias 'vEthernet (WSL)'; Get-NetNat | ? Name -Eq WSLNat | Remove-NetNat -Confirm:$False; New-NetNat -Name WSLNat -InternalIPInterfaceAddressPrefix 192.168.80.0/24;"

date /t
time /t

pause

The default generated /etc/resolv.conf won't work for DNS for Internet access, so that is why we disable generateResolvConf in /etc/wsl.conf, add 8.8.8.8 to it, and changed it to read-only file.

After run above script, you can access from host to WSL2 due to it is host-only static ip now.

PS C:\Users\oldhorse> Get-NetAdapter 'vEthernet (WSL)' | Get-NetIPAddress

IPAddress         : 192.168.80.1
InterfaceIndex    : 24
InterfaceAlias    : vEthernet (WSL)
AddressFamily     : IPv4
Type              : Unicast
PrefixLength      : 24
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Preferred
ValidLifetime     : Infinite ([TimeSpan]::MaxValue)
PreferredLifetime : Infinite ([TimeSpan]::MaxValue)
SkipAsSource      : False
PolicyStore       : ActiveStore

PS C:\Users\oldhorse> Get-NetNat 'WSLNat'

Name                             : WSLNat
ExternalIPInterfaceAddressPrefix :
InternalIPInterfaceAddressPrefix : 192.168.80.0/24
IcmpQueryTimeout                 : 30
TcpEstablishedConnectionTimeout  : 1800
TcpTransientConnectionTimeout    : 120
TcpFilteringBehavior             : AddressDependentFiltering
UdpFilteringBehavior             : AddressDependentFiltering
UdpIdleSessionTimeout            : 120
UdpInboundRefresh                : False
Store                            : Local
Active                           : True

PS C:\Users\oldhorse> ipconfig

Ethernet adapter vEthernet (WSL):

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : 192.168.80.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

PS C:\Users\oldhorse> ping 192.168.80.2

Pinging 192.168.80.2 with 32 bytes of data:
Reply from 192.168.80.2: bytes=32 time=1ms TTL=64
Reply from 192.168.80.2: bytes=32 time=1ms TTL=64

also you can access from WSL2 to host, Internet like google as well,

oldhorse@wsl2:~$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1350 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:ea:e8:22 brd ff:ff:ff:ff:ff:ff
    inet 192.168.80.2/24 brd 192.168.80.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:feea:e822/64 scope link
       valid_lft forever preferred_lft forever

oldhorse@wsl2:~$ ip r
default via 192.168.80.1 dev eth0 
192.168.80.0/24 dev eth0 proto kernel scope link src 192.168.80.2

oldhorse@wsl2:~$ ping google.ca
PING google.ca (142.251.40.99) 56(84) bytes of data.
64 bytes from lga25s79-in-f3.1e100.net (142.251.40.99): icmp_seq=1 ttl=105 time=69.4 ms
64 bytes from lga25s79-in-f3.1e100.net (142.251.40.99): icmp_seq=2 ttl=105 time=60.5 ms

oldhorse@wsl2:~$ cat /etc/resolv.conf 
nameserver 8.8.8.8

Common issue for WSL2 NAT

Since we changed WSL2 NAT manually, sometimes it hanging on wsl cli, as remedy run below reset batch file as admin from win host.
wslreset.bat

taskkill /f /im wslservice.exe

wsl -l 

pause

From now on the WSL2 with static ip and Internet access ready for k8s installation.

k8s installation automation

It is possible to install k8s installation in one shot using my handy script toolkit.
First of all, download it via git clone, or manually download from github.

git clone git@github.com:robertluwang/hands-on-nativecloud.git
cd ./hands-on-nativecloud/src/k8s-cri-dockerd

Here is complete step to install a single node k8s cluster on WSL2.

bash docker-server.sh

exit and enter to WSL2 instance, to make sure non-root user to access docker.

bash cri-dockerd.sh
bash k8s-install.sh
bash k8s-init.sh 192.168.80.2

Next I will explain more details for each step as below.

Linux native Docker server on WSL2

The systemd is ready so just following the docker doc to install docker on Ubuntu,

https://docs.docker.com/engine/install/

I make it as handy script, docker-install.sh.

docker-server.sh

# docker-server.sh
# handy script to install docker on ubuntu 
# run on k8s cluster node (master/worker)
# By Robert Wang @github.com/robertluwang
# Nov 21, 2022

echo === $(date) Provisioning - docker-server.sh by $(whoami) start

sudo apt-get update -y
sudo apt-get install -y ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update -y
sudo apt-get install -y docker-ce docker-ce-cli containerd.io 

sudo groupadd docker
sudo usermod -aG docker $USER

# turn off swap
sudo swapoff -a
sudo sed -i '/swap/d' /etc/fstab

sudo mkdir /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
sudo systemctl enable docker
sudo systemctl daemon-reload
sudo systemctl restart docker
sleep 30 
sudo systemctl restart docker

echo === $(date) Provisioning - docker-server.sh by $(whoami) end

Let's run it,

bash docker-server.sh

re-login to WSL2, docker working for non-root user, in my case it is oldhorse.

docker info 
docker ps

verify docker daemon,

oldhorse@wsl2:~$ systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2024-04-14 12:07:34 EDT; 1h 10min ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 253 (dockerd)
      Tasks: 104
     Memory: 208.5M
     CGroup: /system.slice/docker.service
             ├─253 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

docker bridge docker0 ready,

oldhorse@wsl2:~$ ip addr show docker0
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:a7:dc:0d:c9 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

cri-dockerd setup for k8s + docker on WSL2

As we known the docker support removed since k8s 1.24+, so need cri-dockerd installed to have cri interface with Docker Engine API for k8s cluster. More details please see https://github.com/Mirantis/cri-dockerd.

You might wonder why we want to still keep docker for k8s setup, because I would like get benefit from pure docker test and k8s test in same local lab.

I also make handy script for setup - cri-dockerd.sh

cri-dockerd.sh

# cri-dockerd.sh
# handy script to install cri-dockerd on ubuntu 
# run on k8s cluster node (master/worker)
# By Robert Wang @github.com/robertluwang
# Nov 26, 2022

echo === $(date) Provisioning - cri-dockerd by $(whoami) start

# cri-dockerd 
VER=$(curl -s https://api.github.com/repos/Mirantis/cri-dockerd/releases/latest|grep tag_name | cut -d '"' -f 4|sed 's/v//g')
wget https://github.com/Mirantis/cri-dockerd/releases/download/v${VER}/cri-dockerd-${VER}.amd64.tgz
tar xvf cri-dockerd-${VER}.amd64.tgz
sudo mv cri-dockerd/cri-dockerd /usr/local/bin/
sudo chmod +x /usr/local/bin/cri-dockerd

wget https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.service
wget https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.socket

sudo mv cri-docker.socket cri-docker.service /etc/systemd/system/
sudo chown root:root /etc/systemd/system/cri-docker.*
sudo sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service

sudo systemctl daemon-reload
sudo systemctl enable cri-docker.service
sudo systemctl enable --now cri-docker.socket
sleep 30
sudo systemctl restart cri-docker.service

echo === $(date) Provisioning - cri-dockerd by $(whoami) end

Let's run it,

bash cri-dockerd.sh

Verify it is active running,

oldhorse@wsl2:~$ systemctl status cri-docker
● cri-docker.service - CRI Interface for Docker Application Container Engine
     Loaded: loaded (/etc/systemd/system/cri-docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2024-04-14 12:07:35 EDT; 1h 25min ago
TriggeredBy: ● cri-docker.socket
       Docs: https://docs.mirantis.com
   Main PID: 880 (cri-dockerd)
      Tasks: 17
     Memory: 100.8M
     CGroup: /system.slice/cri-docker.service
             └─880 /usr/local/bin/cri-dockerd --container-runtime-endpoint fd://

k8s packages installation on WSL2

It is time to install k8s following up official k8s doc.

Install k8s packages using below handy script k8s-install.sh,

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

k8s-install.sh

# k8s-install.sh
# handy script to install k8s on ubuntu 
# run on all k8s cluster node (master/worker)
# By Robert Wang @github.com/robertluwang
# Oct 30, 2021

echo === $(date) Provisioning - k8s-install.sh by $(whoami) start

sudo apt-get update -y
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
sudo curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update -y
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

# cli completion
sudo apt-get install bash-completion -y
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -F __start_kubectl k' >>~/.bashrc
echo "export do='--dry-run=client -o yaml'" >>~/.bashrc

echo === $(date) Provisioning - k8s-install.sh by $(whoami) end

Let's run it,

bash k8s-install.sh

Verify installed package version,

oldhorse@wsl2:~/dev$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"29", GitVersion:"v1.29.3", GitCommit:"6813625b7cd706db5bc7388921be03071e1a492d", GitTreeState:"clean", BuildDate:"2024-03-15T00:06:16Z", GoVersion:"
go1.21.8", Compiler:"gc", Platform:"linux/amd64"}
o
ldhorse@wsl2:~/dev$ kubectl version 
Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.3

oldhorse@wsl2:~/dev$ kubelet --version
Kubernetes v1.29.3

k8s cluster init on WSL2

Run handy script k8s-init.sh for first time setup k8s cluster on WSL2,

k8s-init.sh

# k8s-init.sh
# handy script to init k8s cluster on ubuntu 
# run on k8s master node only
# By Robert Wang @github.com/robertluwang
# Nov 21, 2022
# $1 - master/api server ip

echo === $(date) Provisioning - k8s-init.sh $1 by $(whoami) start

if [ -z "$1" ];then
    sudo kubeadm init --pod-network-cidr=192.168.0.0/16 --ignore-preflight-errors=NumCPU --ignore-preflight-errors=Mem --cri-socket unix:///var/run/cri-dockerd.sock | tee /var/tmp/kubeadm.log
else
    sudo kubeadm init --pod-network-cidr=192.168.0.0/16 --apiserver-advertise-address=$1 --ignore-preflight-errors=NumCPU --ignore-preflight-errors=Mem --cri-socket unix:///var/run/cri-dockerd.sock | tee /var/tmp/kubeadm.log
fi

# allow normal user to run kubectl
if [ -d $HOME/.kube ]; then
  rm -r $HOME/.kube
fi
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# install calico network addon
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.24.5/manifests/tigera-operator.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.24.5/manifests/custom-resources.yaml

# allow run on master
kubectl taint nodes --all node-role.kubernetes.io/control-plane-

echo === $(date) Provisioning - k8s-init.sh $1 by $(whoami) end

We need to give eth0 static ip 192.168.80.2 as parameter to k8s-init.sh, let the k8s api server pointing to eth0 ip.

bash k8s-init.sh 192.168.80.2

k8s cluster reset on WSL2

Run handy script k8s-reset.sh to quickly rebuild k8s cluster on WSL2 if anything mess up,

k8s-reset.sh

# k8s-reset.sh
# handy script to reset k8s cluster on ubuntu 
# run on k8s cluster node (master/worker)
# By Robert Wang @github.com/robertluwang
# Nov 21, 2022

echo === $(date) Provisioning - k8s-reset.sh by $(whoami) start

sudo kubeadm reset -f --cri-socket unix:///var/run/cri-dockerd.sock 

echo === $(date) Provisioning - k8s-reset.sh by $(whoami) end

For example,

bash k8s-reset.sh

then following k8s-init.sh with eth0 static ip,

bash k8s-init.sh 192.168.80.2

k8s cluster test on WSL2

oldhorse@wsl2:~$ k get node
NAME   STATUS   ROLES           AGE    VERSION
wsl2   Ready    control-plane   7d3h   v1.29.3
oldhorse@wsl2:~$ k get pod -A
NAMESPACE          NAME                                       READY   STATUS    RESTARTS        AGE
calico-apiserver   calico-apiserver-99d8c8bc6-5csh6           1/1     Running   9 (104m ago)    7d3h
calico-apiserver   calico-apiserver-99d8c8bc6-lgdh4           1/1     Running   9 (104m ago)    7d3h
calico-system      calico-kube-controllers-78788579b8-jdhj4   1/1     Running   9 (104m ago)    7d3h
calico-system      calico-node-vzhwj                          1/1     Running   9 (104m ago)    7d3h
calico-system      calico-typha-7647b5ff5b-dkxks              1/1     Running   15 (40m ago)    7d3h
kube-system        coredns-76f75df574-4dg8d                   1/1     Running   9 (104m ago)    7d3h
kube-system        coredns-76f75df574-5cmgf                   1/1     Running   9 (104m ago)    7d3h
kube-system        etcd-wsl2                                  1/1     Running   32 (46m ago)    7d3h
kube-system        kube-apiserver-wsl2                        1/1     Running   30 (46m ago)    7d3h
kube-system        kube-controller-manager-wsl2               1/1     Running   10 (104m ago)   7d3h
kube-system        kube-proxy-x7mxt                           1/1     Running   9 (104m ago)    7d3h
kube-system        kube-scheduler-wsl2                        1/1     Running   10 (104m ago)   7d3h
tigera-operator    tigera-operator-6fbc4f6f8d-d5fgp           1/1     Running   16 (40m ago)    7d3h

Do dry run on new pod,

oldhorse@wsl2:~$ k run test --image=nginx $do
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: test
  name: test
spec:
  containers:
  - image: nginx
    name: test
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Let's launch test pod if everything looks good,

oldhorse@wsl2:~$ k run test --image=nginx 
pod/test created
oldhorse@wsl2:~$ k get pod 
NAME   READY   STATUS              RESTARTS   AGE
test   0/1     ContainerCreating   0          

oldhorse@wsl2:~$ k get pod -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP              NODE   NOMINATED NODE   READINESS GATES
test   1/1     Running   0          48s   192.168.9.254   wsl2   <none>           <none>

oldhorse@wsl2:~$ curl 192.168.9.254
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Conclusion

So far the one single node k8s cluster lab is ready on WSL2:

k8s cluster has static ip, can be accessed from inside and outside of WSL2
it is docker based so same lab env for any docker related test
it is ready for local LLM AI test on both docker and k8s

About Me

Hey! I am Robert Wang, live in Montreal.

More simple and more efficient.

GitHub: robertluwang
Twitter: robertluwang
LinkedIn: robertluwang
Medium: robertluwang
Dev.to: robertluwang
Web: dreamcloud.artark.ca