Forem: Selvamraju007

Connect GCP VM Instances with VS Code on Windows 10

Selvamraju007 — Mon, 22 May 2023 13:24:32 +0000

In this blog we are going to see how connect GCP VMs with VS Code on Windows 10 machine.

Please refer the below step by step for your reference.

Step 1: Install google cloud cli on your local machine

Install cloud SDLK using the https://cloud.google.com/sdk/docs/install. I’ve installed via PowerShell command on my local windows machine, you can choose installation method based on your OS.

Step 2: Open Cloud SDK and login using below command

gcloud init

And login with your GCP’s account

Step 3: Open PowerShell and execute below command to create SSH key

ssh-keygen -t rsa -f [KEY_FILENAME] -C [USERNAME]

[KEY_FILENAME] is the name of SSH key file that will be created
[USERNAME] is the username that will use to log in that VM

ssh-keygen -t rsa -f keyfile -C mytest@gmail.com

This command will create private and public key.

Step 4: Login GCP console and navigate VM which is created already

Go to Compute Engine-> Click VM instances tab-> Click VM name

Click Edit and find SSH Key option save your public key which is created earlier.

Click + ADD ITEM and save your public key

Save and now ready to connect using VS code.

Step 5: Connect with VS code

Install Remote - SSH extension

Now click Remote— SSH and Press “+”

Enter the below command to create connection

ssh -i C:\\Users\\[YourUser]\\[KEY_FILENAME] [USERNAME]@[VM's IP]
[VM’s IP]- Use external IP of your VM

Now click the IP address to establish connection.

Conclusion:

Now successfully connected GCP VM’s with VS code on windows 10 machine. Hope this helps !

Happy learning !

Install Linux on Windows with WSL

Selvamraju007 — Fri, 19 May 2023 13:53:55 +0000

In this blog, we are going to learn how to install Linux on Windows with WSL.

WSL:

The Windows Subsystem for Linux (WSL) lets developers install a Linux distribution (such as Ubuntu, OpenSUSE, Kali, Debian, Arch Linux, etc) and use Linux applications, utilities, and Bash command-line tools directly on Windows, unmodified, without the overhead of a traditional virtual machine or dualboot setup.

Prerequisites:

You must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11 to use the commands below.

Install WSL command:
You can now install everything you need to run WSL with a single command. Open PowerShell or Windows Command Prompt in administrator mode by right-clicking and selecting “Run as administrator”, enter the wsl — install command, then restart your machine.

wsl --install

This command will enable the features necessary to run WSL and install the Ubuntu distribution of Linux.

If you’re running an older build, or just prefer not to use the install command and would like step-by-step directions, see WSL manual installation steps for older versions.

Change the default Linux distribution installed
By default, the installed Linux distribution will be Ubuntu. This can be changed using the -d flag.

To change the distribution installed, enter: wsl --install -d . Replace with the name of the distribution you would like to install.
To see a list of available Linux distributions available for download through the online store, enter: wsl --list --online or wsl -l -o.
To install additional Linux distributions after the initial install, you may also use the command: wsl --install -d .

Basic commands for WSL:
The WSL commands below are listed in a format supported by PowerShell or Windows Command Prompt. To run these commands from a Bash / Linux distribution command line, you must replace wsl with wsl.exe. For a full list of commands, run wsl --help.

Install:

wsl --install

List available Linux distributions

wsl --list --online

List installed Linux distributions

wsl --list --verbose

See a list of the Linux distributions installed on your Windows machine, including the state (whether the distribution is running or stopped) and the version of WSL running the distribution (WSL 1 or WSL 2). Comparing WSL 1 and WSL 2. This command can also be entered as: wsl -l -v. Additional options that can be used with the list command include: --all to list all distributions, --running to list only distributions that are currently running, or --quiet to only show distribution names.

I’ve tried above stpes and working fine for me.

Happy learning !

Kubernetes Pod Sizing

Selvamraju007 — Wed, 17 May 2023 12:26:25 +0000

Kubernetes has become the de-facto standard for container orchestration and is used extensively in the modern DevOps landscape. When deploying applications to Kubernetes, one of the key considerations is the sizing of the Kubernetes pods.

A Kubernetes pod is the smallest deployable unit in Kubernetes and is composed of one or more containers that share the same network namespace and storage volumes. The pod is designed to run a single instance of a container, and it is responsible for managing the container’s lifecycle, such as starting, stopping, and restarting the container.

When sizing a Kubernetes pod, there are several factors to consider, including the amount of CPU and memory required by the container, the number of containers in the pod, and the amount of storage required.

CPU Sizing

CPU sizing is one of the critical factors to consider when sizing a Kubernetes pod. Kubernetes allocates CPU resources to each pod, and the amount of CPU allocated to a pod determines how much processing power the pod has available.

When sizing CPU resources for a pod, you should consider the amount of CPU required by the container to run effectively. You can specify the CPU resource requests and limits for the container in the pod specification. The requests are the minimum amount of CPU required by the container, while the limits are the maximum amount of CPU that the container can use.

Memory Sizing

Memory sizing is another critical factor to consider when sizing a Kubernetes pod. Memory allocation in Kubernetes is similar to CPU allocation, and the amount of memory allocated to a pod determines how much memory the pod has available.

When sizing memory resources for a pod, you should consider the amount of memory required by the container to run effectively. You can specify the memory resource requests and limits for the container in the pod specification. The requests are the minimum amount of memory required by the container, while the limits are the maximum amount of memory that the container can use.

Number of Containers

The number of containers in a Kubernetes pod is another factor to consider when sizing a pod. When you add multiple containers to a pod, you need to ensure that each container has enough CPU and memory resources to run effectively. You should also consider the inter-container communication requirements when sizing a pod with multiple containers.

Storage Sizing

Storage is another critical factor to consider when sizing a Kubernetes pod. The amount of storage required by a container depends on the nature of the application being deployed. You can specify the storage requirements for the container in the pod specification, including the type of storage, the amount of storage, and the storage class to use.

Here are some examples of Kubernetes pod sizing:

Example 1: Web Application

Suppose you want to deploy a web application to Kubernetes using a pod. The application requires 2 CPU cores and 4GB of memory to run effectively. In this case, you would specify a CPU request of 2 cores and a memory request of 4GB in the pod specification. You may also specify higher CPU and memory limits if needed. If the application requires persistent storage, you would also specify the storage requirements in the pod specification.

apiVersion: v1
kind: Pod
metadata:
  name: web-app-pod
spec:
  containers:
  - name: web-app-container
    image: web-app-image
    resources:
      requests:
        cpu: "2"
        memory: "4Gi"
      limits:
        cpu: "4"
        memory: "8Gi"
    volumeMounts:
    - name: data-volume
      mountPath: /data
  volumes:
  - name: data-volume
    persistentVolumeClaim:
      claimName: data-pvc

Example 2: Microservices Architecture

Suppose you have a microservices architecture where each microservice runs in its own container within a Kubernetes pod. Each microservice requires 0.5 CPU cores and 1GB of memory to run effectively. If you have 10 microservices, you would need a pod with a CPU request of 5 cores and a memory request of 10GB. You would also need to ensure that the inter-container communication between the microservices is efficient.

apiVersion: v1
kind: Pod
metadata:
  name: microservices-pod
spec:
  containers:
  - name: microservice-1
    image: microservice-1-image
    resources:
      requests:
        cpu: "500m"
        memory: "1Gi"
  - name: microservice-2
    image: microservice-2-image
    resources:
      requests:
        cpu: "500m"
        memory: "1Gi"
  # Add more microservices containers as needed

Example 3: Machine Learning Application

Suppose you want to deploy a machine learning application to Kubernetes using a pod. The application requires 8 CPU cores and 16GB of memory to run effectively. You would specify a CPU request of 8 cores and a memory request of 16GB in the pod specification. If the application requires GPU resources, you would also need to specify the GPU resource requirements in the pod specification.

apiVersion: v1
kind: Pod
metadata:
  name: ml-app-pod
spec:
  containers:
  - name: ml-app-container
    image: ml-app-image
    resources:
      requests:
        cpu: "8"
        memory: "16Gi"
      limits:
        cpu: "16"
        memory: "32Gi"
    volumeMounts:
    - name: data-volume
      mountPath: /data
    # Add GPU resources if needed
  volumes:
  - name: data-volume
    persistentVolumeClaim:
      claimName: data-pvc

Example 4: Stateful Application

Suppose you have a stateful application that requires persistent storage and runs in a single container within a Kubernetes pod. The container requires 1 CPU core and 2GB of memory to run effectively, and the application requires 100GB of persistent storage. In this case, you would specify a CPU request of 1 core, a memory request of 2GB, and a persistent storage request of 100GB in the pod specification.

apiVersion: v1
kind: Pod
metadata:
  name: stateful-app-pod
spec:
  containers:
  - name: stateful-app-container
    image: stateful-app-image
    resources:
      requests:
        cpu: "1"
        memory: "2Gi"
    volumeMounts:
    - name: data-volume
      mountPath: /data
  volumes:
  - name: data-volume
    persistentVolumeClaim:
      claimName: data-pvc
      # Specify the size of the persistent volume
      resources:
        requests:
          storage: 100Gi

These are just a few examples of Kubernetes pod sizing. The actual resource requirements for your applications may vary depending on the nature of the application, the workload, and the expected usage patterns. It is essential to monitor the pod’s resource utilization and adjust the pod sizing as needed to ensure optimal performance and prevent resource starvation.

Conclusion

Sizing a Kubernetes pod involves careful consideration of several factors, including CPU and memory requirements, the number of containers, and storage requirements. By considering these factors, you can ensure that your Kubernetes pods are sized appropriately for the applications they host. Remember to monitor your pods’ resource utilization and adjust your sizing as necessary to optimize performance and prevent resource starvation.

Thanks for reading the article, hope this helps !

Kubernetes Logging Best Practices

Selvamraju007 — Tue, 16 May 2023 12:09:42 +0000

Kubernetes is a popular open-source platform for container orchestration that is widely used by developers and DevOps teams for deploying and managing containerized applications. One critical aspect of running any application on Kubernetes is logging, which helps to monitor the health and performance of the application and troubleshoot issues quickly.

In this blog, we will discuss Kubernetes logging and the best practices for logging in a Kubernetes environment.

What is Kubernetes Logging?

Kubernetes logging refers to the process of capturing and storing log data generated by the Kubernetes cluster and the applications running on it. This data includes information about application performance, errors, warnings, and other events. Kubernetes logs are critical for debugging and troubleshooting issues in a distributed system and are used by developers, operators, and security teams to monitor and maintain application health.

How does Kubernetes Logging work?

Kubernetes logs are generated by various components of the Kubernetes cluster, including the Kubernetes API server, kubelet, container runtime, and the applications running on the cluster. These logs are collected and aggregated by a logging agent that runs as a Kubernetes pod and sends them to a central location for storage and analysis.

The most common logging agent used in Kubernetes is Fluentd, which is an open-source data collector that can collect, transform, and forward log data to various destinations, including Elasticsearch, Splunk, and Kafka. Other popular logging agents include Logstash and Fluent Bit.

Here is the diagram for how logging using Fluentd works.

Best Practices for Kubernetes Logging:

Here are some best practices for logging in a Kubernetes environment:

Use a centralized logging solution — A centralized logging solution allows you to store and analyze logs in a central location, making it easier to troubleshoot issues and monitor application performance. Elasticsearch, Splunk, and Loggly are some popular centralized logging solutions used with Kubernetes.
Implement log rotation — Log rotation is the process of deleting or archiving old logs to prevent disk space from filling up. It is essential to configure log rotation for your Kubernetes logs to avoid running out of disk space.
Use structured logging — Structured logging formats log messages in a structured way, making it easier to search, filter, and analyze log data. Use JSON or other structured logging formats instead of plain text to make it easier to extract information from logs.
Avoid logging sensitive information — Avoid logging sensitive information such as passwords, API keys, and other credentials in your Kubernetes logs. Instead, use environment variables or secrets to store this information securely.
Include contextual information — Including contextual information such as timestamps, hostnames, and request IDs in your logs makes it easier to correlate log events and troubleshoot issues.
Tagging privacy information:
To ensure the security and privacy of sensitive data, it is advisable to tag or mask any sensitive information present in your logs. You can use a standardized format like "priv" or "****" to identify and mask sensitive data such as passwords, credit card numbers, or personally identifiable information (PII).
Using different log levels:
Utilizing different log levels allows you to categorize log messages based on their severity or importance. Common log levels include DEBUG, INFO, WARNING, ERROR, and CRITICAL. By using appropriate log levels, you can filter and prioritize log messages based on their significance during troubleshooting and debugging.
Streaming logs to a log server:
Rather than relying solely on local log files, it is recommended to stream logs to a centralized log server or a log management system. This ensures that logs are collected and stored centrally, making it easier to search, analyze, and retain logs for longer periods. Popular log servers and management systems include Elasticsearch, Splunk, and Graylog.
Enabling log rotation:
Configure log rotation to prevent log files from growing indefinitely and consuming excessive disk space. By setting log rotation parameters, you can control log file size, number of log files to retain, and frequency of rotation. This practice ensures that logs are managed efficiently and prevents disk space issues.

While kubectl logs is the standard command for retrieving logs in Kubernetes, there are also several open-source CLI tools available that provide additional features and enhanced log collection capabilities. One such tool is kubetail, which allows you to tail logs from multiple pods simultaneously. Here are a few open-source CLI tools that you can use for log collection in Kubernetes:

Kubetail:
Kubetail is a simple utility that enables you to tail logs from multiple pods within a Kubernetes cluster. It aggregates logs from multiple pods and streams them to your terminal, allowing you to view real-time logs from various sources at once. Kubetail is particularly useful when troubleshooting distributed systems or monitoring application behavior across multiple pods.
Stern:
Stern is another powerful log tailing tool specifically designed for Kubernetes. It allows you to tail logs from multiple pods and containers within a Kubernetes cluster, providing color-coded output for easy identification. Stern also supports regex-based filtering, making it convenient to focus on specific log messages or patterns.
Kail:
Kail is a CLI tool that enables log streaming and tailing from multiple pods in a Kubernetes cluster. It supports log output in both raw and JSON formats, making it flexible for different use cases. Kail also provides options for filtering logs based on labels or namespaces, allowing you to narrow down log output as needed.
Logcli:
Logcli is part of the Loki ecosystem and provides a command-line interface to query logs stored in Loki, a horizontally scalable log aggregation system. It allows you to search and retrieve logs based on various parameters, including labels, time ranges, and log levels. Logcli is useful when you have a centralized logging solution based on Loki and want to query and analyze logs efficiently.

These open-source CLI tools provide additional functionality and flexibility for log collection and analysis in Kubernetes. Depending on your specific requirements, you can choose the tool that best fits your needs and integrates well with your existing logging infrastructure.

Please note that when using any third-party tools, ensure that they are from trusted sources and review their documentation for installation and usage instructions.

There are various commands that you can use to check logs in a Kubernetes environment. Here is a list of commands that you can use with kubetail to check Kubernetes logs:

To tail logs from a specific pod:

kubetail <pod-name>

This command will stream and display logs from the specified pod. You can include multiple pod names to tail logs from multiple pods simultaneously.

To tail logs from pods matching a label selector:

kubetail -l <label-selector>

This command will stream and display logs from all the pods that match the specified label selector.

To tail logs from pods in a specific namespace:

kubetail -n <namespace>

This command will stream and display logs from all the pods in the specified namespace.

To tail logs from multiple pods using wildcard expressions:

kubetail <pod-name-1> <pod-name-2> ...

This command allows you to specify multiple pod names or use wildcard expressions to tail logs from multiple pods. For example:

kubetail my-app-*

This will tail logs from all pods that start with "my-app-".

To include timestamps in log output:

kubetail -t <pod-name>

This command will display logs with timestamps, allowing you to see when log messages were generated.

To stop tailing logs from a specific pod:

kubetail --stop <pod-name>

This command will stop streaming and displaying logs from the specified pod.

These are some of the common commands you can use with kubetail to efficiently tail logs from multiple pods in a Kubernetes cluster. Make sure to install and set up kubetail before using these commands. Refer to the kubetail documentation for installation instructions and additional features.

When using kubetail, you can also view the logs of Kubernetes control plane components by specifying their pod names. Here's how you can view the logs of some commonly used control plane components:

To view the logs of the Kubernetes API server:

kubetail -n kube-system kube-apiserver-<node-name>

Replace <node-name> with the actual name of the node where the API server pod is running. This command will tail the logs of the API server.

To view the logs of the Kubernetes controller manager:

kubetail -n kube-system kube-controller-manager-<node-name>

Replace <node-name> with the name of the node where the controller manager pod is running. This command will tail the logs of the controller manager.

To view the logs of the Kubernetes scheduler:

kubetail -n kube-system kube-scheduler-<node-name>

Replace <node-name> with the name of the node where the scheduler pod is running. This command will tail the logs of the scheduler.

To view the logs of the etcd cluster:

kubetail -n kube-system etcd-<node-name>

Replace <node-name> with the name of the node where the etcd pod is running. This command will tail the logs of the etcd cluster.

Remember to replace <node-name> with the actual node name where the respective control plane component pod is running. The -n kube-system flag is used to specify the namespace where the control plane components are typically deployed.

By tailing the logs of these control plane components, you can monitor their activity, detect any issues or errors, and gain insights into the behavior of your Kubernetes cluster.

Conclusion

Kubernetes logging is a critical aspect of running applications on Kubernetes. A centralized logging solution, log rotation, structured logging, avoiding sensitive information, and including contextual information are some best practices for logging in a Kubernetes environment. By following these best practices, you can ensure that your Kubernetes logs are stored securely, easy to search and analyze, and help you troubleshoot issues quickly.

Thanks for reading the article, hope this helps !