Forem: Dallin

Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS) - Part 3

Dallin — Tue, 14 Nov 2023 17:24:22 +0000

This is the third and final part of the series on Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS). In the last two articles, we discussed various topics. These include what Service Mesh is, what Istio is, what technologies we used, the prerequisites and architecture overview, reviewed the Terraform code used to deploy the AWS resources and successfully deployed the Terraform Code to AWS. We also installed the the Istio CLI tool, istioctl, ran the "configure.sh" script to configure the Flux Repository, and installed Flux to the Amazon EKS cluster. Finally, we reviewed the Addons and Applications managed by Flux, discussed the Istio Components and Addons used by Istio, and reviewed how Istio works with Applications and Microservices.

In this final and third article, we will cover the following topics.

Access the Applications managed by Flux
Demonstrate how Istio works
Review the Istio Addons
Clean up apps and infrastructure

You can access the code in my GitHub Repository.

Access Applications managed by Flux

Let's access the Applications managed by Flux. Retrieve the public_base_domain_name specified in the "locals.tf" when you ran Terraform earlier.

Access Bookinfo App

For the Bookinfo App, you can access the Bookinfo URL "https://bookinfo./productpage" in your favorite web browser.

Access Podinfo App

For the Podinfo App, you can access the Podinfo URL "https://podinfo." in your favorite web browser.

Access Istio Addons

Let's access the Istio Addons before we demonstrate how Istio works. Retrieve the public_base_domain_name specified in the "locals.tf" when you ran Terraform earlier.

Access Kiali Dashboard

To access the Kiali Dashboard, follow these steps.

Access the Kiali URL "https://kiali." in your favorite web browser.

Generate and retrieve token by running the following command

kubectl -n istio-system create token kiali-service-account

Copy and paste the generated token from above to the Kiali Login page.
Log into the Kiali Dashboard.

Access Grafana Dashboard

To access the Grafana Dashboard, follow these steps.

Access the Kiali URL "https://grafana." in your favorite web browser.

Log into Grafana with the following username and password.

Username: admin
Password: Grafana&Git0ps

Demonstrate how Istio works

Now that we know how to access the Applications let's demonstrate how Istio works. We will send several requests to the Bookinfo and Podinfo applications by running curl commands from the terminals or shells. We will do this to simulate enough traffic for Istio and the Istio Addons to capture the network traffic so the dashboards in Kiali and Grafana can display correctly.

Populate Data for Kiali and Istio Dashboards in Grafana

Retrieve the public_base_domain_name specified in the "locals.tf" when you ran Terraform earlier.
Open two separate terminals or shells.

NOTE: Run these commands as often as you want to populate the data in Kiali and Grafana.

Run the following command in the first terminal or shell to send traffic to the Bookinfo application. Replace with the actual domain name.
```
for i in {1..720}; do curl -s -o /dev/null "https://bookinfo.<public_base_domain_name>/productpage" ; done
```
In the second terminal or shell, run the following command to send traffic to the Podinfo application. Replace with the actual domain name.
```
for i in {1..720}; do curl -s -o /dev/null "https://podinfo.<public_base_domain_name>" ; done
```

It will take 4 to 5 minutes for the above commands to finish.

Access and Review Data in Kiali Dashboard

While the commands above run, re-access the Kiali Dashboard and review the data.

Let's change the time range to the last 10 minutes by clicking in the upper right corner of the web page. Click on "Last 1m" to "Last 10m".
Click on "Graph" on the left side of the Kiali Dashboard.
Click on "Select Namespaces" and choose "bookinfo".
Click anywhere on the Kiali Dashboard, and you should see something similar.
Click on "Display" and select "Response Time" and "Security".
Click anywhere on the Kiali Dashboard again, and you should see the response times and a little green lock on each connection.
Click on "productpage v1" on the Kiali Graph Dashboard, and you should see something like this. It will display information on the right-hand side of the Dashboard, such as Traffic and Traces.
Let's add the podinfo namespace to the Kiali Graph Dashboard. Click Namespace again and then podinfo. After selecting, you should see something like this.
Feel free to explore other parts of the Kiali Dashboard. The Kiali Dashboard pulls its data from Prometheus, Grafana, and Jaeger.

Access Istio Dashboards in Grafana

Let's re-access the Grafana Dashboards for Istio and review the data. For details on what each Istio Dashboard does, click here.

NOTE: If the data is not showing in the Grafana Dashboards for Istio, re-run the commands from above to populate data for Bookinfo and Podinfo again.

Change to the Istio Dashboards in Grafana by clicking on Home in the upper left of the Grafana home page
Click on Dashboards
Hover over Istio and then click "Go to folder".
Istio Dashboards should now appear.
Click on Istio Mesh Dashboard In Grafana. This dashboard provides a global view of all services and applications used in the Service Mesh.
Let's navigate to the Istio Service Dashboard by clicking on Istio in the upper left-right next to "Home > Dashboards > Istio" and then click "Istio Service Dashboard".
Click on service on the Istio Service Dashboard and change to "productpage.bookinfo.svc.cluster.local" service and then expand General, Client Workloads, and Service Workloads sections.
Review the General, Client Workloads, and Service Workloads sections. In the Client Workloads and Service Workloads sections, you can see that mutual TLS (mTLS) is enabled and working correctly.
Feel free to explore other parts of the Istio Dashboards in Grafana.

Clean Up

Clean up Applications managed by Flux from Kubernetes

Suspend Applications managed by Flux

flux suspend source git flux-system
flux suspend kustomization apps flux-system \
  monitoring-controllers monitoring-configs

Delete Applications managed by Flux

kubectl delete -f ./k8s/apps/base/bookinfo/config.yaml
kubectl delete -f ./k8s/apps/base/bookinfo/cert_request.yaml
kubectl delete -f ./k8s/apps/base/podinfo/config.yaml
kubectl delete -f ./k8s/apps/base/podinfo/cert_request.yaml
kubectl delete -f ./k8s/apps/base/bookinfo/release.yaml
flux delete helmrelease -s podinfo
flux delete helmrelease -s -n monitoring loki-stack
flux delete helmrelease -s -n monitoring kube-prometheus-stack

Wait 1 to 5 minutes for Applications to be removed from Kubernetes

Delete Application sources managed by Flux

flux delete source helm -s podinfo
flux delete source helm -s -n monitoring grafana-charts
flux delete source helm -s -n monitoring prometheus-community

Verify Applications are removed

kubectl -n bookinfo get all
kubectl -n istio-ingress get ingresses bookinfo-ingress
kubectl -n podinfo get all
kubectl -n istio-ingress get ingresses podinfo-ingress
kubectl -n monitoring get all

Clean up Infrastructure Applications managed by Flux from Kubernetes

Suspend Infrastructure Applications managed by Flux
```
flux suspend kustomization infra-apps
```

Delete Infrastructure Applications managed by Flux

kubectl delete -f ./k8s/infrastructure/apps/kiali/config.yaml --force=true --grace-period=0
kubectl delete -f ./k8s/infrastructure/apps/kiali/cert_request.yaml
kubectl patch kiali kiali -n istio-system -p '{"metadata":{"finalizers": []}}' --type=merge
kubectl delete kiali --all --all-namespaces
kubectl delete jaegers.jaegertracing.io -n observability jaeger
flux delete helmrelease -s kiali
flux delete helmrelease -s jaeger-operator
kubectl delete crd kialis.kiali.io

Wait 1 to 5 minutes for Kubernetes Addons to be removed from Kubernetes

Delete Application sources managed by Flux

flux delete source helm -s jaegertracing
flux delete source helm -s kiali

Verify Kubernetes Addons were removed successfully

kubectl -n kiali-operator get all
kubectl -n observability get all
kubectl -n istio-ingress get ingresses kiali-ingress

If any resources are not deleted, manually delete them.

Clean up Kubernetes Addons managed by Flux from Kubernetes

Suspend Kubernetes Addons managed by Flux

flux suspend kustomization infra-configs infra-controllers

Delete Kubernetes Addons managed by Flux

flux delete helmrelease -s aws-load-balancer-controller
flux delete helmrelease -s cert-manager
flux delete helmrelease -s cluster-autoscaler
flux delete helmrelease -s external-dns
flux delete helmrelease -s istio-base
flux delete helmrelease -s istiod
flux delete helmrelease -s istio-ingressgateway
flux delete helmrelease -s metrics-server

Wait 1 to 5 minutes for Kubernetes Addons to be removed from Kubernetes

Delete Application sources managed by Flux

flux delete source helm -s cert-manager
flux delete source helm -s cluster-autoscaler
flux delete source helm -s eks-charts
flux delete source helm -s external-dns
flux delete source helm -s istio
flux delete source helm -s jetstack
flux delete source helm -s metrics-server

Verify Kubernetes Addons were removed successfully

kubectl -n kube-system get all -l app.kubernetes.io/name=external-dns
kubectl -n kube-system get all -l app.kubernetes.io/name=aws-load-balancer-controller
kubectl -n kube-system get all -l app.kubernetes.io/name=aws-cluster-autoscaler
kubectl -n cert-manager get all
kubectl -n istio-ingress get ingresses kiali-ingress
kubectl -n istio-ingress get all
kubectl -n istio-system get all
kubectl get ingressclasses -l app.kubernetes.io/name=aws-load-balancer-controller

If any resources are not deleted, manually delete them.

Uninstall Flux from Kubernetes

Uninstall Flux
```
flux uninstall -s
```
Verify Flux was removed successfully
```
kubectl get all -n flux-system
```

Clean up Terraform

Navigate to the root of the directory of the GitHub repo and run the following commands
```
cd terraform

terraform destroy
```
Check Terraform destroy results

Conclusion

In conclusion, this guide provided a comprehensive overview of utilizing Istio, a service mesh, and Amazon EKS. A service mesh is an essential infrastructure component that enhances communication and operational capabilities within a microservices architecture. Istio, as a leading service mesh implementation, provides a robust set of features for traffic management, security, and observability without heavy lifting from developers. Istio works seamlessly with Amazon EKS, as EKS supports Kubernetes natively, and Istio is designed to work with Kubernetes-managed services. Users can leverage Istio within EKS to manage microservices traffic, enforce security policies, and observe communication patterns between services, all while taking advantage of the managed Kubernetes environment that EKS provides. This combination allows for a powerful, scalable, and secure system for managing microservices at scale.

Using Flux, a GitOps Tool, Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS) - Part 3

Dallin — Fri, 10 Nov 2023 22:20:31 +0000

In this final and third article, we will cover the following topics.

Access the Applications managed by Flux
Demonstrate how Istio works
Review the Istio Addons
Clean up apps and infrastructure

Access Applications managed by Flux

Let's access the Applications managed by Flux. Retrieve the public_base_domain_name specified in the "locals.tf" when you ran Terraform earlier.

Access Bookinfo App

For the Bookinfo App, you can access the Bookinfo URL "https://bookinfo.public_base_domain_name/productpage" in your favorite web browser.

Access Podinfo App

For the Podinfo App, you can access the Podinfo URL "https://podinfo.public_base_domain_name" in your favorite web browser.

Access Istio Addons

Let's access the Istio Addons before we demonstrate how Istio works. Retrieve the public_base_domain_name specified in the "locals.tf" when you ran Terraform earlier.

Access Kiali Dashboard

To access the Kiali Dashboard, follow these steps.

Access the Kiali URL "https://kiali.public_base_domain_name" in your favorite web browser.

Generate and retrieve token by running the following command

kubectl -n istio-system create token kiali-service-account

Copy and paste the generated token from above to the Kiali Login page.
Log into the Kiali Dashboard.

Access Grafana Dashboard

To access the Grafana Dashboard, follow these steps.

Access the Kiali URL "https://grafana.public_base_domain_name" in your favorite web browser.
Log into Grafana with the following username and password.

Username: admin
Password: Grafana&Git0ps

Demonstrate how Istio works

Populate Data for Kiali and Istio Dashboards in Grafana

Retrieve the public_base_domain_name specified in the "locals.tf" when you ran Terraform earlier.
Open two separate terminals or shells.

NOTE: Run these commands as often as you want to populate the data in Kiali and Grafana.

Run the following command in the first terminal or shell to send traffic to the Bookinfo application. Replace with the actual domain name.
```
for i in {1..720}; do curl -s -o /dev/null "https://bookinfo.<public_base_domain_name>/productpage" ; done
```
In the second terminal or shell, run the following command to send traffic to the Podinfo application. Replace with the actual domain name.
```
for i in {1..720}; do curl -s -o /dev/null "https://podinfo.<public_base_domain_name>" ; done
```

It will take 4 to 5 minutes for the above commands to finish.

Access and Review Data in Kiali Dashboard

While the commands above run, re-access the Kiali Dashboard and review the data.

Let's change the time range to the last 10 minutes by clicking in the upper right corner of the web page. Click on "Last 1m" to "Last 10m"..
Click on "Graph" on the left side of the Kiali Dashboard.
Click on "Select Namespaces" and choose "bookinfo".
Click anywhere on the Kiali Dashboard, and you should see something similar.
Click on "Display" and select "Response Time" and "Security".
Click anywhere on the Kiali Dashboard again, and you should see the response times and a little green lock on each connection.
Click on "productpage v1" on the Kiali Graph Dashboard, and you should see something like this. It will display information on the right-hand side of the Dashboard, such as Traffic and Traces.
Let's add the podinfo namespace to the Kiali Graph Dashboard. Click Namespace again and then podinfo. After selecting, you should see something like this.
Feel free to explore other parts of the Kiali Dashboard. The Kiali Dashboard pulls its data from Prometheus, Grafana, and Jaeger.

Access Istio Dashboards in Grafana

Let's re-access the Grafana Dashboards for Istio and review the data. For details on what each Istio Dashboard does, click here.

NOTE: If the data is not showing in the Grafana Dashboards for Istio, re-run the commands from above to populate data for Bookinfo and Podinfo again.

Change to the Istio Dashboards in Grafana by clicking on Home in the upper left of the Grafana home page
Click on Dashboards
Hover over Istio and then click "Go to folder".
Istio Dashboards should now appear.
Click on Istio Mesh Dashboard In Grafana. This dashboard provides a global view of all services and applications used in the Service Mesh.
Let's navigate to the Istio Service Dashboard by clicking on Istio in the upper left-right next to "Home > Dashboards > Istio" and then click "Istio Service Dashboard".
Click on service on the Istio Service Dashboard and change to "productpage.bookinfo.svc.cluster.local" service and then expand General, Client Workloads, and Service Workloads sections.
Review the General, Client Workloads, and Service Workloads sections. In the Client Workloads and Service Workloads sections, you can see that mutual TLS (mTLS) is enabled and working correctly.
Feel free to explore other parts of the Istio Dashboards in Grafana.

Clean Up

Clean up Applications managed by Flux from Kubernetes

Suspend Applications managed by Flux

flux suspend source git flux-system
flux suspend kustomization apps flux-system \
  monitoring-controllers monitoring-configs

Delete Applications managed by Flux

kubectl delete -f ./k8s/apps/base/bookinfo/config.yaml
kubectl delete -f ./k8s/apps/base/bookinfo/cert_request.yaml
kubectl delete -f ./k8s/apps/base/podinfo/config.yaml
kubectl delete -f ./k8s/apps/base/podinfo/cert_request.yaml
kubectl delete -f ./k8s/apps/base/bookinfo/release.yaml
flux delete helmrelease -s podinfo
flux delete helmrelease -s -n monitoring loki-stack
flux delete helmrelease -s -n monitoring kube-prometheus-stack

Wait 1 to 5 minutes for Applications to be removed from Kubernetes

Delete Application sources managed by Flux

flux delete source helm -s podinfo
flux delete source helm -s -n monitoring grafana-charts
flux delete source helm -s -n monitoring prometheus-community

Verify Applications are removed

kubectl -n bookinfo get all
kubectl -n istio-ingress get ingresses bookinfo-ingress
kubectl -n podinfo get all
kubectl -n istio-ingress get ingresses podinfo-ingress
kubectl -n monitoring get all

Clean up Infrastructure Applications managed by Flux from Kubernetes

Suspend Infrastructure Applications managed by Flux
```
flux suspend kustomization infra-apps
```

Delete Infrastructure Applications managed by Flux

kubectl delete -f ./k8s/infrastructure/apps/kiali/config.yaml --force=true --grace-period=0
kubectl delete -f ./k8s/infrastructure/apps/kiali/cert_request.yaml
kubectl patch kiali kiali -n istio-system -p '{"metadata":{"finalizers": []}}' --type=merge
kubectl delete kiali --all --all-namespaces
kubectl delete jaegers.jaegertracing.io -n observability jaeger
flux delete helmrelease -s kiali
flux delete helmrelease -s jaeger-operator
kubectl delete crd kialis.kiali.io

Wait 1 to 5 minutes for Kubernetes Addons to be removed from Kubernetes

Delete Application sources managed by Flux

flux delete source helm -s jaegertracing
flux delete source helm -s kiali

Verify Kubernetes Addons were removed successfully

kubectl -n kiali-operator get all
kubectl -n observability get all
kubectl -n istio-ingress get ingresses kiali-ingress

If any resources are not deleted, manually delete them.

Clean up Kubernetes Addons managed by Flux from Kubernetes

Suspend Kubernetes Addons managed by Flux

flux suspend kustomization infra-configs infra-controllers

Delete Kubernetes Addons managed by Flux

flux delete helmrelease -s aws-load-balancer-controller
flux delete helmrelease -s cert-manager
flux delete helmrelease -s cluster-autoscaler
flux delete helmrelease -s external-dns
flux delete helmrelease -s istio-base
flux delete helmrelease -s istiod
flux delete helmrelease -s istio-ingressgateway
flux delete helmrelease -s metrics-server

Wait 1 to 5 minutes for Kubernetes Addons to be removed from Kubernetes

Delete Application sources managed by Flux

flux delete source helm -s cert-manager
flux delete source helm -s cluster-autoscaler
flux delete source helm -s eks-charts
flux delete source helm -s external-dns
flux delete source helm -s istio
flux delete source helm -s jetstack
flux delete source helm -s metrics-server

Verify Kubernetes Addons were removed successfully

kubectl -n kube-system get all -l app.kubernetes.io/name=external-dns
kubectl -n kube-system get all -l app.kubernetes.io/name=aws-load-balancer-controller
kubectl -n kube-system get all -l app.kubernetes.io/name=aws-cluster-autoscaler
kubectl -n cert-manager get all
kubectl -n istio-ingress get ingresses kiali-ingress
kubectl -n istio-ingress get all
kubectl -n istio-system get all
kubectl get ingressclasses -l app.kubernetes.io/name=aws-load-balancer-controller

If any resources are not deleted, manually delete them.

Uninstall Flux from Kubernetes

Uninstall Flux
```
flux uninstall -s
```
Verify Flux was removed successfully
```
kubectl get all -n flux-system
```

Clean up Terraform

Navigate to the root of the directory of the GitHub repo and run the following commands
```
cd terraform

terraform destroy
```
Check Terraform destroy results

Conclusion

Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS) - Part 2

Dallin — Fri, 10 Nov 2023 21:47:57 +0000

This is the second part of the series on Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS). The first article discussed what a Service Mesh and Istio are, what technologies we will use, the prerequisites and architecture overview, and the configuration and setup process.

In this part two article, we will be working on these tasks.

Configure access to Amazon EKS Cluster
Install Istio CLI tool - istioctl
Run the "configure.sh" script to configure the Flux Repository
Install Flux to the Amazon EKS Cluster
Review Addons and Applications managed by Flux
Discuss the Istio Components and Addons used by Istio.
Review how Istio works with Applications and Microservices

You can access the code in my GitHub Repository.

Configure access to Amazon EKS Cluster

Amazon EKS Cluster details can be extracted from terraform output or by accessing the AWS Console to get the name of the cluster. This following command can be used to update the kubeconfig in your local machine where you run kubectl commands to interact with your EKS Cluster. Navigate to the root of the directory of the GitHub repo and run the following commands:

cd terraform

AWS_REGION=$(terraform output -raw aws_region)
EKS_CLUSTER_NAME=$(terraform output -raw eks_cluster_name)
aws eks --region $AWS_REGION update-kubeconfig --name $EKS_CLUSTER_NAME

Results of configuring kubeconfig.

Install Istio CLI - istioctl

Install Istioctl CLI

a. For macOS or Linux, follow these instructions using Homebrew
```
brew install istioctl
```
b. For Windows, follow these instructions with Chocolatey.
```
choco install istioctl
```
c. For Windows, follow these instructions with Scoop.
```
scoop bucket add main
scoop install main/istioctl
```
d. Install instructions for other methods can be found here.
Instructions on how to use istioctl can be found here and here.
Verify that istioctl is installed by running the following command.
```
istioctl version
```
Results of running istioctl.

Configure and Install Flux

Configure Variables needed to install Flux

export GITHUB_TOKEN='<REPLACE_WITH_GITHHUB_TOKEN>'
export GITHUB_USER='<REPLACE_WITH_GITHUB_USER>'
export GITHUB_OWNER='<REPLACE_WITH_GITHUB_OWNER>'
export GITHUB_REPO_NAME='<REPLACE_WITH_GITHUB_REPO_NAME>'

Configure the Flux Repository by running the "configure.sh" script. The "configure.sh" script updates the various applications with the necessary values to run correctly. Navigate to the root of the directory of the GitHub repo and run the following commands:
```
cd scripts

./configure.sh
cd ..
```
Results of running the "configure.sh" script.

Install Flux on the Amazon EKS Cluster

flux bootstrap github \
  --components-extra=image-reflector-controller,image-automation-controller \
  --owner=$GITHUB_OWNER \
  --repository=$GITHUB_REPO_NAME \
  --private=false \
  --path=clusters/eks-istio-lab \
  --personal

Results of installing Flux on the Amazon EKS Cluster.
Wait 2 to 5 minutes for Flux to reconcile the Git repository we specified, During this time, Flux will install and configure all of the defined Kubernetes Addons and Applications.
Run the following command to check if all of the Kubernetes Addons and Applications deployed successfully
```
flux get all -A
```

Managing Flux

Managing Flux is handled by using the Flux CLI. Flux does not come with any Web or UI interface to manage Flux. Please click here if you would like more information on the Flux CLI.

The following are some commands you can use to manage Flux.

flux get all
flux get sources all|git|helm|chart
flux get helmreleases
flux get kustomizations
flux logs
flux suspend kustomization <kustomization_name>
flux reconcile source git flux-system

For additional information on using Flux, please look at the following series I wrote about Flux.

Kubernetes Addons managed by Flux

Below are the Applications that Flux manages, the Kubernetes Addons will be deployed and configured by Flux first. The following Kubernetes Addons will be installed.

AWS Application Load Balancer Controller
External DNS
Cluster Autoscaler
Cert manager
Metrics Server

The AWS Application Load Balancer Controller and External DNS must be deployed first because the Applications need to be accessible by a load balancer and have the DNS Name registered with Route 53.

Applications managed by Flux

Flux manages the following applications. These applications will be used to demonstrate the various Istio features.

Bookinfo - A sample application composed of four separate microservices
Podinfo - A tiny web application made with Go

Istio and Istio Addons managed by Flux

Istio is comprised of the following three components.

Istio - An open-source service mesh that provides a uniform way to connect, manage, and secure microservices, enabling traffic flow control, policy enforcement, and telemetry data aggregation without altering service code.
Istiod - Consolidated control plane daemon of the Istio service mesh, responsible for service discovery, configuration management, certificate issuance, and providing overall operational control.
Istio Ingress Gateway - A dedicated network gateway in the Istio service mesh architecture that manages incoming traffic routing to various services within the mesh.

The following components are not necessary to run Istio but are addons to help the observability features of Istio.

Kiali - An open-source observability platform tailored for service mesh deployments, providing insights into the performance and structure of microservices networks within Istio. Kiali will retrieve data from Prometheus, Grafana, and Jaeger if installed and configured correctly.
Jaeger - Distributed Tracing platform - An open-source, end-to-end distributed tracing system that helps monitor and troubleshoot transactions in complex, microservice-based architectures.
Kube Prometheus Stack
- Prometheus - An open-source monitoring system with a dimensional data model, flexible query language, and powerful alerting functionality for storing and querying time-series data.
- Grafana - An open-source analytics and monitoring platform designed for visualizing and exploring metrics from various databases and time-series data.
Grafana Loki - A horizontally scalable, multi-tenant log aggregation system inspired by Prometheus, designed for cost-effective storage and querying of logs at scale.

Review how Istio works with Applications and Microservices

Review Configuration of Kubernetes Namespaces work with Istio

By default, Istio doesn't inject sidecars/proxies into the pods. The easiest way to have Istio inject a sidecar/proxy to a pod is by adding a label to a Kubernetes namespace. When the label is added to an existing Kubernetes namespace, the existing pods must be deleted to add the sidecar/proxy container.

As part of the Bookinfo and Podinfo installation managed by Flux, the Kubernetes namespaces included the label to have Istio inject a sidecar/proxy. Let's review the Kubernetes namespaces that have the label "istio-injection=enabled".

In the Bookinfo namespace.yaml file. As you can see, when this namespace is created, the label "istio-injection=enabled" is added.

apiVersion: v1
kind: Namespace
metadata:
  name: bookinfo
  labels:
    istio-injection: enabled

Let's review which Kubernetes namespaces have the label "istio-injection=enabled".

Run the following command to determine which Kubernetes namespaces have the label "istio-injection=enabled".
The result is four Kubernetes namespaces with the label "istio-injection=enabled".
Check Pods in the Bookinfo namespace as two containers running for each pod by running the following command.
As you can see, each pod is running two containers

Review Configuration of Istio Gateway and VirtualServices

Istio Gateway is a component of the Istio service mesh that manages incoming and outgoing traffic for microservices-based applications. It handles routing, load balancing, security, and TLS termination tasks. You define routing rules, authentication, and authorization policies, and it provides observability and high availability features to ensure efficient and secure communication within the service mesh.

Istio VirtualService is a Kubernetes custom resource definition (CRD) that defines how traffic is routed to services in a service mesh. It is a powerful tool that can be used to implement various traffic management policies, such as load balancing, fault injection, and rate limiting.

Here is the Istio Gateway that was created. This is an Istio Ingress Gateway.

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: common-gateway
  namespace: istio-ingress
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 443
        name: https-443
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: "wildcard-tls"
      hosts:
        - "*"
    - port:
        number: 80
        name: http-80
        protocol: HTTP
      tls:
        httpsRedirect: true
      hosts:
        - "*"

As part of the Istio Gateway, a TLS certificate is required. The TLS certificate can be created in several ways, but for the guide, Cert-Manager was used to create a wildcard certificate with Let's Encrypt.

When Istio Gateways are created, a Kubernetes Service is created. This Kubernetes Service can be defined and used in Kubernetes Ingresses.

Let's review the Bookinfo Kubernetes Ingress to see how this works.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig":
      { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:us-west-2:012345678910:certificate/3860d571-18bc-4c62-af82-92a5d1cc3aba"
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/healthcheck-path: /healthz/ready
    alb.ingress.kubernetes.io/healthcheck-port: status-port
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/backend-protocol: HTTPS
    external-dns.alpha.kubernetes.io/hostname: "bookinfo.dallin.brewsentry.com"
  name: bookinfo-ingress
  namespace: istio-ingress
spec:
  ingressClassName: alb
  tls:
  - hosts:
    - bookinfo.dallin.brewsentry.com
    secretName: bookinfo-tls
  rules:
  - host: bookinfo.dallin.brewsentry.com
    http:
      paths:
      - backend:
          service:
            name: istio-ingressgateway
            port:
              number: 443
        path: /*
        pathType: ImplementationSpecific

The Bookinfo Kubernetes Ingress has a rule that listens for requests from a host and then routes the requests to the istio-ingressgateway Kubernetes Service.

A VirtualService consists of a set of routing rules. Each routing rule defines a match condition and a destination. A request is routed to the destination if it matches the match condition.

Here is the VirtualService for Bookinfo.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: bookinfo-vs
  namespace: bookinfo
spec:
  hosts:
  - "bookinfo.dallin.brewsentry.com"
  gateways:
  - istio-ingress/common-gateway
  http:
  - match:
    - uri:
        exact: /
    redirect:
      uri: /productpage
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080

Let's review what Istio VirtualServices were created.

Run the following command to determine what Istio virtual services were created.
```
kubectl get virtualservices.networking.istio.io -A
```
The result is that three Istio virtual services were created.

In this article, we configured access to Amazon EKS Cluster and installed the Istio CLI - istioctl tool. We ran the "configure.sh" script and installed Flux on the Amazon EKS Cluster. We reviewed the Addons and Applications managed by Flux and discussed the Istio Components and Addons used by Istio. Finally, we reviewed how Istio works with Applications and Microservices.

Please stay tuned for the third and final part of the series, where we will complete the following tasks.

Access the Applications managed by Flux
Demonstrate how Istio works
Review the Istio Addons
Clean up apps and infrastructure

Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS) - Part 1

Dallin — Fri, 10 Nov 2023 20:57:08 +0000

Introduction

Have you ever wondered if there was a better way to manage, monitor, or secure your distributed microservices without adding extra modules to your code? Well, there is a way. It is called a Service Mesh.

This is the first part of the series on Using Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS). This guide will explain what a Service Mesh and Istio are and show you how to use Istio, a Service Mesh, with Amazon Elastic Kubernetes Service (EKS). You will use Flux to deploy various Kubernetes Services and Applications, including Istio, and supporting applications and microservices to Amazon Elastic Kubernetes Service (EKS).

You can access the code in my GitHub Repository.

Before we begin, let's define what Service Mesh and Istio are.

What is a Service Mesh?

A service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between microservices, often in a cloud-based application. It's designed to handle a high volume of network-based inter-process communication among application infrastructure services using application programming interfaces (APIs).

Here's a breakdown of the key components and concepts of a service mesh:

Sidecar Proxy: In a typical service mesh, each microservice is paired with a lightweight network proxy that intercepts network communication. This proxy, often called a "sidecar," intercepts all network communication to and from the service.
Data Plane: The collection of sidecar proxies that intercept and control the network traffic between services make up the data plane. They are responsible for features like service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Control Plane: This is the administrative layer of the service mesh that provides the management capabilities. It allows operators to configure the proxies in the data plane and apply policies. It's also where the service mesh’s intelligence resides, as it controls the behavior of the proxy servers.
Service Discovery: Service meshes can dynamically recognize the addition of new services and the removal of old ones, adapting to the changes in the infrastructure.
Traffic Management: It can intelligently control the flow of traffic and API calls between services, handling load balancing, routing, and circuit breaking.
Security: Service meshes can handle encryption in transit and enforce security policies, like mutual TLS (mTLS), for service identity verification and secure communication.
Observability: Provides monitoring, logging, and tracing of service interactions, which is vital for diagnosing issues and understanding system behavior.
Policy Enforcement: It allows operators to apply organizational policies related to security, compliance, and governance across all service communications.

Examples of Service Meshes:

Istio: One of the most popular service mesh solutions, Istio is designed to connect, secure, control, and observe services. It works with Kubernetes but can also be adapted to integrate with other environments. It provides robust traffic management, security features, and observability.
Linkerd: Known for its simplicity and ease of use, Linkerd is a lightweight service mesh that offers core service mesh features like service discovery, routing, failure handling, and metrics. It's designed to be as transparent as possible and can be used with Kubernetes and other frameworks.
Consul: Consul by HashiCorp provides a full-featured control plane with service discovery, health checking, and a service mesh. It can be used with Kubernetes or virtual machines and is known for its cross-datacenter functionality.
AWS App Mesh: This service mesh provided by AWS makes it easy to manage and communicate between microservices across multiple types of compute infrastructure. App Mesh standardizes how your services communicate, giving end-to-end visibility and helping to ensure high availability for your applications.

Service meshes are helpful because they offload common functionalities from the application code, allowing developers to focus on the business logic rather than the complexities of networking. However, they also add a new layer to the infrastructure stack, which can increase complexity and the operational burden if not managed properly.

What is Istio?

Istio is an open-source service mesh that helps organizations run distributed, microservices-based applications more securely, reliably, and efficiently. It's designed to connect, monitor, and secure microservices, providing tools to manage traffic flows between services, enforce policies, and aggregate telemetry data. Google, IBM, and Lyft originally developed it and are now part of the Cloud Native Computing Foundation (CNCF). Istio is platform-independent but often associated with Kubernetes, commonly used to orchestrate containers that host microservices.

Here's a brief overview of what Istio can do:

Traffic Management: Istio provides advanced routing capabilities, allowing for A/B testing, canary releases, and more. It does this by managing the flow of traffic and API calls between services, which is crucial in a microservices architecture. It enables request retries, fault injection, and traffic splitting for increased resilience and control.
Security: Istio enhances security by providing inter-service authentication and authorization without requiring changes to the application. It manages certificates and keys and ensures encrypted communication between services with mutual TLS (mTLS), helping to reduce the risk of man-in-the-middle attacks.
Observability: It offers insights into the behavior of the services, including monitoring, logging, and tracing of the communications, which helps in understanding the performance and issues in the applications.
Policy Enforcement: Istio allows you to enforce policies regarding access control and resource usage across your microservices without modifying the services themselves.
Service Discovery: Istio supports service discovery mechanisms, making it easier for services to find and communicate with each other in a dynamic environment.

Istio achieves these features by deploying a special sidecar proxy (Envoy proxy) alongside each service instance. These proxies intercept and control all network communication between microservices, and they are managed by Istio's control plane, which provides the necessary rules for the proxies.

Istio’s service mesh architecture is composed of:

Data Plane: Consisting of intelligent proxies (Envoy) deployed as sidecars that mediate and control all network communication between microservices.
Control Plane: Manages and configures the proxies to route traffic and enforces policies at runtime.

Istio is designed to work on various environments, including on-premises, cloud-hosted, and hybrid environments, and it can run with services written in any language.

Istio operates at the platform layer, providing the infrastructure necessary to manage the interactions between microservices, while the application code remains unaware of Istio’s existence. It is typically used in Kubernetes environments but can also be used with other orchestration solutions. Its control plane architecture is responsible for managing the overall configuration and behavior of the service mesh.

By decoupling development concerns from operational challenges, Istio provides a uniform way to secure, connect, and monitor microservices.

Technologies we are going to use:

HashiCorp Terraform
Istio
Flux
GitHub
Amazon Elastic Kubernetes Service (EKS)
Amazon Elastic Container Registry (ECR)
AWS Key Management Service (KMS)
Amazon Route 53
AWS Certificate Manager (ACM)
Amazon Virtual Private Cloud (Amazon VPC)
IAM policies and roles

Prerequisites

Before you begin, make sure you have the following before starting:

An active AWS account. You can create a new AWS account here.
AWS CLI installed and configured. Instructions can be found here.
Terraform installed. Instructions can be found here.
Helm installed. Instructions can be found here.
Kubernetes CLI (kubectl). Instructions can be found here.
Flux CLI. Instructions can be found here.
A GitHub Personal Access Token. Instructions can be found here.

Architecture Overview

Amazon Elastic Kubernetes Service (EKS)

# Create AWS EKS Cluster
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                    = local.eks_cluster_name
  cluster_version                 = local.eks_cluster_version
  cluster_endpoint_private_access = true
  cluster_endpoint_public_access  = true

  cluster_addons = {
    kube-proxy = {
      most_recent                 = true
      resolve_conflicts           = "OVERWRITE"
      resolve_conflicts_on_update = "OVERWRITE"
    }
    vpc-cni = {
      most_recent                 = true
      resolve_conflicts           = "OVERWRITE"
      resolve_conflicts_on_update = "OVERWRITE"
      service_account_role_arn    = module.vpc_cni_ipv4_irsa_role.iam_role_arn
    }
    aws-ebs-csi-driver = {
      most_recent                 = true
      resolve_conflicts           = "OVERWRITE"
      resolve_conflicts_on_update = "OVERWRITE"
      service_account_role_arn    = module.ebs_csi_irsa_role.iam_role_arn
    }
    coredns = {
      most_recent                 = true
      resolve_conflicts           = "OVERWRITE"
      resolve_conflicts_on_update = "OVERWRITE"
    }
  }

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  depends_on = [module.vpc]
}
...

AWS Certificate Manager (ACM)

# Create SSL Certificate using AWS ACM for Bookinfo
resource "aws_acm_certificate" "bookinfo" {
  domain_name       = "bookinfo.${local.public_base_domain_name}"
  validation_method = "DNS"

  lifecycle {
    create_before_destroy = true
  }
}

# Validate SSL Certificate using DNS for Bookinfo
resource "aws_route53_record" "bookinfo_validation" {
  for_each = {
    for dvo in aws_acm_certificate.bookinfo.domain_validation_options : dvo.domain_name => {
      name   = dvo.resource_record_name
      record = dvo.resource_record_value
      type   = dvo.resource_record_type
    }
  }

  allow_overwrite = true
  name            = each.value.name
  records         = [each.value.record]
  ttl             = 60
  type            = each.value.type
  zone_id         = local.route53_zone_id
}

# Retrieve SSL Certificate ARN from AWS ACM for Bookinfo
resource "aws_acm_certificate_validation" "bookinfo" {
  certificate_arn         = aws_acm_certificate.bookinfo.arn
  validation_record_fqdns = [for record in aws_route53_record.bookinfo_validation : record.fqdn]
}
...

Amazon Virtual Private Cloud (Amazon VPC)

# Create AWS VPC and Subnets
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0.0"

  private_subnets     = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
  public_subnets      = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]

  name                 = local.vpc_name
  cidr                 = local.vpc_cidr
  azs                  = local.azs
  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true
  enable_dns_support   = true

  create_flow_log_cloudwatch_iam_role             = true
  create_flow_log_cloudwatch_log_group            = true
  enable_dhcp_options                             = true
  enable_flow_log                                 = true
  flow_log_cloudwatch_log_group_retention_in_days = 7
  flow_log_max_aggregation_interval               = 60

  public_subnet_tags = {
    "kubernetes.io/cluster/${local.eks_cluster_name}" = "shared"
    "kubernetes.io/role/elb"                          = 1
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/${local.eks_cluster_name}" = "shared"
    "kubernetes.io/role/internal-elb"                 = 1
  }
}

IAM policies and roles

# Create IAM Role for AWS ALB Service Account
module "load_balancer_controller_irsa_role" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version = "~> 5.0"

  role_name                              = "${local.eks_iam_role_prefix}-aws-load-balancer-controller"
  attach_load_balancer_controller_policy = true

  oidc_providers = {
    ex = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["kube-system:aws-load-balancer-controller"]
    }
  }
}
...

Amazon ALB

Amazon ALBs will automatically be created when you deploy the various apps using Flux as the deployment tool. This will be discussed in part 2 of this article.

Amazon Route 53

When you deploy the various apps using Flux, the public domains you will use will be registered automatically on Amazon Route 53. This will be discussed in part 2 of this article.

We just finished reviewing the architecture that will be created by Terraform code. Several of the code blocks from above are just snippets of code. Please see the git repo for the complete code.

Setup and Deploy Infrastructure

Follow these steps to set up the environment.

Set variables in "locals.tf". Below are some of the variables that should be set.

aws_region
aws_profile
tags
public_base_domain_name

Update Terraform S3 Backend in provider.tf

bucket
key
profile
dynamodb_table

Navigate to the Terraform directory
```
cd terraform
```
Initialize Terraform
```
terraform init
```
Validate the Terraform code
```
terraform validate
```
Run, review, and save a Terraform plan
```
terraform plan -out=plan.out
```
Apply the Terraform plan
```
terraform apply plan.out
```
Review Terraform apply results

After completing the above steps, you should have a running and working Amazon EKS Cluster.

Please stay tuned for part two of the series, where we will complete the following tasks.

Configure access to Amazon EKS Cluster
Run the configure.sh script to configure the Flux Repository
Install Flux to the Amazon EKS Cluster
Review the Addons and Applications managed by Flux
Discuss the Istio Components and Addons used by Istio
Review how Istio works with Applications and Microservices