Forem: Javier Marasco

Managed cluster vs unmanaged clusters in Kubernetes

Javier Marasco — Tue, 27 Jun 2023 08:30:00 +0000

As we see in the latest article here a Kubernetes cluster can contain a lot of components just to function properly and as you can imagine, install, maintain, update, upgrade and operate such an infrastructure might be haunting.

What does it look like to build a Kubernetes cluster?

As discussed there are two kinds of nodes for our cluster, the master node also known as control nodes, and the worker nodes. Each of those nodes contains different components that will be supporting our cluster, a minimum configuration will have one control node and one worker node, which can be physical machines or virtual machines connected by a network.

The control nodes will run the API server, an etcd database, the scheduler, and some controllers while the worker nodes will run only tree components being them the kubelet, kube-proxy, and the container runtime.

This might sound simple, right? but then you need to configure all those components, you also need to do all the networking needed for those nodes to communicate, and in the networking part there is one additional thing, you will (eventually) need to expose your applications to the internet using an ingress controller which will require to have an external IP exposed to the internet from where all the incoming traffic to your applications will ingress your cluster, this means you will need to administer the creation (and maintenance) of those public IPs.

This becomes a complex task pretty quick but if this is so complicated, why would anyone want to build a cluster for themselves? well, the answer is pretty easy, security when you build your cluster, you keep control of everything, patching, kernel versions, os versions, packages in the nodes, everything is possible to customize to your needs. It might sound like overkill but think in heavily regulated industries like the financial sector, there you would like to have control over every piece of your infrastructure, you don't want to have a bug in a library running in the OS of your worker node that allows malicious actors to exploit it and gain root in one of your pods.

Managed clusters and the cloud providers offerings

So, we are not working in a highly regulated industry but we do need one or more clusters and we don't want to spend months learning how to build a cluster and then need to maintain them manually, what can we do?

Luckily each cloud provider (at least the most popular ones) does offer automated ways to build a cluster for you and give you access to it to simply deploy resources into them. This is very convenient for most of the scenarios where a Kubernetes cluster is needed, even for production workloads, those cloud providers have (normally) multiple levels of SLAs for those clusters meaning that if you are running a development cluster you can opt for a lower SLA but also a cheaper billing while for production workloads you can opt for a higher SLA which includes highly available resources but that will also cost you more, but you can choose what is best for your case and environment.

Those managed clusters have a clear downside, and that is the fact that the cloud providers will retain the control of the control plane and their control nodes meaning you can't decide on how to configure it and even when it is possible to adjust and tune your worker nodes (even apps, libraries and kernel configurations) as soon as you modify them, you loose support from the cloud provider. This might sound like a bad move from the providers but think what would it mean for them to support every possible change made in any cluster in the world? it would be impossible for them to provide proper support so they stick to a proven configuration and they support it, if you move away from that you are on your own.

Patching and upgrading your cluster

We discussed a lot about building and configuring your cluster, but what about version upgrades? what happens when Kubernetes releases a new version? well, here is again a big difference, cloud providers tend to be a bit delayed with the latest version being released because they need to test it first, pass certain internal validations, and then expose them for you to consume in a trustable manner, this takes time meaning if you are wanting to implement a super shiny new feature of the latest release of Kubernetes and you are running a managed cluster you will need to wait a bit before this version is available for you to pick it up, while in a managed cluster you can upgrade whenever you want.

But what happens if something goes wrong with my upgrade? I moved to a newer version and now everything is broken (see the note after this paragraph for a good example). In a managed cluster it is very complicated to revert to an older Kubernetes version I would say it is impossible but I actually know cases where the cloud provider managed to revert the version in VERY specific cases but I would take this as "something you can't do" but in your unmanaged cluster it is as simple as just reinstall the older version or revert to a backup snapshot from before your upgrade (because of course you do backups before upgrading your cluster version) and you are again in a case like nothing happened, very easy.

A note about this, recently with the introduction of version 1.25 there was a change in a configuration called cgroups in the kernel of the OS being used by Kubernetes, this caused some applications using an old (but still functional) version of their frameworks to start consuming a lot of memory and making the kubelet constantly kill those pods, a simple solution is to upgrade your framework in your app, but if that is not possible you can also revert the change in the worker node kernel, but this last option will make you unsupported for your cloud provider.

So, what is the takeaway of this article?

Essentially you now know what a managed and unmanaged cluster, you know what are the advantages and disadvantages of each one, and also you know what it means to have a managed cluster and what can you do with it.

The main point here is to know your situation, know your use case and know what you expect from Kubernetes and your cluster/s. Will you need to meet very strict standards and provide proof that you have certain configurations? then you are tied to an unmanaged cluster, you will need to learn and understand the details of Kubernetes and manage all by yourself, which might sound haunting but with time and training, you will be perfectly fine.
If on the other hand, you will be deploying applications that you know are secure, you don't need to provide configurations to audits and you can rely on a cloud provider having control over your control plane, you are more than fine with a managed cluster and you can skip the deep part of Kubernetes for now (it is always advisable to learn the concepts anyway, but you will not be rushing to learn and understand everything because you need to build you cluster quickly).

So what are your thoughts? would you prefer a managed cluster? an unmanaged cluster? would you feel comfortable knowing your control plane is not under your control? Let me know in the comments box 👍

Thank you for reading 📚 !

The ever-growing kubernetes manual

Javier Marasco — Thu, 22 Jun 2023 17:13:02 +0000

After several months not posting and trying to think a way to articulate this idea I had in mind, I think I got how to present this to the community.

This post will serve as an index for a series of other articles about kubernetes from scratch, I am planing on writing a more elavorated version of it (possibly in a ebook format) but I also wanted to give a free version to the community with the basics of Kubernetes in an easy to follow format clarifying the basic concepts you will need to go from 0 to a degree where you will feel confortable enough to deploy apps to Kubernetes and do some basic troubleshooting by using a easy to follow narrative to not only read technical information but also to nuderstand the reasoning behind it.

I will try to add more articles into the list so if you feel there are items not being presented please let me know and I will add them into the index.

So, let's jump into it:

1 - What is Kubernetes, how does it works and why do we need it
2 - Managed cluster vs unmanaged clusters
3 - Basic elements of an applications running in Kubernetes
4 - Deployments, replicasets and pods
5 - Services, networking and ingress
6 - Storage classes, volumes and how to store your app data
7 - Secrets, config maps and how they are used

I will be adding those articles soon and update this index with the links to the corresponding articles, follow me to get notified every time I upload one of them and don't miss a single post :)

As always, comments and feedback is always welcome and appreciated.

See you later!

What is Kubernetes, how does it works and why do we need it

Javier Marasco — Thu, 22 Jun 2023 17:12:40 +0000

A lot of people are getting into Kubernetes and the first thing they normally do is google "How to deploy an application to Kubernetes" which could make sense but then you will find thousands of articles (including the official documentation) explaining how to deploy an application, but then you see "deployment", "replica set", "pods", "resource quotas", "secrets" and everything starts getting confusing, complex and not making sense and then you get overwhelmed by the amount of information.

I believe the first thing to start with a technology that is new to you is to understand why such technology exists, what problems it solves, and how it works (conceptually), this will let you have a clear picture of the technology and will ultimately let you decide if it is the best fit for your particular case, so let's start with that.

A little of the history and background

Previous to Kubernetes applications used to be deployed in virtual machines (and previous to that into physical machines) but those machines needed to have libraries, dependencies, networking configurations, etc. You can imagine managing that was very complex and changes needed a long time and coordination between multiple teams. Then in 2013, there was a presentation that changed everything, a person called "Solomon Hykes" presented a new project his company (with two other persons) was working on, it was Docker.

The key difference with Docker was nothing new actually, was a set of capabilities that already existed in the Linux kernel for a long time but now it was being exposed to the application level in a more comprehensive way, with this "Docker" tool, it was possible to pack your code with all dependencies in a "container" which you could take to any other system where Docker was running and it will behave the same, that was exactly what was needed!

So now a few months/years had passed and you have containers everywhere, everyone is happy with the approach but the more containers you have it becomes a bit more complicated to manage, so we started using "docker-compose" which was a way to group containers into "logical units" to deploy them together and have some sense of integration between them, quickly it was obvious that there was a need for some way to manage large amounts of containers, then "Docker Swarm" appeared which was a tool to orchestrate the deployment of complex applications with multiple containers. This worked fine as an orchestrator but then in 2014 Google releases Kubernetes as an alternative to Docker Swarm but with more functionalities, more features were added with time and the community adopted Kubernetes as the de facto orchestrator, Kubernetes used Docker under the hood for a quite long time to execute the containers the same as Docker Swarm but later the community of Kubernetes decided to make it possible to use any container engine you want so they adjusted Kubernetes to support virtually any container engine removing the need to have Docker as the only container engine.

But... how does it work?

Kubernetes at the very top level is very simple in concept, you have your image (your application code and dependencies packed into a single file made of multiple "layers") and you will deploy it into a set of machines running "something" that will take your image and make it run this "something" will take care of checking your image is running in a healthy machine, will restart the image when something bad happens to it, will kill it and restart it if it starts consuming too many resources, it will handle communication in/out from the world to your image, etc.

To do this, Kubernetes has its own internal components, being them:

API Server
Scheduler
Kubelet
Kube proxy
etcd

The API Server is the interface between Kubernetes and the rest of the universe, every time you run a command using kubectl (the CLI to manage any Kubernetes cluster) it will make a REST call to the API server of your cluster and provide the parameters and files you pass to kubectl as the payload of the command.

The Scheduler is a process that will take the container (note I am not talking about an image anymore, more on this later) and will validate which node in the cluster meets the needs to have that container running (resources, exclusions, affinity, etc.)

Kubelet is a process running in each "worker" node that will do the actual work of making your container run, it will do all the needed tasks to ensure your code runs in the node, resource allocation, resource monitoring, process handling, etc. this is a key part as it also is responsible to support different container engines.

Kube proxy is a component also running on each worker node that will take care of all the networking work for our containers, one important piece of information about kube proxy that many people get confused about is that it will not route or be in the middle of the traffic at all, kube proxy does the needed networking configuration in the worker node for the traffic to reach the correct container (adding and removing rules from iptables for example) which means that it can crash and the applications running in your node will continue to work.

And of course, we have a lot of components in the control nodes and the worker nodes, we have container running, network routes defined and a lot of information about our infrastructure but how does the API server "remembers" all this? well, there is where etcd enters into the scene, etcd is a highly available and scalable key/value data store (what a lot of words, no worries, is not that complex) which takes the role to store all the cluster state constantly, it contains information about your cluster and the API server is the one updating it constantly.

Now you have a basic understanding of how is it possible for Kubernetes to run a container, by knowing this you can decide if Kubernetes is the right tool for your application.
I often tend to not recommend Kubernetes for small applications that are just in the initial stages of being deployed or if your application is already running in another infrastructure and the migration to Kubernetes will only bring you more work to do without much gain.

Just remember Kubernetes is a tool and like any other tool it serves a purpose and that should drive your intentions to move into it or not, adopting Kubernetes is a task that will require a lot of investigation, learning, and effort (translated into time), be mindful to move to Kubernetes only if it makes sense for your use case.

I hope this first article of the series helped you to understand the basics of Kubernetes and decide if Kubernetes is the right choice for your next steps.

If you enjoyed this article please consider following me to get alerted on every new article and consider leaving a message with your ideas and/or feedback.

Thank you for reading!

HTTPs with Ingress controller, cert-manager and DuckDNS (in AKS/Kubernetes)

Javier Marasco — Sat, 19 Feb 2022 14:33:32 +0000

This guide pretends to explain a quick and easy way to get our applications in Kubernetes/AKS running exposed to internet with HTTPs using DuckDNS as domain name provider.
This guide is also available in my YouTube channel @javi__codes (Spanish only for now, sorry).

Pre-requisites and versions:

AKS cluster version: 1.21.7
Helm 3
Ingress-controller nginx chart version 4.0.16
Ingress-controller nginx app version 1.1.1
cert-manager version 1.2.0
cert-manager DuckDNS webhook version 1.2.2

(1) Add ingress-controller Helm repo

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

(2) Update repository

helm repo update

(3) Install ingress-controller with Helm

With this command we will be installing the latest version from the repository.

helm install nginx-ingress ingress-nginx/ingress-nginx --namespace ingress --create-namespace

(4) Verify the pods are running fine in our cluster

kubectl get pods -n ingress

You should see something like this:

NAME                                                      READY   STATUS    RESTARTS   AGE
nginx-ingress-ingress-nginx-controller-74fb55cbd5-hjvr9   1/1     Running   0          41m

(5) We need to verify our ingress-controller has a public IP assigned

kubectl get svc -n ingress

We should see something similar to this, the key part here is to have an IP assigned in "EXTERNAL-IP", this might take a few seconds to show, it is expected as in the background what is happening is that Azure is spinning a "Public IP" resource for you and assigning it to the AKS cluster.

NAME                                               TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE
nginx-ingress-ingress-nginx-controller             LoadBalancer   10.0.33.214    20.190.211.14   80:32321/TCP,443:30646/TCP   38m

(6) Deploy a test application

Now we will be deploying a testing application that will be running inside a pod with a service that we will use to access the pods. This might feel a bit overkill as we have only a single pod and having a service for a single pod seems a lot but keep in mind that pods can be rescheduled at any given moment and they can even change their IPs while a service doesnt, so reaching our pods using a service is the best (and the desired) option. This also scales better as if we have more pods we will still use the same service to reach them and the service will load balance between them.

This is the yaml file for our test application:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-app
  namespace: default
spec:
  selector:
    matchLabels:
      app: echo-app
  replicas: 2
  template:
    metadata:
      labels:
        app: echo-app
    spec:
      containers:
      - name: echo-app
        image: hashicorp/http-echo
        args:
        - "-text=Test 123!"
        ports:
        - containerPort: 5678

(7) Deploy our service

apiVersion: v1
kind: Service
metadata:
  name: echo-svc
  namespace: default
spec:
  ports:
  - port: 80
    targetPort: 5678
  selector:
    app: echo-app

(8) Let's deploy an ingress resource

Now we need to deploy an ingress resource, this will tell our ingress controller how to manage the traffic that will be arriving to the public IP of the ingress controller (the one from the step 5), basically we are telling it to forward the traffic from the "/" path to the service of our application.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-echo
  namespace: ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: echo-svc
            port:
              number: 80

We are telling the ingress controller to forward all the traffic of the port 80 to the service echo-svc on it's port 80.

(9) Let's test it all together

To test this we will be accessing the ingress using the public IP that we got in step 5:

Using a web browser, go to https://IP

Using the command line run curl https://IP

Adding certificates with cert-manager for duckDNS

So far all is good, the only (small :) ) detail is that our ingress has an IP and not a domain/subdomain which is a bit hard for humans to remember and our traffic is all going unencrypted over http, we don't have any security (yet).
We will be adding cert-manager to generate TLS certificates for us in our DuckDNS subdomain, cert-manager not only allows us to get certificates, it also rotate them when they are about to expire (and we can configure how ofter we want to expire/rotate them).

(10) Let's install cert-manager

helm install cert-manager jetstack/cert-manager --namespace cert-manager --version v1.2.0 --set 'extraArgs={--dns01-recursive-nameservers=8.8.8.8:53\,1.1.1.1:53}' --create-namespace --set installCRDs=true

After a moment it will be done creating the needed resources, we can verify this by checking the status of the pods in the cert-manager namespace:

kubectl get pods -n cert-manager

Something like the following should appear:

NAME                                            READY   STATUS    RESTARTS   AGE
cert-manager-6c9b44dd95-59b6n                   1/1     Running   0          47m
cert-manager-cainjector-74459fcc56-6dfn8        1/1     Running   0          47m
cert-manager-webhook-c45b7ff-hrcnx              1/1     Running   0          47m

(11) Do you need a domain for free? DuckDNS to the rescue!

With all this in place we are ready to request a TLS certificate for our site/application, but first we need to own a domain or a subdomain to point to our public IP (step 5) so we can reach our pods/service using a name instead of an IP.
Another very important point to note is that cert-manager will only provide certificates if we can proove we own the domain/subdomain (this is to avoid the possibility of anyone requesting a certificate for a well known domain like google.com), to do this it has two methods http-01 and dns-01, we will focus this time in dns-01 which basically works like this: cert-manager requests us to provide credentials to access the domain/subdomain (in DuckDNS this is a token), with that cert-manager will generate a random string and make a TXT record in the domain provider with the value of the random string generated, will wait a moment and will use public DNSs to query that TXT record, if cert-manager finds the TXT record with the correct value it means we own that domain/subdomain and then will remove the TXT record and generate a certificate for us for that domain/subdomain. This will end with a secret in our K8s/AKS cluster containing the certificate and the key for that domain/subdomain, that secret is the one we will tell ingress-controller to use to validate the https traffic reaching our ingress.

(11) Configuring our DuckDNS account

We need to go to https://www.duckdns.org/ and log in with our account/credentials (you have multiple alternatives in the upper right part of the page). Once this is done you will see your token in the screen, that's the token we will need in step 12 of this guide.

A bit below that we will see a text field where we need to enter the subdomain name we want (something.duckdns.org) and a place to assign an IP (IPv4), in there we can enter a name for our subdomain and for the IP enter the public IP of our ingess (the one from step 5), then click save/update.

Now we are telling DuckDNS to redirect all the traffic that arrives to that subdomain to the IP we entered, wonderful!

(12) Deploy a DuckDNS cert-manager webhook handler

Now is time to deploy a DuckDNS webhook handler, this is what will add the functionality to cert-manager to manage records in DuckDNS. We can opt to use a helm chart or deploy by cloning the repository where this solution resides, the helm chart didn't work for me so I will be describing the approach using the code in the repo instead.

Let's clone the repository first

git clone https://github.com/ebrianne/cert-manager-webhook-duckdns.git

Now we install it from the cloned repository

cd cert-manager-webhook-duckdns

helm install cert-manager-webhook-duckdns --namespace cert-manager --set duckdns.token='TOKEN_DE_DUCKDNS' --set clusterIssuer.production.create=true --set clusterIssuer.staging.create=true --set clusterIssuer.email='NUESTRO_MAIL' --set logLevel=2 ./deploy/cert-manager-webhook-duckdns

Now we will see we have a new pod in our cert-manager namespace, we can check with the following command

kubectl get pods -n cert-manager

And you will see something like this

NAME                                            READY   STATUS    RESTARTS   AGE
cert-manager-webhook-duckdns-5cdbf66f47-kgt99   1/1     Running   0          56m

(13) ClusterIssuers y detalles de cert-manager

To generate certificates cert-manager has two certificate generators, one is called XXXX-Staging and the other one XXXX-Production. The main difference is that the Production one will provide a certificate that is valid for all web browsers, this is the one we want in our application, but if we are testing and learning we will make mistakes and too many mistakes in the Production one will cause cert-manager to ban us from using the service. To avoid this there is the Staging one which will provide a valid certifcate that our brownsers will take as "valid buuuuuuut" so you will see the padlock and the https but you will see in the certificate description that it is a Staging certificate. With this Staging one we can try and make as many mistakes as we need to fully understand how this works, once done, you simply change the ClusterIssuer to the Production one and you will get a new certifica but for Production and since it was working when you did your tests in Staging this one should not fail.

When we installed the DuckDNS webhook we told the names to use for those ClusterIssuers, here is what I mean:

--set clusterIssuer.production.create=true --set clusterIssuer.staging.create=true

(14) Let's create an ingress resource using the Staging ClusterIssuer

Create a file called staging-ingress.yaml with the following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo-https-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: cert-manager-webhook-duckdns-staging
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  tls:
  - hosts:
    - superprueba.duckdns.org
    secretName: superprueba-tls-secret-staging
  rules:
  - host: superprueba.duckdns.org
    http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: echo
            port:
              number: 80

In this example the subdomain is called superprueba and I am defining to use the clusterissuer cert-manager-webhook-duckdns-staging and to store the certificate in a secret called superprueba-tls-secret, also all https traffic coming from superprueba.duckdns.org needs to be forwarded to service echo on port 80.

The secret name can be anything we want, is not mandatory to contain the name of the subdomain/domain but is a good practice so we can quickly identify what the secret is for.

Another important detail is that the ingress resource has to be defined in the same namespace as the service that will be forwarding traffic to, but the ingress CONTROLLER can be (and is normal to configure in this way) in another different namespace.

Now we apply it with the following command:

kubectil apply -f staging-ingress.yaml

(15) Verify the creation process for our certificate

Now if we run a kubectl get challenge in the same namespace where we deployed the ingress resource we should see something like this:

NAME                                                        STATE     DOMAIN                       AGE
superprueba-tls-secret-staging-6lmxj-668717679-4070204345   pending   superprueba.duckdns.org      4s

This is the process that cert-manager uses to generate the TXT record in DuckDNS and confirm we own the subdomain/domain (basically we provided a valid token), once this process is done and cert-manager confirms we are the owner of the subdomain/domain this challenge is deleted and a certificate and a key are generated and stored in the secret we specified (superprueba-tls-secret-staging) in our case.

If we check the status of our certificate while the challenge is still pending we will see something like this:

NAME                             READY   SECRET                           AGE
superprueba-tls-secret-staging   False    superprueba-tls-secret-staging   7m15s

And once is done and the challenge is deleted we will see something like this:

NAME                             READY   SECRET                           AGE
superprueba-tls-secret-staging   True    superprueba-tls-secret-staging   7m15s

At this point we can verify that we can access our subdomain superprueba-duckdns.org with a browser or using curl.

With curl we would see something like this.

 curl https://superprueba.duckdns.org/
curl: (60) schannel: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted.

This is correct, we have a certificate but is not a production ready one, is just one to test the configuration for cert-manager is correct, now we can go and change the clusterissuer from staging to production to obtain a real and valid certificate.

(16) Adjusting our ingress resource to request a production certificate

Now let's crete a new file called production-ingress.yaml with the following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo-https-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: cert-manager-webhook-duckdns-production
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  tls:
  - hosts:
    - superprueba.duckdns.org
    secretName: superprueba-tls-secret-production
  rules:
  - host: superprueba.duckdns.org
    http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: echo
            port:
              number: 80

And then let's apply it with:

kubectl apply -f production-ingress.yaml

Once this is done, we can do the same verification steps as before to confirm that the production certificate is issued and stored in our secret and confirm by navigating to our site again

curl https://superprueba.duckdns.org/
Test 123!

(17) Troubleshooting

Ok, this was the article on how to configure and make it work this solution without problems, some times we have a typo, or misconfigure an IP or a wrong name somewhere and is a pain in the neck to know what is wrong if you are just following this tutorial as a first approach to kubernetes.
So here are a few things to check in case something is not working as expected.

a) Check the cert-manager webhook logs, here you will find all the actions that the webhook performs to the duckdns service, if there is a problem with the token you are using, a failure to reach duckdns, etc. here is where you will find that.

b) Check the logs of cert-manager (the core element), those are the pods called cert-manager-XXXX and in here you will find information on what is cert-manager doing, if is requesting a certificate, creating a secret, running a challenge, etc.

c) Verify the logs for the ingress-controller pods, here we can see the requests reaching our cluster, if the requests can't reach our ingress they will not be able to be routed to any service, here we should see the request being ingested.

d) Check the configuration in DuckDNS is pointing to the correct IP as we configured it, this can be done with https://digwebinterface.com/ which is a simple page that you input a domain name and it will return you the IP where it is pointing.

About me

If this article was useful to you or you liked it, please consider give it a like, write a comment or subscribe to my space or my other social networks, that helps me understand what is the best content to share and what people like to read or see.

You can subscript, follow, like, message me in:
Twitter -> @javi_codes
Instagram -> javicodes
LinkedInd -> javiermarasco
Youtube -> javi_codes
GitHub -> https://github.com/javiermarasco

All the code for this is in the following repository:

Repo: https://github.com/javiermarasco/https_duckdns

Let's test our configurations with Powershell and Pester

Javier Marasco — Sat, 12 Feb 2022 16:54:32 +0000

I tend to automate everything, it makes sense that if there is something you are requested to do more than once and the time you need to invest to automate it is not huge, you will spend some time automating it. But I often found myself having a configuration file to not need to deal with modifying my scripts, I simply create a script that does the job and provide the script with configuration files as input.
This is a very nice approach when you don't want to have parameters in your scripts as well.

But then, you communicate this in your company and more people start using your automation, you of course know how the config file should look like, but what if others are not aware? what if that automation ends up in a pipeline? you have the usage documented (of course you do!) but probably others are not aware of it.

So.... how do you ensure your script is used in a safe way and your configuration file is honored? well, keep reading and I will show you how I do it :)

General idea

Let's think that we have a terraform file that will build some resources in the cloud, and you are providing this terraform plan a JSON file, inside the terraform plan you decode the JSON and use the contents to provide of values to you resources.

Since this JSON is something you create, the format is what ever it makes sense for your needs, it can have any format and any amount of fields, but you need to be aware that depending on the structure you give to it the tests might change a bit.

Example configuration file

For this example I will go with a general JSON file that I made for this example, if contains arrays, nested objects, booleans and arrays inside nested objects.

[
    {
        "vnet_name": "demo-vnet-name-1",
        "resource_group_name": "resource_groupname",
        "address_space": [
            "10.0.0.0/23"
        ],
        "dns_servers": [
            "10.0.1.0",
            "10.0.0.128"
        ],
        "vnet_location": "eastus2",
        "Subnets": {
            "name": "misubnet",
            "address_prefixes": [
                "10.0.0.0/24",
                "10.0.0.128/24"
            ]
        },
        "Enabled" : true
    },
    {
        "vnet_name": "demo-vneT-name",
        "resource_group_name": "resource_groupname",
        "address_space": [
            "10.0.2.0/24"
        ],
        "dns_servers": [
            "10.0.1.12"
        ],
        "vnet_location": "eastus3",
        "Subnets": {
            "name": "misubnet2",
            "address_prefixes": [
                "10.0.2.0/24",
                "10.0.2.128/24"
            ]
        },
        "Enabled" : false
    }
]

This configuration file is an array that contains 2 definitions to build two hypothetical network resources.

Thinking process

The first thing we want to do is to sit, relax and watch our configuration file, think on what make sense to validate and what doesn't, we don't want to write kilometers of tests when we don't need to validate all, maybe there are fields that are ok to have any value on them (like tags, we probably don't care if the tag is correct), while others are very important to validate, like a naming convention.

Once we saw what we want to write a test for, our next step is to write down a list of test we want to address, let's do that.

I want to test:

1) My vnet_name is not empty
2) I want to check the vnet_name is following my naming convention (very simple needs to have 4 sections)
3) My vnet_name is composed of only lower case letters
4) The location for the resources needs to be one that I "approve" to build on

With this list, we are ready to start writing our tests.

Pester

What it is

Pester is a test framework for powershell, is very easy to use, contains a lot of methods/keywords to run our assertions and it's installation and configuration can't be simpler.
Using Pester we will be writing blocks called "Descriptions" defined by the word "Describe" which are logical ways to split our assertions, inside each "Describe" block we will be creating one or more "It" asserts, each one of those will run a validation and will be reported as a "Passed" or "Failed" assertion.

In the output, when you run the "Invoke-Pester" command with the "-Output Detailed" parameter, the "Describe" block will be grouping the "It" asserts, so it will be easier to read as an output.

How to install (Windows)

To install Pester is as simple as install it from the PSGallery following this guide.

The key steps are:

1) Open a powershell terminal as administrator
2) Run Install-Module -Name Pester -Force -SkipPublisherCheck

No big mystery here, it will install pester as a module in your host and let it ready to use.

How it works

As mentioned before, writing a test is simply create Describe blocks to group similar assertions and inside those blocks, write one It block for each assertion.

For the Describe blocks, there is not much to say, you place a name on them and nothing more.

For the It asserts is different, in them you can pass a TestCase which is an array of elements that the assert will evaluate one by one or you can simply skip that and inside the It block write the code to make the validation.

When you write an assertion you want to do an operation and then pipe it to the should operator (which is part of what you install with Pester). This should operator has some parameters that you can use to describe what are you expecting the evaluation will return.

Some parameters for should are:

Be: Compares the evaluation result to a desired value
Not: Inverts the boolean of the evaluation
BeNullOrEmpty: Checks if the evaluation is an empty string or not defined at all
BeGreaterThan: Checks if the evaluation is greater than a defined value
BeLessThan: Checks if the evaluation is less than a defined value

Here you can find the complete list

Once you have your test written, you can call it by running Invoke-Pester -Path <file.ps1> and if you like the verbose output (like me) add -Output Detailed

Example Pester test

We are going to be using Pester 5.3.1 in this guide, next I will split my test in sections to explain them next:

Pre tests data

We will need some elements before we can start testing stuff, this is to define certain values for the tests.

# We retrieve the configuration to test
$configfile = Get-Content -path "./configuration.json" | ConvertFrom-Json -Depth 4

# Define the list of approved regions to deploy resources
$Regions = @('eastus2','eastus')

# We create an empty array to pass to our tests
$TestCases = @()

# We populate our test cases creating elements named "Instance" for each entry in our config file
foreach ($item in $configfile) {
    $TestCases += @{Instance = $item }
}

"My vnet_name is not empty"

Let's define the test for this

# Example to verify if the value was defined, this prevents missing important fields.
Describe "Check vnet_name is defined." -Verbose {
    It "Verify the name is set in <Instance.vnet_name>." -TestCases $TestCases -Verbose {
        Param($Instance)
        $Instance.vnet_name | should -not -Benullorempty
    }
}

In here we are using the "should", "not" and "benullorempty" to compare with the value we got from the configuration, Pester already have all this functions ready for use to use.

Check naming convention

In this one we are going to assume our naming convention is something that needs to have 4 segments separated by a "-". This is a very simple check, think that you can even validate each of those sections and see if their values are correct.
Another useful check is to validate if there are no other resources already created with this name.
Keep in mind you can have multiple "It" commands inside a "Describe" section.

# Example to verify namingconvention, this helps to enforce we don't create resources wrongly named.
Describe "Check naming convention for vnet_name." -Verbose {
    It "Verify the vnet_name for <Instance.vnet_name> matches naming convention length." -TestCases $TestCases -Verbose {
        Param($Instance)
        $Instance.vnet_name.split("-").count | should -be 4
    }
}

Only lower letters are allowed on the vnet_name

This one is very interesting, we are going to use regular expressions to check if the name we are providing is composed of lower case letters, regular expressions are very powerfull and we can write really good tests by using them to define what we are expecting.

In this example cmath is for case sensitive matches and imatch is used fo case insensitive matches.

# Example to validate our names are all lowercase, useful for resources that doesn't support uppercase
# here the cmatch uses a regular expression, this can be adjusted to match any patter we need.
Describe "Check name for vnet_name should be all lowercase." -Verbose {
    It "Verify <Instance.vnet_name> is all lowercase." -TestCases $TestCases -Verbose {
        Param($Instance)
        $Instance.vnet_name -cmatch "^[^A-Z]*$" | should -be $true
    }
}

Validate we are only deploying to approved locations

At the beginning of this section we defined a list of approved locations, we will use that to validate the location in our configuration is in that list.

# Example on how to validate a value in an array of values, like in this case where an 
# approved list of regions is given to the test to validate we build in the approved locations/regions.
Describe "Check location/region to deploy." -Verbose {
    It "Verify if region for <Instance.vnet_name> is approved." -TestCases $TestCases -Verbose {
        Param($Instance)
        $Regions -contains $Instance.vnet_location | should -be $true
    }
}

Final notes

As usual, you can find my other networks in here

If you find this useful or have any recommendation, please let me know in the comments, follow me for future posts so I know which content is more desired by the community and I can focus on produce more of it and I hope you enjoyed it.

Thanks for reading!!

(Spanish) Ingress-controller, cert-manager y DuckDNS en AKS

Javier Marasco — Sun, 06 Feb 2022 12:24:55 +0000

Ingress controller con NGINX y cert-manager usando DuckDNS

Esta guía es una forma de configurar ingress controller y cert-manager (usando DuckDNS) para tener rápidamente (y gratis) una URL con HTTPS apuntando a nuestro cluster AKS donde podemos tener nuestras aplicaciónes expuestas a internet.
Esta guía la van a poder encontrar en forma de video en mi canal de YouTube (javi__codes) también. Mas links a mis otras redes sociales al final de la guía.

Ahora, empecemos:

Pre requisitos y versiones:

AKS cluster en version: 1.21.7
Helm 3
Ingress-controller nginx chart version 4.0.16
Ingress-controller nginx app version 1.1.1
cert-manager version 1.2.0
cert-manager DuckDNS webhook version 1.2.2

(1) Agregar helm repo de ingress nginx

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

(2) Update de repos

helm repo update

(3) Instalar nginx ingress-controller con helm

Esto va a instalar la ultima version disponible del chart en el repositorio.

helm install nginx-ingress ingress-nginx/ingress-nginx --namespace ingress --create-namespace

(4) Verificamos que los pods estén corriendo correctamente

kubectl get pods -n ingress

Deberían ver algo asi:

NAME                                                      READY   STATUS    RESTARTS   AGE
nginx-ingress-ingress-nginx-controller-74fb55cbd5-hjvr9   1/1     Running   0          41m

(5) Verificamos que nuestro ingress tiene una IP publica asignada

kubectl get svc -n ingress

Deberíamos ver algo asi, lo importante es que tengamos una IP asignada en "EXTERNAL-IP", a veces demora unos momentos en asignarnos una IP, lo que pasa por detrás es que Azure tiene que generar un recurso de tipo "Public IP" y asignarlo al cluster de AKS. Si no aparece una IP cuando ejecuten el comando, esperen un momento y vuelvan a intentarlo.

NAME                                               TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE
nginx-ingress-ingress-nginx-controller             LoadBalancer   10.0.33.214    20.190.211.14   80:32321/TCP,443:30646/TCP   38m

(6) Desplegamos una aplicación de prueba

Ahora vamos a desplegar una aplicación corriendo en un pod y un servicio que vamos a usar para acceder a los pods de esta aplicación. Si bien vamos a desplegar un único pod y tener un servicio para un único pod puede parecer algo sin sentido, piensen que ese pod puede ser eliminado y otro tomar su lugar, esto no siempre mantiene la IP interna del pod por lo que para acceder a el de forma directa tendríamos que estar constantemente actualizando la IP que usamos para acceder al pod en caso que este sea rescheduleado (borrado y otro creado en su lugar), un service evita justamente esto, nosotros siempre usamos el service para acceder al pod y no importa si tenemos 1, 10 o 70 pods, siempre vamos a usar el mismo service.

yaml de la aplicación de prueba:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-app
  namespace: default
spec:
  selector:
    matchLabels:
      app: echo-app
  replicas: 2
  template:
    metadata:
      labels:
        app: echo-app
    spec:
      containers:
      - name: echo-app
        image: hashicorp/http-echo
        args:
        - "-text=aplicación de prueba"
        ports:
        - containerPort: 5678

(7) Desplegamos un service para nuestra aplicación

apiVersion: v1
kind: Service
metadata:
  name: echo-svc
  namespace: default
spec:
  ports:
  - port: 80
    targetPort: 5678
  selector:
    app: echo-app

(8) Ahora desplegamos un ingress resource

En este paso vamos a desplegar un "ingress resource" esta es una forma de generar un ingress que va a decirle a nuestro ingress controller que el trafico que ingrese a nuestro cluster por la IP publica del ingress controller (la del paso 5 ) sea redireccionado a un servicio interno de nuestro cluster. Recuerden que el servicio de nuestra aplicación no tiene una IP publica asignada.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-echo
  namespace: ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: echo-svc
            port:
              number: 80

Aca le estamos diciendo a Kubernetes que cree un ingress resource para enviar todo el trafico ingresante por el ingress controller en el puerto 80 al servicio echo-svc en su puerto 80.

(9) Probamos que todo funcione

Ahora podemos probar acceder a nuestro sitio usando la IP publica del ingress controller del punto 5.

Usando un web browser https://IP

Usando linea de comando con curl https://IP

Agregando certificados con cert-manager y duckDNS

Bueno, hasta aca todo muy bien, pero nuestro ingress tiene una IP (no es lo mejor para exponer un servicio, es difícil de recordar) y ademas es todo http, no tenemos cifrado.

Veamos de agregar cert-manager, una solución que nos permite obtener certificados TLS para nuestros dominios web y rotarlos automáticamente.

(10) Instalemos cert-manager

helm install cert-manager jetstack/cert-manager --namespace cert-manager --version v1.2.0 --set 'extraArgs={--dns01-recursive-nameservers=8.8.8.8:53\,1.1.1.1:53}' --create-namespace --set installCRDs=true

Luego de que termine de crear sus recursos, podemos verificar que los pods de cert-manager estén corriendo correctamente

kubectl get pods -n cert-manager

Deberíamos ver algo as

NAME                                            READY   STATUS    RESTARTS   AGE
cert-manager-6c9b44dd95-59b6n                   1/1     Running   0          47m
cert-manager-cainjector-74459fcc56-6dfn8        1/1     Running   0          47m
cert-manager-webhook-c45b7ff-hrcnx              1/1     Running   0          47m

(11) Necesitas un dominio temporal y gratuito? DuckDNS al rescate

Hasta este punto, estamos listos para solicitar un certificado TLS para nuestro sitio, PERO tenemos que tener un dominio propio en internet para apuntarlo a la IP publica de nuestro ingress controller (Punto 5) y asi acceder a nuestros servicios usando un nombre de dominio (o subdominio).
Otro punto importante es que cert-manager solo provee certificados TLS si puede comprobar que nosotros somos los propietarios del dominio que queremos usar (para evitar darnos un certificado TLS de google.com por ejemplo), para hacer esto tiene dos formas, pero hoy vamos a hablar de una llamada DNS-01, en este modelo cert-manager va a generar un registro TXT en nuestro DNS con un valor aleatorio, luego de intentar crear este registro TXT, va a intentar leerlo, si lo puede leer significa que nosotros tenemos acceso a ese dominio (porque cert-manager pudo crear el record TXT), en ese momento cert-manager genera un certificado, elimina el registro TXT y almacena el contenido del certificado (y su key) en un secret en nuestro cluster de kubernetes.

Una vez terminado este proceso, podemos crear un ingress resource diciéndole que queremos que el trafico que ingrese desde ese dominio (y un path especifico) use un certificado (que tenemos en un secret) y reenvíe el trafico a un servicio (el de nuestro pod).

(11) Configuremos nuestra cuenta de DuckDNS

Vamos a tener que ir a https://www.duckdns.org/ y loguearnos con algunos de los métodos que están en la parte superior de la pagina.
Una vez hecho esto, vamos a ver que tenemos un TOKEN, ese es el que vamos a usar en el paso 12 de esta guía.

Tambien mas abajo en la pagina vamos a ver un lugar donde podemos generar un subdominio de duckdns.org y asignarle una IP. Esa IP (IPv4) tiene que ser la IP de nuestro ingress controller (la del paso 5 de esta guía). Una vez hecho eso guardamos los cambios clickeando en el botón de al lado de donde pusimos la IP (IPv4).

Con esto ya le dijimos a DuckDNS que todos los requests que vayan a ese subdominio que configuramos sean redireccionados a la IP que pusimos.

(12) Desplegar el webhook handler de DuckDNS

Vamos a desplegar el webhook de DuckDNS, esta pieza es la que nos permite interactuar con DuckDNS. Este webhook tiene un helm chart que podemos usar, pero en mi caso no funciono, la forma que si me funciono fue clonando el repositorio y usando los archivos de ese repositorio.

Clonamos el repositorio

git clone https://github.com/ebrianne/cert-manager-webhook-duckdns.git

Instalamos desde el repositorio que clonamos

cd cert-manager-webhook-duckdns

helm install cert-manager-webhook-duckdns --namespace cert-manager --set duckdns.token='TOKEN_DE_DUCKDNS' --set clusterIssuer.production.create=true --set clusterIssuer.staging.create=true --set clusterIssuer.email='NUESTRO_MAIL' --set logLevel=2 ./deploy/cert-manager-webhook-duckdns

En este punto vamos a tener un nuevo pod en nuestro namespace cert-manager, lo podemos ver con el siguiente comando

kubectl get pods -n cert-manager

y se vería algo asi

NAME                                            READY   STATUS    RESTARTS   AGE
cert-manager-webhook-duckdns-5cdbf66f47-kgt99   1/1     Running   0          56m

(12) ClusterIssuers y detalles de cert-manager

Cert manager maneja conceptualmente dos tipos de formas de generar certificados, tienen un generador de certificados llamado XXXX-Staging y otro llamado XXXX-Production. La principal diferencia es que el de Production nos va a dar un certificado valido que podamos usar en el mundo real, uno que nuestros navegadores van a reconocer mientras que el de staging va a generar un certificado que nuestros navegadores van a reconocer como https, pero van a marcarlo como "de prueba".

La idea es que hagamos todas las pruebas con el de Staging dado que si nos equivocamos y pedimos muchos certificados erróneos, nos vamos a tener problemas, mientras que si hacemos lo mismo en el de production, es muy probable que nos baneen por un tiempo determinado.

Cuando instalamos el webhook de DuckDNS especificamos que queríamos crear los clusterissues de production y staging.

--set clusterIssuer.production.create=true --set clusterIssuer.staging.create=true

(13) Creamos un ingress resource usando el cluster issuer de staging

Vamos a crear un archivo llamado staging-ingress.yaml con este contenido

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo-https-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: cert-manager-webhook-duckdns-staging
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  tls:
  - hosts:
    - superprueba.duckdns.org
    secretName: superprueba-tls-secret-staging
  rules:
  - host: superprueba.duckdns.org
    http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: echo
            port:
              number: 80

En este ejemplo mi sub dominio en DuckDNS es superprueba y lo que estoy definiendo es que usemos el clusterissuer cert-manager-webhook-duckdns-staging y que guardemos el certificado en un secret llamado superprueba-tls-secret y que todo el trafico que ingrese por superprueba.duckdns.org https, lo envíe al servicio echo en su puerto 80.

El nombre del secret puede ser cualquier cosa, no tiene que contener el nombre del subdominio, pero es una buena practica que sea descriptivo y que sea claro para que se usa.

Un detalle importante es que este ingress resource tiene que estar en el mismo namespace que el servicio al cual estamos redireccionando el trafico. El ingress CONTROLLER puede (y recomiendo) estar en un namespace diferente al igual que cert-manager.

Ahora aplicamos con:

kubectil apply -f staging-ingress.yaml

(14) Verificando el proceso de creación del certificado

Ahora si en nuestro namespace donde desplegamos el ingress resource y los pods/servicios hacemos un kubectl get challenge vamos a ver algo asi

NAME                                                        STATE     DOMAIN                       AGE
superprueba-tls-secret-staging-6lmxj-668717679-4070204345   pending   superprueba.duckdns.org      4s

Este es el proceso que realiza cert-manager para generar el record TXT en DuckDNS y comprobar que nosotros poseemos ese dominio/subdominio (que el token que indicamos es correcto), una vez que este proceso termina y se verifica que somos los propietarios de ese dominio/subdominio, este challenge desaparece y se genera un certificado que es almacenado en un secret con el nombre que nosotros indicamos en el ingress resource (superprueba-tls-secret-staging en nuestro caso).

Si vemos el estado de nuestro certificado mientras el challenge existe, vamos a ver algo asi:

NAME                             READY   SECRET                           AGE
superprueba-tls-secret-staging   False    superprueba-tls-secret-staging   7m15s

Y una vez terminado el proceso y cuando challenge desaparece, lo vemos asi:

NAME                             READY   SECRET                           AGE
superprueba-tls-secret-staging   True    superprueba-tls-secret-staging   7m15s

En este punto podemos verificar acceder a nuestro dominio superprueba.duckdns.org usando un navegador o con curl.

Con curl veriamos algo asi:

 curl https://superprueba.duckdns.org/
curl: (60) schannel: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted.

Esto es correcto, tenemos un certificado, pero no es un certificado "productivo", sino uno para comprobar que todo esta configurado correctamente en cert-manager y estamos listos para pedir un certificado definitivo.

(15) Ajustando nuestro ingress resource para solicitar un certificado productivo

Ahora vamos a crear un archivo nuevo llamado production-ingress.yaml con el siguiente contenido:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo-https-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: cert-manager-webhook-duckdns-production
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  tls:
  - hosts:
    - superprueba.duckdns.org
    secretName: superprueba-tls-secret-production
  rules:
  - host: superprueba.duckdns.org
    http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: echo
            port:
              number: 80

Y por supuesto lo aplicamos con:

kubectl apply -f production-ingress.yaml

Una vez hecho esto, podemos aplicar los mismos pasos de antes para revisar que todo esta bien y por ultimo comprobamos con nuestro navegador o con curl si nuestro sitio https ya esta funcionando, si todo salio bien, tendríamos que ver algo asi:

curl https://superprueba.duckdns.org/
aplicación de prueba

(16) Troubleshooting

Hasta aca todo el camino feliz, donde todo funciono perfectamente y no tuvimos ningún problema, no tuvimos errores de tipeo, pusimos las IPs correctas en todos lados y no confundimos los nombres de los secrets (no es que me haya pasado alguna vez, absolutamente no ...). Pero que pasa cuando algo asi nos sucede? como hacemos para poder identificar donde esta el problema? Bueno, aca mis recomendaciones (las que me ayudaron en todas las veces que NO me paso de tener algo mal configurado)

a) Revisar los logs de los pods de cert-manager webhook. En el del webhook de DuckDNS vamos a ver todos los pasos que esta haciendo cert-manager contra duckDNS y podemos ver si nuestro TOKEN por ejemplo esta mal, o si duckDNS no esta contestando correctamente los requests.

b) Revisar los logs del pod de cert-manager, el que se llama simplemente cert-manager-XXXX, este nos va a mostrar información sobre lo que esta haciendo cert-manager, este pod nos va a indicar de la creación o modificación del secret donde va a estar el certificado, el webhook solo se encarga de la comunicación y verificación con DuckDNS, pero este pod se encarga del trabajo dentro de nuestro cluster de kubernetes.

c) Verificar el log de ingress-controller: Aca vamos a ver cuando un request llegue a nuestro ingress controller, de que IP viene, si hay algún error en el request, básicamente si algo esta mal configurado en el ingress, aca es donde vamos a ver que esta pasando.

d) usar una herramienta para verificar que la IP que pusimos en DuckDNS esta apuntando a la IP publica de nuestro ingess-controller, una que uso yo es https://digwebinterface.com/ en esta herramienta podemos comprobar si realmente el cambio que hicimos en DuckDNS esta correctamente configurado.

Notas finales

Como siempre, si esta guía les sirvió o la ven interesante les agradecería mucho que la compartan con todas las personas que puedan, cuanta mas gente la vea mas posible es que me hagan llegar recomendaciones de como mejorar mis próximos artículos o si hay algún error en este y me alientan a seguir escribiendo y compartiendo con el resto de la comunidad.
Esta guía la voy a convertir en un video y subirlo a mi canal de YouTube, los veo por ahi (de paso... subscríbanse, denle like y dejen un comentario si les gusta el video, todo eso me ayuda)

Aprovecho a dejarles links a mis redes sociales y formas de contacto:

Repo: https://github.com/javiermarasco/https_duckdns

Twitter -> @javi_codes
Instagram -> javicodes
LinkedInd -> javiermarasco
Youtube -> javi_codes

(Spanish) PowerGrafana, que es y como se usa

Javier Marasco — Tue, 07 Dec 2021 17:27:39 +0000

PowerGrafana, ¿qué es y para qué se utiliza?

Comencemos con un poco de contexto y cómo comenzó todo

A veces sucede que tenemos que monitorear alguna aplicación, servicio o componente (entre otras cosas) y la herramienta que usamos o usa nuestra empresa es Grafana, mientras que tenemos pocas cosas que monitorear no es un gran problema pero a medida que agregamos más elementos al Estos paneles se vuelven más complejos de mantener y / o configurar.

Imagine que tiene que implementar 20 aplicaciones, cada una ejecutándose en un app service, por lo que seguramente querríamos monitorear el uso de CPU y memoria (para empezar) y seguramente algunas cosas más.
En este escenario simplista tenemos que configurar algunos paneles en un tablero y en cada panel poner las métricas que queremos mostrar (CPU y Memoria) en nuestro caso.

Esto suena simple pero en poco tiempo seguramente necesitaremos agregar otra métrica o desplegar una nueva versión de nuestra aplicación, peor aún, podríamos desplegar una nueva versión de algunos componentes y no de otros mientras tenemos que mantener todas las versiones (el el inicial más el nuevo) de cada componente, como puedes ver esto se vuelve cada vez más complejo de mostrar en Grafana, es mucho tiempo pinchando en la interfaz o (la alternativa) editando archivos json para pegarlos en el Grafana interfaz web y generar los cuadros de mando o paneles (créanme que es muy fácil cometer errores al editar esos archivos).

La alternativa

PowerGrafana fue creado para resolver este problema (o intentar que sea más fácil de manejar) extrayendo toda la complejidad de lidiar con la interfaz web, los archivos json o incluso ingresando los nombres de los recursos a monitorear a mano.
Usando un módulo simple de PowerShell podemos iterar rápidamente a través de nuestros recursos y para cada uno de ellos crear un panel que muestre el uso de CPU y memoria.

Cada comando posee su ayuda, la cual pueden consultar ejecutando:

PS> Get-Help New-GrafanaDashboard 

NAME
    New-GrafanaDashboard

SYNOPSIS
    Creates a dashboard in Grafana


SYNTAX
    New-GrafanaDashboard [-DashboardName] <Object> [[-Tags] <String[]>] [<CommonParameters>]


DESCRIPTION
    This cmdlet will create an empty dashboard in Grafana that can be used as starting point to create your grafana monitoring.

EXAMPLE
    New-GrafanaDashboard -DashboardName "My new dashboard" -Tags @('Web','Azure','Production')

RELATED LINKS

REMARKS
    To see the examples, type: "Get-Help New-GrafanaDashboard -Examples"
    For more information, type: "Get-Help New-GrafanaDashboard -Detailed"
    For technical information, type: "Get-Help New-GrafanaDashboard -Full"
    For online help, type: "Get-Help New-GrafanaDashboard -Online"

Referencias

(Spanish) Prometheus en AKS

Javier Marasco — Tue, 07 Dec 2021 17:24:04 +0000

Introducción

La intención principal de esta publicación (así como de las demás) no es tener un tutorial muy profundo en todos los detalles de cada tecnología/aplicación que se describe, sino una guía breve y concisa de algo en particular que puede ayudar a las personas que están comenzando a conocer el tema descrito en este artículo, por supuesto, si tenés alguna recomendación para mejorar esto, no dudes en enviarme un mensaje/comentario.

¿Qué es Prometheus?

Arranquemos, vayamos directamente al tema que estaremos hablando hoy acá, Prometheus es una herramienta de monitoreo que es muy popular en el mundo de Kubernetes al igual que una de los proyectos de la CNCF (Cloud Native Computing Foundation) y esto significa que es un producto muy maduro con un gran apoyo de la comunidad.

Hay varias características que hacen de Prometheus una de las herramientas favoritas para monitorear entornos, algunas se enumeran aquí:

Metodología "pull" (significa que el servidor de Prometheus extrae métricas en lugar de esperar a que la aplicación envíe las métricas al servidor de Prometheus)
Muy rápido al recopilar métricas y realizar agregaciones
Una arquitectura que permite monitorear no solo los entornos de Kubernetes sino otras aplicaciones como bases de datos o servidores web (usando "exporters")
Soporta cientos de aplicaciones, hardware, plataformas, etc. para monitorear aquí está la lista

Con esto en mente, continuemos instalándolo y configurándolo para un primer intento.

Cómo instalarlo

Para nuestro ejemplo, instalaremos Prometheus en un clúster de AKS (clúster de Kubernetes que se ejecuta como PaaS en Azure).

Hay varias formas de implementar Prometheus en un clúster de Kubernetes, pero la más simple y la que tiene más sentido es utilizar algo llamado Helm charts, imagina a un Helm chart como un conjunto de archivos YAML vinculados como un único recurso para implementar, esto significa que lo instala como un elemento "único" (un chart) pero, en su lugar, implementará los recursos necesarios en tu clúster para que la solución funcione correctamente (como pods, replica sets, deployments, services, secrets, etc.)

Qué es Helm

Helm es una tecnología que nos permite empaquetar un montón de archivos YAML que se usarán como un todo para hacer que una solución funcione (en este caso, la solución es Prometheus), al usar Helm estás eliminando la complejidad de tener que administrar de forma independiente todos los recursos para que la solución funcione, en su lugar, proporcionas un archivo de configuración al chart y lo implementás, que creará todos los recursos y los configurará correctamente.

Una de las ventajas de usar Helm es que se puede confiar en los repositorios donde se mantienen esos charts y usarlos, pero también si querés podés mantener tu propia versión del chart localmente o en un repositorio privado, para que puedas ajustarlo a tus necesidades (como usar una imagen de container personalizada para los deployments del chart en lugar de usar la predeterminada, esto podría ser un requerimiento de seguridad, por ejemplo)

Otra característica es que se puede almacenar esos charts en los mismos repositorios donde se almacenan sus imágenes de contenedor y versionarlos de la misma manera que lo haces con una imagen de contenedor.

Cada chart tiene su propio conjunto de archivos porque representan un grupo de recursos que funcionan para hacer que una aplicación funcione, por lo que los recursos necesarios para un chart para Prometheus no son los mismos recursos necesarios para cert-manager, por ejemplo, pero la idea es la misma, un conjunto de archivos YAML que una vez implementados trabajarán juntos para hacer que la aplicación se ejecute.

Para utilizar los charts de helm, tenés que tener helm instalado en tu sistema y agregar los repositorios que utilizará, cada chart de helm vive en un repositorio que se debe agregar para descargar el chart y sus archivos.

agregando un repositorio *

Para este artículo usaremos los charts de Prometheus estándar.

Requisitos previos

Bueno, para este artículo necesitaremos:

Un clúster de AKS en funcionamiento, nada especial, solo la implementación base está bien, podés seguir este tutorial (Haré un tutorial en el futuro)
Una terminal con kubectl y helm instalados
Instala el repositorio de helm de prometheus-community

Instalación

Lo primero que haremos es agregar el repositorio de helm de prometheus-community y actualizar nuestra lista de repositorios locales ejecutando el siguiente comando:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

Ahora podemos revisar todos los gráficos que podemos instalar ahora que agregamos el repositorio, veamos:

helm search repo prometheus-community

¡Perfecto! Vemos muchos charts allí, esos son elementos diferentes que podemos instalar, pero hoy nos centraremos en el llamado "prometheus".

Hay un nombre de columna ** CHART VERSION **, esta es la versión del gráfico en sí y no de Prometheus, esto se debe a que se pueden realizar modificaciones en la forma en que está compuesto el gráfico y lo que hay dentro, pero aun así usar la misma versión de Prometheus que la versión de gráfico anterior. Se pueden ver todas las versiones de charts ejecutando:

helm search repo prometheus-community/prometheus -l | grep -v "community-prometheus/prometheus-"

El comando grep es para eliminar todos los demás charts de la lista *

Si instalas Prometheus sin decir la versión del gráfico que deseas, se instalará la última (14.11.1 en el momento en que escribo este artículo).

Vamos a instalarlo ahora:

helm install prometheus prometheus-community/prometheus

el "prometheus" antes del nombre del repositorio/gráfico es el nombre que queremos darle a esta implementación, podés elegir otro nombre.

Ahora que tenemos nuestro servidor Prometheus instalado y listo, verifiquemos en Kubernetes lo que implementamos (tenía el espacio de nombres "default" seleccionado al instalar el gráfico, por lo que mi gráfico se implementó en el espacio de nombres "default")

** Una cosa importante es que cuando instala un gráfico de helm, se instala en un namespace en Kubernetes, si cambia a otro namespace e intentas ver los gráficos instalados, no verás el que instalaste en el otro namespace. Dejame poner esto en una imagen para explicarlo mejor: **

Esto es un helm list en mi namespace" default "
Esto es un helm list en otro namespace

En Kubernetes podemos ver todos los recursos creados automáticamente:

kubectl get all

Siguiendo los pasos descritos después de la instalación del gráfico de helm, debemos reenviar un puerto desde nuestra máquina al pod donde se ejecuta el servidor Prometheus con:

export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus, component=server" -o jsonpath="{. items [0].metadata.name}")

kubectl --namespace default port-forward $POD_NAME 9090

** Si estás utilizando WSL para ejecutar tus comandos de Kubernetes, debes realizar algunos pasos adicionales para que esto funcione **

Primero verifica cuál es la IP de tu WSL ejecutando wsl hostname -I, ya que no se puede acceder a los puertos en tu máquina host (Windows) ejecutando localhost: port si está exponiendo los puertos dentro de WSL.
En segundo lugar, el comando port-forward debe incluir --address 0.0.0.0 como kubectl --namespace default port-forward --address 0.0.0.0 $POD_NAME 9090
En tercer lugar, debe usar la IP de WSL (la del paso uno) en lugar de localhost para acceder a Prometheus

Con esto ahora podemos ir a nuestro navegador y acceder a localhost: 9090 para ver este dashboard:

Prometheus tiene muchas métricas predeterminadas que se recopilan de forma predeterminada, para verlas, podes comenzar a escribir algo en el cuadro de búsqueda y se completará automáticamente con las métricas disponibles.

Como ejemplo:

Y ahora, lo único que queda es profundizar en las métricas que Prometheus está recopilando, tal vez agregando algunos exportadores, ¿configurar el administrador de alertas tal vez? (este es un tema para otra publicación), ¿consumir esas métricas desde Grafana? (El artículo sobre esto ya está en camino : guiño :)

Últimas palabras

Espero que esto te ayude a comenzar con Prometheus, ya que es muy simple de implementar y al mismo tiempo muy poderoso, si tenés algún problema siguiendo esta guía o alguna recomendación, hacémelo saber en la sección de comentarios.

(English) PowerGrafana, what is it and how to use it

Javier Marasco — Tue, 07 Dec 2021 17:24:03 +0000

PowerGrafana, what is it and what is it used for?

Lets beggin with a bit of context and how it all started

Sometimes it happens that we have to monitor some application, service or component (among other things) and the tool that we use or our company uses is Grafana, while we have few things to monitor it is not much of a problem but as we add more elements to the panels it becomes more complex to maintain and / or configure.

Imagine that you have to deploy 20 applications, each one running in an app service so we would surely want to monitor your CPU and Memory usage (to begin with) and surely a few more things.
In this simplistic scenario we have to configure some panels in a dashboard and in each panel put the metrics that we want to show (CPU and Memory) in our case.

This sounds simple but in a short time we will surely need to add another metric or deploy a new version of our application, even worse, we could deploy a new version of some components and not others while we have to keep all the versions (the initial one plus the new one) of each component, as you can see this becomes more and more complex to show in Grafana, it is a lot of time clicking on the interface or (the alternative) editing json files to paste them in the Grafana web interface and generate the dashboards or panels ( believe me that it is very easy to make mistakes when editing those files).

The alternative

PowerGrafana was created to solve this problem (or try to make it easier to handle) by extracting all the complexity of dealing with the web interface, json files, or even entering the names of the resources to monitor by hand.
Using a simple PowerShell module we can quickly iterate through our resources and for each of them create a panel that shows CPU and Memory usage.

Each command has it's own help, which can be accessed by executing:

PS> Get-Help New-GrafanaDashboard 

NAME
    New-GrafanaDashboard

SYNOPSIS
    Creates a dashboard in Grafana


SYNTAX
    New-GrafanaDashboard [-DashboardName] <Object> [[-Tags] <String[]>] [<CommonParameters>]


DESCRIPTION
    This cmdlet will create an empty dashboard in Grafana that can be used as starting point to create your grafana monitoring.

EXAMPLE
    New-GrafanaDashboard -DashboardName "My new dashboard" -Tags @('Web','Azure','Production')

RELATED LINKS

REMARKS
    To see the examples, type: "Get-Help New-GrafanaDashboard -Examples"
    For more information, type: "Get-Help New-GrafanaDashboard -Detailed"
    For technical information, type: "Get-Help New-GrafanaDashboard -Full"
    For online help, type: "Get-Help New-GrafanaDashboard -Online"

(English) Installing Prometheus in AKS

Javier Marasco — Wed, 01 Dec 2021 16:14:24 +0000

Introduction

The main intention of this post (as well as the others) is not to have a very deep tutorial in all the details of each technology/application that is described but a short and concise guide for something particular that can help people that are starting to know the topic described in this article, of course if you have any recommendation to make this better, just feel free to drop me a message/comment.

What is Prometheus

So, let's jump directly to what we will be talking today here, Prometheus is a monitoring tool that is very popular in Kubernetes world as is one of the CNCF (Cloud Native Computing Foundation) projects and this means is a very mature product with a big support in the community.

There are several characteristics that make Prometheus one of the favorites tools to monitor environments, some are listed here:

Pull methodology (Means the Prometheus server pulls for metrics instead of waiting for the application to push metrics to Prometheus server)
Very fast when collecting metrics and doing aggregation
An architecture that allows to monitor not only Kubernetes environments but other applications like databases or web servers (using exporters)
Supports for hundreds of applications, hardware, platforms, etc. to monitor here is the list

With this in mind, let's continue to install it and configure it for a first try.

How to install it

For our example, we will install Prometheus in an AKS cluster (Kubernetes cluster running as a PaaS in Azure).

There are multiple ways to deploy Prometheus in a Kubernetes cluster but the simplest one and the one that makes more sense is to use something called Helm chart, think of Helm Charts as a set of YAML files linked as a single resource to be deployed, this means that you install it as a "single" item (one chart) but it will instead deploy the needed resources in your cluster to make the solution work properly (think of pods, replica sets, deployments, services, secrets, etc.)

What is Helm

Helm is a technology that allows us to package a bunch of YAML files that will be used as a whole to make a solution to work (in this case the solution is Prometheus), by using Helm you are removing the complexity of needing to manage independently all the resources to make the solution work, instead you provide a configuration file to the chart and deploy the chart, that will create all the resources for you and configure them properly.

One of the advantages of using Helm is that you can rely on repositories where those charts are being maintained and use them, but you could also want to keep your own version of the chart locally or in a private repository, so you can tune it to match your needs (like using a custom docker image in the chart instead of using the default one, think of security needs for example)

Another characteristic is that you can store those charts in the same repositories where you store your container images and version them the same way you do with a container image.

Each chart has its own set of files because they represent a group of resources that work to make an application to work, so the resources required for a helm chart for Prometheus are not the same resources required for cert-manager for example, but the idea is the same, a set of YAML files that once deployed will work together to make the application to run.

In order to use helm charts you require to have helm installed in your system and add the repositories that you will be using, each helm chart lives in a repository that you require to add in order to retrieve the chart and it's files.

adding a repository

For this article we will use the default Prometheus helm chart

Prerequisites

Well for this article we will need:

An AKS cluster up and running, nothing special, just the base deployment is fine, you can follow this tutorial (I will make a tutorial in the future)
A terminal with kubectl and helm installed
Install the prometheus-community helm repository

Installation

The first thing we will do is to add the prometheus-community helm repository and update our local repository list by running the following command:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 
helm repo update

Now we can check all the charts that we can install now that we added the repository, let's see:

helm search repo prometheus-community

Perfect! We see a lot of charts there, those are different elements that we can install, but today we will focus on the one called "prometheus"

You see there is a column names CHART VERSION, this is the version of the chart itself and not of Prometheus, this is because you can make modifications in the way the chart is composed and what is inside it but still use the same version of Prometheus as the chart version before. You can see all the chart versions by running:

helm search repo prometheus-community/prometheus -l | grep -v "prometheus-community/prometheus-"

The grep is to remove all the other charts from the list

If you install Prometheus without telling the version of the chart that you want, it will install the latest (14.11.1 at the moment I write this article).

Let's install it now:

helm install prometheus prometheus-community/prometheus

*the "prometheus" before the name of the repository/chart is the name we want to give to this deployment, you can choose another name.

Now we have our Prometheus server installed and ready, let's check in Kubernetes what we got deployed (I had the "default" namespace selected while installing the chart, so my chart was deployed in the "default" namespace)

One important thing is that when you install a helm chart, it gets installed in a namespace in Kubernetes, if you change to another namespace and try to see the installed charts, you will not see the one you installed in the other namespace. Let me put this in an image to explain better:

*This is a helm list in my "default" namespace

*This is a helm list in another namespace

In Kubernetes we can see all the resources created automatically:

kubectl get all

Following the steps described after the installation of the helm chart, we should forward a port from our machine to the pod where Prometheus server is running with:

export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")

kubectl --namespace default port-forward $POD_NAME 9090

If you are using WSL to run your Kubernetes commands, you need to do some extra steps for this to work

First check what's the IP of your WSL by running wsl hostname -I as you can't access the ports in your host machine (windows) by running localhost:port if you are exposing the ports inside WSL.
Second, the port-forward command should include --address 0.0.0.0 like kubectl --namespace default port-forward --address 0.0.0.0 $POD_NAME 9090
Third, you should use the WSL IP (the one from step one) instead of localhost to access Prometheus

With this we can now go to our browser and access localhost:9090 to see this dashboard:

Prometheus has a lot of default metrics getting collected by default, to check them you can start writing something in the search box, and it will autocomplete with available metrics.

As an example:

And now, the only thing that is left is to dig into the metrics Prometheus is collecting, maybe adding some exporters, configure alert manager maybe? (this is a topic for another post), consume those metrics from Grafana? (Article about this is already in the queue 😉 )

Final words

I hope this helps you to get started with Prometheus as it is very simply to implement and at the same time very powerful, if you have any problem following this guide or any recommendation, please let me know in the comments section.

(Spanish) Trabajando con Helm Charts

Javier Marasco — Wed, 01 Dec 2021 15:19:31 +0000

Introducción

En esta ocasión vamos a estar viendo como trabajar con Helm Charts, en mis anteriores artículos sobre Prometheus y Grafana hablamos un poco sobre los Helm Charts, pero no mucho en detalle.

En este articulo voy a descargar el chart de Grafana y vamos a ver como podemos modificarlo para que se ajuste a nuestras necesidades.

Que son los charts?

En el mundo de Kubernetes a menudo nos encontramos con que tenemos aplicaciones corriendo en algunos pods, servicios para esos pods, probablemente un ingress controller para exponer esta aplicación al mundo exterior y podemos tener secrets, configmaps, volúmenes u otros recursos de Kubernetes. A medida que nuestra aplicación va creciendo en componentes y complejidad nos encontramos con que tenemos cada vez mas cantidad de archivos yaml (o yml) pero nuestra aplicación es "una sola" compuesta de multiples archivos, manejar todos estos archivos por separado siendo "una sola entidad" es complicado. Es por esto que aparecen los Helm Charts como una forma de simplificar esto al reunir en una sola entidad (un chart) todos los componentes a desplegar, una segunda ventaja es que podemos hacer "templating" de estos archivos yaml de forma que tomen un valor de un archivo de configuración, haciendo aun mas fácil la configuración del chart. Veamos como es esto.

Viendo los charts disponibles en un repositorio

Como primer paso vamos a agregar el repositorio de Grafana (como un ejemplo, estos pasos aplican para cualquier helm chart).

Y vamos a ver que charts hay disponibles en el repositorio

Vamos a concentrarnos en el chart llamado grafana/grafana que es el que usamos en nuestra instalación de Grafana en otro articulo y que despliega los componentes básicos para una implementación de Grafana minima.

Lo siguiente que tenemos que hacer es ir a un directorio donde queramos descargar nuestro chart y correr el comando helm pull grafana/grafana --untar esto va a descargar el chart a un directorio llamado grafana con todos los archivos necesarios dentro.

Y dentro de este directorio están los archivos que componen el chart

Empecemos a ver que hay dentro de un chart

Lo primero que podemos ver es un archivo llamado Chart.yaml, este archivo contiene la descripcion del chart que acabamos de descargar

Vemos información como la version del Chart, la URL del proyecto al que pertenece el chart y algo mas como quien mantiene el chart, etc.

Luego vemos una estructura de directorios

El único directorio que siempre vamos a encontrar en un chart es templates ya que en este directorio es que vamos a poner todos los archivos yaml que componen nuestra solución. Los demás directorios son necesarios para Grafana, en otros charts podemos encontrar otros directorios, pero siempre tengan en cuenta que los templates que forman los recursos en Kubernetes están en el directorio templates.

Por ultimo en el directorio donde descargamos el chart vamos a encontrar un archivo llamado values.yaml , este archivo es el corazón de la configuración de un chart, en este archivo vamos a encontrar todos los parámetros que son configurables para nuestro chart (y si lo deseamos podemos agregar mas configuraciones en este archivo que luego podemos usar en los yaml del directorio templates).

Como son los templates en un chart

Básicamente un template es un archivo yaml que contiene instrucciones para que al momento de desplegar nuestro chart, los valores finales a implementar sean tomados de un archivo de configuración (values.yaml) y no tengamos que hardcodear un valor en el template. Veamos un ejemplo:

En el archivo deployment.yaml podemos encontrar esto:

Donde en la línea 18 hace referencia a .Values.replicas esta es la forma en la que le decimos a Helm que para el valor del atributo replicas de este yaml, tome el valor que tenemos en el archivo values.yaml con la key replicas. El archivo values.yaml se ve así:

En la línea 24 de values.yaml podemos ver que dice replicas : 1 si desplegamos este chart así, veremos que tenemos sola replica del pod grafana

Pero si modificamos este valor y ponemos 3, guardamos el archivo values.yaml y aplicamos el chart nuevamente, veremos que ahora tenemos 3 replicas de nuestro pod grafana

Esto mismo podemos aplicarlo para cualquier configuración, de esta forma podemos alterar nuestra aplicación simplemente actualizando valores en values.yaml y corriendo un helm upgrade grafana . (si estamos en el directorio donde tenemos nuestro chart descargado)

Algo super interesante es que estos charts también podemos almacenarlos en los repositorios donde almacenamos nuestras imágenes de containers, de esta forma podemos versionarlos juntos con nuestras imágenes de containers y hacer un pull desde donde lo necesitemos.

Mas allá de la configuración básica

En values.yaml van a encontrar muchas líneas comentadas, si las des comentan van a estar habilitando nuevas configuraciones del chart, esto se debe a que helm nos permite poner código condicional en nuestros templates, algo como "si existe el bloque X desplegar Y recurso en Kubernetes". Un ejemplo de esto es el ingress controller de Grafana.
En la línea 181 de values.yaml dice "enabled: false" y en el archivo ingress.yaml en la línea 1 dice {{- if .Values.ingress.enabled -}}, esta es una forma de habilitar o deshabilitar (llamado también feature flag) un componente o configuración en base a lo que definamos en el archivo de valores del chart.

Por ultimo

Espero que este articulo los ayude a comprender un poco mejor que son lo Helm Charts y se puedan animar a crear los suyos propios, realmente no es muy difícil y permite tener un mayor control sobre lo que estamos desplegando en nuestra aplicación y que recursos pertenecen a que aplicación.

Si les fue de ayuda el articulo, agradecería que lo compartan para que mas gente pueda llegar a leerlo y podamos ayudarnos entre todos, de la misma forma si encuentran algo que no es correcto en este articulo o que puede ser explicado mas en profundidad, háganmelo saber en los comentarios.

Muchas gracias por leer!

(Spanish) Instalando y configurando Grafana en un cluster AKS

Javier Marasco — Wed, 01 Dec 2021 14:58:38 +0000

Introducción

En este post vamos a ver como instalar Grafana de forma sencilla usando Helm Charts, también vamos a ver que es y como se usa Grafana

Que es grafana?

Vamos a empezar por explicar que es exactamente Grafana y para que podemos usarlo. Grafana es un software de monitoreo muy ampliamente utilizado para todo tipo de ambientes, se puede utilizar en tu casa para monitorear tu smart home (como consumos eléctricos, de gas o luminosidad a lo largo del día) como también en una empresa para identificar utilizacion de recursos, consumo de CPU y memoria por parte de máquinas virtuales o contenedores, etc.
Grafana puede interactuar con múltiples orígenes de datos gracias a sus "datasources" que son adicionales que se pueden instalar en Grafana para poder consumir metrics y logs de distintas fuentes.
Es importante aclarar que Grafana no produce ni modifica metrics ni logs, simplemente los consume y los muestra de una forma amigable con el usuario.
Normalmente, la creación de dashboards, paneles y otros recursos en Grafana es hecho a mano por medio de su panel de administración lo cual puede ser tedioso a veces cuando tenemos que deployar algunos componentes en la nube que van a estar por algún tiempo y queremos monitorearlos, pero a los pocos días los destruiremos, para tareas como esas podemos usar PowerGrafana que es un módulo de PowerShell que puede crear dashboards, paneles y targets de forma programática.

Como instalarlo

Para instalar Grafana tenemos varias opciones, dependiendo de nuestras necesidades podemos elegir entre:

Correr una máquina virtual e instalar Grafana dentro como un binario
Podemos correr Grafana como un container, esto puede ser dentro de una máquina virtual o en un ambiente de containers
Podemos deployar Grafana en Kubernetes de forma manual (nosotros creamos el pod y el servicio básico para que funcione)
Deployamos Grafana en Kubernetes usando un Helm Chart (vamos a explorar esta opción en este post)

Vamos a continuar con el uso de Helm Charts porque realmente creo que es LA forma en la que se deben deployar aplicaciones en Kubernetes, es mucho mas simple de mantener, mas manejable su configuración y además nos permite muy fácilmente deployar la misma aplicación con la misma configuración en múltiples clusters sin ningún cambio.

Para comenzar vamos a agregar el repositorio de Grafana y correr un helm repo update para actualizar la lista de repositorios en nuestra máquina.

Ahora podemos inspeccionar ese repositorio y ver que Charts tenemos disponibles, veamos:

Genial, tenemos el Chart que necesitamos Grafana, pero también tenemos en el mismo repositorio otras configuraciones de Grafana para distintos propósitos, continuemos.

Prerrequisitos

Como prerrequisitos vamos a necesitar muy pocas cosas:

Nuestro cluster de kubernetes
Un namespace en nuestro cluster
Una terminal de nuestra preferencia (bash, zsh, powershell) con kubectl y helm instalados
El repositorio de Grafana agregado en nuestro Helm local (como hicimos en el paso anterior)

Instalación

Para este ejercicio vamos a instalar los recursos de Grafana en un namespace separado, lo vamos a llamar monitoring, para eso ejecutamos:

kubectl create namespace monitoring

En mi terminal tengo instalado kubens que es una aplicación que nos permite setear el namespace en el que vamos a trabajar cuando ejecutemos kubectl y de esta forma nos ahorramos tener que especificar el -n namespace con cada comando

Ahora veamos nuevamente la lista de charts que agregamos:

helm search repo grafana

Perfecto, vemos que hay muchos charts además del que necesitamos, esto es porque en el mismo repositorio de Helm se distribuyen otras configuraciones de grafana (combinaciones e Grafana con otros productos).

Vamos a instalar el chart de Grafana:

helm install grafana grafana/grafana

Vemos que el comando instalo el chart de Grafana y ya nos esta indicando algunas cosas que son interesantes, veamos que son:

Warnings:

W1115 09:12:11.499698    9800 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W1115 09:12:11.529960    9800 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W1115 09:12:12.040639    9800 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W1115 09:12:12.040639    9800 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+

Esos warnings suelen aparecer en varios Charts, se deben a que la API de Kubernetes está en constante evolución y muchas veces algunos features van mutando y se van deprecando en favor de otros, estos warnings en particular hacen referencia a que resource kind PodSecurityPolicy está deprecado en la versión 1.21 de Kubernetes y va a ser totalmente removido en la versión 1.25 (mi clúster corre 1.22), por lo que hasta que actualice a 1.25, esto va a seguir funcionando, pero si actualizo mi clúster (o deployo este chart) a 1.25, este resource kind no va a existir más y por ende el chart va a fallar.

Password de admin de grafana:

Ni bien terminemos de instalar Grafana, vamos a querer loguearnos en su dashboard, para esto necesitamos el password del usuario admin y este password la podemos encontrar en un secret en el mismo deploy (en el mismo namespace), el comando que nos muestra helm install es lo que debemos hacer para conocer el password (Este password se guarda como todos los secrets en Kubernetes como un base64 encoded string)

kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Con este comando vamos a obtener el secret primero y luego vamos a pasárselo al comando base64 para que lo decodifique y nos lo muestre.

Si no tienen el comando base64, pueden simplemente usar brew (Linux y MacOS) o chocolatey (Windows)

Dado que estoy corriendo estos comandos en Windows, mi ejemplo es:

kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 -d

Y en mi caso el output es xzUvc2esXl2aSivqhQQ5X3ZZ01TZ2HMCEPdpWVSJ, ese es el password de mi usuario admin, este valor va a ser distinto en cada caso.

Accediendo Grafana por primera vez!

El último paso es acceder a Grafana, nos menciona que podemos acceder a grafana por medio del service grafana.monitoring.svc.cluster.local, pero esto solo funcionaria en caso de que estén corriendo el clúster de kubernetes de forma local en sus máquinas, en caso de que estén corriendo el clúster en la nube (como en mi caso) tienen que hacer un port-forward del puerto 3000 (el puerto default de Grafana) del pod a algún puerto que no estén usando en sus máquinas (podemos usar el 3000 también por practicidad), para esto necesitamos averiguar el nombre del pod, lo podemos hacer simplemente corriendo kubectl get pod o de forma más especifica kubectl get pods --namespace monitoring -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=grafana" -o jsonpath="{.items[0].metadata.name}"

Este comando nos va a decir el nombre del pod de grafana en el namespace monitoring que tenga los labels name=grafana y instance=grafana, en mi caso esto es grafana-5c999c4fd5-czxdw.

Ahora podemos hacer:

kubectl port-forward grafana-5c999c4fd5-czxdw 3000:3000

Usamos admin como usuario y el password que obtuvimos antes como password xzUvc2esXl2aSivqhQQ5X3ZZ01TZ2HMCEPdpWVSJ

En este punto tenemos Grafana instalado y configurado en nuestro clúster, tengan en cuenta que esta configuración de Grafana es muy simple y guarda toda la información sobre Dashboards y DataSources en un storage de tipo EmptyDir que es un tipo de storage volátil (cuando el pod se destruye, el storage también por lo que obtenemos un pod nuevo con un storage nuevo cada vez que matemos el pod, perdiendo toda la configuración que realicemos).

Si queremos cambiar este comportamiento, lo que debemos hacer es descargar el chart a nuestra máquina y ajustarlo para que en lugar de un EmptyDir utilice una storage class que nos dé persistencia. En un próximo artículo sobre Helm Chart voy a avanzar sobre como se puede hacer esto y mucho más.

Por último

Espero que esta explicación ayude a conocer lo básico de como instalar Grafana usando Helm Charts, si este post te fue útil, por favor compártelo con otros y si tienes sugerencias para más contenido o mejoras, por favor hacémelo saber en los comentarios de este post. ¡Muchas gracias!