Tekton 101🥷

Shivam Mukhade — Fri, 06 May 2022 07:14:04 +0000

This article is going to introduce what Tekton is and how we can build a CI/CD pipeline using Tekton.

There are multiple projects under the hat of Tekton but the heart is "Tekton Pipeline ".

Tekton Pipelines

Tekton Pipelines project provides kubernetes style resources for declaring CI/CD-style pipelines.

Before getting into understanding Tekton Concepts lets take couple of general examples and then relate it to Tekton...

(Here we will consider our CI would be doing following things

Build Test,
Unit Test,
Building container image,
pushing it to a remote registry
deploying the image on a kubernetes cluster )

If we want to run a CI manually on our machine how would we do it?

May be by writing a script?
or executing the commands one by one?

Lets take a scenario where we want to run CI/CD on a go project (A):
Step 1: We would clone the project (git clone)
Step 2: Run go build command
Step 3: Run go test command to run the test
Step 4: Using docker build command to build the image
Step 5: Docker push command to push the image to remote registry
Step 6: Create/Updating a Kubernetes Deployment with the newly built image

Now lets take another scenario where we want to run CI on a Java project (B):
Step 1: We would clone the project (git clone)
Step 2: Run javac command to compile the project
Step 3: Run tests
Step 4: Using docker build command to build the image
Step 5: Docker push command to push the image to remote registry
Step 6: Create/Updating a Kubernetes Deployment with the newly built image

If we compare both the scenarios, we can see following things

Step 4 & Step 5 are common in both, and are executed together in an order, isn't it? we would push a image only after building. this is where we introduce Tekton Task.

A Task is a collection of Steps that you define and arrange in a specific order of execution as part of your continuous integration flow.

Step 5 will be always executed after Step 4. So we can have a Tekton Task which has 2 steps -> building the image and then pushing it to a image registry. This task can be shared among both the projects.

Step 1 which is cloning the project can also be Tekton Task which can be shared in both projects.
Step 6 which is creating/updating a Kubernetes Deployment can be Task and can be shared too.

Rest of the Steps were project specific or language specific which could also be Tekton Task. In this scenarios we couldnt share those but for other projects we could definitely reuse those.

How to use the Task in a Pipeline, we are going to see it later in the article.

Here we saw that A Tekton Task is a collection of steps which can be executed in a specific order and the task can be reused across different pipelines by changing the parameters.

_ Tekton has a catalog which is collection of Tasks which are maintained by the community. You can access them hub.tekton.dev. You can directly use the Task in your Pipeline from here or you can modify existing based on your requirement._

In Hub, you can find Tasks for Cloning the code, running the test, and many more scenarios.

But Tekton Task is just a Template, if you apply it on a Kubernetes Cluster nothing would happen.

Consider following hello world Task, if you apply this on a Kubernetes cluster with Tekton Installed, nothing would happen.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: hello
spec:
  params:
    - name: text
      type: string
  steps:
    - name: say-hello-to
      image: registry.access.redhat.com/ubi8/ubi
      command:
        - /bin/bash
      args: ['-c', 'echo Hello $(params.text)!']

To run the Task, you need to create a TaskRun which is an running instance.

A TaskRun allows you to instantiate and execute a Task on-cluster

In above example, you can see there is param defined which can passed while executing the TaskRun.

A TaskRun would look like as below

apiVersion: tekton.dev/v1beta1                             
kind: TaskRun                             
metadata:                               
  generateName: hello-                             
spec:                               
  taskRef:                                 
    name: hello   
  params:
  - name: text
    value: "sm43"

You can create as many TaskRun as required by changing the parameters.

If you create a TaskRun, You can check the logs using tkn CLI.
tkn CLI is another sub project of Tekton which simplify interacting with Tekton resources using command line.

You can check the logs using

# get the name of taskrun created
tkn taskrun ls 

# get the logs
tkn taskrun logs <taskrun-name>

We saw Task and how to run it, but we would have to run multiple Tasks right? one for running test, one for building and pushing image.. that doesn't sound efficient 🤔

So, we have Tekton Pipeline where we can define different Tasks in the order we want to run.

Before we start building the Pipeline we are going another Tekton Concept which is [Workspace](https://tekton.dev/docs/pipelines/workspaces/#overview)

Workspaces allow Tasks to declare parts of the filesystem that need to be provided at runtime by TaskRuns

we will have multiple Tasks in a Pipeline, any multiple Tasks will need access to the application code, to run the test or to build the images, etc.

Workspace can be a volume which is shared among the Tasks in the Pipeline to execute their specific logic on the code.
In the example, we saw before we can use a workspace where git clone task will clone the code and rest of Tasks will have access to that workspace and execute their logic on that code.

Lets build a Pipeline
We are going to use a go application called news-demo.

You can clone the repository and follow the steps.

Prerequisites:
A Kubernetes cluster.

You can install Tekton Using Tekton Operator, steps can be found here.
NOTE: this will also install other Tekton projects.

or You can just install Tekton Pipeline which we are discussing in this article. you can find installation steps here

for the News-Demo application, we are going to build a Pipeline which would be doing following Tasks

clone the code in a workspace
build the code
test the code
build and push the image
check is a deployment alread exist
if no, create a deployment
if yes, then update the deployment

The pipeline can would look as below
You can find the complete pipeline here
below you can see part of the pipeline..

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: news-demo-deploy
  namespace: news-demo
spec:
  params:
    - name: REPO
    - name: REVISION
    - name: IMAGE
    - name: TAG
    - name: NAMESPACE
  workspaces:
    - name: shared-workspace
  tasks:
    - name: fetch-repository
      taskRef:
        name: git-clone
      workspaces:
        - name: output
          workspace: shared-workspace
      params:
        - name: url
          value: $(params.REPO)
        - name: subdirectory
          value: ""
        - name: deleteExisting
          value: "true"
        - name: revision
          value: $(params.REVISION)
    - name: build-test
      runAfter:
        - fetch-repository
      taskRef:
        name: golang-build
      params:
        - name: packages
          value: ./...
        - name: package
          value: github.com/sm43/news-demo
      workspaces:
        - name: source
          workspace: shared-workspace
    - name: unit-tests
      runAfter:
        - fetch-repository
      taskRef:
        name: golang-test
      params:
        - name: package
          value: github.com/sm43/news-demo
        - name: flags
          value: -v -mod=vendor
      workspaces:
        - name: source
          workspace: shared-workspace
    - name: build-push-image
      taskRef:
        name: buildah
      workspaces:
        - name: source
          workspace: shared-workspace
      params:
        - name: IMAGE
          value: $(params.IMAGE):$(params.TAG)
        - name: FORMAT
          value: "docker"
      runAfter:
        - build-test
        - unit-tests
    - name: check-deployment
      taskRef:
        name: kubernetes-actions
      params:
        - name: script
          value: |
            kubectl describe deployment news-demo -n "$(params.NAMESPACE)" >/dev/null 2>/dev/null
            if [[ $? -eq 0 ]]; then
              printf yes | tee /tekton/results/output-result
            else
              printf no | tee /tekton/results/output-result
            fi
      runAfter:
        - build-push-image
    - name: patch-image
      taskRef:
        name: kubernetes-actions
      params:
        - name: script
          value: |
            kubectl patch deployment news-demo --patch='{"spec":{"template":{"spec":{
              "containers":[{
                "name": "news-demo",
                "image": "$(params.IMAGE):$(params.TAG)"
              }]
            }}}}' -n $(params.NAMESPACE)
      when:
        - input: "$(tasks.check-deployment.results.output-result)"
          operator: in
          values: ["yes"]
      runAfter:
        - check-deployment
    - name: create-deployment
      taskRef:
        name: kubernetes-actions
      workspaces:
        - name: manifest-dir
          workspace: shared-workspace
      params:
        - name: script
          value: |
            kubectl -n $(params.NAMESPACE) apply -f <(sed "s@image:.*@image: $(params.IMAGE):$(params.TAG)@" k8s/02-deployment.yaml)
      when:
        - input: "$(tasks.check-deployment.results.output-result)"
          operator: in
          values: ["no"]

Lets understand the pipeline

The first Task is git-clone which we will be using from Tekton Hub. we will install it on the cluster and reference in our Pipeline.
You can see we have used a workspace where the task will clone the code and we pass the same workspace to other Tasks to access the code.
Depending on project we will pass the params.
Similarly we are going to use golang-build and golang-test from Tekton Hub and refer it on our Pipeline.
One thing to notice here is we have a field runAfter. If you see in build-test we have

runAfter:
  - fetch-repository

which means run this task after completing fetch-repostitory which is nothing but git-clone. this is how we can specify order.

After running build test and unit test, we use buildah Task to build and push image.
Finally once image is built, we will create or update a Deployment.
Here we use when expression to decide which action to perform. When we run check-deployment, it checks whether the deployment exist using kubernetes-actions Task from Tekton Hub and add its output to results.
So, the next 2 Tasks in the Pipeline will be executed on the basis of output of check-deployment Task.
if check-deployment output is yes then patch-image will be executed else create-deployment will be executed.

 when:
        - input: "$(tasks.check-deployment.results.output-result)"
          operator: in
          values: ["yes"]

Lets execute it...

Setting up application

You can clone the repository using

git clone https://github.com/sm43/news-demo.git

The application requires a News API account and its key to work. Sign up for a News API account and get your free API key.

Once you have the key

cd news-demo

Edit the configMap in k8s directory and add your API key for the variable NEWS_API_KEY.

After updating the configMap, you can apply the manifest on the cluster using kubectl.

kubectl apply -f k8s/

This will deploy the application in news-demo namespace and create a service for the deployment.

To access the application outside cluster, Create a Route if you are OpenShift cluster or Ingress on a kubernetes cluster.

kubectl apply -f k8s/openshift/

You can get the route and access it on a browser.

echo "https://$(kubectl get routes news-demo -n news-demo -ojsonpath='{.spec.host}')"

You would be able to see the below application

Setting up Pipeline

Before applying the pipeline, we will need to create some resources, run the bash script in pipeline directory.

Edit the script to add your image registry credentials so that pipeline can push image to your registry. and update your registry username in pipelineRun.

Execute the scipt

./pipeline/run.sh

This script install task from catalog, create service account which has access to your registry for pushing the image, rbac required for creating/updating deployment, pipeline and starts the pipeline by creating pipelineRun.

You can use tkn to access the resource

# List the pipelineRuns
tkn pipelinerun ls

# follow the logs of pipelineRun
tkn pipelinerun logs -f

Wait for pipelineRun to be completed and then check the image in deployment..

Previously, when we deployed the application the image was quay.io/sm43/news-demo:latest and now the pipeline has updated the image to quay.io/sm43/news-demo:v0.1.

We have successfully ran the pipeline.

Now, if you do any changes in your code and start the Pipleine then you would see the changes deployed on the cluster after successfully completing the Pipeline.

But this is manually right? we can't keep running manually each time. we want to run based on events.
We want to run the CI when we create a pull request or push a commit to a branch.

So for this we need to set up Tekton Triggers. We will seeing how to setup Tekton Trigger in a following article.

Till then you can checkout the complete series about Tekton here.

If you liked the article please leave feedback.

Forem: Shivam Mukhade

Tekton 101🥷

Tekton Pipelines

Setting up application

Setting up Pipeline