Forem: Preetham

Serverless Golang REST API with AWS Lambda

Preetham — Sat, 29 Jan 2022 18:48:06 +0000

In this post, let us go through the process of deploying a Golang API to AWS Lambda and handling requests through AWS API Gateway, a process which is very straight forward and takes a few minutes once we know the steps.

I am assuming that you have a basic knowledge of Go and an AWS account. If you don’t know anything about AWS Lambda and API Gateway, don’t worry, all we need to get started is explained here. If you already have a good understanding of Lambda and API Gateway, feel free to skip some of the explanations.

I was inspired from may other posts and one of the main reason I am writing this post is because most of the posts involves explanation of additional components like databases, cache, etc which is really useful but not a straightforward post on getting started with AWS Lambda Golang REST API methodologies with API gateway as there are many use-cases without the need of any database or state to be maintained.

So we can take a look at working with databases like dynamodb in a different post and in this post, we will create a simple rest api which does the below

Method	Route	Action
GET	/mirror?name=xxx	Returns the query parameter name like a mirror and an error if name is not found
POST	/mirror	Accepts a json payload and returns it back like a mirror and returns an error when json is invalid or payload is empty

With these APIs, we will get an understanding of how to work with the http methods and parsing the incoming request.

Setting up the aws cli
Creating Simple Golang REST API
Creating and deploying an AWS Lambda function
Setting up the HTTP API in AWS API Gateway
Deploying the HTTP API created

Setting up the aws cli

Installation and basic usage instructions can be found here. We can verify the installation as below

aws --version

aws-cli/2.2.44 Python/3.8.8 Darwin/20.5.0 exe/x86_64 prompt/off

We need to setup an AWS IAM user with programmatic access permission for the CLI to use. Detailed official instructions can be found here here and you can also check this post for a detailed walkthrough. For getting started, we will attach AdministratorAccess policy to this user but for production, recommended to use a restricted policy with only the necessary levels of access. Note down the aws_access_key_id and aws_secret_access_key obtained.
Configure the CLI to use the credentials of the IAM user we just created. We will use the region us-east-2 and the output format as json (json is easy to read).

$ aws configure
AWS Access Key ID (None): aws_access_key_id
AWS Secret Access Key (None): aws_secret_access_key
Default region name (None): us-east-2
Default output format (None): json

More detailed instructions can be found here

Creating Simple Golang REST API

1 .GO Installation

Though the deployment mode is serverless in AWS cloud, we would need Go installed in your machine for compiling the go package.
Follow the instructions in the official page for installing go in your machine.
Verify that Go is installed.

$ go version

go version go1.17.2 darwin/amd64

2 .Creating the Folder structure

mkdir serverlessgolang
cd serverlessgolang

3 .Initiate the project

go mod init <project-name>

The above command initiates the project and creates a file called go.mod - which is similar to package.json in nodejs.

Its a good approach to give the project-name as a repo link (github.com/yourid/ and also to keep this folder structure as part of git, but not compulsory.

4 .Create the golang file and install the packages

Just proceed, we will create this file and proceed with the explanation of each part below.

vi main.go

and in the editor

package main

import (
    "fmt"

    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
    "github.com/valyala/fastjson"
)

func HandleRequest(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    ApiResponse := events.APIGatewayProxyResponse{}
    // Switch for identifying the HTTP request
    switch request.HTTPMethod {
    case "GET":
        // Obtain the QueryStringParameter
        name := request.QueryStringParameters["name"]
        if name != "" {
            ApiResponse = events.APIGatewayProxyResponse{Body: "Hey " + name + " welcome! ", StatusCode: 200}
        } else {
            ApiResponse = events.APIGatewayProxyResponse{Body: "Error: Query Parameter name missing", StatusCode: 500}
        }

    case "POST":    
        //validates json and returns error if not working
        err := fastjson.Validate(request.Body)

        if err != nil {
            body := "Error: Invalid JSON payload ||| " + fmt.Sprint(err) + " Body Obtained" + "||||" + request.Body
            ApiResponse = events.APIGatewayProxyResponse{Body: body, StatusCode: 500}
        } else {
            ApiResponse = events.APIGatewayProxyResponse{Body: request.Body, StatusCode: 200}
        }

    }
    // Response
    return ApiResponse, nil
}

func main() {
    lambda.Start(HandleRequest)
}

Now after creating the file, we can install all the required modules using

go mod tidy

which will download and install the imported files in main.go or if any other go files present in the folder.

The alternative approach is to manually install the packages

go get <package-name>
---
go get github.com/aws/aws-lambda-go
go get github.com/valyala/fastjson

4.1 .Code and Package Explanation

Now, let us go through in detail the packages imported and the code

Package github.com/aws/aws-lambda-go/lambda

This package implements the Lambda programming model for Go and is used to start or initiate the lambda handler. Any Golang Lambda function must have this package.

In the function main(), we call lambda.start() and pass in the HandleRequest() as the lambda handler function.
Our handler function, as programmed will parse the request and returns a welcome message incase of Get method and returns the input body in case of POST method.

More details on this package can be found here.

Package github.com/aws/aws-lambda-go/events

This package provides input types for Lambda functions that process AWS events. There are a variety of events which can be consumed by the lambda function, in which we will be working on API Gateway events and the package also provides two useful types APIGatewayProxyRequest and APIGatewayProxyResponse which contain useful information about incoming HTTP requests and send responses that the API Gateway understands.

type APIGatewayProxyRequest struct {
    Resource              string                        `json:"resource"` // The resource path defined in API Gateway
    Path                  string                        `json:"path"`     // The url path for the caller
    HTTPMethod            string                        `json:"httpMethod"`
    Headers               map[string]string             `json:"headers"`
    QueryStringParameters map[string]string             `json:"queryStringParameters"`
    PathParameters        map[string]string             `json:"pathParameters"`
    StageVariables        map[string]string             `json:"stageVariables"`
    RequestContext        APIGatewayProxyRequestContext `json:"requestContext"`
    Body                  string                        `json:"body"`
    IsBase64Encoded       bool                          `json:"isBase64Encoded,omitempty"`
}

type APIGatewayProxyResponse struct {
    StatusCode      int               `json:"statusCode"`
    Headers         map[string]string `json:"headers"`
    Body            string            `json:"body"`
    IsBase64Encoded bool              `json:"isBase64Encoded,omitempty"`
}

here,
request is of type events.APIGatewayProxyRequest and
response is of type events.APIGatewayProxyResponse

and using

request.HTTPMethod we identify the GET and POST invocations
request.Body we get the request body
request.QueryStringParameters we get the query parameter name
response.Body we send the response message to the user
response.StatusCode we send the response status code

Notice how in all cases the error value returned from our lambda handler is nil. This is because the AWS API Gateway doesn't accept error objects when you're using it in conjunction with a lambda proxy integration (they would result in a 'malformed response' errors again). So we need to manage errors fully within our lambda function and return the appropriate HTTP response.

Package github.com/valyala/fastjson

This package contains json parsing and validating methods which are faster than default json package.
We are using the method fastjson.Validate(request.Body) to validate the json.

We can also validate the input payload in the API gateway, which is recommended as it will not invoke the lambda function on wrong inputs and saving cost. We will look at it later.

5. Creating an executable and zipping

env GOOS=linux GOARCH=amd64 go build -o output/main

here, we are setting the environment variable

GOOS=linux, implying build the executable for linux OS
GOARCH=amd64, implying that the executable is build for execution in amd64 cpu architechture
-o <path/file> is the output file name

Next, we need to zip the file to upload it to AWS Lambda function

zip -j output/function.zip output/main

this will create a zip file

Creating and deploying an AWS Lambda function

1. IAM Role

Before deploying a lambda function, We need to set up an IAM role which defines the permission that our lambda function will have when it is running.

For now let's set up a lambda-function-executor role and attach the AWSLambdaBasicExecutionRole managed policy to it. This will give our lambda function the basic permissions it need to run and log to the AWS cloudwatch service.

First we have to create a trust policy JSON file. This will essentially instruct AWS to allow lambda services to assume the lambda-function-executor role

File: ./tmp/trust-policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Then use the aws iam create-role command to create the role with this trust policy

aws iam create-role --role-name lambda-function-executor \
--assume-role-policy-document file://./tmp/trust-policy.json

Response:

{
    "Role": {
        "Path": "/",
        "RoleName": "lambda-function-executor",
        "RoleId": "AROAZ5VSPHO6WRWZRHZBX",
        "Arn": "arn:aws:iam::682200349629:role/lambda-function-executor",
        "CreateDate": "2022-01-29T13:54:20+00:00",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": "lambda.amazonaws.com"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}

Let us store the ARN value returned in a variable

rolearn=arn:aws:iam::682200349629:role/lambda-function-executor

Now the role has been created and we need to specify its permissions. we can attach the policy AWSLambdaBasicExecutionRole using the aws iam attach-role-policy command as below

aws iam attach-role-policy --role-name lambda-function-executor \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

We can find a list of other lambda permission policies here.

2. Deploy the golang build into Lambda function

Now, let us deploy the actual Lambda function to AWS using the command aws lambda create-function. This function takes the below flags and would take a few minutes to complete.

Flag	Description
--function-name	The name our lambda function will be called within AWS
--runtime	The runtime environment for the lambda function (in our case "go1.x")
--role	The ARN of the role we want the lambda function to assume when it is running (obtained in the above step)
--handler	The name of the executable in the root of the zip file (in our case its main)
--zip-file	Path to the zip file we created in Step 2.5

functionName=gettingstarted #you can give your preferred name
aws lambda create-function --function-name $functionName --runtime go1.x \
--role $rolearn \
--handler main --zip-file fileb://./output/function.zip

Response

{
    "FunctionName": "gettingstarted",
    "FunctionArn": "arn:aws:lambda:us-east-2:682200349629:function:gettingstarted",
    "Runtime": "go1.x",
    "Role": "arn:aws:iam::682200349629:role/lambda-function-executor",
    "Handler": "main",
    "CodeSize": 4458249,
    "Description": "",
    "Timeout": 3,
    "MemorySize": 128,
    "LastModified": "2022-01-29T14:13:10.826+0000",
    "CodeSha256": "VwOZE7cyuYdCtMS2MH7KFaM9NgrQn2fChDyw3B2rnxE=",
    "Version": "$LATEST",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "e07cccd1-42b3-4c7b-9524-fc18b067a4d3",
    "State": "Pending",
    "StateReason": "The function is being created.",
    "StateReasonCode": "Creating",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ]
}

we can store the function arn in a variable

functionarn=arn:aws:lambda:us-east2:682200349629:function:gettingstarted

You can validate or list all the functions using

aws lambda list-functions

Now our lambda function has been deployed and is now ready to use. we can try it out by using the aws lambda invoke command (which requires us to specify an output file for the response — we are using ./output/output.json in the snippet below).

aws lambda invoke --function-name gettingstarted ./output/output.json

We got the output

{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}

Now the Lambda function has been created and deployed.

Setting up the HTTPS API in AWS API Gateway

1 . We can create a REST API using the command aws apigateway create-rest-api. we are choosing type as REGIONAL as we want the API gateway to be reachable from the region. We can also choose EDGE - if we want our API gateway to be accessible via AWS Cloudfront.

aws apigateway create-rest-api --name apitest --endpoint-configuration types=REGIONAL

Response

{
    "id": "knvvn50y71",
    "name": "firstapi",
    "createdDate": "2022-01-29T20:14:39+05:30",
    "apiKeySource": "HEADER",
    "endpointConfiguration": {
        "types": [
            "REGIONAL"
        ]
    },
    "disableExecuteApiEndpoint": false
}

Let us store the id returned, we will be using it a lot

restApiId=knvvn50y71

2 . Next we need to get the id of the root API resource ("/"). We can retrieve this using the command aws apigateway get-resources using the api id obtained above

aws apigateway get-resources --rest-api-id $restApiId

Response

{
    "items": [
        {
            "id": "4hsrin7w54",
            "path": "/"
        }
    ]
}

Again, let us take a note of the root-path-id value this returns as parentId (root=> here is parent)

parentId=4hsrin7w54

3 . Now we need to create a new resource under the root path — specifically a resource for the URL path /mirror. We can do this by using the aws apigateway create-resource command with the --path-part parameter

aws apigateway create-resource --rest-api-id $restApiId \
--parent-id $parentId --path-part mirror

Response

{
    "id": "rgrcck",
    "parentId": "4hsrin7w54",
    "pathPart": "mirror",
    "path": "/mirror"
}

Again, let us take a note of the resource-id this returns.

resourceId=rgrcck

Let us understand that it's possible to include placeholders within the path by wrapping part of the path in curly braces. For example, a --path-part parameter of mirror/{name} would match requests to /mirror/goku and /mirror/vegeta, and the value of name would be made available to our lambda function via the events object. We can also make a placeholder greedy by postfixing it with a +. A common idiom is to use the parameter --path-part {proxy+} if we want to match all requests regardless of their path.

4 . But we will not be doing it, but rather read the values from a query parameter as decided in the start. Let's get back to our /mirror resource and use the aws apigateway put-method command to register the HTTP method of ANY. This will mean that our /mirror resource will respond to all requests regardless of their HTTP method.

aws apigateway put-method --rest-api-id $restApiId \
--resource-id $resourceId --http-method ANY \
--authorization-type NONE

Response

{
    "httpMethod": "ANY",
    "authorizationType": "NONE",
    "apiKeyRequired": false
}

5 . Now let us integrate the api gateway with the lambda function using the command aws apigateway put-integration.

Flag	Description
--type	Value should always be `AWS_PROXY` and this ensures the AWS API Gateway sends information about the HTTP request as an `event` to the lambda function and also automatically transform lambda function output to a HTTP response.
--integration-http-method	Value should be `POST` Let us not confuse this with what HTTP methods our API resource responds to.
--uri	parameter needs to take the format below

arn:aws:apigateway:<region>:lambda:path/2015-03-31/functions/<our-lambda-function-arn>/invocations

If we have not copied our lambda function arn, we can get it using

aws lambda list-functions

The account id of the current cli can be found using the below command

aws sts get-caller-identity

Now call the command

region=us-east-2
accountId=682200349629
uri="arn:aws:apigateway:""$region"":lambda:path/2015-03-31/functions/arn:aws:lambda:""$region"":""$accountId"":function:""$functionName""/invocations"
####
aws apigateway put-integration --rest-api-id $restApiId \
--resource-id $resourceId --http-method ANY --type AWS_PROXY \
--integration-http-method POST \
--uri $uri

Response

    "type": "AWS_PROXY",
    "httpMethod": "POST",
    "uri": "arn:aws:apigateway:us-east-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-2:682200349629:function:gettingstarted/invocations",
    "passthroughBehavior": "WHEN_NO_MATCH",
    "timeoutInMillis": 29000,
    "cacheNamespace": "rgrcck",
    "cacheKeyParameters": []
}

6 . Now, when we test the REST API with the command aws apigateway test-invoke-method we will see an error

aws apigateway test-invoke-method --rest-api-id $restApiId --resource-id $resourceId --http-method "GET"

Error Response

{
    "status": 500,
    "body": "{\"message\": \"Internal server error\"}",
    "headers": {
        "x-amzn-ErrorType": "InternalServerErrorException"
    },
    "multiValueHeaders": {
        "x-amzn-ErrorType": [
            "InternalServerErrorException"
        ]
    },
    "log": "Execution log for request 198564f9-28e4-4578-895d-290af2f6dee6\nSat Jan 29 16:36:37 UTC 2022 : Starting execution for request: 198564f9-28e4-4578-895d-290af2f6dee6\nSat Jan 29 16:36:37 UTC 2022 : HTTP Method: GET, Resource Path: /mirror\nSat Jan 29 16:36:37 UTC 2022 : Method request path: {}\nSat Jan 29 16:36:37 UTC 2022 : Method request query string: {}\nSat Jan 29 16:36:37 UTC 2022 : Method request headers: {}\nSat Jan 29 16:36:37 UTC 2022 : Method request body before transformations: \nSat Jan 29 16:36:37 UTC 2022 : Endpoint request URI: https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-2:682200349629:function:gettingstarted/invocations\nSat Jan 29 16:36:37 UTC 2022 : Endpoint request headers: {X-Amz-Date=20220129T163637Z, x-amzn-apigateway-api-id=knvvn50y71, Accept=application/json, User-Agent=AmazonAPIGateway_knvvn50y71, Host=lambda.us-east-2.amazonaws.com, X-Amz-Content-Sha256=a33d05de2f7337bb6ae5302f16240b5f3e42be56141d90f6954dc0372771ea72, X-Amzn-Trace-Id=Root=1-61f56d15-1e3289d5c10359a5bb24075f, x-amzn-lambda-integration-tag=198564f9-28e4-4578-895d-290af2f6dee6, Authorization=*********************************************************************************************************************************************************************************************************************************************************************************************************************************************21274e, X-Amz-Source-Arn=arn:aws:execute-api:us-east-2:682200349629:knvvn50y71/test-invoke-stage/GET/mirror, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEMj//////////wEaCXVzLWVhc3QtMiJHMEUCIQCE/xgic5zfU9dT5XjUcGjHKmXwV+z2m36xVB2BxHKFqgIgOjEBR08BSPnA6inOTdhJ2P92c0OA9cJaaQzA/zS [TRUNCATED]\nSat Jan 29 16:36:37 UTC 2022 : Endpoint request body after transformations: {\"resource\":\"/mirror\",\"path\":\"/mirror\",\"httpMethod\":\"GET\",\"headers\":null,\"multiValueHeaders\":null,\"queryStringParameters\":null,\"multiValueQueryStringParameters\":null,\"pathParameters\":null,\"stageVariables\":null,\"requestContext\":{\"resourceId\":\"rgrcck\",\"resourcePath\":\"/mirror\",\"httpMethod\":\"GET\",\"extendedRequestId\":\"Mt37aHfKiYcF_QQ=\",\"requestTime\":\"29/Jan/2022:16:36:37 +0000\",\"path\":\"/mirror\",\"accountId\":\"682200349629\",\"protocol\":\"HTTP/1.1\",\"stage\":\"test-invoke-stage\",\"domainPrefix\":\"testPrefix\",\"requestTimeEpoch\":1643474197654,\"requestId\":\"198564f9-28e4-4578-895d-290af2f6dee6\",\"identity\":{\"cognitoIdentityPoolId\":null,\"cognitoIdentityId\":null,\"apiKey\":\"test-invoke-api-key\",\"principalOrgId\":null,\"cognitoAuthenticationType\":null,\"userArn\":\"arn:aws:iam::682200349629:user/preetham\",\"apiKeyId\":\"test-invoke-api-key-id\",\"userAgent\":\"aws-cli/2.2.44 Python/3.8.8 Darwin/20.5.0 exe/x86_64 prompt/off command/apigateway.test-invoke-method\",\"accountId\":\"682200349629\",\"caller\":\"AIDA [TRUNCATED]\nSat Jan 29 16:36:37 UTC 2022 : Sending request to https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-2:682200349629:function:gettingstarted/invocations\nSat Jan 29 16:36:37 UTC 2022 : Execution failed due to configuration error: Invalid permissions on Lambda function\nSat Jan 29 16:36:37 UTC 2022 : Method completed with status: 500\n",
"latency": 36
}

So if we try to isolate the error, it says
Execution failed due to configuration error: Invalid permissions on Lambda function

This is because our API gateway doesn't have permission to execute our lambda function

7 . we can use aws lambda add-permission command to give the API permissions to invoke the lambda

we are generating a random uid using uuidgen command and the region and accountId is part of the session variable as we have defined in one of the previous steps

uid=`uuidgen`
sourceArn="arn:aws:execute-api:""$region"":""$accountId"":""$restApiId""/*/*/*"
aws lambda add-permission --profile preetham --function-name $functionName --statement-id $uid \
--action lambda:InvokeFunction --principal apigateway.amazonaws.com \
--source-arn $sourceArn

Response

{
    "Statement": "{\"Sid\":\"E7B0C866-B985-4F0D-8DEA-AE560A04EBFE\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"apigateway.amazonaws.com\"},\"Action\":\"lambda:InvokeFunction\",\"Resource\":\"arn:aws:lambda:us-east-2:682200349629:function:gettingstarted\",\"Condition\":{\"ArnLike\":{\"AWS:SourceArn\":\"arn:aws:execute-api:us-east-2:682200349629:knvvn50y71/*/*/*\"}}}"
}

8 . we can test the api gateway again

aws apigateway test-invoke-method --rest-api-id $restApiId --resource-id $resourceId --http-method "GET"

Response

{
    "status": 500,
    "body": "Error: Query Parameter name missing",
    "headers": {
        "X-Amzn-Trace-Id": "Root=1-61f57957-74340f97bf9f0046a2c78983;Sampled=0"
    },
    "multiValueHeaders": {
        "X-Amzn-Trace-Id": [
            "Root=1-61f57957-74340f97bf9f0046a2c78983;Sampled=0"
        ]
    },
    "log": "Execution log for request 0f285ec1-58ec-46be-be7b-2f002efcdc75\nSat Jan 29 17:28:55 UTC 2022 : Starting execution for request: 0f285ec1-58ec-46be-be7b-2f002efcdc75\nSat Jan 29 17:28:55 UTC 2022 : HTTP Method: GET, Resource Path: /mirror\nSat Jan 29 17:28:55 UTC 2022 : Method request path: {}\nSat Jan 29 17:28:55 UTC 2022 : Method request query string: {}\nSat Jan 29 17:28:55 UTC 2022 : Method request headers: {}\nSat Jan 29 17:28:55 UTC 2022 : Method request body before transformations: \nSat Jan 29 17:28:55 UTC 2022 : Endpoint request URI: https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-2:682200349629:function:gettingstarted/invocations\nSat Jan 29 17:28:55 UTC 2022 : Endpoint request headers: {X-Amz-Date=20220129T172855Z, x-amzn-apigateway-api-id=knvvn50y71, Accept=application/json, User-Agent=AmazonAPIGateway_knvvn50y71, Host=lambda.us-east-2.amazonaws.com, X-Amz-Content-Sha256=7890b63bd5654eb21207f4af1bcb751ba84c0498a9a9929e309dafd30f9e8cc8, X-Amzn-Trace-Id=Root=1-61f57957-74340f97bf9f0046a2c78983, x-amzn-lambda-integration-tag=0f285ec1-58ec-46be-be7b-2f002efcdc75, Authorization=*********************************************************************************************************************************************************************************************************************************************************************************************************************************************649380, X-Amz-Source-Arn=arn:aws:execute-api:us-east-2:682200349629:knvvn50y71/test-invoke-stage/GET/mirror, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEMn//////////wEaCXVzLWVhc3QtMiJHMEUCIBki4ycUlnDnl5O2JNLu3QZ4YxX6e+oAoPgBMXAZDNb+AiEA6DzZsiRL6GIEAmFjwfPISLGV2cQiMW+IIGk0AG5 [TRUNCATED]\nSat Jan 29 17:28:55 UTC 2022 : Endpoint request body after transformations: {\"resource\":\"/mirror\",\"path\":\"/mirror\",\"httpMethod\":\"GET\",\"headers\":null,\"multiValueHeaders\":null,\"queryStringParameters\":null,\"multiValueQueryStringParameters\":null,\"pathParameters\":null,\"stageVariables\":null,\"requestContext\":{\"resourceId\":\"rgrcck\",\"resourcePath\":\"/mirror\",\"httpMethod\":\"GET\",\"extendedRequestId\":\"Mt_lvGoICYcF6zQ=\",\"requestTime\":\"29/Jan/2022:17:28:55 +0000\",\"path\":\"/mirror\",\"accountId\":\"682200349629\",\"protocol\":\"HTTP/1.1\",\"stage\":\"test-invoke-stage\",\"domainPrefix\":\"testPrefix\",\"requestTimeEpoch\":1643477335759,\"requestId\":\"0f285ec1-58ec-46be-be7b-2f002efcdc75\",\"identity\":{\"cognitoIdentityPoolId\":null,\"cognitoIdentityId\":null,\"apiKey\":\"test-invoke-api-key\",\"principalOrgId\":null,\"cognitoAuthenticationType\":null,\"userArn\":\"arn:aws:iam::682200349629:user/preetham\",\"apiKeyId\":\"test-invoke-api-key-id\",\"userAgent\":\"aws-cli/2.2.44 Python/3.8.8 Darwin/20.5.0 exe/x86_64 prompt/off command/apigateway.test-invoke-method\",\"accountId\":\"682200349629\",\"caller\":\"AIDA [TRUNCATED]\nSat Jan 29 17:28:55 UTC 2022 : Sending request to https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-2:682200349629:function:gettingstarted/invocations\nSat Jan 29 17:28:56 UTC 2022 : Received response. Status: 200, Integration latency: 292 ms\nSat Jan 29 17:28:56 UTC 2022 : Endpoint response headers: {Date=Sat, 29 Jan 2022 17:28:56 GMT, Content-Type=application/json, Content-Length=103, Connection=keep-alive, x-amzn-RequestId=60d92f89-317c-4c29-894d-c157bcc9b79f, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-61f57957-74340f97bf9f0046a2c78983;sampled=0}\nSat Jan 29 17:28:56 UTC 2022 : Endpoint response body before transformations: {\"statusCode\":500,\"headers\":null,\"multiValueHeaders\":null,\"body\":\"Error: Query Parameter name missing\"}\nSat Jan 29 17:28:56 UTC 2022 : Method response body after transformations: Error: Query Parameter name missing\nSat Jan 29 17:28:56 UTC 2022 : Method response headers: {X-Amzn-Trace-Id=Root=1-61f57957-74340f97bf9f0046a2c78983;Sampled=0}\nSat Jan 29 17:28:56 UTC 2022 : Method completed with status: 500\n",
    "latency": 307
}

We can see that the error

Error: Query Parameter name missing

which is the same as configured in our golang code. Yay !!

This is fine, but now we cannot execute this via an URL.

Deploying the HTTP API created

We can run the below command to make it live

aws apigateway create-deployment --rest-api-id $restApiId \
--stage-name staging

Response

{
    "id": "4pdblq",
    "createdDate": 1522929303
}

Now, the url can be accessed by
https://<rest-api-id>.execute-api.<region>.amazonaws.com/staging/mirror

url="https://""$restApiId"".execute-api.""$region"".amazonaws.com/staging/mirror"

GET Method

curl --location --request GET $url

Response

Hey Goku welcome!

POST Method

curl --location --request POST $url \
--header 'Content-Type: application/json' \
--data-raw '{
    "country" : "India"
}'

Response

{
    "country" : "India"
}

Now, for our function, if we want to make any changes or rebuild the code, we can rebuild it, zip it and upload it by by the command

aws lambda update-function-code --function-name $functionName \
--zip-file fileb:///tmp/function.zip

With this, we will be able to build a simple AWS Lambda REST API with Golang. Now, you can edit the handler function to suit your requirement.

Cover Image Source: https://blog.travelex.io/blazing-fast-microservice-with-go-and-lambda-d30d95290f28

What is an Environment?

Preetham — Sun, 02 Aug 2020 16:34:46 +0000

I have always wondered how we could describe an environment? We have Dev, test, Prod environments and so on.

Below is my understanding of an environment.

An environment encompasses the necessary hardware(single or multiple servers)(physical / virtual ), Software dependencies, database, compiler/interpreter, network access for an application to run and is isolated from other environments unless explicitly connected.

Is this correct or is there a better way to describe it ? I want to understand this better as this is something that I use everyday and so I want my understanding of something fundamental to be solid.

Can I go with Pop OS rather than ubuntu?

Preetham — Sat, 01 Aug 2020 17:33:57 +0000

I bought a new laptop with windows 10 home edition pre-installed. Hence I went with WSL and it was not so good.Hence I spun an aws EC2 instance and working on it. But it aint free and doesn't feel like working on a local machine. So I thought of installing Linux and I had previously used ubuntu a couple of years back. Now I stumbled upon Pop OS. Can I go with it rather than ubuntu or install Ubuntu ?

Kubernetes Services and Deployments coming together

Preetham — Thu, 23 Jul 2020 06:24:22 +0000

In this tutorial, we are going to look at how the services and deployments come together and the WordPress web application is up and running.

By the end of this tutorial, we will have

After creating the service and deployment objects, I found another major issue. I was skeptical about whether to write this or write along with the fix for the issue. Still, I wanted to show this, as I believed that we want to understand what purpose each individual component satisfies.

Shush!Shush! Back to our topic. Since we understand the pods, deployments, and services I'll jump into the manifest part.

db-service.yaml

---
apiVersion: v1
kind: Service
metadata: 
    name: wpdb-service
    labels:
        app: wordpress
        type: db-service
spec:
    selector:
        app: wordpress
        type: db
    ports:
        - protocol: TCP
          port: 3306
          targetPort: 3306
    clusterIP: None

The .spec.selector you see above will match with the .spec.template.metadata.labels - which creates the link between the service and the pods in the deployment

db-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
    name: wordpress-db
    labels:
        app: wordpress
        type: db-deployment
spec:
    replicas: 2
    selector:
        matchLabels:
            app: wordpress
            type: db
    template:
        metadata:
            name: wordpress-db
            labels:
                app: wordpress
                type: db
        spec:
            containers:
                - name: wordpress-db-container
                  image: mysql:5.7
                  env:
                  - name: MYSQL_ROOT_PASSWORD
                    value: DEVOPS1
                  - name: MYSQL_DATABASE
                    value: wpdb
                  - name: MYSQL_USER
                    value: wpuser
                  - name: MYSQL_PASSWORD
                    value: DEVOPS12345
                  ports:
                    - containerPort: 3306
                      name: mysql

In the top db-deployment.yaml, always 2 replicas of db pod will be created and maintained. The .spec.selector.matchLabels and .spec.template.metadata.labels must match for the deployment to maintain the pods. We have passed the environment variables to configure the database and we will use these for the WordPress pod to connect to the DB.

Below, I have combined the service and deployment for the WordPress app in a single YAML.

app.yaml

---
apiVersion: v1
kind: Service
metadata: 
    name: app-service
    labels:
        app: wordpress
        type: app-service
spec:
    selector:
        app: wordpress
        type: app
    type: LoadBalancer
    ports:
        - protocol: TCP
          port: 80
          targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
    name: wordpress-app
    labels:
        app: wordpress
        type: wordpress-deployment
spec:
    replicas: 3
    selector:
        matchLabels:
            app: wordpress
            type: app
    template:
        metadata:
            name: wordpress-app
            labels:
                app: wordpress
                type: app
        spec:
            containers:
            - name: wordpress-app
              image: wordpress
              env:
              - name: WORDPRESS_DB_HOST
                value: wpdb-service
              - name: WORDPRESS_DB_NAME
                value: wpdb
              - name: WORDPRESS_DB_USER
                value: wpuser
              - name: WORDPRESS_DB_PASSWORD
                value: DEVOPS12345
              ports:
                - containerPort: 80
                  name: wordpress

In the env section of the WordPress container pod, for WORDPRESS_DB_HOST I have given the name of the DB service which is wpdb-service (found in metadata of db service). The other environment variables match with the appropriate env variables of MySQL container pod.

Once these yamls are applied, let us verify it.

kubectl get pods
NAME                             READY   STATUS    RESTARTS   AGE
wordpress-app-556fbb7c44-67frw   1/1     Running   0          2d12h
wordpress-app-556fbb7c44-mkh75   1/1     Running   0          2d12h
wordpress-app-556fbb7c44-t4m4m   1/1     Running   0          2d12h
wordpress-db-d9949b65d-6d2xr     1/1     Running   0          101m
wordpress-db-d9949b65d-jk7sb     1/1     Running   0          101m

kubectl get deployments
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
wordpress-app   3/3     3            3           2d12h
wordpress-db    2/2     2            2           102m

kubectl get service
NAME           TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
app-service    LoadBalancer   10.101.181.255   <pending>     80:30195/TCP   2d12h
kubernetes     ClusterIP      10.96.0.1        <none>        443/TCP        4d1h
wpdb-service   ClusterIP      10.111.171.206   <none>        3306/TCP       25h

We can see that the pods and deployments are up and running. On the service(app-service), I note that the application is running on port 30195.

Now I can access the web application through my browser - by typing any one of the node server name (master or two workers) with the port.

assuming node name is abc.def.com and port as seen above is 30195, type abc.def.com:30195

I am able to access the web application and I am able to see

If the environment variables of the WordPress and MySQL pods don't match appropriately, there will be no connectivity between the two pods and hence we will see the below:

Now coming back, I finished the initial setup or configuration of WordPress and I am trying to login

Voila, I have the issue I was speaking of in the beginning. I am again sent to the same page.

I dug around and I found that among the two pods, it randomly picks one. So, the reason it brought me back was that when I tried to login, it moved to a different pod. Now, had the pods been connected to a storage and mounted it, the actual data accessible by both the pods would be the same.

We need a solution to fix this, which we will look at the next post in the series.

On a lighter note, we would have been able to achieve the same with pods and services, instead of a deployment.

But it doesn't solve the need for using Kubernetes. Any normal container communication using docker would appear something similar.

Delete unrelated files post-use

Preetham — Thu, 23 Jul 2020 01:56:15 +0000

Now, I would say that it's good to clean up or remove unrelated files from the point of execution after its work is done. We do not want to think it is fairly safe to assume it won't cause any problems.

A very common task that has to be done after any plain VM or application VM (from an image) deployment is to perform post-deployment steps.

A general use-case would be to download the post-deployment script from any blob and perform the post-deployment script. We might even need to pass parameters to the post-deployment script. So we might assume that, as parameters need not be passed, the post-deployment script will not cause any problem. There might be many thousand other factors that might want you to think that there is no harm in this. What or who could even possibly run this again. But I have seen a scenario where it was executed again and that scenario cannot be recreated either.

So, Always prevention is better than cure.

So always delete any files(including scripts and configs) or folders after it has done its job brilliantly. At least archive them and store them in a different folder if you feel it might be needed again.

Please do add other similar scenarios or experiences to take note of in production to prevent breakdowns.

Photo Courtesy: https://unsplash.com/photos/jLtXXn7n16k

Kubernetes Services

Preetham — Tue, 21 Jul 2020 02:50:44 +0000

In this tutorial post, we will see why we need service and then see what is a service and the types of services in Kubernetes.

Let us take our scenario where we have a WordPress and MySQL deployments.

Now, if we want to establish connectivity from a Wordpress pod to another MySQL pod, the MySQL pod does not know which pod to connect to. As a hypothetical scenario, we have to establish a connection via IPs and when we establish a connection by hardcoding IPs, we keep giving work to the same MySQL pod and the other pods are not utilized.

Now that takes away the very benefit that deployments give and the pods become mortal again. Let's say a pod with x IP is deleted, the deployment controller will create another pod with y IP. Now, WordPress pod doesn't know which MySQL pod it should connect to as it only knew pod with x IP.

Its a pandemonium. Now the MySQL pods decide that they will get themselves a manager which will be discovered easily by the WordPress Pod and to others if needed by a single static name and load balance the work among them.

Thus services are born. A Kubernetes Service is an abstract way to expose an application running on a set of Pods as a network service. With Kubernetes, you don't need to modify your application to use an unfamiliar service discovery mechanism.
Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods and can load-balance across them.

Now the WordPress pod doesn't need to know about MySQL pod or its IPs. It only needs to know about the name of the service that front-ends the MySQL pods.

You can have many services within the cluster. Kubernetes services can efficiently power a microservice architecture.

Services provide features that are standardized across the cluster(irrespective of the nodes on which the service and pods are running on):

Load balancing
Service discovery between applications (DNS,port-mapping).
Features to support zero-downtime application deployments (on combination with rollingUpdates of deployments).

There are 3 different types of services.

ClusterIP
NodePort
LoadBalancer

ClusterIP

This is the default Service type. It exposes the service on a cluster internal IP. Choosing this type makes the service reachable only from within the cluster.

So except the web or app service, all other services which should not be accessed from the outside like DB use ClusterIP service.

NodePort

A NodePort is a very basic way to get external traffic directly to your service.
NodePort opens a specific port on your node/VM and when that port gets traffic, that traffic is forwarded directly to the service. The service is applied to all nodes in the cluster(master and workers) and hence can be accessed by http://anyNodeName:port.

For a NodePort service, Kubernetes allocates a port from a configured range ( default is 30000 - 32767). We cannot use standard ports like port 80 or 8080. It is possible to define a specific port number(in the range), but you should take care to avoid potential port conflicts.

There are a few limitations like

Only one service per port
You can only use ports 30000-32767.
Dealing with dynamic or changing node/VM IP is difficult.

LoadBalancer

A LoadBalancer is the standard way to expose a service to the internet.

Only in case of public cloud providers like Azure, GCP, AWS, etc, on creating a service of type LoadBalancer, an external LoadBalancer in that public cloud would be created.

In the case of a private cloud, by default, a service of type LoadBalancer would behave like a service of type NodePort. A custom load balancer like NGINX or load balancers applications like DevCental F5 will use the IP and port of the nodes on which the service is running to load balance it.

A service manifest like other kubernetes object manifests have

apiVersion
kind
metadata
spec - selector and ports

.spec

.spec.type

.spec.type specifies the type of service we need to create.
Default is ClusterIP and other acceptable values are NodePort and LoadBalancer.

.spec.selector

Like we have seen in a deployment manifest, .spec.selector must match the .metadata.labels of the pods which have to be load-balanced and catered by this service. This is the link between the service and the pods in the deployment.
The difference between deployment manifest and service manifest selectors is that deployment manifest selector will have .spec.selector.matchLabels and service manifest will have .spec.selector

.spec.ports

Below, ports[i..n] represents the index of the port object

.spec.ports[i..n].port represents the port on which the service is listening to.
.spec.ports[i..n].targetPort represents the port of the pods to connect to.
.spec.ports[i..n].protocol is the protocol to use - default is tcp
.spec.ports[i..n].name is the name of the virtual server (each port)

.spec.ports is an array and we can define multiple ports here.

---
apiVersion: v1
kind: Service
metadata: 
    name: app-service
    labels:
        app: wordpress
        type: app-service
spec:
    selector:
        app: wordpress
        type: app
    type: LoadBalancer
    ports:
        - protocol: TCP
          port: 80
          targetPort: 80
          name: wordpress

In the above example, we created a service of type LoadBalncer and the values in .spec.selector should match with the label of the pod to connect to and port 80 represents that the service is listening to requests on port 80 and targetPort 80 represents that the service is sending traffic to the nodes on port 80.

Kubernetes Deployments - A way to scale pods - a way to immortality

Preetham — Mon, 20 Jul 2020 10:56:41 +0000

Pods are mortal. They are born and when they die, they are not resurrected. A deployment object is how you can give immortality to pods.

Deployments represent a set of multiple, identical Pods and upgrade them in a controlled way, performing a rolling update(update single or a batch of pods, instead of making changes in all the running pods at once).

A deployment runs multiple replicas of your application and automatically replaces any instances that fail or become unresponsive.

Deployment controller provides declarative updates for Pods and it manages pods running on your cluster.

Deployment controller always strives to make the declared desired state the actual state at a controlled rate.

In this way, deployments ensure that one or more instances of your application are available to serve user requests.

A Deployment manifest, like pod manifest needs

apiVersion
kind
metadata - name, labels, annotations and other information.
spec - replicas, deployment strategy, pod template, selection and other details.

.spec

The Pod Template or .spec.template and .spec.selector are the only mandatory fields of the .spec

1).spec.template - mandatory

The .spec.template has the exact same schema as a pod except it does not have an apiVersion or Kind. Only a .spec.template.spec.restartPolicy equal to Always is allowed, which is the default, even if not specified.

spec:
    template:   
        metadata: 
        name: wordpress-db
        labels:
            app: wordpress
            type: db
        spec:
            restartPolicy: Always
            containers:
                - name: wordpress-db-container
                  image: mysql:5.7
                  env:
                    - name: MYSQL_ROOT_PASSWORD
                      value: DEVOPS1

2).spec.selector - mandatory

The .spec.selector field defines how the Deployment finds which Pods to manage. Basically, the .spec.selector.matchLabels will match with the label of the pod to manage, which in this case is .selector.template.metadata.labels. If they do not match, it will be rejected by the API.

3).spec.replicas - optional - default Value (1)

It specifies the number of desired pods.

spec:
    replicas: 3
    selector:
        matchLabels:
            app: wordpress
            type: db
    template:   
        metadata: 
        name: wordpress-db
        labels:
            app: wordpress
            type: db
        spec:
            restartPolicy: Always
            containers:
                - name: wordpress-db-container
                  image: mysql:5.7
                  env:
                    - name: MYSQL_ROOT_PASSWORD
                      value: DEVOPS1

In the above example, the .spec.selector.matchLabels matches the .selector.template.metadata.labels and has a replica of 3 and hence it will always have 3 pods running and if one pod is destroyed, another pod will be created.

4)spec.strategy - optional - default (RollingUpdate)

It specifies the strategy to replace the old pods by new ones. The possible values are RollingUpdate - kill batch or single pods and recreate those but not all and Recreate - all existing Pods are killed before new ones are created

spec:
    replicas: 3
    strategy:
        type: RollingUpdate

Example

apiVersion: apps/v1
kind: Deployment
metadata:
    name: wordpress.db
    labels:
        app: wordpress
        type: db-deployment
spec:
    replicas: 2
    selector:
        matchLabels:
            app: wordpress
            type: db
    template:
        metadata:
            name: wordpress-db
            labels:
                app: wordpress
                type: db
        spec:
            containers:
                - name: wordpress-db-container
                  image: mysql:5.7
                  env:
                  - name: MYSQL_ROOT_PASSWORD
                    value: DEVOPS1
                  ports:
                    - containerPort: 3306
                      name: mysql

Assume that we create another deployment for MySQL with 2 replicas, we have our current setup which we created as below.

We have not actually created the deployment in this post, but we will create a deployment in the forthcoming posts.

These are the basics of deployment which we need to understand to give immortality to the pods (of course pseudo immortality as the entire Kubernetes cluster may go out of commission).

Create Kubernetes pods

Preetham — Fri, 17 Jul 2020 09:34:20 +0000

Let us create two pods in Kubernetes.
Let us create a pod having a WordPress container and another pod having a MySQL container.

Now, why WordPress and MySQL? What would be a better example than setting up the good old Apache webserver with PHP connecting to a MySQL database? Here WordPress is a free and open-source content management system written in PHP and paired with a MySQL or MariaDB database. So instead of creating your PHP from scratch, we are going to create containers from pre-created WordPress images.

What we will have at the end of this tutorial is as below.

Creating pods in Kubernetes is simple. We will use the YAML based approach.

Let us create a file named wordpressDeploy

ment.yaml and we will use vim editor for this.

vim wordpressDeployment.yaml

A blank page would be opened. Press escape and the type i. The page will open in insert mode. Type the below.

apiVersion: v1
kind: Pod
metadata:
        name: wordpress-app
        labels:
                app: wordpress
                type: app
spec:
        containers:
                - name: wordpress-app
                  image: wordpress

Press escape ,then type :wq! and press enter. The page saves.
Here,
. apiVersion represents the version and type of the API we are
going to use to create this Kubernetes object. Any new
feature, beta, alpha versioning is done at the API level
rather than the field or resource level.
. kind represents the type/kind of object you want to create
. metadata represents the data that helps uniquely identify the
object, including a name string, UID, and an optional
namespace. name in metadata is the name of the object you
are going to create. labels in metadata is the
identification used by other objects to select this object. We
will look at how labels and selectors in future posts.
. spec represents the kind of (what) state you desire for the
object. In containers, we are specifying the name and image
of the container we are going to create in this pod.

So when we apply this, we are going to create a pod named WordPress-app with the official WordPress image in dockerhub.

$ kubectl apply -f wordpressDeployment.yaml
pod/wordpress-app created

#check pod status
$ kubectl get pods
NAME            READY   STATUS    RESTARTS   AGE
wordpress-app   1/1     Running   0          21s

When we check the status, we see that the pod is up and running.

Now let us create the MySQL pod and so we will create/open WordPressDb.yaml

apiVersion: v1
kind: Pod
metadata: 
        name: wordpress-db
        labels:
                app: Wordpress
                type: db
spec:
        containers:
                - name: wordpress-db-container
                  image: mysql:5.7

Apply the yaml to create a MySQL pod, created from the official MySQL image.

$ kubectl apply -f wordpressDb.yaml

$ kubectl get pods
NAME            READY   STATUS             RESTARTS   AGE
wordpress-app   1/1     Running            0          26m
wordpress-db    0/1     CrashLoopBackOff   1          22s

We see that the status is having an error named CrashLoopBackOff.

#checking the logs for the pod
$ kubectl logs -p wordpress-db
Error from server: Get https://172.31.38.44:10250/containerLogs/default/wordpress-db/wordpress-db-container?previous=true: dial tcp 172.31.38.44:10250: i/o timeout

#desribing the pods
$ kubectl describe pods wordpress-db
Name:         wordpress-db
Namespace:    default
Priority:     0
Node:         ip-172-31-38-44/172.31.38.44
Start Time:   Thu, 16 Jul 2020 01:39:37 +0000
Labels:       app=wordpress
Annotations:  cni.projectcalico.org/podIP: 10.13.108.2/32
              cni.projectcalico.org/podIPs: 10.13.108.2/32
Status:       Running
IP:           10.13.108.2
IPs:
  IP:  10.13.108.2
Containers:
  wordpress-db-container:
    Container ID:   docker://e6a336394be171103dc0aacc386d3cfb348ef565a9c69969cc3e5d268c1e79ea
    Image:          mysql:5.7
    Image ID:       docker-pullable://mysql@sha256:ea560da3b6f2f3ad79fd76652cb9031407c5112246a6fb5724ea895e95d74032
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 16 Jul 2020 02:00:45 +0000
      Finished:     Thu, 16 Jul 2020 02:00:45 +0000
    Ready:          False
    Restart Count:  9
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-48pkn (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-48pkn:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-48pkn
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From                      Message
  ----     ------     ----                  ----                      -------
  Normal   Scheduled  <unknown>             default-scheduler         Successfully assigned default/wordpress-db to ip-172-31-38-44
  Normal   Pulled     22m (x5 over 23m)     kubelet, ip-172-31-38-44  Container image "mysql:5.7" already present on machine
  Normal   Created    22m (x5 over 23m)     kubelet, ip-172-31-38-44  Created container wordpress-db-container
  Normal   Started    22m (x5 over 23m)     kubelet, ip-172-31-38-44  Started container wordpress-db-container
  Warning  BackOff    3m25s (x96 over 23m)  kubelet, ip-172-31-38-44  Back-off restarting failed container

We still don't understand what the issue is, but we can understand that the container starts, crashes, again restarts.
I am still not sure why it's crashing, but I'll take a look at the official MySQL container page in dockerhub.

I can observe that the environment variable MYSQL_ROOT_PASSWORD is mandatory. So I am going to pass it.

apiVersion: v1
kind: Pod
metadata:
        name: wordpress-db
        labels:
                app: wordpress
                type: db
spec:
        containers:
                - name: wordpress-db-container
                  image: mysql:5.7
                  env:
                          - name: MYSQL_ROOT_PASSWORD
                            value: DEVOPS1

Now let's apply it and observe.

$ kubectl apply -f wordpressDb.yaml
pod/wordpress-app created

$ kubectl get pods
NAME            READY   STATUS    RESTARTS   AGE
wordpress-app   1/1     Running   0          10m
wordpress-db    1/1     Running   0          21s

It works and both the pods are up and running.

What are Kubernetes pods?

Preetham — Fri, 17 Jul 2020 07:07:36 +0000

A pod is the smallest compute unit object you create in Kubernetes, and it's how you group one or more containers. Containers in a pod share the same networking, the same storage, and are scheduled in the same host. A pod would feel and act like a container would in case of single container pods.

Thus A Pod represents a unit of deployment: a single instance of an application in Kubernetes, which might consist of either a single container or a small number of containers that are tightly coupled and that share resources.

Application-specific "logical host"

So you should think of a pod as if it was a representation of a virtual machine, but it actually isn't.

Let us understand what we mean by that.

If an application needs some other applications or components which run in the same VM or(any physical machine), we can call it an application-specific "logical host". Basically, it has every relatively tightly coupled components for the app to run. An example would be having a Redis cache along with the web app in the same VM.

Pod models an application-specific "logical host" - it contains one or more application containers(Eg: Redis and App running in nodejs) which are relatively tightly coupled — in a pre-container world, being executed on the same physical or virtual machine would mean being executed on the same logical host and communicate using localhost.

why do we need multiple containers to run inside a pod? Why can't we put all of these in a single container?

Let us take the example of a web application running on node js and uses a Redis cache.

In any Linux machine, init (short for initialization) is the first process started during booting of the computer system. Init is a daemon process that continues running until the system is shut down.
But, generally, containers are designed to run a single process. It doesn't have an init system like the Linux VM, that manages all processes. So if we run multiple processes inside a container, it is our(the developer's) responsibility to keep both the node js and Redis processes running, manage their logs, handle all IPC signals, and so on. It is not so easy. Hence it is recommended to run only one process per container, and we will be able to monitor this process for logs, and all other management becomes easier.

Hence, with pods, we can run multiple containers together. Thus the pod of containers models an application-specific "logical host".

These shared containers in a pod will have the following advantages:

They communicate with each other using localhost(i.e) in the same network and have the same IP but vary by port.
They share the storage volumes.
They form a cohesive unit of service.

Now, if you can observe, I have changed the single container pod-style given in Kubernetes architecture post, to the current grouped container pod style. Though the previous depiction is also correct, this is an easier, standard way to handle and scale the application. You can also observe that the pod containing apache is a single container pod.

Check out this post for more info on the pod networking: https://dev.to/kanapuli/what-are-kubernetes-pods-anyway-1app

Dependency hell in Linux (centos 7)

Preetham — Fri, 17 Jul 2020 03:08:02 +0000

I needed to update the Kubernetes and docker in three test centOS 7 servers, a master and two slaves, installed using kubeadm. Now, this was my first time using centOS. The server was not maintained for nearly a decade. I checked the status of existing Kubernetes and found that many files were missing. Updating it was also not possible as the required file structure was messed up. It might probably the result of a previously failed installation or upgrade :p .

Hence I decided to uninstall both the docker and Kubernetes and tried to reinstall it and voila, that's when I stumbled into the dependency hell.

I checked the repositories with

yum repolist
repo id                                repo name                                                     status
Server                                 local                                                           3831
base/7/x86_64                    CentOS-7 - Base                                                      10070
azure-cli                              Azure CLI                                                         76
docker-ce-stable/x86_64                Docker CE Stable - x86_64                                         79
jenkins                                Jenkins-stable                                                   106
kubernetes/x86_64                      Kubernetes                                                    14+528
spcclient2.4                           Spacewalk Client 2.4                                              27
treasuredata/7/x86_64                  TreasureData                                                      15

In one worker server, I saw the CentOS-7 - Base repository was also missing.
So I tried installing it by

yum install centos-release
No packages found

Now, among my available repositories, none had centos-release.

So I went in and searched for the Centos-release Download for Linux (rpm)

I went into https://pkgs.org/download/centos-release and selected the link to rpm in CentOS x86_64.

I copied the binary package URL under the downloads section, and tried to install it in the machine.

#yum install -y <copied link>
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/centos-release-7-8.2003.0.el7.centos.x86_64.rpm

This installed it properly and I verified it using yum repolist.

I also came to know that the extra packages repository was needed and I tried to install it.

yum install epel-release

I got the same issue - package not found.

I then found a workaround - use official fedora repository

cd /tmp
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
ls *.rpm

It was installed smoothly and I verified it.

yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: repos-tx.psychz.net
 * epel: d2lzkl7pfhq30w.cloudfront.net
 * extras: centos-distro.1gservers.com
 * remi-safe: repo1.dal.innoscale.net
 * updates: mirror.teklinks.com
 * webtatic: us-east.repo.webtatic.com
repo id                          repo name                                                           status
Server                           local                                                                 3831
base/7/x86_64                    CentOS-7 - Base                                                      10070
docker-ce-stable/x86_64          Docker CE Stable - x86_64                                               79
epel/x86_64                      Extra Packages for Enterprise Linux 7 - x86_64                       13383
extras/7/x86_64                  CentOS-7 - Extras                                                      412
kubernetes/x86_64                Kubernetes                                                          14+528
remi-safe                        Safe Remi's RPM repository for Enterprise Linux 7 - x86_64            3836
spcclient2.4                     Spacewalk Client 2.4                                                    27
updates/7/x86_64                 CentOS-7 - Updates                                                     900
webtatic/x86_64                  Webtatic Repository EL7 - x86_64                                       725
repolist: 33277

Now, I tried installing docker and It went smoothly in the machines where I manually installed the base release repository.

But in my master machine, I still faced the dependency issue. It even said that you have installed x.1.23 and the required is x.1.23_noarch.
But when I checked the repolist, it was the same in all 3 machines.

I tried many different approaches. I even manually installed the dependencies in the same way I installed the Centos-release package. That was daunting, as the dependency went in loop, parent package needed 2 child packages and the child packages needed 2,3,4 etc child packages and so on.

After checking, even after

yum update

the actual centos-release base repository was not updated.

checking the available repos

ls  /etc/yum.repos.d
CentOS-Base.repo                       remi-glpi92.repo
CentOS-CR.repo                         remi-glpi93.repo
CentOS-Debuginfo.repo                  remi-glpi94.repo
CentOS-Media.repo                      remi-modular.repo
CentOS-Sources.repo                    remi-php54.repo
CentOS-Vault.repo                      remi-php70.repo
CentOS-fasttrack.repo                  remi-php71.repo
CentOS-x86_64-kernel.repo              remi-php72.repo
docker-ce.repo                         remi-php73.repo
epel-testing.repo                      remi-php74.repo
epel.repo                              remi-safe.repo
jkastner-dnf-plugins-core-epel-7.repo  remi.repo
kubernetes.repo                        spcclnt24_7.repo_bkp
local.repo                             vault.centos.org_centos_7.4.1708_extras_x86_64_.repo
mesosphere.repo                        webtatic-archive.repo
old                                    webtatic-testing.repo
remi-glpi91.repo                       webtatic.repo

I checked the contents of CentOS-Base.repo

cat /etc/yum.repos.d/CentOS-Base.repo

And voila, someone, manually edited this file.

yum clean packages Eliminate any cached packages from the system and hence I ran it.

yum clean

I manually removed the base repository files in CentOS-Base.repo /etc/yum.repos.d

rm CentOS-Base.repo

I installed the Centos-release manually in the master machine also once again, and I was able to install the docker and kubeadm.

Finally, the dependency hell was resolved. My lesson here was to

make sure the necessary repositories are added
check the repository file for issue if it still persists, and if it is an error, remove and reinstall it.
check if it solves the issue.

If this solves it, it is great, else, keeeeeep diggin!!!

Set up a Kubernetes master-slave architecture using kubeadm

Preetham — Wed, 15 Jul 2020 03:21:22 +0000

We will go into the detail of how to install Kubernetes using kubeadm. Kubernetes can be installed and set up step by step on your own or we can use multiple installation tools or stacks available. We are going to use kubeadm.

According to the kubeadm source, Kubeadm is a tool built to provide kubeadm init and kubeadm join as best-practice “fast paths” for creating Kubernetes clusters.

kubeadm performs the actions necessary to get a minimum viable cluster up and running. By design, it cares only about bootstrapping, not about provisioning machines.

We can use kubeadm for creating a production-grade Kubernetes environment.

We will consider building a Kubernetes setup with one master node and 2 worker nodes.

Let us assume that we have three Ubuntu Linux machines named master, worker1, and worker1 in the same network. For practice purposes, you can create 3 VMS in VirtualBox or you can create 3 VMs in the cloud. The VMs will be accessible from each other. As the name suggests, we will add the necessary configuration in the master machine to make it a Kubernetes master node, and connect the worker1 and worker2 to it.

In all the 3 machines do the following. Follow only the ubuntu approach and follow the centos approach only when you have centos installed instead of ubuntu.

Step 1: Installing Docker as the container runtime Interface

Installation in Ubuntu

#update the repository
sudo apt-get update

#uninstall any old Docker software
sudo apt-get remove docker docker-engine docker.io

#Install docker
sudo apt install docker.io

#Start and automate docker to start at run time
sudo systemctl start docker
sudo systemctl enable docker

#verify docker installation
docker container ls
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS               NAMES

Installation in CentOS 7(only when required)

#removing existing docker
sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

sudo yum remove docker*

#adding repository
sudo yum install -y yum-utils
sudo yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

#install docker engine
sudo yum install docker-ce docker-ce-cli containerd.iosystemctl enable docker -y

#Start and automate docker to start at run time
sudo systemctl start docker
sudo systemctl enable docker

#verify docker installation
docker container ls
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS               NAMES

Kubeadm will by default use docker as the container runtime interface. In case a machine has both docker and other container runtimes like contained, docker takes precedence. If you don't specify a container runtime interface, kubeadm will automatically search for the installed CRI by scanning through the default Linux domain sockets.
Runtime Path to linux domain socket

Docker **/var/run/docker.sock**
contained **/run/containerd/containerd.sock**
CRI-O **/var/run/crio/crio.sock**

Step 2: Installing kubeadm tool

Installation in Ubuntu

#add the required repository for kubeadm
sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

#update the repository
$ sudo apt-get update

#installing kubelet, kubeadm and kubectl
sudo apt-get install -y kubelet kubeadm kubectl

#setting apt-mark
sudo apt-mark hold kubelet kubeadm kubectl

apt-mark will change whether a package has been marked as being automatically installed. Hold is used to mark a package as held back, which will prevent the package from being automatically installed, upgraded, or removed.

Restart the kubelet if required

systemctl daemon-reload
systemctl restart kubelet

Installation in CentOS 7(only when required)

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

# Set SELinux in permissive mode (effectively disabling it)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

systemctl enable --now kubelet

Perform the following step only in the master node

Step 3 Initializing the control plane or making the node as master

kubeadm init will initialize this machine to make it as master.
kubeadm init first runs a series of prechecks to ensure that the machine is ready to run Kubernetes. These prechecks expose warnings and exit on errors. kubeadm init then downloads and installs the cluster control plane components. This may take several minutes.

We have to take care that the Pod network must not overlap with any of the host networks: you are likely to see problems if there is any overlap. We will specify the private cidr for the pods to be created.

$ kubeadm init --pod-network-cidr 10.13.0.0/16

In case you are not actually creating the production environment and your master machine has only one CPU(minimum recommended CPU is 2), if an error occurs due to preflight check of CPU, let us run the below:

#swapoff -a (if swap issue is seen - only in testing or practice)
kubeadm init --ignore-preflight-errors=NumCPU --pod-network-cidr 10.13.0.0/16

The output should look like

[init] Using Kubernetes version: vX.Y.Z
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kubeadm-cp localhost] and IPs [10.138.0.4 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kubeadm-cp localhost] and IPs [10.138.0.4 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubeadm-cp kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.138.0.4]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 31.501735 seconds
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-X.Y" in namespace kube-system with the configuration for the kubelets in the cluster
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "kubeadm-cp" as an annotation
[mark-control-plane] Marking the node kubeadm-cp as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node kubeadm-cp as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: <token>
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a Pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  /docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

Now as seen in the output above, we need to run the below commands as a normal user to use the cluster.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

if you are the root user, run

export KUBECONFIG=/etc/kubernetes/admin.conf

Now the machine is initialized as master.

If we check the nodes, we will see only the master node.

kubectl get nodes
NAME               STATUS   ROLES    AGE   VERSION
ip-172-31-45-165   Ready    master   48s   v1.18.5

Step 5 Installing a Pod network add-on

You must deploy a Container Network Interface (CNI) based Pod network add-on so that your Pods can communicate with each other. Cluster DNS (CoreDNS) will not start up before a network is installed.

We will use **Calico** as our CNI tool.

Calico is a networking and network policy provider. Calico supports a flexible set of networking options so you can choose the most efficient option for your situation, including non-overlay and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts, pods, and (if using Istio & Envoy) applications at the service mesh layer.

Calico will automatically detect which IP address range to use for pod IPs based on the value provided via the --pod-network-cidr flag or via kubeadm's configuration.

Run the below

kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml

Now calico is installed as our CNI.
We can verify if the coreDNS and calico are running perfectly fine by running the command

kubectl get pods --all-namespaces
NAMESPACE     NAME                                                    READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-76d4774d89-5rz9t                1/1     Running   0          70s
kube-system   calico-node-lj8cp                                       1/1     Running   0          70s
kube-system   coredns-66bff467f8-h7k7r                                1/1     Running   0          2m23s
kube-system   coredns-66bff467f8-rqdcj                                1/1     Running   0          2m23s
kube-system   etcd-dlesptrnapp3v.jdadelivers.com                      1/1     Running   0          2m33s
kube-system   kube-apiserver-dlesptrnapp3v.jdadelivers.com            1/1     Running   0          2m33s
kube-system   kube-controller-manager-dlesptrnapp3v.jdadelivers.com   1/1     Running   0          2m33s
kube-system   kube-proxy-4b4pj                                        1/1     Running   0          2m23s
kube-system   kube-scheduler-dlesptrnapp3v.jdadelivers.com            1/1     Running   0          2m33s

Run step 5 only in the worker nodes and run it in both the worker nodes.

Step 5 Join the worker nodes to the master node

Now in the worker nodes, run the kubeadm join common which you obtained when running the kubeadm init in the master machine/ control-plane-host.

kubeadm join 172.31.45.165:6443 --token jjyon1.bfztgwfuxavldki2 --discovery-token-ca-cert-hash sha256:29b6281be8bd08fd2d993e322f8037491f32006d925a4cd96cd1568563d12e5f

You will see an output like below.

W0715 03:14:44.854420   13222 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

Run the command as sudo

sudo kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

step 6 verifying if the nodes are added in the master/control-plane machine

kubectl get nodes
NAME               STATUS   ROLES    AGE     VERSION
ip-172-31-33-64    Ready    <none>   119s    v1.18.5
ip-172-31-38-44    Ready    <none>   116s    v1.18.5
ip-172-31-45-165   Ready    master   5m30s   v1.18.5

Thus we have set up a Kubernetes master-slave architecture using kubeadm.

Kubernetes Architechture

Preetham — Tue, 14 Jul 2020 13:59:12 +0000

Before we enter into the practicals, we will finish the theory first (i.e.) understanding the terms and components involved.

Let us start with the nodes.

A node is a machine, physical or virtual, on which the Kubernetes is installed. A node is a worker machine and this is where the containers will be launched by Kubernetes. It is also known as minions.

Now, what if the node machine on which the application is running goes down.

Obviously, our application goes down and we don't want that to happen. That is why we have clusters.

A cluster is a set of nodes grouped together. Hence, even if one node fails, you have your application still accessible from other nodes. Also, having multiple nodes helps in sharing the load.

We have nodes and clusters, but,

who will be the orchestrator to manage these clusters and nodes?
Where is the information about the members of the clusters stored?
How are the nodes monitored?
When a node fails, how do you move the workload of the failed node to another worker node?

This is where the master or the orchestrator comes into the picture. The master is another node with Kubernetes installed in it, and is configured as Master.

The master watches over the nodes in the cluster and is responsible for the actual orchestration of containers on the worker nodes.

When Kubernetes is installed in a system, the following components are installed along with Kubernetes.

API Server: The API server acts as the front-end for Kubernetes. The CLI, users, management devices all talk to the API server to interact with the Kubernetes cluster.
ETCD Key store: ETCD is a distributed reliable key-value store used by Kubernetes to store all data required to manage the key-value store. When you have multiple nodes and multiple masters in your cluster, ETCD stores all the information on all nodes in the cluster in a distributed manner(sync). ETCD is responsible for implementing locks within the cluster to ensure that there are no conflicts between the masters .
Scheduler: It is responsible for distributing work or containers across multiple nodes. It looks for newly created containers and assign them to nodes.
Controllers: They are the brains behind an orchestration and are responsible for identifying and responding when nodes, containers, or endpoints(Services) goes down. The controllers make the decisions to bring up new containers in such cases.
Container Runtime: It is the underlying software used to run the containers. We use docker for container runtime.
Kubelet: Kubelet is the agent that runs on each node in the cluster. The agent is responsible for making sure that the containers are running on the nodes as expected.

Now we saw the different components involved in kubernetes. But what makes one a master and others a node.

Master is the node, which has the kube-api server and that's what makes it a master. It also has the ETCD, controller and scheduler components.
Worker node or simply node is the one which has the container runtime and the Kubelet.

Forem: Preetham

Serverless Golang REST API with AWS Lambda

Table Of Contents

Setting up the aws cli

Creating Simple Golang REST API

1 .GO Installation

2 .Creating the Folder structure

3 .Initiate the project

4 .Create the golang file and install the packages

4.1 .Code and Package Explanation

Package github.com/aws/aws-lambda-go/lambda

Package github.com/aws/aws-lambda-go/events

Package github.com/valyala/fastjson

5. Creating an executable and zipping

Creating and deploying an AWS Lambda function

1. IAM Role

2. Deploy the golang build into Lambda function

Setting up the HTTPS API in AWS API Gateway

Deploying the HTTP API created

What is an Environment?

Can I go with Pop OS rather than ubuntu?

Kubernetes Services and Deployments coming together

db-service.yaml

db-deployment.yaml

app.yaml

Delete unrelated files post-use

Kubernetes Services

ClusterIP

NodePort

LoadBalancer

.spec

.spec.type

.spec.selector

.spec.ports

Kubernetes Deployments - A way to scale pods - a way to immortality

.spec

1).spec.template - mandatory

2).spec.selector - mandatory

3).spec.replicas - optional - default Value (1)

4)spec.strategy - optional - default (RollingUpdate)

Create Kubernetes pods

What are Kubernetes pods?

Application-specific "logical host"

why do we need multiple containers to run inside a pod? Why can't we put all of these in a single container?

Dependency hell in Linux (centos 7)

Set up a Kubernetes master-slave architecture using kubeadm

Step 1: Installing Docker as the container runtime Interface

Step 2: Installing kubeadm tool

Step 3 Initializing the control plane or making the node as master

Step 5 Installing a Pod network add-on

Step 5 Join the worker nodes to the master node

step 6 verifying if the nodes are added in the master/control-plane machine

Kubernetes Architechture