Forem: Muhammad Hamza Hameed

Go Workspaces (multi-module)

Muhammad Hamza Hameed — Mon, 21 Nov 2022 07:48:39 +0000

Workspace Deserves a Spotlight

A new feature in Go 1.18 go workspace mode which makes it simple to work with multiple modules. With multi-module workspaces, you can tell the Go command that you’re writing code in multiple modules at the same time and easily build and run code in those modules.

Why Workspace?

In short, the workspace makes the multiple-module dependency and switch easier in local development.

You must have encountered such an issue: The multiple modules in the repo reference each other directly. But your local changes to a module A fail to reflect in the module B that depends on A. What we take to overcome this is

Push the modified code of module A to GitHub, and pull the update in module B.
Replace the module B go.mod file’s dependency on A to a local absolute path.

   replace github.com/hamza/a v0.1 => /Users/hamza/workspace/repo/module-a

Both solutions have their own cons, which bring us pain in developing in multi-module mode somehow.

The first method is not designed in the way you like it if you try to avoid releasing new versions before more tests. Besides, after you publish it on Github, more releases and more versions are required for fixing the bug once they occur.

For the second method, you should always remind yourself to revert the changes in the local go.mod file before submitting; otherwise, you might drag your co-developers into trouble.

In light of this, Go released the go worspace mode in its version 1.18.

Set Up Workspace

Let’s test whether the workspace can ease the above-mentioned issues. Make sure that you’ve installed Go at Go 1.18 or later using the links at go.dev/dl.

Create a module for your code

To begin, create a module for the code you’ll write.

Open a command prompt and create a directory for your code called workspace.

   $ mkdir workspace
   $ cd workspace

Initialize the module

In this example, I will create a new module hello that will depend on the golang.org/x/example module.

   $ mkdir hello
   $ cd hello
   $ go mod init example.com/hello
   go: creating new go.mod: module example.com/hello

Add a dependency on the golang.org/x/example module by using go get.

   $ go get golang.org/x/example

Create hello.go in the hello directory with the following contents:

   package main

   import (
    "fmt"
    "golang.org/x/example/stringutil"
   )

   func main() {
     fmt.Println(stringutil.Reverse("Hello"))
   }

Now, run the hello program:

   $ go run example.com/hello
   olleH

Create the workspace

In this step, we’ll create a go.work file to specify a workspace with the module.

Initialize the workspace

In the workspace directory, run:

$ go work init ./hello

The go work init command tells go to create a go.work file for a workspace containing the modules in the ./hello directory.

The go command produces a go.work file that looks like this:

go 1.18

use ./hello

The go.work file has similar syntax to go.mod.

The go directive tells Go which version of Go the file should be interpreted with. It’s similar to the go directive in the go.mod file.

The use directive tells Go that the module in the hello directory should be main modules when doing a build.

So in any subdirectory of workspace the module will be active.

Run the program in the workspace directory

In the workspace directory, run:

$ go run example.com/hello
olleH

The Go command includes all the modules in the workspace as main modules. This allows us to refer to a package in the module, even outside the module. Running the go run command outside the module or the workspace would result in an error because the go command wouldn’t know which modules to use.

Next, we’ll add a local copy of the golang.org/x/example module to the workspace. We’ll then add a new function to the stringutil package that we can use instead of Reverse.

Download and modify the golang.org/x/example module

Clone the repository

From the workspace directory, run the git command to clone the repository:

   $ git clone https://go.googlesource.com/example

Add the module to the workspace

   $ go work use ./example

The go work use command adds a new module to the go.work file. It will now look like this:

   go 1.18

   use (
     ./hello
     ./example
   )

The module now includes both the example.com/hello module and the golang.org/x/example module.

Add the new function.

We’ll add a new function to uppercase a string to the golang.org/x/example/stringutil package.

Create a new file named toupper.go in the workspace/example/stringutil directory containing the following contents:

   package stringutil

   import "strings"

   // ToUpper uppercases argument string.
   func ToUpper(s string) string {
     return strings.ToUpper(s)
   }

Run the code in the workspace

References

Get the full source code from this github repository.

GoCD - Introduction (A tool that use for continuous Integration and continuous delivery CI/CD)

Muhammad Hamza Hameed — Mon, 21 Nov 2022 07:40:34 +0000

GoCD is an open-source tool which is used in software development to help teams and organizations automate the continuous delivery (CD) of software. It supports automating the entire build-test-release process from code check-in to deployment.

GoCD system requirements

Client (browser) requirements

GoCD supports the latest versions of the following browsers

Google Chrome
Mozilla Firefox
Microsoft Edge
Apple Safari

GoCD server requirements

Hardware

RAM - minimum 1GB, 2GB recommended
CPU - minimum 2 cores, 2GHz
Disk - minimum 1GB free space

Supported operating systems

Windows - Windows Server 2012, Windows Server 2016, Windows 8 and Windows 10
Mac OSX - 10.7 (Lion) and above with Intel processor
Debian - Debian 8.0 and above
CentOS/RedHat - CentOS/RedHat version 6.0 and above
Ubuntu - Ubuntu 14 and above
Alpine Linux - Alpine Linux 3.6 and above

Installation

GoCD consists of two installable components. The GoCD Server and one or more GoCD Agents. They have a one-to-many relationship in that many GoCD agents can connect to one GoCD Server. To do any real work, you need at least one agent, since agents are the real builders or work executors in the system.

Make sure your system has installed Java / JRE version 13 (for releases 21.1.0 and higher)

Follow GoCD Server installation instructions to install server
Follow GoCD Agent installation instructions to install agent

Follow the the given links instructions, select your operating system and install GoCD Server and (at least one) Agent and run server and angent according to the instructions.

Whatever operating systems you install the GoCD server and (at least one) GoCD agent on, the default ports used by the server are 8153 (HTTP port) and 8154 (HTTPS port). So, after installation you should be able to access either http://localhost:8153 or http://your-server-installation-hostname:8153 and you should see a screen such as,

Once the agent is started, switching to the agents tab by clicking on the Agents link in the header should take you to a screen where the agent shows up and is idle. Like this:

An agent or agents can be installed on any node, and not necessarily the node that the GoCD Server is installed on. The only requirements are that ports 8153 and 8154 of the GoCD Server are accessible from the node that the agents are installed on.

Concepts in GoCD

Server and agents

In the GoCD ecosystem, the server is the one that controls everything. It provides the user interface to users of the system and provides work for the agents to do. The agents are the ones that do any work (run commands, do deployments, etc) that is configured by the users or administrators of the system.

The server does not do any user-specified work on its own. It will not run any commands or do deployments. That is the reason you need a GoCD Server and at least one GoCD Agent installed before you proceed.

Pipelines and materials

A pipeline, in GoCD, is a representation of workflow or a part of a workflow. For instance, if you are trying to automatically run tests, build an installer and then deploy an application to a test environment, then those steps can be modeled as a pipeline. GoCD provides different modeling constructs within a pipeline, such as stages, jobs and tasks. We will see these in more detail soon. For the purpose of this part of the guide, you need to know that a pipeline can be configured to run a command or set of commands.

Another equally important concept is that of a material. A material is a cause for a pipeline to trigger or to start doing what it is configured to do. Typically, a material is a source control repository (like Git, Subversion, etc) and any new commit to the repository is a cause for the pipeline to trigger. A pipeline needs to have at least one material and can have as many materials of different kinds as you want.

The concept of a pipeline is extremely central to Continuous Delivery. Together with the concepts of stages, jobs and tasks, GoCD provides important modeling blocks which allow you to build up very complex workflows, and get feedback quicker.

Now, back at the Add pipeline screen, you can choose your own material here, you can change Material Type to Git and provide the URL for your git repository in the URL textbox. If you now click on the Test Connection button, provide github username and github personal access token. it should tell you everything is OK.

Scroll down and let's provide the pipeline a name, without spaces, and ignore the Pipeline Group field for now.

This step assumes that you have git installed on the GoCD Server and Agent. Like git, any other commands you need for running your scripts need to be installed on the GoCD Agent nodes.
If everything went well, press Next to be taken to step 3, which deals with stages, jobs and tasks.

Stage details, jobs, and tasks

let's provide the stage a name, without spaces. Scroll down you will see the job and task field. Provide the job a name, without spaces and define your task bellow in the task field.

If all went well, you just created your first pipeline! Leaving you in a screen similar to the one shown below. If you give it a minute, you'll see your pipeline building (yellow) and then finish successfully (green)

The pipeline is building

The pipeline is finished successfully.

Clicking on the bright green bar will show you information about the stage

and then clicking on the job (terminal) icon will take you to the actual task and show you what it did

Scrolling up a bit, you can see it print out the environment variables for the task and the details of the agent this task ran on

Pipeline dependency material

Instead of using a source code repository on GitHub as a material, it is possible to use a stage of any pipeline as a material for another pipeline. This seemingly simple feature provides a lot of power. This allows pipelines to be chained together, allowing very complex workflows to be modeled well and is a basis for the more advanced features in GoCD, such as Value Stream Map (VSM) and fan-in/fan-out.

A pipeline dependency material links pipelines together. The pipeline which has the material which points to another pipeline is called the Downstream Pipeline. The actual pipeline which is the dependency material is called the Upstream Pipeline. Even though it is called a Pipeline Dependency the real dependency is to a stage in an upstream pipeline.

As soon as that stage completes successfully, the first stage of each of the configured downstream pipelines triggers. If the stage does not complete successfully, its configured downstream pipelines do not trigger.

Let's see how we can get a pipeline dependency to work. Clicking on Admin and then Pipelines takes you to the Pipeline Configuration page which lists all the pipelines in the system. It looks like this

we can mange previous piplines or we can add more pipelines here. Lets make another pipeline.

scroll down and chose some task in job and task field and click on Run

This is how it will look on the Pipelines list (dashboard) when finished. That doesn't look very different. But, this allows for some powerful features such as fan-in, fan-out and the ValueStream Map (VSM) Click on the label of the Demo_PipeLine (the part highlightedin the image above) VSM for a sneak peek at a small Value Stream Map.

You now know how to chain pipelines together. Let's now learn about artifacts, which are very useful when used with pipeline dependencies.

Artifacts

An artifact in GoCD is a file or directory which is most often produced during the run of a pipeline. Every job in a pipeline can be configured to publish its own set of artifacts and GoCD will ensure that these artifacts are moved from the Agent where it is created, to the Server and stored there, so that it can be retrieved at any time.

Typically, artifacts are created during a job run, by one of the tasks. Some examples of artifacts are: Test reports, coverage reports, installers, documentation, meta information about the build process itself and anything else that needs to be stored after a pipeline has finished.

These artifacts, published by a job, can be fetched and used by any downstream pipeline or any stage after the one that produced the artifact in the same pipeline, using a special task called a Fetch Artifact task. As jobs are independent of each other. So, a job in the same stage as another job that produced an artifact cannot use that artifact. It needs to be used in a stage after that one.

Let's see how to publish an artifact. In the upstream pipeline, Demo_Pipeline, let's first declare an artifact. The build script used throughout this guide creates a file called my-artifact.html after it finishes. We can use that as the artifact for this example.

Go to Admin -> Pipelines and select the stage, demo_stage and then the job, build.
Click on the Artifacts tab in the job.
Enter my-artifact.html as the artifact source.
Leave the Destination box empty for now.

Since we have chosen to leave the Destination empty, and that means that the artifact will be accessible at the root, by its own name. We have also chosen the type as Build Artifact, to signify that it is not a Test Artifact. Marking it as a Test Artifact means that GoCD will try and parse the file as a test report and if it can, it will use it in the test aggregation reporting it does. Typically, you'll want to use Build Artifact for installers, binaries, documentation, etc.

After saving this change, retrigger Demo_Pipeline by going to the pipeline dashboard and clicking on the play button against it. Once it is finished, going to the Artifacts tab of the pipeline run shows you the artifact for that run. Every run of that pipeline, from now on, will have that artifact.

Clicking on it will show you its contents

Fetching and using artifacts

We can now use this artifact in any downstream pipeline, or any subsequent stage of the same pipeline. Let's fetch this artifact in the pipeline My2ndPipeline and display it as a part of the output. To do this, we go to the task configuration section of the echo job inside the Demo_Stage stage of My2ndPipeLine pipeline, and add a Fetch Artifact task.

Once you provide all the details and click save, you can move the Fetch Artifact task above, so that it is done first. Then, for this demonstration, let us display the fetched file in a Custom Command task.

After it is moved up, you can edit the Custom Command task to output the contents of the file (for instance). If you are running this on Windows, use type instead of cat

Running My2ndPipeLine now will show you the contents of the file created in Demo_Pipeline and fetched as an artifact. The importance of fetching an artifact this way is that GoCD will ensure that the correct version of the artifact will be fetched and provided to the job in My2ndPipeLine.

If more instances of Demo_Pipeline ran (because it is fast) while My2ndPipeLine has run fewer times, GoCD will ensure that every time My2ndPipeLine runs, the correct (and compatible) version of the artifact is fetched and used. When you now check the output of the pipeline, you should see something like this

Hope this helped you.

References

GoCD - An Introduction

Installation GoCD

Concepts in GoCD

Value Stream Map

Serverless application in golang with AWS (Lambda Service)

Muhammad Hamza Hameed — Mon, 21 Nov 2022 07:30:26 +0000

What is Serverless Architecture?

Serverless architecture is a way to build and run applications and services without having to manage infrastructure. Your application still runs on servers, but all the server management is done by AWS.

AWS Lambda

AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software as a service (SaaS) applications and only pay for what you use.

Advantages of Using AWS Lambda

Run code without provisioning or managing infrastructure. Simply write and upload code as a .zip file or container image.
Automatically respond to code execution requests at any scale, from a dozen events per day to hundreds of thousands per second.
Save costs by paying only for computing time you use — by per millisecond — instead of provisioning infrastructure upfront for peak capacity.
Optimize code execution time and performance with the right function memory size. Respond to high demand in double-digit milliseconds with Provisioned Concurrency.

Serverless Application in Go with AWS

Prerequisites

You’ll need an AWS account for this. If you don’t yet have one, sign up for a free account here.
For building and deploying your functions, you’ll be using the Serverless Framework, which is the most widely used tool for the job. Assuming you have a recent version of Node.js installed, you can install the Serverless CLI with the following npm command:

  $ npm install -g serverless

Once you have the Serverless CLI installed, you must configure it to use the AWS access keys of your account:

  $ serverless config credentials --provider aws --key <access key ID> --secret <secret access key>

You can get the access key and secret key from the My Security Credentials option. Use Create New Access Key if you do not have one already.

If you don’t have Go installed yet, you can either download an installer from the official website or use your favorite package manager to install it

The Lambda-Time Function

Now that you have everything you need, let’s create a new project using Go modules for your function. For example, let’s name the module lambda-time. In a new, empty directory, initialize the Go module, and install the AWS Lambda for Go library:

$ go mod init lambda-time
$ go get github.com/aws/aws-lambda-go

After this, you can proceed to create a main.go file that implements your handler function and starts the process:

package main

import (
    "context"
    "encoding/json"
    "log"
    "time"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
)

func main() {
    lambda.Start(handleRequest)
}

func handleRequest(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    resp := &response{
        UTC: time.Now().UTC(),
    }

    body, err := json.Marshal(resp)
    if err != nil {
        return events.APIGatewayProxyResponse{}, err
    }

    return events.APIGatewayProxyResponse{Body: string(body), StatusCode: 200}, nil
}

type response struct {
    UTC time.Time `json:"utc"`
}

This previous code can be broken into a few simple steps:

Define a response struct that supports JSON serialization and defines the HTTP response body of a successful invocation of your AWS Lambda function.
Create a request handler function, which creates a response struct containing the current time in UTC and then proceeds to serialize it as JSON. In case the serialization fails, you return the error; if everything goes well, you respond with your serialized JSON as the response body and a status code of 200.
Register your handler function in the main function using the AWS Lambda for Go library.

The Handler Function

It’s worth taking some time to understand how the handler function works. While there are multiple valid handler signatures, the one you used is the complete one. The context argument provides information on the invoked function, its environment and the deadline of the invocation. Returning an error value from the handler function signals that the invocation failed and automatically logs the value of the error.

That leaves the request and response structs in your handler function signature. Lambda functions are invoked either by AWS services or by using an AWS SDK (e.g., from another Lambda function). Data passed in and out of a Lambda function is in JSON format. In your case, the AWS Lambda for Go library automatically handles the serialization and deserialization between JSON and Go values.

When calling Lambda functions using the AWS SDK, the structure of the input and output JSON data is up to the developer. For AWS Lambda functions invoked by AWS services, the data structure depends on the invoking service. Amazon API Gateway is the service that triggers Lambda functions in response to HTTP calls. For API Gateway, this means the request is always of type events.APIGatewayProxyRequest and the response will always be of type events.APIGatewayProxyResponse.

The AWS Lambda for Go library contains the data definitions for each AWS service that can invoke Lambda functions.

Integration with Golang Routing Libraries

If you are using some package for routing, there is a package available that may be used to connect their routing library with lambda.
github.com/awslabs/aws-lambda-go-api-proxy It currently supports the following routing libraries.

chi
gin
gorillamux
fiber
echo
iris
negroni

Lets see the chi example for routing:

package main

import (
    "context"
    "encoding/json"
    "log"
    "time"

    chiadapter "github.com/awslabs/aws-lambda-go-api-proxy/chi"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
)

func main() {
    lambda.Start(handleRequests)
}

func handleRequests(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    router := chi.NewRouter()

    router.Get("/", func (w http.ResponseWriter, _ *http.Request) {
        resp := &response{
            UTC: time.Now().UTC(),
        }

        body, err := json.Marshal(resp)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            return
        }

        w.Header().Set("Content-Type", "application/json; charset=utf-8")
        w.WriteHeader(http.StatusOK)
        w.Write(body)
    })

    return chiadapter.New(router).ProxyWithContext(ctx, req)
}

type response struct {
    UTC time.Time `json:"utc"`
}

Deployment

Your function is now ready and you can proceed by deploying it with the Serverless Framework. Your application deployed using the Serverless framework based on the serverless.yml configuration file.

If you are not familiar with the .yml syntax, you can read this serverless.yml guide.

We must first create a serverless.yml file that defines what we are deploying.

service: lambda-time
provider:
  name: aws
  runtime: go1.x
package:
  exclude:
    - ./**
  include:
    - ./bin/**
functions:
  lambda-time:
    handler: bin/lambda-time
    events:
      - http:
          path: /
          method: get

Here you name both your service and the function lambda-time, but the service could instead contain multiple functions with different names. You also need to configure your API Gateway by specifying that the function responds to HTTP events of a particular HTTP method and at a given request path.

Next up, build the code as an x86-64 Linux executable, and deploy it:

$ GOOS=linux GOARCH=amd64  go build -o bin/lambda-time .
$ serverless deploy

Once finished, the command prints the URL for the endpoint.

Open it, and make sure it responds with the current time.

Now you can visit your aws account. You can see your application at AWS services > Lambda > Applications

In your application you can see your ApiGatewayRestApi , Lambda Function and ServerlessDeploymentBucket where your api is deployed.

In ApiGatewayRestApi there are api paths Method Execution where you can run Test.

Stage where you can create multiple stages, By Default we have a dev stage.

In Lambda Functions you can see your lambda functions and in your function there is runtime settings where your handler file and runtime language already selected.

Let's visit Cloud Watch where you can see your logs:

References

See full source-code at this github repository.

Background process in Lambda using SQS

Muhammad Hamza Hameed — Mon, 21 Nov 2022 07:19:34 +0000

AWS Lambda

The Problem

We want some work to process at background (asynchronously) in Lambda even if the Lambda function returns. But when the lambda function returns, all the background services automatically stop.

The Solution

So here we are going to use Amazon SQS for this purpose. We will send a message to a queue, and it will trigger another lambda function that will run in the background.

What is Amazon SQS?

Amazon Simple Queue Service (SQS) is a managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. Using SQS, you can send, store, and receive messages asynchronously between software components at any volume, without losing messages or requiring other services to be available.

SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.

Golang Example to Run Background Processes in Lambda Using SQS

Prerequisites

For building and deploying your functions, you’ll be using the Serverless Framework, which is the most widely used tool for the job. Assuming you have a recent version of Node.js installed, you can install the Serverless CLI with the following npm command

  $ npm install -g serverless

Once you have the Serverless CLI installed, you must configure it to use the AWS access keys of your account

  $ serverless config credentials --provider aws --key <access key ID> --secret <secret access key>

You can get the access key and secret key from the My Security Credentials option. Use Create New Access Key if you do not have one already

If you don’t have Go installed yet, you can either download an installer from the official website or use your favorite package manager to install it

Create a Messaging Queue

Let's create a queue. Go to Simple Queue Service (SQS) in your AWS account. You will see the following interface there

Create queue from here and select type FIFO and let's name it test.fifo as we want to process exactly once message, in the exact order that they are sent. you can also use the standard queue according to your need. And we'll keep all other settings to default for now.

And now let's code.

Sending Message on the SQS Queue

Now that you have everything you need, let’s install the AWS SDK for Go library.

$ go get github.com/aws/aws-sdk-go

After this, we can proceed to write the code to send a message on our SQS queue to trigger the lambda function that will perform the background task.

package handlers

import (
  "github.com/aws/aws-sdk-go/aws"
  "github.com/aws/aws-sdk-go/aws/credentials"
  "github.com/aws/aws-sdk-go/aws/session"
  "github.com/aws/aws-sdk-go/service/sqs"
  "os"

  "encoding/json"
  "log"
  "net/http"
)


func yourHttpHandlerFunction(w http.ResponseWrite, r *http.Request) {
  // tasks you need to perform before the background task
  ...

  // sending details to other lambda function to perform the background task
  message := SqsTriggerMessage{
    Message: "You can add different fields here according to your data requirements in the background task.",
  }

  err := sendMessage(message)
  if err != nil {
    log.Println(err)
    w.WriteHeader(http.StatusInternalServerError)
    return
  }

  // other tasks you need to perform
  ...

  // sending user an immediate response
  w.WriteHeader(http.StatusOK)
}

type SqsTriggerMessage struct {
  Message string
}

func sendMessage(data interface{}) error {
  b, err := json.Marshal(data)
  if err != nil {
    return err
  }

  svc := sqs.New(
    session.Must(
      session.NewSession(
        &aws.Config{
          Credentials: credentials.NewStaticCredentials(os.Getenv("AWS_ACCESS_KEY_ID"), os.Getenv("AWS_SECRET_ACCESS_KEY"), ""),
          Region:      aws.String(os.Getenv("AWS_REGION")),
        },
      ),
    ),
  )

  result, err := svc.GetQueueUrl(&sqs.GetQueueUrlInput{
    QueueName: aws.String("queue-name"),
  })
  if err != nil {
    return err
  }

  _, err = svc.SendMessage(&sqs.SendMessageInput{
    MessageBody:            aws.String(string(b)),
    QueueUrl:               result.QueueUrl,
    MessageGroupId:         aws.String("group-id"),
    MessageDeduplicationId: aws.String("deduplication-id"),
  })

  return err
}

This previous code can be broken into a few simple steps:

Define a message struct that contains the data you need in the background task
Marshal your struct into JSON and send that JSON as message to the SQS queue We can also send this through the message attributes, but I found this method more convenient to use
Give the user a 200 response and return. Your lambda function will finish after this, but the message you sent on the SQS queue will trigger the other lambda function to perform the background task

The Lambda Function to Handle SQS Event

Let's create another program to handle sqs events.

package main

import (
  "github.com/aws/aws-lambda-go/events"
  "github.com/aws/aws-lambda-go/lambda"

  "encoding/json"
  "log"
)

func main() {
  lambda.Start(handleSqsRequest)
}

func handleSqsRequest(sqsEvent events.SQSEvent) error {
  for _, message := range sqsEvent.Records {
    var request SqsTriggerMessage
    err := json.Unmarshal([]byte(message.Body),&request)
    if err!=nil {
      log.Println(err)
      continue
    }
    // task you need to perform based on the data in the message
    log.Println(request.Message)
    ...
  }

  return nil
}

type SqsTriggerMessage struct {
  Message string
}

This previous code can be broken into a few simple steps:

We wrote a handleSqsRequest function that will receive the messages from queue and registered it in the main function using the AWS Lambda for Go library
In the event handler function, we are unmarshalling the message body into our message struct, and printing the message on the console. You can send the details of the task you need to perform and use those to call appropriate function to do that task

Now we have 2 Lambda functions the first one sends the message to SQS queue, that needs to trigger our second lambda function to perform the background task

Deployment

Our lambda functions are now ready, and we can proceed by deploying it with the Serverless Framework. Our application is deployed by the Serverless framework based on the serverless.yml configuration file.

If you are not familiar with the .yml syntax, you can read this serverless.yml guide.

We must first create a serverless.yml file that defines what we are deploying.

service: your-service-name
provider:
  name: aws
  runtime: go1.x
package:
  exclude:
    - ./**
  include:
    - ./bin/**
functions:
  main-program:
    handler: bin/main-program
    events:
      - http:
          path: /
          method: get
      ...
  sqs-handler:
    handler: bin/sqs-handler
    events:
      - sqs: <replace_this_with_your_sqs_queue_arn>

In the above file, we defined both our lambda functions.

The first one runs the main program on the HTTP requests through Amazon API Gateway. In this program we need to perform the background task, so we will send a message on the SQS queue
The second one runs our sqs handler on an SQS event in the queue which arn we provide here. You can get the arn from your aws account where we created it. Amazon SQS > Queues > test.fifo

Next up, we will build our code, and deploy it using the serverless deploy command.

GOOS=linux GOARCH=amd64 go build -o bin/main-program .
GOOS=linux GOARCH=amd64 go build -o bin/sqs-handler ./sqs

serverless deploy

You can see the logs of all your lambda functions through the CloudWatch. To access the logs, go to CloudWatch > Log groups in your amazon account.

You can see the logs from here. Let's check the sqs-handler logs. It must have printed the received message.

References

Get the full source code from this github repository.

Forem: Muhammad Hamza Hameed

Go Workspaces (multi-module)

Workspace Deserves a Spotlight

Why Workspace?

Set Up Workspace

Create a module for your code

Create the workspace

Initialize the workspace

Run the program in the workspace directory

Download and modify the golang.org/x/example module

Run the code in the workspace

References

GoCD - Introduction (A tool that use for continuous Integration and continuous delivery CI/CD)

GoCD system requirements

Client (browser) requirements

GoCD server requirements

Hardware

Supported operating systems

Installation

Concepts in GoCD

Server and agents

Pipelines and materials

Stage details, jobs, and tasks

Pipeline dependency material

Artifacts

Fetching and using artifacts

References

Further reading

Serverless application in golang with AWS (Lambda Service)

What is Serverless Architecture?

AWS Lambda

Advantages of Using AWS Lambda

Serverless Application in Go with AWS

Prerequisites

The Lambda-Time Function

The Handler Function

Integration with Golang Routing Libraries

Deployment

References

Background process in Lambda using SQS

AWS Lambda

The Problem

The Solution

What is Amazon SQS?

Golang Example to Run Background Processes in Lambda Using SQS

Prerequisites

Create a Messaging Queue

Sending Message on the SQS Queue

The Lambda Function to Handle SQS Event

Deployment

References