Forem: Pol Milian

Building a Twitter bot with Go, GPT-3 and AWS

Pol Milian — Tue, 27 Dec 2022 14:52:20 +0000

I've recently been learning Go, as you can read in my previous post. What better way to learn a programming language than with a real project?

Everyone is talking about Twitter and its bots. I thought, how difficult would it be to do it? Is it even legal?

The bot I made is like a "wise Indian guru" that tweets love advice every 2 hours. This advice is generated using the GPT-3 API.

How did I do it?

We call the GPT-3 API and get the generated response body. From that, we parse it and trim it to remove blank spaces. This string is going to be the tweet for our bot
If all this process is successful, we will then make a POST request to the Twitter API.

How can we trigger this every 2h?

I decided to use an AWS Lambda for this. Once the code is uploaded, it's very easy to trigger it every X amount of time using AWS EventBridge + a cron or rate expression.

What do we need?

A Twitter Developer account with Elevated privileges.
A second Twitter account for our bot. A lot of guides don't tell you this (maybe it's too obvious?), but it should be step one in order to secure the handle you want.

All the code for this project is open-source, and is available at: https://github.com/el-pol/lovebot. Feel free to fork it & submit PRs 😇

Get the Twitter credentials to tweet from the bot

First of all, you need to apply for a Twitter developer account from your main account: https://developer.twitter.com/en. Then, apply for Elevated Access so that your applications can read & write. Here is a good step-by-step: https://www.extly.com/docs/perfect_publisher/user_guide/tutorials/how-to-auto-post-from-joomla-to-twitter/apply-for-a-twitter-developer-account/#apply-for-a-developer-account
Then, create a new regular Twitter account for the bot. I suggest to do this in another browser, so you don’t confuse your Main Account with your Bot Account.
In the bot account settings, find the Managed by section and refer it to your main account. This will add an Automated badge to the bot account and Managed by @yourhandle.
Once you’ve done steps 1-2-3, follow this guide: https://medium.com/geekculture/how-to-create-multiple-bots-with-a-single-twitter-developer-account-529eaba6a576
1. What we do here is authorize our main account to control our bot account. That’s why we need another authentication pin. This uses Oauth1a.
2. The final step is different. As of today, you will need to do a POST request instead of visiting the URL: curl --request POST --url '<https://twitter.com/oauth/access_token?oauth_token=><TOKEN>&oauth_verifier=<PIN>'
3. After this, you should get the response with the key & the token
4. There is a better way to do this with Oauth2; it’s more secure but a bit more complicated since you will need to re-authenticate with this flow every time you make a new tweet. If you have the time and energy, I suggest you to do it this way. Here is a good example: https://developer.twitter.com/en/docs/authentication/oauth-2-0/user-access-token
By the end, you should have 4 different secret keys:
1. Main account consumer key
2. Main account access token secret
3. Bot acess token (the one you just generated)
4. Bot access token secret (the one you also just generated)
You will use all of those: the first two to tell Twitter you have a developer account authorized to read/write, and the last two to tell Twitter that you will be tweeting from your bot account. You are really authenticating in two different steps. If you would only want to tweet from your main account, you would only need your own consumer key & secret.

Get the OpenAI API credentials

OpenAI is a paid API, so you will need to go to https://beta.openai.com/overview and add your billing details.

Then, you will get an API key that you can authenticate with. You can see the documentation here: https://beta.openai.com/docs/api-reference/making-requests

As you can see, we need to pass our key to the Authorization Headers.

The code

Believe it or not - the hardest part is over. If you have some experience in programming, what we are going to do next is two sequential POST requests: first to OpenAI to get the text, and next to Twitter to post it from our bot account.

This is the file structure I used:

The fetch package contains our logic for the OpenAI request. We will then import that into main.go
In main.go we do the Twitter POST request. This could be refactored into another package for cleaner code.
Your .env file should look like this:

AWS Lambda-specific warning

Usually, you would put all the code into func main() and it would just work. But when we work with AWS Lambda, we need to add a small modification. We have to put our code in a handler and then put it in func main(), as explained here: https://docs.aws.amazon.com/lambda/latest/dg/golang-handler.html

So for a Lambda, all the code you would normally put in main, has to go into another function and then pass it like this:

func main() {
    lambda.Start(HandleRequest)
}

Where HandleRequest is the function where we will execute our main code.

This is important because it tells the Lambda when to start. I did the mistake of not doing that. Then, the code is executed before the Lambda event starts, so it keeps erroring.

With his Lambda handler, we can pass information to the context, so we can do fun things like passing arguments to our code from HTTP query params. For example: https://aws.amazon.com/premiumsupport/knowledge-center/pass-api-gateway-rest-api-parameters/

Code review

In main.go:

package main

import (
    "context"
    "fmt"
    "log"
    "os"
    "strings"
    "time"

    "github.com/aws/aws-lambda-go/lambda"
    "github.com/dghubble/oauth1"
    "github.com/el-pol/lovebot/fetch"
    "github.com/joho/godotenv"
)

func HandleRequest(ctx context.Context) (string, error) {
    godotenv.Load()

    consumerKey := os.Getenv("CONSUMER_KEY")
    consumerSecret := os.Getenv("CONSUMER_SECRET")
    accessToken := os.Getenv("ACCESS_TOKEN")
    accessSecret := os.Getenv("TOKEN_SECRET")
    prompt := os.Getenv("PROMPT")

    if consumerKey == "" || consumerSecret == "" || accessToken == "" || accessSecret == "" {
        panic("Missing required environment variable")
    }

    // Will return a trimmed string
    fetched := fetch.GetGenerated(prompt)

    // From here on, Twitter POST API
    config := oauth1.NewConfig(consumerKey, consumerSecret)
    token := oauth1.NewToken(accessToken, accessSecret)

    httpClient := config.Client(oauth1.NoContext, token)

    // Necessary - you don't want to be charged for long Lambdas timing out.
    httpClient.Timeout = time.Second * 10

    path := "https://api.twitter.com/2/tweets"

    body := fmt.Sprintf(`{"text": "%s"}`, fetched)

    bodyReader := strings.NewReader(body)

    response, err := httpClient.Post(path, "application/json", bodyReader)

    if err != nil {
        log.Fatalf("Error when posting to twitter: %v", err)
    }

    defer response.Body.Close()
    log.Printf("Response was OK: %v", response)
    return "finished", nil
}

func main() {
    lambda.Start(HandleRequest)
}

The code is very basic - as this is just a fun project for me to learn Go. BTW, if you see any errors or improvements, please let me know.

And then in fetch.go we have:

package fetch

import (
    "bytes"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"
    "strings"
    "time"

    "github.com/joho/godotenv"
)

func GetGenerated(prompt string) string {
    godotenv.Load()

    apiKey := os.Getenv("OPENAI_API_KEY")

    if apiKey == "" {
        panic("Missing required environment variable")
    }

    // Create the request body
    jsonBody := fmt.Sprintf(`{"prompt": "%s", "max_tokens": 256, "model": "text-davinci-003"}`, prompt)

    // Create the request
    req, err := http.NewRequest("POST", "https://api.openai.com/v1/completions", bytes.NewBuffer([]byte(jsonBody)))

    if err != nil {
        log.Fatalf("Error when creating request: %v", err)
    }

    // Add the headers
    req.Header.Add("Authorization", fmt.Sprintf("Bearer %s",
        apiKey))
    req.Header.Add("Content-Type", "application/json")

    client := &http.Client{
        Timeout: time.Second * 10,
    }

    resp, err := client.Do(req)

    if err != nil {
        log.Fatalf("Error when sending request: %v", err)
    }

    defer resp.Body.Close()

    // Check the response
    if resp.StatusCode != 200 {
        log.Fatalf("Response was not OK: %v", resp)
    }

    // Parse the response
    var respBody struct {
        Choices []struct {
            Text string `json:"text"`
        } `json:"choices"`
    }

    err = json.NewDecoder(resp.Body).Decode(&respBody)

    if err != nil {
        log.Fatalf("Error when decoding response: %v", err)
    }

    trimmed := strings.TrimSpace(respBody.Choices[0].Text)

    if trimmed == "" {
        log.Fatalln("Result is empty")
    }

    if len(trimmed) >= 280 {
        log.Fatalln("Result is too long")
    }

    return trimmed
}

Packages used

github.com/aws/aws-lambda-go/lambda: Necessary only if we are going to use AWS Lambda.
github.com/dghubble/oauth1: To authenticate to the Twitter Oauth1.a API.
github.com/el-pol/lovebot/fetch: I put all the OpenAI code in fetch.go
github.com/joho/godotenv: To load .env variables

Packaging the code & uploading it to AWS Lambda

For this part, I followed this amazing guide written by Toul: https://dev.to/toul_codes/infrahamburglar-a-cybersecurity-news-bot-built-with-aws-lambda-golang-and-twitter-api-v2-198e

First, we need to build our application & then zip it. This zipped file is what we will uploadto AWS Lambda.

To build in Go, write in your terminal:

GOOS=linux  GOARCH=amd64 go build -o your-twitter-bot main.go

Once this is done, do:

zip your-twitter-bot.zip your-twitter-bot

This zip file is what you will upload as the code for your Lambda. Every further modification that you do will force you to repeat this two steps & upload again.

Adding the trigger

Every time this Lambda runs, you will post a new tweet. The way to schedule it is using a cron expression using AWS CloudWatch Events, or EventBridge. The reference for EventBridge is here: https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents.html

Make sure to refer to the docs, AWS’ cron expressions are a bit different from the standard: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html

Before adding the trigger, you should test that your Lambda works. If you go into the Test section of your Lambda, you can trigger it. We don’t care about the content of the event - since our Lambda does not depend on any parameters or arguments. So we can trigger any event, and it should fire and work.

If it worked, you should see a new tweet in your bot account.

If everything is OK, you can add the trigger so that the bot tweets every X amount of time with a cron expression.

Finished!

If you’ve reached the end, you should have a working bot.

This is the one I made, give him a follow: https://twitter.com/LoveAdviceWiseG

First impressions of Go as a JavaScript developer

Pol Milian — Tue, 20 Dec 2022 12:16:59 +0000

In my latest role, I had the chance to learn a bit of Go (or Golang).

My whole programming career I've been using JavaScript and TypeScript. I dabbled a bit with PHP. So let's just say that Go is officially the second programming language I'm learning.

So, here are my takeaways:

It's easier to pick up a second language

When I started with JavaScript, it was daunting. I had no idea about programming. All these concepts of memory, loops, data types, algorithms... Everything was new. You start to connect the dots after a while, and once you get the logic right, it's easy to spend hours on MDN and deepen your knowledge.

Now, with Go, the thought process is more like:

"So this is how you do a for loop, it's different from JavaScript in this and that way"

I have the basics of a programming language in my mind, so with Go, it's just figuring out how those basics work. It's a great exercise to constantly compare both.

JavaScript is a mess

Everything in Go is more organized. go.mod and go.sum are so much simpler and clean than package.json and the dreaded package-lock.json.

The Go CLI is a joy to use. The thing is that everything is built into the language. When we use JavaScript, we have to use NodeJS to interact with the OS. And NodeJS is great, but it is a library/framework hacked on top of JS. I'm sure tho that if we had to rebuild NodeJS today, it would be much better (hello Deno).

The documentation in Go is excellent. Every package has its documentation page. For example, let's have a look at the HTTP package.

Everything is faster in Go

Or at least it feels that way. I don't know enough about the innards of Go to tell you why it is faster. But literally every action I do feels a bit faster than in JavaScript.

Go is making me a better programmer?

Pointers

There are some basic programming concepts that you never even touch in JavaScript. The biggest one I've had to spend some time with are pointers. I'm still wrapping my head around when to use * and &, but at least the concept of memory addresses is clear in my head.

Error handling

I like the style of defensive programming that Go enforces. There's a pattern you see all the time:

f, err := os.Open("filename.ext")
if err != nil {
    log.Fatal(err)
}
// Do something if there's no error

So whenever we do anything, we first check if there's an error. If not, we proceed.

This is great, because how many times you've had an unhandled error in JavaScript? In JS you always need to actively make sure that you catch an error, usually with a try/catch block.

In Go, by putting the error first, we make sure we always handle it.

You can easily compile

Whatever code you write, you can build it as an executable file for multiple systems.

I'm not sure if you can do this in NodeJS - but for sure it's not built in. To run NodeJS programs you usually need to have NodeJS installed, and then run node your-program.js.

JavaScript is simple & it works

In JS, it's pretty easy to write some messy code in a few minutes and see its effect. Hell, you can even right now open the dev tools in your browser, go to the console and just do alert('hello') and it will work. JS is still the language of the web; so I don't think it's going anywhere.

Around the web, there are thousands of libraries written in JS. There's more documentation, and more people using JavaScript than Go. I've already had quite a hard time finding some answers for Go questions - and I always find the answer for JS.

I can see Go succeeding at replacing NodeJS, but for websites, JS and its frameworks will still prevail.

That's why, if you are a Full-Stack developer, I recommend you to take a look at Go, mostly for the backend: https://go.dev/tour/welcome/1

How to improve your technical interview? Start livestreaming!

Pol Milian — Wed, 10 Aug 2022 05:49:00 +0000

Recently I saw a video where Midudev, one of my favorite devs, gets interviewed:

It was a great learning experience for me. I've rarely seen other people interview, so it was very enlightening in a few areas.

The interview Miguel does is to join arc.dev. This is a company that connects companies to developers, something like gun.io, g2i, and many others.

Justin is the interviewer. I couldn't find a lot of information regarding him, but at the end of the interview he gives out a few details:

He has more than 10 years of experience, mostly in backend development.
His favorite languages are Python and Go.
Mostly USA based.

Why livestreaming will improve your interviewing skills:

You get used to the pressure

I always get very nervous at live coding, I really dislike it. It's not an uncommon feeling: there's someone observing you and scrutinizing your every move and thought. The pressure is there to not let the interviewer hanging and to "think fast".

Miguel, on the other hand, is very good at it. This comes probably from streaming on Twitch. He is a successful streamer, usually having around 1000+ viewers per stream. By streaming so much he is used to people watching him code live. That's why I think thatin the coding part of the interview he does so well.

He is still nervous, but the way he deals with the problem and explains all his actions is exactly what Justin (the interviewer) wants to see. To be fair, I believe the speed at which he solves the problem is because a very common interview problem: performing a two objects join with JS.

Still, it would have been hard for a lot of people; especially doing it live. I like how he uses node/assert as a very quick way to show he knows TDD and to prove his assumptions.

You have to explain everything

When you are streaming or being interviewed, there should not be "dead air". You have to talk, to explain everything you are doing. That's a skill that streaming will give you that's useful for live interviews.

It's never good to have assumptions about what the other person knows. That's also a good thing Miguel does: explain absolutely everything he is thinking and doing. This way, Justin is not left there wondering and just staring. He gets a feel for how Miguel thinks, which ultimately is the number one skill in programming: problem solving - and all the thought process behind it.

You'll have a live directory of your skills

If you have been streaming for a while, the VODs (stored files of past streams) will be available for everyone to see. Imagine a recruiter clicking on your name and being able to see hours and hours of you coding. There's no better social proof than that.

How to start streaming?

You just need a webcam and a Twitch account.

To record yourself, there's software like OBS or Streamlabs.

What I learned from Frontendmasters' Full Stack for Front-End Engineers Course

Pol Milian — Sat, 16 Jul 2022 07:50:42 +0000

In this post, I will reflect on what I learned from Jem Young's course "Full Stack for Front-End Engineers" on FrontendMasters.

To be fair, I already knew most of what he taught; so the following notes are just little tidbits that I didn't use in the past or concepts that I want to go over again.

I really recommend the course to everyone, even if you are just starting programming it will give you a good overview of how the Internet works and how servers work.

The command line

Why do people still use the command line, when so many things can be done with a GUI (Graphical User Interface)?

GUIs are opinionated. Interfaces always have details that force you to do things a certain way; and sometimes you have to fight around them to do what you really want.

The command line is always faster. If you know what you want to do, there is no faster way than the command line. You just type it and do it, without using your mouse. Kind of like using VIM.

Useful commands

cat: If we do cat FILE, it will show us the file contents.
less: It's like cat, but will show us less contents, one page at a time.

Shells

A terminal runs a shell. The shell is what interacts with the system and interprets our commands.

If we do echo $0, we can know the shell we are using. Common ones are bash, fish, zsh...

IP addresses and protocols

HTTP is based on TCP. UDP is faster, but TCP is lossless and that's why we use it.

We can ping a site, to know if it is working. For example, ping twitter.com.

Setting up a server

We can use a cloud provider like DigitalOcean or AWS to set up a virtual server. There are some steps we always need to do:

Update software
Create a new user
Make that user a super user
Enable login for new user
Disable root login

sudo !! returns the last command as super user.
If we check the auth.log file, we can see everyone that is trying to log in.
A common command to check a log is tail -f FILE. It will give us the last lines, and follow it in real time.

SSH keys

SSH is comprised of two keys: public and private. We will create a public key and copy it into ~/.ssh/authorized_keys, just so that we don't need to enter the key every time we try to log into the server.

It's good practice to disable root login in our SSH config: PermitRootLogin no.

SSH runs in port 22, but some people change it for safety reasons.

Nginx

Nginx helps us route our requests to wherever we want. It's used as a reverse proxy, web server and proxy server.

Nginx + Express

We can set up an Express server and with Nginx, route the request to Express. This is called proxy pass. We edit the Nginx config and proxy pass to localhost:3000, or whatever port our Express server uses.

After any config change we make, we can run nginx -t to validate the config.

Nginx config

We can write redirects in Nginx. 301 are temporary redirects, and 302 are permanent redirects. For example:

location /help {
    return 301 https://google.com
}

We can also have a subdomain. We can have a lot of different servers and a lot of different subdomains in Nginx.

Process manager

It's good practice to add a process manager to our server. This way we can manage restarts, and keep the server running. A common one is pm2.

Bash basics

Bash is a Unix shell and a command line language used everywhere. There are some common operators and tricks that we should know:

| and >: the pipe operator (|) gets the output and pipes it into something else. For example ps | grep bash. It's a redirection operator. > will write to a file.
find and grep: find will search file names, and grep will search file contents.

Security

If we do sudo apt install unattended-upgrades, it will auto upgrade packages to prevent against known vulnerabilities.

We should have a firewall. Keep all the ports closed unless needed. To check which ports are open, we can use nmap. If we do nmap OUR_IP, we can see which ports are open and closed.

There is a tool called ufw (uncomplicated firewall), that will make closing and opening ports easy for us. For example, ufw allow ssh.

HTTP

HTTP is HyperText Transfer Protocol, it is based on a request/response model. We send and receive packets, and each packet contains HTTP data. These packets have HTTP headers and the contents.

The headers are like the envelope where we see information about the packet. Example of headers: host, user-agent, accept, accept-encoding. Custom headers usually start with X-.

Everything in HTTP is stateless, that's why we use cookies, which are a way to persist information over time.

We can create custom headers and custom status codes.

HTTPS

HTTPS is encrypted HTTP. It encrypts all data in-transit

HTTP2

Requires SSL. It's faster because it can do multiplexing, we can do everything in one connection. For example: https://http2.akamai.com/demo

Cookies cannot be compressed.

Load balancers

top and htop tell us everything that goes on in the server. Kind of like Activity Monitor, for a server.

Nginx can be used as a load balancer: https://nginx.org/en/docs/http/load_balancing.html

WebSockets

Bidirectional connection between server and client. We keep the connection open. Usual example is for a chat app, or a chatbot.

Primitive vs reference values in JavaScript

Pol Milian — Sat, 16 Jul 2022 07:47:20 +0000

In this post, we will take a look at a super common interview question: primitive vs reference values in JavaScript.

What is a primitive value?

In JS, there are 6 primitive values: number, string, boolean, undefined, null and symbol.

You can think of them as the base, hence the name primitive. These values are not objects and they don't have methods.

What is a reference value?

The short answer is: anything that is not a primitive value is a reference value. They are typeof "object". We are talking about objects, arrays, functions, etc.

Key differences

The key difference is how these values behave under the hood. Specifically, how they store their values in memory.

The primitive values would have the value in memory, while the reference values would have a reference to that location in memory (the memory address), and not the value itself.

For example:

let name = 'Jesus'
let nickName = name

name = 'Juan'

console.log(name) // 'Juan'
console.log(nickName) // 'Jesus'

We are only using primitive values here, so everything makes sense. Right?

Now, let's take a look at a reference values example:

let jose = {
  type: 'Person',
  name: 'Jose'
}

let juan = jose

juan.name = 'Juan'

console.log(jose.name) // 'Juan'
console.log(juan.name) // 'Juan'

See? In here, the juan variable is referencing the same spot in memory as the jose variable. That is why when we change the name property in one of them, we will be changing it in both.

This behavior also explains why if we compare two seemingly equal reference values, the result will be false. For example:

const fruit = {
  color: 'Yellow',
  name: 'Banana'
}

const snack = {
  color: 'Yellow',
  name: 'Banana'
}

fruit === snack // false

So primitive types are compared by their value, while reference types are compared by their reference.

Lastly, remember that primitive values are immutable. So when you change a primitive value, what you are doing is replacing the in-memory value. You are not modifying it like you can do with reference values.