Forem: Ramki Pitchala

Build a Dynamic Portfolio With the Github API

Ramki Pitchala — Thu, 22 Apr 2021 21:13:37 +0000

This article was originally posted on Medium. If you prefer reading it from there, please do check it out.

Introduction

With the advent of countless templates and web page design applications, the task of creating a personal portfolio transitioned from a painful endeavor into a painless task.

However, the act of constantly updating the hard-coded information and rebuilding and redeploying your portfolio turns into a tiresome burden.

In this tutorial, I will show you how I integrated the Github API into my portfolio to pull the most recent project information, solving the issue of directly updating and redeploying my portfolio.

Agenda

Setup
Service
Integrate

Setup

Before we build the solution, let’s first understand the core idea and configure Github to ease the integration.

First, let’s talk about the Github repositories. To elaborate, I house my projects in the repositories themselves: I host the source code, keep detailed descriptions, and maintain links to demo gifs or images.

This repository houses my Path Finding Visualizer project.

In the right side, the section outlined in blue contains the description of the project. On the other hand, the section outlined in red has a link to the demo. Finally, the section outlined in black contains topics associated with the project.

To sum it all up, the Github API will return the above fields upon an HTTP request to:

GET /users/{username}/repos

This endpoint will return a list of public repositories for a user.

Here is what the return json object looks likes for the Path Finding Visualizer repository.

Although it is a bit difficult to see, the API returned the fields we need: description, homepage, and topics.

This is fantastic news, since it shows that we can store and retrieve information from our repositories.

Let’s now work on configuring Github to simplify the integration.

All we need to do is generate a personal access token in the developer settings. We will use the personal access token to authenticate our requests to the Github API so we can take advantage of the generous rate limits.

First, navigate to your profile settings.

From your profile settings, navigate to the developer settings.

Finally, go to the personal access tokens tab.

At this point, you can click “Generate new token” and Github will walk you through the rest. Once you have generated the token, make sure you save it to the clipboard, since we will need it for later.

Now that we have generated our personal access token, we can work on the actual integration.

Service

To start, let’s conceptualize how the portfolio would interact with the Github API.

Building a separate service that communicates between the portfolio and the Github API is a good idea, since we should house our Github personal access token in a secure environment.

Here is a depiction of the network flow:

Let’s start building out the service.

I will be using Python and Flask, but just understand that you can use any stack of your choice.

First, let’s create a config.py file. In this file, we will store our Github username and our personal access token. Make sure you do not push this file to version control.

    USERNAME = "YOUR GITHUB USERNAME"
    TOKEN = "YOUR PERSONAL ACCESS TOKEN FROM EARLIER"

Let’s create a server.py file and make a simple Flask app.

    from flask import *
    from flask_cors import CORS
    from config import *

    app = Flask(__name__)
    CORS(app)

    if __name__ == "__main__":    
       app.run()

Now, let’s expose an endpoint that our portfolio can call to get the project data.

    @app.route("/projects", methods=["GET"])
    def getProjects():
       pass

The getProjects() method will do three things.

It will make a request to the Github API for the repository data.
It will extract the relevant fields from each of the repositories.
It will return the filtered data back to the portfolio.

Let’s tackle these steps one by one.

Making a request to Github

We can simply use the Python requests library to make the HTTP request.

    url = "https://api.github.com/users/USERNAME/repos"        
    headers = {"Accept":"application/vnd.github.mercy-preview+json"}        

    repos = requests.get(url, headers=headers, auth=(USERNAME,TOKEN)).json()

The Github API will also include the topics field for each of the repositories if we pass in the above headers.

If all goes correctly, the request should have a status of 200 and the repos variable should be a list of json objects.

Extracting relevant fields

The main fields of concern are id, name, html_url, description, topics, and homepage. The homepage field represents the text stored in the black outlined area:

We can put links to our demo gifs in homepage.

The code to parse each of the repositories is this:

    projects = []
    for repo in repos:
        if repo["homepage"]:
            project = {
                "id": repo["id"],
                "name": repo["name"],
                "url": repo["html_url"],
                "description": repo["description"],
                "topics":repo["topics"],
                "images": repo["homepage"].split(";")
            }
            projects.append(project)

Just to clarify, I added more than one demo link for certain projects. I separated each link with a semi-colon.

Here is what project would look like for the Path Finding Visualizer repository:

    {
      "id": 206887035,
      "name": "Pathfinding-Visualizer",
      "url": "https://github.com/Ramko9999/Pathfinding-Visualizer",
      "description": "Built a colorful visualizer for shortest path algorithms: A star search, Breadth First Search, and Bidirectional Breadth First Search. Utilized Queues and Priority Queues to reduce collection operations such as pop() and push() to O(1) and O(log(n)) run times.",
      "topics": [
        "data-structures",
        "graph-algorithms",
        "java",
        "pathfinding",
        "swing"],
      "images": [
        "https://media.giphy.com/media/U6GuVGBOQoHUHaZ7LV/giphy.gif",
        "https://media.giphy.com/media/gg9eCb2ZBqcU4LlSML/giphy.gif",
        "https://media.giphy.com/media/l4Wyj6PQwwxZdPjfSe/giphy.gif",
        "https://media.giphy.com/media/LRgBCMGhCvqkzHzeaY/giphy.gif"]},

All that remains now is to return projects back to the initiator of the request.

Returning a response

We can return the filtered data back to the client with the following:

    return {"projects": projects, "error": False}

So with all the pieces put together, the getProjects() method will look as such:

Please be sure to test your service locally by calling the /projects endpoint. Once it is working, you can deploy it wherever you like. Personally, I deployed the service on pythonanywhere.com.

Pat yourself on the back, as the hardest part is now over!

Integrate

All that remains now is that we have to make an HTTP request from our portfolio website to the service we have just created.

For example, I used React to build out my portfolio, so here is how I made the call and fetched the projects.

All you need to do now is to build out the user interface based on the data.

With that said, feel free to head over to my portfolio to check out how I implemented my user interface.

That's it folks!

Client-Side Image Compression on the Web

Ramki Pitchala — Wed, 21 Apr 2021 18:29:30 +0000

This article was originally posted on Medium. If you prefer reading it from there, please do check it out.

Huge shoutout to Codú Community for inspiring this blog. All the code for this project is linked on GitHub.

Introduction

There are multiple ways of optimizing server performance.

One way is to make the client do some of the work.

Consider uploading images for profile pictures. Since high-quality images take up several MB, it is costly to send them over the network to the server. Also, since profile pictures don’t need to be extremely detailed, it would be nice to compress them and save space.

Thankfully, with HTML Canvas, we can compress our images on the client itself. After the compression, we can then send the images to the server, reducing upload time and the work the server must do.

Agenda

Setup demo HTML page
Listen to the image input
Resize and compress image with HTML Canvas
Demo of the compression working

Setup demo HTML page

To follow along, create a new project with the following files:

- index.html
- main.js

We will create the basic UI in index.html.

Put the following in index.html:

In addition to accepting file uploads, we will preview both the initial image the user uploads and our compressed version in the UI.

Let’s go to main.js to handle when a user inputs an image.

Listen to the image input

In main.js, let’s first define getImageDimensions, which returns a Promise of an input image’s width and height. We need the initial image’s dimensions to maintain the aspect ratio when we resize.

    function getImageDimensions(image){
        return new Promise((resolve, reject) => {
            image.onload = function(e){
                const width = this.width;
                const height = this.height;
                resolve({height, width});
            }
        });
    }

Let’s now add an event listener to handle when our input tag, image-input, changes.

The above listener will trigger whenever a user uploads an image. We take the uploaded image, display it to the user, and acquire its dimensions. All that is left is to resize and compress the image.

Resize and compress image with HTML Canvas

Let’s get to the fun part and make the compressImage function in main.js.

This is the magic!

Given an HTML image, the scale factor, and the initial width and height of the image, the function creates an HTML Canvas and draws the image downscaled on it.

Finally, we turn the downscaled image into a blob and resolve it from the Promise. The resolved blob represents our compressed image.

We can now use this function to compress whatever image we want.

Let’s integrate this into the event listener we created earlier.

Let’s break this down.

First, we create two compressed images with differing scales: the ratio of MAX_WIDTH and the initial image’s width and the ratio of MAX_HEIGHT and the initial image’s height (You can parameterize MAX_WIDTH and MAX_HEIGHT based on the use case).

Then, we pick the smaller blob out of the two to be our compressed output and display it to the user. Finally, we check if our compressed version is smaller than the initial image. If the initial image was smaller, we can use it instead.

We now can compress images whenever the user inputs an image on the client. optimalBlob represents the image with the smallest size among both the compressed versions and the initial image.

Demo of the compression working

I took the above image and submitted it into our file input.

Here is what occurred:

Here is the compressed result:

The initial size of the image was roughly 299 KB and the compressed result was only 45 KB, a huge reduction.

With this reduction in size, it will be much faster to send the image to the server, and the server doesn’t need to worry about compressing it either.

It is a win-win situation!

However, if image quality is important, this approach is not a good idea, since resizing through HTML Canvas is lossy.

Despite that, this is a great way to handle the uploads of profile pictures.

That’s all I got. Thanks for reading!

Host and Use Redis for Free

Ramki Pitchala — Wed, 21 Apr 2021 01:22:31 +0000

This article was originally posted on Medium. If you prefer reading it from there, please do check it out.

Introduction

Although building software projects can be challenging, deployment can be a pain.

In fact, in situations where speed is necessary, deployment must be seamless. For that reason, I want to show you how to deploy Redis for free and use Redis remotely.

Redis Labs, a Redis cloud hosting service, offers a free plan with simple deployment steps. We will leverage that plan to create an instance and integrate that instance with Node.js.

Walk Through

Do everything quickly and well.
― G.I. Gurdjieff

To start, head over to https://redislabs.com/ and sign up.

Once you create and verify your account, you will end up on this screen.

Click “Create your subscription”.

Scroll down to “Fixed size” and pick the free option.

Click “Create”. We now need to create our database under the subscription.

Enter in the database name. Be sure to copy “Redis Password”. Once you are ready, click “Activate”.

We can use the endpoint to connect to our remote instance so be sure to copy it down. For security reasons, be careful with who you expose your endpoint to.

It is really that quick!

Integration

Let’s connect to our remote Redis instance! Although I will use Node.js, the connection process will most likely be similar to the other technologies.

Start by creating a new npm project.

npm init

After filling out the project details, cd into your project and install redis, a Node.js client for Redis, and dotenv, an environment variable loader.

npm install redis

npm install dotenv --save-dev

In the root directory, create a file called .env . In it, let’s put our Redis instance hostname, port, and password.

We can find the hostname, port, and password in the View Database section:

Contents of .env:



    REDIS_HOSTNAME=YOUR REDIS HOSTNAME
    REDIS_PORT=YOUR REDIS PORT
    REDIS_PASSWORD=YOUR REDIS PASSWORD

Create index.js. Let’s say this is where we want to connect to our remote instance. Let’s create a client and test if we can connect to it.

Now we can connect to our Redis instance from Node.js!

Conclusion

I wanted to show a simple way to deploy and use Redis for free, whether it be for demoing a project at a hackathon or evaluating a proof of concept idea.

Thank you for taking the time to read this blog!

Set a Time Limit on Async Actions

Ramki Pitchala — Mon, 19 Apr 2021 17:01:56 +0000

This article was originally posted on Medium. If you prefer reading it from there, please do check it out.

Introduction

Asynchronous programming is incredible!

With it, we can run our IO-intensive tasks without having to block the execution of other code.

However, in situations where the code after a blocking task depends on the task’s result, we must wait. Now, imagine if the blocking task took a long time to finish or it never finished. This could be problematic in the context of the application.

We can solve this issue by setting a time limit on our task. If our task doesn’t finish in the span of the time limit, we can return a failure value instead. Let me elaborate.

Concept

Let’s suppose that the blocking task is a promise called longTask, and it was used in the following function:

    async function doSomething(){
       let data = await longTask;
       doSomethingImportantWithData(data);
    }

If the time it takes longTask to settle is longer than our requirements or if longTask is never settled, we won’t be able to execute the code after longTask in a timely manner.

However, imagine if we could set a time limit on our blocking tasks. In the case that the blocking task doesn’t settle within the time limit, we can return a failure value from the task. In the scenario the task resolves, we can return the value it resolved to.

To elaborate, suppose there was a function called fulfillWithTimeLimit which takes in milliseconds, the time limit, task, the task promise we would like to set a time limit on, and failureValue, the value that would be resolved from fulfillWithTimeLimit if task never completes within the time limit.

In the case that longTask is resolved before the time limit, fulfillWithTimeLimit returns with the value resolved from longTask.

In the case that longTask never finishes within the span of the time limit, the function should immediately return failureValue.

With this approach, we ensure that we never have to wait on longTask for more than the specified time limit.

Let’s dig into the approach.

Code

In order to “set a time limit” on the task, we can create another promise, timeoutPromise, which resolves to failureValue after the time limit. After that, we can race both our timeoutPromise and task with Promise.race.

Promise.race takes in a list of promises and resolves or rejects to the value of the promise in the list that is settled first.

To provide an example, suppose I had the two following promises:

    const a = new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve("A finished before!");
        }, 100);
    });

    const b = new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve("B finished!");
        }, 1000);
    });

Suppose I raced these promises and got the value.

    const finishString = await Promise.race([a, b]);

Since a resolves after 100 milliseconds while b resolves after 1000 milliseconds, a will be the first promise to resolve. As a result, finishString will be equal to “A finished before!”. If you would like to learn more about Promise.race, please check out the following:

Promise.race()

Nonetheless, let’s apply the promise racing idea to create the fulfillWithTimeLimit function.

To begin, we create our timeoutPromise and ensure it resolves with the failureValue after the time limit. Then, we race to see whether our task or timeoutPromise finishes first. For safety, we can clear the timeout and return response, the resolved value of the race.

Here is how doSomething looks now:

In the above example, I set failureValue to null. However, it may be better to set it to a value of the same type as what is resolved from the task. In fact, it may be better to call reject in the timeoutPromise than to resolve with a failureValue.

That’s it! We can easily reuse fulfillWithTimeLimit in our application code where we need a time limit.

Conclusion

In this blog, I aimed to show a solution with Promise.race to handle situations where blocking tasks may fail to settle or take too long to settle. Though I did not cover all the functionalities of promises, I hope this article amplifies your curiosity to explore them more.

Resources

Promise

Promise.race()

setTimeout

Crunch Pokemon Data with Python and Deta Base

Ramki Pitchala — Sun, 18 Apr 2021 21:09:50 +0000

This article was originally posted on Medium. If you prefer reading it from there, please do check it out.

Introduction

Performance is a must when it comes to building software. However, in certain situations, the speed to set up and integrate new services for proof of concept evaluation or infrastructure is overlooked. Specifically, in the realm of databases, there are many options, but I find Deta to be most seamless to set up and use. Deta offers Deta Base (I will refer to it as Base), a NoSQL data store optimized for developer simplicity. In this article, I aim to show how to set up a Deta project and interact with your Base to store and manipulate Pokemon data.

Let’s get started!

Agenda

Setup
CRUD
Querying

Setup

To begin, navigate to Sign Up, create a new account, and verify your account. Once you sign in, you should be on the following page:

Click the arrow in the top left, we will create a new project from scratch.

Enter a name of your choice and hit create! A popup with your project key and project id will appear. Make sure you save the key!

With the project key saved, create a new directory, walk into it and run the following command.

pip install flask deta

Flask is a web framework that we will use to create endpoints to listen to incoming requests. Based on the requests, we will interact with our remote database using the deta package.

Here is how the folder structure will look like:

app.py

config.py

In config.py, we will store our project key

DETA_KEY= "YOUR_COPIED_KEY"

In our project, we will create a Base called pokemon and utilize it to store, access, and manipulate Pokemon data. Before that, let’s go over the schema for a record in pokemon:

Each record in a Base must correspond to a unique identifier called key. When we insert a Pokemon record into our Base, we will provide name as our key. As a result, if we needed to get information on Charizard, we just have to ask pokemon to find the associated record with Charizard as its key.

In app.py, let’s set up the Flask app and our connection to our Base, pokemon.

    from config import DETA_KEY
    from flask import Flask, request
    from deta import Deta
    from json.decoder import JSONDecoder

    app = Flask(__name__)

    deta_project = Deta(DETA_KEY)
    db = deta_project.Base("pokemon")
    decoder = JSONDecoder()

    if __name__ == "__main__":
        app.run()

All the data access and manipulations will occur through db.

Our setup is now complete! Onto the CRUD!

CRUD

CRUD is an acronym for creating, reading, updating, and deleting data in a database. We will explore how to perform each of the above operations in our pokemon Base. All the work will be done in app.py.

Insertion

Deta provides 2 ways of inserting data. The first is through the put method. put is the faster method of insertion. If you call put on a record that already exists in the base, put will overwrite the record. In contrast, insert is 2x slower than put. In the case that you try to insert into the Base with an already existing key, it will throw an error.

Let’s create an endpoint to insert a new Pokemon with put.

Here is what the endpoint would look like using insert :

Deta also supports inserting multiple records at the same time with put_many. According to the documentation, it is possible to insert at most 25 items into the Base in a single call (Deta Base SDK).

Let’s test out what we have so far. Let’s insert the data for Pichu into the Base via an HTTP POST through /pokemon on Postman.

Once you hit send, navigate back to Deta.

Click on your Base under the Bases section. You will now be able to view the data in your Base.

As you can see, Pichu is in our pokemon Base!

Access

We can use the get method to access the record of a given key.

Since our Pokemon name is the key, we can directly access its record by providing its name. We can test it with the following GET request.

Updation

As stated previously in Insertion, the put function can be used for overwriting records. However, put completely overwrites the record and can remove prior fields that are not part of the new updates. As result, if we want to partially update the record, we can use the update function. In fact, update also allows for fine-grained updations like incrementing values and appending, prepending, and removing elements in a list (Deta Base SDK).

Let’s make sure updating works with an example. First, I will insert the following data for Charizard.

    {
        "region": "Johto", # Charizard is from Kanto region
        "name": "Charizard",
        "height": 1.7,
        "weight": 90.5,
        "type": ["Fire"], # Charizard also Flying type
        "evolution": ""
    }

region should be Kanto, and Charizard is also a flying type. Let’s update Charizard with a POST request to /pokemon/update/Charizard.

After a GET request to /pokemon/Charizard, it is clear that region and type are updated.

Deletion

Deta provides delete, a function that takes in a key and deletes the record associated with the key.

If I wanted to delete Charizard, I would make an HTTP DELETE request to /pokemon/delete/Charizard.

We have now explored how to insert, access, update and delete from our Base. Let’s learn how to query our Base.

Querying

Prior to querying, make sure to fill up your Base with some more Pokemon.

Querying is done through the fetch method. To elaborate, fetch takes in a query or a list of queries and accumulates a list of records whose fields match the query or queries.

A query is nothing more than a dictionary where the mapping between the keys and values represents the query condition. For instance, suppose I wanted to get Blastoise’s record with a query instead of get, here is how it would work:

    query = {"name": "Blastoise"} 
    results = next(db.fetch(query))
    blastoise = results[0]

We accumulate all the records with name equal to Blastoise.

It is also possible to query based on inequalities. For instance, we can query for all Pokemon that weigh greater than 100kg and are less than 1 meter tall.

    query = {"weight?gt": 100, "height?lt": 1}
    pokemon = next(db.fetch(query))

We can append “?gt” and “?lt” at the end of numerical fields to query for records with respective values greater than or less than a threshold. There are a lot more suffixes that can be added to the end of a query field, so I recommend reading the documentation for your specific use case (Deta Base SDK).

Let’s create an endpoint that will return Pokemon which are of a parameter type.

“?contains” checks if a provided query element exists in the list associated with the field. To provide an example, if I wanted all the fire-type Pokemon, I would send a GET request to /pokemon/type/Fire.

There are 13 Pokemon returned from the above request.

This is a little overwhelming since this query will return all the fire-type Pokemon in our Base. Deta also provides us with the ability to limit the number of query results with buffer. Let’s set the buffer arg in fetch to 2.

Now, we only get two records back.

Finally, by changing the pages argument in fetch, it is possible to spread the result data over multiple pages.

Conclusion

My primary goal with this writing was to shed greater light on a database that you can set up and work on in the blink of an eye. It took hardly much time to create the project and based on the above examples, CRUD and Querying are as simple as they can get. For these reasons, Deta Base is perfect for proof of concepts, serverless applications, hackathons, and many more situations and projects that require simplicity and speed.

Resources