Forem: Anuj Kapoor

Deliver bug-free with Slack bots

Anuj Kapoor — Sun, 18 Apr 2021 10:02:04 +0000

It's every engineer's dream to deliver bug-free code, having a constant eye for catching those edge cases is the most important part to reach that. Working on a data-search product for a year, I have realized regular testing and health checks are quite important regularly just to make sure general sanity for the product.

Slack has been an important tool for us, not only for communication but also for delivering good products. Initially used as a tool to maintain communication among the team, I have thought of ways in which we can integrate slack for having the ease of consideration of tests and feedbacks at the dev level. It helps in collating everything in a single place , spoilers: slack 🙊

Most of the processes, I have worked on, uses slack just to notify about the results. One can use any other tool of choice to replicate the same. But slack's bot network was quite helpful in making this easy to implement and scalable. I have use slack api python client, directly using api and other clients also work.

Example Pipelines -

Testing:

Taking the example of our data-search product, data is updated daily and we have to make sure the sanity of it till frontend of our products. We use regular test runs and slack bot to notify about the same to make it seamless across the team.

code sample:

test.sh a simple bash script to test the service and sending out results to slack, which eases the process of not referring to a dashboard for the same

    #!/bin/bash

    echo "*********************"
    echo "* Testing *"
    echo "*********************"
    echo "Running tests"
    echo "............................"
    run_test --parallel=8 2>&1 | tee -a ./test_result.txt

    echo "****************************"
    echo "*Sending Results to Slack*"
    echo "****************************"
    python3 slack_message.py

slaclk_message.py utilizes standard code base of slack api client in python to have a bot sending out the results in specific channel

    import os
    from slack import WebClient
    from slack.errors import SlackApiError

    def status(str):
        if "OK" in str:
            return "OK"
        else:
            return "FAILING"

    test_resp = open('test_result.txt', 'r').read().replace('\n', ' ')

    test_status = status(test_resp)

    client = WebClient(token='SLACK_BOT_TOKEN')

    try:
        response = client.chat_postMessage(
            channel='#CHANNEL_NAME',
            text=""" Service status : {test_status}""".format(test_status=test_status))
    except SlackApiError as e:
        assert e.response["ok"] is False

Quality assurance(getting edge cases and more):

Notify if the quality of results from an API or Service degrade below certain threshold. Getting feedback from the user is quite important for improving the quality and they do the free labor for you by using your product. We can use similar slack client to notify about the same.

Health-monitor:

Keeps track of production infrastructure, and give health checks if utilization goes beyond limit

Random use-cases:

It can be used with different analytics tools like metabase to send regular insights about your products
One of team member uses this to keep track of indexing for one of the service

Possibilities are endless. These processes apply well for early-stage startups but might get out of hand easily for managing large infrastructure. But I think similar pipelines will help to catch those production and tailor your products in line with the expectations of your clients.

Do share feedbacks and some better pipelines you can think of

Getting back at blogging

Anuj Kapoor — Sat, 17 Apr 2021 17:14:11 +0000

Between the summer of 1665 and the spring of 1667, Isaac Newton made two long visits to Woolsthorpe in order to escape the plague affecting Cambridge. The bubonic ‘Great Plague’ of 1665–6 was the worst outbreak of plague in England. Issac did some amazing experiments with prism, hyperbola and gravity(famous apple incident)

Where I have been …

Me who got endless time and freedom to pursue my personal endeavors, had my share of different explorations, with bullets:

🎓 Started the year as a college student and became alumini overnight.

🎉 Started my first full-time job

👷 Had a constant love-hate relationship with remote work.

🧑‍🍳 Learned some basic cooking

👨‍👦‍👦 Met a lot of people and friends online, for some of them it was literally impossible without the pandemic

💻 Got weirdly into raspberry pi's, homelabs, and servers.

🌳 Had some amazing life lessons overall

Conclusion, Idk how to write blogs.xD

Using time efficiently doesn’t always mean being productive, having a fair share of explorations is important. Last year has been a whole new experience be it life, career or technical escapades.

2020 if nothing gave me a good port to host my internal services on 😆

I am trying to get back to blogging again. I think bringing structure to the week is important, will try to have a weekly and biweekly schedule from now.

Thanks for sticking around 🙂.

Sentiment Analyzer using SSVM

Anuj Kapoor — Sat, 19 Sep 2020 04:40:03 +0000

Recently I came across SSVM and WebAssembly on the server-side. It is a new way of development with ease of javascript and performance of rust. This approach combines Rust's performance, WebAssembly's security and portability, and JavaScript's ease-of-use. A typical application works like this.

The host application is a Node.js web application written in JavaScript. It makes WebAssembly function calls.
The WebAssembly bytecode program is written in Rust. It runs inside the SSVM and is called from the Node.js web application.

Description:

My project uses an express API to receive 'phrase' as a parameter and then uses a rust function to do the sentiment analysis using the AFINN-111 wordlist.

Bearbobs / ssvm-nodejs-starter

A template project to run Rust functions in Node.js through the Second State WebAssembly engine.

Sentiment Analyzer Backend using SSVM

It is an implimentation of sentiment analyzer using ssvm. It is inspired from 'Sentimental' which is in javascript Main advantage here is that the actual logic is based on rust which is faster and the api is exposed with express which makes it simple to impliment.

Example Usage:

/GET http://127.0.0.1:3000/?phrase="I am happy" //Score: 3, Comparative: 1

TODO:

Senitiment Analyzer logic.
File analysis
- API
- Rust logic(Facing issue with Rexiv2 with SSVM)
Make logic of Analyzer more refined.
Frontend to wrap it.

Getting started with SSVM

Fork this project to create your own Rust functions in Node.js. Learn more

The Rust functions are in the src directory. You can put high performance workload into Rust functions.
The JavaScript functions are in the node directory and they can access the Rust functions.
Use the node node/app.js command to run the application in Node.js.

Use Docker to build and

…

View on GitHub

The current implementation is very simple, as it is my first project with rust as well. I am working on adding file support to it so that it can parse text directly.

The underlying architecture is like:

Special thanks to second-state for an amazing getting-started guide, I was able to set it up on my local within 10 minutes.
Overall it was an amazing learning experience and I can think of many use cases where this type of infrastructure will work.

Open Code-Coverage Bot

Anuj Kapoor — Sat, 22 Aug 2020 06:32:39 +0000

My Workflow

Background:

When you try to search for some code-coverage solution that can utilize Github actions, you will definitely stumble upon two major players Coedcov and Coveralls. But both of them are proprietary and comes with an extra cost attached to it. When you are working on a personal project or an early-stage startup you don't want to spend extra bucks for the same.

Action:

So I designed this action which makes use of action artifacts to generate coverage reports and comment them on pull-requests. It also has a feature to upload reports to a file server. I used the same for managing a very data-intensive Django project. With my test-suite workflow, I used coverage.py to generate reports. I store these reports in a text file in actions-artifacts and also upload them. There is a little preprocessing done on the report to make it comment friendly.

The whole process is divided into two parts:

Running tests and generating the report.
Modifying the report to send it as an action-bot comment, and uploading it.

Expected Outcome:

the first part was fairly simple, I used coverage.py to run standard Django tests.you can find the syntax in the Yaml file below. For the second part, I created a new action repository open-coverage-action, It takes a text file as input and comments it on a pull-request. I am also working on adding preprocessing of the report to it. Currently, it uses basic bash to extract the part which contains the final result and add some polish, after that stores it in a new file for open-coverage-action. I also use file.io to upload a full report for further reference and also upload-artifact action to upload it as a an artifact.
it pretty much does the job required for my workflow.

Submission Category:

Maintainer Must-Haves

Yaml File or Link to Code

name: pr-coverage

on: pull_request

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 4
      matrix:
        python-version: [3.6]

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Cache pip
        uses: actions/cache@v2
        with:
          # This path is specific to Ubuntu
          path: ~/.cache/pip
          # Look to see if there is a cache hit for the corresponding requirements file
          key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-
            ${{ runner.os }}-
      - name: Install Python Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - name: Generating and Uploading Coverage Report
        run: |
          touch local.env
          echo DEBUG_VALUE=1 > local.env
          cd data_api/
          python manage.py makemigrations
          python manage.py migrate
          coverage run manage.py test --noinput --parallel 16
          coverage report > results.txt
          # Modifying coverage report
          echo "Coverage Report: " > report.txt
          echo " Statements | Miss | Cover " >> report.txt
          tail -4 results.txt >> report.txt
          echo "   " >> report.txt
          echo "Link to full report :" >> report.txt
          # uploading report to file.io any other services can be used
          curl -F "file=@results.txt" https://file.io |     python3 -c "import sys, json; print(json.load(sys.stdin)['link'])" >> report.txt
        env:
          SECRET_KEY: ${{ secrets.SECRET_KEY }}
          ADMIN_URL_KEY: ${{ secrets.ADMIN_URL_KEY }}
          DATABASE_NAME: github_actions
          DATABASE_USER: postgres
          DATABASE_PASSWORD: postgres
          DATABASE_HOST: 127.0.0.1
          DATABASE_PORT: 5432
      - uses: actions/upload-artifact@v2
        with:
          name: report
          path: data_api/report.txt
      - uses: actions/download-artifact@v2
        with:
          name: report
      - name: Open Coverage Notifier Action
        uses: Bearbobs/open-coverage-action@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          path: data_api/report.txt

Full file can be found in a gist @ workflow

Additional Resources / Info

This is just my first take on this idea. I am planning to reiterate on the preprocessing of the reports, to make it more polished. Currently, this workflow can be directly used by any Django project maintainers. Special thanks to file.io for free text uploading service and Github actions. please refer to the open-coverage-action repository for further reference.
Please reach out for suggestions and feedbacks. Thank You.

Links to the services used:

Open-Coverage-Action

File.io

Coverage.py

The Last Project

Anuj Kapoor — Wed, 20 May 2020 11:13:54 +0000

My Final Project

It's a blog which is developed using Javascript and Graphql

Demo Link

https://www.escapades.works/

Thanks, #githubsdp for helping with the domain
service used: Name.com

Link to Code

Bearbobs / Blog

Repository for Blogging my Experiences

Blog

http://escapades.works/

Simple Blog implimentation using React and Graphql

View on GitHub

How I built it (what's the stack? did I run into issues or discover something new along the way?)

It all started with a simple blog implementation using gatsby. I added some extra functionality like dark/light mode to provide a better outlook. Techstack is Javascript with Graphql.
In next rollout, I am adding options to comment and upvote posts, and according to these ratings ordering my posts. There will also be a timeline implementation of the post.

Additional Thoughts / Feelings / Stories

I am happy to implement it, as I always thought of writing articles but was never committed enough to do the same. Now I can basically spin up some new articles while parallelly polishing my own platform.