Forem: Kannav Sethi

Reflecting on the ORAssistant Project: A Journey of Collaboration and Technical Growth

Kannav Sethi — Sat, 07 Dec 2024 04:32:01 +0000

Introduction

If you've been following my last two blogs, you already know the context of this project. This is the final installment of a three-part series documenting my journey with the OpenROAD Project's ORAssistant. If you missed the previous entries, I recommend checking out those blogs to get the full background on this exciting open-source contribution.

blog1
blog2

When I first stumbled into the world of the OpenROAD Project's ORAssistant, I had no idea how much I'd learn. What started as a simple desire to contribute turned into a rollercoaster of technical challenges.

The first challenge was understanding the existing system. I spent hours poring over the code, trying to figure out how the current feedback mechanism worked. It was like solving a puzzle where each piece was slightly more complicated than the last.

Final Thoughts

The core of my technical work involved creating a MongoDB-based feedback submission system. This required:

Carefully analyzing existing feedback submission mechanisms
Designing a prototype function mirroring the Google Sheets integration
Developing and refining a validator schema for the context collection
Debugging and improving the submit_feedback function in mongoClient.py

Each of these steps wasn't just a technical task – it was a learning journey, a chance to understand not just how code works, but why it works the way it does.

Challenges Faced

Remember those minor issues that can drive a developer crazy? I had plenty. My validator schema for the context collection was a mess at first. And the submit_feedback function? It was returning None like a stubborn child, throwing error messages even when everything seemed correct.
But the real kicker was the CI pipeline issue. Picture this: I'd crafted what I thought was a perfect pull request, only to have it shut down because of secret propagation problems. Talk about frustrating!

The frustration was real. But so was the learning.

The Community That Saved Me

This is where the magic of open-source really shines. The project maintainers weren't just gatekeepers – they were problem-solvers. They granted me repository access, offered guidance, and showed me that open-source is as much about people as it is about code.

What I Learned (Beyond Code)

Sure, I picked up some technical skills. MongoDB integration? Check. CI/CD troubleshooting? Got it. But the real lessons were deeper:

Patience is more than a virtue; it's a necessity in software development
Every error is an opportunity to learn
Communication is the real programming language

Conclusion

To anyone thinking about contributing to open-source: do it. It's messy, it's challenging, and sometimes it'll make you want to throw your computer out the window. But trust me, the rewards are worth every moment of frustration.

I've said this previously as well , but I love open-source and all of this was really a good experience for me

Progress on Automatic Feedback Loop

Kannav Sethi — Fri, 29 Nov 2024 02:28:32 +0000

Introduction

This blog will elaborate on my progress towards contributing to the Automatic Feedback Loop on the repo ORAssistant

to know more about these you can have a look at my previous blog

Well this repo has given me a lot of good experiences, I will briefly discuss some of them over here

Integrating the MongoDB function for feedback

Now in the first PR that I made to this repo, I proposed a design for MongoDB,

In this PR, I am integrating the functionality to submit the feedbacks that the user is getting to the MongoDB collections

My approach was as follows

Check the existing functionality to find how the parameters were being passed to the google sheets function
Create a prototype function following a similar logic to the Google Sheets function
Test it out with dummy data from the frontend
Check if the data is being loaded to the MongoDB collection

Now this approach actually succeeded but with one major issue and a couple of minor issues

The minor issues were as follows :

My validator schema for the context collection was wrong and because of this my data was not being sent to the collection,
The submit_feedback function in mongoClient.py file was returning None in all the cases, thus leading to error messages even when the result was correct

I will talk about the major issue in the next point

Fork Issue

The major issue that I encountered while submitting a PR was that even though all of my new code in the PR was correct, the CI pipeline wasn't accepting any new PRs due to the fact that the secrets from the upstream repo were not propagating to the fork repo, thus the CI pipeline was failing

Seeing this issue, the maintainer did grant me the read/write permissions to contribute to the repo , and I did so.

I volunteered to fix this issue and correct the pipeline as not correcting it would lead to all of the potential new contributors having read/write access to the repo which isn't good

Conclusion

my journey with the ORAssistant repo has been an insightful learning experience. Integrating the MongoDB function for feedback has not only enhanced my technical skills but also provided valuable insights into database handling and testing. Although the initial implementation was successful, addressing the encountered issues helped me refine my approach. Moving forward, I plan to continue building on this foundation and contribute further to the project, ensuring smoother functionality and better user experience.

Planning To Resolve Automatic Feedback Loop

Kannav Sethi — Wed, 27 Nov 2024 02:27:02 +0000

Introduction

One of the things that I have learned during my Open-Source Journey, is that this domain is filled with experiences, which will not only level up my skills as a Developer but would also give me a chance to work on technologies that I would've worked at a Big Company

To broaden my horizon of tech-stack, I looked out for another issue that would enhance my skills as a Developer, I saw this repository called ORAssistant

Why this Repo

I have always wanted to work on AI applications that are industry standard or have industrial standard designs, and this repository is a RAG-based application

At first glance over this repo, I knew it was a match made in heaven, the amount of details that the maintainers have highlighted in the architecture of this application is gigantic.

This is one of the diagrams explaining the architecture

I have always wanted to see how the designs behind such apps look like

This was my first time seeing a design like this and I loved it.

All in all, my passion for learning more about AI design was the one that attracted me to this project

Issue and My Approach

I have talked about the issue in one of my previous blogs, you can go and check it out for a comprehensive overview of the issue that I am dealing with

For this issue, my first PR for this project was to deal with the MongoDB design, again I have given a comprehensive report for the same in my previous blog

I am working on getting my second PR merged which is related to adding the MongoDB functionality to the existing Feedback functionality,

My approach to this is to add to the existing functionality without breaking the previous code and testing it out to see if it's working as expected

This is my second PR contribution to this project and so far I have loved every moment of it, there's just too many things to learn and develop

Conclusion

Contributing to the ORAssistant project has been a remarkable learning experience that aligns perfectly with my passion for AI design and development. It has not only deepened my understanding of industry-standard architectures but also allowed me to tackle real-world challenges. This journey has truly reinforced my enthusiasm for open-source development and its ability to foster growth and innovation.

Contributing to ChatCraft

Kannav Sethi — Sat, 23 Nov 2024 04:07:20 +0000

Adding API Key Management for Non-LLM AI Providers in ChatCraft

Introduction

In this blog post, I'll discuss my recent contribution to ChatCraft.org, where I implemented support for non-language model AI providers, specifically focusing on Jina.ai integration for PDF to Markdown conversion. This work was done as part of PR #740.

The Challenge

ChatCraft already supported API key management for LLM providers like OpenAI, but there was a need to extend this functionality to other AI services. The primary goals were:

Separate LLM providers from other AI services in the UI
Implement API key management for Jina.ai
Create a foundation for supporting additional document processing providers

Implementation Process

1. UI Modifications

First, I modified the settings dialog to clearly distinguish between different types of providers:

Changed "Providers" label to "LLM Providers"
Added a new "Other AI Providers" section
Implemented a consistent table layout matching the existing providers section

2. Provider Architecture

I created an abstraction for non-LLM providers:

export abstract class NonLLMProviders {
  id: string;
  name: string;
  apiUrl: string;
  apiKey?: string;
}

3. Jina.ai Integration

Implemented the JinaAIProvider class:


export class JinaAIProvider extends NonLLMProviders {
  constructor(apiKey?: string) {
    super(JINA_AI, JINA_API_URL, apiKey);
  }

  static fromSettings(): JinaAIProvider {
    const settings = getSettings();
    const provider = settings.nonLLMProviders["Jina AI"];

    // case 1 when the provider already exists in the settings
    if (provider instanceof JinaAIProvider) {
      return provider;
    }
    // case 2 when the provider doesn't exist but the api key is set
    if (provider?.apiKey) {
      return new JinaAIProvider(provider.apiKey);
    }

    // case 3 when the provider doesn't exist and the api key is not set
    return new JinaAIProvider();
  }

Key Learnings

Provider Abstraction: Initially, I created separate implementations for each provider type. However, after code review feedback, I realized there was a significant overlap with ChatCraftProvider. This led to discussions about future refactoring to create a more generic provider system.
Error Handling: Working with Jina.ai's API taught me the importance of graceful degradation. When users hit free-tier limits, we needed to provide clear guidance about API key usage.
Document Processing Power: During testing, I discovered Jina.ai's impressive capabilities - it successfully converted an 85-page PowerPoint PDF to markdown with high accuracy, opening possibilities for future document processing pipelines.

Challenges and Solutions

UI Consistency: Initially, the new provider section had different sizing from the existing table. This was standardized after code review feedback.
Code Organization: The original implementation had some redundancy in provider management. This was resolved by better encapsulation and static utility functions.

Future Improvements

Better error handling for free-tier limitations
Refactoring provider classes for better code reuse
Support for additional document processing services

Conclusion

This repository was too good to work on, in fact some of the knowledge that I have gained from here would be used in my own project DialectMorph, because both of the projects are utilising different AI clients but are encapsulating logic in a way which makes it easier to be integrated into any app

Contributing to ORAssistant

Kannav Sethi — Sat, 23 Nov 2024 03:57:32 +0000

Introduction

I was pleased after having contributed to various repositories during Hacktoberfest, but as soon as Hacktoberfest, I had this new profound excitement to contribute to more Open-Sourced Projects. I had contributed to a lot of projects that had a tech stack that included both backend and fronted, but this time around I wanted to contribute to an AI-based project, particularly something related to RAG (Retrieval Augmented Generation) as I wanted to delve into it

ORAssistant

While looking for a lot of repos based on RAG, I stumbled across a perfect open-sourced RAG tool, ORAssistant, which is again, a chatbot that responds to common questions or inquiries for a larger project.

The architecture of this tool is quite complex, I am still trying to figure out how the main query architecture operates, but this is the exciting part, learning as I contribute.

Issue

For my first issue, I picked up one of the issues where the task was to automate the feedback loop, in Leyman terms, what this entails is that an RAG app usually depends on feedback from the user to further fine-tune the response, the task was to get this feedback from the user and store it in a database and feed it back to the model itself

The architecture would look something like this

Currently, the system stores the feedback in Google Sheets, which again is not an optimized approach

This issue in itself would take around 4-5 PRs to get solved, but for the focus of this blog, I will restrict it to the first PR I have made.

First Pull Request

For the first pull request, as evident from the discussions from the issue, my task was to first get the database design set up and running. In doing so, I encountered a lot of issues

Issues faced

During the setup, the documentation to get the GOOGLE_SERVICE_KEY wasn't straightforward, so I had to double-check it with the maintainer and tweak a lot of settings in my personal Google account to get it up and running, the maintainers were helpful during the entire process
There were some inconsistencies in the backend which led to the frontend not operating correctly, but one of the good things about this project is since the backend is changing dynamically, they have a mock backend so that while the backend is being developed, the fronted doesn't suffer.

Main solution

The solution that I had proposed for this PR, revolved around the discussion of choosing the right database, after an elaborate discussion with the maintainer, we decided that it was best to use MongoDB for the project, thinking about scalability and flexibility in the fields due to the schema-less nature of MongoDB.

After making the initial design, I opened a PR, which was related to having the initial design setup for the frontend

One of the issues faced during the process of getting this merged was that it wasn't passing a test in the CI pipeline, which wasn't related to an error in my code, but because some repository secrets were not being propagated to the fork of the repository that I was working on, so the maintainer had to give me write access to the repo to get my PR merged

Further Contributions

This PR would now act as the base for further PRs that would eventually solve the entire issue. To be honest, this is one of the best projects I have worked on in a while, it is taking me about 6-7 PRs to just solve one issue, this shows how intricate and developed the project is.

I am really enjoying how my Open-Source Journey is shaping out to be.

Publishing my Tool

Kannav Sethi — Sat, 23 Nov 2024 02:54:56 +0000

Introduction

I wanted to share my experience of publishing the package to NPM. This blog post details the process, challenges, and lessons learned during the release.

Choosing the Package Registry

For DialectMorph, I chose to publish on NPM (Node Package Manager) as our primary package registry. The decision was influenced by:

NPM's widespread adoption in the JavaScript/TypeScript ecosystem
Excellent integration with both Node.js and Bun
Robust documentation and community support
Built-in versioning and dependency management

Release Process

1. Package Preparation

First, I had to prepare the package for NPM publication. This involved:

{
  "name": "dialect-morph",
  "version": "1.0.0",
  "bin": {
    "dialectMorph": "./dist/index.js

}

The critical changes included:

Setting up the version correctly
making sure that the binary was being made available to the user when downloaded as an independent package
Ensuring all dependencies were correctly listed

2. Build Process Configuration

I configured the build process using Bun:

Added TypeScript compilation settings
Set up the build script to create the dist directory
Ensured the binary was properly executable

3. Testing and Verification

Before publication, I:

Ran local installation tests using bun link
Verified the CLI worked globally
Tested all supported language conversions
Validated configuration file handling

Issues Encountered

Before going to the user testing phase, I wanted to install this globally on my machine to test but since I already have it on my machine, so I ran a docker instance of NodeJS to verify all of the installation instructions

P.S The docker instance is really more useful than using a VM in these instances, if you want to set it up, you can use the following command

docker run --rm -it node:latest /bin/bash

this will open up a bash terminal to an instance that has node installed locally

when I installed this, what I found was the binary was not being included in the dialectMorph package , which was bad, so I had to make fixes for the same and publish my package again.

User Testing Insights

One of my friends tested the total installation process out and the following was found from his experience:

Installation Confusion: Users were unclear about the difference between global and local installation. This led to adding clearer installation instructions:

npm i -g dialect-morph  # for global installation

API Key Configuration: Users struggled with setting up API keys. We added multiple configuration options:
- Environment variables
- TOML configuration file
- Command-line arguments

Current Installation and Usage

DialectMorph can now be installed globally via NPM:

npm i -g dialect-morph

Basic usage:

dialectMorph ./path/to/file.py -l java  # Convert Python to Java
dialectMorph ./path/to/file.cpp -l python  # Convert C++ to Python

For configuration, users can either:

Create a ~/.dialectMorph.config.toml file
Use command-line arguments
Set environment variables

Full documentation is available on my GitHub repository.

Conclusion

This release process has been a valuable learning experience, highlighting the importance of thorough documentation, user testing, and flexible configuration options. The NPM ecosystem provides an excellent platform for distributing CLI tools, and my friend's feedback has been invaluable in improving DialectMorph.

CI and Dev Containers

Kannav Sethi — Mon, 18 Nov 2024 04:58:55 +0000

Introduction

For this week of contribution, I planned to implement a CI pipeline that will automate all of the checks made to any of the changes pushed to my main branch directly or via a PR

In addition to this, since my project is becoming bigger and is open-sourced, I believe more people are going to contribute to this, so I also added the option to have dev-containers for the ease-of-use of a developer and better dev experience.

Again, all of this work is being done on my project, DialectMorph

CI Pipeline

There are a lot of tools that could help with making CI/CD pipelines, but for this project, I decided to go with Github Action, this was because I was already familiar with it and it would be a perfect fit for my project

Github Actions work on YAML Scripts, these are nothing but configuration files, similar to how JSON is used to provide configuration settings. YAML is much preferred in the field of DevOps to make configuration easy and human-readable

Setup

For my project, I have used the following configuration setting

name: Linting and Formatting
on:
  push:
    branches:
      - main
      - master
  pull_request:
    branches:
      - main
      - master
jobs:
  lint:
    name: Prettier and ESLint
    runs-on: ubuntu-latest
    steps:
      - name: Checks Out the Repository
        uses: actions/checkout@v4
      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 20
      - name: Sets up the Bun enviornment
        uses: oven-sh/setup-bun@v2
        with:
          bun-version: "latest"
      - name: Installing Bun Dependencies
        run: |
          bun install .
          bun install -D jest ts-jest @types/jest typescript
      - name: Linting The Code
        run: |
          bun eslint .
      - name: Checking for Formatting
        run: |
          bunx prettier . --check
      - name: Running Tests
        run: |
          npm test

in the Github Actions config file, three things are necessary for a valid configuration which are

name: Informs the name of the script
on: Event that triggers the CI/CD pipeline
jobs: Tasks that would run when the event is triggered

My specific configuration is as follows:

Push or Pull Requests to the Main Branch trigger the CI script
Once the event is triggered, the first command sets up the environment to run on an Ubuntu virtual machine
Then the repository is cloned on the VM and bun installs dependencies and runs the project
The project is checked for any listing errors
The project is then checked for consistent formatting based on the prettier rules that are in the project
At last, the test-suite is run to check for any errors occurring during the changes that are coming from a different branch

Github actions helped me automate all of the checks that I would have to do manually just to ensure no major errors were occurring in my codebase and no tech debt was being stacked on.

Dev Containers

Dev-Containers, are one of the best tools for developers collaborating on a project and wanting to have a consistent platform to keep up with development, ignoring the dependency issues on each of their local machines.

Env Setup

To get this setup for my project, I had to provide the developer with the option to develop inside a container, which meant I had to configure the settings for my dev-container

My project utilizes bun as the main package-handler tool, and it is not that easy to set it up on different machines, so I had to provide a Dockerfile as the base for my dev-container

for my Dockerfile, I utilized the following settings

FROM --platform=$TARGETPLATFORM ubuntu:latest

RUN apt-get update && apt-get install -y curl unzip git

RUN curl -fsSL https://bun.sh/install | bash
ENV PATH="/root/.bun/bin:${PATH}"

this ran the container in an Ubuntu image, it installs git curl and unzip as the machine dependencies, so that the user shouldn't have the issue of installing them manually

then it sets up the bun package manager on the container for

now for the actual dev-container.json configuration , I have used the following settings

{
    "name": "Dev Container For DialectMorph",
    "build": {
        "dockerfile": "Dockerfile"
    },
    "customizations": {
        "vscode": {
            "settings": {
                "window.zoomLevel": 0.5,
                "workbench.tree.indent": 20,
                "editor.minimap.renderCharacters": false,
                "files.autoSave": "afterDelay",
                "files.autoSaveDelay": 1000,
                "editor.cursorBlinking": "smooth",
                "files.eol": "\n",
                "files.trimTrailingWhitespace": true,

                "editor.defaultFormatter": "esbenp.prettier-vscode",
                "editor.formatOnSave": true,

                "prettier.trailingComma": "es5",
                "prettier.semi": true,
                "prettier.tabWidth": 2,
                "prettier.useTabs": true,
                "prettier.printWidth": 80,

                "editor.codeActionsOnSave": {
                    "source.fixAll.eslint": "always"
                },
                "eslint.validate": ["typescript"]
            },
            "extensions": [
                "dbaeumer.vscode-eslint", // ESLint support
                "esbenp.prettier-vscode", // Prettier formatting
                "ms-vscode.vscode-typescript-next", // TypeScript support
                "aaron-bond.better-comments", // Better code comments
                "christian-kohler.path-intellisense", // Path autocompletion
                "streetsidesoftware.code-spell-checker", // Spell checking
                "eamodio.gitlens", // Git integration
                "yoavbls.pretty-ts-errors", // Better TypeScript errors
                "wayou.vscode-todo-highlight", // Highlight TODOs
                "gruntfuggly.todo-tree" // Track TODOs
            ]
        }
    }
}

in this, I have provided the base of my dev-container to be the Dockerfile and then added the recommended vs code setting and extensions which would help the user in the development

Conclusion

I successfully implemented a CI pipeline using GitHub Actions for my DialectMorph project, automating all the necessary code checks and testing processes. As my project grows, I wanted to ensure quality and consistency, which led me to configure the pipeline to run on both direct pushes and pull requests to the main branch. To make it even more contributor-friendly, I added dev container support with custom VS Code configurations, making it significantly easier for other developers to jump in and start contributing without worrying about environment setup

Testing with Jest

Kannav Sethi — Mon, 11 Nov 2024 03:23:16 +0000

A Guide to Testing in TypeScript Projects: Overcoming Challenges with Jest and Bun

In this blog post, I’ll walk you through my experience setting up and using Jest for testing in a TypeScript project. Along the way, I’ll share the hurdles I faced, particularly when trying to use Jest with Bun (my preferred package manager), and how I handled mocking LLM (Large Language Model) responses.

Again I will be demoing this on my project, DialectMorph

Using Jest

Framework

When it comes to testing, Jest is one of the most popular frameworks for JavaScript and TypeScript projects. I chose Jest because:

Ease of Setup: Jest works out of the box with minimal configuration for most projects.
Built-in Mocks and Spies: Jest has built-in support for mocking and spying, which makes it easy to isolate parts of your code, especially when working with external dependencies.
Snapshot Testing: Jest provides the ability to perform snapshot testing, useful when dealing with complex objects.
Active Ecosystem and Community: With a large community and lots of plugins, Jest offers solid support and continuous improvements.

For more information, you can visit Jest’s official documentation: Jest.

Bun: The Package Manager

I use Bun as my package manager for managing dependencies in my TypeScript projects. It is known for its speed and simplicity. However, I encountered a significant issue with Bun and Jest compatibility that I’ll elaborate on later. You can learn more about Bun at Bun.

Setting Up the Project

Initial Setup

To get started, I first installed Jest, TypeScript, and related dependencies:

bun add -d jest ts-jest @types/jest

Adding Bun to the Mix

Since I use Bun as my package manager, I ran into issues while trying to use Bun with Jest, as Bun doesn't natively support Jest out-of-the-box. This led to several frustrating days of trying to resolve compatibility problems. After many attempts at using bun test for Jest, I eventually had to work around it by running Jest directly using npm(even though Bun was the default package manager).

I used Bun for dependency management and npm for testing. This workaround helped resolve the Jest compatibility issue but resulted in additional setup time and confusion.

bun install
npm test

Mocking LLM Responses

One of the key challenges in testing is handling external dependencies, especially when dealing with large language models (LLMs) or API calls. In my project, I needed mock responses from LLMs interacting with the file system.

To achieve this, I used Jest’s powerful mocking abilities. I mocked several modules (fs, os, path, and toml) to isolate the functions under test. For example:

jest.mock("fs");
jest.mock("os");
jest.mock("path");
jest.mock("toml");

This allows me to simulate different scenarios without actually interacting with the file system or external dependencies.

Writing Test Cases

File Utility Functions

I had a set of utility functions to test, including file operations like creating directories and files. Here's how I structured the test cases for one of the functions, makeDir:

describe("makeDir Function", () => {
  it("Creates a Directory and returns its path", () => {
    const result = makeDir("testDir");
    expect(fs.mkdirSync).toHaveBeenCalledWith("/root/testDir", { recursive: true });
    expect(result).toBe("/root/testDir");
  });

  it("Doesn't create directory if it already exists", () => {
    (fs.existsSync as jest.Mock).mockReturnValue(true);
    const result = makeDir("testDir");
    expect(result).toBe("/root/testDir");
    expect(fs.mkdirSync).not.toHaveBeenCalled();
  });
});

Each of the utility functions (makeDir, createFile, extractCodeBlock, loadTomlConfigFile) had corresponding test cases that mocked the necessary parts of the code (like file reading and writing) to avoid side effects.

What I Learned from Writing Test Cases

Writing test cases helped me discover several interesting things:

Mocking complexity: Mocking certain modules, especially system-level ones like fs, os, and path, proved tricky. However, Jest’s jest.mock function made this easier once I figured out how to properly mock these external dependencies.
Mocking non-trivial dependencies: One of the most challenging parts of mocking was the toml parsing, as it involved simulating both file reading and parsing logic. But with proper mocks, I could easily simulate various file contents.
Test organization: I found that grouping related tests together (as seen in the nested describe blocks) kept my tests organized and made them easier to read and maintain.

Issues

Jest and Bun compatibility issues: The biggest obstacle I encountered was integrating Jest with Bun. This issue caused a lot of wasted time, and I had to use npm instead of Bun for running tests. This was a frustrating experience and a learning moment about the importance of checking compatibility early in the process.
Mocking non-obvious behaviors: Another moment of realization was learning how to mock file system behaviors, especially with Jest's ability to mock implementation details like file reads and writes. This was crucial to avoid side effects in tests.

Uncovered Bugs and Edge Cases

While writing tests, I discovered some edge cases in the file utility functions. For instance:

Handling non-existent files: There were scenarios where the code didn't handle the absence of files correctly. By testing this behavior, I was able to ensure that the code didn’t break and returned expected results.
Unexpected behaviors with path resolution: By mocking the path module, I ensured that the directory and file paths were correctly resolved across different environments.

Conclusion

This process has been an eye-opener in terms of testing and mocking in JavaScript/TypeScript. Prior to this project, I hadn’t done much testing, and it was rewarding to see how structured testing could catch bugs and ensure code correctness.

I believe personally, I would rather avoid writing test cases at least after most of the code is written as it gets too complicated, for me If I know I have to write tests, I would rather begin with a TDD approach

Local Dev Environment and Static Code Analysis

Kannav Sethi — Fri, 01 Nov 2024 18:58:23 +0000

For this week, I decided to improve the developer-friendliness nature of my project, DialectMorph. I took on the responsibility to automate the various setup procedures that a developer needs to get the project running locally on their machine,I will now elaborate the use-case of each of them in this blog

Code Formatting

Given that my codebase is open-sourced, a lot of developers might contribute to it. This sometimes leads to the having inconsistent code-formatting, which might not be a good thing when it comes to providing a clean and smooth developer experience

So for this, I utilized Prettier, which is a library used to support the task of formatting

Overview

This tool is used to specify the formatting options which would remain consistent throughout the codebase,
All of the configuration settings for the same go into a config file called .prettierrc or any other supported file formats the tool provides which can be found in the documentation.

For my codebase, I have applied the following settings which would be tailored to my project

File-Name: .prettierrc

{
    "trailingComma": "es5",
    "semi": true,
    "tabWidth": 2,
    "printWidth": 80,
    "useTabs": true
}

the options here provide me with the following functionality
having trailing commas whenever it is included near the es5 syntax
semicolons at the end of every statement
Sets the number of spaces per indentation level to 2
Limits line length to 80 characters.
It uses tabs for indentation instead of spaces

Now I would also want to ignore certain files or certain code blocks to not be formatted, for that use-case I have two options that is

Using a file called .prettierignore to include all the files that I want to ignore
I can also place the statement called prettier-ignore before the code blocks I don't want to be formatted

My initial formatting with prettier changed about 500 lines in my codebase

I have configured the following scripts in my package.json to have ease-of-use to call this tool

        "format": "bunx prettier . --write",
        "format:check": "bunx prettier . --check",

Linting

Linting is one of the common practices used in codebases to ensure the code is more consistent and to identify patterns that could lead to potential bugs

For my project, I utilized the tool called ESLint since it is compatible with typescript based projects

Overview

I used this tool to make sure that there was no errors or unused variables in my projects, also I had a couple of configuration settings as well

import globals from "globals";
import tseslint from "@typescript-eslint/eslint-plugin";
import tseslintParser from "@typescript-eslint/parser";

export default [
    {
        files: ["**/*.{js,mjs,cjs,ts}"],
        languageOptions: {
            globals: {
                ...globals.browser,
            },
            parser: tseslintParser,
            parserOptions: {
                ecmaVersion: "latest",
                sourceType: "module",
            },
        },
        plugins: {
            "@typescript-eslint": tseslint,
        },
        rules: {
            ...tseslint.configs.recommended.rules,
        },
    },
    {
        files: ["dist/**/*.js"],
        rules: {
            "@typescript-eslint/no-unused-expressions": "off",
            "@typescript-eslint/no-require-imports": "off",
        },
    },
    {
        files: ["examples/**/*.js"],
        rules: {
            "@typescript-eslint/no-unused-expressions": "off",
            "@typescript-eslint/no-require-imports": "off",
            "@typescript-eslint/no-unused-vars": "off",
        },
    },
    {
        files: ["src/index.ts"],
        rules: {
            "@typescript-eslint/no-explicit-any": "off",
        },
    },
];

This configuration file manages linting rules for my project through multiple configuration blocks. The primary block sets up basic TypeScript linting for all code files with browser globals and recommended rules, while subsequent blocks relax specific rules for distribution files and examples. Finally, it includes a special configuration for src/index.ts that allows the use of the any type.

I have configured the following scripts in my package.json to have ease-of-use to call this tool

"lint": "bun eslint .",

Local Dev Environment

I wanted to make sure that the developer that is working on my repository has a smooth experience while setting up my repository.

I have been working on my codebase in VSCode mostly, so I decided to include a .vscode folder which would allow me to specify the necessary things needed for a smooth experience

`extensions.json` file

This file contains the name of all the extensions that would provide a better experience to the developer while they are working on my repository, when the users open ups my repository for the first time,
they will see the prompt to install the recommended extensions, for my program you can see the recommended extensions as follows

{
    "recommendations": [
        "dbaeumer.vscode-eslint", // ESLint support
        "esbenp.prettier-vscode", // Prettier formatting
        "ms-vscode.vscode-typescript-next", // TypeScript support
        "aaron-bond.better-comments", // Better code comments
        "christian-kohler.path-intellisense", // Path autocompletion
        "streetsidesoftware.code-spell-checker", // Spell checking
        "eamodio.gitlens", // Git integration
        "yoavbls.pretty-ts-errors", // Better TypeScript errors
        "wayou.vscode-todo-highlight", // Highlight TODOs
        "gruntfuggly.todo-tree" // Track TODOs
    ]
}

`settings.json`file

The file contains all the editor settings for VSCode, that again is not necessarily needed but is again used for a great and seamless experience for the developer. I believe if you want people to work on your codebase, automate the manual setup things and provide them with a great experience.

This settings.json file is particularly used for specifying the setting for VSCode editor and environment

I have provided the following settings for the same

// arbitary settings for vs code to have good config settings

{
    "window.zoomLevel": 0.5,
    "workbench.tree.indent": 20,
    "editor.minimap.renderCharacters": false,
    "files.autoSave": "afterDelay",
    "files.autoSaveDelay": 1000,
    "editor.cursorBlinking": "smooth",
    "files.eol": "\n",
    "files.trimTrailingWhitespace": true,

    "editor.defaultFormatter": "esbenp.prettier-vscode",
    "editor.formatOnSave": true,

    "prettier.trailingComma": "es5",
    "prettier.semi": true,
    "prettier.tabWidth": 2,
    "prettier.useTabs": true,
    "prettier.printWidth": 80,

    "editor.codeActionsOnSave": {
        "source.fixAll.eslint": "always"
    },
    "eslint.validate": ["typescript"]
}

Git Hooks (Pre-Commit)

Git-Hooks are scripts that automatically run at a specified timeline in the git execution pipeline, they are triggered at different events like having a pre-commit hook, a post-commit hook. They are really useful to perform additional sanity testing task or basic cleaning tasks before the data is pushed or committed

For my use-case I have only utilised pre-commit hook as I didn't see the use of having other hooks right now in my project

To implement these hooks I have utilised another library called husky

How this works is you can specify the type of git events you want and what command should run when the git event occurs,for my package.json I have the following settings


    "husky": {
        "hooks": {
            "pre-commit": "bun run format"
        }
    },

the above command specifies that there is only a pre-commit event that would run the command to format my entire code before committing it.

This is by far one of the best things I have come across recently.

Conclusion

Through the implementation of tools like Prettier, ESLint, VSCode configurations, and Git hooks, I've significantly reduced the manual setup burden for potential contributors. These improvements not only make the codebase more maintainable but also create a welcoming environment for developers who might want to contribute to the project in the future.

My Fourth and Final Contribution to Hacktoberfest

Kannav Sethi — Mon, 28 Oct 2024 02:36:49 +0000

My 4th and final contribution to Hacktoberfest was made to a repository that I have contributed to in the past, DiceDB. The project is centered around the idea of being an in-drop replacement of Redis while being more optimized than it.

Issue

Previously I had contributed to this repository by solving an issue that was related to testing out a particular Database Operation ensuring the functionality aligns with the documentation and if needed making changes to the documentation.

This time around, I wanted to contribute to the codebase, part of the reason being I wanted to learn GoLang and the other being getting a gist of how tools/software like DiceDB are built, what is the intuition to such a good design.

So I found an issue that was not too hard to grasp initially but would also help me achieve the goals I mentioned earlier.

The issue was centered around the idea of resolving and fixing a certain error message and giving the right error message in an appropriate situations

Process of finding and correcting the error

The idea was to first find the issue and then resolve it

Since I had pretty much no knowledge of how GoLang works, I first went to the docs for the same, familiarized myself with the first principles and the syntax of GoLang, and then went back to solve the issue

Since this project was vast, I utilized VSCode's search functionality to find where the error was being logged, and what functions were associated with the error message.

After I found the Point-of-Interests for me, what I did was to isolate the issue from the other parts of the codebase, made the current logic make sense to me, separate the error message, and then add the fixes for the same.

PR

After I had resolved the issue, I opened a PR to get my changes approved and merged into the main branch, but turns out someone else had there changes merged into the main, so there was conflicts

I then had to use the rebase command to rebase my topic-branch on top of the new changes that were merged into main


git rebase main

After doing so, there were no conflicts and soon the PR was approved by the maintainer to be merged into the repo

Conclusion To Hacktoberfest

In wrapping up my Hacktoberfest contributions, working on DiceDB allowed me to dive deeper into Go and learn firsthand how a Redis alternative is built. Resolving an error-handling issue helped me grasp Go’s syntax and debugging in a real-world codebase.

This entire experience shaped how the dilemma that I used to have with contributing to OSS projects, it gave me one of the best skills that any developer could have i.e. adaptability as during my experience with Hacktoberfest, Almost all of my contributions have been to projects that have used a tech-stack that I've never had any experience with

I have loved each and every aspect of Hacktoberfest!

Hacktoberfest contribution to ZTM-Quest

Kannav Sethi — Wed, 23 Oct 2024 04:03:39 +0000

Introduction

For my third contribution to Hacktoberfest, I found a repository that I never thought would be the one that I would contribute to, this repository was particularly centered around 2D-Game Dev in JavaScript since I already had a working knowledge of JavaScript, diving and exploring the codebase wouldn't have been a problem so I picked up one of the issues in this repository to contribute to.

ZTM-Quest

The repository was named ZTM-Quest

Issue

The issue that I picked up to solve was related to adding a end-credits scene to the game which would dynamically fetch the name of the contributors from the Github API and would also fetch the asset credits as well and display them in a scrolling manner, something similar to how normal 2D games would do nowadays.

Further features which were to be included were

Adding a button to trigger the credits scene and to exit the scene
Providing the options via the keyboard to exit/trigger the credits scene

KAPLAY

The entire program used the KAPLAY framework, which is a framework that can be used in Javascript to make 2D games

The whole abstract idea behind this is to have various components or characters that can interact with each other to trigger interactions. The background image that a developer could provide in the program is known as a sprite here, it could even be a map where the character or the components could be placed and can lead to different interactions

What I found was most of 2D-Game Dev involved manipulating objects geometrically in a 2D plane with the added features of interaction and dynamic resizing

Core Changes Made

The core changes that I made over here were as follows

Intuition

The intuition that followed this was that the program would have 3 components, a background, a text container or text itself, and an exit button

My first iteration of changes involved making those but later on I realized it would be better to have a sprite in the background as well, so I made the following changes

    const background = k.add([
        k.sprite("backgroundImage", { width: k.width(), height: k.height() }),
        k.pos(0, 0),
        k.opacity(0.2),
        k.scale(1),
        k.z(60),
        k.fixed()
    ]);


    const overlay = k.add([
        k.rect(k.width(), k.height()),
        k.pos(0, 0),
        k.color(0, 0, 0),  // 40% transparent black
        k.z(50),  
        k.fixed()
    ]);

    // Add the credit text
    const center = k.vec2((k.canvas.width / 2), k.canvas.height / 2);
    const text = k.add([
        k.text(creditText, {
            size: 15 / camScale,
            width: k.width() * 0.9 / camScale,
            lineSpacing: 15 / camScale,
            align: "center"
        }),
        k.pos(k.width() / 2, k.height()),
        k.anchor("top"),
        k.color(255, 255, 255),
        k.z(101),
        k.scale(camScale)
    ]);

    const crossButton = k.add([
        k.rect(65, 35),
        k.color(255, 0, 0),
        k.anchor('topright'),
        k.z(101),
        k.area(),
        k.scale(1),
    ]);

    const exitText = k.add([
        k.text("Exit", { size: 26 / camScale }),
        k.color(255, 255, 255),
        k.anchor("topright"),
        k.z(102),
        k.scale(camScale)
    ]);

as can be seen, I put an overlay as well to make the image be less in focus than the text itself

as for the text that is to be displayed I made two functions that would fetch the data for me, which then I could put in the creditText field

export async function getAssets(){
    const fileSHA = await fetch("https://api.github.com/repos/zero-to-mastery/ZTM-Quest/contents/asset_credits.md")
    const data = await fileSHA.json()
    const fileContents = atob(data.content)
    return fileContents.split("\n").filter(content=>content.trim()!=='').join("\n")
}

export async function getContributors(){
    const contributors = await fetch("https://api.github.com/repos/zero-to-mastery/ZTM-Quest/contributors")
    const data = await contributors.json()
    return data.map((person)=>{
        return person.login
    }).join("\n")

}

I believe that whenever an API is being requested for the data, 90% of the task is to transform the data so that the developer can utilize it optimally, that's what I did here.

My Experience With the Repo

To be honest, this is one of the most interesting issues I have worked on before I never knew how to work with 2D-GameDev frameworks in JavaScript, this gave me the much-needed experience, and I enjoyed contributing to this

In fact one of the other things that I have learned while contributing to this was the proper usage of git rebase and git checkout -B

These commands literally saved the day for me,

I utilized git rebase frequently to make my issue branch in sync with the changes being merged into the main branch upstream and to avoid conflicts as well

I utilized git checkout -B when I used to mess up the code on the branch and wanted to go back to the latest working commit

I usually used to do the following


git checkout -B issue-branch latest-working-commit

Conclusion

This is my 3rd Hacktoberfest contribution and I am loving contributing to Open-Source, the amount of hands-on experience I have gained this time around is unbeatable

My Hacktoberfest Contribution To DiceDB

Kannav Sethi — Thu, 10 Oct 2024 19:40:33 +0000

This week I had the opportunity to work on DiceDB which is a drop-in replacement of Redis and is much faster than it as well

Background

Redis is an open-source in-memory database, that can be used for the purpose of storing data, caching, or as a message broker

DiceDB improves on the benchmarks and also differs in two aspects from Redis i.e.

It is multi-threaded
Listens to SQL query and informs the client about the changes as soon as possible

My Contribution

Issue

I saw an issue that was related to auditing the documentation concerning the PFMERGE command

To briefly describe what the issue was :

The documentation for the PFMERGE command might have become stale, audit and fix it
As the tool is a drop-in replacement of Redis, check if the functionality of the command matches the functionality mentioned in the Redis documentation
Make the documentation for the command consistent with the new proposed format that is to make appropriate use of headers, add terminal examples and use proper table format for arguments and error output types

My Approach

I've been wanting to learn Redis for a while, but have never taken the time to do so, this issue helped me learn more about how Redis operates and how DiceDB is different from it

To get myself acquainted with the technology, I went to the official documentation page of Redis, read through the documentation, different data types, and quickstart guide, and then ran a Docker container for the same on my local machine to experiment with it

After I was done exploring Redis, I moved on to get DiceDB working on my machine, most of the commands that I ran in DiceDB ran through the help of redis-cli tool, which allowed me to connect to the Docker instance of DiceDB running on my machine

`PFMERGE`

I was acquainted with common data types like, JSON, String, Sets and Lists

But this time around I came across a new Data Type, known as HyperLogLog, which estimates the cardinality of the elements in its set, this was a type of a probabilistic data structure

Any command that started with a PF probably dealt with the use of HyperLogLog Data structure, my task at hand was to see how PFMERGE was performing and how is it giving outputs and handling error

Some of the examples I ran here were

127.0.0.1:7379> PFADD hll1 "a" "b" "c"
(integer) 1
127.0.0.1:7379> PFADD hll2 "c" "d" "e"
(integer) 1
127.0.0.1:7379> PFADD hll3 "e" "f" "g"
(integer) 1
127.0.0.1:7379> PFMERGE hll_merged hll1 hll2 hll3
OK
127.0.0.1:7379> PFCOUNT hll_merged
(integer) 7

127.0.0.1:7379> PFADD hll_merged "x" "y" "z"
(integer) 1
127.0.0.1:7379> PFMERGE hll_merged hll1 hll2 hll3
OK
127.0.0.1:7379> PFCOUNT hll_merged
(integer) 7

127.0.0.1:7379> PFMERGE hll_merged hll1 hll2 non_existent_key
OK
127.0.0.1:7379> PFCOUNT hll_merged
(integer) 5

all of these examples gave me an overview of how this command worked

Making Changes

Once I was done with all of the pre-requisite stuff, I finally went on to audit the documentation and my changes were as follows

Changed the terminal to reflect the correct port number being used to access the docker instance
Added another example, in the example usage section demonstrating invalid usage of the command
Converted the formatting of the Return Values and Parameters sections to a table format
Modified the expected behavior to match the functionality of how the command was working

A full descriptive view of my PR can be found here

Conclusion

Contributing to DiceDB this week provided me with valuable insights into in-memory databases and the HyperLogLog data structure. By auditing and updating the documentation for the PFMERGE command, I ensured that DiceDB's documentation remains accurate and user-friendly

Forem: Kannav Sethi

Reflecting on the ORAssistant Project: A Journey of Collaboration and Technical Growth

Introduction

Final Thoughts

Challenges Faced

The Community That Saved Me

What I Learned (Beyond Code)

Conclusion

Progress on Automatic Feedback Loop

Introduction

Integrating the MongoDB function for feedback

Fork Issue

Conclusion

Planning To Resolve Automatic Feedback Loop

Introduction

Why this Repo

Issue and My Approach

Conclusion

Contributing to ChatCraft

Adding API Key Management for Non-LLM AI Providers in ChatCraft

Introduction

The Challenge

Implementation Process

1. UI Modifications

2. Provider Architecture

3. Jina.ai Integration

Key Learnings

Challenges and Solutions

Future Improvements

Conclusion

Contributing to ORAssistant

Introduction

ORAssistant

Issue

First Pull Request

Issues faced

Main solution

Further Contributions

Publishing my Tool

Introduction

Choosing the Package Registry

Release Process

1. Package Preparation

2. Build Process Configuration

3. Testing and Verification

Issues Encountered

User Testing Insights

Current Installation and Usage

Conclusion

CI and Dev Containers

Introduction

CI Pipeline

Setup

Dev Containers

Env Setup

Conclusion

Testing with Jest

A Guide to Testing in TypeScript Projects: Overcoming Challenges with Jest and Bun

Using Jest

Framework

Bun: The Package Manager

Setting Up the Project

Initial Setup

Adding Bun to the Mix

Mocking LLM Responses

Writing Test Cases

File Utility Functions

What I Learned from Writing Test Cases

Issues

Uncovered Bugs and Edge Cases

Conclusion

Local Dev Environment and Static Code Analysis

Code Formatting

Overview

Linting

Overview

Local Dev Environment

extensions.json file

settings.jsonfile

Git Hooks (Pre-Commit)

`extensions.json` file

`settings.json`file

`PFMERGE`