Forem: Ayush Deb

Research AI Assistant Application - RAG

Ayush Deb — Sat, 09 Nov 2024 21:14:50 +0000

This is a submission for the Open Source AI Challenge with pgai and Ollama

What I Built

I created an AI-powered research assistant application designed to make finding and understanding relevant research papers much easier. When you’re writing a research paper, it’s often a struggle to locate high-quality papers that match your topic closely. This app streamlines that process. It first searches and pulls up research papers closely aligned with your query, and then leverages LLM (Large Language Models) and RAG (Retrieval-Augmented Generation) to generate insightful summaries, suggested responses, and information based on your specific prompt. Not only does it provide summaries, but it also includes links, titles, and other essential details for each recommended paper, so you can get a comprehensive view of what's available. This tool essentially simplifies the process of gathering and interpreting research, helping you focus more on writing and analysis than on the hunt for resources.

Key Features : -

Relevant Paper Discovery: Quickly finds research papers closely related to your topic or query.
Intelligent Summarization: Uses Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) to generate clear, concise summaries for each paper.
Prompt-Based Response Generation: Provides insightful responses and context-specific information based on your prompt, helping you understand complex topics.
Link and Title Integration: Includes direct links, titles, and summaries of recommended papers, saving you time and ensuring easy access to source material.
User-Friendly Interface: Simplifies the research process, allowing you to focus on writing and analysis rather than sourcing information.

Demo

Here is my Github Link : - https://github.com/Ayush-developer/Reasearch_Assistant_RAG_Application

Tools Used

Timescale Cloud: Database hosting
pgai through TimescaleDB for AI
pgVector: For efficient data storage and retrieval of embeddings
Ollama: For generating embeddings and handling response generation using Mistral and LLama
React.js: Used for building the user interface
Flask API ; - For backend API Handling
Postman : - Testing Queries

Technologies Used : -

Ollama with Nomic Embeddings: Utilizes Nomic embeddings for accurate text representation, supporting semantic search and relevance scoring for research queries.
PostgreSQL with PG Vector and Psycopg2: Stores embeddings in PostgreSQL using pgvector, enabling efficient similarity searches, with psycopg2 managing database connections.
Llama Model via Ollama: Leverages the Llama model for natural language processing, generating precise responses and summaries based on user queries.
Flask API for Backend: Manages backend processing and data flow between the frontend and AI models, ensuring smooth interaction.
React for Frontend: Provides a responsive, intuitive interface for query input, viewing results, and accessing summaries.

Final Thoughts

This is my first time participating in a Dev.to Hackathon, and it’s been an incredible learning experience! Building this AI application has opened my eyes to so many new technologies, from working with Large Language Models and embeddings to managing backend integrations. I’m still figuring out deployment and improving the application’s functionality, so I’d love to connect with anyone willing to help or share advice. As a 4th-year student currently interning with React, this project marked my first foray into advanced AI and NLP, and I’m grateful for the opportunity to showcase my work and grow through this experience. Thank you for creating such an encouraging platform!

Prize Categories:
Open-source Models from Ollama: AI Bot leverages Ollama’s open-source models for embedding and understanding text-based queries.

Create a Full Stack Mern Social Media Application - Part 2

Ayush Deb — Mon, 14 Oct 2024 21:43:24 +0000

So it looks like this series is going to be long if I have to put every single detail here. But if it can help somebody I am up for it .

1. Configure the file storage on your index.js

FILE STORAGE

Stores files in the "public/assets" folder.
Keeps the original file name when saving the uploaded file.
Multer is configured to handle these uploads in your app.

2. Configure MongoDB Atlas

Create and configure a MongoDB Atlas server.

Create a .env file and put the mongodb url along with your password inside the url.

Put the port number

3. Create the Auth.js file

Use brcypt to hash the password and create the user with all credentials and details.
Use jsonwebtoken or 'jwt' for authentication
import user module from the userSchema made using mongoose.

Create a Full Stack Mern Social Media Application - Part 1

Ayush Deb — Thu, 10 Oct 2024 06:01:59 +0000

Hi Guys,

This is my first full stack applications to put all my skills to use and highlight this in my resume to crack a full time Software Developer Role. So this is how I started.

First we are creating our server then we will code our client side.

SERVER

Install all the Dependencies

How to : - npm - i express brcypt ....

express
body-parser
bcrypt
cors
dotenv
gridfs-stream
multer-gridfs-storage
helmet
morgan
jsonwebtoken
mongoose

Create a index.js file with importing all the dependencies

import express from "express";
import bodyParser from "body-parser";
import mongoose from "mongoose";
import cors from "cors";
import dotenv from "dotenv";
import multer from "multer";
import helmet from "helmet";
import morgan from "morgan";
import path from "path";
import { fileURLToPath } from "url";

Note: - fileURLToPath in Node.js is a function that takes a special kind of URL (called a file URL) and turns it into a regular file path that your operating system can understand.

For example, a URL like file:///C:/path/to/file isn't something you can directly use with file system functions in Node.js, but by using fileURLToPath, it becomes something like C:\path\to\file on Windows or /path/to/file on Linux

Adding the configurations

1. const __filename = fileURLToPath(import.meta.url);

What it does: This converts the file's URL (which is provided by import.meta.url) into a path that the operating system understands.

Why it's used: It helps you get the full file path of the current file, which is especially useful in ES modules where __filename isn't available by default.

2. const dirname = path.dirname(filename);

What it does: This extracts the directory name of the current file from the file path.

Why it's used: It allows you to know the directory where the current file is located, which is useful for working with relative paths (e.g., serving static files).

3. dotenv.config();
What it does: This loads environment variables from a .env file into process.env.

Why it's used: It helps you securely store configuration like API keys and database URLs in an external file without hardcoding them into your code.

4. const app = express();

What it does: This creates an Express application instance.

Why it's used: Express is a framework for building web servers and APIs. This line initializes the app so you can start adding routes and middleware.

5. app.use(express.json());

What it does: It allows the Express app to automatically parse incoming JSON requests (for example, when someone sends JSON data in a POST request).

Why it's used: This makes it easier to handle JSON data in your API or web server.

6. app.use(helmet());

What it does: Helmet is a middleware that adds security-related HTTP headers to your app.

Why it's used: It helps improve security by setting headers that protect against various web vulnerabilities like cross-site scripting (XSS) and clickjacking.

7. app.use(helmet.crossOriginResourcePolicy({ policy: "cross-origin" }));

What it does: This adds a policy to allow resources from your server to be accessed by external domains (cross-origin).

Why it's used: It provides a layer of control over what resources can be shared across different websites, while still maintaining security.

8. app.use(morgan("common"));

What it does: Morgan is a middleware for logging HTTP requests. Here, it’s set to log in the "common" format.

Why it's used: It helps track incoming requests for debugging, analytics, or monitoring purposes.

9. app.use(bodyParser.json({ limit: "30mb", extended: true }));

What it does: This allows the app to handle large JSON payloads (up to 30MB) in incoming requests.

Why it's used: If your app is expected to receive large JSON data (e.g., file uploads), you increase the limit so the app can process it.

10. app.use(bodyParser.urlencoded({ limit: "30mb", extended: true }));

What it does: This is similar to the previous line but handles URL-encoded data (like form submissions).

Why it's used: It allows your app to process form submissions with larger amounts of data.

11. app.use(cors());

What it does: CORS (Cross-Origin Resource Sharing) middleware enables your app to accept requests from other domains.

Why it's used: It's necessary if you're building an API or web app that will be accessed from a different domain, such as a front-end app hosted separately.

12. app.use("/assets", express.static(path.join(__dirname, "public/assets")));

What it does: This serves static files (like images, CSS, or JavaScript) from the public/assets directory at the /assets path.

Why it's used: It allows users or browsers to access these files directly by visiting http://yourdomain.com/assets/filename.