Forem: Captain jay

Hault - launch ai customer support with a click

Captain jay — Mon, 24 Jun 2024 06:47:33 +0000

This is a submission for the Twilio Challenge

What I Built

Hault is an open-source software that helps connect database that contains information about your business and FAQs that help you automate customer support using AI.

Introduction

The project is mainly a WhatsApp bot which you can scroll down and host via one click deploy on render by pressing Deploy on render and the user can talk to the application via WhatsApp to whatever number you have configured on twilio.
In my case, you can check the bot out right now by messaging me at (+13476958493)
Since the number is blocked because of WhatsApp verification problem you can check the website out and try the project to do so you will need to log in via Google and you will need to ask the admin permission via email at capja778@gmail.com
then head over to https://hault.vercel.app/login
I keep it as raw as possible so that the people using this project find it easy to host and edit the website too.
Here are some of the screenshots of the application

How to host the software for your business?

:1. First get the following authentication token and api keys from the following services

Google Gemini API key
Airtable account Table id, base id, api key
Twilio account sid , phone number and auth token

well actually i added 2 AI servuce that is gemini and workers ai which can be switched using LLM variable set to either GEMINIAI or WORKERAI.But for ease we will use gemini ai for now.

Same for database plan was to in corporate Mongodb too but we are only gonna use Airtable for now so set UI as AIRTABLE

For twilio you will need to create a whatsapp sender by purchasing a number, then create a service there you will choose the number, then in integration section choose webhook and add the call back url as

your_url+/api/v1/twilio/message

in status add

your_url+/api/v1/twilio/status

we will get the url after deployment

So overall you will need these values

LLM=GEMINIAI
UI=AIRTABLE
OWNER_EMAIL
OWNER_PHONE_NUMBER
TWILIO_PHONE_NUMBER
TWILIO_ACCOUNT_SID
TWILIO_AUTH_TOKEN
AIRTABLE_TABLE_ID
AIRTABLE_BASE_ID
AIRTABLE_API_KEY

Let everything else stay empty or add "123" as a dummy value when inputting in the deployment.

:2. Go to my repo https://github.com/captain0jay/Hault
and click the deploy on render button

:3. Then you will be redirected to render website where the values we got will be inputted

:4. Then just press launch button and your application will be live and your customers can start messaging and take the benefit of AI customer support

:5. One last step as we discussed earlier that in console.twilio.com -> messaging -> service -> your_created_service -> integration

the call back url as

your_url+/api/v1/twilio/message

in status add

your_url+/api/v1/twilio/status

here add your_url as the one that you got in hosted application see below image as reference

Demo

So you can check the bot out right now by messaging at (+13476958493) and ask about Hault that is the project itself.
Since the number is blocked because of whatsapp verification problem you can check the website out and try the project to do so you will need to login via Google and you will need to ask the admin permission via email at capja778@gmail.com
then head over to https://hault.vercel.app/login

Below is the repo

captain0jay / Hault

Hault is an AI-powered platform designed for businesses and professionals to easily showcase their products and services using AI. With intuitive database management and seamless WhatsApp integration, Hault enhances customer engagement and automates support using advanced AI technologies like Gemini and Workers AI.

Hault

AI Customer support and buisness launch software

Host it online

Install locally
·
Request Feature

Table of Contents

Introduction

The project is mainly a WhatsApp bot which you can scroll down and host via one click deploy on render by pressing Deploy on render and the user can talk to the application via WhatsApp to whatever number you have configured on twilio.
In my case, you can check the bot out right now by messaging me at (+13476958493) Since the number is blocked because of WhatsApp verification problem you can check the website out and try the project to do so you will need to log in via Google and you will need to ask the admin permission via…

View on GitHub

Tech stack

For backend - Nodejs
For frontend - Nextjs, Shadcn
Main services - Twilio
AI - Gemini
Database - Airtable

Twilio and AI

Twilios whatsapp and SMS functionalities were very helpful in making the software as the code required is very minimal

I connected airatble database and fetched the details from the database feeding it to Gemini AI and then final step was to give response back to twilio after webhook is called and all it took for twilio to connect is 5 lines of code

router.post('/message', async (req, res) => {

    const userMessage = req.body.Body || "hi";
    const twiml = new MessagingResponse();

    let response = await getChat(userMessage);


    twiml.message(response);
    res.type('text/xml').send(twiml.toString());
});

Additional Prize Categories

My submission qualifies for Twilio Times Two, Impactful Innovators

Thanks for reading till the end :)

CF-assist - Visual smart assistant for visually impaired

Captain jay — Mon, 15 Apr 2024 05:41:02 +0000

This is a submission for the Cloudflare AI Challenge.

What I Built

CF-assist is a visual assistant that helps visually impaired individuals get visual insights via the application with just a click and hold action. Users can hold and talk to the assistant, and it will help them understand what the image is. The image can be from their camera, constantly running when the app is open, or users can talk to the LLM via the application.

How to use

There are 2 modes which can be toggled from the navbar:

Text mode
- Type and send to talk to LLM normally.
- Click an image by clicking the photo button, and the very next text you send can be about the image.
- Drag and drop or choose a file, then send the next text about the image to get insight.
CF-assist mode
- Hold and speak, then leave after a short interval; the assistant will give you the result via audio. (Important: hold the line at the bottom or a few cm outside the area; that's the area where the application picks up a hold movement. The same goes for a click; click the line; it can detect a few cm off clicks too. A detailed video will be uploaded tonight.)
- Click and hold when the user wants to know what's happening in their background and get visual assist. First click the line, then hold the line, speak, and leave after a short time; the assistant will respond via audio.

My Code

Here is the GitHub code: cfassisttesttwo

Installation

Clone the repository:
```
git clone <repository_url>
```
Navigate to the project directory:
```
cd cfassist
```
Install dependencies:
```
npm install
```
Navigate to the server directory:
```
cd server
```
Install server dependencies:
```
npm install
```
Return to the project directory:
```
cd ..
```
Start the development server:
```
npm run dev
```

Environment Variables

Server:

Create a .env file in the server directory.
Obtain CLOUDFLARE_API_TOKEN and CLOUDFLARE_APP_ID from your Cloudflare account.

Add the following lines to the .env file:

CLOUDFLARE_API_TOKEN=<your_cloudflare_api_token>
CLOUDFLARE_APP_ID=<your_cloudflare_app_id>

Client:

Create a .env file in the cfassist directory.
Define NEXT_PUBLIC_SERVER_URL with the server URL. For local development, use:
```
NEXT_PUBLIC_SERVER_URL=http://localhost:4000
```

Note: Ensure that all environment variables are correctly set before running the application.

Demo

You can check the website right now
cfassisttesttwo.pages.dev

(Disclaimer: While testing it on other devices and browser the permission of using sound doesn't seem to work well on mobile devices, for optimal experience of using cfassist mode on full potential I recommend using the website on only desktop devices or devices which gives more option on audio and camera permission without explicitly changing it)

Meanwhile, you can look at the images of the running application:

Journey

I started this application using the Next.js framework as the base and TypeScript as the language of choice for both frontend and backend. The models I was particularly interested in for making my application are:

Automatic speech recognition
Image to text
Text generation

For immediate speech-to-text service, I used browser transcribing service, which is faster as it runs on the user's device, but it is not as accurate as Cloudflare AI model. So, I used both, browser one for immediate transcribing and Cloudflare AI for final transcription.

As of now, no login method is implemented, and it's free to use; everything is stored in local storage, ensuring the lowest latency.

I made this application in such a way that it's easy for visually impaired individuals or people without any application knowledge to easily access the application; it's just click, hold, speak, and get visual assistance for anyone in need.

Glam up my markup challenge - Camp activities, how good could I possibly make it using just plain HTML, CSS, JS???

Captain jay — Sun, 31 Mar 2024 18:15:38 +0000

This is a submission for DEV Challenge v24.03.20, Glam Up My Markup: Camp Activities

What I Built

I built absolutely nothing and didn't even bother adding elements. Needless to say, I took the plain HTML template and challenged myself as to how good-looking can I possibly make the form.

So I landed on the idea of making the form multipage using javascript without actually changing the HTML, which was completed made a changing image and colour background that together blends and transitions perfectly to make this amazing form...

Demo

Journey

Well it all started when I was little .... jk XD
I learned how to manage my timing and do a project fast as I only had less than a day or so due to my examinations going on.

Also I learnt a new way to make slides without touching the HTML itself so that was something.

Building the animation on transition was also fun you can see it sliding when you press next or back really small detail but worth adding for my inner peace.