Forem: Chinwe O.

Vacation Planner Application using AWS

Chinwe O. — Mon, 12 Feb 2024 01:04:37 +0000

Traveling is a popular and enjoyable activity for people all over the world. I've been going on some spontaneous trips lately, and it's been tough to plan for them properly. So, I started working on a little side project to help with that. It's still in the early stages, but it could turn into something big eventually.

Let's start building! The main goal is to create a web app that gives users estimated flight costs, recommended travel dates, and hotel choices for a specific destination, all tailored to their budget.

Leveraging AWS for Vacation Planning Application

Technologies and Services

Frontend: HTML, CSS, JavaScript using AWS Cloud9 (I used MaterialUI for the design)
Backend: AWS Lambda for serverless backend logic.
Database: Amazon DynamoDB to store any necessary data like user preferences or historical data.
APIs: Integrate third-party APIs for flight data, hotel listings, and travel recommendations. Hosting: Amazon S3 for hosting the static website, Amazon API Gateway for RESTful API.

Key Features of an Vacation Planner App

Destination Input: users can input their desired vacation location.
Travel Date Suggestions: the tool suggests the best times to visit based on factors like weather, local events, or historical data.
Flight Cost Estimates: this will provide estimated flight costs for the suggested dates.
Hotel Recommendations: to list the different hotel options in the destination area.
Budget: users can input their budget and the app will provide recommendations within that price range.

Frontend Development

I had to create a basic web app where users can enter their vacation preferences and use CSS Framework (with MaterialUI) to manage user inputs and show results.

Step 1: Set Up AWS Cloud9

Log into your AWS Account and go to the AWS Cloud9 service.
Create a new environment: I gave it a name and proceeded with the default settings.
Launch the environment: This will open the Cloud9 IDE in your browser.

I personally love the Preview feature offered in Cloud9, as it allows me to observe the website in real time while developing the application. Once I had created the static website, I required a method for hosting it to make it accessible to the public. This is where Amazon S3 became essential. To achieve this, I:

Created an S3 Bucket
Uploaded my Website Files
Enabled Static Website Hosting
Set Bucket Permissions (which is set to Public). What this means is that the bucket will be public is a necessary step when hosting a static website on Amazon S3 because it allows anyone on the internet to view and access your website.
Got Access to the website endpoint AWS provided to me and launched it on my browser

Web View

Launched the web app using Cloudfront URL

Backend Setup with AWS Lambda

Here, I will be using AWS Lambda for serverless backend logic. I set up the backend with AWS Lambda to process user inputs and also interact with the third-party API. I chose a preferred API of my choice (there are others out there that are available which are also free to use). I built the Lambda function using python

Testing out the API using POSTMAN

Before testing out the API, I had to employ the use of Postman to ensure the API was indeed going to fetch results. As you can see in the image below, it does return the expected results

What to expect next?

The next phase will be to implement security measures for protecting user data on AWS, Cost Analysis of Running a Vacation Planner on AWS, Testing my AWS-Based Vacation Planner Application amongst others.

I am excited about this fun-little project and look forward to using it to plan my trips, you should too !

Sentiment Analysis on the Launch of Amazon Q using Amazon Comprehend and Sagemaker Studio

Chinwe O. — Tue, 02 Jan 2024 20:06:08 +0000

Sentiment analysis is a powerful tool that allows companies to analyze customer opinions and sentiments towards their products or services. At AWS Re:Invent in Las Vegas on November 28, 2023, Amazon unveiled Amazon Q, an innovative AI-based assistant tailored for workplace use. This cutting-edge tool is designed to deliver quick responses to inquiries, create content, and enable actions using data from customer information databases, codebases, and corporate systems. Amazon Q provides personalized interactions to streamline processes and speed up decision-making while also fostering a culture of creativity and innovation in the workplace. The launch of Amazon Q marks a significant advancement in the field of AI-powered assistants for work.

This mini-project came as a result of my curiosity with the whole hype of AI which seems to be dominating the technology landscape. Also as a Product Manager this is of importance in order to determine customer satisfaction and engagement and ultimately make data driven decision in order to scale the product/service. As a Data Analyst, just like how Duet AI is to Google Looker, that is how Amazon Q is to AWS Quicksight. This tool integrated with Quicksight provides real time and fast analysis of data in seconds. When a new product is released, there is a need to determine the business value of its customers. Sentiment analysis can help identify the overall sentiment towards the new product by analyzing customer opinions and emotions expressed in online reviews and social media posts.

Getting Started

In this tutorial, I will show how to perform a Sentiment Analysis using 2 major tools from AWS: AWS Sagemaker Studio and Amazon Comprehend.

Architectural Diagram

Data Collection: data was extracted from twitter using Python. This data contained tweets from users. Here are some samples of the tweet that were scraped for this project:

Using #AmazonQ in the IDE is reeediculous! I know AI isn't always right but it has been absolutely amazing. It even overcomes my HORRIBLE spelling! So Sick!
reInvent2023 unveiled #AmazonQ as a standout highlight. At Caylent, we're excited to streamline the integration of Amazon Q into MeteorAI, making adopting GenAI solutions that much faster and easier.
Do I know anyone at AWS who can comment on when the Amazon Q Code Transformation for .NET will be available? If this works, it would be amazing for some legacy applications we need to modernize at work.

Data Transformation: most of the transformation done here was data cleaning, this is because the data contained certain special characters such as "#" and "@". This was done using Excel Sheet.

IDE: Amazon Sagemaker Studio served as the coding environment to run and perform the task

Sentiment Analysis: AWS offers a range of services with strong NLP capabilities, including entity recognition, key phrase extraction, and sentiment analysis. Amazon Comprehend is one of such service. It utilizes machine learning to uncover insights and connections within text, enabling the detection of sentiments.

This article is best suited for programmers familiar with Python.

Scraping of Tweets
I have a detailed article on Web Scraping here, so ensure to follow the link to get started. After this is done, the tweets are saved in the Excel Sheet and uploaded to S3 Bucket

Sentiment Analysis
Import the CSV File needed to run the analysis in Amazon Sagemaker Studio.

import boto3
import pandas as pd

comprehend_client = boto3.client('comprehend', region_name='us-east-1')

# Read CSV file
amazon_q_df = pd.read_csv('amazon_q.csv')
texts = amazon_q_df['Text']  
len(texts)

Perform the sentiment analysis using detect_sentiment in Amazon Comprehend to determine different sentiments of users tweets. Read more on how it works here

sentiments = []

# Analyze text using Amazon Comprehend
for text in texts:
    sentiment_response = comprehend_client.detect_sentiment(Text=text, LanguageCode='en')
    sentiment_scan = sentiment_response['Sentiment']

    sentiments.append({'Text': text, 'Sentiment': sentiments})

sentiment_df = pd.DataFrame(sentiments)
print(sentiment_df.head())

Output:

Visualize these texts by sentiments (mixed, neutral, positive, negative)

import seaborn as sns
import matplotlib as plt
percentage = (sentiment_df.groupby(['Sentiment']).size())
percentage

# Adjusting the explode parameter to match the number of unique sentiment categories in the data
unique_sentiments = sentiment_df['Sentiment'].nunique()
explode = [0.1 if i == 0 else 0 for i in range(unique_sentiments)]

plt.figure(figsize=(8, 8))
plt.pie(percentage_new['Percentage'], labels=percentage_new['Sentiment'], autopct='%1.1f%%', startangle=140, explode=explode)
plt.title('Percentage of Tweets by Sentiment')

# Show the plot
plt.show()

The data collected were only specific to Twitter and the tweets were quite limited too, which may skew the overall findings and limit generalizability.

This shows:

A large majority of tweets, 74.4%, are classified as Neutral, indicating no particular positive or negative emotion.
20.9% of tweets are Positive, suggesting a favorable or optimistic sentiment of the Amazon Q.
A small proportion, 2.3%, are Negative, implying a pessimistic or unfavorable sentiment.
An equally small proportion, another 2.3%, are Mixed, indicating the presence of both positive and negative sentiments within the same content.

This analysis is useful for gauging public opinion or reaction to the launch of Aamzin Q on social media platforms like Twitter.

Amazon Comprehend and Amazon Sagemaker Studio played a crucial role in carrying out the Sentiment Analysis of Amazon Q. With Amazon Comprehend's advanced NLP capabilities such as entity recognition, key phrase extraction, and sentiment analysis, we were able to uncover valuable insights from the tweets extracted from Twitter. The use of Amazon Sagemaker Studio also provided an efficient coding environment for running and performing these important tasks. These tools not only facilitated the analysis process but also enhanced my understanding of public sentiments towards Amazon Q. The use of Amazon Comprehend and Amazon Sagemaker Studio showcased the power and effectiveness of AI-powered tools in sentiment analysis.

Thank you for taking the time to read my article. Don't forget to follow me on here and feel free to reach out to me. Looking forward to connecting again soon!

Language Whisperer - AI/ML Transformer Tools Hackathon

Chinwe O. — Fri, 30 Jun 2023 04:28:09 +0000

LanguageWhisperer: Facilitate your language learning with Transformers!

EDIT: We came 1st Place in the AWS Community Builder Hackathon 🥳🥇

We live in a world full of different objects, images and languages. However not everyone is well versed with learning new languages. But what if we want to learn a foreign language? This is where our tool, the LanguageWhisperer comes in.

HOW DID WE GET HERE?

On Friday 26th May, 2023, Lily Kerns, a Community Manager in the AWS Community Builder program announced our first ever Community Builders Hackathon! This is a fantastic opportunity for all of us to learn, be creative, and create something remarkable. 🥳

We teamed up together - @ronakreyhani, @dashapetr, @anja and I to work on this project

Project Description

Project Title: Language Whisperer

Language Whisperer as the name implies is a simple-to-use application that enables you to translate and learn a new language through images. Think of Language Whisperer as your primary application whenever you encounter an unfamiliar object in any given situation. This application will assist you in gaining knowledge and understanding about the object in question, regardless of your prior familiarity with it.

This application can be used in different scenarios such as education, entertainment, tourism etc.

Motivation

When visiting a foreign country and expressing a desire to acquire knowledge of its language, it would be advantageous to promptly access a comprehensive vocabulary for a specific location. Manually searching for numerous words would consume a considerable amount of time. However, with the LanguageWhisperer, efficiency is achieved. By capturing an image, multiple words can be translated simultaneously, enabling users to listen to the translations and gain deeper insights into their meanings.

Architectural Diagram

Process

During the initial stages of the project, our team engaged in brainstorming sessions to explore various ideas on how to leverage Transformers. We finally selected her concept, the Language Whisperer, for our project. To facilitate collaborative development, @dashapetr created a Google Colab notebook where team members could experiment and iterate on the initial codebase. As the project progressed, we transitioned to using GitHub, to streamline collaboration and ensure efficient code management among team members. This transition allowed for smoother coordination and enhanced teamwork throughout the development process. The following tools were used:

StarCoder agent, an extensive language model (LLM), offers a natural language API built on top of transformers and has been employed for the purpose of implementation. Detailed documentation for the agent can be found at the provided link https://huggingface.co/docs/transformers/main_classes/agent.
Wiki Searcher (a custom tool) was implemented by utilizing BeautifulSoup, a Python library designed for extracting data from HTML and XML files. This library played a crucial role in parsing and navigating the HTML structure of web pages, enabling the extraction of relevant information for the Wiki Searcher application.
gTTS (Google Text-to-Speech) library (a custom tool) was used for converting text into high-quality speech output with natural-sounding voices. This decision stems from the observation that the default translator within the agent does not meet our desired level of effectiveness when it comes to accurately reading text in various languages.
Streamlit was used for the Frontend

Challenges Faced

Finding the appropriate voice for the task proved to be quite challenging. Unfortunately, the built-in StarCoder text-to-speech tool rendered foreign phrases in English with a noticeable accent, causing confusion. Additionally, a decision was made to conduct research in order to find a suitable solution. One option considered was the utilization of Amazon Polly, although integrating it with streamlit presented difficulties, as it necessitated authorization to an AWS account. Alternatively, the gtts library offered a viable option, requiring no keys or access and easily installable via pip install. It simply required the addition of a language code as input, yielding natural-sounding voice output.

One of the challenges we encountered was determining the appropriate front-end stack for our machine learning application. Initially, we embarked on building a Next.js React application with Python APIs. However, in an effort to simplify the process, we made the decision to utilize Next.js embedded APIs instead of deploying lambda functions and an API gateway. Unfortunately, this decision led to significant issues due to dependencies. We found ourselves needing to containerize the Python library dependencies. Considering the urgency of implementing our idea as quickly as possible for a quick proof of concept, we altered our approach and opted to implement the user interface using Streamlit.

Outcome

With Language Whisperer, all you have to do is

take a picture
upload the image you would want to transcribe
select your preferred choice of language
play and listen to the transcribed language

E.g: Let us have a look at a sample image:

Step 1: Image Analysis - Language Whisperer receives this image and generates the following

Step 2: Image Caption: 'a plate of food with eggs, hams, and toast'

Step 3: Translate the caption into a language of your choice: E.g Spanish

Step 4: Learn/Read the caption: un plato de comida con huevos, jamones y tostadas

Step 5: Search for a word meaning in Wiki: (comida) –> food, something edible <…>

PROJECT OUTPUT

Github Repository: https://github.com/RonakReyhani/LanguageWhisperer
Demo Video: https://youtu.be/zaYRAKcPHOk

Lesson Learnt (New Skill Developed)

Use of Session States
Use of Amazon Polly
Use of Streamlit

Reflection

@anja: As I didn’t have any experience with the Transformers library before the Hackathon, I was hoping to be able to contribute enough anyway. Luckily, the documentation is very beginner friendly even for people that aren’t experienced with Machine Learning yet. I have been reminded that you should never be afraid to try new tech tools, often it’s not as difficult as you think. I will definitely dive more into Machine Learning in the future. Also it was the first Hackathon I participated in, it was awesome to work together with my brilliant teammates.

@ronakreyhani: I have always harboured a deep passion for machine learning (ML), which makes every new concept or topic in the field incredibly enticing. Recently, my curiosity led me to explore the realm of Generative AI, specifically the renowned Hugging Face LLM models. Although I had heard about them in passing, I had never had the opportunity to delve into their intricacies. This project presented a remarkable chance to step out of my comfort zone and venture into the unknown.

Throughout this journey, I gained extensive knowledge about various aspects of the Hugging Face ecosystem, including Hugging Face Hub, models, transformers, pipelines, and the widely acclaimed "agent" that our app heavily relies on. Beyond the technical advancements, what truly made this experience exceptional was the opportunity to collaborate with an extraordinary team spanning across the globe. Through virtual meetings and vibrant discussions, we pooled our ideas and arrived at a common understanding. Working with them was truly inspiring, and their unwavering support allowed me the freedom to implement my ideas using the tools I was most comfortable with.

As the hackathon reached its conclusion, I not only acquired a wealth of knowledge about LLM models and Hugging Face agents, but I also forged incredible friendships. The prospect of meeting these newfound friends in person fills me with anticipation and excitement. In retrospect, this sense of camaraderie and connection stands as the greatest achievement of this endeavour.

@dashapetr: As a Data Scientist, I had a bit of experience with Transformers, but I haven’t used tools and agents. I found the hackathon idea very interesting because I saw the huge tools' potential. Choosing the concept was quite challenging; I had several of them, but when the LanguageWhisperer came to my mind, I was so excited that I decided to put aside all the rest of the ideas. The LanguageWhisperer is something I wish existed when I was struggling to learn French and Chinese. I am grateful that my team decided to go with this idea, and I am extremely happy to get to know my fellow female builders better; it’s an enormous pleasure to build together!

@chinwee__o: One of the many things that stood out for me was employing different alternatives available needed to get the work done. One of which was trying out Amazon Polly for the purpose of this project. While transformer had a text-to-speech agent which could have been implemented in this project, however the outcome did not produce the best result. This further buttressed that these alternatives are available in order to meet a specific need if others fail. 1 month, 4 weeks, 29 days, 696 hours, 41760 minutes, 2505600 seconds and every single meeting, conversations, chat, lines of code, new learnings with these ladies was worth the while.

FURTHER STEPS

We believe that the LanguageWhisperer can be extended and improved.

Firstly, its functionality can be expanded to facilitate the comparison of translations in multiple languages, generate illustrative usage examples for a given word, and provide a feature to "listen" to and rectify the user's pronunciation.
Secondly, LanguageWhisperer has the potential to be transformed into a mobile application, enabling users to access it conveniently from any location.

Getting Started with AWS Storage tools — s3

Chinwe O. — Wed, 04 Jan 2023 04:10:28 +0000

Just getting started with Cloud Computing in Amazon Web Service as a Data Analyst and not sure of the different cloud services to should explore. This article is for you.

As a data analyst, it is imperative for you to have a basic knowledge of cloud services and learn how to use them for whatever projects you will be working on. Just like you, I recently just started exploring and working with the different storage (s3) and analytical tools (AWS Athena and Quicksight) available in AWS.

In this article, we will focus on STORAGE SERVICES (Amazon S3), which allow us to store structured sets of data used by applications just like SQL Server, Oracle, and MySQL. In my previous article which outlined the steps needed to ingest data from Excel (.xlsx) to Azure Server Database, here I will be guiding you on how to ingest data into one of AWS Cloud Storage Service, S3.

Before I go on, it is important you have Credit in your AWS Account and also have a dataset available.

For this walkthrough, we would be using structured data and storing it in a .csv format. Here are some links to free datasets:

AWS Datasets: https://registry.opendata.aws/

Data World: https://data.world/

Kaggle: https://www.kaggle.com/datasets

Data.gov: https://data.gov/

Datahub.io: https://datahub.io/search

In doing so, Here is a step-by-step process:
Step 1: Log into your AWS account, and search for S3 (Amazon Simple Storage Service) in the console home. S3 is a fully managed object-based storage service and it's highly durable and cost-effective. S3 is used for storing large files like videos, images, static websites, and even backup archives as well.

Step 2: Create a Bucket. Think of a bucket as a container for your data to store objects.

By default, you can create up to 100 buckets in your AWS Account.

Click the Create icon
Next, create a bucket name (note that your bucket name has to be completely unique).
Change the AWS Region as you desire
For the other options such as Object Ownership, I usually just use the default recommended option.
Check the Public access settings for your bucket (this protects against public access and also allows you to inspect and change already existing policies and ACLs for your buckets and objects).
Enable Bucket Versioning, I will recommend you enable this so that you can have a different history of the changes made on your buckets. This will help you recover objects when you accidentally overwrite or delete (Just think of this like Github Version Control)

Go on and click the Create Bucket icon. Once it has been created, you will then see a success message e.g “Successfully created bucket “rfmdataworld”

Step 3: Select the newly created bucket “rfmdataworld”

Step 4: Upload your data into the bucket

In the objects option, click the “Upload” Icon, you will then be redirected to the UPLOAD PAGE.

Select “Add files”, these files in this case would be any dataset you already have stored on your computer.
Go ahead and Upload the file
Once it has been successfully uploaded, a success status, “Your file has been successfully uploaded” will be shown.

You can also view the details of the file that has just been uploaded.

This file is now currently in your newly created s3 bucket.

Now you have seen how to upload and store data in AWS S3