Forem: Boi N Moriba

Q. Quarterly Inc: A Flatiron School DS Capstone Project

Boi N Moriba — Fri, 29 May 2020 15:22:28 +0000

Guys It's Finally HERE, My Final Project

Here is the direct link to the Github if this project: https://github.com/bmor2552/QQuarterlyInc/blob/master/README.md

Here is direct link to the video presentation of the project: https://www.youtube.com/watch?v=8tBPOyyXLzE&t=11s

The Idea

I must have missed something this time around because I had on my overthinking pants for this project. Let me tell you, I underestimated the power of overthinking!!

Originally I had an idea which came from my very faint but present experience with fundamental analysis. I wanted to see if the sentiment of certain documents had an influence on the stock price change of a company.

The Process

The steps I had to take were as followed:

Gather a list of companies I wanted to analyze
Gather the documents of focus for the list of companies
Clean the documents (stops words, html, etc...)
Parse through the documents
Run a sentiment analysis on the documents
Compare the sentiment values to the price change in the companies

Seems simple enough right?

But what if you have never done it before, and can't seem to find anyone else who has done it either? Then what?

Figuring It Out As I Go

After my project idea was approved, I got straight to work. I had an idea of how to begin because of very helpful blogger named Osho Jha. He shared the beginning steps of how to obtain the documents you want to focus on for your sentiment analysis; I chose quarterly reports. This is where the help stopped ... dun dun duuuuuuuuunnnnnnn!!

Here is a link to his blog post: https://towardsdatascience.com/useful-sentiment-analysis-mining-sec-filings-part-1-358942fc98ed

This did not stop me, with the guidance of my Instructor and Coach I was able to proceed with the steps listed in "The Process", but not without overthinking for two days straight, searching for data for a week, cleaning the data for another week, and finally conducting the sentiment analysis a couple of days before my presentation/final grade is due (We had a total of 3 weeks FYI)!

To those who are more level headed when it comes to tackling a task, this may come across as complaining. But I assure you this was a test of my patience with the process & with myself.

I have been over analyzing everything since the beginning, and after 27+ years of doing this, task start to take longer then they should. Please NEVER ask me what I want to eat!

But it's ok, because through trail and error along with three different notebooks I was able to find signs of correlation between the stock price change of a company and it's quarterly reports. (See Github link at the beginning of the blog, don't be shy be curious lol)

Images Of The Results

Below you will see a scatter plot of the price change of a company and the subjectivity of the quarterly reports.

Below you will see a scatter plot of the price change of a company and the polarity of the quarterly reports.

Below you will see a scatter plot of the price change of a company and the price change of the overall market. (This is kind of obvious but also cool to plot, give me a break I'm trying here lol)

What's Next

Because of the timeframe I was unable to look into more companies, although I did find a way to automate the document retrieval process, something was wrong with the os.listdir method, the function kept returning only one company (got to love those for loops!). Feel free to take a swing at it if you dare!

I also did not get a chance to tokenize the reports or parse through the digits of the reports, this will be my next task after I fix the function that I mentioned above.

Closing Statement

This was one of the most frustrating, exciting, encouraging experiences of my life and I feel like I am on cloud nine! If you are interested in doing a coding bootcamp I say go for it. You will come out with a new set of skills and as a more evolved human. Sounds like a win win to me, anyways happy coding people!!

One Chapter Ends, Another One Begins

Boi N Moriba — Fri, 08 May 2020 15:41:36 +0000

Chapter One - The Journey

In the beginning of bootcamp, it felt as if this fast paced learning would never end! For the first month all I could do was sleep. This was the only way I could process all of the information from the day.

As the second month of bootcamp progressed, the corona virus had spread across the states and rumors of going remote started to flood the school. Eventually that became our reality and I was back at home struggling to learn new material in an environment I had kept separate from educational and income based work.

With the constant battle of new challenges I faced, I chose to keep my head down and keep working. I will admit I had my bad days but I kept pushing forward! I had a goal, and a student loan sooooo......yeah.

Now we are in the last month of our Data Science Bootcamp here at Flatiron School, I am currently working on my final project and graduation is set to be on May 29, 2020. It doesn't feel real.

Words Of Wisdom

When you are starting something new, imagine yourself as a child. Remember how scared you were to ride your bike without training wheels, or jump in the pool without your floaties? That fear is still with us in adulthood, so be gentle with yourself. It's ok if you do not get things right the first time, you've never done it before!

Know that you will face some challenges along the way, and while you are facing them it will feel like you are getting no where.

Keep pushing, because when you reach the end you will see just how far you have come and that fear that I mentioned before will be replaced with confidence and skill.

Final Words

My plan for this blog post wasn't to go into detail about my personal journey through the bootcamp. It was more so to acknowledge that I did it, I made it to the end of the program (well, not really I still have my final project but that should be easy).

I wanted to share that, as well as some kind words to encourage anyone thinking of doing a coding bootcamp or embarking on a new journey themselves. Hopefully this reaches someone in need of it. Happy coding, and stay well.

Chapter Two - Job Hunting Post Corona Virus ...

Binary Classification Problem: Random Forest & OneHot Encoder

Boi N Moriba — Thu, 16 Apr 2020 22:54:00 +0000

For more projects checkout my github: https://github.com/bmor2552

With all the uncertainty in the air right now, picking up a new project is one of the best things one can do to kill idle time at home. So, why not learn how to solve a binary classification problem?

This week for our Module 3 Project here at Flatiron School, I was able to team up with my fellow 02-17-2020 Data Science cohort member Taki Yasuoka (github link: https://github.com/Tyasuoka) to figure how to predict whether or not a customer would discontinue their services with a telecommunication company. Below you will find the github link to this project.

https://github.com/Tyasuoka/Module_3_Project/tree/master

The Data

The data was provided by the Kaggle member david_becks and consisted of churns and non-churns in a telecommunication company. The columns contained information on the features customers used, from area codes to evening minutes. The link below will lead you to it.

https://www.kaggle.com/becksddf/churn-in-telecoms-dataset

The Process

First things first clean, explore, & transform the data!! With this data set there were no missing values. We did find some object columns that would interfere with the model of our choice. So we dropped the ones we didn't need and transformed the ones we did need.

Another thing we noticed about the data was that the column names seemed as if they would interact with one another, i.e. evening minutes and evening charges. The churn column was also imbalanced; there were less churns than there were non-churns!

This would later become an issue, so we decided to use the random forest model to take care of it. To measure how are model was performing we went with ROC_AUC (how confident our model is when making predictions) and Accuracy Scores (how correct those predictions are).

The Model
The random forest model is a group of decision trees, THE END.
Just kidding, let's start with what a decision tree is by using our data as an example.

A decision tree model in our case will split its predictions into churn and non-churns. Think of it like sorting apples and oranges, or sorting change. The decision tree is classifying/sorting its predictions! The only issue with using this model is that our non-churns outweigh our churns so the sorting of our predictions will be weighed heavily on the non-churn side.

We don't want a model to tell use everyone will keep their services and not tell us who will disconnect their services! We want a model that can predict both who is a loyal customer and who needs some extra attention before its too late!!

To fix the imbalance of our data we can use, you guessed it, more trees! This is where the random forest came in!! Random forest uses the bagging method and random selection to make its collection of decision tree. This is done to get rid of multicollinearity (aka columns interacting with one another) and balance out the data.

Side Note: Since random forest is using random sampling this helps even out the non-churns and churns in our data, the more samples you pull from your given data the better the distribution of churns and non-churns in your predicted data.

The Results:

Here is the github link to the project; the notebooks include details on findings and the code to obtain those findings.

https://github.com/Tyasuoka/Module_3_Project/tree/master

Random Forest without OneHot Encoder

Accuracy Score: 0.936 aka about 94%
ROC_AUC Score 0.909 aka about 91%

Random Forest with OneHot Encoder

Accuracy Score: 0.942 aka about 94% (but a higher 94%)
ROC_AUC Score: 0.934 aka about 93%

Side Note: Use OneHot encoder on a column that is distributed better than your target. In our case, area code was well distributed between 3 numbers, so to assist the models learning we applied OneHot encoder to the area code column, hence the increase in the scores above.

Conclusion

When you are dealing with a binary classification problem in the real world, keep in mind that if there is multicollinearity present and your data is imbalanced, a good model to consider would be the random forest. To assist in the balancing of the data and performance of the model try an ensemble method like encoding.

Future Recommendations

If I were to do this project all over again without a time frame on the outcome, I would start with a less complicated model like Logistic Regression. To see if I can improve the performance of that model I would use feature engineering and ensemble methods. Then I would move one to decision tree, and finally random forest. From there I would compare the models and choose the one that best fits the business problem I am trying to solve.

References: Below are great sites to help breakdown everything I discussed in this blog.

Random Forest & Decision Trees

https://victorzhou.com/blog/intro-to-random-forests/

Accuracy Score

https://blog.floydhub.com/a-pirates-guide-to-accuracy-precision-recall-and-other-scores/

ROC_AUC Score

https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5

OneHot Encoding

https://www.analyticsvidhya.com/blog/2020/03/one-hot-encoding-vs-label-encoding-using-scikit-learn/

Hopefully this will motivate you to learn something new or even help you solve a different binary classification problem. Happy coding!!!

Transition To Remote Learning

Boi N Moriba — Mon, 23 Mar 2020 06:22:33 +0000

As the days go by, I find my motivation declining. Things just aren't the same. The more I sit in one place, the more idle I become. If this restlessness continues, it will not end well.

This weekend I took it upon myself to end this madness by digging deep into the problem. One question after another:

Am I just being lazy?
Is remote learning for me?
Why is the switch to online learning so difficult?

The answers weren't easy to find so I decided to compare the pros and cons of remote learning.

Pros

Sleeping in
As a person who would wake up at 500am just to be on the road early enough to avoid downtown rush hour traffic, I am happy about not having to commute or wake up as early. 630am - 700am works just fine.
Savings on gas
Have you seen the gas prices?
Savings on parking garage
This monthly bill can be eliminated all together, who doesn't like having less bills?
No more cold buildings
Usually all of the buildings I go to have the A/C set to a temperature that makes me very uncomfortable. It's good to be home where I can control how cold or hot I want to be.
Getting out of my comfort zone
I have never had to attend classes remotely, but this gives me a chance to grow. I'm learning how to do something I am obviously not comfortable doing all while learn something new about myself.

Cons

Decrease in inspiration
I joined the Flatiron Data Science Bootcamp to be surrounded by like minded people who I can bounce ideas off of and learn with. I also wanted to be able to talk to my instructor and coat as much as possible. Although the access to my instructor and coach have not changed, the access to being around my cohort has. Video chat is not the same as being next to someone.
The energy of the environment
The energy on campus was productive and determined vs the energy at home which is relaxing and distracting. I am not use to working remote so this is a culture shock for me.
Decrease in in-person networking opportunities
While having access to the Wework building we were able to connect with different companies on different floors. Now we are restricted to our homes (especially with the spreading of COVID 19).

Solution

Remote learning/working is a new experience for me, but I am willing to work at it by incorporating the following:

To increase my inspiration I will begin reaching out to my cohort for study sessions via video chat. I will also ask my instructor and coach to join me in a study session via video chat. This way can I trick my brain into feeling as though I am in a class room.
To address the energy in my environment, I will change locations until I find a spot that I feel productive in (sitting and staying at a desk in my house was surprisingly harder than I thought, so the search continues)
For the sake of my sanity and my bootcamp experience I deleted all social media, I didn't want that distraction. Now that I am entering the professional world I will use sites like Linkedin, Kaggle, Dev.to etc... to network on a more professional level.

This is not something that is going to change overnight, but it is a process I am ready to begin. Wish me luck and feel free to comment any advice you have on how to make this remote transition as smooth as possible.

Why Not Data Science?

Boi N Moriba — Mon, 02 Mar 2020 00:58:23 +0000

I'm 27 years old and I have just realized that I've been going about this life thing all wrong 🤦🏾‍♀️. Good thing is I'm still alive thank God, so I can start going about it the right way.

The How

After getting my Bachelor's Degree in 2014 I was on a mission to start my career in the health field. I was going to be a physical therapist!! 6 years later and I haven't even touched anything that has to do with physical therapy, let alone my degree (ask me if it's on my wall, go ahead ... it's not).

Don't get me wrong I loved furthering my education and I appreciate the experience I got from going to a university (Virginia State University 😏), but I was going after something I didn't want, I was going after something that sounded good.

Now before you judge me for picking a major that sounded appealing, have a conversation with my mother, then tell me how you feel (love you mom 💕). Let's face it, at 18 I had no idea what I wanted to do with my life and staying at home was not an option. Long story short, it just wasn't for me.

Like a lot of my peers in the same predicament I got a job and started working until I grew numb. I got tired of wasting my life in an enviroment that stumped my growth. I deserved a challenge, I deserved mental stimulation, I deserved a job where I could sit down all day!!

The When

In 2019 one of my really good friends mentioned coding to me and how much I would like it. Right after he told me that, I recalled the two other times I had been told to try coding as a career. Third times a charm right? I took it as a sign and went all in. I haven't stopped coding since October of that year.

I still haven't explained why I want to become a Data Scientist, have I?
Well to break it down to its simplest form I have come up with a list. This list will not only show you why I chose data science but it will also show you what I like to do in my spare time. Prepare yourself...

The list

I love problem solving (and the rise and fall of my emotions during problem solving. Once I figure it out, that feeling makes it all worth it.)
I love Googling things (it's a skill, and it's not up for debate.)
I love math (for the record, I more so like statistical math, don't through any geometry at me please because I will NOT catch it).
I'm a Virgo ♍️ (which means I already made a life career out of over analyzing EVERYTHING plus I like to organize EVERYTHING so I am VERY task oriented)

The WHY

It's a match made in heaven!! As a Data Scientist I will be looking at all sorts of data and googling ways to manipulate that data. My day will revolve around completing task that have to do with the data. From time to time I will have to use math, as well as present what I have discovered and how it is helpful (which is a plus because I like to talk, can't you tell from my super long blog post).
Last but not least I will be sitting down majority of my work day 🤯😫!!! So, the REAL question is Why Not Data Science?