Forem: Ayodele (eye-ya-deli)

4 Tips to Speed Up Your Data Science Workflow, Using Observable

Ayodele (eye-ya-deli) — Mon, 29 Aug 2022 22:20:00 +0000

According to a report by Forbes, Data Scientists spend 80% of their time on data management—cleaning, labeling and annotating. This means there's much less time for doing the core data work that businesses need.

Here, we offer 4 tips to reduce time spent on those time-consuming tasks, and free up time to refine algorithms and mine data for patterns. Added benefit: the time recovered also helps teams make better data-driven decisions.

More Tools, More Problems

One of the reasons cleaning, labeling, and annotating data is so time consuming is because of the number of tools being used. It’s common to start with SQL or a Jupyter notebook, and then create charts in Excel, Tableau, or Power BI, then take screenshots of them to embed in presentation slides.

Hopping between three, four, five, or more tools to stitch together an end-to-end data workflow - exploration, analysis, modeling, and communication of findings - adds complexity, room for misinterpretation, and manual errors. It certainly doesn’t help streamline the workflow.

Our 2021 research report — The State of DataViz — found that the team members involved in the above workflow use two, three, four, and sometimes even more tools to stitch together an end-to-end flow to analyze data and communicate insights.

In fact, there were more than 180 unique tools named by respondents, optimized for different roles with different skills (see color-coded diagram below), that are involved in the data workflows.

Fewer tools reduce knowledge workers' mental strain

Tool-hopping also forces us to switch context. While it may be subtle to those experienced and accustomed to stitching together custom data workflow solutions, moving between tools requires us to stop working with the muscle memory of one tool and pick up another.

According to a joint report by Qatalog and Cornell University’s Idea Lab, it takes people nine and a half minutes, on average, to get back into a productive workflow after switching between digital apps. And half of the knowledge workers reported a decrease in productivity and an increase in fatigue from constantly switching between tools.

Tip #1: Use one tool to take you from dirty cleaning to exploratory analysis

In Observable, you can see where your data is coming from, manipulate it with visual tools, create charts in seconds, and pair prose alongside code.

See where your data is coming from. Pull data into an Observable notebook, whether that data is written inline, saved in a local file, fetched from a public or private web API, or stored in a live database. Learn more here.

Manipulate data with visual tools. Use Observable's Inputs to quickly create sliders, radio buttons, and other custom filtering tools to see different cuts of the data more clearly. Learn more here.

Create charts in seconds. Observable Plot's goal is to help you get a meaningful visualization quickly. It has a free JavaScript library, a concise and (hopefully) memorable API to foster fluency, and plenty of examples to learn from and copy-paste. Learn more about Observable Plot here.

Pair prose alongside code. This very article pairs prose alongside code. You're reading the prose here. Take a moment to scroll up to the image showing 180 unique tools. If you click on the caret to the left of that image, you'll see the JavaScript code that corresponds to that image.

Tip #2: Speed up data-cleaning

Cleaning data can be both time consuming and frustrating, especially when data is collected from different sources. Dealing with format mismatches, null or missing data, and unruly joins can eat up time that is better spent on mentally challenging work that still must get done, no matter how long data-cleaning takes.

On Observable, use SQL natively as a Swiss Army knife to transform, join, and clean data on your terms, securely connected to your data.

Tip #3: Accelerate the data collection process

You may be collecting data from qualitative surveys, fetching product data, or creating new datasets from various sources. The disparate tools used by most data scientists to collect data for analysis range from a menagerie of free, open-source tools to pricey options that do one thing really, really well.

Rather than using several different tools, Observable allows data teams to manage their data workflow (from import to analysis), and present findings in the same place, with no screenshots or exports required. When it comes to data collection, Observable offers several ways to streamline the process.

Munge qualitative data
Observable’s powerful visualization tools let you munge and group text data with ease. Generate word clouds, analyze sentiment with Tensorflow, or find surveys that most frequently express dissatisfaction with TF-IDF.

Minimaps offer transparency and security
Most product data lives in a cloud-based source. However, the security of these connections is always a concern. Often, it's hard to see exactly what database tables are used to generate a chart or dashboard.

No more trudging through every cell to find a connection error or all the charts that rely on that connection. With Observable’s minimap, you can clearly identify dependencies and the cells that rely on them.

Clear database management in settings makes it easy to always stay connected. On Observable, you can connect private Observable notebooks directly to PostgreSQL, MySQL and BigQuery databases. Use the DatabaseClient() to plug live data into reactive visualizations.

Plus, sharing settings allows each team member to be able to interact with your analysis. This way, they see the source of truth the way you can-without having to download, setup, or manage anything new.

Combining datasets
Sometimes the relevant data lives in two wildly different formats, and in different places. Pull together and combine those datasets easily with Data Wrangler, a UI tool that lets you join and select the data you want with just a point and a click.

Tip #4: Quickly mine for patterns

Data visualization is often discussed in terms of what to show stakeholders and clients. However, we can use visualizations ourselves in order to better understand and tell stories about data.

Observable’s Summary Table is a fast way to get a quick, visual overview of a dataset. With Summary Table, you can create scannable data tables and summary charts with the click of a button.

You can also use it to understand the variability and spread of any column. These intuitive tooltips help you decide what further analysis or modeling techniques you want to take.

To get started with a more streamlined data science workflow, and enable your team to get to analysis faster, try Observable for free.

Originally posted on Observablehq.com

How I Got Started in Tech

Ayodele (eye-ya-deli) — Thu, 23 Apr 2020 19:08:29 +0000

The year is 2012 and I just transferred to Texas State University. I was as excited as any other Category 5 Nerd for the first day of class. I was officially a computer science major. Just saying that out loud made me feel smart. You could walk to the CS building and other students would go "oh you're smart if you have a class in there". I navigated to the class with ease having cased the building the day before. I sat next to a brunette and bounced my knee nervously trying to pull up the syllabus on my laptop before the professor arrived.

Unlike many of the freshman around me, I was a junior in college. After a disastrous freshman year in a small town in Texas, I spent a year and a half studying film in Austin, a 3-hour drive from my parents home. I loved what I was doing, I excelled in film editing and could see myself directing dramas and music videos. The only problem was when reality hit. When I was looking for film work I stumbled on $8/hr gigs that wanted you to work 16 hour days with few benefits and often only paid at the end of a 3-month shoot. This was not the lifestyle I could take on considering I didn't have a large savings to live on.

I decided after unsuccessfully trying to find work in film to continue my education and landed on computer science. I had always liked computers. I was spoiled my senior with a pretty silver MacBook to apply to and go to college with. Despite my pleas for a Dell with a pink flowery back cover the MacBook was perfect for my needs and had minimal issues. I blame my dad for making me an Apple fangirl before I held an iPhone.

Our professor finally arrived to class 5 minutes late. He looked like he was in his late-30s and carried his laptop tightly in his armpit. He connected to the projector without a word to us then scanned the room slowly. His eyes fell upon my section of the room and he quickly walked over to the whiteboard, wrote out his name then turned to face us again. "It looks like we have two girls in here this semester, don't feel bad if you fail. I'm Dr. _______ and this is Computer Science 1". This was my first introduction to tech, no, this was my first introduction to misogyny in tech.

After 'don't feel bad if you fail' left my professor's mouth I felt all eyes on me, I looked around taking in the faces of my pale male classmates. Some were confused some badly hid their amusement. I was one of two women and the only black person in the room. So I resolved to not wanting to seem stupid. You can't come off stupid if you never say or ask anything. Now I had 20 boys waiting with a joke for when I failed. Weeks later I was doing poorly, as my professor expected. I scowled noticing the flyer on the bulletin board adjacent to the shitty professor's room.
"Are you a girl interested in Computer Science? Apply for the CS dept Scholarships for girls!"

I thought "Why the fuck would anyone want this?" The courses weren't set up so we'd learn, but copy how our instructor coded. We started in C++ and did so many tests by hand I spent more time writing than typing. My willingness to ask questions was at 0 the whole semester. I feared failing my gender or my race and nobody in that room would think I just sucked on my own merits after our Professor calling us out. It didn't take long for the semester to come to a close with my grade sitting at a C. I met with my professor during after hours where he berated me for not attending office hours earlier (gee, I wonder why) and that he could tell I was putting in genuine effort, but "if this part is hard for you it's unlikely you're going to get it. You're very bubbly though, you look like a Communications major not a Computer Science major."

Let's take a trip down memory lane to 1998. I was 7 and sitting in front of a big box TV watching Star Trek: The Next Generation after school. When my dad got back from work he lugged in a massive desktop computer and what seemed like a larger than life satellite dish. I was lucky to have parents who were early adopters of tech and apparently, he got a good deal for signing on early. He got to work setting up a big stake in the backyartd to secure the dish. He brought home our first family computer and I was hooked from day 1. I wish I could say I was a coding wunderkind or took the thing apart just to put it back together, but that's not the case. I was your average kid, an internet consumer playing flash games on the Nickelodem websote. You see my dad is a lover of all things new and techy from big screen TVs to Xboxes, Smartphones, the lot of it. I understand that my experience using technology is unique compared to most of my peers and especially most black people. As an only child, I benefitted from my dad's love of gadgets and disposable resources.

We soon had a dedicated computer room with a desk for my parent's shared computer and a desk for mine. My mom was a nurse who spent time in community college coding in COBOL and FORTRAN. My dad was a security freak with networking and TC/IP books strewn around. Tech was a quiet omnipresent influence in my life. Through my teenage years, I'd spend hours posting on Xanga, updating my top 8 on myspace, and writing fanfiction on what's now a quiz site. To say I feel like a digital native is an understatement. However, my math skills were lacking. I voluntarily took summer school math in high school to catch up with my advanced and AP math friends. It just didn't click for me long before I knew there was a stereotype about black people and mathematics. My parent's invested in after-school math tutoring to make sure I was college ready. I felt just as insecure in my college classrooms as I did answering questions in private tutoring sessions.

I wanted to give it another shot, wanting to believe I could do it. I was a 5th year senior studying digital media in Pittsburgh and had the time to take some CS courses as electives. I even had my first female professor. She was an awesome teacher and actually broke down the problem and how to translate it to code. I was the only woman in the Intro to Python course and would often go to office hours to talk about career paths. She went above and beyond to explain logic concepts that helped me with projects. Turns out being able to communicate is important. Once when we were getting coffee and chatting about front end vs back end she opened up to me about the harassment she'd been facing in the department. I wasn't blind to how her male student's treated her. I was concurrently in a web design course with most of those guys and they never talked back to our male professor. They never made jokes about his boobs or how attractive or unattractive they found him. I found myself handcuffed with fear in those moments. I thought they'd direct the vitriol towards me next. It was my professor and I in a room of 25 white guys spewing unnecessary anger. I didn't get why they hated her. Why they made fun of her voice or why they'd make comments if she'd bend over to do ANYTHING. Trust me, if Deep Nude technology existed when I was in school, the guys around me would have been using it.

She was in the middle filing another complaint with the department (the right thing to do right?) and she needed a witness to what was going on as her previous complaints had been met with "you should learn how to control your class". I enthusiastically filled out the necessary emails and spoke to the dean of the department personally. Within a week we had a new class "monitor", a male admin in the department offices. Of course on this day the guys were on their best behavior. With that singular male opinion, there was no problem and her complaint was dismissed. When she found out the department had no intention in doing anything about it, it was a cold, cloudy day and we stood outside the CS building chatting for nearly two hours. She felt like she wanted to quit. She admitted she felt no reason to continue teaching before thanking me for taking the class and having her back.

My third encounter with sexism in tech was at my very first "big girl" job. It was hardly that, working for a 1-person startup at $12 an hour, but I had experience in SEO from some internships I had done during my undergrad. I was dressed for the job every day and so excited to show my boss I was going to work hard and have kickass ideas. It was perfectly fine at first, we brought on a social media manager and the three of us felt like rockstars. I'd ace client calls and they'd compliment the level of detail in my work.

One day I was fired unexpectedly. I came into work to find the doors locked. I called my boss and he explained how he just didn't need my services anymore. I was confused, we got along, I was doing a good job, I even went to dinners with him, his wife, and our other coworker. In the end, I got no explanation. A few months later while I was on a business trip to Minneapolis for my new company my old boss texted me out of the blue. After I picked up my last paycheck we hadn't communicated at all. He asked how I'd been and I bragged about the fancy hotel being comped for us. He took the conversation to an inappropriate level and even went as far to send me nude images of himself, before casually admitting he had to let me go because he felt like he couldn't control himself around me. I blocked his number and told his wife what was going on.

Since then I've done social media for a hotel marketing agency, app analytics for a lottery app company, videography and marketing for a cannabis startup, social and analytics for a cannabis website, data science for a cannabis startup, a drone startup, a fitness software company, and my current role at SambaSafety.

A harsh reality we need to confront is that 22-year-old cornballs coming fresh out of CS programs that allow them to harass their professors are making tech that impacts human beings. They're seen as the archetype, the ideal, the standard. Truth is, people of color have been in tech, used tech, retrofitted tech and we need to tell our stories. Homogenous groups are often at the helm of tech startups and can't guide companies to success without mentorship, legal guidance, or a basic concept of the very complex ethics software engineering entails, and yet. If we don't fix this early, we can't fix tech.

To my computer science professor that told me I looked like a communications major, I'd like to give a middle finger to you. I have a degree in Communications and went on to get a Master's Degree in Data Science. You can do both.

Top 10 Soft Skills for Data Scientists

Ayodele (eye-ya-deli) — Tue, 17 Dec 2019 22:35:00 +0000

You might read a Data Scientist job description and get overwhelmed by the skills listed to do the job well. What they don’t tell you is your soft skills are just as crucial as being able to perform K-fold cross-validation. These are things you can learn and highlight in an interview that won’t completely make up on lacking experience with a specific technology, but still, help you be seen as a “great” Data Scientist in your next role.

Communication 🙊

Being able to explain to not only stakeholders but coworkers and customers alike is a huge asset. One of the biggest things you’ll have to do as a Data Scientist is to explain things to people. I have to explain Machine learning models, basic probability, why forecasts predict the way they do, and why correlations don’t equal causation. This also includes coordinating appropriate meetings with stakeholders do better understand the use case of models you create. Translating our industry jargon for the people around us and adjusting how we speak to technical and non-technical folks is a requirement in the modern Data Scientist's toolbox.

Time Management ⏳

‍Estimating project timelines is a huge part of my role. I can decide how much of our team needs to be involved and how long we think it’ll take to create and QA the deliverables. Whether this is an Excel document, Tableau dashboard, or Machine learning model, we scope out if it’s a small-ish, individual project or requires a team to work in two-week sprints.‍

Professional Writing 📃

You’d be surprised at the number of people who don’t know how to write well in emails. Being able to write a professional sounding email is a skill that can be learned just like any other. Truth is, we shouldn’t always write the way we speak, and we should be able to cater to the same insights in different levels of depth depending on who we’re talking to.

Technical Writing ✏

This mostly pertains to writing good documentation of your models and assumptions. We have to keep good, clear documentation of our work so it’s not only easy to reproduce, but when we go back to tinker with it, we have an understanding of what we were trying to do. It IS possible to write about your model or package in a way that’s not completely dry and hard to follow. Check out these tips on better documentation writing so it’s clear for users.

Tactful Critique 🔥

One of the hardest things about transitioning into data science for me was learning to be fearless when giving criticism and delivering it with tact. I had the tact down, but I was always worried about critiquing a company’s way of doing things. While that was easy for me as a people pleaser I had to learn that’s not the best thing for the business. I had to understand that if I didn’t push us to use better products and new techniques that I wasn’t creating the kind of environment that grows from its experiences.

File Management 🗃

Being able to manage your versions well locally is a big deal. Don’t be like me and prioritize speed for file organization. You’ll thank yourself later. Save often and come up with a system that works door you. Some people add a V1, 001, or “needs_cleaning” to the end of the file names to help sort through what changes you’ve made or which version you should send a colleague. Structuring your projects properly will help you stay organized and integrate well with any Data Science team.

Active Listening 👂🏽

You need to be able to decipher what your customers say they want and what they want. For instance “I need to see all customers that purchased though brand ambassadors last year” might really mean “I need to determine if the brand ambassador program is worth the cost to the company”. Many Data Scientists are in positions where they deliver data to other parts of their organization. This often means boiling down their work in layman’s terms, educating customers on their models, and framing how predictive models aren’t ground truth. To translate their needs into tasks you need to be able to ask the right questions that help customers reveal their motivations.‍

Humility 🙌

One tactic my colleagues and I employ in interviews is asking candidates what they would change about their past projects. We find insight in how people think if they’re able to critique their past projects and demonstrate they’ve learned new techniques. If you think there’s nothing you could have done to a make a project you've worked on better, it’s a big red flag. One of the best ways to do this is to detach your self-worth your work. You're not bad at your job because your model doesn't predict well. Take the extra step to understand why and even if you made something that failed or only predicted accurately for 12% of cases, you can explain what you'd do to improve performance or even better, call out how you may not have had the best grasp on the data and could have posed your hypothesis better.

Data Analysis 🔍

While this seems to be a hard skill, being able to stare at a table of numbers and decipher what's going on can be learned. I was practicing data science for years without stopping to really analyze my data. This happens by forming questions and building new biological neural pathways that act as a mental decision tree of questions to interrogate your data with. It wasn’t until I completed the Head First Data Analysis book that I fully understand how to pull insights from raw data. Without the pressure of a deadline or deliverable, I was able to study a small table of data and have an intuitive idea of what was going on.

Business Awareness 📈

It’s pertinent to know about the key performance indicators (KPIs) for your industry, especially in large organizations. Data science is useful not just in tech, but also in healthcare, marketing, and real estate to many a few verticals. Each industry has its unique challenges and the metrics for success are drastically different. Having a good knowledge of this wins you a lot of “nice-to-have” points while interviewing for a new role.

BONUS: A technical skill everyone in Data Science should have is an advanced knowledge of Excel. As much as we in the industry like to joke that Excel will never go away, it’s true. Maybe if you’re working at an AI-first or research company you won’t touch much Excel, but customers often still ask for deliverables in Excel workbooks. Advanced Excel usually means being able to perform V LOOKUPS, Pivot Tables, and Visual Basic.

Thankfully, the soft skills are a little easier to learn that becoming familiar with a new language. If you’re transitioning careers, you can practice a few of these in your current role and prepare to highlight your soft skills when you interview for jobs in Data Science.

How I Got My Job in Data Science (and a 95K pay increase)

Ayodele (eye-ya-deli) — Tue, 17 Dec 2019 22:31:53 +0000

In 2014, I graduated from the University of Pittsburgh with a Digital Media and Communications degree. I worked in social media marketing for a travel marketing agency and ended up getting let go a week after moving to Denver (after they assured me I’d be able to work remotely).

This was the beginning of my career transition. I thought I'd only be able to get out of the field I was in by getting a degree in something else, which is not the case, but that's what I did.

A few months into grad school a friend got me a Data Science job at a six-person startup making $19 an hour. At the time this was the most money I had made so I was fine with doing it at that rate. It started off as data entry and gradually progressed to analytics, dashboarding, and regression and forecasting near the end. After a year of stressful work and going to school full-time, I quit my job with a full year of experience under my belt. I wouldn't suggest it, but I have worked myself to the bone and burnt out and 12/10 would not recommend. I took random jobs to pay the bills. Ever need a cracked screen fixed? I worked as a technician fixing phones at the Apple genius bar. I was able to balance part-time jobs and school, but I cut back on a LOT of my expenses. Almost a year after I left my DS burnt out and angry, I got a job at a drone for good startup and while I got “sweat equity” I was making $0 every two weeks from August until December when I left.

This is when I started working at a gas station. I worked overnights in a shitty part of town and calling the cops at 4 am to remove people from the story was common. I was making a whopping $9/hr with the only benefit being enough time overnight to get my grad school reading assignments done.

I’d come home after an overnight shift, take a nap, train models to recognize firearms, go to class, then work overnight again. It was hell, but the product I was working on could actually save lives. The difference in my willingness to sacrifice my sleep was knowing how many people of color have been murdered by cops with no recourse. While I was working a lot, it wasn’t 60-80 hours to prove to my boss I was worthy. I was doing it for free in hopes my work would reduce just one police killing.

The exception to my mentality against working yourself sick is if it’s directly for the greater good. 99.9% of jobs don’t get close.

I also had the benefit of having a job where I could work on other things. Aurora, CO is pretty slow at night and in winter I’d maybe see a single customer per hour. On average I spent 40 hours at work at the gas station, 10-15 hours on school (mostly done at work) and 20 hours on my ML job. The ML job was great for my resume, but working there wasn’t a complete necessity. There’s not a lot of cross over in what my day to day looks like now so it’s likely I’d still be in the role I’m in without that experience.

Here are some ways I made my career change:
‍
Strategic education 🏫

I had the advantage of already having a Bachelor's degree, but since it wasn’t technical I went back to school for my Master's degree. I chose a school that was the least expensive I could find that wouldn’t require me to take the GRE (another $200 I couldn’t afford). I was wary of online masters programs so I was happy to see Regis University has a physical campus as well as a long history. I have some guidelines I use to help determine which online programs are legit and which are selling snake oil and hype.

In grad school, I took on additional loans so I could pay for books ($400-600/semester), rent and utilities ($500/mo), and food. knowing the average entry-level Data Science job I'd eventually get would be good enough to pay them back easily. When I started grad school in June 2016 I was on unemployment and itching to learn something new. I was able to bet on my major being a hot field with fairly low stress and high enough wages to live in an expensive city AND pay back any loans I took.

Networking 📶

I went to PLENTY of tech conferences free just applying for their student scholarships. Not all conferences restrict this to students. Many offer free tickets (and sometimes expenses paid) to underrepresented folks and ones who just and afford conference tickets.

I went out of my way to go to meetup events when I just want to sit at home and watch tv. I made time to talk to people who seemed influential in the field while I absorbed the tools and techniques they used. Going to events and talking to people got me my first job as a Data Scientist. I met someone who would become a good friend at a networking event and they introduced me to the CEO of the company I worked as a Data Scientist for.

Cold emailing/messaging 🧤

What I love about machine learning especially is that people in Data Science and ML want to talk to other people about their work. I could fill binders for the messages I sent data people I admired on LinkedIn. There’s most definitely a right and wrong way to reach out (which I plan on covering in a blog post soon), but asking what their day to day is like or how the data science department is situated at their company can help you get insight on if it’s even a field you want to get into. I had a cold outreach a template and just changed some details to match who I was reaching out to. Some people didn't respond but the ones that did gave me so much insight on what the job was like before I was too deep to back out of it.

Find Community 🌈

The best thing I did for my career was to join a bunch of public data science and tech slack groups. I have a Twitter thread detailing some of my favorites, but I had a constant influx of job listings and resources coming at me each day via slack. I made great friends, met some cool slack friends at conferences, and helped another get a job at my company. While it may be worth noting you might want to turn off all slack notifications once you're in 30 channels like I am. Even if it feels like you’re learning in a vacuum, you don’t have to. Don’t just lurk, participate, ask questions, and get rewarded.

Balance learning and practicing ⚖

Data science is extremely hands-on. It’s impossible to just listen to theory and be able to implement it. I spent about half my time reading about theory and another half coding these projects out. Ideally, you'd spend a quarter of your time learning about new topics and the rest implementing it. I say this because in my experience it takes a lot longer to work through a project, tune hyperparameters, test on metrics, etc that it does to watch a video explaining the concept. This practice helped me get an intuitive understanding of what the models I made were doing.

Calculated risk 📈

I weighed a lot of tech career options before settling on Data Science. Like someone I saw on twitter mentioned, tech jobs, in a way, are a cheat code. We earn high wages with relatively low experience, have low(er) stress environments, and get great fulfillment from working on high impact projects (my company’s app has 30 million users!). I wouldn’t have seen this kind of rapid change and financial stability had I gone into nursing or law from marketing. For other tech careers, I seriously considered software engineering, front end development, and data engineering before deciding on data science.

I also recognize to most my degree has less merit than one from MIT or Stanford, but I paid $28k for tuition total instead of taking on nearly double that at more prestigious schools. I also knew that risk meant I was going to have to prove my work the employers differently.

Projects 💻

So most of my hands-on projects came from school assignments but for a few, I made sure to go above and beyond what was required for class. Now that I’m interviewing candidates I can’t stress enough, you can take a class or toy project meaningful with the right tweaks and interviewers want to see that.

The truth is, you can get these projects done in your free time, but guidance is one of the big aspects of learning data science that I needed. It’s also one of the reasons I took a lot of in-person classes in grad school. I suggest mentorship for those not in a formal degree program. It's one of the best ways to stay on track while not having to ask Stack Overflow or Cross Validated (it's Data Science-y cousin) for help. You can learn to be a Data Scientist or Machine Learning Engineer completely online via a formal degree or YouTube but it may not be a straight path or go by quickly.

Impactful work 💕

Do good shit and get recognized for it. When I was at my first data science job I reduced our customer churn and increased sales by 3X over a year. When I was at the drone company I was training sensors on real firearms in real-time. It’s not always easy to find roles where you can do this kind of thing, but that was my trade-off on being severely underpaid. I was hardly paying rent but had a HUGE role in building these machine learning models since I was the only one who knew how.

I suggest if you want to try this use Upwork or another freelance site like Incluzion to find small companies who can pay you a grand or two for some analysis. Find a passion project and pull the tools you need to get started.

Interview prep 🧠

Only a year into my two-year program, I started interviewing for full-time data science roles. I fully knew I wasn’t ready for a job but part of it was collecting intel. I didn’t finish nearly all the take-home tests I was assigned but I started to have a good understanding of the things companies would want me to answer. I spent most of my time nose deep in interview questions and learning the right and wrong ways to answer them. I have some favorite interview books like As Heard in Data Science Interviews by Kal Mishra.

The secret sauce 💦

I did a lot of research before I was going to go into Data Science. I took a hard look at careers like software engineering, UX research, and data engineering before deciding.

I mean it when I saw changing careers as a calculated risk. Some people risk a lot and some have less to lose. I didn’t have much to lose but I calculated what I stood to gain. I looked at entry-level data science salaries, job hours, stress levels, and what skills I had that would transfer over easily. I think most people forget this, but leverage the skills you already have. If you’re in another industry you can learn to code and build tools that will fill a need in your industry. If you’re in retail like I was you can use your interests to guide you. Please don’t forget to remind the interviewers of the skills needed for the role that you gained before the role.

I talked to people about what their day to day was like. I asked what tasks they did a lot and what tools they used. I listened to a million podcasts like Becoming A Data Scientist and Super Data Science. Both give great insight into what it can be like to work in data science. One of which has been kind enough to have me on as a guest where I talked about my journey in more detail.

I found what data scientists complained about like data cleaning and decided if that was something I could handle dealing with. There are plenty of studies and surveys on the hot new field, but if there wasn’t I would ask people at meetups what they did.

Helpful hint: ask people about a recent project that was difficult or particularly enjoyable.

I know this was very specific advice but that doesn’t mean you can’t apply it to your goals. Now, I’ll tell you what most people who had a come up don’t want you to know. I’m not special for having done this. The situation around me was shitty and desperate enough to where I didn’t need more motivation to do better. My motivation was not being able to buy my family Christmas gifts. It was having to walk by the Salvation Army ringers with shame because I could use more help than I could help other people. I was food pantry broke, couldn’t afford a Wendy’s 4 for $4 broke, but I’m not some kid genius who understood this stuff from the jump. I was in remedial math through college and worked hard to learn the new concepts. Learning math was easier than calling the cops to remove the 4th druggie of the night who wouldn’t leave.

I made a lot of mistakes along my way so I’ll save you the time. If I were to redo my career transition now here’s what I would do:

Research career paths
This is so much more than just Googling, but asking people on Twitter, LinkedIn, finding podcasts, and talking to people at events about what their job is like.
Cut out most media that isn’t about what you’re studying
Once you start consuming the types of media that practitioners do it’ll be easier to talk the talk. I wish I had spent more time around the subjects of statistics and math before starting in Data Science, but you have to start consuming new media.
Find a flexible job
Obviously, it needs to pay the bills but try for a job that gives you a little free time (if you can) this way you can follow other industry nerds on Twitter or study up while getting paid. If you can’t, come up with a weekly learning plan and start putting in 10 hours a week after work. Dedicate one day on the weekends to studying for your new career. Even if you can’t do your homework at work, if you can read a couple of industry-related articles it’s a start.
Go to events in your local area
I can’t overstate this for career changers. You need to hear the vocab used in person, get ideas from other people, and just be a sponge to the new information. Some people live in remote areas and for that there are webinars. Plenty of free ones too that way you can watch one on AI over lunch and spend an hour or two learning to code when you get home.
Stop lurking and engage
For years I lurked on Twitter, Reddit, LinkedIn and I realized just asking questions was enough. Even in the past few months, I'm realizing the kind of response I can get from just asking into the ether. Now I have enough knowledge to be valuable but engage with people instead of being passive.
Get experience ASAP
The field of ML and AI is booming and can often leave you feeling like you're perpetually behind, I feel this way a lot! It's all good because you can get experience in Data Science by working on Kaggle competitions, getting a freelance gig, building robust projects from mere online videos. Whatever you can do to

If you’re in survival mode like I was, you have options. If you can get loans for school that’s one route. If you can work on this clandestinely at work, that’s another. You can make a career change part-time, after your regular job with some guidance and the right tools without burning out or working 80 hours a week.
Don’t listen to people who tell you if you’re not spending 20 hours outside your job learning something else, you just don’t want it enough. Their smugness will fill their pockets faster than cash will. There are billion-dollar companies where the expectation is to work 40 hours and GO THE FUCK HOME. There ARE industries that prioritize balance vs wanting you to work as much as you're physically capable.

Not one of these folks on chess, not checkers Twitter has a $100 billion company so why should you listen to their "grind til you die or you're just lazy" message? Fuck. That.

Finally, when you're interviewing for a job, ask them how frequently they work overtime. Ask them what their PTO policy is and stay away from crooks that offer unlimited PTO. If you can find a tech company that still accrues PTO, you’re golden. Ask them what their longest task was or the longest project. Ask them the last time they stayed up late working on a work assignment and then ask if it’s because they were interested and going down a rabbit hole or if they felt pressured to get it done. Do your digging to find out if the company's culture is toxic or one you thrive in.

If you might be considering a career in Data Science try taking this quiz to understand which roles you might want to go after.

For those wondering, My gas station job came out to $17,280/yr before taxes. My Data Science offer was $112,500/yr.