Forem: Claudio Davi

What does a Machine Learning Engineer actually do?

Claudio Davi — Mon, 21 Jun 2021 18:29:24 +0000

Recently Paige Bailey - Twitter, Github - Tweeted an analysis she did on the state of Machine Learning cohorts. Using her Article as a basis, I'd like to share my view on what I've doing for the past three years as a Machine Learning Engineer and where I believe I find myself in.

There's a lot of contradiction of what a Machine Learning Engineer is today and most job openings just assume what it is based on the company's needs. Researches like Paige's help us understand and begin to draw lines of what we should be doing or even know what our job title should be.
I constantly receive job offers asking if I'm interested in positions such as Data Engineer (ML) or Software Developer - Data, which to my understanding, does not fit my profile.

With this in mind here's my take on what I, as a Machine Learning Engineer, am doing:

Basic Exploratory Data Analysis (EDA): It is impossible to do any Machine Learning work without knowing your data first. I say BASIC here because I'm looking to find the underlying structure of the data, not doing a fancy report or discovery. What I usually do here is to find which features I'm going to use, which are irrelevant and which machine learning methods are the most appropriate for the problem at hand. What I DON'T do: Fancy reports, customer dashboards and presentations. That would be a Data Scientist's or Data Analyst's job.
Traditional Machine Learning: As I stated before, it is quite hard to do ML without EDA, so after finding out the problem I'm dealing with, I'll generally mess around with some ML techniques. That could be Sklearn, XGBoost, or even Tensorflow/Pytorch. Again, I'm not looking for state of the art performance, nor big fancy models as you see on NeurIPS article's or OpenAI releases. I'm looking to solve a small very specific problem inside my domain. Nothing fancy. Considering this, it would put me in somewhere along the lines of the Data Scientist – Business, Traditional ML.
Product Integration: Here is where I think the Engineering part of ML Engineering comes in. I already have built a model that solves a very specific domain problem, now what can I do to make it available for everyone? Here I'll usually look for the best approach to put the model in production, create a data pipeline and if needed updating and training pipelines for future use. Generally this is done with Docker containers and some frameworks like MLFlow and a hosting platform like AWS or GCP. This will fill into the ML Ops practitioner bucket

As the team evolves and more people come in, I can see the ML Engineer focusing more and more on 2 and 3. Sometimes you can receive a ML Model built on Sklearn with several preprocessing steps, which will yield a model with hundreds of MB if not GB and poor performance. Those can be optimized and I already have found myself doing these kind of optimizations.

As a finishing thought, according to Paige's cohort studies I'd be a mix of MLOps practitioner and Data Scientist - Business, Traditional ML. These are the roles I'd fit in the most and probably the mix that - for me - would describe what a ML Engineer does.

What do you think? Am I missing something?
What is a Machine Learning engineer to you?

You Have Built a Machine Learning Model, Now What??

Claudio Davi — Fri, 26 Jun 2020 14:57:41 +0000

This topic feels too close to home to many of us, it does for me at least.

Almost every tutorial you will find on the Internet will help you build a very simple model to show you that Machine Learning works and how awesome it is. You followed it, finished, proud even. You used it on the few examples that they gave you.

IT WORKS!

Cool, but now what? Should I use my Jupyter Notebook to make predictions?

That doesn't feel right, does it? It only does not, as it is not very useful, like, really, you are not going to use it like that.

One of the first steps that you go through when building a machine learning model is how to preprocess your data.
This process alone raises one of the most important questions that the tutorials are not telling you:

How will I preprocess the incoming inputs (requests from clients)?
If I create another sklearn's StandardScaler(), won't be I just creating a scale based on nothing?

Spoiler: Yes you are.

How can I export to save this thing then?

Now it depends on your stack.

If you are following the most common pipeline which uses Pandas for manipulation, Scikit-learn for preprocessing and Keras or even one of sklearn's models, you can have a sklearn Pipeline().

That way you can divide your pipeline into small steps, one for each preprocessor and then use your model.

This process tends to be the easiest one for beginners, and I highly recommend it. Although, you will probably see that your pipeline can become a huge .pkl/.joblib file if you are using big datasets.

My workflow for this type of machine learning environment is something along the lines of:

## Create Pipeline
clf = Pipeline(steps=[
        ('preprocessor', preprocess), 
        ('classifier', sklearn_wrapper.KerasClassifier(keras_model, args))
])

clf.fit(train, labels)

## Save model as a different file, so keras can save everything correctly

clf.named_steps['classifier'].model.save(f'./model/{MODEL_NAME}.h5')
clf.named_steps['classifier'].model = None

## Dump the pipeline (this file is the one that can be huge)
with open(f"./new_model/{MODEL_NAME}-pipeline.pkl", "wb") as model_file:
    pickle.dump(clf, model_file, protocol = 4)

To use it, just rebuild the pipeline with pickle.load and the model with keras.load_model(), set model back, and it should work fine.

Another alternative, is to use Tensorflow for everything.
You can still have your keras model, but now you will have to preprocess everything inside the tensorflow graph. And for newcomers, I would not go this way first.

With tensorflow, you should be at least familiar with feature_columns, tf.data and tensorflow-transform. As I said, I would not go straight to this route. I rarely use it myself.

Now you have your model saved that will preprocess your data for you on the fly.

Well, how do I use it now?

Glad you asked.
You can use Google Cloud's Machine AI Platform Models or any other model hosting platform of your choice. However with preprocessing and everything you should build a custom prediction routine. Be aware of the prices, request-rate and auto-scaling options.

I personally use Docker + Flask + Gunicorn, deployed under a load balancer of your choice. Why? Simplicity. Docker and Flask make everything easy for me to edit and improve. Flask handles my requests, Docker my infrastructure, gunicorn the workers. There are several tutorials for those. I'll link one from a follow Dev.to writter Docker-Flaks-Gunicorn.

Best Practices for this architecture:

Save your model on the cloud - I use AWS S3 and sometimes Google's Cloud Storage - download it on your docker CMD command. so every instance will have it on startup.
Build the model before your first endpoint. Alongside with your Flask(app) instance. This will save you tons of resources, because your are not building it every request.
Stress test. Check your input/output rate, scale as you see fit.

This is what works for me. Your situation may be different, I hope I have provided you with a good few options to solve your problem.

I will finish this article with the elephant in the room.

What about TFX?

I may not be the best person to talk about it, as I have very little experience with it. However, the fact that most of you have never heard of it proves the point that I want to make.
There's no one using or writing about it, in other words, there's no community. There's the documentation and the source-code, and that's all.
I've had the privilege to talk with one of the TFX's core devs a couple of weeks ago. They are aware that the documentation is lacking and everything is very experimental at this point.
However it has the potential to be my go to technology for machine learning pipelines. It solves most of the problems I have when building machine learning models for production. But not at this stage of development.

I will definitely keep experimenting with it and hopefully change my mind in the near future.

That's it.

What are your thoughts on ML Processes and pipelines?
if you are using your machine learning models in production, what is your go to stack?

What do you do to stay productive?

Claudio Davi — Tue, 19 Nov 2019 20:47:16 +0000

I'm worried about my mental health, but I'm also worrying - and I feel like I'm not alone - that I am not doing enough.

We were all sold the idea that we should be productivity machines, working all hours and building the future. However, we forget to take care of our mental health in this process. Hours that should be labeled 'me time' becomes 'learning time' and we find it very hard to make that distinction.

I work 8 hours of my day and I want to make the most of it. So, what do you guys do to stay productive and always on top of your game, without sacrificing 'me time'?

State of AI 2019: TL;DR;

Claudio Davi — Thu, 11 Jul 2019 00:27:29 +0000

Another year has passed since the very first state of AI report, so this year I decided to do (as it turns out not so) quick run down into a couple of topics that I find interesting on the report.
The report is a compilation of 'points of interest' put together by a VC and an Angel investor both focused on AI technologies. The report can be found on slideshare

First things first, they considered the following key dimensions in their report:

Research: Technology breakthroughs and their capabilities
Talent: Supply, demand and concentration of talent
Industry: Large platforms, financing areas of application for AI-driven innovation today and tomorrow
China: With two distinct Internets, they considered AI in China a different category
Politics: Public opinion on AI, economic implications and emerging geopolitics of AI

Research

Conquered

AlphaStar: DeepMind beat a world class Startcraft II player by a whooping 5-0. This represents a huge breakthrough because the AI was running on imperfect information and controlling a huge action space in real time, on top of that, it had to make strategic decisions for long term goals, I personally was watching the match live on Youtube, as a former SC player, it was thrilling.
- How: The network was trained watching humans play (supervised learning) then it created agents and played against itself (Reinforcement Learning).
OpenAi Five: Dota2 bot win rate goes up to 99.4% win rate over 7,000 games with 15,000 live players.
- How: They upped their hardware, this new version now consumes about 800 petaflop/s-days. ### Next
Creating Creativity: We use rewards in Reinforcement learning to make our algorithm learn, as in real life it's hard to encode rewards, researchers are starting to also give rewards when the algorithm tries new approaches to the problem.
From Lab to Prod: With the recent release of Facebook's Horizon a Reinforcement Learning platform as an Open Source Software, RL systems optimizations starts to become more widespread. This platform can be used to optimize a set of actions based on the state of the agent at any given moment.
AlphaFold is at it again: Despite the huge breakthrough last year in protein folding, this year Alpha Fold has significantly outperformed itself.
Natural Language Processing: This going almost without saying, after the scandal around OpenAI's GPT-2 Model going dark, several labs reported great improvements on this area. Pre-trained models is where it is at guys!
Common Sense - Some Machines have it, most humans don't: Researchers at NYU trained inferential knowledge to acquire common sense and reason about previously unseen events.
Growing Interest on Federated Learning: This is like, distributed systems, but for AI. Instead of having a monolithic centralized machine, training is distributed into mobile devices for computation. Google just released their architecture on Federated Learning and how they are using it on GBoard. On top of that, Tensorflow gains a new flavor: Tensorflow Federated. Keep one eye on this one, the health care industry is all over this one.
Privacy: Companies like Google are starting to give emphasis on data privacy for ML models, Tensorflow Privacy now gives a strong mathematical guarantee that they do not remember your personal details.
Reading Thoughts: Neural networks can decode your thoughts from brain waves. The technique is still invasive requiring an electrocorticography (for now) but who knows about what can we develop in the next few months???
Evolutionary AutoML: Evolutionary algorithms are being used for neural network architecture and hyperparameter optimization. The previous technique consisted of RL algorithms.
The age of Deep Fakes: State of the art GANs are evolving fast, from images to audio, some companies are creating full body pictures with clothes for retail purposes. You guys remember the AI that sounded like Joe Rogan? So, that, and image, and video, and basically everything now.

Talent

Top Players (By number of articles on NeurIPS):

Google
MIT
Stanford
CMU
Berkeley

88% of the researchers at NeurIPS, ICML or ICLR were men.
80% of AI professors are men and 75% of students of AI are also men
Data Labelling jobs are on the rise. ($1.47/h)
China's studies are growing in average citation rate, despite Europe securing the most number of AI papers published.
China again: AI courses enrollment is on the rise.
US, China, UK, Germany and Canada account for 72% of the authors of papers in 2018-2019
Hiring AI researchers is going down. Perhaps this is a sign to bring this applied research into production environments.

Industry

With big investments, comes ..: Venture Capital in AI grows to reach > $27B/year
Big Tech investments: Big Tech continues to eat AI-first startups
Robots taking over: Robotics are on the rise, several processes can be improved like, warehouse management, supply-chain for e-commerce and even robots for building robots.
Self Driving Road Bump: Self Driving Cars are not the hype at the moment, several players retreated and most companies are missing deadlines.
Demand forecasting: As more data becomes available in digital form, it becomes increasingly more interesting to use machine learning to forecast demand. With better forecasts, business can prepare supply and reduce waste increasing their profitability. Usage Examples: Travel, Local Businesses, Logistics, Retail and others.
Reading Machines: With the advances of Natural Language Processing, there's a growing interest in reading and writing machines.
AI Drugs: The FDA approved 3 AI based diagnostic products and pharma companies are investing in AI-driven drug development.
Increasing the knowledge gap: AI Patents are on the rise. The most patented field is Computer Vision
AI Hardware: AI hardware seems to be an interesting field to be in. Several global startups are investing into this technology
Handheld AI: Samsung, Huawei and Xiomi top the list for performance in AI tasks for mobile devices.
Huawei Hold the Doors: Huawei owns most of the essential patents for 5G tech, likely becoming the key player for building an ecosystem for network providers.

Politics

This section is mostly about two surveys, so I'll summarize the results here:
Americans are not in favor of AI for Warfare unless they know someone is doing it
Americans don't know who should decide how AI is developed and deployed
Most people agree that companies should have an AI review board to address ethical decisions
The general public believe that high level AI is just around the corner - 9 years - (considerably sooner than the predictions made by experts)
The majority of Americans trust the US military or academic researchers to develop AI
Facebook was rated the least trustworthy AI developer.
Governance Challenges: Preventing AI-assisted surveillance from violating privacy and civil liberties; Preventing AI from spreading fake or harmful content; Preventing cyber attacks; Protecting data privacy.
Angela Merkel plans to invest 3 billion Euros in AI by 2025
Finland is training 1% of its population in the basis of ML, becoming the first European country to put a national AI strategy in place.
Finland's goal is to become the a world leader in practical AI applications.
Trump signed an executive order creating a program called the "American AI Initiative" with unclear goals.
With Google's exit from project Maven (ML enabled military imagery) a startup was founded and takes over the project and is pushing to employ drones into large-scale conflicts
With recent developments in cameras, mass surveillance is growing in technical sophistication
California approved a bill last year that criminalize the use of bots to interact with a California person "with the intent to mislead".
As stated before, deep fakes are being discussed in the American House of Representatives.
Researchers find racial bias in Amazon's Rekognition
Georgetown Centre for Security and Emerging Technology is the largest US centre focused on AI and policy.

China

Alibaba and JD.com have entered the animal and insect husbandry business.
China R&D spending grows fast but lags in market share
China is going through increases in industrial automation and job displacement.
Robots are taking over warehouses in China
China is publishing more high impact machine learning academic research

Predictions

There is a new wave of start-ups for NLP. They will raise over $100M
Self-driving will stay in R&D.
Privacy-preserving ML will be adopted by Fortune 2000 companies to beef up data security and privacy policy.
Institutions will build AI undergraduate degrees to fill talent void.
Google will have a major breakthrough in quantum computing hardware, triggering at least 5 startups to do quantum ML.
Governance and AI will become a bigger topic and one major AI company will make a substantial change to their governance model.

What are your thoughts on their report? What are you most excited about in AI for 2019-2020?

Show us your current Wallpaper!

Claudio Davi — Thu, 04 Jul 2019 12:08:53 +0000

Reducing VSCode Memory Consumption

Claudio Davi — Wed, 19 Jun 2019 19:54:57 +0000

How to reduce VSCode memory usage

I've been using VSCode for quite a while now and one of the most annoying things that I have noticed is how much memory it uses, specially in comparison with Sublime Text.

I will present you a few tips that I have found to at least make it usable for big projects.

Disclaimer: I mostly use Python, therefore I'm sure that you will find a lot more options to optimize for Javascript or your preferred language.

Most of the tips below must be placed into your user settings (JSON)

Telemetry

First, did you know that VSCode sends data to Microsoft about it's usage?
If you want to turn it off is quite simple, place

"telemetry.enableTelemetry": false

into your configurations and you are all set.

Search Indexing

Search is one of the most memory consuming activity VSCode does. It has to keep an index of all your files and their contents. You probably don't want to search inside your node_modules/ or env/ folder right?
I have had this problem before and I don't know if they come disabled by default now, but it is on my config file so here it goes:

"search.exclude": {
    "**/node_modules": true,
    "**/bower_components": true,
    "**/env": true,
    "**/venv": true
  }

File Watcher

The file watcher is used to detect changes in your working files and folders.
If you are like me and every new package you have to pip | npm install it on the fly you probably have a lot of changes in your secondary files and folders.
So we are going to disable the watcher for those folders and whatever else we don't want to follow, like our git/objects folder.

 "files.watcherExclude": {
    "**/.git/objects/**": true,
    "**/.git/subtree-cache/**": true,
    "**/node_modules/**": true,
    "**/env/**": true,
    "**/venv/**": true,
    "env-*": true
  },

Organizing your Explorer

Ok, now we have optimized for performance, now we want to optimize for productivity. One of the most important things to do is to reduce the clutter in your workspace.

To do that we are going to remove files from the Explorer tab. Why? We don't really want to see files that we are not going to work with.

"files.exclude": {
    "**/.git": true,
    "**/.DS_Store": true,
    "**/.vscode": true,
    "**/__pycache__": true,
    "**/.pytest_cache": true,
    "**/node_modules": true,
    "venv": true,
    "*.sublime-*": true,
    "env*": true
  }

That will leave you with most of the clutter out of sight. Which is Great!

Extra Tips

A few extra tips to help you:

Workspaces

Workspaces are great, create several of them. I always have a few instances of VSCode open for the same project, if you are doing Full-Stack use one for back-end and another for front-end development. If you, like me, are building microservices you can use one workspace for each service. You are going to see how clean it all becomes.

Extensions

Keep the minimum amount of extensions that you can possibly have, most of them are not optimized. Keep what you use daily and disable or uninstall the others. I go through my extensions once a month and do a Marie-Kondo-style cleanup.

Full Script

"files.exclude": {
    "**/.git": true,
    "**/.DS_Store": true,
    "**/.vscode": true,
    "**/__pycache__": true,
    "**/.pytest_cache": true,
    "**/node_modules": true,
    "node_modules": true,
    "venv": true,
    "*.sublime-*": true,
    "env*": true
  },
 "search.exclude": {
    "**/node_modules": true,
    "**/bower_components": true,
    "**/env": true,
    "**/venv": true
  },
"files.watcherExclude": {
    "**/.git/objects/**": true,
    "**/.git/subtree-cache/**": true,
    "**/node_modules/**": true,
    "**/env/**": true,
    "**/venv/**": true,
    "env-*": true
  },

Do you have any more tips that can improve VSCode performance? Share with us!

The Journey From Java to Python

Claudio Davi — Tue, 11 Jun 2019 19:44:28 +0000

Introduction

I'm definitely not one of these wizard kids that started programming at 13 and were a genius by the time they completed 20. I started programming at 18 years old and I started with Java. I was terrible at it, the first time I saw a for loop I thought it was magic.
I'm 25 now and have been programming since then, 3 years professionally and two of them coding in Java.

Background and Java

I went to college get the title of BSC in Software Engineering, I thought all my colleagues were at the same level - level 0 - but boy was I wrong. Almost everyone there could program, I did not. That made me the underdog, and as such I had to learn and study harder than most people.
From 2011 to 2016 I've been studying Java, it's patterns, code styles, best practices and idiosyncrasies. I'm sure I barely scratched the surface of the language, but after some time I started to feel productive. That's when I landed my first job as a Software Engineer Jr, back in January 2017.
Fast forward to September of the same year. At that time I was into Machine Learning and had already developed a few ML algorithms in Java by hand, then I experimented developing a few of them in Python.

Python with Love

At this point I already knew how to code and collaborate. Picking up a new language was not going to be as hard as it was the first time, but Python was so friendly and the community so active that I barely had any problems.
It took me another year to get a job as a full time Python developer, but today I'm way more productive in Python than I have ever been in Java. It feels fresh, clean and powerful.
It used to take me a day to go from a Factory to a full REST endpoint with Java, now it takes me a couple hours to model a basic CRUD application with Python and Flask.

What I miss the most about Java

Python also has its weaknesses, after I transitioned what I miss the most is the static checked type system. That saved me a lot of trouble and ensured my code would work wherever I needed to use it. In python to remedy this, I use the type hints and doc strings. It is still not checked and still gives me run-time errors, but when your software is growing, it can save you a lot of time that you would spend reading and refactoring your old code.

How could you transition to another language?

I started reading and experimenting, published a few articles on Medium in Portuguese (my native language), started tweeting about it and got some contacts. I suggest you follow the same path, study, write about it, tell people about your experiences and teach.
I started programming fairly late considering my peers, If I did it, you can do it too.

Did you change paths in your career? Tell me in the comments!