Forem: Sergios Karagiannakos ( AI Summer )

Top 10 courses to learn Machine and Deep Learning (2020)

Sergios Karagiannakos ( AI Summer ) — Mon, 17 Feb 2020 21:49:51 +0000

Machine Leaning Courses - The ultimate list

You know what I was hoping to have when I started learning Machine Learning. An all in one Machine Learning course. At the time, it was really tricky to find a good course with all the necessary concepts and algorithms. So we were forced to search all over the web, read research papers, and buy books.

Luckily that’s not the case any more. Now we are in the exact opposite situation. There are so many courses out there. How I am supposed to know which one is good, which includes all the things I need to learn. So here I compiled a list of the most popular and well- taught courses.

I have personal experience with most of them and I highly recommend all of them. Every Machine Learning Engineer or Data Scientist I know suggests one or many of them. So don’t look any further.
Ok, let’s get started.

1) Machine Learning by Stanford (Coursera)

This course by Stanford is considered by many the best Machine Learning course
around. It is taught by Andrew Ng himself ( for those of you who don’t know him,
he is a Stanford Professor, co-founder of Coursera, co-founder of Google Brain
and VP of Baidu) and it covers all the basics you need to know. Plus, it has a
rating of a whopping 4.9 out of 5.

The material is completely self-contained and is suitable for beginners as it
teaches you basic principles of linear algebra and calculus alongside with
supervised learning. The one drawback I can think of, is that it uses Octave (
an open-source version of Matlab) instead of Python and R because it really
wants you to focus on the algorithms and not on programming.

Cost: Free to audit, $79 if you want a Certificate

Time to complete: 76 hours

Rating: 4.9/5

Syllabus: Linear Regression with One Variable

Linear Algebra Review
Linear Regression with Multiple Variables
Octave/Matlab Tutorial
Logistic Regression
Regularization
Neural Networks: Representation
Neural Networks: Learning
Advice for Applying Machine Learning
Machine Learning System Design
Support Vector Machines
Dimensionality Reduction
Anomaly Detection
Recommender Systems
Large Scale Machine Learning
Application Example: Photo OCR

2) Deep Learning Specialization by deeplearning.ai (Coursera)

Again, a course taught by Andrew Ng and again it is considered on the best in
the field of Deep Learning. You see a pattern here? It actually consists of
5 different courses and it will give you a clear understanding of the most
important Neural Network Architectures. Seriously if you are interested in DL,
look no more.

It utilizes Python and the TensorFlow library ( some background is probably
necessary to follow along) and it gives you the opportunity to work in real-life
problems around natural language processing, computer vision, healthcare.

Cost: Free to audit, $49/month for a Certificate

Time to complete: 3 months (11 hours/week)

Rating: 4.8/5

Syllabus:

Neural Networks and Deep Learning
Improving Neural Networks: Hyperparameter Tuning, Regularization, and
Optimization
Structuring Machine Learning Projects
Convolutional Neural Networks
Sequence Models

3) Advanced Machine Learning Specialization (Coursera)

The advanced Machine Learning specialization is offered by National Research
University Higher School of Economics and is structured and taught by Top Kaggle
machine learning practitioners and CERN scientists It includes 7 different
courses and covers more advanced topics such as Reinforcement Learning and
Natural Language Processing. You will probably need more math and a good
understanding of basic ML ideas, but the excellent instruction and the fun
environment will make up to you. It surely comes with my highest recommendation.

Cost: Free to audit, $49/month for a Certificate

Time to complete: 8-10 months (6-10 hours/week)

Rating: 4.6/10

Syllabus:

Introduction to Deep Learning
How to Win Data Science Competitions: Learn from Top Kagglers
Bayesian Methods for Machine Learning
Practical Reinforcement Learning
Deep Learning in Computer Vision
Natural Language Processing
Addressing the Large Hadron Collider Challenges by Machine Learning

4) Machine Learning by Georgia Tech (Udacity)

If you need a holistic approach on the field and an interactive environment,
this is your course. I have to admit that I haven’t seen a more complete
curriculum than this. From supervised learning to unsupervised and
reinforcement, it has everything you can think of.

It won’t teach you Deep neural networks, but it will give you a clear
understanding of all the different ML algorithms, their strengths, their
weaknesses and how they can be used in real-world applications. Also, if you are
a fan of very short videos and interactive quizzes throughout the course, it’s a
perfect match for you.

Cost: Free

Time to complete: 4 months

Rating:

Syllabus:

Supervised Learning
Unsupervised Learning
Reinforcement Learning

5) Introduction to Machine Learning (Udacity)

src="https://www.youtube.com/embed/ICKBWIkfeJ8"
frameborder="0"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen>

This introductory class is designed and taught the co-founder of Udacity
Sebastian Thrun and the Director of Data Science Research and Development Katie
Malone. Its primary audience is beginners who are looking for a course to get
started with ML. Again if you like Udacity’s environment (which I personally do),
it is an amazing alternative to get your foot in the door.

Cost: Free

Time to complete: 10 weeks

Syllabus:

Welcome to Machine Learning
Naïve Bayes
Support Vector Machines
Decision Trees
Choose your own Algorithm
Datasets and Questions
Regressions
Outliers
Clustering
Feature Scaling

6) Deep Learning Nanodegree (Udacity)

The Deep Learning Nanodegree by Udacity will teach you all the cutting-edge DL
algorithms from convolutional networks to generative adversarial networks. It is
quite expensive but is the only course with 5 different hands-on projects. You
will build a dog breed classifier, a face generation system a sentiment analysis
model and you’ll also learn how to deploy them in production. And the best part
is that it is taught by real authorities such as Ian Goodfellow, Jun-Yan Zhuand,
Sebastian Thrun and Andrew Trask.

Cost: 1316 €

Time to complete: 4 months

Rating 4.6/5

Syllabus:

Project 1: Predicting Bike-Sharing Patterns (Gradient Descent and Neural
Networks)
Project 2: Dog Breed Classifier( CNN, AutoEncoders and PyTorch)
Project 3: Generate TV Scripts (RNN, LSTM and Embeddings)
Project 4: Generate Faces (GAN)
Project 5: Deploy a Sentiment Analysis Model

7) Machine Learning by Columbia (edX)

The next in our list is hosted in edX and is offered by the Columbia University.
It requires substantial knowledge in mathematics (linear algebra and calculus)
and Programming( Python or Octave) so if I were a beginner I wouldn’t start
here. Nevertheless, it can be ideal for more advanced students if they want to
develop a mathematical understanding of the algorithms.

One thing that makes this course unique is the fact that it focuses on the
probabilistic area of Machine Learning covering topics such as Bayesian linear
regression and Hidden Markov Models.

Cost: Free to audit, $227 for Certificate

Time to complete: 12 weeks

Syllabus:

Week 1: maximum likelihood estimation, linear regression, least squares
Week 2: ridge regression, bias-variance, Bayes rule, maximum a posteriori
inference
Week 3: Bayesian linear regression, sparsity, subset selection for linear
regression
Week 4: nearest neighbor classification, Bayes classifiers, linear
classifiers, perceptron
Week 5: logistic regression, Laplace approximation, kernel methods, Gaussian
processes
Week 6: maximum margin, support vector machines, trees, random forests,
boosting
Week 7: clustering, k-means, EM algorithm, missing data
Week 8: mixtures of Gaussians, matrix factorization
Week 9: non-negative matrix factorization, latent factor models, PCA and
variations
Week 10: Markov models, hidden Markov models
Week 11: continuous state-space models, association analysis
Week 12: model selection, next steps

8) Practical Deep Learning for Coders, v3 ( by fast.ai)

Practical Deep Learning for Coders is an amazing free resource for people with
some coding background (but not too much) and includes a variety of notes,
assignments and videos. It is built around the idea to give students practical
experience in the field so expect to code your way through. You can even learn
how to use a GPU server on the cloud to train your models. Pretty cool.

Cost: Free

Time to complete: 12 weeks (8 hours/week)

Syllabus:

Introduction to Random Forests
Random Forest Deep Dive
Performance, Validation, and Model Interpretation
Feature Importance. Tree Interpreter
Extrapolation and RF from Scratch
Data Products and Live Coding
RF From Scratch and Gradient Descent
Gradient Descent and Logistic Regression
Regularization, Learning Rates, and NLP
More NLP and Columnar Data
Embeddings
Complete Rossmann. Ethical Issues

9) Machine Learning A-Z™: Hands-On Python & R In Data Science

Definitely, the most popular AI course on Udemy with half a million students
enrolled. It is created by Kirill Eremenko, Data Scientist & Forex Systems
Expert and Hadelin de Ponteves, Data Scientist. Here you can expect an analysis
of the most important ML algorithms with code templates in Python and R. With 41
hours of learning + 31 articles, it is certainly worth a second look.

Cost: 199 € (but with discounts. At the time of writing the cost was 13.99€)

Time to complete: 41 hours

Syllabus:

Part 1 - Data Preprocessing
Part 2 - Regression: Simple Linear Regression, Multiple Linear
Regression, Polynomial Regression, SVR, Decision Tree Regression, Random
Forest Regression
Part 3 - Classification: Logistic Regression, K-NN, SVM, Kernel SVM, Naive
Bayes, Decision Tree Classification, Random Forest Classification
Part 4 - Clustering: K-Means, Hierarchical Clustering
Part 5 - Association Rule Learning: Apriori, Eclat
Part 6 - Reinforcement Learning: Upper Confidence Bound, Thompson Sampling
Part 7 - Natural Language Processing: Bag-of-words model and algorithms for
NLP
Part 8 - Deep Learning: Artificial Neural Networks, Convolutional Neural
Networks
Part 9 - Dimensionality Reduction: PCA, LDA, Kernel PCA
Part 10 - Model Selection & Boosting: k-fold Cross Validation, Parameter
Tuning, Grid Search, XGBoost

10) CS234 – Reinforcement Learning by Stanford

The most difficult course on the list for sure because arguably Reinforcement
Learning is much more difficult. But if you want to dive into it, there is no
better way to do it. It is in fact actual recorded lectures from Stanford
University. So be prepared to become a Stanford student yourself. The professor
Emma Brunskill makes it very easy to understand all these complex topics and
gives you amazing introduction to the RL systems and algorithms. Of course, you
will find many mathematical equations and proofs, but there is no way around it
when it comes to Reinforcement Learning.

You can find the course website
here and the video lectures in
this Youtube
playlist

Cost: Free

Time to complete: 19 hours

Syllabus:

Introduction
Given a model of the world
Model-Free Policy Evaluation
Model-Free Control
Value Function Approximation
CNNs and Deep Q Learning
Imitation Learning
Policy Gradient I
Policy Gradient II
Policy Gradient III and Review
Fast Reinforcement Learning
Fast Reinforcement Learning II
Fast Reinforcement Learning III
Batch Reinforcement Learning
Monte Carlo Tree Search

Here you have it. The ultimate list of Machine and Deep Learning Courses. Some
of them may be too advanced, some may contain too much math, some may be too
expensive but each one of them is guaranteed to teach all you need to succeed in
the AI field.

And to be honest, it doesn’t really matter which one you’ll choose. All of them
are top-notch. The important thing is to pick one and just start learning.

Originally published in AI Summer

Best Artificial Intelligence books to read

Sergios Karagiannakos ( AI Summer ) — Sat, 15 Feb 2020 00:00:00 +0000

Best Artificial Intelligence books to read

The ultimate list of Artificial Intelligence and Machine Learning non fiction books

In this list, I tried to collect the top 10 most important and popular books around Artificial Intelligence, Machine Learning and Robotics. I spent the last year reading each one of them and I totally recommend all of them (Ok maybe some more than others). This collection is a nice break from all the technical stuff so don’t expect to find technical books filled with math and algorithms.

Instead, there are only nonfiction books which analyze Artificial Intelligence from a philosophical or a business point of view. In my opinion, every MachineLearning Engineer, Programmer with an interest in Machine Learning, Professional who wants to apply AI into his business should read at least two of them. Don’t be surprised if I tell you that every person on the planet should read one of them.

Artificial Intelligence has slowly proven itself as the major force of all future technological advancements and is expected to play a significant role in shaping out every-day lives more than most of us imagine.

Let’s begin:

Homo Deus: A Brief History of Tomorrow

After publishing the best-seller Sapiens, which describes the evolution of mankind through the ages from apes to super-intelligent beings, Yuav Noah Harari(a historian, philosopher, and a professor in the Department of History at theUniversity of Jerusalem) continues his search into the future.

In his book Homo Deus, he argues that humanity will increase his efforts to achieve total happiness, immortality and God-like powers and that may result into various possible futures. Will humans lose control to machines? Will the man be worshiped as God? But the man idea throughout the book is that it will end in uncoupling our intelligence from emotions. Harari dives deep into philosophical issues such as consciousness, human emotions, individualism so if you'd like a little philosophical questioning and thinking, make sure you read this book.

The Singularity is Near: When Humans Transcend Biology

Ray Kurzweil is an inventor and futurists who has written 5 best-sellers so farand is partially responsible for popularizing the term Technological Singularitythrough his book “The Singularity is Near”. The book focuses on a more technical aspect of AI rather than a philosophical point of view and analyzes the sociological impacts of intelligent robots in human life. It also introduces the possibility to merging with machines and live as a cybernetic being, like a cyborg(to make a Battlestar Galactica reference, sorry I couldn’t help myself).

As Bill Gates put it: ‘Ray Kurzweil is the best person I know at predicting the future of artificial intelligence. His intriguing new book envisions a future in which information technologies have advanced so far and fast that they enable humanity to transcend its biological limitations - transforming our lives in ways we can’t yet imagine.’

Superintelligence: Paths, Dangers, Strategies

Superintelligence by prof. Nick Bostrom is the book on Artificial Intelligencesafety. Bostrom imagines how we can create an Artificial Intelligence far superior than we could even think and what risks does it entail. He thinks of examples of how things can go wrong and if superintelligence can replace us as the dominant lifeform on Earth.

One thing that stood out to me was the parallelization of humans with gorillas. If the fate of gorillas depends more on humans than themselves, could this meant hat the fate of humans will depend more on AI than on our species? Another great philosophical book on AI which raises more questions than it answers (and that how it should be)

Life 3.0: Being Human in the Age of Artificial Intelligence

Life 3.0 by cosmologist and MIT professor Max Tegmark. Life 1.0 comes from biological evolution, life 2.0 from cultural evolution and Life 3.0 from technological evolution. It describes once again how things could go wrong. But it does so by using tangible examples with real-life elements and it proposes specific actions to prevent them.

Hear this: a company called Omega took over the world using a super-intelligent AI agent, called Prometheus who was able to develop breakthrough systems, manage global resources optimally and even create other machines. All of that were achieved without anyone realizing AI was behind it. That’s how the book starts. Does this sound plausible to you? It sure does to me.

AI Superpowers: China, Silicon Valley, and the New World Order

Kai-Fu Lee ( who developed the world’s first speaker-independent speech recognition system and held executives positions in Google, Apple and Microsoft)argues that dramatic changes due to AI may happen much sooner than we expect and explores the impact China will have in future. According to him: “If data is the new oil, then China is the new Saudi Arabia”. Maybe the future is not western after all.

He focuses on the problem of global unemployment as a result of AI and he provides a clear description of which jobs will be affected, how soon and how we can provide solutions. The best thing about his book is that it won’t go into vague apocalyptic predictions but it forms educated guesses based on real-world data and his experience in the field.

Analytics of Life: Making Sense of Artificial Intelligence, Machine Learning and Data Analytics

Analytics of Life by Mert Damlapinar is an amazing book for businessmen, managers, marketers and entrepreneurs who want an introduction in ArtificialIntelligence and advanced Data Analytics. It starts by an explanation of what is Machine Learning and Big Data and then it covers real examples of applications on healthcare, marketing, governments and nature, explains what jobs will be replaced and how companies and startups can apply AI to solve their use cases. To summarize the main idea: AI can and will transform almost every industry. A must if you are a professional, who wants to dive into the world of MachineLearning

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

If you want the best high-level overview of Machine Learning, look no further than The Master Algorithm by Pedro Domingos. Domingos manages to organize the entire field of ML in one book and covers everything from the history of the field to the latest breakthroughs.

It creates a conceptual model of the field by categorizing algorithms into 5different school of thoughts. Each school has its own perspective of what the best generalized algorithm is. Then he goes into more details about its school and its algorithms and finally, he suggests that the ultimate master algorithm is the combination of all these and that we gradually drive towards that goal.

How to Create a Mind: The Secret of Human Thought Revealed

How to Create a mind is the second book in our list by Ray Kurzweil. This time he takes the exploration into AI a step further. What’s the best way to create an artificial brain? The answer is to reverse engineer our own biological brain, understand precisely how it works and then apply this accumulated knowledge to create intelligent machines.

Inspired by the latest neuroscience research he describes how our brain is nothing more than “a self-organizing hierarchical system of pattern recognizers” and those insights will enable us to reconstruct it using silicon and programming.

Our Final Invention

In this book James Barrat (a documentary filmmaker for National Geographic, Discovery, PBS and more) characterized Artificial Intelligence as humanity’s final invention. It clearly exposes the risks that may arise from General Artificial Intelligence, it indicates that superintelligence does not necessarily imply benevolence and it summarizes the last years of research on potential AI threads. And it does so thought extensive research and detailed interviews with people in the field.

Our final Invention might have a slightly pessimistic tone and it might leave you with a sense of hopelessness but that’s why it’s a great book. It forces you to think about our future, to try to find new ways to prevent all that from happening. Social awareness at its best.

Human + Machine: Reimagining Work in the Age of AI

Accenture leaders Paul R. Daugherty and H. James Wilson use their experience to reveal how companies use AI to drive innovation and increase profitability and how AI clearly transforms all business processes from customer service and new inventions to productivity and workplace culture. I like to describe it as a playbook for other business leaders to understand the positive impact AI will have on their companies but also the need for education and training to prevent the disruption caused.

How to get hired as a Machine Learning Engineer

Sergios Karagiannakos ( AI Summer ) — Sat, 08 Feb 2020 00:00:00 +0000

How to get hired as a Machine Learning Engineer

7 steps to land a Machine Learning Engineer Job

Becoming a Machine Learning Engineer is by no means an easy task (a positive note to start the article). But it is completely doable if you have the patience and the discipline. The bad news is that you have to study a lot to land a job in a tech company. The good news is that there is a shortage of skilled MachineLearning Engineers, even in big tech, and the salaries are crazy. Seriously crazy. Is it worth it? For me it totally is. But that’s up to you.

So how do I start?

Know your s**t

Before even thinking of applying, you have to know the basics. And by basics, I don’t mean Convolutional Neural Networks. Not even K- Means. I mean basic computer science principles. Algorithms and Data Structures, Programming languages (preferably Python), Debugging, Testing, Version control, Cloud computing. The list goes on and on.

But remember. A Machine Learning Engineer is first and foremost a Software Engineer. He is not a Data Scientist or a Data Analyst. To learn everything, I personally would choose this amazing course by Coursera on Python programming and this by Udacity on Algorithms and Data Structures

Know your Machine Learning

Not as important as the first step, but still useful. You should familiarize yourself with basic algorithms such as Regression, Decisions trees, K Means and get your hands dirty with data preprocessing and modeling. Again, don’t go too fancy. Companies, in general, don’t look for people who can prove mathematically back-propagation. They look for developers to code and build their machine learning pipelines. And in those pipelines machine learning takes about only 5% of the work. Again these courses byCoursera and Udacity are yourfriends.

For a more comprehensive list of resources, you can also check out my blog , where I publish Machine Learning and Artificial Intelligence articles AI Summer.

Get experience

Assuming that you know the basics ( and you should), the next important step is to get experience. I recommend to start working on some personal projects you find interesting (maybe you can predict bitcoin price with neural networks or run quicksort in a massive dataset, I don’t know), participate in one or two Kaggle competitions and maybe land a few small freelancing gigs from clients.

The important thing is to build the whole thing from scratch. From the database and server to the deployed API in production. It is the only way to really comprehend the whole stack and get in touch with all components of the pipeline. Trust me. No course or lesson can match this. It will immediately give you a whole new perspective.

Build Portfolio and Resume

Now that you’re feeling confident and you have worked on some real-worldprojects, it’s time to build your resume. However, your resume is not just apdf. It is a personal website showcasing all your projects and courses. It is aLinkedIn account (extremely important) with all the up to date information aboutyou. It is a GitHub profile containing all the code you wrote over the pastmonths. And maybe is a blog showcasing what you have learned so far. Preferablyis all the above.

That’s how you will grab the attention of the recruiter from your dream company.That’s what the hiring manager will discover when he googles your name.

Prepare for coding interviews

Now to the fun (not so much) part. You have to prepare for coding interviews.Don’t assume that you know what you’re doing. Even if you have a master’s from MIT. Let me say it one more time. You have to prepare.

If you feel confident in your Algorithms and Data Structures skills, here what you’ll need to do. Grab a copy of Cracking the coding interview (the bible for software interviews) and open a Leetcode account. Start practicing on easy problems.Try to come up with a brute force solution, then try to optimize it. When you are stuck, think about what other data structures you can use. Or check the book for similar problems. But don’t give up.

Then do another one. And another one. As you’re solving more and more, you start to identify the patterns and you can proceed to medium or even hard problems.How many problems you should solve? The more the better. Perhaps 150 if you want to work in a FAANG. Otherwise about 50.

Also, my advice is to try and simulate the actual interview experience as soon as possible when you practice. Set up a timer. Explain your thoughts out loud.

Study system design

One integral part of interviews is system design rounds, where you describe how you would build a popular architecture such as Instagram or Netflix. It evaluates all your technical abilities, your background and your general knowledge. Hence, it’s not something you can learn overnight.

Although you can prepare. You can start by dive into the system design of the 10 most popular apps and then try to design a different one. And repeat the process until you feel confident.

You should also emphasize in Machine Learning architecture such as recommendation systems or search autocompletion. In general, this is the round where the company will test your ML background. But keep in mind that it’s avery high-level talk.

Apply

The final step is to start applying. You can, of course, submit applications onthe company’s online platform. But don’t expect any results. To expedite the process, I would focus on 3 things:

Find recruiters on LinkedIn, send them a friend request expressing your interest in a position and let your CV / GitHub account / Website do all the work for you. But do it subtly. Express your interest in the company, ask for an informational call etc.
Ask a friend who works in a tech company for a referral (50% of people hired are via referrals).
Attend job fairs and networking events

That was it? It’s that easy? Lol. Arguably it’s a long process and it takes courage and determination to continue after rejections. But don’t discourage yourself. There is not a single person who hasn’t been rejected. You should also know that unfortunately, it’s also a matter of timing and luck.

But eventually, all your efforts are going to pay off. And that’s not a matter of timing or luck. It’s a certainty.

Graph Neural Networks - An overview

Sergios Karagiannakos ( AI Summer ) — Sat, 01 Feb 2020 00:00:00 +0000

Graph Neural Networks: An overview

Over the past decade, we’ve seen that Neural Networks can perform tremendously well in structured data like images and text. Most of the popular models like convolutional networks, recurrent, autoencoders work very well on data that have a tabular format like a matrix or a vector. But what about unstructured data?What about Graph data? Is there a model that can learn efficiently from them? Probably you guessed it from the title. The answer is Graph Neural Networks.

Graph Neural Networks were introduced back in 2005 (like all the other good ideas) but they started to gain popularity in the last 5 years. The GNNs are able to model the relationship between the nodes in a graph and produce a numeric representation of it. The importance of GNNs is quite significant because there are so many real-world data that can be represented as a graph. Social networks, chemical compounds, maps, transportation systems to name a few. So let’s find out the basic principles behind GNNs and why they work.

Let’s define our basic problem first: We want to map a given graph to a single label, which can be a numeric value, a class or whatever really. In other words:

F ( Graph ) = embedding

And we want to find the function F. For example, imagine that each graph is a chemical compound or a molecule and the label is the likelihood that this molecule can be used to produce a certain drug. If we have a way to extract the label from every graph, we essentially found a way to predict which molecules are more likely to be used in a drug. Cool, right?

How do we do this? We already know a type of Neural Network that can be used on graphs (sort of). If you think about it, recurrent neural networks can operate on a special type of graph. A chained graph (This a graph that is basically a line). Time series are actually chained graphs, where each timestamp is a node followed by the next timestamp.

So, in fact, we can build a network where each graph node is a recurrent unit(LSTM or something else) and the information of the node is an embedding that will be transferred through the chain (like a message). And because the units are all recurrent, the information won’t be lost when the embedding travels through the graph. It is our familiar Recurrent neural networks. Exactly the same as the ones used in language translation and the other natural language processing applications.

We can of course extend this idea to proper graphs and we get this:

Graph neural networks: Variations andapplications

This slide is taken from an awesome talk about GNNs by MicrosoftResearch. Let’s see for a moment what is going on here.

Each orange triangle used to be graph node and it’s now replaced by a recurrent unit. The envelopes represent the embeddings of the nodes that will travel through the graph. Each graph edge is also replaced by a Neural network to capture the information of the edge (its weight).

Now for the learning part. At a single time step, each node pulls the embedding from all its neighbors, calculates their sum and passes them along with its embedding to the recurrent unit, which will produce a new embedding. This new embedding contains the information of the node plus the information of all the neighbors. In the next time step, it will also contain the information of its second-order neighbors. And so on and so on. The process continues until every node knows about all the other nodes in the graph. Each one of the embedding has now information from all the other nodes. The final step is to collect all embeddings and add them, which will give us a single embedding for the whole graph.

[https://tkipf.github.io/graph-convolutional-networks/(https://tkipf.github.io/graph-convolutional-networks/)

That’s it. We did it. We manage to capture the whole graph in a single embedding. This embedding can now be used in some other model to perform some classification, prediction, clustering whatever. Let your imagination wander.

If you want to experiment with Graph Neural Networks, I got you covered:

I would personally go with the third one since it has better documentation but is your choice.

Here you go. Ta-ra lad.