Machine Learning is exactly what it sounds like. It’s when machines learn things. Seriously. I’m not being trite, I’m being honest. That’s the best definition of what Machine Learning is. 

Of course, there are many different kinds of Machine Learning that exist. There’s supervised and unsupervised learning, reinforcement learning, genetic algorithms, symbolic learning, and more. The list goes on. It’s fascinating to study each and every one of these fields. Amazingly, each field has many different ways of doing the same thing. What is that thing they’re all doing? Learning something, anything, using computers as the method of learning.

Before we dig deeper, let’s clear up one thing really quickly. Machine Learning algorithms are not new. There are no algorithms being used today that are new in any sense of the word. This includes Deep Learning (the hype-machine favorite). All Machine Learning being done today has its roots in ideas that have been studied and developed since the mid 1950s. So, if the algorithms have been around, in their nascent state, since then, why are they getting popular now?

It Always Starts with an Idea

Nothing can earn you more attention faster than great success. Machine Learning has seen a lot of success in recent years. Things like Google Translate, Siri, Alexa, Netflix and Amazon recommendation systems, and, of course, self-driving cars all rely heavily on Machine Learning to work. But why did it start working now?

Let’s consider the invention of the water turbine. In the beginning, the water turbine was a wheel that someone stuck in a river near their village or house. Someone figured out how to use the river to turn that wheel and create energy. Then they funnelled that energy into various tasks, such as grinding wheat or pumping water into a well. As time went on, more technology was developed and more effective tools became available. They had better metals, electricity, and, most importantly, concrete. Combine those things together and you have the Hoover Dam. Now you can power an entire city with the electricity generated by what started off as a simple water wheel.

During the building of the Hoover Dam, was the water wheel a new invention? No. The concept of the Dam was already a known idea. The force of gravity is being used, via water, to turn something in a circle. When it comes to Machine Learning, it’s basically the same thing. However, instead of the force of gravity being used to power cities, a machine is learning a set of parameters by updating them via a loss function, that is back propagated through the layers. Don’t worry about those details now, we’ll get into them on subsequent blog posts. But, know that the idea of back propagation has been around since the 80s. Right now, I want you to focus on one question. What are the new technologies (like concrete) allowing Machine Learning to thrive now?

The Evolution of Machine Learning

There are 4 main reasons Machine Learning has turned into a Hoover Dam:

1. Data
2. General Purpose Graphics Processing Units (GPGPUs)
3. Cloud Computing
4. Improved Algorithms

Data

All Machine Learning algorithms are hungry for one thing. Data. You cannot do the popular Machine Learning that is done today without it. Think of data as the water in the dam. Want to make a model that predicts whether or not your photo has a cat in it? Great! Give me 1 million photos with cats and 1 million photos without cats. If you only give me 10 of each, the chances of success go way down. In today’s world 1 million photos of anything is possible. We have massive amounts of data in the cloud, on Google photos, Facebook, Instagram, you name it. So! Tons of water (data) in the dam, but how do you make electricity?

GPGPUs

We need to funnel that data in the right direction and through all our energy harvesting instruments. Here is where the GPGPUs come into play. They are like the steel pipes that funnel the water and withstand the pressure without bursting. Nearly all Machine Learning algorithms require heavy amounts of linear algebra. GPGPUs are exceptionally high performant in these kinds of computations. Why? Linear algebra relies on matrices, which happens to be the same data type that monitors use. Machine Learning developers realized they could increase their algorithm’s speed if they used a GPU to do the processing. GPUs physical circuitry are ideal for this job. They have hundreds to thousands of cores which run in parallel and are designed to handle matrices and linear algebra routines.

After this discovery, the GPU manufacturers embraced this new purpose and started designing GPUs for Machine Learning (what we today call GPGPUs). The increase in Machine Learning speed was significant. Seriously significant. Combined with the large amount of data now available, Machine Learning researchers have been able to design and run experiments that they could never even dream of 10 years ago.

Cloud Computing

Simultaneously there has been another progression on another front for increased and easily available computing power. Enter cloud computing, accessible to anyone anywhere (who can pay for it). Today you can “spin up” a virtual machine with as many GPGPU’s as you’d like within minutes. You can run your machine learning models for as long as you’d like and you only pay for the time you provisioned the machine. This is the power of internet + computing + data + algorithms. It’s become very easy for everyone to access Machine Learning because algorithms are open sourced and the compute power is available to anyone from anywhere. You no longer need to buy or maintain hardware in order to perform machine learning.

Improved Algorithms

Improved algorithms are the most fascinating component of Machine Learning’s evolution. All Machine Learning, at its heart, is an algorithm. When given a set of data, we can write an algorithm that will learn something from that data! A machine can learn how to recognize dogs, transcribe your voice into text, or find a face in a photo, all by applying an algorithm to data. It’s amazing! We have algorithms that, given data, will find something we could not learn on our own.

Umm. Ok. That last part is wrong. We absolutely could learn it on our own. The same code that the computer runs, we could do by hand, right? Wait a minute. Could we really do it by hand? Common Machine Learning models today may learn over 1 million parameters. That’s tons of iterative number crunching which, honestly, would take us lifetimes to do by hand. So no, we can’t do it on our own. It would take too long! Let’s get a computer to do it for us.

My point here is that computers are not thinking. They are computing. They are processing information that we gave them and in a way that we designed. When you hear about Machine Learning, always remember it’s a really cool algorithm that learns something because a really clever human designed the algorithm to do so.

Why Should You Care?

As you can see, Machine Learning is popular today because of the advent of new hardware, greater accessibility to data, and better algorithms. These three factors together have combined to create a Machine Learning boom.

But, what is Machine Learning actually good at? What does it struggle with? Finally, and most importantly, how does it affect you, both in your professional life and in your day-to-day life as a consumer of technology?

I will answer all these questions in my next post. I will point out a few common products that are already imbedding Machine Learning into your daily life. I will also discuss how you may soon be interacting with Machine Learning in your professional life. Thanks for reading and please let me know your thoughts and questions in the comments section below!

 There's still plenty to cover as Machine Learning continues to advance. Keep an eye out for future posts and dive into the details with Gilad's expert navigation of Deep Learning with Python. 

{{cta('b45204e1-6662-4cc5-8c14-4ac60f14c6fc')}}