Machine learning is multi-faceted. Learning mechanisms are proving incredibly useful to many disciplines. When machine learning gets applied to a discipline, like Biology or Psychology let’s say, the methodology usually takes on a form in line with that discipline.
In Pedro Domingos’ book, The Master Algorithm: How The Quest for the Ultimate Learning Machine Will Remake Our World, he categorizes the types of machine learning algorithms into five classes, which he calls the tribes of machine learning. Each group supports a set of principles, and, from them, stem different machine learning models.
From many to one, his approach ends by saying that all tribes will center upon a single, master algorithm to know everything. This final machine learning algorithm, The Master Algorithm, would be the artificial general intelligence many predict is to come. In order to get there, Domingos wants to take a little from each of these fives tribes and mix them together.
The 5 Tribes of Machine Learning:
- Symbolists
- Connectionists
- Bayesians
- Evolutionaries
- Analogizers
Symbolists
Symbolists focus on logic. They develop formal systems to create an AI. From the 1950s to the 1980s, symbolic AI was the dominant paradigm. Most computers were based on this kind of thinking, and, for decades, it was the most accessible to code. Symbolists will use technologies that follow a rigid path to make a decision based on logic. Their hard-coded rules would look something like, “Always go left when you encounter a stop sign”.
Systems that produce this kind of decision-making include:
- Decision trees
- Random decision forests
- Production rule systems
- Inductive logic programming
Connectionists (Neuroscience)
Connectionists are the group that create models based on the brain. They love to point out that neural networks, a key part of their internal machine learning architecture, were modeled after the way neurons work in the brain. They use models such as:
- Artificial neural nets
- Reinforcement learning
- Deep learning
When they say connections, they do not refer to connections like analogies, like a kitten is to a cat as a puppy is to a dog. For Connectionists, they use connections to term the signals that pass from one neuron to another. There are strengths of signals and numbers of signals.
A large problem people have with the Connectionist framework is that the way that decisions are made is hidden from view—the proverbial black box. Every step of a decision made in a decision tree is known. In Connectionism, an input may have an output, but the path that input took to get to that output goes unseen. A person cannot ask, “How did the model come to that conclusion?”
Bayesians (Statisticians)
The Bayesian tribe values statistics. It is a different form of logic, one that exists in probabilistic outcomes, and not in hard coded or either/or outcomes. For the Bayesians, machine learning is a form of probabilistic inference. Given an input, there is some chance that the output is Y. Nothing is determined; the possible outcomes have some chance of occurrence.
Bayesians might use Connectionist or Symbolist methods to generate decision models, but the focus in their thinking lies in having only probabilistic outcomes. The infrastructure to get from X to Y isn’t as important as the way the result is interpreted. Nonetheless, the Bayesians will use models such as:
- Hidden Markov chains
- Graphical models
- Causal inference
(Source)
Evolutionaries (Biologists)
Evolutionaries like to think of things, not infrastructure, or determined outcomes, or even in the interpretations of the outcomes. These guys are the process theory types. They are concerned with steps and iterations.
Looking at machine learning from a biologist stance, they are concerned with the evolution of an AI. They’re curious with how it grows, mutates…with how it becomes. Similar to how people evolve at each mutation, the human genome itself having some ability to learn and adapt, producing a human better suited to navigate its environment at every turn, Evolutionaries want to create machine learning models that, too, learn and adapt.
Evolutionaries use genetic algorithms, and evolutionary programming. A common application of Evolutionary AI is on learning tasks. Below, the DeepMind team created a little stick figure object with legs, gave it an environment and simple physics, and made the goal to go forward. After many iterations of trial and error, the figure’s intelligence evolved to be able to leap over the objects and progress forward. Evolutionary algorithms are used a lot in game play. They will be one of the primary models that get cars driving autonomously.
Analogizers (Psychologists)
Analogizers are usually the storytellers. They can create classes of entities. If an input, old or new, is identified as part of one of those classes, then Analogizers believe they can predict the outcome of the input as being like the outcome of that class.
Problems that are tackled through this approach are recommender systems. For example, if a user were to watch a romantic comedy on Netflix, the Analogizer will say, “Given your watching behavior and that you liked this rom-com, you will like these other sets of movies because other people who have watched the same movie also like these other movies.” Analogizers use classes and types to identify groups of people, and predict future outcomes based upon the outcomes of other members of the class.
The machine learning models Analogizers will use are K-Nearest Neighbor algorithms and SVMs, unsupervised learning methods, to place members of a population into their classes. The below chart places its members into two classes 0 and 1. The second picture classifies a member into one of three classes.
The 5 ML Tribes
Each approach has its usefulness. Combinations of these Machine Learning methods might be the answer to an AGI. Self-driving cars might learn to drive safely on the roads through an Evolutionary method, but they will use Connectionist methods to give the cars’ sensors sight. The car might get a huge boost in its user-driver interactions by utilizing the Analogizer’s way of classifying its drivers into types, Aggressive, Defensive, or Passive. Finally, there are rules to the road such as stopping at stop signs and staying between the lines, where methods from the traditional Symbolist tribe prove beneficial.
Additional resources
For more on this topic, browse the BMC Machine Learning & Big Data Blog or check out these articles and how-to’s:
- 4 Types of Artificial Intelligence
- How Machine Learning Benefits Businesses
- What Democratization of AI Means for Enterprise IT
- Bias and Variance in Machine Learning
- How to Create a Machine Learning Pipeline
- Machine Learning with TensorFlow & Keras, a multi-part Guide
- Amazon Machine Learning and Analytics Tools
