It’s Time to Fundamentally Rethink Artificial Intelligence

• Read original article here

Get smarter at writing
It’s Time to Fundamentally Rethink Artificial Intelligence
It’s time to think of a solution that is smaller, simpler and less data-intensive that will set AI up for the future. Introducing Active Memory Learning.
Jan 5 · 7 min read
Background
In 2018, I began my path to learning computer programming with the intention of learning about Artificial Intelligence (AI). Originally, I only wanted to learn the surface level concepts because AI was becoming more common in everyday life. After immersing myself in the subject of AI for the next two and a half years, I began to fall in love with the entire field and began to learn the subject.
As nice as that story may sound, studying AI by myself was quite difficult at times. Most frameworks (starting points/generic functionality, i.e: Tensorflow, Pytorch) often assume you either have prior education in AI or Math. At the time, I was neither of these two things. I felt I had good ideas and if AI was just a bit easier for a beginner to understand, I could excel and make meaningful applications. Fast forward a few years later, I got over the initial growing pains and began making AI projects.
With only 2 years in the field, I have already seen trends come and go. There is one trend that I believe is a pressing issue in AI that needs to be addressed for the next generation of AI. I believe I have found a strong contender for a solution, and it doesn't involve Neural Networks.
Quick Summary: Neural Nets, datasets and Model Size
For those who aren’t familiar with Neural Networks, terms like Model Size and dataset, what do I mean? Let me first briefly explain Neural Nets, then I’ll explain Model Size and datasets. If you already know all this, skip to the next section.
A visualization of a simple Artificial Neural Network
Artificial Neural Networks (Neural Nets) is a software method used to build AI models. The reason the method is called Neural Nets is that the original creators of Neural Networks wanted to mimic how we learn as humans in computers. Neural Nets contain “Artificial Neurons” which allow computers to do basic tasks like recognizing a picture of a dog or do complicated tasks like driving a car. How does it work? Neural Nets go through a process called “Training”. I’ll spare you the technical details, but for a Neural Net to “learn” a task, it’s shown thousands of inputs (like images) so it can generalize to do that specific task. So for example, if you wanted a Neural Net to recognize traffic objects, you’d show it 10,000 images of a stop sign, 10,000 images of traffic lights and so on. After that, the model would practice (Train) recognizing these images for hours, days or even weeks after it got through the dataset. This is what a dataset is: A set of inputs that allow an AI model or Neural Network to learn a task.
Now that you have a brief understanding of datasets, let’s talk about Model Size. An easy way to think about model size in the context of Neural Nets is to recall the artificial neuron portion of Neural Nets. Like our human brains, each neuron has a connection to another neuron. This is similar to Neural Nets and an easy way to think about Model Size when talking about AI. Each artificial connection has a function that allows a Neural Net to make a prediction or a decision. So, Model Size can be thought of as the raw number of artificial neurons in one model.
Now that you have a general understanding of datasets and model size, let me continue explaining the pressing issue: The Compute First Era.
The Compute First Era of AI
During my time learning about AI, I have noticed three prominent trends:
Model sizes, compute requirements and dataset requirements are increasing at an exponential rate.
Source: https://openai.com/blog/ai-and-compute/
This graph is from a research study done by OpenAI. The graph shows the computing power required in Peta-Flop days for older vs. modern AI Models (Note: One Peta-Flop day is defined as a computer’s ability to do perform ~1 Quadrillion Operations per second, for one full day.)
What can be concluded from this graph? For starters, the graph shows a significant spike in required compute power from the year 2011 when Deep Neural Networks began to gain traction. When AI model sizes increase, compute requirements also increase. To the unsuspecting eye, why is this a big deal?
Transistor count in computers are doubling at a rate of ~2 years (Computing power)
AI model sizes are doubling at a rate of 3 to 4 months (Required computing power)
The required computation from AI models is being increased, but we don’t have the computing to support this trend long term.
Surely though, there is a reason for this right? Yes, there is. The reason is in nature of how Neural Networks improve:
Depiction of how Neural Nets improve
When more data is provided, Model Size and compute power are increased. In turn, Neural Networks improve. However, this trend is increasing the cost of developing AI models, which effectively squeezes people out of AI. To be a startup or a hobbyist in today's AI ecosystem, you need to have the scale to back it up. For most people and startups, this is not feasible.
So what can be done? Surely we don’t want to move AI progress back, but we also can’t continue on this trend.
Today I’d like to present something I call Active Memory Learning, which addresses all these issues.
Introducing Active Memory Learning (AML)
Active Memory Learning is a new AI method I built to address the shortcomings of Neural Networks. With AML, you can build highly accurate AI models with as little as 3 data points. AML is a subset of a developing AI field called Few-Shot-Learning (FSL). FSL is the ability for AI to learn a task with very little data while maintaining the accuracy of State-of-the-art methods.
The problem with FSL is simple: Most methods suck compared to Neural Networks or they miss the point of FSL by adding more computing needs. This is where Active Memory Learning comes in: With AML, you can build AI models with barely any data, maintain accuracy and get rid of training.
Wait…Did I say get rid of training? Yes. AML models build on average in 10 seconds on an entry-level CPU. No training required, no expensive hardware required.
Not only do AML models not need training, they can be edited. If your model happens to have low accuracy you can dynamically edit your model by showing your model more inputs. This is done because AML models have an Active Memory, meaning models can remember exactly what was shown to them prior. By being able to remember exactly what was shown to it previously and apply it to a new situation, this creates less need for big data.
Another massive advantage of AML is that one method has been very generalizable to text, images, numeric and categorical data, making it highly adaptable compared to Neural Networks.
Overall, AML poses a lot of advantages compared to Neural Networks for businesses, developers and hobbyists looking to get started in Artificial Intelligence.
The future of AI is Active Learners, not one time learners
Photo Credit: Johannes Plenio — Unsplash
Going forward in AI, there is one goal amongst most companies and research groups: Artificial General Intelligence (AGI). AGI describes(by most people's definitions) when Artificial Intelligence is as intelligent as a human. Humans can pick up tasks by being shown little examples and adapt it to new situations. No one exactly knows what AGI would look like, but people can speculate how we get there.
I personally believe we need to move towards Active Learners and away from one time learners. This is the reason I built Active Memory Learning. Neural Networks are one time learners: they learn a task and don’t improve past that point. Active Memory Learning is a method that allows machines to be Active Learners and improve past a point where they started.
To refer to the previous graph (Neural Network improvement), we will eventually hit a bottleneck in either computing or performance. What happens then? The AI community has largely relied on Neural Nets for a solution to problems without much thought of what’s next, other than scaling up.
I will add, AML is not an AGI solution, at least not right now. However, it raises a good question of where we should be focusing in AI development.
We need to think forward and we need to think where we are headed. Active Memory Learning is a solution to the growing compute needs, big data problem and moving towards active learners.
Use AML for your next project
If you want to tryout AML, go check out the only Developer API online that uses Active Memory Learning: clevrML. clevrML aims to make the process of building, deploying and editing AI for everyone easier, while keeping costs low. No more complicated documentation, tutorials or long learning process. Just create an account, copy and paste a code snippet and use the API. Trust me, it’s worth a try.
Check it out here at www.clevrml.com .
Thank you for reading this post and feel free to share. If you want to keep updated on AML/clevrML, please check out the website now! A blog will be coming soon.
Landon