Deep learning can be applied to different learning paradigms, LeCun added, including supervised learning, reinforcement learning, as well as unsupervised or self-supervised learning.
But the confusion surrounding deep learning and supervised learning is not without reason.
For the moment, the majority of deep learning algorithms that have found their way into practical applications are based on supervised learning models, which says a lot about the current shortcomings of AI systems.
Reinforcement learning and unsupervised learning, the other categories of learning algorithms, have so far found very limited applications.
“If you take deep learning from Facebook, Instagram, YouTube, etc., those companies crumble,” LeCun says.
Reinforcement learning agents must be trained on hundreds of years’ worth of session to master games, much more than humans can play in a lifetime (source: Yann LeCun).
Reinforcement learning systems are very bad at transfer learning.
Reinforcement learning really shows its limits when it wants to learn to solve real-world problems that can’t be simulated accurately.
“My suggestion is to use unsupervised learning, or I prefer to call it self-supervised learning because the algorithms we use are really akin to supervised learning, which is basically learning to fill in the blanks,” LeCun says.
“Basically, it’s the idea of learning to represent the world before learning a task?
Once we have good representations of the world, learning a task requires few trials and few samples.”.
The second challenge is creating deep learning systems that can reason.
Current deep learning systems are notoriously bad at reasoning and abstraction, which is why they need huge amounts of data to learn simple tasks.
LeCun, however, did admit that “nobody has a completely good answer” to which approach will enable deep learning systems to reason.
The third challenge is to create deep learning systems that can lean and plan complex action sequences, and decompose tasks into subtasks.
Deep learning systems are good at providing end-to-end solutions to problems but very bad at breaking them down into specific interpretable and modifiable steps.
But learning to reason about complex tasks is beyond today’s AI.
The idea behind self-supervised learning is to develop a deep learning system that can learn to fill in the blanks.
The closest we have to self-supervised learning systems are Transformers, an architecture that has proven very successful in natural language processing.
“This is the main technical problem we have to solve if we want to apply self-supervised learning to a wide variety of modalities like video,” LeCun says.
This is what’s going to allow to our AI systems, deep learning system to go to the next level, perhaps learn enough background knowledge about the world by observation, so that some sort of common sense may emerge,” LeCun said in his speech at the AAAI Conference.
In supervised learning, the AI system predicts a category or a numerical value for each input.
In self-supervised learning, the output improves to a whole image or set of images.
To learn the same amount of knowledge about the world, you will require fewer samples,” LeCun says
“If artificial intelligence is a cake, self-supervised learning is the bulk of the cake,” LeCun says
10 hours ago
11 hours ago
11 hours ago
12 hours ago
Get monthly updates and free resources.
CONNECT WITH US