Self-supervised learning is the future of AI
Apr 05, 2020 2 mins, 41 secs
Supervised learning is the category of machine learning algorithms that require annotated training data.

Deep learning can be applied to different learning paradigms, LeCun added, including supervised learning, reinforcement learning, as well as unsupervised or self-supervised learning.

But the confusion surrounding deep learning and supervised learning is not without reason.

For the moment, the majority of deep learning algorithms that have found their way into practical applications are based on supervised learning models, which says a lot about the current shortcomings of AI systems.

Reinforcement learning and unsupervised learning, the other categories of learning algorithms, have so far found very limited applications.

“If you take deep learning from Facebook, Instagram, YouTube, etc., those companies crumble,” LeCun says.

Reinforcement learning agents must be trained on hundreds of years’ worth of session to master games, much more than humans can play in a lifetime (source: Yann LeCun).

Reinforcement learning systems are very bad at transfer learning.

Reinforcement learning really shows its limits when it wants to learn to solve real-world problems that can’t be simulated accurately.

“My suggestion is to use unsupervised learning, or I prefer to call it self-supervised learning because the algorithms we use are really akin to supervised learning, which is basically learning to fill in the blanks,” LeCun says.

“Basically, it’s the idea of learning to represent the world before learning a task?

Once we have good representations of the world, learning a task requires few trials and few samples.”.

The second challenge is creating deep learning systems that can reason.

Current deep learning systems are notoriously bad at reasoning and abstraction, which is why they need huge amounts of data to learn simple tasks.

LeCun, however, did admit that “nobody has a completely good answer” to which approach will enable deep learning systems to reason.

The third challenge is to create deep learning systems that can lean and plan complex action sequences, and decompose tasks into subtasks.

Deep learning systems are good at providing end-to-end solutions to problems but very bad at breaking them down into specific interpretable and modifiable steps.

But learning to reason about complex tasks is beyond today’s AI.

The idea behind self-supervised learning is to develop a deep learning system that can learn to fill in the blanks.

The closest we have to self-supervised learning systems are Transformers, an architecture that has proven very successful in natural language processing.

“This is the main technical problem we have to solve if we want to apply self-supervised learning to a wide variety of modalities like video,” LeCun says.

This is what’s going to allow to our AI systems, deep learning system to go to the next level, perhaps learn enough background knowledge about the world by observation, so that some sort of common sense may emerge,” LeCun said in his speech at the AAAI Conference.

In supervised learning, the AI system predicts a category or a numerical value for each input.

In self-supervised learning, the output improves to a whole image or set of images.

To learn the same amount of knowledge about the world, you will require fewer samples,” LeCun says

“If artificial intelligence is a cake, self-supervised learning is the bulk of the cake,” LeCun says


1. Union: Increasing number of employees test positive for COVID-19 at Tucson UPS facility - Arizona Daily Star
May 30, 2020 # health 33 secs
2. Kentucky restaurateur killed, police chief fired amid protests
Jun 02, 2020 # breaking 40 secs
3. John Kelly defends James Mattis after Trump rebuke: 'The president did not fire him' - Fox News
Jun 04, 2020 # politics 47 secs
4. More Twin Cities restaurants halt takeout after staffers test positive for COVID-19 - Minneapolis Star Tribune
Jun 03, 2020 # health 1 min, 39 secs
5. Virgin Galactic stock soars following SpaceX success - CNN
Jun 01, 2020 # politics 21 secs
6. A University Is Under Fire After A Professor Tweeted About Comparing The "Behavioral Profile" Of Black Americans To Asian Americans
Jun 05, 2020 # breaking 1 min, 34 secs
7. CORONAVIRUS LIVE UPDATES: Kansas food plant shuts down after workers contract virus - KMBC Kansas City
Jun 03, 2020 # health 3 mins, 10 secs
8. Hulu, Amazon & Starz Join Netflix In Support For Black Lives Matter As Curfews Take Hold In America Over George Floyd Death Protests – UPDATE - Deadline
May 31, 2020 # entertainment 1 min, 2 secs
9. Some U.S. officials are blaming outside agitators for unrest. That tactic has a long history, experts say | CBC News
Jun 03, 2020 # breaking 1 min, 54 secs
10. Billions Recap: Family Drama - Vulture
Jun 01, 2020 # entertainment 1 min, 25 secs
11. Venture firms rush to find ways to support Black founders and investors - TechCrunch
Jun 02, 2020 # politics 1 min, 33 secs
12. Cole Sprouse arrested at Los Angeles Black Lives Matter protest - Page Six
Jun 01, 2020 # entertainment 1 min, 4 secs
13. Sofia Richie and Scott Disick Have Been "Texting" Since Their Breakup - Cosmopolitan.com
Jun 01, 2020 # entertainment 33 secs
14. Emma Watson's Blackout Tuesday posts draw criticism amid BLM movement - Insider - INSIDER
Jun 03, 2020 # entertainment 1 min, 55 secs
15. Wait, Suicide Squad Almost Had An Even Bigger Justice League Connection? - CinemaBlend
May 31, 2020 # entertainment 34 secs
16. 'Insecure' actor Kendrick Sampson says he was shot 7 times with rubber bullets by police while protesting - Fox News
May 31, 2020 # entertainment 33 secs

SUBSCRIBE

Get monthly updates and free resources.

CONNECT WITH US

© Copyright 2020 365NEWSX - All RIGHTS RESERVED