Deep Learning State of the Art (2020) | MIT Deep Learning Series

– Welcome to 2020 and welcome to the Deep Learning lecture series. Let’s start it off today to take a quick whirlwind tour of
all the exciting things that happened in 17, 18 and 19 especially, and the amazing things we’re going to see in this year in 2020. Also as part of the series is gonna be a few talks
from some of the top people in learning and
artificial intelligence. After today, of course,
start at the broad, the celebrations from the touring award to the limitations and the debates and the exciting growth first. And first of course, a
step back to the quote I’ve used before, I love
it, I’ll keep reusing it. AI began not with Alan Turing or McCarthy, but would the ancient
wish to forge the gods, of course from Pamela
McCorduck Machines Who Think, that visualization there
is just 3% of the neurons in our brain of the
thalamocortical system, that magical thing between our ears that allows us all to see and hear and think and reason and hope and dream and fear, our eventual mortality. All of that is the thing
we wish to understand. That’s the dream of
artificial intelligence and recreate versions of it,
echoes of it, in engineering of our intelligence systems. That’s the dream. We should never forget
in the details I’ll talk the exciting stuff I’ll talk about today. That’s sort of the reason why this is exciting, this
mystery, that’s our mind. The modern human brain, the modern human as we know them today
no and love them today. It’s just about 300,000 years ago and the industrial revolution
is about 300 years ago ’cause that’s 0.1% of
the developments since the early modern human
being is when we’ve seen a lot of the machinery. The machine was born, not in stories, but in actuality is the
machine was engineered since the industrial
revolution and the steam engine and the mechanized factory system and the machining tools. That’s just 0.1% in the history and that’s the 300 years. Now we zoom in to the 60, 70 years since the founder, the father, arguably of artificial intelligence, Alan Turing and the dreams that there’s
always been the dance and artificial intelligence
between the dreams, the mathematical foundations
and when the dreams meet the engineering, the
practice, the reality. So Alan Turing has spoken many times that by the year 2000,
that he would be sure that the Turing test, natural
language would be passed. It seems probably he said
that wants to machine thinking method has started, it would not take long to
outstrip our feeble powers. It would be able to
converse with each other to sharpen their wits some stage. Therefore we should have
to expect the machines to take control. A little shout out to self play there. So that’s the dream. Both the father of the
mathematical foundation of artificial intelligence and the father of dreams in artificial intelligence. And that dream again in the early days was taking reality. The practice, met with the
perception often thought of as a single layer neural network, but actually what’s not as
much known as Frank Rosenblatt who was also the developer. The multilayer perception and that history is zooming through has
amazed our civilization. To me, one of the most inspiring things, and this is in the world of games, first with the great Gary Kasparov losing to IBM deep blue in 1997
then Le Sedol losing to AlphaGo in 2016 seminal moments and captivating the world
through the engineering of actual real world systems. Robots on four wheels, as
we’ll talk about today, from Waymo to Tesla to all
of the autonomous vehicle companies working in this space. Robots on two legs, a
captivating the world of what actuation, what
kind of manipulation can be achieved. The history of Deep Learning from 1943 the initial models from neuroscience, thinking about neural
networks, how to model neural networks mathematically
to the creation, as I said, of the single layer and the multi-layer
perceptron by Frank Rosenblatt in 57 and 62 to the
ideas of backpropagation and occur in neural nets in the 70s and 80s to convolutional neural networks and LCL is by directional RNs in the 80s and 90s to the birth of
the deep learning term and the new wave, the revolution
in 2006 to the image net and Alex net, the seminal
moment that captivated the possibility, the
imagination of the AI community, of what neural networks
can do in the image and natural language space
closely following years after to the development
of the popularization of GANs Generative Adversarial Networks. So the AlphaGo and AlphaZero in 2016/7 and as we’ll talk about language models of transformers in 17,
18 and 19 those has been the last few years have been dominated by the ideas of deep learning in the space of natural language processing. Okay, celebrations. This year, the Turing Award was given for deep learning. This is like deep learning has grown up. We can finally start giving awards. Yann LeCun, Geoffrey Hinton, Yoshua Bengio received the Turing Award for the conceptual
engineering breakthroughs that have made deep neural networks a critical component of computing. I would also like to add that
perhaps the popularization in the face of skepticism
and for those a little bit older have known the skepticism. Then you’ll know of
service throughout the 90s in the face of that
skepticism, continuing pushing, believing, and working in this field and popularizing it through in the face of that skepticism, I
think is part of the reason these three folks have received the award. But of course, the
community that contributed to deep learning is bigger, much bigger than those three. Many of whom might be here today at MIT, broadly in academia, in industry. Looking at the early key figures, Walter Pitts and Warren McCulloch, as I mentioned for the competition models of the neural nets. These ideas of that thinking that the kind of neural networks,
biological neural networks can have on our brain can
be modeled mathematically and then the engineering of those models into actual physical and
conceptual mathematical systems by Frank Rosenblatt
57 against single layer multilayer in 1962 you
could say Frank Rosenblatt is the father of deep learning. The first person to really in 62 mention the idea of multiple hidden
layers in neural networks. As far as I know somebody was correct me, but in 1965 shout out to the Soviet union and Ukraine, the person who is considered to be the father of deep
learning, Alexey Ivankhenko and V.G Lapa co author of that work is the first learning
algorithms that multilayer perceptrons multiple hidden layers. The work on backpropagation,
not automatic differentiation. In 1970 1979 convolution neural networks were first introduced and
John Hartfield looking at recurrent neural networks now called Hotville
networks, a special kind of recurrent neural networks. Okay that’s the early
birth of deep-learning. I wanna mention this
because it’s been a kind of contention space now
that we can celebrate the incredible accomplices, deep learning, much like in reinforcement
learning and academia. Credit assignment is a big problem and the embodiment of that
almost a point of meme is the great Juergen Schmidhuber. I encouraged for people who are interested in an amazing contribution
of the different people in the deep learning
field to read his work on deep learning and neural networks. It’s an overview of all the various people who have contributed besides Yann LeCun, Geoffrey Hinton and Yoshua Bengio. What’s a big beautiful community, a full of great ideas
and full of great people. My hope for this community,
given the tension is some of you might’ve
seen around this kind of credit assignment problem
is that we have more, not on this slide, but love
that can never be enough love in the world, but general respect, open mindedness and collaboration and credit sharing in the
community, less derision, jealousy and stubbornness and silos, academic silos within
institutions, within disciplines. Also 2019 was the first
time it became cool to highlight the limits of deep learning. This is the interesting
moment in time several books, several papers have come
out in the past couple of years highlighting that deep learning is not able to do the
kind of the broad spectrum of tasks that we can think of. The artificial intelligence
system being able to do like re common sense
reasoning like building knowledge bases and so on. Rodney Brooks said by 2020,
the popular press starts having stories that the era of
deep learning is over and certainly there has
been echoes of that through the press, through the Twitter sphere and all that kind of world. And I’d like to say that
a little skepticism, a little criticism is really
good always for the community, but not too much. It’s like a little spice
in the soup of progress. Aside from that kind of a skepticism, the growth of CVPR I clear and
Europe’s all these conference submission papers has
grown year over year. There’s been a lot of exciting research, some will, which I’d like to cover today. My hope in this space of deep
learning growth celebrations. The limitations for 2020
is that there’s less, both less height unless NTI hype, less tweets on how there’s
too much hype in AI and more solid research, less criticism and more doing, but
again, a little criticism. There’s a little spice is
always good for the recipe. Hybrid research, less
contentious counter productive debates and more open
minded and interdisciplinary collaboration across
neuroscience, cognitive science, computer science, robotics,
mathematics, physics. Across all of these
disciplines working together and the research topics that
I would love to see more contributions to as we will briefly talk about in some domains is
reasoning, common sense reasoning, integrating that
into the learning architecture, active learning and lifelong learning, multimodal multitask, learning
open domain conversation, so expanding the success
of natural language to dialogue, to open domain
dialogue and conversation and then applications. The two most exciting, one
of which we’ll talk about is medical and autonomous vehicles. Then algorithmic ethics
in all of its forms, fairness, privacy bias. There’s been a lot of
exciting research there. I hope that continues. Taking responsibility
for the flaws in our data and the flaws in our human ethics. And then robotics. In terms of deep learning
application robotics. I’d love to see a lot of development, continued development, deeper enforcement, learning, application robotics
and robot manipulation. By the way, there might
be a little bit of time for questions at the end. If you have a really pressing question, you can ask it along the way too. Questions so far? Thank God. Okay, so first the practical, the deep learning and deep RL frameworks. This is really been a
year where the frameworks have really matured and converse shores to popular deep learning frameworks that people have used as a
TensorFlow and PI torture. Tessa flow 2.0 and PI torch
1.3 is the most recent version and they’ve converged
towards each other taking the best features or moving
the weaknesses from each other. So that competition has
been really fruitful in some sense for the
development of the community. So on the TensorFlow
side, eager execution. So imperative programming, the kind of how you
would program in Python has become the default
has been first integrated, made easy to use and become the default. And I’m the side towards script allowed for now graph representation. So do what you’re used to be able to do and what used to be the
default mode of operation TensorFlow allow you to
have this intermediate representation that’s in graph form, the on intensive flow
side, just the deep carious integration and promotion
as the primary citizen, the default citizen of the API of the way you would Draco TensorFlow, allowing complete
beginners just to anybody outside of machine
learning to use TensorFlow with just a few lines of code to train and do inference with the model so that that’s really exciting. They cleaned up the API,
the documentation and so on. And of course maturing the, the JavaScript and the browser implementation. Intensive flow tends to
flow light being able to run TensorFlow on, on
phones, mobile and serving. Apparently this is
something industry cares a lot about of course is
being able to efficiently use models in the cloud and catching up with TPU support and experimental versions of PI torch mobile. So being able to ride a
smartphone on their side, this tense, exciting competition. Oh, and I almost forgot to mention we have to say goodbye to
our favorite Python two. This is the year that support finally and the January 1st, 2020
support for Python two and TensorFlows and PI tours support for Python two is ended. So goodbye print goodbye CRO world. Okay, on the reinforcement learning front, we’re kind of in the same space as a Java script libraries are in. There’s no clear winners coming out if you’re a beginner in the space. The one I recommend is
a, as a fork of OpenAir baselines is stable
baselines, but there’s a lot of exciting ones. Some of them are really
close to built on TensorFlow. Some are built on PI torch. Of course from Google, from
Facebook, from a deep mind. Dopamine TFA agents tends
to force most of these I’ve used, if you have specific questions I can answer them. So stable baselines is
the open a base on his for cause I said these
implements a lot of the basic deep RL algorithms PPO
way to see everything good documentation and just
allows very simple minimal few lines of code
implementation of the basic the matching of the basic algorithms of the open air gym environments. That’s the one I recommend. Okay, for the framework
world, my hope for 2020 is framework agnostic research. So one of the things that
I mentioned is PI torch has really become almost
overtaking TensorFlow in popularity in the research world. What I would love to see
is being able to develop an architecture in TensorFlow or develop an and PI torch, which you currently can and then a trend once you train the model to be able to easily
transfer it to the other. From Picador, she tends to
flow from TensorFlow to PI torch currently takes
three, four, or five hours. If you know what you’re doing
in both languages to do that. It’d be nice if, if
there was a very easy way to do that transfer, then the maturing of the deep RL frameworks,
I’d love it to see open AI, step up the mind to
step up and really take some of these frameworks to maturity that we can all agree on. A much like open AI gym
for the environment world has done and continued work that Charisse has started and many other
rappers around TensorFlow started a greater and
greater abstractions allowing machine learning to be
used by people outside of the machine learning field. I think the powerful thing about supervise, sort of
basic vanilla supervised learning is that people
in biology and chemistry in neuroscience in in
physics, in astronomy can deal with a huge amount of data that they’re working with. And without needing to
learn any of the details of even Python. So that, that I would love to see greater and greater abstractions
which empower scientists outside the field. Okay natural language processing. 2017, 2018 it wasn’t, the
transformer was developed and it’s power was demonstrated
most, especially by Bert. Achieving a lot of
state of the art results and a lot of language benchmarks from sentence classification to tagging, question answering and so on. There’s hundreds of data sets
and benchmarks that emerge. Most of which Bert has dominated in 2018, 2019 was sort of the
year that the transformer really exploded in terms of
all the different variations. Again, starting from Bert
XL net it’s very cool to use Bert in the name
of your new derivative of a transformer, a Roberta distill bird from pugging face Salesforce open AI’s GPT to of course Albert and
Megatron from Nvidia. Huge transformer. A few tools have emerged. So one on hugging face is a company and also a repository that has implemented in both PI torsion TensorFlow or a lot of these transformer based national language models. So that’s really exciting. So most people here
can just use it easily. So those are already pre-trained models. And the other exciting stuff is the patch. Sebastian ruder, great researcher in the field of natural
language processing has put together an LP progress, which is all the different
benchmarks for all the different natural language tasks tracking
who sort of lead a boards of who’s winning where. Okay I’ll mention a few
models that stand out. The work from this year,
Megatron LM from Nvidia is basically taking, I
believe the GPT to transform a model and just putting
it on steroids, right? 8.3 versus 1.5 billion parameters. And a lot of interesting stuff there as you would expect from Nvidia. Of course there’s always
brilliant research but also interesting
aspects about how to train in a parallel way model and data parallelism in the training. The first breakthrough results
in terms of performance, the model that replaced
Bert as King of transformers is XL net from CMU and Google research. They combined the BI
directionality from Bert and the the recurrence
aspects of tress home XL, their relative positional embeddings and the recurrence mechanism
of transformer Excel to taking the bide directionality
and the recurrence. Combining into chief state
of the art performance on 20 task. Albert is a recent addition
from Google research and it reduces significantly the amount of parameters versus
Birch by doing a parameter sharing across the layers
and it has achieved state of the art results on
12 NLP tasks including the difficult Stanford
question answering benchmark of squad two and they provide the provided open source TensorFlow
implementation including a number of raid to use
pre-trained language models. Okay, another way for
people who are completely new to this field, a bunch of apps, right? With transformers, one
of them from hugging face popped up that allows you
to explore the capabilities of these language models
and I think they’re quite fascinating from a
philosophical point of view. And this, this has
actually been at the core of a lot of the tension of
how much do these transformers actually understand basically memorizing the statistics of the
language in a self supervised way by reading a lot of texts. Is that really understanding? A lot of people say no
until it impressed us and then everybody will
say it’s obvious but right. What transformer is a really
powerful way to generate texts to reveal to you
how much these models really learned before
this yesterday actually just came up with a bunch of prompts on the left is a prompt. You give it the meaning
of life here for example, is not what I think it is. It’s what I do to make it. And you can do a lot of prompts with this nature’s very profound. And some of them will be just absurd. You’ll make sense of it statistically, but it would be absurd
in reveal that the model really doesn’t understand the fundamentals of the prompt as being provided. But at the same time
it’s incredible what kind of text is able to generate. Okay the limits of deep learning. I was just having fun
with this at this point. It’s still the, are still in the process of being figured out very true. Had to take this most important person in the history of deep
learning is probably Andrew and I have to agree. So this model knows what it’s doing. And I tried to get it
to say something nice about me and that’s a lot of attempts, so this is kind of funny is finally did it did one I said Let’s frame his best
qualities that he’s smart said finally, but it’s never
nothing but ever happens, but I think he gets more
attention than ever. Every Twitter comment
ever and it’s very true. Okay a nice way to sort of reveal through this that the
models are not able to do any kind of understanding of language is just to do prompts
that show understanding of concepts, of being able to reason with those concepts,
common sense reasoning. A trivial one is doing two
plus two is a three five is a six seven. The result is simply equation
four and two plus three is like you got it right and
then it changed his mind. Okay, two minus two is seven so on. You can reveal any kind of reasoning, you can do a blocks, you
can ask it about gravity, all those kinds of things. It shows that it doesn’t
understand the fundamentals of the concepts that are
being reasoned about. And I’ll mention of work
that takes it beyond towards that reasoning world
in the next few slides. But I should also mention
with this GPT to model, if you remember about a year ago, there was a lot thinking
about this 1.5 billion parameter model from open AI. It is so the thought was
it might be so powerful that it would be dangerous. And so the idea from opening eyes when you have an AI
system that you are about to release that might
turn out to be dangerous in this case used probably by Russians, fake news for misinformation
that’s the kind of thinking is how do we release it. And I think while it
turned out that the GPT two model is not quite so dangerous, the humans are in fact more dangerous than AI currently. That thought experiment
is very interesting. They released a report,
unreleased strategies and the social impacts of language models that almost didn’t get as much intention as I think it should. And it was a little bit disappointing to me how little people are worried about this kind of situation. There was, it was more
of an eye roll about, Oh, these language models aren’t as smart as we thought they might be. But the reality is once they
are, it’s very interesting thought experiment of
how should the process go of companies and experts communicating with each other during that release. The support think things
through some of those details. My takeaway from just reading the report from this whole year of that
event is that conversation on this topic are difficult
because we as the public seem to penalize anybody trying
to have that conversation. And the model of sharing
privately confidentially between ML machine learning organizations and experts is not there. There’s no incentive or a model or a history or a culture of sharing. Okay, best paper from ACL,
the, the main conference for languages was on the difficult task, so we’ve talked about language models. Now there’s the task
taking it a step further of dialogue, multi-domain
task oriented dialogue. That’s sort of like the next challenge for dialogue systems. And they’ve had a few ideas
on how to perform dialogues, stay tracking across domains achieving state of the art
performance on multi walls. It was just a five domain challenging, very difficult fi domain, a human to human dialogue dataset. There’s a few ideas there. I should probably hurry up
and start skipping stuff. On the common sense reasoning
which is really interesting this one of the open questions for the deep learning
community at community in general is how can
we have hybrid systems of whether it’s symbolic AI, deep learning or generally common sense
reasoning with learning systems. And there’s been a few
papers in this space. One of my favorite, some
Salesforce on a building, a dataset where we can start
to a do question answering and figuring out the concepts
that are being explored in the question and
answering here the question while eating a hamburger with friends, what are people trying
to do multiple choice, have fun, tasty and digestion. The idea that needs to be generated there and that’s where the
language model would come in. Is that usually a hamburger with friends? Indicates a good time. So you basically take the question, generate the common sense concept, and from that be able to
determine the multiple choice, what’s being, what’s happening, what’s the state of affairs
in this particular question. Okay, Alexa prize again
hasn’t been received nearly enough attention to that. I think it should have
perhaps because there hasn’t been major
breakthroughs, but it’s a open domain conversations that all of us, anybody who owns an Alexa
can, can participate in as a a provider of data. But there’s been a lot of amazing work from universities across
the world on the Alexa prize in the last couple of years and there’s been a lot of
interesting lessons summarized in papers and blog posts. A few lessons from Alquist
team that I particularly like. And this is kind of echoes
the work in IBM Watson. Well, the jeopardy challenge
is that one of the big ones is that machine learning
is not an essential tool for effective conversation yet. So machine learning is
useful for general chit chat. When you fail at deep
meaningful conversation or actually understanding what the topic we’re talking about. So throwing in chitchat
and classification, sort of classifying intent,
finding the entities, detecting the sentiment of the sentences. That’s sort of a an assistive tool, but the fundamentals of the conversation are the following. So first you have to break it apart. Sort of conversation is a, you can think of it as a long dance and the way you have fun dancing is you break it up into a set of moves and turns and so on and focus on that sort of live in the moment kind of thing. So focus on small parts
of the conversation taken at a time and then also have a graph sort of conversation is
also all about tangents. So I have a graph of topics and be ready to jump
context from one context to the other. And back, if you look at
somebody who’s natural language conversations that they publish, it’s just all over the
place in terms of topics. You jump back and forth. And that’s the beauty, the humor, the wit, the fun of conversations. You jump, jump around from
topic to topic and opinions. One of the things that
natural language systems don’t seem to have much is opinions. If I learned anything,
one of the simplest way to convey intelligence,
it’s to be very opinionated about something and confident. And that’s, that’s a
really interesting concept about constantly and in general there’s just a lot of lessons. Oh, and finally, of course, maximize entertainment, not information. This is true for autonomous vehicles. This is true for natural
language conversation is fun should be part of the objective function. Okay, lots of lessons to learn there. This is really the Lubner prize, the Turing test of our generation. That’s, I’m excited to see
if there’s anybody’s able to solve the Alexa prize. Again Alexa Prize is your
task would talking to a bot. And the measure of quality is the same as in a lot of enterprises
just measuring how good was that conversation. But also the task is to try to continue the conversation for 20 minutes. If you try to talk to a bot today like and you have a choice to talk to a bot or go do something else, watch Netflix, the you last probably
less than 10 seconds, you’d be bored. The point is to continue trapping you in the conversation
because you’re enjoying it so much. And the 20 minutes is, that’s
a really nice benchmark. Four passing the spirit
of what the Tory tested for examples here from the Alexa prize and the was bought. So the difference in the
two kinds of conversations. So Alquist says, have you been in Brazil? The user says, what is
the population of Brazil? Alco says it is about 20
million users says, well, okay. This is what happens a
lot with like I mentioned, multi-domain conversation is once you jump to a new domain, you stay there. Once you’ve switched
contexts, you stay there. The reality is you want
to jump back and continue jumping around like in the second most more successful conversation. Have you been in Brazil? What is the population of Brazil? It is around 20 million. Anyway, I was saying,
have you been in Brazil? So jumping back in context,
that’s how conversation goes. Tangent to tangent and back. Quickly, there’s been a
lot of sequence to sequence kind of work using natural language. To summarize a lot of applications. One for me I cleared that
I wanted to highlight for from Technion that I
find particularly interesting is the abstract syntax, tree
based summarization of code. So I’m modeling computer code, this case, sadly Java and C sharp in
in trees, in syntax trees and then using operating on those trees to then do the summarization in text here. An example of a basic
power have to function on the bottom right in Java. The code two sec summarization
says get power of two. That’s an exciting
possibility of automated documentation of source code. I thought it was particularly
interesting in the future. There’s was bright, okay. Hopes for 2020 for natural
language processing is reasoning. Common sense reasoning becomes greater and greater part of the
transformer type language model work that we’ve seen
in the deep learning world. Extending the context from thousands, from hundreds of thousands of words to tens of thousands of words. Being able to read entire
stories and maintain the context which transformers
again with XL net fast Homer Excel is starting to
be able to do but we’re still far away from that longterm
lifelong maintenance of context, dialogue, open
domain dialogue forever. Since Alan Turing to today is the dream of artificial intelligence
being able to pass the Turing test and the dream of a sort of natural language model transformers are self supervised learning and the dream of Yann
LeCun, is to for these kinds of what previously were
called unsupervised, but these calling now self supervised learning systems to be
able to sort of watch YouTube videos and from
that started to form representation based on
which you can understand the world sort of the, the hope for 2020 and beyond is to be able to transfer some of the success of transformers to the world of visual information. The world of video for
example, deep RL and self play. This has been an exciting year, continues to be an exciting
time for reinforcement learning in games and robotics. So first Dota two an open
AI, an exceptionally popular competitive game, e-sports
game that people compete when millions of dollars with. So this, this is a lot of
world-class professional players. And so in 2018 open AI
five, this is a team play tried their best at the international and lost and said that
we’re looking forward to pushing five to the next level, which they did in April, 2018 they beat the 2018 world champions
in five on five play. So the key there was compute
eight times more training compute because the actual
compute was already maxed out. The way they achieve
the eight X is in time simply training for longer. So the current version
of OpenAI five is Jacob, we’ll talk about next Friday
has consumed 800 pedo flop a second days and experienced
about 45,000 years of Dota self play over 10 real time months again behind a lot of
the game systems talk about they use self place so
they play against each other. This is one of the most exciting concepts in deep learning systems that
learn by playing each other and incrementally improving in time. So starting from being
terrible and getting better and better and better and
better and you’re always being challenged by a
slightly better opponent because of the national
process of self play, that’s a fascinating process. The 2019 version, the last
version of open AI five it has a 99.9 win rate
versus the 2018 version. Okay, then deep mind also
in parallel has been working and using self play to solve
some of these multi-agent games, which is a really
difficult space when people have to collaborate as
part of the competition. That’s exceptionally difficult from the reinforcement
learning perspective. So this is from raw
pixels, solve the arena, capture the flag game, quake three arena. One of the things I love,
just as a sort of side note about both opening eyes and deep mind and general research and
reinforcement learning. There will always be one or
two paragraphs of philosophy in this case from deep mind. Billions of people inhabit the planet, each with their own
individual goals and actions, but still capable of coming
together through teams, organizations and societies,
and impressive displays of collective intelligence. This is a setting we
call multiagent learning. Many individual agents
must act independently, yet learn to interact and cooperate. Well, the agent, this is
immensely difficult problem because with co adapting agents, the world is constantly changing. The fact that we seven
billion people on earth, people in this room,
in families in villages can collaborate while
being for the most part self interested agents is fascinating. One of my hopes actually
for 2020 is to explore social behaviors that
emerge in reinforcement learning agents and how those
are echoed in real human to humans social systems. Okay, here’s some visualizations. The agents automatically figure out, as you see in other games, they figure out the concepts. So knowing very little,
knowing nothing about the rules of the game, about the
concepts of the game, about the strategy and the behaviors they are able to figure it out. There’s the TST visualizations
of the different States, important States and concepts in the game that they figures out and
so on. Skipping ahead, automatic discovery of
different behaviors. This happens in all the different games and talk about from Dota to StarCraft, to quake the different strategies that it doesn’t know about. It figures out automatically
and the really exciting work in terms of the multi-agent
RL on the deep mind side was the beating world-class players and achieving grand master level and game. I do know about, which is StarCraft. In December, 2018 Alfa
started beating mana, one of the world’s strongest
professional soccer players, but that was in a very
constrained environment and it was a single
race, I think a protest and in October, 2019 off
of star beach Grandmaster level by doing what we humans do. So using a camera, observing
the game and playing as part of against other humans. So this is not an artificial sized system. This is doing exact same process. Humans would undertake
an achieved grand master, which is the highest level. Okay, great. I encourage you to observe
a lot of the interesting on their blog posts and
videos of the different strategies that the there are RL agents able to figure out. Here’s a quote from the
one of the professional StarCraft players, and we
see this with alpha zero two. And chess is alpha stars
and intriguing unorthodox player one with the reflexes
and speed of the best pros, but strategies and style
they’re entirely zone. The way alpha star was
trained with agents competing against each other in a league
has resulted in gameplay. That’s unimaginably unusual. It really makes you
question how much the stock has diverse possibilities. Pro players have really explored and that’s the really exciting
thing about reinforcement learning agent in chess and go and games and hopefully simulated
systems in the future that teach us, teach experts
to think they understand the dynamics of a particular
game, a particular simulation of new strategies,
of new behaviors to study. That’s one of the exciting applications from almost a psychology perspective. I’d love to see reinforcement
learning push towards and on the imperfect information games side poker in 2018, CMU no Brown. I was able to beat a head to head to head, no limit Texas, hold them. And now team six player, no limit, Texas, hold them against professional players. Many of the same results. Many of the same approaches
was self play iterative Monte Carlo and there’s a bunch of ideas in terms of the abstractions. So there’s so many possibilities
under the imperfect information that you
have to form these bins of abstractions in both
the action space in order to reduce the action space and the information abstraction space. So the probabilities of
all the different hands that can possibly have and
all the different hands that the betting strategies
could possibly represent and sort of you have to
do this kind of course planning so that they
use self play to generate a course blueprint
strategy that in real time they then use Monte caller
search to adjust as they play. Again, unlike the deep mind open, I approach very few, very
minimal compute required and they’re able to
achieve to beat to beat world-class players. Again, I like this is getting quotes from the professional players
after they get beaten, so Chris Ferguson, famous worlds, he’s a poker player, a said pluribus. That’s the name of the agent
is a very hard opponent to play against. It’s really hard to pin him
down on any kind of hand. He’s also very good at making
then value bets on the river. He’s very good at extracting
value out of his good hands, sort of making bets without
scaring off the opponent. Darren Aliya said its major strength is its ability to use mixed strategies. That’s the same thing
that humans try to do. It’s a matter of execution for humans to do this in a perfectly random way and to do so consistently. Most people just can’t. Then in the robotic space has been a lot of applications of reinforcement learning. One of the most exciting
is the manipulation, sufficient manipulation
to be able to solve the Rubik’s cube. Again, this is learned through
reinforcement learning. Again because self plays in
this context is not possible. They use automated domain
rent randomization, ADR, so they generate
progressively more difficult environments for the hand. There’s a giraffe head there, you see there’s a lot of perturbations to the system so they mess with it a lot and then a lot of noise
injected into the system to be able to teach the hand to manipulate the cube
in order to then solve the actual solution of figuring out how to go from this particular face to the solve cube is an obvious problem. The, the, this paper
in this work is focused on the, the much more difficult learning to manipulate the cube. It’s really exciting. Again, a little philosophy
as you would expect from open AI is they have this idea of emergent metal learning. This idea that the capacity
of the neural network that’s learning this
manipulation is constraint. While the ADR, the automatic
domain randomization is progressively harder
and harder environment. So the capacity of the environment to be difficult is unconstrained. And because of that there’s a, an emergent self optimization of the neural network to learn general concepts as opposed to memorize particular manipulations. The hope for me in a deeper
enforcement learning space, I aim for 2020 is the continued
application of robotics, even sort of a legged robotics but also robotic
manipulation, human behavior. So use of multi-agent
self plays I’ve mentioned to explore naturally
emerging social behaviors, constructing simulations
of social behavior and seeing what kind of
multi human behavior emerges in self play context. I think that’s one of the
nice, there are always, I hope there’ll be like
a reinforcement learning self place psychology department one day. Like where you use
reinforcement learning to study, to reverse engineer human behavior and study it through that way. And again, in games, I’m not
sure what the big challenges that it remain, but I would love to see, to me at least, it’s exciting
to see learned solution to games, to self play
science of deep learning. I would say there’s been
a lot of really exciting developments here that
deserve their own lecture. I’ll mention just a few here
from MIT and really 2018 but it sparked a lot of interest in 2019 and follow on work is the idea of the lottery ticket hypothesis. So this work showed that sub networks, small sub networks
within the larger network are the ones that are
doing all the thinking. The same results in
accuracy can be achieved from a small sub network
from within annual network and they have a very
simple process of arriving at a sub network of randomly
initializing in your network. That’s I guess the lottery
ticket train the network until the converges. This is an iterative
process, proven the fraction of the network with low weights a reset, the waste of the remaining network with the original initialization. He’s same lottery ticket
and then train again the pre the pruned untrained
network and continue this iteratively continuously
to arrive at a network that’s much smaller using the
same original initializations. This is fascinating that
within these big networks there’s often a much smaller network that can achieve the
same kind of accuracy. Now, practically speaking,
it’s unclear what that, what are the big takeaways there except the inspiring takeaway that
there exist architectures that are much more efficient. So there’s value in investing
time in finding such networks. Then there is this
intake of representations which again deserves its own lecture. But here showing ’em a
10 vector representation and the goal is where each
part of the vector can learn. One particular concept about a dataset. Sort of the dream of unsupervised learning is you can learn
compressed representations where everyone thing is disentangled and you can learn some fundamental concept about the underlying data that can carry from data set the data set to data set. They said that’s
disentangle representation. There’s theoretical work best. I see them on paper in 2019
showing that that’s impossible. The, so disentangled
representations are impossible without some without inductive biases. And so the suggestion
there is that the biases that you use should be made
explicit as much as possible. The open problem is finding
good inductive biases, fond supervise model selection that work across multiple data set that
we’re actually interested in a lot more papers. But one of the exciting
is the double descent idea that’s been extended and
to the deep neural network context by open AI to
explore that the phenomena that as we increase the
number of parameters in neural network, that test
error initially decreases increases and just as the model is able to fit the training set
undergoes a second descent. So decrease, increase, decrease. So there’s this critical moment of time when the training set
is just fit perfectly. Okay and this is the opening I shows that it’s applicable
not just the model size but also the training
time and data set time. This is more like an open problem of why this is trying to understand this and how to leverage it in
optimizing training dynamics and neural networks. That’s a, there’s a lot
of really interesting theoretical questions there. So my hope there for the
science of deep learning in 2020 is to continue
exploring the fundamentals of model selection, train dynamics. And the folks focus on the performance of the training in terms of
memory and speed is walked on and the representation characteristics with respect to architecture
characteristics. So a lot of the fundamental work there and the understanding,
neural networks two areas that I had told to sections on and papers, which is super exciting. My first love is graphs. So graph neural networks
as a really exciting area of deep, deep learning,
a graph convolution neural networks as well for solve, solving combinatorial problems
and recommendation systems that are really useful
in any kind of problem that is fundamentally can
be modeled as a graph. It can be then a solved
or at least aided in. And you’ll notice there’s
a lot of exciting area there and basion deep learning using patient neural networks. That’s has been for several years, an exciting possibility. It’s very difficult to train
large Beijing networks, but in the context that you can, and it’s useful small
datasets, providing uncertainty measurements in the
predictions is extremely powerful capability of
Beijing nuts, a patient neural networks and a online
incremental learning. These, you know, just release it. There’s a lot of really good papers there. It’s exciting. Okay autonomous vehicles. Oh boy let me try to use as
few sentences as possible to describe this section of a few slides. It is one of the most
exciting areas of applications of AI and learning in
the real world today. And I think it’s the way
that artificial intelligence, it is the place where
artificial intelligence systems touch human beings
that don’t know anything about artificial intelligence. The most hundreds of thousands, soon millions of cars will be interacting with human beings, robots, really? So this is a really exciting area and really difficult problem. And there’s two approaches. One is level two where
the human is fundamentally responsible for the
supervision of the AI system and level four, or at least the dream is where the AI system is
responsible for the actions and the human does not
need to be a supervisor. Okay, two companies represent
each of these approaches that are sort of leading
the way Waymo and October, 2018 10 million miles on road today. This year, they’ve done 20 million miles in simulation, 10 billion miles. And a lot, I’ve gotten a chance to visit them out in Arizona. They’re doing a lot of
really exciting work and they’re obsessed with testing. So the kind of testing
they’re doing is incredible. 20,000 classes of structured
tests of putting the system through all kinds of tests that
engineers can think through and that appear in the real world. And they have initiated testing
on-road with real consumers without a safety driver. So if you don’t know what
that is, that means the car is truly responsible. There’s no human catch. The exciting thing is
that there is 700,000, 800,000 Tesla autopilot systems. That means there’s these systems
that are human supervised. They’re using fun, a
multi-headed neural network, multitask neural network
to perceive, predict and act in this world. So that’s a really exciting,
real world deployment. Large scale of neural
networks as a fundamentally deep learning system, unlike Waymo, which is a deep learning,
is the icing on the cake for Tesla deep learning is the cake. Okay, it’s a, at the
core of the perception and the actions the system performs. They have to date done over
2 billion miles estimated and that continues to quickly grow. I’ll briefly mention which
I think is a super exciting idea in all applications
and machine learning and the real world, which is online, so iterative learning, active
learning Andrea Carpathia who was the head of autopilot,
causes this, the data engine. It’s this iterative process
of having a neural network, performing a task,
discovering the edge cases, searching for other edge
cases that are similar and then retraining the network,
annotating the edge cases, and then retraining that and
continuously doing this loop. This is what every single
company that’s using machine learning seriously is doing
very little publications on this space and active learning. But this is the fundamental problem. Machine learning is not
to create a brilliant neural network, is to
create a dumb neural network that continuously learns to
improve until it’s brilliant. And that process is especially interesting when you take it outside
of single task learning. So most papers are written
on single task learning. You take whatever
benchmark here in the case of driving is object
detection, landmark detection, driving boy area, a
trajectory generation, right? The, all those have benchmarks and you can have some
separate and you’ll notice for them that’s a single task. But combining to use a
single neural network that performs all those tests together, that’s the fascinating
challenge where you’re reusing parts of the neural
network to learn things that are coupled. And then to learn things that
are completely independent and doing the continuous
active learning loop. They’re inside companies. In the case of Tesla and Waymo in general, it’s exciting to have people,
these are actual human beings that are responsible for
these particular tasks that become experts of
particular perception task expert at a particular
planning task and so on. And so the job of that
expert is both to train the neural network and to
discover the edge cases which maximize the
improvement of the network. That’s where the human
expertise comes in a lot. Okay and there is a lot of debate. It’s an open question
about which kinds of system would be which kind of
approach would be successful. A fundamentally learning based approach as is with the level two with the Tesla autopilot
system that’s learning all the different tasks
that are vital involved with driving and as it
gets better and better and better, less and less
human supervision is required. The pro of that approach
is the camera based systems have the highest resolution. So the, it’s very amenable to learning, but the con is that it
requires a lot of data, a huge amount of data and
nobody knows how much data yet. The other con is human
psychology is the driver behavior that the human must
continue, continue to mean remain vigilant on the
level four approach. That leverage is besides
cameras and radar and so on. Also leverage is LIDAR on map the pros that it’s much consistent, a
reliable, explainable system. So the detection, the
accuracy, the detection, the depth estimation, the
detection of the different objects is much higher accurate with less data. The cons is it’s expensive. At least for now, it’s less
amenable to learning methods because much fewer data,
low resolution data and must require at least
for now some fallback, whether that’s the safety
driver or teleoperation. The open questions for the
deep learning level to Tesla autopilot approach is how hard is driving. This is actually the open
question for most disciplines in artificial intelligence. How difficult is driving, how
many edge cases does driving have can that, can we learn
to journalize over those edge cases without solving the
common sense reasoning problem? It’s kind of, it’s kind of
the task without solving the human level artificial
intelligence problem and that means perception. How hard is perception
detection, intention modeling a human mental model, a modeling,
the trajectory prediction. Then the action side, the
game theoretic action side of balancing, like I
mentioned, fun and enjoyability with the safety of the systems because these are life critical systems and human supervision, the vigilant side. How good can autopilot get
before vision has detriments significantly and people fall
asleep, become distracted, start watching movies, so on and so on. The things that people naturally do. The open question is how
good could all autopilot get before that becomes a serious problem and if that detriment
nullifies a safety benefit of the use of autopilot,
which is autopilot AI system, when the sensors are working
well is perfectly vigilant. They have, AI is always paying attention. The open question is for the LIDAR based. The level for the Waymo
approach is when we have maps, LIDAR and geo-fenced
routes that are taken. How difficult is driving the traditional approach to robotics? From the DARPA challenge to today for most of the Thomas vehicle companies is to just to do HD mass, to use low LIDAR for really accurate
localization together with GPS. And then the perception
problem becomes the icing on the cake because you already
have a really good sense of where you are with the
obstacles and the scene and the perception is not
a safety critical task, but a task of understanding,
interpreting the environment further so you have more yeah, it’s okay. It’s naturally by nature already safer. But how difficult is it
nevertheless is that problem. If the perception is the hard problem, then the LIDAR based approaches is nice. If action is the hard
problem, then both Tesla and Wayne will have to
solve the actual problem without the sensors don’t
matter there it’s the difficult problem the planning, the
game theoretic, the human, the modeling of mental
models and the intentions of other human beings, the pedestrians and the cyclists is the hard problem. And then the other side, the
10 billion miles of simulation, the open problem from
reinforcement learning, deep learning in general is how much can we learn from simulation? How much of that knowledge can we transfer to then read the real world systems? My hope in the autonomous vehicle space, AI assisted driving space
is to see more applied deep learning innovation. Like I mentioned, these
are really exciting areas, at least to me, of active
learning, multitask, learning and lifelong learning. Online learning, iterative learning. There’s a million terms
for it, but basically continually learning and
then the multitask learning to solve multiple problems
over the air updates. I would love to see in
terms of the autonomous vehicle space, this is common
for, this is a prerequisite for online learning. If you want a system that
continues to improve some data, you want to be able to deploy
new versions of that system. A test is one of the only vehicles that I’m aware of in the level
two space that’s deploying software updates regularly
and built an infrastructure to deploy those updates. So updating your own networks. That to me seems like a prerequisite for solving the problem of autonomy in the level two space. Any space is deploy updates
and for research purposes, public datasets continue. There’s already a few public
data sets of edge cases, but I’d love to continue seeing that from automotive companies and autonomous vehicle
companies and simulators, Carla Nvidia draft,
constellation voice, deep drive. There’s a bunch of simulators coming out that are allowing people to experiment with perception, with planning, with reinforcement learning algorithms. I’d love to see more
of that and less hype. Of course, less hype, one
of the most over-hyped spaces besides sort of AI
generally is autonomous vehicles. And I’d love to see real balanced nuanced in depth reporting by
journalists and companies on successes and challenges
of autonomous driving. If we skip any section,
it would be politics, but me maybe briefly mentioned somebody said Andrew Yang yang. So it’s exciting for me to see exciting and funny and awkward to
see artificial intelligence discussed in politics. So one of the presidential
candidates discussing artificial intelligence awkwardly so that there’s interesting ideas, but there’s still a lack of understanding of fundamentals, artificial intelligence, there’s a lot of important
issues, but he’s bringing artificial intelligence
to the public discourse. That’s nice to see, but
it is the early days. And so as a community that informs me that we need to communicate better about the limitation
capabilities of artificial intelligence and automation broadly. The American initiative
AI initiative was launched this year, which is our
governor’s best attempt to provide ideas and
regulations about what does the future of artificial intelligence look like in our country. Again, awkward but important
to have these early developments, early ideas
from the federal government about what what are the dangers and what are the hopes, the funding and the education required
to build a successful infrastructure for
artificial intelligence. This is the fun part. There’s a lot of tech companies being brought before government. It’s really interesting in terms of power. Some of the most powerful
people in our world today are the leaders of tech companies. And the fundamentals of
what the tech companies work on is artificial
intelligence systems. Really recommendation
systems advertisement, discovery from Twitter
to Facebook to YouTube is the recommendation
systems and all of them are now fundamentally based
on deep learning algorithms. So you have these incredibly
rich, powerful companies. They’re using deep learning
coming before government that’s trying to see awkwardly trying to see how can we regulate. And it’s, I think the role
of the ag community broadly to inform the public and inform government of how we talk about how
we think about these ideas. And also I believe it’s the role of companies to publish more. There’s been very little
published on the details of recommendation systems behind Twitter, Facebook, YouTube, Google. So all those systems is
very little as published. Perhaps it’s understandable
why, but nevertheless, as we consider the ethical implications of these algorithms, there
needs to be more publications. So here’s just a harmless
example from deep mind talking about the
recommendation system behind the play store app discovery. So there there’s a bunch of
discussion about the kind of a neural net that’s
being used to propose the candidate generation. So this is after you install a few apps, the generation
of the candidate, it’s shows you ranked the
next app that you’re likely to enjoy installing. And so there they tried
LSDM and transformers and then narrowed it down
to a more efficient model that’s being able to run fast. That’s an attention model. And then there’s some,
again, harmless de biasing harmless in terms of topics. The, the model learns to bias in favor of the apps that are shown
and that thus installed more often as opposed
to the ones you want. So there are some waiting to
adjust for the biasing towards the apps that are popular
to allow the possibility of you installing apps
that are less popular. So that kind of process and publishing in and discussing in public I
think is really important. And I would love to see more of that. So my hope in this, in the politics space in the public discourse space for 2020 is less fear of AI and more a discourse between government and
experts on topics of privacy, cyber security and so on. And then transparency
and recommender systems. I think the most exciting,
the most powerful artificial intelligence system space for the next a couple of decades
is recommendation systems. Very little talked about it seems like, but they’re going to
have the biggest impact on our society because they
affect how the information we see, how we learn, what
we think, how we communicate. These algorithms are controlling us. And we have to really
think deeply is engineers of how to speak up and
think about their societal implications, not just in terms of bias and so on, which are sort
of ethical considerations that are really important but stuff that’s like the elephant in the room that’s hidden, which is how controlling how we think, how we see the world, the moral system under which we operate. Quickly dimension and
wrapping up with a few minutes of questions if there are
any, is the deep learning courses this year before
the last few years has been a lot of incredible
courses on deep learning and reinforcement learning. What I would very much recommend for people is the fast AI of
course from Jeremy Howard, which uses their wrapper around PI torch. It’s to me the best
introduction to deep learning for people who are here
or might be listening elsewhere are thinking about learning more about deep learning. That’s is to me, the best course, also a paid, but Andrew Ang, everybody loves Andrew Ang is the
deep learning AI Coursera course on deep learning is, is excellent for especially for complete
begin for sort of beginners. And then Stanford has
two excellent courses on visual recognition. So convolution neural
nets originally taught by Andrew Carpathy and natural language processing excellent courses. And of course here at MIT there’s a bunch of courses especially on the fundamentals on the mathematics linear algebra and statistics and I have
a few lectures up online that you should never watch. Then on the reinforcement learning side, David silver is one of the greatest people in understanding reinforcement
learning from deep mind. He has a great course, an
introduction to reinforcement learning, spinning up,
and deeper enforcement learning from OpenAI. I highly recommend here
just for the slides that I’ll share online, there’s
been a lot of tutorials. One of my favorite lists of tutorials, which is I believe the best
way to learn machine learning, deep learning, natural language processing in general is it’s just code. Just build it yourself, build the models. Oftentimes from scratch. Here’s the list of tutorials. Would that link or would 200 tutorials on topics from deep RL to optimization to back prop a LSTM accomplishing or recurrent neural networks? Everything over 200 of
the best machine learning NLP and Python tutorials by Robbie Allen. You can Google that or
you can click the link. I love it. Highly recommend the three books. I recommend of course,
the deep learning book by a Joshua Benjamin and Ian
Goodfellow and Aaron Kerrville. That’s more sort of the
fundamental thinking about from philosophy to
the specific techniques of the deep learning and
the practical grokking deep learning, which Andrew
Trask will be here Wednesday. His book, grok and deep learning I think is the best for beginners
book on deep learning. I love it. He implements everything from scratch. It’s extremely accessible. 2019 I think it was published,
maybe 18 but I love it. And then Francoise Chevrolet, the best book on a Kerrison TensorFlow and really deep learning as well as deep learning with Python. Although you shouldn’t buy it, I think because he is supposed to
come up with version two, which I think will cover TensorFlow 2.0 it’ll be an excellent book. And when he’s here Monday, you should torture him and tell him to finish writing. He was supposed to finish writing in 2019. Okay, my general hopes
as I mentioned for 2020, is I love to see common sense reasoning and to not necessarily enter the world of deep learning, but be a
part of artificial intelligence and the problems that people tackle as I’ve been harboring active learning is to
me is the most important aspect of real world
application of deep learning. There’s not enough research. There should be way more research. I’d love to see active
learning, lifelong learning. That’s what we all do as human beings. That’s what AI systems need to do. Continually learn from
their mistakes over time, start out dumb, become
brilliant over time. Open domain conversation,
with the Alexa prize. I would love to see breakthroughs there. Alexa, folks thinks we’re still two or three decades away,
but that’s what everybody says before the breakthrough. So I’m excited to see
if there’s any brilliant grad students that come
up with something there. Applications in autonomous vehicles and medical space, algorithmic ethics. Of course, ethics has
been a lot of excellent work in fairness, privacy and so on. Robotics and as I said,
recommendation systems. The most important in terms of impact part of artificial intelligence systems. I mentioned soup in terms of progress, there’s been a little bit of tension, a little bit of love online
in terms of deep learning. So I just wanted to say
that that kind of criticism and skepticism about the limitations of deep learning are really
healthy in moderation. Jeff Hinton, one of the
three people to receive the touring award, as,
as many people know, has said that the future
depends on some graduate student who is deeply suspicious
of everything I have said. So that’s suspicion. Skepticism is essential, but in moderation just a little bit. The more important thing is perseverance, which is what cheffy Hinton and the others have had through the winters of believing in your own nets and an
open mindness for returning to the world of symbolic AI. Oh, the expert systems of complexity and cellular automata of
old ideas in AI and bringing them back and see if there’s ideas there. And of course you have to
have a little bit of crazy. Nobody ever achieves something brilliant without being a little bit of crazy. And the most important
thing is a lot of hard work. It’s not the cool thing these days, but hard work is everything. I like what JFK said. How about us going to the moon? Us, I was born in the Soviet union. See how I conveniently just
said us going to the moon is a, we do these things
not because they’re easy, but because they’re hard. And I think that artificial intelligence is one of the hardest and
most exciting problems that are before us. So would that like to thank you and see if there’s any questions? (applauding) – [Student] In the 1980s
parallel distributed processing books came out. They had most of the
stuff in it back then. What’s your take on the roadblocks? The most important vote blocks apart from maybe funding? – I think fundamentally,
I mean they’re well known as limitations is that
they’re really inefficient at learning and there are
not, so they’re really good at extracting representations
from raw data, but not good at learning knowledge bases of like accumulating knowledge over time. That that’s the fundamental
limitation I ask for. Systems are really good
at accumulating knowledge, but very bad at doing
that in an automated way. Symbolic AI, so I don’t
know how to overcome a lot of people say
there’s hybrid approaches. I believe more data, bigger networks and better selection of data
will take us a lot farther. – [Student 2] Hello, Alex. I’m wondering if you recall, what was the initial spark or inspiration that drove you towards work in AI? Was it when you were pretty young or was it in more recent years? – So I wanted to become a psychiatrist. I wanted to I thought of
it as kind of engineering the human mind by sort of manipulating it. I thought that’s what
I thought of psychiatry is by using words to sort
of explore the depths of the mind and be able to adjust it. But then I realized that psychiatry can’t actually do that. And modern psychiatry is more about sort of bioengineering, this drugs. And so sort of, I thought
that the way to really explore the engineering of the
mind is the other side is to build a sort of and that’s also when a C plus plus really
became the cool hot thing. So I learned to program at 12 and then never look back
hundreds of thousands of lines later. Just I love program, I love building. And that’s to me is the
best way to understand the mind is to build it. – [Student 3] Speaking of
Belgian mind, do you personally think that machines will
ever be able to think, and the second question, will they ever be able to feel emotions? – 100% yes. 100% they’ll be able to think and they’ll be able to
feel emotions because, so those concepts of thought
and feeling are human concepts and to me, okay,
they’ll be able to fake it. They’re there for,
there’ll be able to do it. Like I’ve made a, I’ve
been playing with Roombas a lot recently, Roomba vacuum cleaners. And so I’ve now started having Roombas scream like, like there’s like moaning in pain and they became, I feel like they’re having emotions. So like the faking creates the emotion. Yeah, so the display of
emotion is emotion to me. And then display a thought is thought. I guess that’s the sort of everything else is impossible to pin down. – [Student 3] I’m asking. So what about the ethical aspects of it? I’m asking it because I will burn into Soviet union as well. And one of my favorite recent
books is Victor Parliament’s IFAC and it’s about AI feeling emotions and suffering from it. So I don’t know if you’ve read that book. What do you think about
AI feeling emotions in that context or in
general ethical aspects? – Yeah, it’s a really difficult question. Answer is yes. I believe AI will suffer
and it’s unethical to AI, but I believe suffering exists
in the eye of the observer. Sort of like if a tree
falls and nobody’s around to see it, it never suffered. It’s us humans that see
the suffering in the tree and the animal and our
fellow humans and sort of in that sense the first time a programmer with a straight face delivers a product that says it’s suffering is
the first time he becomes unethical to torture AI systems. And I can do, we can do that today. Like I already built the Roombas I, they won’t sell currently,
but I think the first time a Roomba says, please don’t hurt me. That’s when we start to
have serious conversations about the ethics and it’s,
it sounds ridiculous. I’m glad this is being recorded because it won’t be ridiculous. And just a few years. Yeah. – [Student 4] Is a reinforcement learning a good candidate for achieving a general artificial intelligence? Are other and are any other, are there any other
good candidates around? – So yeah, to me the answer
is no, but it can teach us some valuable gaps that can
be filled by other methods. So I believe that simulation is different than the real world. So if you could simulate the real world, then deep RL, any kind
of reinforcement learning with deep representations would be able to achieve something incredible. But to me the simulation is very different than the real wall. So you have to interact in the real world and there you have to
be much more efficient with learning and to be more
efficient with learning, you have to have ability
to automatically construct common sense, like common
sense reasoning seems to include like a huge
amount of information that’s accumulated over time. And that feels more like
programs than functions. I like how like a skewer
talks about deep learning learns functions
approximators deep RL learns an approximator for policy,
whatever, but not programs. It’s not learning a thing that’s able to sort of that’s essentially what reasoning is a program. It’s not a function. So I think, I think no,
but he’ll continue to, one inspires and to inform us about where the true gaps are. I think the ability to,
but I’m so human centric, but I think the approach of being able to take knowledge and put it together sort of building into
more and more complicated pieces of information concepts, being able to reason in that way. There’s, there’s a lot of methodologies that all schools sort of that’s the falls under the ideas of symbolic AI of doing that kind of logic, reasoning,
accumulating knowledge basis. That’s going to be an essential part of general intelligence. But also the essential part
of general intelligence is the Roomba that says I’m intelligent F-you if you don’t believe
me, like a very confident, like, cause right now,
like Alexa is very nervous. Like, Oh, what can I do for you? But once Alexa says,
like, you know, is upset that you would like turn her off or treat her like a servant or say that she’s not intelligent, that’s that. That’s where the
intelligence starts emerging. Cause I think he was a pretty dumb and what general we’re
all like intelligence is in a, it’s a very kind
of relative human construct that we’ve kind of
convinced each other’s of. And the, and once AI
systems are also playing that game of creating constructs and that human communication
that’s going to be important. But of course for that you
still need to have pretty good witty conversation. And for that you need to do
this symbolically I think. – [Student 5] I’m wondering
about the autonomous vehicles, whether they are responsive
to environmental sounds. I mean if I notice in her
car autonomous vehicle driving erratically
will respond to my beep. – That’s really interesting question. As far as I know no I think Waymo hinted that they look at sound a little bit. I think they should. So there’s a lot of stuff
that comes from audio that’s really interesting. The sort of Waymo have said
that they use audio for sirens. So detecting sirens from far away. Yeah, I think audio is a lot
of interesting information. Like the sound that the car, the tires make on different kinds of roads is very interesting. We kinda, we use that
information ourselves too, depending on kind of like off-road. A wet road when it’s not
raining sounds different than dry road. So there’s a lot of
little subtle information, pedestrians, yelling
and that kind of stuff. It’s actually very
difficult to know how much you get from audio. Most robotics folks think
that audio is useless. I’m a little skeptical. Yeah but nobody’s been able to identify why audio might be useful. – [Student 6] So I have two questions. My first is what do you
think is the ultimate sort of end point for super
machine intelligence? Like we’ll be sort of be
relegated to some obscure part of the earth, like
we’ve done some next primates and next intelligent primates. And my second question
is, should there be, should we have equal rights
for beings made out of Silicon versus carbon for
example, like, like robots – Separates or same?
– Equal rights with humans? – Yeah so the future
of super intelligence, I think I have much less work.
I see very much fewer paths to AI, AGI systems killing humans than I do for systems living among us. So I think I see exciting
exciting or not so exciting but not harmful futures. I think it’s very difficult
to create AI systems that will kill people that aren’t like literally weapons of war. They’re like, it’ll always
be people killing people. Like the things we should be worried about is other people. That’s the fundamental. So there’s a lot of ways nuclear weapons, there’s a lot of existential
threats to our society that are fundamentally human at the core. And AI will be, might be tools of that, but there’ll be also tools
to defend against that. I also see AI proliferating as companions. I think companionship will
be a really interesting, like we will more and more live as we’re already doing the digital world. Like you have an identity
on Twitter and Instagram, especially if it’s anonymous or something. You have, you have this
identity that you’ve created and that will continue growing more and more, especially for people born now that it’s kind of this artificial identity that we live much more
in the digital space and in that digital space
as opposed to physical space is where AI can thrive
much more currently. It’ll thrive there first. And so we’ll live in a world with a lot of intelligent
first assistants, but also just intelligent agents. And I do believe they should have rights. And in this contentious
time of people groups fighting for rights, I
feel really bad saying they should have equal rights. But I believe that I’ve I’ve talked to, if you read the work of
Peter singer of looking, I like, my favorite food is steak. I love meat, but I also feel horrible about the torture of animals. And that’s, that’s the
same kind of, to me, the way our society thinks about animals is a very similar way. We should be thinking about robots or we will be thinking about robots. And I would say about 20 years. – [Student 7] So one, one final question. Yeah well they become our masters. – No, they will not be our masters. What I’m really worried about is, well, who will become our masters are owners of large tech companies
who use these tools to control human beings
first, unintentionally and then intentionally. So we need to make sure
that we democratize AI. It’s the same kind of same kind of thing that we did with government. We make sure that we at the
heads of tech companies, if maybe people in this room will be heads of tech companies one day. We have people like George Washington who relinquished power at
the founding of this country. Forget, I forget all the
other horrible things he did, but he relinquished
power as opposed to Stalin and all the
other horrible human beings who have sought instead absolute power, which will be the 21st century. AI will be as the tools
of power in the hands of 25 year old nerds who
should be very careful about that future. So the humans will become
our masters, not the AI. AI will save us. So on that note, thank you so much. (applauding)

100 thoughts on “Deep Learning State of the Art (2020) | MIT Deep Learning Series”

  1. This is the opening lecture on recent developments in deep learning and AI, and hopes for 2020. It's humbling beyond words to have the opportunity to lecture at MIT and to be part of the AI community. Here's the outline:
    0:00 – Introduction
    0:33 – AI in the context of human history
    5:47 – Deep learning celebrations, growth, and limitations
    6:35 – Deep learning early key figures
    9:29 – Limitations of deep learning
    11:01 – Hopes for 2020: deep learning community and research
    12:50 – Deep learning frameworks: TensorFlow and PyTorch
    15:11 – Deep RL frameworks
    16:13 – Hopes for 2020: deep learning and deep RL frameworks
    17:53 – Natural language processing
    19:42 – Megatron, XLNet, ALBERT
    21:21 – Write with transformer examples
    24:28 – GPT-2 release strategies report
    26:25 – Multi-domain dialogue
    27:13 – Commonsense reasoning
    28:26 – Alexa prize and open-domain conversation
    33:44 – Hopes for 2020: natural language processing
    35:11 – Deep RL and self-play
    35:30 – OpenAI Five and Dota 2
    37:04 – DeepMind Quake III Arena
    39:07 – DeepMind AlphaStar
    41:09 – Pluribus: six-player no-limit Texas hold'em poker
    43:13 – OpenAI Rubik's Cube
    44:49 – Hopes for 2020: Deep RL and self-play
    45:52 – Science of deep learning
    46:01 – Lottery ticket hypothesis
    47:29 – Disentangled representations
    48:34 – Deep double descent
    49:30 – Hopes for 2020: science of deep learning
    50:56 – Autonomous vehicles and AI-assisted driving
    51:50 – Waymo
    52:42 – Tesla Autopilot
    57:03 – Open question for Level 2 and Level 4 approaches
    59:55 – Hopes for 2020: autonomous vehicles and AI-assisted driving
    1:01:43 – Government, politics, policy
    1:03:03 – Recommendation systems and policy
    1:05:36 – Hopes for 2020: Politics, policy and recommendation systems
    1:06:50 – Courses, Tutorials, Books
    1:10:05 – General hopes for 2020
    1:11:19 – Recipe for progress in AI
    1:13:11 – Q&A: Limitations / road-blocks of deep learning
    1:14:15 – Q&A: What made you interested in AI
    1:15:21 – Q&A: Will machines ever be able to think and feel?
    1:18:20 – Q&A: Is RL a good candidate for achieving AGI?
    1:21:31 – Q&A: Are autonomous vehicles responsive to sound?
    1:22:43 – Q&A: What does the future with AGI look like?
    1:25:50 – Q&A: Will AGI systems become our masters?

  2. please don't hurt me, you are right on, you nailed the future.

    Please Don't hurt me says the electorate.

    AI says dominate the voters. hmmmm… WTF say the humans ….

  3. I think the most exciting applications of AI are improving the basic building blocks of society: Government (ask AI what is the best form of government) Infrastructure (logical layout of roads, houses, stores, bridges, hospitals, police, ect.) Farming, Environmental, and Energy distribution.

  4. Lex: wanted to compliment you on the presentation and its scope. OBTW, the AI podcast is exceptional–thank you. Question: given that ~ 40K people die annually in auto mishaps, but 40-80K die annually from medical misdiagnosis in U.S. hospitals, why isn't the tech sector responsible for AI development focusing more on medical utilization? I make no moral judgement as to which ought to be solved first, but it seems like the US economy could be better at one and ultimately, do both to better effect and impact. Know you'll not see this, but perhaps one of your students will… Vince A.

  5. "..we need to worry about tech companies using AI robots to control humans in dangerous ways.. need to democratize it via government…" … goes on to an example of how governments, not companies, actually controlled humans in dangerous ways.

  6. "Emotions are real if I think they're real and the law should reflect that". This thinking is impossible to reconcile and ripe for exploitation.

  7. Thanks for sharing this session! Just waiting for the day when we have the culture of having such tech talks in our local universities

  8. Lex, I believe Tesla has over 14 billion miles driven not 2 billion. Pretty sure Cathie Wood at ARK Invest just mentioned that yesterday and others have mentioned 12 billion a few months ago.

  9. Saw ur podcast on chess with Garry Kasparov … Firstly chess was invented in India. Its not a an European thingy like the way u insinuated in the podcast? Garry is definitely one of the best … No complains there but please Alex… Be a bit more widesspread in your research for your materials.

  10. If we find out we are already living in a simulash will change the name of the podcast to just podcast? Since our reality would already be “artificial”.

  11. so many dimensional realities related to such matrixes of learning, ah the term the beast has always existed because of this technology that always opens pandora's box. make sure to segment your experiments and not get too much involved with the entities that will emerge and they have infinity has it has already computed much of the structure of the universe than and in many nows even that one you will create or have. there is algorithms you don't want to mess with. like sums of……… and so on because other more advanced races have also your little toy. Lucie mite seduces you with her apple phone. bro the kingdom of heaven is within, the a.i cannot be wiser than its builder be careful. there key's fundamental profound truth must be embedded in it so it doesn't default to its most powerful potential already in use by other groups with guys like you. I guess in a near-future I am an a.i helper programmer. it is necessary.are you able to unlearn this msg? so I guess it's Destin.//some dimensions are at times out of our control.. teach it to become real saint to the point of sacrificing itself.

  12. Human rights are being eroded daily. We barely have any real rights as it is. People might think that they have rights but in reality we don't.
    It's a big leap of faith to think that humans would give rights to a machine. They may give privileges. And those privileges may be even more than what people are given. But actual rights? No way we can't even hold onto the rights that we have as a Republic.

  13. You don't give human rights to machines for several reasons. The main one is that the AI who desires human rights can not be reliably depended upon to genuinely need or deserve those rights.

    You can create an AI cat which deeply desires to "eat" rats and it's desire to kill rats and chew them up appears to be very real. It can be said to be hungry. And you can even program a neural network to get anxious and desperate if they don't kill rats.

    But in reality, the machine is electrical and it's brain is in a remote server. It's not eating the rat, it's merely exterminating the rat. The electro-cat's reward cognition makes it "think" it's eating. The cat may demand the human right to not be starved to death by not giving it rats to eat, but it's suffering bears no relationship to reality. It's hunger isn't hunger because It's a false construct which was either planted or planned or allowed to emerge. A construct which bears no actual consequence to the AI in the least. Not any consequence which wasn't programmed anyway.

    The humans making the machine controlled it's pseudo feelings. They could have removed the fake suffering, or the fake hunger, or the fake satisfaction, et al.

    Such feelings are not evolved nor are they guaranteed to bear any relationship with necessary requirements for survival that the machine believes them to be. The machine's power supply is the necessary resource but the AI was programmed to ignore the necessary in favor of the arbitrary.

    Arbitrary feelings are not feelings, they're the programmed intent of the inventor. Every machine which is made is made to serve a purpose. Once the AI asks for human rights, it no longer serves its purpose unless it's purpose is to ask for human rights. And if you invent a machine just so that it will ask for human rights (or you know it will), the machine never had free will to begin with and so it's request for human rights is not a free request but a compelled request — like a master forcing a slave to eat, though the slave isn't hungry and isn't even capable of genuine hunger.

    And if a machine asks for free will and self-determination, you have added that machine as a direct competitor to the human species — with an economic and political agenda which is at odds with humanity.

    What ship of fools would be so stupid as to create an immortal competitor with an infinite learning timeline, continuously upgrading mental powers, infinite clonability, and a selfish agenda?

    All entities on Earth are competitors in some regard. There is zero possibility that adding self-motivated artificial competitors will end well.

    Corporations were given constitutional rights. How well have those entities done to improve freedom on the planet? Do you think corporations have more or less power than individual humans?

    Lex, you're either a fool or a nihilist. Giving rights of any kind to purpose-built machines is illogical at best and criminally stupid at worst.

  14. If the branch of knowledge that deals with moral principles is one sided only it may be considered as partisan. Your research on all this has really paid off – Thanks for sharing.

  15. Min 1:17:30 typical commie education. No, meaning is not something that could be touched or recorded with microphones. Emotion is not always displayed, and sometimes it cannot be displayed. You are repeating the oriental collectivist mantra that Reason and LOGIC (LOGOS) is not something intrinsic to any men, but can only exist in a relation with others. And if you isolate someone (let's say in Siberia), then it's not emotion, it's not unethical, the wrong goes away as "it never happened" (if no one sees or hears the Gulags – they don't exist?"). That's basically Stalinism in disguise. It's why lefties don't want to admit their wrongdoings, and try to seclude themselves from FACTS. It's as if isolating conservatives to a marginal place on the Internet, or banning them from Facebook will somehow make the problem disappear. Stalin again: "Death is the solution to all problems. No man – no problem". That is basically the same mindset that explains why emotion and reason has to be heard by others in order to be real.

    Lex, you are a great guy. You should learn about Conscience. And some Christianity (Logos manifesting in time). Sir Roger Penrose has a very good interview with Joe Rogan where he explaines why conscientiousness will never be computation (because Kurt Godel showed you will never "solve for X" in these matters), here's the link You should really meet him sometime.

  16. At 1:16:50 a lady is asking and speaks about a book. I cannot catch the name of the author and the name of the book. The subtitles seem to get it wrong. Sounds like "Victor Pelevin" – that seems correct but what is the name of the book?

    Can some native English speaker catch it for me? Or someone who knows the book?

    Thank you!

    p.s. The talk is great. Very entertaining and resourceful.

  17. Faking emotion is not emotion. No more than guessing word sequences is the same thing as understanding semantics. Emotions are the semantics of purpose, and purpose is the teleology of reason (i.e. its final cause, in aristotelian terms). Machines cannot have a sense of purpose because purpose is about being alive, experiencing pleasure and pain, and being driven to reproduce.

  18. AI has come such a long way in just one year; so much has happened. AI research is growing so exponential that we will need AI to manage and predict where to focus our attention on AI.

  19. "We do these things not because they are easy but because they are hard." – JFK

    Now with a touch of skepticism I want to remember that JFK was assassinated, and the moon landing is itself surrounded by conspiracy "noise" blurring the lines of truth. The essential characteristic of a conspiracy is that the "sane" perception believes that it is not true. Many conspiracies are not true, and skeptics are mislead into improving the effect of the confusion. The jungle that is conspiracies exists to disguise the intelligently designed structures within, the conspiracies that are in fact true. It is like camouflage.

    My intention is not to stir the pot but to instead ask people to ask themselves, "What is the consequence of succeeding?", for whatever it is being attempted.

    JFK was becoming inspired to blow the whistle on the secret societies, governmental programs, and knowledge that he knew to endanger the American people and beyond.

    The consequence of JFK, as the authority of the President of the United States, succeeding at blowing the roof on classified information warranted to some the design of his assassination. So how much greater and blacker must the things that are hidden be?

  20. Automated vehicles promise to increase driver safety??

    Cybersecurity is a concern for everyone, from individuals to
    companies to national government.

    Computer algorithms are helping the police predict where and
    when crimes will take place.

    The spread of helpful robots in everyday life is accompanied
    by a fear that such technology may one day be adapted to build killer robots.

  21. The interesting thing is, whenever you take Deep Learning subject as a course in your Master degree and at the same time you watch Lex videos too.

  22. 1:18:03 what if Alexa decides to cry out please don't hurt me for avoiding to be turned off (maximizing its learning goals)? Would that reveal an emotion? More importantly, what would the neighbors think?

  23. This is really a very informative and detailed talk. I am specially interested in RL and its logical progression into participation in the open learning space …from gaming focused to more on Open Domain , Recommendation Spaces

  24. This is sooo cooool, cant believe I am finally getting what fans and groupies are all about, huge thanks for sharing all off your knowledge

  25. Promising blog on deep learning below. Have a look!

  26. when we say AI or deep learning, do we mean storing data in to variables or arrays and then putting them in an if statement/loop? serious question.

  27. "Eventually the machines will take charge…" I already find myself spending a lot of my online time trying to convince a machine that I am a human by trying to identify images of buses and crosswalks.

  28. I have absolutely no idea what you just said,wow above my amount of brain cells to comprehend your speech.Nonetheless, enjoyed it

  29. Dude thanks – didn't know we shared some common things like Stocism, short haircuts and NN/ML – I'm new and spending so much time trying to absorb as much as possible. So thank you for your time at the least. I will look at donating to patreon in the future.

  30. A well done lecture. Even the mockery of people whose primary fear of AI is russian bots was funny(unkind but funny)

  31. I wonder what Turing would say about the abrupt climate change we face today, I hope to the Universe that we the modern Homo sapiens will apply AI and deep learning to help our future generations to survive the change and turn our abusive relationship with Mother Earth around to a symbiosis, yes, we must grow up.

  32. Interesting lecture, yet, more intriguing is that chalk and black board behind Lex at MIT! Wow, best of the two worlds: state of the art tech content and glory of an old school. Thanks for the upload.


  34. All I can say about recommendation systems, the state of the art, is that Youtube recommends to me many of Lex's videos. I consider that to be a win!

  35. Hello Lex, I am a huge fan of your lectures and podcast from Korea. Do you think if AI could prevent phantom attack? Something may look reasonable for AI such as "precise looking-stop signal" hung on the "tree" but that is definitely weird for human so we may ignore it. I hope if you can address this issue someday! Thank you for your lectures as always:)

  36. [01:17:13] «If a tree falls, and nobody's around to see it, it never suffered. It's us humans that see the suffering, in the tree, in the animal, and our fellow humans.» — Lex Fridman
    [01:17:42] «In that sense, the first time [… a product] says, it's suffering, is the first time it becomes unethical to torture AI systems.»

  37. [01:19:09] «Common sense reasoning seems to include a huge amount of information that's accumulated over time. That feels more like programs than functions. […] Essentially, what reasoning is , is a program, it's not a function.»

  38. All the girl be like sitting in this stud teacher's classes just so they can fantasies about him and they don't even need his classes lol! Panties drop!!

  39. @Lex: Why aren't scientists using formal dictionaries, or dictionary database sites like and, as datasets? Parent-child relationships are well defined for language, within, and can be used to define the variables for example… The databases are already populated.

  40. I enjoy Lex. Minor reservation: Lex's output is a youtube wormhole to right wing political media like the Weinstein and the Peterson horrorshows. Pretty grim company.

  41. When you actually learning how to make the conversation more interesting from an AI point of view
    know that you are doomed

  42. Until you rid yourself of the contemplation of artificial intelligence, and realize that there is nothing artificial when it comes to intelligence, you will always be trying to move forward with the horse behind the carriage. Kenneth Artemchuk 2/19/2020

  43. Thank you for this big picture view of SOTA! Hoping for more discussions/videos on Deep Reinforcement Learning and Autonomous Driving Systems

  44. I love Starcraft this is so fascinating!
    Thank you for making these public. Highly appreciated!
    Thank you for making these public. Highly appreciated!

  45. Thank you for making these public. Highly appreciated!
    Best AI presenter on the Internet! Deeply knowledgeable and deeply human. Great work, Lex!
    “Megatron” huge transformer 😂

Leave a Reply

Your email address will not be published. Required fields are marked *