Fast.ai, Learning to Learn | Machine Learning, Kaggle & Blogging | Interview with Radek Osmulski


Sanyam Bhutani: Hey, this is
Sanyam Bhutani and you’re listening to “Chai Time Data
Science”, a podcast for data science enthusiasts, where I
interview practitioners, researchers, and Kagglers about
their journey, experience, and talk all things about data
science. Hello, and welcome to another
episode of the “Chai Time Data Science” show. In this
interview, I interview not just one of my machine learning
heroes but a machine learning hero to the complete fast.ai
family. I like to call it family instead of community. Fast.ai
hero Radek I think this is one of the most honest, and the best
interviews in terms of the transparent Kaggle learning to
learn, and how to approach machine learning problems or
machine learning materials advice. Radek has shared his
journey with complete honesty about how he went about learning
the fast.ai materials, how he went on to smashing the kaggle
competitions that he participated in, and his journey
on learning to learn about all of the materials fast.ai and
beyond in the machine learning world. We of course talked a lot
about learning to learn fast ai, and Kaggle all three together
and individually as well. I’m really excited to share this
interview. I honestly feel this has been the one of the best
interviews I’ve done on this series. And I really recommend
that you stick to the end of the interview for all of the amazing
advice is, this is one interview that has every single answer
that I wish I could hang near the, near my work desk. So I
hope you enjoy it as much as I did. A quick reminder to the non
native English audience this along with all of the interviews
have checked English subtitles, so remember to enable them on
YouTube if you’re watching it on YouTube. Without further ado,
here’s my conversation with fast.ai hero Radek, all about
learning to learn fast.ai Kaggle and machine learning. Please
enjoy the show. Hi everyone today on the show I
have a, if I may peak for the fast.ai community, a fast.ai
hero, thank you so much for joining me on the podcast. Radek: Thank you so much Sanyam,
for having me, super happy to be here. Thank you for your very
kind words, Sanyam Bhutani: Really excited
to have you on the show. So this podcast for the audience is
slightly different. We’ll be talking more about your learning
journey and how you went about learning to learn. I believe
this is a theme that both of us, I think I became interested in
it through you. So can you tell us why you started learning
machine learning? As far as I know, it wasn’t because you were
interested in getting a job it was out of pure passion for you. Radek: Yes, it may definitely
seem like that. Maybe in this setting, I can share the full
story. So you know, it all started way back when I was
working as a manager at multinational corporations in
90s, so I had a couple of jobs like that. And these jobs for
making me absolutely miserable. I wanted to change something in
my life, in my professional life, and back then my
perception was that, hey, if you want to be doing something else,
you have to learn something that was, now I see it as a as a
misconception probably, but that was my take. And there were all
these MOOCs that started coming out. And I thought, you know,
let me do that. So I loved the experience of doing MOOCs a lot.
It was a lot of fun. And I came across the machine learning MOOC
from Andrew Ng, right-The one on the Coursera with the little
robot. Sanyam Bhutani: Hehehe. Radek: You know, I, I find
enough. I feel like then with machine learning, and now, I
think that for quite a while I had the idea of wanting to do
machine learning professionally as a job but it seems so far out
to me that I didn’t admit it even to myself back then. Sanyam Bhutani: Okay. Radek: For quite a while. But
yeah, that that’s how it started. And since I had such a
good time learning machine learning, I guess I guess that’s
where I am now. Sanyam Bhutani: Okay. Now, I
want to pick up another theme, which is setting goals. Now,
this is one theme that I picked from you through the fast.ai
forums. But before we talk all about that, can you tell us why
what led you to starting fast.ai or even discovering fast.ai. Pure desperation, you know, pure
desperation and luck. So desperation in a sense that up
until the discovering fast.ai, you know, I would try to
recreate the academic curriculum and sort of go through it
myself. So a lot of math and things like that, and calculus
and real analysis and proofs and, you know, there’s a lot of
amazing material on YouTube lectures where, you know, if you
if you want to you can, you can do that. But this was not moving
the needle for me, like I would be doing good on that and I
still wouldn’t be able to approach a real life machine
learning problem. And the last thing I did before I, I learned
about fast.ai, I implemented Jeffrey Hinton’s papers from the
80s in NumPy. I didn’t even know that people were using Pytorch
or TensorFlow, honestly. And, you know, and then I saw this
post from Richard Thomas on Hacker News saying that hey,
we’re doing this course, I became interested. So I did the
fast.ai view on part two. And, you know, I did that, but their
ideas shared their own learning seems so far out to me. And it
was hard to catch up on part one of the course and be doing part
two, that you know, I have given up, I said, you know, this is
it, I stopped doing machine learning at all, because, you
know, I, I already had my older daughter and then and I thought,
well, I cannot be spending that much time on an activity that
will probably not add value to my professional life at any
point in time. So it was basically like pursuing a hobby
and a hobby that would require a lot of time, so I dropped
machine learning altogether. And for five months, I focused on
learning, web development continuing to learn web
development. I already had the web development job back then,
and learning about software engineering because that’s what
I thought that you know where I would go in my life
professionally. And you know, I checked the server looks later
and for five months, I haven’t even touched machine learning.
But then, I saw the I saw a post with fast.ai V2 to come [] The new version Radek: The course the diversion
of the courses that it was there for fast.ai two or one and I
said to myself, well, it was purely emotional. It was not
very logical, but I felt like, you know, the thoughts from the
course the journey, certain learning, they were sort of sort
of like growing in my mind over those five months, and I thought
maybe I will give this completely different approach a
try. And just one last try. Let me do it for a couple of months.
If I will see that there are no results. I drop it, for sure.
But it turns out that the approach works really well. Sanyam Bhutani: Yeah. Radek: I haven’t dropped machine
learning yet. And it seems that maybe I will never do so. Sanyam Bhutani: Awesome. I think
I think I definitely share the struggles of everyday MOOCs with
you. I’ve already written about it in a few blog posts, but
we’ll talk more about that. Can you tell us what made you stick
through fast.ai even on the forums you had constantly
mentioned you were always on a fight with yourself, also about
your schedule, how did you stick to fast.ai? Radek: Fast.ai works. You know
that’s, that’s the main reason I feel it brings results like I
could see myself making progress and I was discovering this new
way of learning. I could see that I was starting to do things
that I would, I was not able to do a couple of weeks or months
before. So that gave me the energy to keep going. Now, yeah,
that that’s basically it. Sanyam Bhutani: Now this, for me
at least was a completely new paradigm because I was in
university back then where you’re taught the bottom up
approach with pretty much all of the world follows. Did you find
the top down approach very intuitive, were there any
fallout of that? Radek: I unfortunately used to
be a very good student. It’s, it’s; Sanyam Bhutani: Hehe. Radek: It’s, you know, one of
the biggest downfalls in my life. So I, I had this bottom up
approach engraved in my mind and know like, what was what Jeremy
was what Jeremy Howard was saying in the videos. It sounded
absolutely crazy. It sounds crazy. It felt crazy. It felt
like I was not doing any work. Sanyam Bhutani: I think it still
does. For me at least. Radek: Yeah, you’re right. And
you know, never never this notion. If you spent 12 years
plus in this academic system, that’s very strict and very old
school. Yeah, the this feeling doesn’t go away where, you know,
you want to know things and the way you know things is if you
can speak out the theory or you can write out the proof and
mathematical proof. But it turns out, I was horribly mistaken and
actually trying to learn something without this hands on
approach. I, I’ve seen so many people try to go down this
route. And because you don’t get constant feedback from what
you’re working on, you are learning either theory that is
not relevant to what you’re doing. Or you think that you
have learned the theory but in practice, you know, your
reasoning has a fatal flaw, but you never discover it because
you have no experience doing what you’re learning about. So,
yes, now I’m definitely in the top down learning approach or so
I thought Sanyam Bhutani: You’d also
shared your approach, your initial thoughts and I remember
the title was what I will focus on during the course it was very
interesting trade. What were your plans and efforts? How did
you invest into the course for the audience who doesn’t know
you from the forums very well. Radek: So it helps to take pick
what you want to focus on in any activity that you’re doing, you
know, and this is what this post service for me where, you know,
I before, as I was writing the post, I would take some time to
think about how I want to approach this already. It was
probably part two of you, too. I’m not sure when I wrote this.
But yeah, I outlined what I wanted to work on how I wanted
to work on things the projects I wanted to do. And also very
importantly, I said what I would not do, you know, this section I
feel was very important because I was still stuck a little bit
in this bottom down bottom up way of thinking when it comes to
learning. And there are these books I still have them on my
shelf, you can actually see Sanyam Bhutani: that is watching
on YouTube. Yes, you know the books that I
would absolutely love to read that I would enjoy to read by
and and to learn some of the mathematical concepts. But I had
the choice either I will do something that will be effective
that will teach me how to do machine learning, which is what
I wanted or I could invest my time somewhere else. So I chose
to invest my time where I felt I would get the best return in
becoming stronger machine learning would say practitioner
because that’s what we say. But in fact, this is what machine
learning researchers do. Definitely, I think, and I don’t
want to start a debate it maybe just my opinion from past a
courses also it’s it’s a flaw that has been ingrained in us
that unless you know the math, you cannot do anything, whereas
applying something is also completely different situation.
And that’s also a learning experience. Radek: Yes, but the thing is
that you could know the math, and you could know the math at a
really high level. And if you have just disliked to stand on,
you know, you will not succeed and by succeeding, you know, so
it depends what definition of success you have, for me in the
context of learning machine learning, learning to do machine
learning, I wanted to be in a situation where if I’m presented
with a problem, or if I’m presented with some set of data,
I would be able to do something useful in a given context, and,
you know, it’s not that I don’t know math, or I didn’t know
math, or that I would not be able to learn math at a high
level of, of some, some sort of high level because this in fact,
felt easier for me to do. This is what I have been doing for my
entire life one school. It’s; Sanyam Bhutani: Yeah. Radek: But it was not bringing
me the results that I wanted. So it was it was hard to start
doing something, as you know, to open up a Jupiter notebook and,
you know, call a method that did something and then try to
understand how it worked and how we can use it to good effect
that was actually hard to So when somebody portrays learning
math as the absolute pinnacle of we can do in life, you know, I
feel that maybe they feel like that and that’s fine. But if I
were to keep learning math and theory and you know, whatnot, I
would have chosen the easy way out, at least for myself. So; Sanyam Bhutani: Yeah. Radek: Doing fast.ai was a
challenge. Sanyam Bhutani: I definitely
agree with you on that. Now, I remember you were very laser
focused on studying what Jeremy suggested and working on a very
few things just around the lectures, not way too out of the
lecture. Can you elaborate on your plans for the future
students who might like to get inspired or follow this path? Yes, I was. I was desperate, you
know. So if I didn’t know how to get where I wanted to get. And
there was Jeremy Howard, and he offered actionable advice. Like
why should they not try following that regime of, you
know, of going about things. So I approached this, I somehow
managed to approach the advice with an open mind and I tried to
implement it as, as best as I could. And initially, it was
just working with fast.ai material. But very soon I was
able to apply the same approach to, you know, come up with my
own ideas or with my own data sets to to work on and then
Kaggle, so yeah, it was just a natural extension of what
happened in the course. Okay, now, especially I and I
think a few of us really struggle with finding the
homework because Professor Jeremy doesn’t, isn’t very
strict with homework per se for the students. Radek: Yes, yes. Okay. All
right. So, I see I see. Yes. As for the homework. So in the
lectures, Jeremy Howard explains how you should probably go about
learning this material. And the way is, you listen to the
lecture, you see what is being done. Then after the lecture,
you open a notebook, and you step through each cell, you run
the code, but you don’t run it blindly. You you run it, and
then you see what happened. You see what were the inputs, you
see what are the outputs. If you’re feeling particularly
adventurous, you can even make you look at the definition of
the methods that’s recording. So that’s the thing you should do,
first of all, then secondly, you try to reproduce the notebook
without looking too much in the notebook. Now that’s obviously
impossible. You know, to do this, you still probably have to
look at the notebook quite a lot of this, okay. And then the
third step is to take the lessons you’ve learned in the
lecture and apply it to maybe a different data set, or maybe you
know, write some post on the forums or write a blog post
about what you learned, sort of to hold ideas in your head for a
little while longer and look at it from a different perspective.
And that’s how you learn. Now, the interesting thing here for
me is that Jeremy Howard explains how to find the
homework, how to do the homework, and quite a few times
in the lectures and also on the forums. So I think that if
people are asking the questions or you know, I don’t know what
homework I should be doing and where to find the exercises,
what they might be saying is they are having maybe, you know,
like a motivation problem where they don’t feel that this
approach would be effective. And to that, I can say that probably
the first point to start to overcome this is to actually
realize that, yeah, this is the reality of life. If you want to
learn this, you have to sit in front of your computer and, and,
you know, spend some time with with things that might seem
basic, but they’re not actually basic. This is the hard work of
you know, of learning that learning something doing that
top to bottom approach, and I’m convinced that this approach
works. Sanyam Bhutani: I definitely
agree on that. Another thing that Jeremy mentions quite often
but very often running in the courses, do Kaggle. Now I used
to feel very intimidated about going on Kaggle because the
notebooks are complete. They work very well on the data set
that Jeremy showcases us, but there’s absolutely no clue of
how to take that approach and apply it on Kaggle. At what
point did you feel confident enough to jump on kaggle and
absolutely smash it on the leaderboard. So, it was a natural
progression, you know, I felt basically I was in this line of
thought of learning from the lectures and coming up with my
projects to practice things on. And initially when I started
with Kaggle, I didn’t smash anything apart from maybe no
smashing my head against the computer screen. Because, yes,
you need a completely different, no, you need to learn additional
skills, skills. Again, that might seem you know, very down
to earth, but they’re actually the meat of doing machine
learning. The reward. So, you know, moving data around
processing data, I remember the first competition I worked on.
And I started very late in the competition, but I didn’t even
get past copying the data and resizing the data so that I
could feed it to a model. Because there was so much data,
I had to learn how to use a cloud VM. So, you know, and then
how do you move code across from you know, your workstation to
the cloud VM? How do you ensure that you have the right things
installed there to do work? How do you even connect to that
machine from your local computer, you know, how do you
edit code, all of this, you know, you pick up along the way.
And in fact, as you start making money doing machine learning,
you will find that the these are the skills that are superly
important and where you will spend most of your time, you
know, delivering real value to a business. It will not be in
seeing whether I don’t know, ReLu or Leaky ReLu works better
in a problem. It will be, you know, having a budget of 40
hours and trying to blow your customer away with the results
that you can bring. And so you you are learning doing Kaggle
you’re learning something that is extremely vital. I definitely concur with your
advice and especially things such as setting AWS, where you
just put in your credit card and you have absolute no idea about
you type in a few words and determinable goes crazy. Now you
feeling your money’s going away into that terminal? Yes, yes, exactly. And the good
thing is that you can search for answers because you know it’s
not 10 other people in the world who are trying to do the same
thing. It’s you know, thousands of people who are faced with the
same problems. So, yeah, you try to solve the problems and you
learn. I think there’s also two
different sides of it. You I know are on the extreme other
end the person who didn’t who wouldn’t ask for advice after
you’ve exhausted every single option. And then there’s the
other crowd of I think the bottom of applause from
university who just go to the professors for every single
question. How do we continue searching even though you feel
completely desperate, you don’t have any answers? It feels like
drowning and you’re gasping for air. You don’t know anything?
What’s going on? I think you have to be humble in
that. You know, we all mainly or some of us may be imagined that
we will spend a week on doing an online course and tomorrow we
will be doing cutting edge research. We want to focus on I
don’t know, coming up with a new activation. Because that might
seem you know, amazing or interesting, but it’s like
martial arts or in any discipline, you need to learn
the basics you need to learn the fundamentals. If you want to
become a good cook, you know, you need to learn how to add
solid, you know, we don’t go into fancy things. When you want
to be a martial arts expert, you will need to practice basic
moves over and over again for years. And same with machine
learning. You need to practice the basics and the basics are
moving data around. They are the vibe, the train, test split, you
know, and you need to be doing this over and over again. So,
yeah, this is hard. This is hard to even have done this before.
And it’s okay if something that is may be frowned upon by people
in certain circles that diffuse challenging to you. But if you
learn things like using the terminology, like a bit of
DevOps, you know, you suddenly start having these superpowers.
And it’s absolutely worth your time. So I would say do not
despair, be accepting of going back to learning basic things
about doing things on the computer. And also, if, if it’s
the case that you’re having a hard time understanding some
concept, and it seems advice to you. Then the best piece of
advice I can give you here because I struggled with that
myself as well at some point, and learning how we learn had me
quite a lot. So you need to give yourself time for the
connections in your minds to form And that basically means
that you need to keep reading the material, you know, over
some period of time and trying to learn about the concept from
different perspective sort of like arranging a jigsaw puzzle
in your mind. But the key here is, you cannot cram things into
your mind or you cannot spend two days and it doesn’t work
like that. But if you spend 15 minutes, over 10 days, you know,
15 minutes every day, then you will see completely different
results. I wish I could hang this near my
desk and look at this advice every day. I think, and I
believe I read this in an article like Steve Jobs had an
intuition for great products. Great data, scientists have
intuition for data and I like you said it, it’s intuitive
intuition that builds over years of experience or hours of having
absolutely no clue of how things work. Yes, and initially to do
something, it takes you two hours, but then, you know, you
suddenly can do the thing in five minutes. And you know, and
it’s like a snowball effect, you know, where it just keeps
growing. Definitely. Now talking about
teaching basics. I’m not sure at one point, this happened, but
you’ve been sharing tutorials on Kaggle starter packs for the
community. How was this idea born? And was in this taking
time away from your learning journey? Why did you invest so
much time into it? Because it does take a lot of time. It does take a lot of time. And,
you know, like, fundamentally, I believe that we’re all in this
together. And like, I mean, people who are trying to learn
something man, or machine or people who do machine learning.
So if you learn something, it’s propably nice to see if you can
maybe help someone who’s on the same path as you are. So there’s
definitely that but as a source of motivation for my actions,
but that would definitely not be enough to warrant the amount of
time that it took to create all these materials. And the reason
I did it is that it was a natural extension from, you
know, from how we talked about learning the fast.ai. Basically,
I did it to learn, like each of these starter packs I created
because there was some concept I wanted to practice. It might
have been a new version of the library or a new technique. It’s
also awesome for learning how to write code clearly, how to
create those tutorial style notebooks, which is what I’m now
using at work, you know, I work remotely and I don’t see myself
ever working from an office ever again. And you have these
notebooks, you shared them with your colleagues and they can
understand what’s going on. You also learn good software
engineering because software engineering is about dealing
with complexity and making complexity become tractable,
something that you can put your hands on. And if you’re working
on a software project, you know, you want this clarity in your
code and you want this understanding of what is
happening where, so working on this definitely have has been
helpful. Now the last to be completely honest with with you
and with anyone who might ever listen to this so so another
aspect. Obviously, sharing things with people is a lot of
fun. There is this component where, you know, you share
something and somebody likes it and, you know, you get the
dopamine shock and are, you know, so that happens. But
that’s something that I try to be quite careful with. Because I
don’t want to be in my actions. I want to be strategic about
what I want to achieve. So I don’t want to be driven by the
fact that somebody I care about, you know, something you
retweeted what I dumped 10 hours of work into, or 20 or 40 hours.
That’s not what this is about. So this is something that I want
to limit and to not make this answer even longer. I will say
that very briefly that there’s also this whole concept of
personal branding and then awesome post by Rachel Thomas.
Not making peace with personal branding. So I haven’t made
peace with personal branding myself. But I can see how this
can be useful. So if you’d like to learn more, definitely go
read that post. You can find it in the show
notes. Please scroll down. Don’t be lazy. It’s an amazing write
up. Now, I know you’ve tweeted about this, but how did you
overcome the fear of being able to share your code publicly
about something that you just learned a few days ago? Radek: Yeah, this fear never
goes away. You know. And I think this is a good thing. This is
the just how things are. If you don’t experience that fear, you
might actually be an imposter. You know, so having the imposter
syndrome means that you’re probably doing something right.
And I think and now even starting to think that going in
the direction of what you fear and that is a very good way to
pick your route. You know, and because that means that you’re
doing something important, most likely, it can mean that. And
obviously still, for every tweet that I write, there are
probably, you know, I published maybe every second tweet, I’m
not sure. And especially when I press that tweet button, I’m
like, Oh my gosh, you know, everyone will think I’m a retard
and you know, what am I even saying? But what helps is
thinking, you know, what are the likely consequences of my
actions? Like, will I lose my livelihood? Will I lose my job
because of that? Well, probably not. In 100 years, everyone I
care about will no longer be a life anyhow, you know, in
several billion years something will happen to the universe. So
you know, like, look at the perspective and it will maybe
make it easier for you to press the press the tweet button with
with code I feel it’s simpler because yeah, I don’t feel that
strongly about showing my code but definitely sharing my
thoughts. I still struggle with that. And also the fear doesn’t
go away, but it becomes better over time. So, you know, just
you have to you have to you have to muster up the courage in the
face of the horrible to be the button and you know, whatever
hundred people that even read what you wrote so. Sanyam Bhutani: Awesome. Now
coming to something that you performed really well on Kaggle
How do you suggest someone should start working on a Kaggle
competition and at what point do you says is that they should
jump on to Kaggle? As an as an extension of doing
homeworks know, if you feel that you’re quite happy was covered
in the course and you’re looking for a new challenge, then pick
up Kaggle I would suggest that you start in a competition
early, ideally just after it launches. And you know, that
makes an enormous difference, because you will not, if you’re
just starting out, you will not anticipate how much progress you
can make if you work on something here regularly over
two or three months, which is how long these competitions
last. Then, each day, read the forums for the competition.
People share very useful things. It’s important to stay in sync
with what people are doing. Check out the kernels. You
should start with a notebook which does nothing but submits
to something submission. You know, if you get to this point,
that will mean that you already have a working environment set
up that you already you know, have your API key or you know,
your you can you know how to download the spreadsheet with
the results and you know how to upload them to Kaggle. And then
you build on top of the of that you establish a baseline, a
baseline can be just taking very simple, a very just or something
like that, you submit that you see whether your score changes.
And each day you try to if you don’t have the time to do the
work, then at least you you know, spend five minutes while
you’re taking the best work and think what you could choke and
improve. Okay, now, there’s also another
point when you’re in the heat of the competition and you’re stuck
at a point you can’t go up the leaderboard people are
overtaking you. How do you come up with new ideas at that point,
and how do you focus your efforts then? Radek: So when working on a
Kaggle competition you are actually building a software
product. It’s it’s not like training a single model, it’s
not one smart idea that can, you know, get you to the top of the
leaderboard, you’re building something that will reading in
the data will process the data will train some models. And then
will combine the models in some way up onto the submission. So
you the way it worked for me as I read the Kaggle forums, and
write ups for previous submissions that did well in
previous competitions, and I’ve built this mental map of what a
good solution to a Kaggle competition looks like. And
basically, you know, you look at the map and you see, you know,
maybe I should work on data processing I read there that
people are having good results if they use black and white
data, for example. So let me try that. Or maybe I’m only taking a
mean of my, of outputs of my models, maybe I should, you
know, train a model on that. The secret sauce to Kaggle, that
takes you from struggling to all the way up to silver medal ranch
and below and beyond is training with cross validation, and also
understanding these two things. So if you feel that, you know,
you’re at the point where you have a hard time improving
,study both these concepts, and you will find that it’s like an
elevator to the top of the leaderboard. You’ve done right.
There are many ways you can go wrong with that. But that would
be. Sanyam Bhutani: Okay. There’s
the other approach as well copy paste everything from the kernel
and just keep working on some link. Should we do that do
recommend that approach initially. Radek: I don’t know. If you’re
having fun or if you feel that you’re learning something, then
sure, you know, there are people who just make submissions and I
haven’t tried that. And also, I find the menu of the kernels to
be a little bit messy. So I know the code is hard to follow. So I
when I see some something interesting, I might try to run
the kernel, but then I open up my own notebook. And I tried to
follow in the footsteps of what the author did, refactoring the
code as I go. And that is very good for your performance on
Kaggle, because chances are you can improve on the approach of
the original creator of the kernel. And the act of
refactoring, actually, you can learn a lot to doing that. Sanyam Bhutani: Did you ever
finally feel comfortable with starting with a blank notebook?
At what point did you feel comfortable because that’s
another point that someone who’s very new to this struggles with
and that leads them to copying and starting from there. Yeah, I don’t think you ever
feel comfortable with that. You know, when you have a blank
notebook, there’s just so much work that you need to put in to,
you know, to get to some results. Ideally, I would look
up the fast.ai notebook and start there, or maybe I would go
back to some of my notebooks and use that as a starting point.
But if I were to honestly start with the blank notebook, I would
import what I need, which is Pandas, NumPy and fast.ai. And
from that point, I would focus on reading in the data, you
know, and once I am able to read the data, then while so much
work has already been done, you know, now just that quickly,
train a very basic model and submit and that would be a lot
of good work that has been accomplished. Okay. Now, another thing that
intimidates beginners I know it’s not a factor. Once you get
over it is hardware, people feel you need a huge chunk of
hardware to do deep learning competition. And I know your
thoughts are very interesting on this if you could share your
thoughts with the audience. Yes. If your thought is on
winning a Kaggle competition or on hardware, then it means that
you’re not doing Kaggle right, you know, like, competing is
fun, but it’s fun the same way as playing with shiny objects,
you know, it gives you the emotional energy to, to
continue. But the real wins of Kaggle are learning and getting
to know people. And in fact, you don’t need a lot of hardware for
that, or a lot of hardware can actually be hurtful to these
approaches. Yeah. Radek: Like, if you place a
constraint on yourself, you can create very interesting
notebooks, you can write very interesting things. Having said
that, there’s still a lot of challenges where you can do very
well without having extensive hardware on the tabular data
competitions which might require maybe a smaller GPU or my
require a lot of CPU cores, but CPU VMs you know, with 96, six
cores, super, super cheap, like, unbelievably affordable. Even in
an, even for computer vision competitions, you can be very
dangerous with a single GPU. And it is true that the people on
the top they will often be people who have been doing this
for a long time who have access to a lot of GPUs who have
scripts who trained a lot of models. But I don’t know the
only competition that I won. It was an unranked competition. But
still, it had some prize and I won it using a 1080 ti Singleton
at TI. So; Sanyam Bhutani: Which was
uncommon in the top five rankings for the audience. I’d
like to highlight that. It was a fashion competition, a
materialists fashion competition. And yeah, there was
some prize I think 1200 dollars. The competition was quite
serious, there were teams consisting of no professionals.
And but this is like this is fun to discuss, but this is really
not where the real value lies. So, I would absolutely not offer
over emphasize this point. And the biggest upside I got from
this competition is that I actually learned a lot because I
started in the competition early and I worked really hard on it.
And also, you know, interestingly, I got a
freelancing gig out of the competition. Okay. Radek: And you know that that
was quite a good freelancing gig. So, you know, you need to
take a step back and sort of figure out where the real value
lies. And the real value for most people doesn’t lie being in
the, you know, gold medal ranch, which is fun. Sanyam Bhutani: Yep. Radek: But in doing all these
other things, Sanyam Bhutani: I also want to
ask about your opinion. Now, that Kaggle is growing bigger
people start to paint Kaggle as not real data science, you don’t
learn a lot of things that are useful in the ideal world.
What’s your take on that? Radek: Yeah, I don’t know, I
feel it is extremely useful, you know, like the it’s a very good
approximation to what doing machine learning in real life
looks like, yeah. It’s extremely challenging. You get feedback
through the leaderboard. You learn about it. And about, you
know, the leakages, you can introduce yourself from your
validation set to your trainset, which is a very important skill.
And like, sure, there’s maybe five or 10% of what is useful to
know that you will not learn, like how to navigate an
organization or you know how to think about what data to get,
or, you know, whether you should, you should focus your
organization on getting more data or of doing doing something
else with the data that you have, you know, processing in
different way. This is something that you don’t learn on Kaggle
because you have the data presented to you. But I wouldn’t
discredit Kaggle at all, and I’m not very active on Kaggle right
now, but it’s a great learning environment. And I think many
people in academia would find Kaggle extremely challenging,
you know, so It’s sort of easy to discredit something if you
know you maybe have not tried it to the extent that you could. Sanyam Bhutani: Very true. Now
talking about another real world skill that’s very helpful,
blogging, you shared many great ideas and thoughts and dogs
again, audience, please scroll down to the show notes, check
out Radek’s medium profile. But can you tell us what efforts and
or what does your pipeline look like while writing a blog? And
how do you suggest someone should start with blogging ideas
even though they’re in the early stages of the journey? Radek: So the best advice,
again, comes from a member of the fast.ai group, team. Yes.
And that’s Rachel Thomas, again with her blog posts on why you
should start blogging. It’s something that I shared multiple
times. And the advice is very simple. There is something that
was challenging to you six months ago, three months ago,
you know, know how to do this right about that. It’s, this is
a great way to start. Now as you built up your blogging muscles,
you might consider blogging about something that you’re
learning. You know what, while you’re learning that thing, it’s
it’s amazing for, for structuring your thoughts for
going deeper on the subject. Sanyam Bhutani: Okay. Now uh
you’ve also; Radek: I just just just just
would like to add, can I add one thing. I just wanted to add that
writing seems to be extremely useful in the modern world. And
blogging allows you to learn how to be a more effective writer
and also the everything that comes with sharing your blogs.
So, you know, the experience of putting your work out there
become more accustomed to how it feels, then writing or doing
something in a way that is useful to other people, like
being useful and adding value is probably the most amazing
business skill that you could have. Then also making it
interesting in a way that other people want to listen to what
you’re saying, you know, that is also those are skills that allow
you to operate in the modern world. So if you can learn all
that by blogging, I think that’s, that’s an extreme win. Sanyam Bhutani: And you had
mentioned and we’ve discussed this on the forums as well.
These are not intuitive ideas, these isn’t these all of these
skills aren’t something you are born with. You have to feel
uncomfortable. You have to go out there and put something out
there, especially if, if you’re broadcasting something if you’re
creating videos or blog posts. Radek: Yes, yes, this is not
comfortable. This is not this is not easy for some people it
might be even harder than for others. And it’s not intuitive
like you said, like, that’s the value of the fast.ai course you
know that you have Jeremy Howard, and he tells you all
these things that a person living in which Poland, the
middle of nowhere, would never learn about otherwise, that they
are important, or you know how to structure a machine learning
project. Try learning that from a, you know, a machine learning
book, like good luck with that, you know, or that it actually
might be useful to be on Twitter, to read the thoughts of
other people who you know, maybe know way more about the field
than you do that’s already very useful. But to try to squish
your thoughts into those, whatever, 260 characters, and
that’s also another skill that is. And if you asked me five
years ago, does this sound like something useful to you or
something hard? I would say, No son, yeah, you’re crazy, you
know? Leave me alone. I’m trying to understand this mathematical
proof or, and working on like, back propagating through all the
layers of CNN. That’s what I would do back then. But no, this
is not the entire story. Sanyam Bhutani: To add to that,
and now that I’m officially out of university, I can say this to
someone who also goes to a university and studies of proper
computer sciences. These things weren’t taught there either.
Those courses are taught by a academics who haven’t ever done
a real data science job or a real data science project. Even
they have absolutely no idea even though someone might feel
intimidated. Okay, I’m sitting next to a computer science
major. No, you’re not you’re sitting next to someone who just
knows that theory. Radek: Yes, yes and no computer
science theory. Yes, it’s useful in very specific situations, you
know, on the job, like that’s putting it very nicely. You
know, I know that in the United States, in colleges the material
is even slightly different, slightly more practical than it
is in colleges, such as in Poland. I’m guessing it’s
probably similar to India. Sanyam Bhutani: Definitely. Radek: You know, so some of the
things people learn here when they go to college to study
computer science, you could learn some, like, for example,
in which not long ago, they were teaching ADA, which is computer
language from so many years ago. And it’s just because the
professor knew this and they are, what, 60-70 years old?
Yeah, but let’s not go to that. Because obviously, we love all
the people who study and learn. And you know, if you’re learning
computer science, I think you should feel good about yourself.
And, you know, so yeah, it’s just many paths in life. And you
might want to compliment your skill set with something that
feels different. Sanyam Bhutani: And if you’re
not, please don’t be intimidated by it. I can guarantee you. Now,
coming to working on tutorials, what was your motivation
continuing to do it, you don’t monetize, so to speak your
content, you don’t do it to break into the field Why do you
spend so much time away from doing projects on creating
courses, creating forums, creating blogs or video
tutorials? Radek: To learn, to continue
learning, that’s the that’s the answer to continue learning and
you know of course right now that I have a full time job as a
machine learning person, I can give maybe less time to these
activities but I, I still feel that continuing to learn and
doing something that can be useful for other people. Those
are two things that I want to continue working towards in one
way or another. And writing tutorials is super useful way of
going about this, it’s a super good way. Sanyam Bhutani: Awesome. Now I
want to touch upon another topic. You have a wonderful
family that you’ve shared with us on the fast.ai family forums
as well. How do you balance this work about the community, your
full time work and your family time, and if you have any
suggestions? Radek: Yes, absolutely. So if
I’m reading your question right, basically, because we couldn’t
go into discussing how having kids actually allows you,
teaches you so many things that are useful in many contexts, we
could discuss that as well. But I think that your question deals
with having a constraint on your time. How do you deal with that,
right? Sanyam Bhutani: Yes. Radek: So you know, many things
become better because there is a constraint and a great piece of
software, the Ruby on Rails library, was written initially
on 10 hours a week. You know, I’m sure that if David
Heinemeier Hansson, the creator of the library was able to spend
more time on it, it would be different, substantially
different, probably not as good. So, or not as good in the ways
that I care about. But specifically with learning, if
you have a constraint on your time, you first start to
appreciate your time so you know, your time becomes
important. And that’s already such an enormous mental shift.
And secondly, when you also try to come up with ways how to use
your time more most effectively and figure out what the
priorities are. So there are a couple of things that can be
very useful here. First of all, working large inter
uninterrupted blocks of time, an hour and a half, maybe two
hours, where you have no distractions, you know, you just
focus on the work that you’re doing be that learning or
programming. Programming is so complex, you need so much
information in your head that it takes a while for you to load
all the important context in your mind to actually be doing
something useful. So if you give 30 minutes to working on
something, and then another 30 minutes later on, you will be
much less effective than if you were to spend an hour straight.
And there can be no distractions. So you cannot, you
know, look at Twitter, you cannot look at Reddit while
you’re doing that. Because what happens then is all the
information that you put in your mind, you know, it goes away and
you have to put in work and it really feels like hard work.
That’s where you get tired, to load up all the information. So,
you know, I’m 100% convinced that an hour and a half or two
hours of uninterrupted time working on something can easily
you can deliver more value than a person who’s sitting at an
office and an open space floor, you know, trying to program when
where they have all these distractions, people asking them
to go grab a coffee with, you know, with, and you know,
meetings and things like this. So that’s, that’s one thing and
it’s super useful. Second thing. You don’t become more productive
by doing more things. That’s a competing misconception. You
don’t become more effective by chasing more priorities. You
become more productive by focusing on one or two things at
most, you know? And so so the trick is not what do you put
into your schedule will do take out of your schedule. For me one
such thing for me one such thing that has been taking a lot of my
attention and emotional energy that was Facebook, you know, I
stopped using Facebook and right now when I listen to people talk
about Facebook, it’s like, you know, I don’t know even what
they’re referring to because the the idea that they can keep you
connected, the Facebook keeps you connected with people, while
it’s very shallow connection and it’s in a sense, you know, you
stay connected to the people you maybe don’t need to stay
connected to anyhow. And the advantage are so tiny as
compared to, you know, going through all the emotional drama
and being served targeted at that. To me, this was a super
good decision. But maybe there are other things in your life
that you might want to remove from your schedule, you know, so
that’s the that’s the second thing. Now, what else what else?
These are definitely the big things. Sanyam Bhutani: You also call
this I remember in one of your blog was the curse of control
plus your command list if you’re from a fancy laptop, but the
Command T in a browser is very easy to be tempted into you open
a new tab, you see your bookmarks, and you’re lost in
time. Radek: Yes, you’re lost in time
because all these multi billion companies they spend a lot of
money and a lot of engineering talent to make it appealing to
you emotionally to hook you on all these products. And you have
to be aware of that. And if you play this game where it’s
impossible because you only have this much attention in a day. So
remove the distractions. Figure out what you wants to give your
attention to. Sanyam Bhutani: Awesome. Now
this has been an interview full of amazing advises and very
transparent to use. Can you give one final best piece of advice
to future fast.ai family members future students? Radek: Yes. Okay. Well, first of
all, congrats to you if you found fast.ai, you’ve already
found an amazing place on the internet. So you must be doing
something right. There’s a lot of snake oil salesman in the
internet and the education on the offerings are actually other
crop, you know, the fast.ai is different, So congrats to you.
And the best piece of advice that I can give you is I think
that enjoy yourself, you know, have fun. And the reason is that
I feel that this is a good advice is that obviously life is
short, so it makes sense to enjoy oneself anyhow. But also,
it’s a key ingredient to doing something for a long time. And
that’s only how you really get to that different things once
you’ve been doing something for not for six months, but maybe
for two years for three years for four years and longer. Sanyam Bhutani: Okay, now,
before we end the call, I’ll have your profiles linked in the
description. Can you mention the platforms, I’ll have Twitter
You’re medium link, any any other platform that you’d like
to have linked in the show notes? Radek: I’m mostly an active on
Twitter by active I mean, not active, like some people where
they tweet a lot. But if I am to, you know, say about what I’m
working on it with you on Twitter and fast.ai forums. So
definitely, you know, you can look me up there and I look
forward to seeing you on the forums. Sanyam Bhutani: Awesome. Thank
you so much for all of your contributions to Kaggle, the
fast.ai community and thank you so much for joining me on the
interview series. Radek: Sanyam Thank you. Thank
you so much. It has been an absolute pleasure being here.
And yeah, my first college podcast or interview with wasn’t
my first, okay, my my my first both podcast an interview so
Sanyam thank you so much. And yeah, see you around. Sanyam Bhutani: Thank you so
much for listening to this episode. If you enjoyed the
show, please be sure to give it a review or feel free to shoot
me a message. You can find all of the social media links in the
description. If you like the show, please subscribe and tune
in each week to “Chai Time Data Science”.

2 thoughts on “Fast.ai, Learning to Learn | Machine Learning, Kaggle & Blogging | Interview with Radek Osmulski”

Leave a Reply

Your email address will not be published. Required fields are marked *