A Beginner's Guide to Machine Learning with ml5.js

hello welcome to a new playlist of videos this is a playlist I mean you know hopefully there's a bunch of videos in this playlist because I'm just starting to record them now so they don't exist yet but this is about machine learning and it is for the total beginner now there's some caveats here as opposed to that but I really wanted to make something you don't have to have watched any other videos about machine learning on my channel I have others that go more in depth and use different tools this is really meant if you're starting it's totally new to you you heard you've heard the words machine learning you've typed them into some search bar and somehow you landed here watching from the future leaning back you're you've taken your brain out for its afternoon defying and you're gelatinous couch is comfortably supporting your skillet skeletal anyway the point is you're here I'm here hello so I'm going to be using for the set of tutorials a pretty new JavaScript library called ml 5 the purpose of this first video in the series is really just to talk about two things what is machine learning what is ml 5 so the one caveat I would say possibly to being a beginner here is that you probably want to have some knowledge and experience with coding itself now that isn't to say that you aren't if you don't you're welcome here please watch I will point you to other videos that can help you fill in some missing gaps but I'm going to assume a little bit of knowledge about programming particularly in JavaScript and I'm also going to be making fairly heavy use of another JavaScript library called p5 so if p5 is new to you I'll refer you to some other p5 tutorials I will but that's going to be a larger large piece of how I build various examples throughout all these videos ok so before I start talking to you about what is ml 5 which is really what I'm here to do let's just ask ourselves a question you know first what's called ml 5 ml for machine learning what is machine learning now there are lots of terminology out there machine learning deep learning artificial intelligence neural networks the list goes on and on and hopefully the definition that whatever I say to you right now doesn't really matter because we're going to sort of explore this topic of machine learning and all these other other all this other terminology throughout these videos but I think one thing to realize about machine learning you think of this idea of artificial intelligence computers robots taking over the world and they've learned their own language and they're defeating our but you know really when it comes down to it brass tacks so to speak machine learning is examining some data a whole lot of numbers and try to make sense of that data and teaching a computer basically to learn patterns that we can present that's really a and we're not talking about something called supervised learning Europe or even more terminology supervised learning where I'm gonna get I'm gonna get into all that and I've gotten into some of that in other videos so I just pulled up some notes that I made for a course about artificial intelligence and and and at NYU here and this is sort of the classic definition of machine learning from 1959 Arthur Samuels a field of study that gives computers the ability to learn without being explicitly programmed and so let's think about that for a second this is actually useful and I'm gonna get into ml5 in a second but just just a little bit longer right I might say something like the most simplistic sense I could write a program that has a specific algorithm that says if Mouse X is greater than so let's say this this window has a width between zero and four hundred if Mouse X is greater than 200 then that is the right side can you see this yes greater than 200 is the right side else if Mouse X is less than 200 or just else that's the left side this is me providing the computer so to speak I've written a program with some intelligence it knows how to tell if a point is on the right side or the left side of the window I have written the algorithm for that well machine turning is about teaching a system to understand a data without explicitly writing your own algorithm this might be the most trivial simplistic algorithm ever but the way I would do machine learning is I would say hey machine learning system I'm going to give you lots of information like 350 is on the right side and 301 is on the right side and 212 is on the right side and 273 is on the right side and 100 is on the left side and 97 is on the left side and 24 is on the left side and 198 is on the left side I'm creating a data set and I'm just gonna say here's a lot of stuff I'm not gonna tell you how to don't I'm not gonna give you some specific hard coded algorithm to know what's on the right or the left I'm just going to create a data set it's going to have a lot of information in it and the machine is going to learn the patterns in that data learn how to make some kind of prediction that's a key word here the prediction being is this point on the right or left based on that data this is the essence of machine learning teaching a computer to perform a task without explicitly programming the algorithm for that task now there's a little convoluted cuz aren't we programming the machine learning algorithm and a sort of program but this is the conceptual idea okay so now that we as best as I could made some definition created a definition of machine learning one that the least we can start with and I'm sure I'll hear from you in the comments let's talk about what is ml5 so before I talk about what is ml5 I want to just mention someone to you red burns who is the founder of the program where I currently teach called ITP red burns is a huge was a huge source of inspiration she was a mentor to me and I wanted to start say I wanted to read a few quotes and we actually have a whole book of red burns quotes but these I think these quotes are really important and they are the kind of driving principles behind a lot of the work that I do but in particular a lot of the behind the ml5 project so technology is not enough consider the technology I should really move the mouse technology is not enough consider the technology as a tool which in itself could do nothing treat the technology as something that everyone on the team could learn understand and explore freely so this is really important don't believe the hype that expectant people say but it's easy to get caught up in the shining technology stuff but we have to remember that without human beings here on this planet what's the point so while I'm really interested and I get lost in my like interest in just sort of numbers and algorithms and patterns and things like that that's I think a core value that I want to make sure I stay grounded to throughout any time that I'm looking at algorithms and thinking about how to how to use technology how to teach technology how to make use of it so this is actually from an article from the ITP journal adjacent there will be a link to this in this video's description which describes sort of the background about the Machine ml5 project itself at the end of this video I want to talk about various people who made contributions which I have here on this link so I will do that but but but let's talk about what is ml5 so in order to kind of figure out what ml5 is I'm going to try to make some diagrams this is gonna be great I can feel it right now so I'm gonna put ml 5 right here in the center now an important component of this discussion is a library called p5.js another really important component of this discussion I'm gonna put it up here is a library called tensorflow da chasse and oh boy oh boy another really important part of this discussion is just the library tensorflow itself as well as maybe sort of this thing called Kerris which i might refer to so first of all what is ml 5 j s ml 5 GS itself is just a JavaScript library it is a library of functions that you can call from your own JavaScript code so again I mentioned that I'm gonna rely heavily on p5 p5 is also just a JavaScript library p5 is a little bit of a special JavaScript library make a little heart there couple stars because it is really I it doesn't necessarily conform to the standard patterns you might find in other JavaScript libraries it makes a lot of decisions to be as friendly and welcoming and well-suited for a beginner as it can be so one of these you could do in regular in p5 is you can write a function called set up and any code that you put in set up executes the moment the program starts you can write a function called draw and any code that you put in draw happens in a loop over and over again this is this a standard kind of design pattern for writing a program that generates an animation like a game or a piece of interactive art so what I'm saying here is that my examples that I'm going to show you in these video tutorials are using p5 Jess almost s like it did the base starting point and then I am also going to import the ml 5 library so I need to import both of these libraries and I'll show you how to do that in the in the HTML file which is something you'll see in the code examples and once I have the ml 5 library I can start to do things like this ml 5 image classifier and then so this is a function setup is something that sort of specific to p5 draw is something specific to p5 and then image classifier well we could know it's specific to ml 5 because I had to write ml 5 dot image classifier so this is one thing a little bit attached which is no surprise if you watched my videos before but a lot of JavaScript libraries are namespace meaning if I'm going to call a function like the image classification function I've got to say that name the library dot that function name so image classification this is one of the common tasks of common machine learning tasks right I'm gonna say hey machine learning here are some images of kitty-cats here are some images of puppy dogs learn about them so and we're gonna see how you could do that exact kind of task with ml5 itself okay so this is the basic idea now why did why am I talking about this so here's the thing if you just were wandering around the streets of I don't know Venice Italy I thought of that there are damn streets they know it's all like oh my god that's crazy you're you're paddling through the canals of Venice Italy and you go to those little canoe sounds I got no the boat next to you you say hey do you know how to do machine learning I'm probably oh you just go use the thing called tensor flow now this is possible they say something like oh you just go use this other thing called like pite or something else but there are many many examples of libraries most open-source libraries that have a lot of functionality for common machine learning tasks one of the most well-known ones of those is called tensor flow this is a project from Google it is open source and most of the time if you were using it you would be writing code in the Python programming language well in my tutorials I'll be writing code in the JavaScript programming languages programming language not in Python well amazingly there happens to be this other thing called tensorflow das also developed by Google researchers I will include a lot of information about where how you can find out more information about Tesla s than the creative potential are addressed in the video description but this is a version of tensor flow written in JavaScript down truth tensorflow it's really run plus good so even when you write your code in Python it's just like running other stuff that's C++ but that's not the case with tetra fluro chassé tensorflow digest is everything's happening in JavaScript and ml5 in the end and you could go and just use tensorflow digest directly to do your machine learning stuff and I have some video tutorials that talk about that but ml5 is a project that actually wrap that is a layer it includes tensorflow digest so nl5 is a library where additional functions were added on top of tensorflow j/s to make certain tasks easier and in truth this is a low-level library meaning if you were to use it directly you have to really worry about things like memory management and uh a and its kind of stuff you know it's interesting you can do it but if you're just getting started and want to learn the basics of machine learning and experimented play this is going to be in your way it's gonna make things take longer and so ml5 is going is al its goal is to make a lot of the things that you're doing tensorflow Jess a bit easier and more accessible okay so here's the other thing that I really want to discuss there are three I would say I'm gonna give you I'm gonna talk about three components of the field of machine learning right now there are particular machine learning algorithms one example of that is something that you might have heard of called a neural network this is an algorithm to perform a machine learning task there are many other examples of algorithms there are also there's something known as machine learning models and in particular there's this idea of a pre trained model okay so I want to talk about that in a second and then of course we have this idea of data and data sets so I'm going to circle this here for a moment I'm going to put my stars next to it because this is essentially where we begin you and me and the people from the future who are watching this video or there have time-traveled into the path I don't know whatever this is where we're gonna begin now most machine a lot of machine learning tutorials we'll start with something like AI now we will look at linear regression it's kind of similar to that thing where I was like I had that line in the middle of the window are the points on the right or the left and we're gonna do linear regression and statistical components of machine learning algorithms but one of the things that is happening in the world today the reason why I'm hoping that this video tutorial and this ml5 library is important and useful is that our lives are increasingly being governed by pre train models corporations train models to perform a particular task to recognize your face to turn the words you're speaking into text there are questionable as many questionably ethical applications of this there are also fun artistic and playful applications of this so one of the ways I think to get started as a beginner to machine learning is actually not to learn all the math and learn how to train all your own machine learning systems but to actually make use of some pre trained models that are out there for example the thing that I'll start with in the next video is a model called mobile net this is a model that's been trained to recognize the content of images but there are lots of images that I could show to that model that it will have no knowledge about and other ones that will have wierdly have a incredibly obscure detailed knowledge about why is that well what was the data that was used to train that model what was the algorithm what algorithm is that model using these are the critical important questions that we of citizens of this world increasingly governed and controlled by software need to feel confident and capable of asking and so by taking the get by friendly and accessible or approachable ways of being able to experiment with pre-trained models this is a good place to start alright so machine learning is a thing the thing people are talking about we want to get some experience playing around a bit experimenting with it understanding what it is we're going to use this ml5 library I'm going to show you how to import it I'm going to show you how to import it into a p5.js sketch and I'm going to show you how to start to use pre trained models and we're gonna look at what those models do well what they don't do well where do they come from and why do they exist okay so actually before I go let's look at a couple more things all right so you know I would encourage you actually before you watch the next video if you're continuing to watch these videos to just check out the end map go to ml5 jazz org I don't encourage you to click on there's lots of stuff to read about here but I encourage you to click on examples and go down and just try all of these so you can see there's some pre-trained models that I'm gonna be using I'm gonna do a tutorial basically about each one of these notably like I mentioned one called pose net which actually can take an image and estimate your pose meaning where your hands where your shoulders where your eyes you know where your hips I might be getting that somewhat wrong there but that's a pre trade model just like a pre trade model learns how to recognize the content of an image here's a pre trained model that looks I'd like if I go out where the human body is in a particular image so we're be looking at those I encourage you to like sort of click around here I also want to just quickly go to this Twitter thread that I posted just to kind of give a little bit of a background and pretty good credit because MFI jess is not a life that I created it's a it's a project certainly that I have worked on but it came from many many other people so first thing to mention is and I did talk about this that ml 5gs is a layer on top of tensorflow gotcha so all of the heavy lifting behind the scenes really comes from this library which was a prod many people are working on in contributing this library at Google and otherwise but the project was initiated by Nikhil and Daniel researchers at Google the development was funded by with the support of the Google Pig mixture group the Google big picture group and the people plus AI research initiative which is led by Martin Wattenberg and Fernanda Viegas and they've done a lot of wonderful projects about visualization and machine learning the style and approach of our library really comes from the processing foundation I mentioned the relationship I guess I meant to say like the reason why the five is here is because of the sort of symbiotic relationship with p5 we've made sure ml five library you absolutely do not need to use p5 with it but it is fully compatible with p5 and the spirit of p5 I think in in its welcoming and inclusive a sensibility is a present in the ml 5 library and so thank you to p5 we've used the five fair ok Cristobal Valenzuela Valenzuela who is also the creator of runway I'm out which you should definitely check out started this project at ITP by creating a few examples with the deep learn jazz library which is what tensorflow which is what became tensorflow DJ yes Yanina has done amazing work on porting style transfer meaning she and and picks two picks models into ml 5 this is really wonderful Pikachu edge-to-edge Pikachu I unique hopefully will bow to me making her own videos that will point you to all have her and as a guest to go over some of these amana Davis has done a lot of work around data collection and responsible data collection that encourage you to read about and find on the ml5 website through an artist residency nikita Huggins a resident research at ITP really did a ton of work a project management work vital important work in shepherding the library through many different stages the ml5 logo on the website is designed by Alejandro hate hey Eun Kwang also gave lots of support to that Dan Ovid actually ported the PostNet model to tension flow touch Jessen ml v JS i'll be showing this in some video tutorials in the future and the a J's ITP adjacent journal which I did mention also thank you to Stephanie Colton who did all the editing for that article and anchored if you really want to read more about the background instead of listen to me ramble you can check that out we had a lot of guest artists who came to visit us and give us a lots of inspiration and ideas throughout the process and then so many organizations allowed us to do some of the first experimental workshops school for poetic computation Eyebeam creative coding fest ITP camp and yeah and there have been many other contributors that you can find if you go to ml5 j s dot org and you click on code and browse the github repositories and see all the wonderful people have contributed so this has been my introduction to ml 5 the library itself and guess what you made it through this video congratulations because in all the rest of the videos I'm really just going to be coding and talking you through how to make certain examples how they work and what the pieces of those examples are so thanks for watching this first video and I'll see you in the next one which the first thing I'm going to do is just make a simple image classification example using the mobile net model see there maybe

38 thoughts on “A Beginner's Guide to Machine Learning with ml5.js”

  1. Thank you for this wonderful library. I am a web developer and always wanted to learn ML using javascript as hobby. this library here, answers my question. and I am also grateful to Daniel for this wonderful series.. I want more and more tutorials in this channel.

  2. Here is the link to the twitter thread he mentions: https://twitter.com/shiffman/status/1008082575016054784

  3. Please check out my game i used PoseNet to create breakout game learnt from you, how do i start game once posenet is fully loaded and is tracking my face?


  4. hey dan, i was researching about pose estimation with tensorflow, and found this paper -> http://gvv.mpi-inf.mpg.de/projects/VNect/ do u think it would be possible to somehow implement that with tensorflow.js ? I've found the code for the paper also -> https://github.com/timctho/VNect-tensorflow

    greetings and lots of love! -jurgis

  5. " but the machine learning is algorithm "
    I will wait for that day to know why everybody ignores the reality.

    >< really why? .. I must work smarter to get it.

  6. If you want to implement ml5js with React.js, check this article:

  7. dude i am your greatest fan right now, you have no idea how much you made me love math , physics, and code at the same time, you are a great teacher, one day when you come to South Africa for no reason at all(which will probably not happen) hala at me man, i want a signature 🙂

  8. Hi dan, can you make a video on how to read machine learning research papers (like how to understand then maths and terminology) and then show how to apply those findings in ml5js? That would be interesting and would help alot

Leave a Reply

Your email address will not be published. Required fields are marked *