Kart Racing Game with Machine Learning in Unity! (Tutorial)

In this video, we’ll be looking at using
Unity’s MLAgents with a Kart Racing Game. MLAgents, or the Unity Machine Learning Agents
Toolkit, is an open-source Unity plugin that enables games and simulations to serve as
environments for training intelligent agents. Agents can be trained using reinforcement
learning, imitation learning, neuroevolution, or other machine learning methods through
a simple-to-use Python API. In this video, we’ll be exploring a simple
integration of the MLAgents Toolkit to the Kart Racer microgame. The sample project includes an example scene,
the MLAgents package, the KartAgent implementation, a pre-trained brain, and a basic racetrack.
These assets are all available for free download at a link in the description below. We have provided an example KartLearningBrain
trained using the default Reinforcement Learning model, that has been trained for a few hours.
When we enter Play Mode, we can see the AI attempt to navigate the track. Let’s take a look at how the KartLearningBrain
is controlling the Kart. We have added a custom KartAgent component
that acts as a controller, and linked it to the Input field of the KartMovement component.
This replaces the default KeyboardInput component and allows our KartAgent to control the Kart
directly. The KartAgent component inherits from the
base Agent class. These top parameters of the KartAgent component
are inherited from the base Agent class. They relate to the input of the Agent, and how
it is trained. If you’d like to learn more, visit the MLAgents GitHub repository for more
information on these parameters. The bottom parameters are custom fields we
have made for this specific project. Our KartAgent uses 9 different raycasts to
“see” the world. Each raycast is defined by a Transform in our Raycasts Array. The
KartAgent uses these raycasts as input into the brain. At every step, the KartAgent will
see how far each Raycast travels before hitting a wall, and then feed that number into the
brain. The KartAgent’s brain will then process the raycasts through its neural net and output
the acceleration and steering values for the Kart. The brain used by the KartAgent is defined
by the Behavior Parameters. The Behavior Parameters define which brain model to use, and how it
functions. Our model is set to the KartLearningBrain. You can have multiple different brains for
a MLAgent. In your game, you may want to have different brains for different difficulty
levels. There are many ways to train a neural net
brain. We have provided a sample KartAgent brain, that has been trained in a basic Reinforcement
Learning environment. Reinforcement Learning works on a simple principle.
The brain will learn by continuously changing its behavior, trying different things to maximize
its rewards, and to minimize its punishments. We have set the Max Step to 2000, which will
end the training episode after a 2000 training steps. We have set the Kart to be rewarded in two
different ways: Reward the agent a very small amount while
it is moving forward. Reward the agent a large amount when it reaches
a checkpoint. The KartAgent is only punished in one way:
Reset the agent if it touches a wall. To reward the agent while it’s moving forward,
we used the AddReward() function inside of KartAgent, passing in Kart.LocalSpeed, and
multiplying it by a small number. To reward the agent with checkpoints, we have
placed checkpoints all around the track. We have created a ReachedCorrectCheckpoint
event on the TrackManager and subscribed the KartAgent.OnReachCheckpoint function. We have set up multiple KartAgents that use
these rules to run in parallel. Having multiple agents can speed up training time. Each KartAgent
contributes its experiences to the same neural net, while trying to reach as many checkpoints
as it can. If we try out the pre-trained brain, we will
see the karts move along the track, detecting walls and turning using its raycasts. We have trained the brain for several hours,
and it can navigate the track, but it can still use some improvement.
We encourage you to try training the KartAgent yourself, and to experiment with the reward
structure. We can train our KartAgent through the KartAcademy.
The KartAcademy script inherits from the base MLAgent Academy class. It determines what
parameters to run Unity in while in training mode and while in normal playback inference
mode. We have set our Time Scale to a higher number,
allowing us to run the game faster than normal and therefore speeding up training. Our KartAcademy will connect to the Python
MLAgent API, feeding it the raycast sensors we created. You will need the MLAgent Python environment
installed to conduct training. With the MLAgent environment installed, you
can monitor the status of the Agent using Tensorboard during or after training. For information on how to set up the MLAgent
Python environment for your platform, visit the MLAgents GitHub repository, which is linked
below.. https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md We hope that this project can help you get
started with Unity’s MLAgents Toolkit, to help you create machine learning AI for your
own project. We encourage you to download the free sample project, and try training
the Kart yourself. Thanks for watching!

85 thoughts on “Kart Racing Game with Machine Learning in Unity! (Tutorial)”

  1. Fantastic! Looking forward to exploring this further! It's great that this sophisticated gaming engine is so accessibly to the average person.

  2. I am doing the Udacity RL course and have played with UnityML. Can you tell more about neural evolution? The last time I checked, the main focus was on Proxy Policy Optimalisation.

  3. Good video, don't get me wrong, but I think it's naive try to teach somebody how to use ml-agents in 5 mins…I think ml-agents are a good approach to Machine Learning, but there are FAAAAR away to be used for users, it is a GIANT black box.

  4. I got this question from one of my uncle who argues that Machine Learning is just an algorithm like any and shouldnt be called an AI. Which upon further argument seemed right. What do you guys think? Also, where can I practically use it in video games ?

  5. Correct me if I'm wrong, but in the example shown here the car is being trained specifically for this track with no other opponent than itself.

    Then how will it handle collision with player when it's trained? (Since player won't be a static obstacle)

    Also, if I want to train my AI for any two positions in a single map, how should I approach this?

  6. what are the system requirements? Will a GPU be necessary for fast-learning? How much time will the agents take to learn if trained on intel i5-8th gen.

  7. Any easy tutorial to understand the overall concept? Exactly what you are trying to mention, any live training session exists on this?

  8. Lol, I managed to do this a few months back, but with genetic evolution.

    This is really cool, and seems this is a start for easy implementation of machine learning agents in games!

  9. but on the script programming i dont understand some of the matter.
    Please help me. I am now doing my tutorial about 3 days. What will help me and tell me when i will able to make a games?

  10. i hate to say that but i cant see the best tutorial on this channel. please upload for beginner to understand than other channel

  11. Hi everyone, Im Ivan, a student and im also a new young game developer.

    I've made a simple, fun and authentic game on google play store, it is named Flapping Pooper.

    I would really appreciate if you could go check it out and share it with your friends and family.. All of you here, thank you!:)

  12. whoa. I started trying to make something similar a while ago from scratch. Never got to far though. cool to see an official unity tutorial on it now.

  13. I dont have unity but want to start using it, when making a game, is there already ways to make things like sprites and characters?

  14. add hdr srbg high nits 2k+, and next gen detail above ray trace and maximum resolution, and improved render system for cross platform

Leave a Reply

Your email address will not be published. Required fields are marked *