RAIN MAN 2.0, Blackjack AI – Part 1 – Counting Cards Using Machine Learning and Python

I'm you paying attention yeah you watching that yeah you seen that right you get you know ya falling off yeah well okay now what what do I have left two checks one eighty one King one six two aces one ten one nine one five one five you are beautiful man you just watched a clip from Rain Man an Academy Award winning film or Dustin Hoffman stars as Anytus –tx Avant who among other things can instantaneously count his way through a deck of cards his brother exploits his talent to bring down a Vegas casino in one of the most classic scenes in movie history Rain Man is one of my top 10 favorite movies and it inspired me to create a card counting blackjack playing AI that I call Rain Man 2.0 Rain Man 2.0 currently consists of a camera and a Python script that runs on my PC s cards are dealt in front of his camera they're passed into a machine learning neural network that identifies and counts each card Rain Man only needs to see the rank and suit information in the corner of the card to identify it the cards can be identified even when they're overlapping or when they're against a noisy background Rain Man 2.0 can count his way through a full deck of cards he keeps track of what's come out of the deck so I know exactly what's left when a few enough cards remain I can tell you what they are without even looking at them just like Rain Man can in the movie ready let's try it five eight jack two three there we go it works so you might be wondering when am I gonna take this thing to a casino well there's a couple caches stay tuned to the end of the video or I'll explain its applications and how I plan to use them first though I'll go over the basics of card counting and then talk about the object detection model and Python script that make this AI work card counting is a strategy employed by blackjack players to gain a slight advantage over the casino here's how it works at the beginning of a new deck of blackjack the count starts at zero as cards are dealt out the player adds or subtracts from the running count depending on the value of the card if the card is a 2 through a 6 the count is increased by 1 if the card is a 10 face card or an ace the count is decreased by 1 if the card is a 7 through 9 the count doesn't change right now we have plus 1 minus 1 and 0 so the running count is at 0 let's finish dealing and count the rest of the cards it's another minus 1 0 minus 1 and plus 1 brings us to a running total of minus 1 that's all there is to it when the count is high it means that are significantly more 10 value cards and aces remaining in the deck than there are low cards 10 cards and aces are more valuable to the player than they are to the dealer at high counts the player can increase their bet since they know they're more likely to win they can also change their betting strategy and do things like splitting tends to take advantage of the extra high cards in the deck boom boom that's a winner counting cards increases the players overall expected return on their money from negative 0.6% to as much as 1.3 percent this means they can expect to win money in the long term effectively beating the casino for a brief introduction to card counting and its impact on house edge checkout wizard of odds calm which I've linked in the video description below Rain Man 2.0 is programmed to implement the same plus one minus one counting strategy also known as the high-low strategy he identifies high cards middle cards and low cards as they come out of the deck to see the cards Rain Man uses Yolo a machine learning object detection model that's trained to identify each card by looking at the rank and suit symbol in the corner to train an object detection model hundreds or thousands of images have to be provided to the training algorithm so it can learn what the objects look like for my previous card detection models I had manually taken hundreds of pictures of playing cards and then painstakingly labeled each one of them it's a very time-consuming process since you need to use thousands and thousands of images to Trainer a bust model manually taking pictures and labeling them takes too much time to be a viable option instead for this model I use synthetic image generation to create fifty thousand pictures of playing cards for training to create these images I first captured video of each of the 52 cards in a variety of lighting conditions then I used OpenCV to extract hundreds of isolated images of the cards from each video I used a script to overlay these extracted images on random backgrounds in random orientations I set the scripts to create 50,000 images for training the script automatically labels the card corners and each image and creates XML files to store the bounding box data it's awesome and it saves me a ton of work I owe a huge shout-out to a youtuber whose name I have no idea how to pronounce but I'm gonna try anyway Jax Jax made an excellent video and github repository showing how to create these synthetic images his video is linked in the description below if you'd like to try yourself also if you want to see some amazing computer vision projects check out his other videos once I generated the 50,000 images I used them to train a yellow v3 model for card detection the model was trained using the darknet framework and training took about 8 hours to complete using the train to yellow model Rainman can easily identify cards that are dealt in front of him yellow is a lightweight but accurate detection model that works very well for this application the model is fast robust and can identify cards in a variety of conditions it runs quicker and is more accurate than my old faster our CNN model even when the old model was trained with the same 50,000 images on tensorflow here's a side-by-side comparison of the yellow model versus the faster our CNN model I wrote a Python program to implement the yellow model into a real time card detection application the program continuously passes each frame from the camera into the Yola model to detect all cards in the frame it actually detects both corners of the cards but I fill through the results out to only show the upper corner detection when a card has been detected for two consecutive frames it gets counted I created a user interface to show the running count and display which cards are still remaining in the deck when a card is counted the program updates the count and draws an X over the counted card so what are the applications for this card counting AI you know I would love to take it to a casino and count cards and win tons of money but there's a few limitations one I can't walk in and sit down at a blackjack table with a webcam strapped to my head most casinos won't even let you have a cell phone at the table let alone any sort of camera I could use a hidden camera but most hidden cameras have a low definition resolution that doesn't provide a clear enough picture of the cards the cards would be blurry small and viewed at an angled perspective the detection model would have to be extremely robust to identify cards in these conditions it's doable but it would take a lot of work second I'm currently running Rain Man on my high powered gaming PC it's a little too bulky to take into the casino alright I'm ready let's play some blackjack I'll need to run it on a more portable platform if I want to take it inside a casino third Rain Man can currently only count through one deck of cards most casinos use eight deck shoes at their blackjack tables my script needs a few tweaks to make it work with multiple decks this is my next goal for the project for now I use Rain Man to help me practice counting cards I point him at the table and play through a few hands of blackjack keeping count of the cards in my head once I'm more than halfway through the deck I pause and check the count reported by Rain Man to see if it matches the count in my head alright counts at three ready yes we got it with a few changes he could be used to help players make informed decisions in other card games like Texas Hold'em hmm or cribbage hmm Rain Man 2.0 could also be used to help visually impaired people play card games the player could show their hand to Rain Man and then half the cards read back to them alright Rain Man tell me what cards I have you have a five of hearts nine of Hearts six of clubs Jack of Clubs and a King of Diamonds this functionality could be implemented in a smartphone app giving visually impaired users a convenient and easy way to see playing cards when Braille cards aren't available ultimately I plan to turn Rain Man 2.0 into a full-fledged blackjack player an AI that can sit at the table have cards dealt in front of him and make hitter stand decisions based on the cards in his hand he'll implement basic strategy and adjust his play based on the running count making him the perfect blackjack player my plan is to run Rain Man on a Raspberry Pi so he'll be small and portable why am i doing this mostly I just want a friend I can play blackjack with maybe someday I'll be able to take him to the casino and bring down the house this video is the first part in a series of videos about Rain Man 2.0 I'll post additional videos as I continue to build out his functionality the project will involve lightweight machine learning computer vision algorithms Python programming and a whole lot of blackjack if this sounds interesting to you please stay tuned for future videos thanks for watching

10 thoughts on “RAIN MAN 2.0, Blackjack AI – Part 1 – Counting Cards Using Machine Learning and Python”

  1. It seems like eventually this will also be able to deviate from basic strategy and even the card counting published deviations because the data will be more granular than simple high lo count. Very cool video.

  2. Ah ah I was wondering why I had a sudden spike in the count of my subscribers :-)) Thanks for the big shout out, bro ! Your project is really great, I am already impatient to watch the following parts and curious about the port on the raspberry ! Your video editing is excellent and you seem to have fun doing it (I know how time consuming this task is 😉

    FYI, since the post of my video one year ago, I had the opportunity to test the following: take only one good picture of each card (instead of a video under varying lighting conditions) and rely on the image augmentation library to simulate the lighting. That works as well as previously, but is much less cumbersome.

    And good guess, you pronounce my id correctly !

Leave a Reply

Your email address will not be published. Required fields are marked *