Dice Detection using OpenCV

I’m working on a small project that will eventually involve object detection and sorting using a robot arm. In terms of object I considered various items from small plastic bottles to figurines and eventually settled on dice given their size, weight and suitability for what is basically a $20 plastic robot arm. Easy, I figured, grab some training images and train a tiny-yolo model for detection and we should be good to go, right? Not really.

This is what we are trying to detect, below. We want to know the color of the dice, as well as the number of pips it sees, from 1 to 6. We want to detect this off video, ideally, too, and fast, in real-time.

Dice

Training a yolo model, well any CNN basically requires a fair number of training images. In my previous post I trained a yolov3 model to detect rats and that took 600 images, carefully labelled and trained and I’ll be the first to admit that labeling hundreds of images is not my idea of a good time. It worked well, and I even managed to retrain it on tiny-yolo to fit on a Raspberry Pi3 and was happy with the result. The FPS rate wasn’t great but it worked well enough. So I figured I’d give it a go first with 40 odd images of a white dice, divided into 6 classes to denote the number of pips. Several hours later I had a model which detected pretty much nothing. Zero. Maybe I messed up the parameters or whatnot, but it made me consider alternatives.

Searching around I found a number of promising examples using OpenCV which I tried, with mixed results. OpenCV is fast, doesn’t require 60MB plus trained models and you can nicely break the problem down into different parts. So I started from scratch and assembled a fairly good model to detect not only different color dice but the pip count on each as well. Keep in mind I have specific requirements including dice location, rotation, color and distance from dice to camera being within a specific, fixed range. So don’t expect this to work for your casino tables right off the bat.

Key to making this work, and also the most painful part, is choosing an HSV color mask to extract the dice from the background. Now I assume that most of the time, and very much for what I need to do, you will have an idea what the background is going to be, say flat black, a green gaming board or whatever. You also have an idea of the distance between camera and dice too.

So the first step is to figure out the HSV color mask (lower and upper bounds of each) as shown below here on a white dice, within your own parameter constraints. It turns out green and red is easy, white is quite a pain to get right.

HSVConfig

You will notice in the screenshot above that I tuned the parameters using the trackbars to isolate the dice as much as possible. This won’t be 100%, there will be residual noise, but you want to be able to detect the pips, as circles, which you count using OpenCV’s HoughCircles method. We know what color we have if we detect pips within one of our 3 defined HSV color masks, basically. For bonus points you can detect the dice using contours too if you wish.

Having done the above for all three dice we get some very good results as shown below:

Red Dice:

Red3

Green Dice:

Green6

White Dice:

White3

The full Python code is included below. You’ll need to tune the HSV masks and the parameters for the HoughCircles method (minRadius, maxRadius) depending on your own requirements.

from imutils.video import VideoStream
import numpy as np
import cv2 as cv2
import imutils

# dice color in HSV
# measure these while on a typical expected background
greenLower = (43, 83, 103)
greenUpper = (99, 115, 182)
redLower = (137,26,149)
redUpper = (202,59,208)
whiteLower = (0,0,0) 
whiteUpper = (191, 160, 150) 

font = cv2.FONT_HERSHEY_SIMPLEX
topLeftCornerOfText = (10,30)
fontScale = 1
fontColor = (0,0,0)
lineType = 2

vs = VideoStream(src=1).start() #1=external USB cam

while True:
	frame = vs.read()
	if frame is None:
		continue

	frame = imutils.resize(frame, width=600)
	blurred = cv2.GaussianBlur(frame, (11, 11), 0)
	hsv = cv2.cvtColor(blurred, cv2.COLOR_BGR2HSV)
	
	#try red?
	mask = cv2.inRange(hsv, redLower, redUpper)
	mask = cv2.bitwise_not(mask) # invert
	circles = cv2.HoughCircles(mask, cv2.HOUGH_GRADIENT, 1, 20, param1=30, param2=15, minRadius=6, maxRadius=30)

	if circles is not None:
		circles = np.round(circles[0, :]).astype("int")
		if ((len(circles) > 0) and (len(circles) <=6)): # no point guessing
			cv2.putText(mask,"RED: " + str(len(circles)), topLeftCornerOfText, font, fontScale,fontColor,lineType)
	else:
		# try green?
		mask = cv2.inRange(hsv, greenLower, greenUpper)
		mask = cv2.bitwise_not(mask) # invert
		circles = cv2.HoughCircles(mask, cv2.HOUGH_GRADIENT, 1, 20, param1=30, param2=15, minRadius=6, maxRadius=30)
		if circles is not None:
			output = mask.copy()
			circles = np.round(circles[0, :]).astype("int")
			if ((len(circles) > 0) and (len(circles) <=6)):
				cv2.putText(mask,"GREEN: " + str(len(circles)), topLeftCornerOfText, font, fontScale, fontColor, lineType)
		else:
			# try white
			mask = cv2.inRange(hsv, whiteLower, whiteUpper)
			mask = cv2.bitwise_not(mask) # for white, depending on background color, remark this out
			circles = cv2.HoughCircles(mask, cv2.HOUGH_GRADIENT, 1, 20, param1=30, param2=15, minRadius=6, maxRadius=30)
			if circles is not None:
				output = mask.copy()
				circles = np.round(circles[0, :]).astype("int")
				if ((len(circles) > 0) and (len(circles) <=6)):
					cv2.putText(mask,"WHITE: " + str(len(circles)), topLeftCornerOfText, font, fontScale,fontColor,lineType)

	cv2.imshow("Preview", mask)
	key = cv2.waitKey(1) & 0xFF
	if key == ord("q"):
		break

vs.release()
cv2.destroyAllWindows()

 

Advertisements
Posted in AI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s