IMPLEMENTATION
INTEGRATION
TWO NODE
The Gomoku playing robot contained two nodes: /gomoku_brain (publisher) and /gomoku_controller (subscriber). The /gomoku_brain contained the computer vision and Gomoku AI game engine and the /gomoku_controller control the motion of Baxter. Two nodes would communicate via the topic called /gomoku_channel.
MESSAGE
We defined two new message type Position.msg and ToDoList.msg. Position.msg represented the 2D position of our 10x10 Gomoku board. Eg. (0,0), (5,3), (9,9) etc. It represents the 2D position of our 10x10 Gomoku board. Eg. (0,0), (5,3), (9,9) etc.ToDoList.msg contains two arrays: eraser and drawer. Both eraser and drawer are Position array. /gomoku_brain publishes these two arrays to /gomoku_controller to tell Baxter which position to eraser or to draw.
SYSTEM WORKFLOW
The webcam would capture the raw chessboard image and the computer vision algorithm computed the 10x10 matrix of the chessboard based on the image. Gomoku AI engine would base on the 10x10 matrix to generate the ToDoList message and then published it to the /gomoku_channel topic. Once gomoku_controller received the ToDoList message, it would control the Baxter use right hand to erase the points based on the eraser array and then use left hand to draw the points based on the drawer array.
COMPUTER VISION
RAW IMAGE PROCESSING
-
Firstly it will detect all the intersections on the grid and determine the largest quadrilateral according to the distance between the intersections and the corner of the picture.
-
Then we do a linear transformation to corp the entire chessboard into a perfect square in order to increase the accuracy that all pieces are recognized correctly.
COLOR DETECTION
-
According to the intersections detected, we are now able to determine each grid. Then we are able to know whether there is a piece in the grid or not.
-
Setting a threshold for values of RGB for red and green. If the fraction of pixels falls into the threshold is greater than â…“, we say there is a green/red piece in the grid.
-
In the end, we produce a 2D numpy array to represent the status of the chessboard.
-
The 2D array will be passed into Gomoku AI.
GOMOKU AI
GOMOKU GAME ENGINE
For the inital gomoku game engine, we use the Expectimax Algorithm implemented by Zitong Mao (https://github.com/ZitongMao/gomoku-ai)
​
We mainly focused on designing the cheating detection algorithm and then intergal both algorithm to make it more poweful.
ALGORITHM WORKFLOW
With the processes shown in the picture, we endue robot with the abilities to detect whether the state is valid, make a good decision of putting next piece chess if received valid chessboard state by Expectimax Algorithm and find where to erase or draw to recover the chessboard if received invalid chessboard state.
In detecting part, the robot compares the new state matrix with the latest valid matrix generated after robot operating in the previous turn. Only if the new matrix has exactly one more player’s piece than the latest valid matrix and other values in the matrix remain same can the robot say that the new matrix is valid.
VALID STATE
We call the function in expect max algorithm to find the best location and add this location to “drawing list” because the robot needs to continue the game if the player did a valid operation.
In the pictures shown in the right, the robot receives a valid new matrix comparing to the latest valid matrix.
"erasing list " = [ ]
"drawing list" = [(4,4)]​
"reminding list" = [ ]
INVALID STATE
The robot will first find all locations that have different values in the new matrix and the latest valid matrix. In order to recover to the latest valid matrix, robot adds all these locations in the “erasing list”. Since robot is able to recover its chess pieces by drawing again on the board but is not able to add player’s chess pieces, it adds locations that should represent robot chess pieces in the “drawing list” and locations that should represent player’s chess pieces in the “reminding list ” to remind player to draw his/her chess pieces in order to recover to the valid state.
​
For pictures shown in the right:
erasing list” = [(4,4), (6,6), (7,7)]
“drawing list” = []
“reminding list” = [(4,6)]
MOTION
MOTION HARDWARE
The marker with buffer
Considering the arm is likely to stop just over the chess board within the error acceptance of the marker, the buffer part is added to the marker to increase the tolerance error of the marker so that arm can always stop within the tolerance range of the marker.
The implementation of the buffer:
The marker is not that tightly fixed to the branch of the gripper. It is limited in the pipe which is fixed on the gripper and falls down to the lowest position under gravity and most of the marker is over the tip of the gripper. If the tip fo the marker hits the board, the marker will be pushed up along the pipe while still keeps up straight. The marker can shill draw on the board as expected and the tolerance distance is much increased.
MOTION HARDWARE
Eraser
The eraser is made up with a dish-washing sponge. The sponge is fixed on the suction cup and since the suction cup and sponge are both soft, their fixture can operate as a buffer to counteract the uncertainty of the arm position.
CODE IMPLEMENTATION
Path Planning
The motion of the arm is based on LAB 7, path planning Many obstacles are added to avoid unnecessary collision with the operator, with the wall and with other robots.The motion/rotation of gripper is based on LAB 3, the Joint velocity setting
​
After the gripper moves to the target location, the gripper, with a marker tied to one branch of the gripper, starts to rotate to draw. The gripper will open at the position of 60 (out of 100) to decide the radius of the circle it plans to draw. The wrist joint will rotate with a certain speed for a certain time clockwise for a circle,(in our implementation, speed = 4.6, time = 1.8) and then with the opposite speed to draw another circle counterclockwise. Then the gripper position reduces to 40 and draws another circle with a smaller size. Another even smaller circle with a radius of 20 is drawn and these processes leave a large enough mark on the board. The arm returns to the origin position after finishing drawing along the path planned by path planner.
The mechanics for the eraser arm generally symmetric. For most of the time, the only difference is that the coordinate on the y-axis is the opposite because of the difference between left and right arm. Only one rotation is needed for the eraser gripper. It is clean enough.