Imagine a world where humans and robots could interact safely. We believe that one of the steps could be a world where robots could play games with humans safely. BaxterChess helps us gain intuition on this problem. By teaching Baxter how to play chess in our world, we are forced to draw upon concepts in robotics, signal processing, and machine learning. This mixture of fields is why our project poses an interesting challenge. Our goal is to show Baxter how to play chess with an ordinary chess set without any aid from AR markers or other perception tools other than Baxter's cameras. Future applications with our project could be, with a little effort, to teach Baxter how to play other games on the same kind of board.

Our design must incorporate some sort of sensing, which implies that we must create some sort of filter to account for the noise in our sensors, as well as some for of actuation. Our desired functionality is the ability for Baxter to play a game of chess with minimal additional sensors.

Our design model goes as follows. As Baxter receives some sensory input from its left hand camera, it finds the chessboard within the image and transforms that picture into a 2D chessboard. That unperspectivized image is then passed to the brain where it is then analyzed for the location of pieces. The piece detection algorithm will detect whether or not there is a piece in a given square, and if there is a piece, detect the color. Once the pieces are detected, Baxter will figure out which of the next possible boards is the most probable and assume that that is the current board. Then Baxter communicates with its internal chess engine and retrieves a move. After retrieving that move, Baxter converts the move into a set of coordinates of the pieces involved in the move and a move type. Using these coordinates and the move type, we can then compute the inverse kinematics needed to either advance a piece, castle, or capture. Baxter then re-examines the board and waits for its next turn.

Some of our design choices include: not using AR tags, and not using suction cup grippers nor flat chess piece cutouts. We thought that it would be much more interesting if we could make Baxter manipulate chess pieces just as a human would. However, this made our problem a lot more complicated. Without AR tags, we had to sense real world coordinates using a different means. Without suction cup grippers and chess piece cut outs, we had to create or own grippers to optimally grasp chess pieces. These design choices helped challenge us to come up with solutions that could be better generalized in the real world. Since we are not using any extra help for our sensing, we have less parts to keep track of.