Results

Problems

A few major issues we had were with the MoveIt-Python interface. We were unable to add an obstacle using the programming interface even though we were able to add an obstacle using the Rviz interface. This feature was important because the z height of the obastacle would be computed during calibration, so we needed the program to add the obstacle. Another problem that occurred was with our random forest. Because of the nature of learning algorithms, we weren't able to obtain 100% accuracy when predicting the next board state. When the Brain makes an error with the board state, everything goes downhill very quickly, such as Baxter making moves relentlessly. This is probably the result of the difference in lighting between the training set and the real world, which is alsways a problem in Computer Vision. One way to shave off some of the error would be to gather a multitude of data and then train on that dataset, but there will always exist some irreducible error.

Some issues with Baxter is that it performs larger movements better than smaller precise movements. That means when dealing with chess pieces, we needed to make sure that the arm traveled a large enough distance in order to guarantee a chess piece move. Near the end, the gripper seems to have started to wobble a little, which made picking up the rook particularly difficult for Baxter. Towards the end, it was very difficult to get Baxter to make precise movements, and that was a bit of a hinderance for us.

Conclusion

Giving Baxter the ability to manipulate real world chess pieces is pretty challenging. Many problems from different disciplines appear in this task, namely, object recognition and homography from Computer Vision, and inverse kinematics and gripping from Robotics. By incorporating all our knowledge from our classes and researching current robotics and vision techniques, we were able to give Baxter some ability to play chess in the real world.

However, there are still some rough edges that need smoothing. For example, our sensing capabilities are around 98% accurate when using 300 trees in our random forest and about 7,000 training images. That 2% error adds up very quickly when we get a new board image every second and there are 64 squares that we need to classify per image. Another issue that arose when we were trying to find the inverse kinematics for Baxter when moving pieces on the chessboard. Because of the RRT algorithm, the positions Baxter's arms end up in make it very difficult for it to play very long games of chess without hitting something when it is readjusting its arm. These are just some things that could be improved in further work, but throughout the duration of our project, we have shown that it is possible to get Baxter to play at least a little chess.