Calibrating robot targeting system

RoboMaster Robotics Competition focuses on the comprehensive application and engineering practice ability of the participating members in science and engineering disciplines, fully integrating many robot related technical disciplines such as “machine vision”, “embedded system design”, “mechanical control”, “inertial navigation”, “human-computer interaction”, etc. At the same time, the innovative combination of e-sports presentation and robotic competition makes the robot confrontation more intuitive and intense, attracting the attention and participation of many technology enthusiasts and the public.

cool robots

When I joined the newly-formed RoboMaster team by the end of 2016, I collaborated with my partner Tong Gao, developed motion controlling algorithms on 4 infantry robot at first. We made some advanced motion functions on our robot for tactics purpose, such as “Dancing Mode”. In this mode, we fixed the gimbal while make chassis pivot, so that the armor on chassis won’t be shot while we can still shoot enemy stably. Moreover, the robot can still move towards any direction in dancing mode.

After that, we turned to the advanced part - targeting system based on Computer Vision. RoboMaster Competiton encourages participants to explore CV algorithms by introducing a subgame. The upper screen shows password with 5 7-segment digits while 9 lower screens show unique hand-written digits from MNIST dataset. Both upper and lower screens refresh every 1.5 seconds. During the competition, the team that succeed in shooting 5 MNIST digits corresponding to the password sequentially will be awarded with a double attack buff, which could be the decisive factor of whole match. We utilized OpenCV and tiny_dnn to perform detection, segmentation and recognition. To ensure our robot’s shooting accuracy, we performed camera calibration, PnP solving and physical error correction. Finally, we optimized our program with multithreading and network pruning for onboard computer. All these efforts led to 98% overall accuracy. Our code can be found here.

Processed Image Find Target
processed_image find_target
Jiarui Xu
Final Year Undergraduate Student of HKUST / Research Intern at MSRA

My research interests include computer vision, robotics and deep learning.