Advancing Robot Motion Capture with Low-Cost, High-Precision Technology

In an article recently published in the journal Drones, researchers proposed an affordable, open-source, and easy-to-use robotic motion capture system, Easy Rocap, for unmanned platforms such as drones.

Limitations of Existing Approaches

Rapid technological advancements in robotics systems have greatly facilitated the progress of unmanned platforms like unmanned aerial vehicles (UAVs). In robotics applications, pose estimation plays a critical role in trajectory planning and execution as a robot cannot build an accurate spatial representation of the environment without correctly estimating velocity and position.

Specifically, real-time pose information is critical for many mobile robots. For instance, accurate and fast pose estimation is crucial for local motion control of drones. Camera-based motion capture (Mocap) systems display a good performance in robotics applications. Currently, robots utilize these systems extensively.

However, most camera-based Mocap systems, like infrared camera-based systems, are easily affected by camera occlusion and light noise. Moreover, common commercial Mocap systems are expensive and lack open-source hardware and software, further increasing the challenges of deploying them on robots.

The Proposed Solution

In this study, researchers proposed Easy Rocap, a simple, open-source, and low-cost robot motion capture system, to address the challenges related to lack of access to the high-precision Mocap systems’ source code, high cost of commercial Mocap systems, and ambient light noise.

Easy Rocap can quickly and effectively capture/estimate the accurate orientation and position of drones in real-time using special material markers as the mobile robot’s tracking objects and several fixed cameras to perform three-dimensional (3D) intersections.

Unlike the infrared camera-based Mocap systems, the proposed system employs multi-object tracking (MOT) and object filtering algorithms that combine object detection technology to ensure precise and continuous trajectories considering the potential obstacles and noise.

The Easy Rocap system was developed using consumer-grade commercial IR RealSense cameras and comprised three key components, including multi-view camera triangulation, multi-view correspondences, and a two-stage key points detection model (TKDM).

The TKDM with fine filtering was proposed to address the environmental light noise interference on visual detection. The proposed multi-view correspondence method combines MOT and object detection. The dual-layer detector can effectively detect objects in complex environments, and the MOT stably tracks markers under short-term occlusion. Geometric constraints are utilized by multi-view correspondences to match corresponding image points accurately.

Initially, a trained object detector was used to generate bounding boxes of markers and robots in each view. An object-filtering algorithm using class and confidence was designed based on training a real-time object detector to eliminate false detections.

Weights were added to matrix coefficients of multiple views based on confidence to reduce the occlusion effects from a certain view on the 3D pose estimation results, ensuring that contributions from every view were reasonable. Then, MOT was applied to maintain the trajectories’ continuity, and the epipolar constraint was applied to multi-view correspondences.

Eventually, the calibrated multi-view cameras were employed to calculate the markers’ 3D coordinates and effectively determine the target robot’s 3D pose. Specifically, the orientation and position of the robot were estimated by the coordinates of multiple markers. The proposed system can be easily integrated into the robot system as it utilizes real-time multi-camera data streams.

Researchers performed extensive experiments in real scenes and simulated environments using multiple cameras to evaluate the performance/precision and rapidity of the proposed motion capture system. They also investigated whether the system can offer real-time pose information to robots. UAVs and unmanned ground vehicles (UGVs) were used for the experiments.

Significance of the Work

The Easy Rocap system demonstrated fast speed and high precision for pose estimation. In the simulation scenario experiment, the proposed system’s average position estimation error was less than 0.008 m, and the average orientation error was less than 0.65^o without any obstruction in the simulation environment. These outcomes indicated the system’s ability to provide high-precision real-time and consistent position data. However, the system’s average position estimation error increased to 1.32 cm/0.0132 m during the fast drone flight around the obstacle.

In the real scenario experiment, researchers compared the proposed method’s localization results with the results of advanced light detection and ranging (LiDAR)-inertial simultaneous localization and mapping (SLAM) algorithm. In this experiment, the SLAM generated drifts during turns, while the proposed Easy Rocap overcame the accumulated errors and drifts of SLAM and made the trajectory more accurate and stable, which displayed the practical applicability of the system. Additionally, the system’s pose estimation speed could reach 30 Hz.

To summarize, the findings of this study demonstrated the feasibility of using the proposed Easy Rocap system for accurately capturing the orientation and location of an unmanned platform in complex scenes.

Journal Reference

Wang, H., Chen, C., He, Y., Sun, S., Li, L., Xu, Y., Yang, B. (2024). Easy Rocap: A Low-Cost and Easy-to-Use Motion Capture System for Drones. Drones, 8(4), 137. https://doi.org/10.3390/drones8040137, https://www.mdpi.com/2504-446X/8/4/137