Leveraging Mocap Data for Localization of Logistical Systems
Markus Knitt, a young researcher in the field of technical logistics, had always been fascinated by the efficiency of robots moving around warehouses and factories. For efficient movement, these robots require precise localization systems that can track their movements in real-time accurately. Thus, when he joined The Institute for Technical Logistics (ITL) at the Technical University of Hamburg (TUHH), he worked eagerly with Jakob Schyga, an experienced Mechanical Engineer at the institute.
The institute decided to use a Qualisys Motion Capture (mocap) system as their ground truth measurement tool. They were drawn to its open system usage and supportive customer base, which Jakob had been introduced to at a Qualisys User Workshop at the Gothenburg, Sweden Headquarters. After participating in objective conversations with industrial users of Qualisys technology, he was convinced that a Qualisys system was the ideal tool for their research.
They selected an 8-camera Miqus M5 camera system for the vast 20m x 12m x 9m test hall, where they mounted the cameras to capture the movement of various equipment within 90 square meters. This space is filled with a variety of logistics equipment, including conveyors and industrial trucks, tracked as 6DOF (rigid body) models in QTM. The ceilings reach a height of 6m and Miqus cameras are mounted high to seamlessly track the rigid bodies in the test hall.
Variety in applications
In the world of logistics and warehousing, the movement of goods and products within a facility is a critical aspect that requires constant monitoring and analysis to ensure efficiency and safety. Tracking and analyzing the movement of these vehicles such as forklift trucks can give valuable insights to warehouse managers, helping them identify maneuvers that can lead to crashes, damages, and other mishaps.
The first step in building the warehouse vehicle analysis application was to prototype a simple system and cross-check it with localization data for testing and validation. However, instead of relying on the localization data from a typical localization system, the researchers used the data from the Qualisys system given its precise accuracy.
Researchers deliberately scattered the data collected by the mocap system to simulate faulty data input with the intention to forecast the potential errors in equipment maneuvers. The advantage of using mocap data is the ability to obtain localization data in real-time, which can not only be used as a ground truth but also to controllably generate faulty data to simulate other localization systems commonly applied in the field of logistics.
This enables the researchers to analyze the movement of the vehicles in the context of the warehouse environment, identifying potential hazards and suggesting ways to optimize the routes and operations of the vehicles. Data collection may also focus on the motion of the vehicle operator, to identify patterns of operator behavior that can lead to accidents, such as over-speeding, sudden turns, or lack of attention.
In the world of logistics and warehousing, the movement of goods and products within a facility is a critical aspect that requires constant monitoring and analysis to ensure efficiency and safety. Tracking and analyzing the movement of these vehicles such as forklift trucks can give valuable insights to warehouse managers, helping them identify maneuvers that can lead to crashes, damages, and other mishaps.
The first step in building the warehouse vehicle analysis application was to prototype a simple system and cross-check it with localization data for testing and validation. However, instead of relying on the localization data from a typical localization system, the researchers used the data from the Qualisys system given its precise accuracy.
Researchers deliberately scattered the data collected by the mocap system to simulate faulty data input with the intention to forecast the potential errors in equipment maneuvers. The advantage of using mocap data is the ability to obtain localization data in real-time, which can not only be used as a ground truth but also to controllably generate faulty data to simulate other localization systems commonly applied in the field of logistics.
This enables the researchers to analyze the movement of the vehicles in the context of the warehouse environment, identifying potential hazards and suggesting ways to optimize the routes and operations of the vehicles. Data collection may also focus on the motion of the vehicle operator, to identify patterns of operator behavior that can lead to accidents, such as over-speeding, sudden turns, or lack of attention.
As the team continued their experiments, they could see the expansive potential of the Qualisys system. They used it to benchmark against other localization systems, and the motion capture system’s superior performance was evident. The team even realized they could leverage the system’s potential to train Artificial Intelligence algorithms, a feat that could shape the future of the industry for years to come.
Variety in applications
They then went on to use the system for automating robots. This involved capturing motion data while applying Robot Operating System (ROS) techniques to direct and train robots to perform various tasks. The data collected from the Qualisys system enabled the robots to move with fluidity and accuracy. Additionally, the researchers combine the mocap data with wheel odometry, cameras, LIDAR and other sensor systems to track the motion of the custom-built robot. Combining data from multiple sources, also known as Sensor fusion, entails multiple benefits, such as allowing for continuation of an experiment even when the tracked object steps outside of the captured volume.
Another innovative project performed at the Institute included a virtual reality (VR) firefighter training simulation. In partnership with another institution that specializes in maritime logistics, they create a training scenario that would help firefighters learn how to respond to container ship fires. To make the training even more effective, researchers integrated the Qualisys system into the simulation. This allowed them to track the movements of the firefighters as they responded to the fire.
They then went on to use the system for automating robots. This involved capturing motion data while applying Robot Operating System (ROS) techniques to direct and train robots to perform various tasks. The data collected from the Qualisys system enabled the robots to move with fluidity and accuracy. Additionally, the researchers combine the mocap data with wheel odometry, cameras, LIDAR and other sensor systems to track the motion of the custom-built robot. Combining data from multiple sources, also known as Sensor fusion, entails multiple benefits, such as allowing for continuation of an experiment even when the tracked object steps outside of the captured volume.
Another innovative project performed at the Institute included a virtual reality (VR) firefighter training simulation. In partnership with another institution that specializes in maritime logistics, they create a training scenario that would help firefighters learn how to respond to container ship fires. To make the training even more effective, researchers integrated the Qualisys system into the simulation. This allowed them to track the movements of the firefighters as they responded to the fire.
Furthermore, the mocap data is used for modeling robots’ kinematic systems with machine learning algorithms. The trained models are later used for different use-cases. For instance, such models allow for the monitoring of the robot during its operation. If faulty or safety-critical behavior is identified, a warning can be automatically generated or the robot be stopped immediately. Such applications are particularly relevant, due to the increased utilization of black-box AI algorithms for localization. Additionally, the trained models are utilized to predict the robot’s behavior in real-world scenarios and to identify edge cases for testing of the robot’s components.
Overall, the Technical Logistics team found the Qualisys Motion Capture system to be an incredibly valuable tool in their research. Its versatility and accuracy made it an ideal choice for a range of applications, and they were able to generate a wealth of data that will inform future research in this rapidly evolving field.
As technology advances and mocap systems become more accessible, we can expect to see more applications of this technology in the logistics and warehousing industry, revolutionizing the way we manage and optimize the flow of goods within our facilities. The group looks forward to acquiring more cameras in the near future.
We intended to use the Qualisys system for mainly two things: first as the ground truth for the position and orientation that the other systems would provide us and second, we used it for automating the robots. Since recognizing the versatility of its use applications, we continue to adopt it towards more and more.
Jakob Schyga, M.Sc.
Do you have a story to share?
Tell us all about how our technology has helped you and your submission is likely to be published on our customer stories page.