The concept of a “digital twin” is bringing a whole extra dimension to industrial robotics. In this case study, RoboDK users at the Dubai Robotics Lab combined neural reconstruction with robot simulation into an innovative digital twin setup.
A digital twin is a virtual replica of a physical system. It allows you to make better informed strategic decisions about your automation system, helping you to detect and iron out problems long before they become a major problem.
By combining state-of-the-art robotic technology and neural reconstruction, the researchers brought more precision and efficiency to the 3D modeling.
Let’s look at how the team at Dubai’s Robotics Lab used RoboDK to create their innovative system.
Digital Twins: The Future of Robotics
What is a digital twin?
In robotics, a digital twin is essentially a virtual model that closely replicates a physical robotic system.
This model is as detailed as it needs to be for the task at hand — it need not be a hyperrealistic simulation. For example, it will certainly include the kinematic and physical properties of the robot itself. It will also include other components that are important for the robotic task, such as sensors, end effectors, and task objects.
The use of digital twin technology holds tremendous potential. In manufacturing, a digital twin of a robotic arm, for instance, can help you optimize your production processes, identify bottlenecks, and predict maintenance needs without disrupting the robot’s productivity.
RoboDK is a popular platform for digital twin creation. For example, previous research from Western Washington University involved creating a simulated changeable learning factory and connecting it to the physical system to create a digital twin.
Introducing… Dubai’s Robotics Research Lab
The Robotics Research Lab is situated within the Dubai Institute of Design and Innovation. Led by researcher Raffi Tchakerian, this cutting edge facility is committed to pushing the boundaries of robotics and advanced manufacturing.
Tools like RoboDK have been instrumental in advancing research and student projects at the Dubai Institute of Design and Innovation into realms traditionally dominated by seasoned engineers. From 3D printing with sand to bio-printing garments on pre-existing 3D objects, RoboDK stands out as a pivotal enabler in our journey, explains Raffi Tchakerian
As part of the lab’s FabLab setup, Tchakerian’s research team uses a KUKA robotic arm to develop solutions and ideas for advanced manufacturing automation.
In this latest project, the researchers aimed to improve digital twin technology by combining their industrial robot with the latest in neural reconstruction technology.
The Setup: KUKA KR 150 Robotic Arm, Jetson and RoboDK
The aim of this research project was to see how neural reconstruction technology can improve digital twin creation.
To achieve this, the research team used the following hardware and software components:
- KUKA KR 150 Robotic Arm — At the core of the project is the lab’s KR 150 industrial robot. In a variety of manufacturing and other industrial settings, manufacturers and other industries use this 6-axis robotic arm.
- Intel RealSense D435i camera — An off-the-shelf depth camera that combines robust depth sensing with inertial measurements to create point cloud data.
- NVIDIA Jetson Nano — The Jetson is a single-board, AI-powered computer system targeted at embedded applications. We have a version of RoboDK specially designed to run on Jetson boards, opening up a world of new possibilities for AI-powered robotic solutions.
- RoboDK — Finally, the software for the team’s project was based on RoboDK. This widely used robotic offline programming and simulation software is ideal for digital twin creation and already includes the KR 150 in our extensive Robot Library.
The Role of NVIDIA’s Neural Kernel Surface Reconstruction (NKSR)
An important part of the project was NVIDIA’s Neural Kernel Surface Reconstruction (NKSR) technology.
This cutting edge set of algorithms helps to generate highly detailed and accurate 3D meshes from large-scale point clouds of noisy location data.
NKSR technology can scale to large scenes, handle noise, and minimize training requirements. It can reconstruct millions of points in seconds, even when the scan data is messy.
The team used this technology to clean up the point cloud data captured from the RealSense depth camera. These data points were then fed through the NKSR algorithm to create clean models for use with the robotic digital twin.
How the Setup Works
The researchers’ system operates with the following process:
- The Intel RealSense camera captures a rough 3D model of the scene, creating a point cloud of data.
- This point cloud is captured by the Jetson Nano board.
- Each frame of 3D data is synchronized and transformed into a refined point cloud using the Open3D library.
- An initial mesh representing the scanned object is generated and sent to RoboDK.
- RoboDK then accurately positions this mesh within the simulated robot scene.
- The mesh is then further refined using the NKSR algorithm.
This process shows the immense potential for integrating off-the-shelf imaging technology with advanced neural reconstruction for digital twins.
Advancing 3D Modeling with Robotics and Neural Reconstruction
What is next for this type of neural digital twin technology?
The researchers from the Robotics Research Lab showed how you can create powerful simulated digital twins using simple components. Many industries could use this type of setup, from aerospace to pharmaceutical manufacturing.
This project also shows how accessible advanced neural processing algorithms are becoming. With technologies like the NVIDIA Jetson Nano and NKSR algorithms, you can now access powerful functionality in an easy-to-use setup. And with RoboDK, you can seamlessly integrate this functionality with your industrial robot.
If you are looking for a way to integrate your robot with advanced algorithms, this case study is a powerful example of what is possible.
What questions do you have about RoboDK? Tell us in the comments below or join the discussion on LinkedIn, Twitter, Facebook, Instagram, or in the RoboDK Forum.. Also, check out our extensive video collection and subscribe to the RoboDK YouTube Channel.
