SPOT AR-VR APP
A R&D PROJECT BY INTUITIVE ROBOTS
Benefits of Merging Mobile Robotics
with Virtual & Augmented Reality
At Intuitive Robots, we are dedicated to significantly enhancing the performance of the Spot robot, both in terms of hardware and software. By integrating cutting-edge technology and innovative solutions, we strive to make Spot an indispensable asset for industries looking to leverage automation and intelligent systems in their inspection processes.
Latest R&D project to date, we can now combine virtual reality (or augmented reality) and mobile robotics like Boston Dynamics Spot robot. For industrial inspection, it offers numerous benefits that can significantly enhance the efficiency, safety, and accuracy of inspection processes.
From Remote Inspection…
Today, it is possible to control and monitor the robot, capture data, and create inspection missions using the robot’s tablet or the Orbit application (recently launched by Boston Dynamics). While the tablet requires the operator to be present on-site, Orbit allows the robot’s missions to be monitored from anywhere in the world.
Teleoperating the robot during an inspection mission or while creating an autonomous route can sometimes be complex. The human eye may overlook a point of interest or even miss a potential anomaly source while watching the robot navigating or inspecting.
… to Remote and Immersive Inspection
Now with AR-VR App, you would be able to inspect your facility in real-time and in total immersion, as if you were right there alongside the robot, even in the most hazardous locations, while you are comfortably seated in your velvet armchair.
With this next-generation application, you have the ability to combine the performance of autonomous mobile robotics with the full power of AR-VR, enabling even more precise industrial inspections.
Deep Immersion
Natural Vision & Interaction
Focus and Engagement
- Enveloping Experience: A VR headset completely covers the user’s field of vision, creating a sensation of being “in” the environment rather than viewing it from the outside. This significantly enhances the feeling of presence.
- 360° Perspective and Gaze Control: Users can turn their heads and see the environment all around them, making navigation more intuitive and interaction with the environment more fluid, which is not possible with a tablet or a laptop.
- Controllers and Gestures: VR headsets often come with controllers or motion sensors, enabling richer and more precise interactions with the environment, adding possibilities for interaction that tablets cannot offer.
- Realistic Dimension: This allows for more accurate judgment of the size and distance of objects, which is crucial for applications such as industrial inspection.
- Elimination of Distractions: By wearing a VR headset, users are isolated from real-world distractions, which can improve concentration and engagement in the experience.
- Increased Engagement: The immersion and natural interaction in AR-VR tend to keep users’ attention more effectively than traditional devices like tablets.
Efficiency & Accuracy
Better Collaboration
- Real-Time Data Visualization: AR can overlay real-time data, metrics, and information directly onto the physical environment, helping operators to quickly address issues.
- Precision in Inspection: VR can provide highly precise measurements and identify defects or issues that might be missed by the human eye.
- Enhanced Cooperation: AR-VR with Spot Enables remote experts to guide on-site personnel or control robots in real-time, facilitating better decision-making and troubleshooting.
- Training and Simulation: Can be used for training purposes, allowing operators to simulate inspections in a virtual environment before executing them in the real world
A Wide Range Of Features
Navigate & Look Around
Control of the Spot Robot:
- Control Spot’s movements and orientation (Motors On/Off, Stand/Sit, Navigation, etc.)
- Control the robot’s speed
Control the PTZ Camera
- Orient the PTZ via the controllers or according to the position of the headset (Head Gaze) in real time
- Zoom in/out
- LEDs On/Off
- Switch from the PTZ camera to the infrared sensor
Collect Visual & Thermal Data
Capture of Points of Interest (PoI):
- Take photos of your inspection points with the PTZ or the infrared sensor
- View the collected data in your gallery
- Add “Warning” points
- Leave comments via your headset (Speech-to-Text) and change the visual typology
- Augmented reality and geolocated PoIs
Intuitive
Interaction
User-friendly Interface
- Usable without the controllers, using only your hands for an even more intuitive experience (Specific UX with hand and finger gesture recognition)
- Intuitive and interactive virtual cockpit (Human-Machine Interface) with controllers or virtual hands
- Use Honk & Megaphone features
OpenAI, What Do You See?
Use OpenAI (GPT-4):
- Ask OpenAI to analyze the image captured by the PTZ camera, using your voice (inputs). It allows to generate PoIs (Points of Interest)
- It uses deep learning and Levatas solutions
Technical Configuration
SPOT + CAM+IR + DOCKING
RICOH THETA Z1
META QUEST 3
How does it work?
Requirements
Robot Platform
Spot Robot
The most efficient inspection robot so far. Several additional payload options are available for Spot:
- Combo CAM+IR + Ricoh Theta360 Z1: This configuration is recommended for optimal use, offering a dual video stream: 360° bubble view + PTZ stream in Picture In Picture (PIP) mode.
- With CAM+IR only: Enables viewing the 360° bubble from the CAM+IR but does not offer control of the PTZ camera.
- With Theta360 only: Allows viewing the 360° bubble from the Theta360 but does not provide an additional second stream in PIP mode (PTZ camera from CAM+IR).
Other Robots
Our AR-VR application can operate on any robot equipped with the Ricoh Theta360 Z1. Control of robot is possible with software integration.
VR Headset
MetaQuest 3 & MetaQuest Pro
These headsets are compatible with our system, offering standalone use without the need for an additional computer.
We have developed our own SDK (C#) to control Spot from a Meta Quest VR headset.
360° Camera
Ricoh Theta360
Using this camera provides a 4K stream for the 360° bubble, along with a dual stream in PIP (picture in picture) mode from the PTZ camera if the CAM+IR payload is installed on Spot.
Internet Connection
Not Required:
For basic use. Spot robot, Theta360 camera, and VR headset just need to be connected to the same network.
Required:
An Internet connection is required to use OpenAI modules for vision analysis, as well as automatic speech recognition (ASR) and text-to-speech (TTS) functions.
Performance & Quality
Headset/Robot Latency: Video stream latency is under 100ms, ensuring a smooth experience.
Stream Quality:
- Theta360: Video stream is transmitted at up to 4K (3840 x 1920) at 30 frames per second, providing high image quality.
- PTZ CAM+IR: Video stream is transmitted at up to 2K (1920 x 960) at 30 frames per second.
For optimal quality, we recommend a minimum bandwidth of 60 Mbps.