Crossing Realities: Connecting the Virtual
and the Physical World for Remote Inspections

Nuclear disarmament is the only solution to the threat of a global nuclear war. Traditional verification approaches of nuclear arms-control agreements towards disarmament have placed a strong emphasis on onsite inspections. Considering the large numbers of U.S. and Russian warheads as well as situations of political tensions or restricted global travel, in-person onsite inspections require a significant amount of time, resources, and trust between parties. As an alternative, remote inspections could facilitate and accelerate the verification process without the same requirements. While prior work has discussed how virtual reality could potentially be used for training and capacity-building in nuclear verification, we extend this idea by using virtual reality to combine virtual and in-person activities into remote nuclear verification inspections that reduce the intrusiveness and cost of inspections. Our approach utilizes bidirectional data flows and interaction between the inspector’s virtual environment and the host’s physical nuclear facility. We implement a full virtual reality environment and a physical demonstration to explore the technology and challenges of remote inspections conducted in virtual reality. For one direction of data flow, we use fiducial markers to estimate the position and orientation of key inspection elements in the physical world (e.g., warhead, robot, room) via an RGB camera, provide the estimated position and orientation data to a virtual reality project in Unreal Engine 5, and use 3D models of the tracked objects to show their relative positions and orientations in an immersive virtual reality environment with real-time updates. This provides inspectors in virtual reality with the key information from the physical world without revealing sensitive information. For the other direction of data flow, we use a mobile robot that can be controlled from the virtual reality environment so that it moves in the physical world. The robot acts as a placeholder for objects relevant to interact with during inspections (e.g., a radiation detector) and provides inspectors with a way to interact with the physical world even though they are not present. Combined, this work demonstrates and discusses how virtual reality could be used for remote nuclear verification inspections.