HERITAGE XR – Social Mixed Reality for Cross-Location Heritage Tourism

Arctron applied a combination of 3D surveying and data processing techniques to create a high-resolution 3D reconstruction of the site in its current state. Using this model as a reference and thanks to research of Jörg Reimann, Arctron then added 3D models of the historic architecture. An augmented-reality application allows visitors to freely explore the historic appearance and to retrieve additional information. Please see this video for a detailed documentation.

The software suite VR4more from Consensive enables the exploration of cultural heritage twins in social virtual reality. Despite the large extent and tiny details of the photogrammetric reconstruction and the manual reconstructions of the historic architecture, their output-sensitive data streaming allows for smooth visualisation on almost any device and platform, including mobile VR-headsets. As a result, users can travel to reconstructed heritage places from any place in the world. Please test a current version of VR4more.

The combination of the above mentioned technologies, products, and services allows virtual visitors of cultural-heritage digital twins to meet those on the real site, who are using the AR-app for localisation. People using any of both endpoints can see and hear each other – almost as if they were truly together in Regenstein, for example.

The cross-location experience is built on a very modular, highly compatible and scalable tech stack using the Unity engine as a common software framework. The AR and VR apps can be developed completely independent by the involved companies. Arctrons AR-app builds on outdoor 3D localization and tracking provided by Vuforia. The social-VR application from Consensive relies on a core module for output-sensitive data streaming. The distribution functionalities of their social-VR-software are also modular and build on a flexible network architecture called Common Ground. Connecting both applications required Arctron to plugin the corresponding module VR4more-People into their AR-app.

Fortunately, the required data transfers are minimal. User poses and voice can be transmitted with a few Kbit/s. Accessing the digital twin via data streaming to remote locations benefits from a larger bandwidth, but it already works well with 5-10 Mbit/s and data caching ensures, that no geometry needs to be loaded twice. Once the relevant data have been seen, it can be continuously explored without further data transfer.

To learn more, please watch:

Dr. Alexander Kulik from Consensive

kulik@consensive.com