Mixed Reality (MR) – Addressing the Requirements of Immersive Experiences with AWS
MR combines aspects of both AR and VR. It not only overlays but also anchors virtual objects to the real world, allowing users to interact with these objects as though they are there. Users can manipulate virtual objects in real time, offering a seamless blend of both realities.
Like AR, high-resolution (or outright transparent) passthrough is required for MR – meaning only a few VR headsets do a good job of supporting MR use cases. Higher-end AR glasses are normally the devices used for MR.
MR is used in medical visualization, remote work, training simulations, and design prototyping.
XR development platforms
Given how complex cross-platform interactive 3D applications are, you probably don’t want to write your own real-time rendering engine from scratch. Therefore, the first thing you should do is identify an existing platform/toolset to build with:
3D game engines: AAA studios have so much expertise in real-time rendering with interactive elements that the most mature VR platforms are 3D gaming engines. These are often the best choices, even if the application you have in mind isn’t a game. Examples include Unreal Engine, Unity, and CryEngine.
Media production platforms: Another category has emerged over time from tools that were originally designed for 3D animation in film, television, and similar entertainment media. Blender and Aero come from this background.
Computer-aided design (CAD): These evolved from 3D design tools meant for architects, civil engineers, mechanical engineers, and others. Examples are Maya and Autodesk VRED.
Online XR frameworks: Some tools were developed from the start to be simplified XR development tools. They are typically easier to get going with but lack support for advanced features available with other types. Babylon.js and O3DE are popular entries in this category.
The rest of this chapter will focus on how things would be implemented with Unity, which has a good balance between performance, features, and ease of adoption.
Online gaming with Unity
Unity is a cross-platform game engine and comprehensive set of tools developed by Unity Technologies. It supports both 2D and 3D applications on a variety of desktop, mobile, console, and AR/VR platforms. Because it is based on .NET, its primary scripting API is C#.
Unity provides an IDE called the Unity editor. In the Unity editor, you can develop and compile applications that include the Unity runtime for a variety of target devices. Available build targets include Linux, macOS, Windows, Android, iOS, PlayStation 4 and 5, and web servers meant to support browser-based clients via HTML5.