Creating a project with a virtual production template

Connecting to Mars using the Live Link Plug-in

Assigning Live Link Controller to a CineCameraActor

Recentering tracking for CineCameraActor

Configuring the media source using Composure

Inputting calibration data into Unreal Engine (4.27 or earlier)

Importing a Lens File (.ulens) into Unreal Engine 5

Applying Lens File to CineCameraActor

<aside> <img src="/icons/warning_red.svg" alt="/icons/warning_red.svg" width="40px" /> WARNING: None of the IBIS (In Body Image Stabilizer) or lens optical stabilizer work with VIVE Mars tracking system, please turn them off before using with VIVE Mars. Image stabilizer artificially eliminates micro jitter on camera body but VIVE Tracker (Rover) still detects this micro jitter. This causes composited footage jitter or impact the accuracy of lens calibration results.

</aside>

Overview

1. In order to implement real-time composite in Unreal Engine, the user needs to enable the proper plugins of virtual production, such as Live Link, Composure, and Media Framework. To simplify the process, please create a new Unreal project using the Virtual Production template. See Creating a project with a virtual production template.
2. In order to control the transform and lens parameters of the virtual camera, the user needs to connect the Mars tracking system to the virtual camera in Unreal Engine. See Connecting to Mars using the Live Link Plug-in and Setting up the virtual camera using the Live Link Controller.
3. Using MediaBundle to stream video sources to Unreal via capture devices and using a native keyer to chrome key the green screen. See Configuring the media source using Composure.
4. In order to map the virtual and real cameras, the user needs to align camera intrinsic parameters, such as focal length, lens distortion, and nodal point offset. Using Lens File to create the mapping of cameras. See:
   • Unreal Engine 4.27: Inputting calibration data into Unreal Engine 4.27.
   • Unreal Engine 5: Importing a Lens File (.ulens) into Unreal Engine 5.
5. Assign a Lens File to Live Link Controller to set the intrinsic parameters of virtual camera in Unreal Engine. See: Applying Lens File to Live Link Controller.

https://embed.notionlytics.com/wt/ZXlKM2IzSnJjM0JoWTJWVWNtRmphMlZ5U1dRaU9pSnZjRlk0UVVOa2JFSndhemhYWmxvek1tYzVaU0lzSW5CaFoyVkpaQ0k2SW1abE56SmlNVEEwTVdVMlpqUmpOMkZoWkdKbU1EUmxPRGhtWmpOaU1HSmlJbjA9