Description

The increasing popularity of XR applications is driving the media industry to explore the creation and delivery of new immersive experiences, while pushing engineers and inventors to address the challenges of real video content manipulation. Considering this reality, the talk will discuss these challenges and introduce key technologies that leverage open standards to enable large scale distribution of new immersive experiences.

A volumetric video is comprised of a sequence of frames, and each frame is a static 3D representation of a real-world object or scene capture at a different point in time. Volumetric video is bandwidth-heavy content that can be presented as dynamic point clouds, multi-view plus depth, or multi-plane image representations.

These bandwidth requirements can be achieved through dedicated compression schemes that produce data rates and files sizes that are economically viable in the industry. Standards play a crucial role in ensuring interoperability across these different types of content and experiences, and this talk will spotlight the Moving Picture Experts Group (MPEG) Visual Volumetric Video-based Coding (V3C) standard as an open standard solution for efficient streaming of volumetric video.

To enrich XR experiences, inventors and industry must synchronize to create and distribute additional media
and interactions mechanisms alongside all the volumetric video components. In fact, the MPEG-I Haptics and MPEG-I Scene description standards are currently under development and will soon propose solutions to industry.

From the perspective of real content creation, it remains challenging but crucial to enable technologies that cover 3D capturing, calibration, depth processing, format conversion, transmission, and rendering. This session will explore two creation pipelines that leverage the aforementioned standard codecs: (1) a real-time pipeline for telepresence using depth and color cameras; (2) an offline pipeline for sports and media applications using color cameras and prior geometric information.

Speakers

Valerie Allie

Video Solutions Group Director , InterDigital

Christiaan Varekamp

Senior Scientist , Philips Group Innovation, Research