Gracia's Moving Volumetric Captures: A Revolutionary Leap in Immersive Technology
The world of virtual reality (VR) and augmented reality (AR) is witnessing a groundbreaking development with Gracia's innovative volumetric capture technology. This cutting-edge solution enables the streaming of dynamic, fully volumetric scenes, marking a significant leap forward in the industry.
A Journey from Static to Dynamic
Gracia initially made its mark in 2024 with a Quest app for viewing static, photorealistic objects. However, their true potential was unveiled with the introduction of a PC VR app for moving, photorealistic scenes, a concept known as volumetric video. This technology goes beyond simple parallax or limited-perspective synthetic depth, offering an immersive experience where users can physically walk around in VR or mixed reality.
Gaussian Splatting: The Secret Sauce
At the heart of Gracia's success lies Gaussian splatting, a technique that fits millions of semitransparent colored blobs (Gaussians) in 3D space. This enables realistic rendering of arbitrary viewpoints in real-time. Moving splats, or 4DGS (four-dimensional Gaussian splats), add the dimension of time, creating dynamic and captivating scenes.
Overcoming Streaming Challenges
Initially, streaming these volumetric scenes required a 2.4 gigabit internet connection, a significant hurdle for most users. To address this, Gracia developed a technique for sending keyframes and motion change deltas, encoding only the changes over time. This approach, inspired by video codecs, reduces the data transfer and allows for near-instant streaming.
WebXR and WebGPU: Unlocking New Possibilities
Gracia's technology leverages WebXR and WebGPU, eliminating the need for app downloads and enabling streaming directly from the web browser. This breakthrough makes volumetric captures accessible to a wider audience, with no hard file-size cap, theoretically supporting entire concerts or other large-scale events.
Streaming Quality and Internet Requirements
To stream high-quality volumetric scenes, a constant 75 Mbps connection is necessary. While this was a challenge a decade ago, the widespread availability of fast internet in developed countries now makes it feasible. Gracia also offers a 17 Mbps mode for lower-quality streaming, ensuring accessibility even in areas with slower internet speeds.
Immersive Experiences and Use Cases
The streaming capability of Gracia's volumetric scenes opens up exciting possibilities. Users can enjoy a musician or comedian performing as a 6DoF volume in their living room or preferred VR environment. This level of immersion surpasses traditional 3DoF 180° 3D videos, offering a more engaging and realistic experience.
Capture and Processing: The Underlying Challenge
Despite the streaming success, capturing these volumetric scenes requires a significant setup. Around 60 shutter-synced and genlocked cameras are needed, with studios charging at least $15K for this service. Gracia's processing costs add to the expense, but the company is working on reducing these barriers to make volumetric capture more accessible.
The Future of Volumetric Content
The author speculates that the 'YouTube of truly volumetric content' could become a reality by the end of the decade. With the rapid advancements in AI and 3D reconstruction, the technology is poised to revolutionize XR (Extended Reality) technology. The potential for mainstream use is vast, and the compounding advantages of a social content feed platform could propel its adoption.
Gracia's streaming capability is a significant milestone, pushing the boundaries of what's possible in immersive technology. As the industry continues to evolve, we can expect even more groundbreaking innovations that will shape the future of VR and AR.
