In December of last year, Meta revealed a new partnership with James Cameron’s Lightstorm Vision, aimed at enhancing 3D entertainment experiences for users of Meta’s Quest headsets. During the recent Meta Connect conference, Cameron joined Meta’s CTO Andrew Bosworth on stage to showcase the first outcome of their collaboration: Quest headset users now have access to an exclusive preview clip from Cameron’s forthcoming film, Avatar 3, through the newly introduced Horizon TV app.
Prior to their keynote presentation, I had the opportunity to engage in a conversation with the two innovators about the possibilities that mixed reality headsets offer for 3D video, the complex past of 3D television, and Cameron’s perspective on the rise of generative AI.
The following interview has been condensed for clarity and length.
What led you to collaborate with Meta on 3D entertainment?
James Cameron: It felt like a perfect match. I’ve been advocating for stereoscopic media and entertainment for 25 years. There was a lull, as cinema seemed to be the only suitable venue for showcasing it. While it had a brief resurgence on flat panel TVs, the technology was often inadequate.
However, with mixed reality headsets, users inherently become stereoscopic viewers. When I encountered these headsets, I realized it was an opportune moment to revive the technology I had developed over the years. Consequently, I founded Lightstorm Vision, focusing on stereoscopic production, which aligned perfectly with Meta’s search for a partner to explore stereo content creation for entertainment.
Andrew Bosworth: It was clear we were meant to find one another. We pitched ideas back and forth, and it’s been an incredible partnership. James excels not only as a storyteller, but as an innovator. He provides insights on how what we’re building can better serve the needs of storytellers, which is invaluable.
Many VR storytelling efforts have aimed to immerse viewers within the narrative and make the experience more interactive. It appears your focus is more aligned with 3D stereoscopic and traditional passive entertainment?
Cameron: You’re correct. My career has revolved around crafting linear narratives—both documentaries and fictional stories—within a defined rectangular frame.
Critics have often dismissed this rectangular format. Yet, its structure successfully guides the viewer’s focus. The Avatar films exemplify my approach, presenting numerous elements to appreciate within a single frame while ensuring that the frame itself plays a critical storytelling role.
There is a century-long cinematic language that reflects how our brains process memories. This shared cinematic vocabulary extends across cultures, from China and India to Japan and the Americas, as we all interpret visual narratives through the same rectangular lens.
Bosworth: The timing of our efforts is crucial. The reluctance to embrace this long-established cinematic language until now can be attributed to the lack of appropriate display technology. In the past, watching movies on headsets was impractical given the inferior resolution compared to TVs or smartphones.
Today, however, advancements in resolution, brightness, and refresh rates in the current generation of headsets, including ours, allow for both traditional television viewing and immersive media experiences. I believe there is space for both formats in this evolving landscape.
Cameron: I believe episodic television represents a significantly overlooked opportunity. The use of stereoscopic production can heighten viewer engagement when watching characters on-screen. However, there has been a lack of options for distributing episodic television in stereoscopic format until now. That is changing, and I anticipate it will be transformative.
Intriguingly, you reflect on 3D televisions, which have largely failed. Critics have drawn parallels between VR and the fate of 3D TVs. Are you suggesting that VR will indeed succeed where 3D TVs did not?
Bosworth: There’s a misconception that all 3D experiences are interchangeable. During my first visit to James in Los Angeles, he showcased a segment of his upcoming film. He displayed it as it would typically be projected in a stereoscopic theater with lower brightness, then again using a laser projector to reveal how he intended it to be viewed. The contrast was profound.
Audiences often perceive 3D films as homogenous. To appreciate differences in quality, such as what Dolby Vision or IMAX offer, viewers need to have a certain level of understanding. Previous 3D TVs provided merely a subpar stereoscopic experience due to factors like poorer glasses and limited projection depth.
Cameron: The drawbacks were evident…
Bosworth: Absolutely. The experience wasn’t satisfactory. The advantage of headsets is that they can guarantee a superior viewing experience every time. This product stands apart, and we may undermine ourselves if we equate it to consumer electronics of the past.
James, you are involved with Stability AI. From your vantage point as a filmmaker and CGI pioneer, what excites you about generative AI?
Cameron: The escalating costs of visual effects have become a significant hurdle for many filmmakers, limiting the types of projects that receive financial backing. Labor cost increases paired with a notable decline in the theatrical market—at least by 30 percent—highlight a pressing need for solutions.
Innovative custom AI models tailored to enhance existing visual effects workflows may serve as the answer. While the allure of a straightforward text-to-video approach is strong, I’m more interested in refining high-caliber production processes involving various effects. I aim to bolster productivity without sacrificing the jobs of talented individuals.
As I collaborate with Stability, I’ve connected with numerous generative AI developers who create tools in isolation from practical production needs. The robust production-focused tooling that has emerged over the past three decades in CGI and visual effects arose from actual production demands, not from theoretical premises.
Do you foresee generative AI democratizing filmmaking for emerging filmmakers? Historically, similar hopes have been associated with game engines and real-time production tools, yet this hasn’t materialized as anticipated. Instead, Hollywood has leaned towards using massive, costly LED screens…
Cameron: I believe generative AI will open new pathways for aspiring filmmakers, allowing them to enter the industry with creations made through prompts. While this could lower barriers, the fundamental structure of the industry may remain intact.
It is my hope that we will never reach a point where actors are replaced. The unique joy of filmmaking lies in collaborating with fellow artists, creating genuine, emotional moments and characters.
The assertion that generative AI lacks creativity is misguided. I believe it can be just as inventive. However, what it cannot replicate is the distinct lived experience and individual perspective that defines literature and film. It can support and enhance creative visions, and I intend to leverage it to the fullest, always prioritizing the creative process.
This piece is Lowpass by Janko Roettgers, a weekly column exploring the shifting landscape of technology and entertainment, available exclusively to Technology News subscribers.
0 Comments
