Universal Scene Description: The HTML of the metaverse

Ever see the movie, “Finding Dory?”

The 2016 Pixar film about a blue tang fish with anterograde amnesia might not be your thing, but it could be compared to CERN, the first-ever website that went live August 6, 1991. 

What’s the connection? The animated film was the first to be built using Universal Scene Description (USD) — which, many say, is a foundational building block of the metaverse. 

In other words, USD is the HTML for 3D virtual worlds. 

“We weren’t thinking about the metaverse when we made USD,” Steve May, vice president and CTO of Pixar, said during a virtual panel discussion at Nvidia’s GTC event this week. “We did not anticipate that USD would grow this rapidly and this broadly.” 

[Follow along with VB’s ongoing Nvidia GTC 2022 coverage »]

Without a doubt, the metaverse is one of the hottest topics of discussion in the tech world — how to build it, govern it, monetize it — and USD is being lauded for its pivotal role in speeding up its evolution. 

And, in this, USD is on a journey the world has seen before. 

An easily extensible, open-source framework for the interchange of 3D computer graphics data, USD was specifically built to be collaborative, to allow for non-destructive editing, and to enable multiple views and opinions. 

Many compare its current iteration to HTML: Assets can be loaded and representation can be specified. Its next phase will be enhanced interactivity and portability — the CSS moment, so to speak. The general consensus is, “Let’s get to the JavaScript of USD,” said Natalya Tatarchuk, distinguished technical fellow and chief architect for professional artistry and graphics innovation at Unity Gaming Services. 

But first: Universal Screen Description origins

As May explained, USD came to be because Pixar was looking to solve workflow problems around film creation. The studio’s movies involve complex and often whimsical worlds that must be believable. Many animators work on scenes at the same time, so Pixar needed a tool that fostered collaboration and was also expressive, performant and fast. 

USD essentially merged, distilled and generalized numerous spread-out systems and concepts that had been around within Pixar for some time. The framework was fully leveraged for the first time in “Finding Dory,” which was released in June 2016. The next month, Pixar made USD open source. 

Ultimately, May described the platform as “old and new”; it is nascent and evolving rapidly. And, because it is so versatile and powerful, it is being widely adopted in many other areas beyond filming and gaming — design, robotics, manufacturing, architecture. 

Nvidia, for instance, took notice because the company had begun to develop content and apps internally for simulation and AI — particularly building worlds for simulating autonomous vehicles, explained Rev Lebaredian, Nvidia’s vice president of simulation technology and Omniverse engineering.

The company needed a common way to describe and build worlds, “really large ones, collaboratively in many spaces,” said Lebaredian, and USD “stripped down to the essence of the problem.” 

Many file formats had come and gone over the decades, he said, but USD felt like “there was a lot of wisdom imbued in it.” 

Taking it home

Similarly, home supply store Lowe’s had been leveraging 3D and augmented reality to present items to consumers, and the company wanted to expand such 3D visualization to operations, store design and the supply chain. 

Also, the company was looking for a way to describe digital twins for its stores — of which it has 2,000 with 20 different layouts and unique features to each, explained Mason Sheffield, director of creative technology at Lowe’s Innovation Labs.

The company’s existing ad hoc system had different departments using Autodesk Revit, 2D CAD, SketchUp and others, he said. Understandably, this provided scaling challenges.

But, in early 2021, Lowe’s adopted an Omniverse platform using USD that bridged its internal warehouse databases, shelf planning, store layout tools and product library. The company went from flat 3D models that had to be batch generated, to a hierarchical, shared file format (for instance, a planogram that can be altered and propagated throughout all stores), said Sheffield. 

“USD feels like a democratization of 3D that we hadn’t seen in other platforms,” he said.

Collaborative evolution

All that said, building blocks aren’t perfect.

As Tatarchuk pointed out, USD is a vehicle for interoperability, and standards need to be evolved to get to portability. “It’s going to take all of us to align on it,” she said. 

Guido Quaroni, senior director of engineering of 3D and immersive at Adobe, said he would like to see the framework approach the web surface. This would enable authoring and not just consumption; also, there should be increased interoperability between apps and surfaces. 

Matt Sivertson, vice president and chief architect for media and entertainment at Autodesk, underscored the importance of allowing artists to use any tool they want. One longer-term potential of USD is driving down the cost — from a workflow perspective — of switching between apps.

“It’s not just about the tools anymore,” he said. “A differentiation feature [will be] how well you support USD.”

The ability to scale to different surfaces is important as well, said Sheffield; he would also like to see native solutions for USD deployment, and a gentler developer learning curve. 

“I’m excited for that evolution toward the real HTML of the metaverse,” said Sheffield. 

Ideally, get directly to HTML 5 and TypeScript, said Mattias Wikenmalm, senior expert for visualization at Volvo Cars.

That said, while the concepts in USD have been “battle proven,” there is a “risk of making USD too complex too fast,” he said. We don’t want to end up in a situation where there are all sorts of plugins for different companies. 

“The building blocks are there, it’s just refining them and building on top of the solid foundation that’s already in USD,” said Wikenmalm. 

To continually support the evolution of the tool it released in the wild, Pixar is ramping up hiring for its USD team. This will help the company explore USD applications beyond filmmaking, said May.

“There are a lot of things we still want to do, a lot of functionality we don’t yet have,” he said. 

Going forward, it will be critical to deeply engage with the community: “What does go into USD? What doesn’t go into USD? How do we avoid USD collapsing from its own weight?”

“We need to make the right decisions — collectively,” said May.