Web Standards for the Win: Building the Open Metaverse with Browsers with Ken Russell and Corentin Wallez from Google

Announcer:

Today, on Building the Open Metaverse.

Corentin Wallez:

Web standards need to make sure technology is coalescing in native before they bring it to the web. 

Otherwise, we risk pushing for something that ends up being a technological dead end or breaks the web by removing it.

Announcer:

Welcome to Building the Open Metaverse, where technology experts discuss how the community is building the open metaverse together. Hosted by Patrick Cozzi and Mark Petit.

Marc Petit:

Hello, and welcome back, metaverse builders, dreamers, and pioneers. I am Mark Petit, and this is my co-host, Patrick Cozzi.

Patrick Cozzi:

Hey, Mark, great to be here. We're going to cover a topic dear to my heart with some of my favorite people.

Marc Petit:

Yes, it's great, and you're listening to Building the Open Metaverse, season five. As you know, this podcast is your portal into open virtual worlds and special computing.

We bring you the people and the projects that are on the front lines of building the immersive internet of the future, the open and interoperable metaverse for all.

Patrick Cozzi:

Today we have two special guests joining us on that mission. Over the years, they've made huge contributions to graphics programming on the web with their work on WebGL and WebGPU.

Please welcome Ken Russell and Corentin Wallez from Google.

Marc Petit:

As you probably have guessed already, this episode is going to be all about web browsers and 3D graphics.

Before we dive in, we'd like to hear from our guests about their journey to the metaverse in their own words.

So let's start with you, Ken.

Ken Russell:

Hi, my name is Ken. I still chair the WebGL working group and manage the combined WebGL WebGPU team at Google.

Journey to the metaverse, my first job in the industry was interning at Silicon Graphics, working on their Cosmo Player verbal plugin for the web browser on the PC.

Corentin Wallez:

Hi, I'm Corentin Wallez, co-chair of the WebGPU group since it started in 2017 and tech lead of the WebGPU implementation in Chromium.

And my journey in the metaverse, I guess, started with, like a lot of other people, in various MMORPGs. In my case, it was a MUD, a multi-user dungeon. I can't remember the name off hand, but that was my metaverse when I was a kid.

Patrick Cozzi:

So, Ken, you mentioned chairing the WebGL working group, and first, I wanted to give you just a huge thanks and shout out. 

I feel you've built a very inclusive community over the years, the work that we did with glTF. I mean, it was inspired by your community building and your culture there.

Ken Russell:

Thanks. We try to keep things collegial, respectful, and moving forward, sometimes slowly, sometimes quickly.

Marc Petit:

Yeah, that's hugely important. Thank you for that.

Google just released WebGPU into Chrome. Tell us why that's such a big deal.

Corentin Wallez:

It's a big deal because first, it's a milestone after, for personally, more than six years of engineering, but collectively as a team in Google and the standards body at the W3C, I think it's been decades of engineering, if not a century right now.

The nice thing about WebGPU is that it takes graphics on the web to the next level. I mean, graphics and GPU computing in general. The kind of key feature of WebGPU is that it brings compute shaders, which are the better way to represent the parallel nature of GPUs when you want to do arbitrary computations compared to what you could do with WebGL, and you can just do so much more, a lot more algorithms, a lot more flexibility, potential performance improvements, et cetera.

The other thing that's nice is that WebGPU builds both on solid foundations because it builds on the new class of graphics APIs, let's call them modern, that's going to age well. And on the other side, it's built for the web first, so it's more idiomatic than what you see in WebGL.

Patrick Cozzi:

Let's talk a bit about adoption. I love programming for the web because folks are so forward-looking, so innovative, and there are engines out there like Babylon and Three that will adopt things quickly. I’m curious how WebGPU adoption is going.

Corentin Wallez:

WebGPU has been adopted before it was even released. You mentioned Babylon. They built basically a full WebGPU backend years before we released the spec and released the first implementation in Chrome. That meant they had to keep up with the changes we had to do once in a while, but everything was working.

I think right now, Babylon has full WebGPU support. Other frameworks like Three are working on WebGPU support. I don't think it's complete yet, but it's definitely way in progress. Things like TensorFlow.js have picked up WebGPU for their own needs, and they are able to run things much faster on WebGPU. It's more like they're able to do a lot more optimizations than they used to.

Ken Russell:

We've worked with Unity for well over a year on their WebGPU backend specifically. There's been a long partnership on WebGL exports to the web, and now they've built up a fully functional WebGPU backend and are busy getting functionality in there that you simply can't do with WebGL, like Particle System, Visual Effect Graph.

This stuff is predicated on having compute shaders and on read-write storage textures, which other team members graciously implemented and Intel contributed to really rapidly. We're getting the features in; it's straightforward to get them into the WebGPU framework, where it would've been a lot more complicated in the OpenGL pipeline.

Corentin Wallez:

But there's a lot more. For example, PlayCanvas announced they have WebGPU support in their engine, and they're able to do more cluster lighting or something much more efficiently. You see demos using WebGPU popping up left and right. A lot of frameworks are opting to use it. I'm thinking of Apache TVM as well, which is a machine learning toolbox that spans native and web.

Internally, we have a lot of people looking to use WebGPU. Earth had the port for a very long time, Meet is working on the port. Of course, the goal here is not just to port, is to get the improvements you can get with WebGPU. Skia is porting to WebGPU, but we'll get back to that because that's not exactly web, and there's a lot more.

Marc Petit:

Do you foresee adoption of WebGPU by other web browsers?

Corentin Wallez:

Yes, definitely. WebGPU has been a collaborative effort with other browsers from the start. The WebGPU group was actually created by Apple, and Microsoft is in there, Mozilla is in there, and a lot of others, like Intel, are contributing a lot. Firefox is implementing WebGPU fast, and they're trying to release it soon. Of course, we don't know the exact timeline, but they're working on it really hard, and you can test their WebGPU support with some flags; they're pretty far along.

Safari doesn't comment on their progress, but if you look at their open-source repository, you can see that there's a bunch of work going on in the WebGPU directory. And so they're working on it. When they're looking to release that, if at all? We don't know, but they're very engaged.

Ken Russell:

The OpenGL state machine carried us all in the real-time graphics realm forward for what, 30 years, and just an amazing innovation, but it's one big state machine, and the difficulty with it is composing different applications and different libraries together because none of these know anything about the graphic states that the other ones are setting.

When you try to put them together into one big application like the metaverse, which is necessarily going to take tons of user-generated content, it becomes very, very difficult, costly, and sometimes even impossible to abstract all that state setting and make sure that these things interoperate well. WebGPU, built on the new modern APIs, is stateless. This is a huge step forward and will allow more libraries to be written or little snippets of code and useful techniques that actually will all interoperate together.

We're already seeing that in some of these partnerships, where they're working better together, and they're able to pick up WebGPU helper libraries more easily.

Patrick Cozzi:

Ken, I think that's a great point about the developer experience and that the API is more modern and representative of today's GPUs. Plus, the access to compute shaders is game-changing to get that level of compute at the edge.

Is there anything else you'd want to comment on with respect to the developer experience, whether it's tooling or just other things? Because it seems like people are able to get started pretty quickly.

Corentin Wallez:

Both in the spec and in our implementation, we've tried to make an effort on the developer experience. In the spec, we have a bunch of niceties to help developers, like tag objects or parts of their commands or command buffers, et cetera. Then in Chromium, we make a really good effort to give very actionable error messages. We treat it as a bug if an error message does not give enough information to be able to find the bug quickly.

We'd like to have a GPU debugger at some point, but that's a massive effort; it's also something that's floating in our minds. But at least right now, we're trying to make the developer experience good through things like error messages, the integration of promises, and making it really easy to import other HTML things into WebGPU. For example, like zero copy video import is five lines of code or less, and it mostly just works. Just the developer experience is something we focus on.

The other thing is that WebGPU is already a large ecosystem. You have the spec and the browsers that implement it, but you have a lot of people writing tutorials, blog posts about it, et cetera. But also, the two implementations from Chrome and Firefox are separated as native libraries that can be used in their own right in native. People use that. But when they use that, then they can use Emscripten to compile their native application using WebGPU in native to the web. And so that works.

But on the reverse side, we also made Node.js bindings for WebGPU using this library. So you have JavaScript that runs in the browser, you put it on the server, and it can run WebGPU on the server. Of course, all of that is not settled right exactly as we would like it.

But Emscripten is still slightly in flux. The Node.js things are still slightly in flux, but the foundation for a big ecosystem is there. That's a thing that's really dear to us. And it's not just spec; it's building a developer community.

Marc Petit:

Who would want to use WebGPU for a native application? There are probably some interesting use cases right there.

Corentin Wallez:

One such application that's looking to use WebGPU native is Chromium, for example. Chromium uses the GPU for all of its 2D rendering through a library called Skia. We're looking to bring the next generation of Skia's GPU algorithms to Chrome and could name Graphite. The way we're doing this is by building Graphite on top of Chromium's WebGPU library. Chrome's 2D rendering will go through Chromium's WebGPU library. This is useful because WebGPU is a portability layer for the various GPU APIs there are.

If you're making a library or software that uses GPU today, you need to learn about Vulkan and write a Vulkan renderer, and then write a D3D12 renderer, and then a metal renderer. The joke is that in Vulkan, we give you all the pieces to make a Ferrari car, but then you have to build it yourself. It's really, really hard.

WebGPU is much simpler in addition to being portable. All of that combined means that it's very interesting to use WebGPU in native because it lowers your development cost to be portable and to use the GPU so you can focus more on the high-level optimizations and let the WebGPU implementation deal with everything else.

Marc Petit:

That makes sense. So it's some sort of abstraction layer.

Corentin Wallez:

If you look at the Rust world, WebGPU is already the defacto graphics API in native using Mozilla's implementation of WebGPU.

It's really exciting.

Patrick Cozzi:

On that note, I mean, how do you see the future of developing for the web?

Will we develop in Rust and then use WebAssembly? Will we write in TypeScript? Is there a combination? How does it play out?

Ken Russell:

I think it's going to be all of the above. The beauty of the way that things have evolved, most recently with the introduction of WebAssembly and the ability to compile native languages to the web is that it interoperates really well with web APIs. They look like a C function call out into something that's well-defined in a header somewhere. Oftentimes you can actually compile inside the web and then recompile, and it just runs. The rapid iteration of the web development with JavaScript and, to some degree, TypeScript as well, increases productivity massively compared to developing with non-web applications and non-web languages.

It's really the interoperability, and it's the ability to develop some portions of your code outside the web, and script identify them, and bring them in, which is a tremendous boon.

You look back at the Java days. You'd build your Java applet, but it was still somewhat painful to iterate, recompile, and now it's really a reload away from seeing the next version of your code if you're developing on the web. That's a nice benefit to developers.

Marc Petit:

So glTF or USD?

Ken Russell:

glTF is an open standard and has been developed as one since the beginning. USD is a wonderful technology developed by industry experts for the film industry, and it's nice to see the strides being taken toward developing a community around it and making it more open. At the same time, it seems to me that the USD community should have recognized all the innovations that went into the development of glTF, making it an excellent delivery vehicle for the web and, frankly, for the metaverse, and just adopted that for that last mile delivery, at least, if not also interchange.

Patrick Cozzi:

Ken, so you bring up a really interesting point for the delivery there, and Marc and I were trying to do a lot of work within the Metaverse Standards Forum to just share information among the communities for use cases for USD and use cases for glTF in ways that maybe we can transcode and interoperate.

I mean, do you think there is a subset of USD that you think will be efficient and appropriate for web delivery, or do you think the right route is going to be, hey, it's going to transcode to glTF for that final mile?

Ken Russell:

I mean, I'm not a USD expert. I am sort of peripherally aware that USDZ is that subset. I think, as far as I know, it's not that well specified that people are reverse engineering it. And teammate Ricardo Cabello, who runs the three.js project, has written a USDZ exporter for three.js so that you can basically just write your scene and your assets in glTF, and then with one click, it'll export to Quick Look on iOS, and it'll produce, on the fly, the USDZ that is necessary in order to see that set of materials and assets.

So anyway, admittedly not an expert in USDZ especially. Really not in glTF either. So an innovation, Patrick, that you drove into glTF was the transition away from using shaders as the materials, which was glTF 1. It was like WebGL in a nutshell, all wrapped up in your shaders and geometry, and everything was in the same file.

And the problem with it was that it wasn't composable. Those shaders cannot possibly interoperate in the same scene. You and Microsoft developed the innovation, which was the glTF-PBR model, and that all of a sudden made all of the materials physically based and easily composable in the same scene, no matter where your glTF asset came from. That's the predecessor, the precursor of what you need in order to bring many assets from many different places into the metaverse.

I think that core innovation is already there and has been for a number of years, and I think that the industry can just run with that, frankly. Another beauty of it is that in the web, we now have the CPU bandwidth in JavaScript and the ability to send out a lot of data to the GPU with WebGPU run compute shaders in order to do things like skinning entirely offloaded onto the GPU.

I think we have the ability to mutate these assets efficiently on the web platform in a metaverse site to build everything basically on glTF assets.

Patrick Cozzi:

I also want to point out that I am also the guilty person who, in 2012, really wanted GLSL as the materials, but then Microsoft definitely convinced us otherwise, and we saw the way for PBR, and a lot of people contributed to that for where it is today.

Just the compute power at Edge for, like you mentioned, things like skinning. There's even talk of maybe subsurface scattering and... Sorry, well, subsurface scattering but also subdivision surfaces. I think runtime efficiency is an interesting conversation now compared to a decade ago because, one, there's more compute available.

Marc Petit:

I know you guys are graphics experts, but let's try to zoom back on browsers at large and run some hypothetical conversation here because of the instant access of the web and the whole conversations about stores.

I think we see a lot of interrogation about the role of web browsers for 3D in the short term and even in the long term. We look at the internet embracing virtual worlds. So I'm going to ask you to do a little bit of an exercise to think about it.

The first question is really like, I'm a user, and honestly, even simple applications, the web application is not always quite as crisp as the native application. You see that outlook. Let's take an example. You have a fantastic web app, it's really featured, but it doesn't have the crispness of the desktop app. So does WebGPU help?

How do we bridge that gap, or is that gap there forever?

Ken Russell:

Let me give one counterpoint. Figma. Figma is a design application that was built on top of web primitives since the beginning. Its shape rendering was built by Evan Wallace, former Figma CTO, in WebGL, used shaders to figure out how to draw these multi-junction paths, which was different than what Illustrator supported web since the beginning. Adobe saw enough value in it to say, you know what, I think we're going to acquire them to our tool suite. Was publicly advertised several months ago. 20 billion valuation for Figma, which makes the WebGL ecosystem $20 billion plus change.

The point is that I'm not sure that it's really crispness or sluggishness or anything of that nature that distinguishes web applications from native ones. It's really the security model and the portability aspect. Those are two of the major things that web provides as advantages.

It's definitely true that the security model imposes some performance overhead, and the portability aspect probably imposes some corners of the APIs that you may need to cut off and not access, or maybe they need to be optional features in order to get that large device reach. But our goal is to make sure that the overhead is as minimal as possible and that we reach the broadest device deployment that you possibly can with the web. And that's what we're striving for.

Corentin Wallez:

If you look at Outlook, it was developed first as a native application, and building a native application compared to building a web app is a very different endeavor, if only because most of your data is somewhere far away that you need to load on demand or whatever. There's a different engineering culture there.

The other thing is that if you look at other desktop applications like Slack or Teams, et cetera, they're actually using web technologies, and it's because web technologies bring you that amazing portability. 

You can build a native app, but also, you can have Slack in a... I have one in a pin tab on my Chrome instance, and it works; the web has a lot of tooling and developer niceties that make it faster to develop some sorts of applications.

Of course, you pay that with more overhead, like if you use React, things are slower than if you use raw dom operations, most likely, but you gain developer productivity, and computers are fast. It can be a worthy trade-off.

Marc Petit:

I saw some headlines last week after the release of the new iPhone saying, finally, a phone can compete with consoles on graphics.

The question for me is, let's take this as a bar or milestone. How far are we from web browsers hitting similar capabilities, being able to compete with consoles?

Corentin Wallez:

Competing with consoles here, I think, was about basically flops of the hardware and the browser. Of course, if you run a browser on Raspberry Pi, it might not be able to compete with Xbox Series X, but maybe it will compete with a Nintendo 64. It's more like, can you access enough of the hardware power through the browser? And it depends, right? The browser is always going to lag behind native in terms of capabilities just because of the portability.

Web standards need to make sure technology is coalescing native before they bring it to the web, otherwise we risk pushing for something that ends up being a technological dead end, and we have to support it forever or break the web by removing it from the web; that would be terrible.

We have to wait for things to coalesce. I think we can reach good enough console-level quality.

Marc Petit:

I'm convinced, and actually, through the next conversation, everyone established this notion that it feels as an observer that we should not be far from doing something like Fortnite in the browser. Looks like the elements are there, and I think that that would be an amazing achievement to have that level of quality of game, which is not a AAA game. It's close to AAA but not, and have that level of a multi-user.

Maybe we can look at all the components that would be required to do that, right, Corentin?

Corentin Wallez:

Right. You mentioned something like Fortnite, and if you look at the web today, in terms of graphic capabilities, we definitely have what it takes to run something like Fortnite. Fortnite can run on terrible phones as well, but the web can even reach the quality of desktop Fortnite in terms of image quality and rendering and, potentially, GP performance.

What matters to make something like Fortnite now is everything that goes around the rendering. The tricky part is streaming the assets. How do you keep things cached in the browser when the user is browsing the web in many other places at the same time? There's the input latency. How do you get that down, dealing with the authentication of users when they're moving from page to page? All these constraints that are around the core just leads to computations. That's the hard part.

Something like Cesium, for example, tries to, or solves, the streaming use case nicely for that. And so that's one break that's needed, but there's others.

Marc Petit:

In terms of compute performance, we've got WebAssembly with some good support for some multithreaded capabilities. That should be a good boost.

Do you guys see that as an option of WebAssembly going hand in hand with WebGPU, or how do you look at that?

Ken Russell:

Absolutely. I'll point out that Google Earth from the browser and maybe Cesium as well is already multithreaded, and they see the performance advantages in doing that. You can compare Earth's single-threaded to multithreaded building is much smoother when running multithreaded. WebGPU is designed for multithreading basically from the beginning.

Of course, it's not quite there, even at the JavaScript API level; there are a couple of aspects of the API that do need to be tuned. But our team is already working on the multithreading implementation at the native level, and we fully expect to be able to get that reduction in overhead that we're seeing at the native API level on the web eventually because you can offload texture loading and probably geometry uploading to other threads.

I think that we're getting closer and closer to what you can do by tuning for a particular platform on the web easily. Wasm uses the portable SIMD subset that Wasm has defined, using multithreading, and using WebGPU for your rendering and probably accessing it from multiple threads simultaneously.

This is a big step forward compared to what was possible with WebGL, which is inherently a single-threaded API. We didn't even go the route of multiple WebGL contexts because it actually changed the semantics of the API so much that it was not workable.

Patrick Cozzi:

Do you see this blended ecosystem? For example, today, you might have Draco mesh decompression, which Google has done, and then WebAssembly in order to use that in the browser.

But then also folks writing a complete engine or complete application like Filament, that Google’s done, then WebAssembly put that whole thing into the browser.

Ken Russell:

This is absolutely a direction that we think a lot of people are going to want to take advantage of. Yes. 

Can I mention a particular use case of unity? They were working on their WebGPU backend, and they got the whole thing running, but they couldn't figure out a couple of issues and were getting the dreaded black screen bug. Oh, no black screen and computer graphics. What is it? Did I forget to turn on the lights? Is the camera pointing in the wrong direction? Is the final frame buffer blip broken? So many things that can go wrong. And the in-browser Wasm development experience was still coming up and wasn't quite there, and they couldn't quite figure out where the bug was. So they ported their web app back to native and built Chromium's native implementation of WebGPU, Dawn, along with their engine as a native application.

It debugged it there in Visual Studio, and they found a few bugs that were legitimate in their usage of the API; they fixed them, and then they recompiled for the web, and it just worked. That's the power of this combined native and web development environment; you really don't have to worry about where your developments happen.

Do it in the place that best works for you, and then you can bring it to the web seamlessly.

Corentin Wallez:

On the other side, if you are making a JavaScript engine for reasons and you would like to use WebAssembly modules for decompression, or maybe you want to use Filament, but the rest of your engine is in JavaScript, you can make that happen by doing function calls and passing data across JavaScript and WebAssembly.

In the past, this was okay, decent enough, but not the most useful. But nowadays, after the first WebAssembly MVP that was released years ago, there are a bunch of extensions to WebAssembly that make it easier and faster to exchange things between Wasm and JS. I'm thinking of things like the WebAssembly, JavaScript promise integration. It means you can do async in WebAssembly without changing your C code and without the huge cost of Asyncify. There are things coming, like reference types. WebAssembly can move around things that are in the JavaScript heap.

Of course, it can't access JavaScript objects directly, but it can reference them in the value and move them around, and they're still garbage collected correctly, et cetera. I'm not a WebAssembly expert, so I'm sure there's more. But things like this mean we can more easily mix and match WebAssembly and JavaScript, and so, basically, choose whatever tool is best for the job.

Marc Petit:

You guys mentioned the server-side Node.js being potentially GPU accelerated now, and we have technology like WebSocket and WebRTC. You mentioned the latency of multi users.

I think it looks like all the ingredients are there, and the question is, when you put them together, do you get the right level of performance? Personally, I wish that we could see more browser-based gaming. It's so very surprising to me that we have all those components. We have great engines like Three.js, Babylon.js, and PlayCanvas.

Do we expect to see a lot more multi-user 3D experience in the browser? Do you see that happening?

Corentin Wallez:

We would love to see that happen, and we see it happening more and more.

Ken Russell:

If you look at dotbigbang.com, it's in-browser, multi-user construction, and multiplayer.

Corentin Wallez:

But there's this, and there's other games that are just web-based purely, and so that's starting to happen. But talking with some people in the Paris office here who looked at web gaming, the major problems for web gaming are not about APIs to do input or latency or video or graphics or whatever. The problem is very much discoverability, monetization, and user identity.

For example, if you want to play a game on your phone, you open an app store, and you get a game. If you want to play a game on your desktop, you open Steam or the Epic Games store, you find games, and you play them.

If you want to play a game on the web, how do you do this? You search with Google the name of games, maybe, but it's not as integrated, and so there's a discoverability problem.

Monetization on the web is also different because it's usually more ads-based. People on the web expect things to be free. And so that's another challenge.

Marc Petit:

At the end of the day, gamers are attracted by good games, and the accessibility that you get on the web by just passing a new URL around, I think, would be highly valued.

Corentin Wallez:

Would you call things like Worldle a game in that respect?

It was discovered and shared and made super popular because of how easy it was to share with people. Of course, it's not very graphically intensive, but that benefited from the super wide reach of the web.

Patrick Cozzi:

Shall we talk about VR for a moment? We have WebXR out there on the web, and how are you seeing things evolve?

Corentin Wallez:

WebGPU doesn't have WebXR support yet, and that's one of the last few, let's call it, regressions from WebGL in WebGPU because today, if you want to use WebXR, you either have to use WebGL, or you have to do crazy–and please don't do them–workarounds with WebGPU. It's terrible.

We want to get that out soon, but it needs a little bit more work in the Immersive Web Working Group and then implementation work in the browsers. Hopefully soon, and hopefully, it's very useful.

Marc Petit:

We talked a lot about browsers today, so now I want to ask you guys to take out your crystal ball. Always a difficult exercise. Let's look forward.

As you know, in this podcast, we think the metaverse is just the result of the internet embracing real-time 3D and also the web.

From your experience, if you could start from a blank page being given the mandate to create a version of the internet and the web that is optimized for shared persistent virtual worlds, how would you start?

Ken Russell:

Maybe it's instructive to look at the current technology stack. Unreliable packet transport is super critical for low-latency games, and the web has had difficulty accessing that in the past. WebRTC has the data connection, API, which lets you access UDP. That's critical, but I gather that web transport to make it a little bit easier to access this stuff is under development, active developments, and will, I think, be a big missing piece of it.

When it comes to asset delivery, though, you don't want to have to write your own reliable transport layer, you just want to use regular fetch, and the web is eminently well-suited to delivering that sort of content. You just have to wait for potentially very, very large assets to be downloaded.

That's where I think more work on streaming is needed, and level of detail exports in the game engines that you'll get instant on and see something, and then, iteratively, you'll get the higher resolution, textures, and assets.

I mean, that's one aspect of the transport communication layers. I think the technology is really good already on the web for this sort of thing because you have some sort of server serving up your web content already. It's usually an incremental delta to adding some amount of back and forth to the client as opposed to simply delivering assets. Hopefully, it’s not too, too difficult to get a multiplayer, which is an inherent piece of the metaverse, into your existing server infrastructure.

Corentin Wallez:

The composability to move between worlds seems doable almost today. Thinking of portals, for example, even with worlds that are of different origins, because we have a new HTML element called Portal that is able to control how the transition between websites works. I'm pretty sure there would be a way to punch a hole through the canvas of the world you're currently in that's used to show the canvas of the next URL that's loaded behind it.

It would require a lot of synchronization between the two, but probably doable. However, there's a huge challenge to compose multiple worlds in the same page with things that are programmable because there are security and sandboxing concerns there, I would imagine. And so that's still very much an open question.

Ken Russell:

Think about a metaverse instance being hosted on a single website. You can pretty much already do what you need to do today. You can get untrusted assets and code coming from a lot of different origins and running securely within a sandbox. One of the challenges comes... Okay, the web has the single origin security policy, and what do you do when you're talking about multiple origins?

The way that Apple has solved this with the visionOS is you have to delegate all responsibility when you're in the shared space to the operating system. They provide a scene graph, and you must supply your assets in that form to the operating system. You can't program it. You can't upload vertices anymore. You can't dynamically touch stuff in the ways that we expect to be able to, for good reason. 

They're saying, okay, well, we have to have some amount of control over the shared stuff. I think that that's where a big part of the challenge would be in a multi-origin metaverse.

Marc Petit:

We touched earlier in the conversation on the overhead of security. We believe in the next generation of browsers, we will still have to deal with that.

Corentin Wallez:

Security of the user is the top most concern of browsers. In specs, we call them user agents, and for a reason, they are there for the user first. We are all trying to make browsers more secure. I mean, when I say all, all browsers are trying to be more secure while reducing the overhead of that security as much as we can. And WebAssembly was a step in that direction.

A bunch of JavaScript optimizations are that. WebGPU is this, and a lot of effort was put into designing low overhead security for WebGPU. But that overhead will always be a little bit there because that's the number one property of web security.

Patrick Cozzi:

It's been really awesome to have you both on the show today. We've covered so many topics, from the benefits of WebGPU to its developer experience and current adoption to how to transport 3D assets on the web, to what it would take to build a Fortnite-like experience with the web tech today, and also some potential future of the web technology stack.

As you know, we like to wrap up with a shout-out. But before we do that, I would actually like to give both of you, Corentin and Ken, a shout-out.

Marc, you may know I've spent several years teaching GPU programming architecture here at the University of Pennsylvania. And every year, Ken, Corentin, Shannon, Brandon, the Chrome GPU team, always comes out, gives a guest lecture, big supporter of the course, and they've hired three amazing students that I've had, Austin, Kai, and Trek.

As an educator, I'm just so thrilled that these students are out there working with Ken and Corentin and contributing to WebGPU and WebGPL, or WebGL. Thank you both for that.

Now I'd like to ask each of you for your shout-outs to a person or organization.

Corentin Wallez:

My first shout-outs would go to Kelsey Gilbert, who's the co-chair of the WebGPU Working Group and a very active contributor. She's been there from day one, and she's also chair of the WebGL Working Group, I believe.

Also, Myles Maxfield at Apple, who's a speculator of WebGPU and contributed a ton of things to the spec. Discussions can be tough in standard bodies, but I think being all together makes for a better outcome. So shout out to them.

Ken Russell:

I'd personally like to give a shout-out to Intel's web graphics team in Shanghai. This team, for many years, has contributed enormously to Chromium, the graphics stack in particular, to the WebGPU implementation.

We wouldn't be where we are today without their tremendous contributions and collaboration. Thanks to that team for investing so much on the web.

Corentin Wallez:

Two other people I'd like to give shout-outs to, actually.

WebGPU is like browser implementers, but it's also a community around the tech, and two people in the community stand out by either building things or helping other people.

The first person is Connor Fitzgerald. I'm not sure what his day job is, but he's one of the maintainers of the WebGPU library used by Firefox. He's been a constant help trying to figure out how to implement things, helping other people to use WebGPU or to use graphics APIs underneath WebGPU, et cetera. A huge shout-out to him.

The other person is Jasper St. Pierre. They're the person behind the noclip.website virtual museum for video games. And likewise, they contributed a ton to the spec. They are helping users that have questions about graphics, they give best practices, and they're just a pillar of the community. That's amazing to see.

Marc Petit: 

Thank you, Ken Russell and Corentin Wallez from Google, for talking to us today and shining a light on web browsers and the important future they could play in 3D graphics. Thanks in particular for your contribution. Thank you so much for that.

Hugely important for every one of us that the web plays a big role and will play a big role and will be the metaverse. Browsers are a critical component of that. I hope that the conversation was enlightening for everybody. Thank you so much.

Also, a  huge thank you to our ever-growing audience. You can reach us for feedback on our website, buildingtheopenmetaverse.org, as well as subscribe to our LinkedIn page, our YouTube channel, and most podcast platforms.

Thank you, Corentin. Thank you, Ken, and thank you, Patrick.

‍