Texturing the Metaverse

Announcer:

Today on Building the Open Metaverse.

Sébastien Deguy:

And the metaverse, by the way, is only whatever we think about the word. It's only creating more appetite for creators and companies alike to have more content. And when you think about, also, I was thinking about that the other day. But maybe I'm late to the game. But I was realizing, well, actually there is a limited space on earth. So there is a limited amount of things we can do with land, with a number of shoes or a number of objects we can sell, manufacture. And by the way, we should be very careful about what we do with the land, what we build and how we exploit the earth and all. But in the virtual space, of course it's using energy as a trade-off. But when it comes to the space itself, it's virtual, infinite.

Announcer:

Welcome to Building the Open Metaverse, where technology experts discuss how the community is Building the Open Metaverse together. Hosted by Patrick Cozzi from Cesium, and Marc Petit from Epic Games.

Marc Petit:

Hello everybody, and welcome to our show, Building the Open Metaverse. The podcast where technologists share their insight on how the community is building the metaverse together. My name is Marc Petit from Epic Games. And my cohost is Patrick Cozzi from Cesium. Patrick, how are you?

Patrick Cozzi:

Hi, Marc. Doing fantastic. Looking forward to today's topic.

Marc Petit:

Yeah, today we have another very special guest. He founded his company straight out of school, more than 20 years ago, and has had a deep impact on the computer graphics industry. He now holds a position which will allow him to make even more of an impact on that industry, as he's leading 3D development at Adobe. It's my pleasure to welcome Sébastien Deguy, founder of Allegorithmic and VP for 3D and Immersive at Adobe. Welcome, Sébastien.

Sébastien Deguy:

Thank you, Marc. Thank you, Patrick, for having me today. I'm very happy to be here.

Marc Petit:

Yeah. No, we're super happy to have you with us. Sébastien, you're both a scientist and an entrepreneur, and a musician, and a bunch of other things we won't talk about it here. But in your own words, please tell us your journey to the metaverse, and how you got where you are today. I think it's fascinating for people.

Sébastien Deguy:

Okay. Well, it's a long journey, as you say. I completed my PhD more than 20 years ago. Actually last December was the 20th anniversary of my PhD. It's been a long time coming, I would say. Yeah. As you say, I created Allegorithmic just after I completed my PhD. And just before I actually studied my PhD, I was very passionate about computer graphics, video games, movies, a lot of things. Everything images, really. And I remember in my room back in France, and you've been there as well, but basically trying to catch every piece of information about this space and computers. At the time it was very complicated because I was living on the countryside, so I would need to actually ask my mother to bring me to a kiosk and buy magazines about computer science. Mostly games, actually, I have to admit.

Sébastien Deguy:

Basically, I started there. I started developing a very strong passion for it. I even developed a program that takes two input videos and turn that into an anaglyph, the ones that you wear, a sign and the red glass, and you see it in relief. I started trying to apply what I knew about computers and computer graphics and computer science into how to make movies, how to make visual effects, and have fun with it. So it started there.

Sébastien Deguy:

And then, I studied math and applied math, and I did a PhD in applied math and computer science in this field of random processes. One cool application of the mathematical model I was working on at the time was for... It was used for simulating complex phenomena, like clouds. But one thing you could also do is, if you don't do cloud in volume, which also on computers at the time was complicated to generate, because you needed a lot of computation power, you could generate 2D images of such clouds. And when looking at the studio images, you would see what you call noises now in computer graphics. And I'd been asked to teach a program that was called Softimage 3D. Before Softimage, actually, you know that very well, Marc. But Softimage 3D.

Marc Petit:

I know a thing or two about it. Yes.

Sébastien Deguy:

Yeah. You know a thing or two. Softimage 3D, basically, I was asked to teach that tool, and it was late '90s. At the time it was the best you can find out there. It was amazing tool. And I had to learn it before I actually taught it, so I was one week ahead of my students. Basically, what I saw in the tool was you had these steps in the process. You had the modeling, you had the animation, you had the rendering. And on the rendering side of things, you could apply textures and materials. And one way to achieve that was to use noise functions. At the time there was the fractal noise, the Worley noise, the Perlin noise, and all the noises you know now.

Sébastien Deguy:

Actually, I made the connection. The first connection I made between my research and the world of computer graphics was at that time. Because I realized, okay, what I find in Softimage 3D and what I can do with my prototype, like trying to simulate clouds. Actually, I could do the same and more. I started thinking, maybe, oh, wait a minute. It's really passion of mine, and maybe I could apply the research I'm doing for this specific field. I started talking in the field of computer graphics about the math behind it. And then the head of the R&D from [inaudible], contacted me. He told me, "How do you do that? Come and tell us a little bit about what you do." This is where it planted a bug in my ear, basically of, maybe there is an interest from the actual world of VFX, which I was completely mad about. And the world of science I was having my PhD in.

Sébastien Deguy:

As soon as I completed my PhD, I started the project of building a company that would become Allegorithmic. At the time I thought it would take two years and two guys to just come up with something that was maybe guys power tools. I don't know if you remember that one, but that was just having fun and producing cool stuff. And it took 10 years. And more years in fact, but 10 years before we actually got some success out of it. And a lot more people than two. A lot more trouble as well.

Sébastien Deguy:

But yeah, fast forward, then we started a company in 2003 officially. And I completed my PhD end of 2001. Started this company year and a half later. And then we grew with what became Allegorithmic at the time we joined Adobe in 2019. We were 140 people when we joined. And in the meantime, we created the tools that are known as Substance. And especially as the Substance designer, Substance painter, and then Substance sampler and more. And the Substance source library of materials. We focused on really first applying the math that was developed, the noise functions, and then expending to everything materials and textures. And Substance designers started to have success in 2011, in fact 2012 even. So it took really 10 years to get to a point where it started to have an impact. Mostly in games at first, not VFX to my dismay. But then came back to the VFX world.

Sébastien Deguy:

And now all the Substance tools are used by 90... Last time we did the count it was 97% of AAA game developers. For the past five years, I think, all the VFX Oscar winners use Substance in some form. Sometimes short, things here and there, but then a lot more with Dune and Blade Runner, and this type of movie anyway. Sorry, it was a long answer. And then in 2019 we joined Adobe, where we started this new division called 3D Immersive. I don't know. Maybe too long.

Marc Petit:

No. I mean, I can attest that when I was at Autodesk you were coming every year, or every six months actually, dropping by Montreal and showing us what you were doing, and the progress was there. The thing I remember that was very specific about what you and Allegorithmic is the support you got from the team. Even if there was not a lot at the beginning, there was always a lot of belief that you guys were onto something big. And we did the deal that you remember, and all of those things.

Marc Petit:

I think what people need to realize is how much resilience you need to have. It's, again, an overnight success 10 years in the making. Something like this. Sébastien, I remember that you've been so resilient. You've been there so humble every time coming back every six months showing progress. I think that built the credibility and the trust in people, and they are willing to stand up for you when you need them to stand up.

Sébastien Deguy:

Exactly. It is very good to hear. By the way, I started writing, just like you, some kind of a book for Allegorithmic. Part of the reason why I went into random processes at the very beginning is that I had an accident, I lost some part of my memory. So I got very interested in this idea of memory and randomness. And, are we predictable? Because I remember going through a phase where I would go through the exact same cycle of questions after the accident, go through the exact same cycle of questions that ended up with the same joke because I was in the same environment. So every three minutes my father would tell me I was waiting for the MRI. And I would... "Where am I? What happened?" And then a joke, something. And then three minutes later, "Where am I? What happened?" And then the same joke.

Sébastien Deguy:

I realized, okay, are we predictable somehow? So I started asking myself the question, does randomness exist really? And my PhD advisor at the time had told me, "Welcome. Finally, you're one of us, asking the right question. Does it exist?" And so, I still did it. Anyway. I started writing this because, well, it's a good excuse for me to say, "Well, I had an accident so I have memory issues now." Anyway, so I'm writing this thing. I found a very old photograph. It's funny you mentioned this, Marc, because a very old photograph where there is the Autodesk. I think it was a discrete booth, even, at the time. And I'm sitting on the edge of the photo, you see some guy laying on the ground, sitting there with a huge laptop. That was me.

Sébastien Deguy:

And I remember, okay, I must have been waiting for you to talk to me, or have a chance to talk to you and show you the stuff. Yeah. I mean, to your point, it takes a lot of time, and it took a lot of time for us. I'm not saying I think it's a general rule, but it took a lot of time. It took a lot of resilience, that's very true too. And I think if there's one thing, it's this one. And of course you need luck. Of course, you need to be good at what you're doing. But at some point you need to work and be resilient, and so it's the big one.

Marc Petit:

So Patrick, let's geek out now.

Patrick Cozzi:

For sure. So Seb, first, I mean, congrats on all your accomplishment in both research and in business, these are great lessons learned. Thank you for sharing them with the community. But let's jump into Substance. I mean, can you tell us more about what the end users do with it, how it incorporates your research, what problems it solves?

Sébastien Deguy:

Right. Substance today, I mean, for the texturing products, and now we have more products. But the texturing products allow for people to create what we call textures or digital materials, which is when you model a 3D scene, a 3D environment, you go with the shape first. We see this empty naked shapes in 3D floating in a 3D space. But then when you want to give it the aspect of wood, you want to give the aspect of skin or fabric, or any material that there is in real life. When you want to have something realistic or not, but that looks like it's a wood or an expression thereof. You need what we call textures, which are images that you're applying onto your 3D object. And these images describe how the light will interact with that surface, and giving you, your eye, your brain, the impression of, okay, this is wood because I recognize the colors. I recognize the patterns. I recognize the way the specularity of how it's reflected, if it is, or transparent it is, et cetera.

Sébastien Deguy:

We have a few tools dedicated to this specific area, which is very narrow in their niche area, but it's necessary in every 3D world and representation, or experience. And so Substance designer is a very technical tool that lets you do that with a node-based approach, a procedural approach that has its advantages. And then Substance painter is some kind of a Photoshop in 3D, where you have a layer stack. And instead of painting with color on a 2D canvas, you paint with materials on a 3D object. And so you extend somehow the idea of the artistic gesture and the Photoshop workflow in 3D.

Sébastien Deguy:

And so artists have been loving the tool for that reason. I mean, it's not the first one to be able to do that, but it was one of the first ones, if not the first one to actually be a truly completely GPU powered, full 3D, full material painting application. So you would paint the all the layers, all the information textures at the same time in one stroke. And so it was nondestructive, et cetera, et cetera. And then you have Substance sampler that lets you, from an input among other things, you can do many things. But one thing can allow you is, and it's very strong about, is take one picture of a material that you like in the street, and turn that into full material in a very simple way. This is for the materials part. And so that's historically what Substance has been about. And now we have more products.

Marc Petit:

Yeah. And it's interesting because it completely blurs the limit between surface geometry and shading. I mean, you can add stitches, you can add a level of details to your object and simplify the process of creating the shape. It's been fascinating to see the sophistication of that technology. Actually, you've been for 20 years at it. Is there still a lot of innovation to be done in that space? Are you running out problems to solve?

Sébastien Deguy:

Well, I think we will never run out of problems to solve, either in that field or others. Yeah. I mean, it's a fairly mature... Actually, it's interesting, because the 3D space as a whole is not that mature, and so there's still a lot of innovation to be developed. Which is interesting. Also, it's a bit of a curve because it means that we are a little bit far still from total democratization, because you still need a good GPU. You still need good hardware. You still need good input. You still need interactivity. And so, there are ways to reach this by a new way, but it's still very demanding. It's a demanding space. And it's a complex space as well because there is one more dimension, it's involving way more complexity in terms of interactions and filters, and all the mathematical functions you can apply in this space anyway.

Sébastien Deguy:

Yeah. I mean, when it comes to texturing, there are still many things to work on. And standardization is one. How to make sure that we have more complex and more complete ways of describing some surfaces that we're bad describing right now. We've been doing a lot of progresses in the past years, especially thanks to the physically based rendering techniques and the mechanism and schemas that have been developed. But, yeah, there is still a lot to be done. And now, of course, it's converging a little bit. Right? And so it's less of a complete blue ocean open to total reinvention, but still a lot of fun to have, I would say.

Patrick Cozzi:

Seb, given how widely used Substance is, I mean, it feels like it's really the defacto standard for describing these materials and textures. Have you thought about open sourcing parts of it?

Sébastien Deguy:

Yes, we always have the discussion. The framework we have in mind is always the same. How can we help the ecosystem? And so, I think we're on the same page in that sense. How do we help the ecosystem? Of course, we're a business, so how do we help the ecosystem in which every actor has to find its way? And including us, right? It's an ecosystem in a sense that it has to benefit everybody. And so we always try to find, okay, well, are there any ways to make sure that, for instance, the Substance format itself could be used more widely? And lately, I don't know if you've seen, but we made available the SDK for the Substance engine.

Sébastien Deguy:

Now anyone can just go to Adobe.io, I think it's the address. And you can download the SDK, and you have all the documentation, you can write your plugin to connect your own application with a Substance engine so that you can then generate the texture. So it's a start. It's the first step. We very much like the open source community for various reasons, and we support it. We actually provide a lot of support to a lot of open source initiatives. We don't talk about all of them, but we do support all the ones that we use, for obvious reason. And then we try to support more, and lately we've been supporting Blender, which is a good example as well. And we have also help develop this bridge between Blender and the Substance applications, because a lot of, actually, our users are using Blender as well.

Sébastien Deguy:

And so the format itself, we've been looking into it, and it's always on the table. We keep turning around, deciding or not deciding. Lately what happened with MaterialX, USD, and other formats. But USD and MaterialX, especially, is of the highest interest to us. Because maybe finally we're onto something that any company could use, and we could exchange things. And the reality of things, back to what I was saying before, we're still in that early age, I feel, of the metaverse and 3D, and experiences. I feel like there's still so much to be done.

Sébastien Deguy:

And you, as a user, what we see is that studios mix a lot of tools coming from a lot of companies. So if we just being realistic and honest with our user and clients, we have to take into account the fact that they won't stay in Adobe only products, and they won't stay into, I don't know, any other company only products. Usually it flows, so it has to flow. So finally, maybe with USD and MaterialX we're onto something that could be leading to a place where, I mean all these materials, all these models, or potentially all these experiences could flow.

Marc Petit:

Do you think you could, in the current state of MaterialX, you could transport all the sophistication of the Substance content?

Sébastien Deguy:

Not everything right now, but we try to make sure that it's easy, feasible if you have some kind of Substance extension to MaterialX somehow. Yeah. I mean, the discussions are ongoing about, should we want to make that more a default thing? And if so, do we need to open source it? Do we need to open it? So these discussions are ongoing. But yeah, definitely we see the interest. We don't want to fall into traps. Also, for legal reasons, it's a complicated thing, but definitely on the table. Yeah.

Marc Petit:

That's great to hear. I mean, because SBS R5 are right now the defacto standards.

Patrick Cozzi:

Seb, in general, I love your perspective on ecosystems and trying to rise a tide for all ships, and connect all the parts there. So, great work.

Marc Petit:

And I think it's one thing we like to underline in this podcast is how people like to work together in our industry. Because it's a given for us because we've grown, we went to SIGGRAPH And we know that we like to work together. And even though we work in competing companies, we're made out of the same wood. I mean, we're part of the same community. But it's not true of every industry, by the way. And I think it's something we need to celebrate of our industry and carry to the metaverse is that appetite for collaboration. We can compete and collaborate, and may the best man win. But I won't name industries, but we all know other industries where data hoarding and there is no spirit of openness and collaboration, actually. But it will change.

Sébastien Deguy:

I agree. I agree. I agree with you, we should celebrate this. Because, to your point, we've all been passionate and fascinated by the first movies. Right? When I saw Tron, the first time when I saw, I don't know, Jurassic Park or Terminator, we all were in awe with what we saw. And same goes for computer graphics and video games. I mean, of course we do compete. Yes. But also, we have the luxury of having this fast expending space. I think there is space for everybody. So if we're smart, then there is space for a lot of people. And a lot of companies. And a lot of ways to address that demand of creators willing to adopt that space.

Sébastien Deguy:

And enter that space. And the metaverse, by the way, is only whatever we think about the word, right? It's only creating more appetite for creators and companies alike to have more content. And when you think about it, also, I was thinking about it the other day. Maybe I'm linked to it again, but I was realizing, well, actually there's no limit in there. I mean, there is a limited space on earth. There is a limited amount of things that we can do with land, with number of shoes or a number of objects we can sell, manufacture. And by the way, we should be very careful about what we do with the land and what we build, and how we exploit the earth at all. But when in the virtual space, of course it's using energy as a trade-off. But when it comes to the space itself, it's virtual, infinite. So there is an opportunity we've never... A scale for this opportunity that the human being has never encountered before, when you think about it. It gives me vertigo sometimes.

Marc Petit:

Yeah. No, it's fascinating. And you create many orders of magnitude of opportunity. So as you said, it's going to be a boon for all of us at all of our companies. Let's switch gears a little bit and talk about your company. So Adobe is a relative newcomer to 3D. You mentioned it really started with the acquisition of Allegorithmic in 2019. But they seems to be now moving by leaps and bounds under your leadership. What are your goals for the division that you created, and what role does Adobe want to play in the metaverse?

Sébastien Deguy:

Yeah. I mean, Adobe has always been interested in 3D. There has been a lot of initiatives actually along the years, for quite some time. But it was always limited by, I would say, the internal appetite for that space. Which was seen as very technical, maybe too narrow, to nascent for a company like Adobe. But the way it's seen now, it has changed in the last year or so, or two and a half... I mean, three years. Because with the acquisition, yes, actually it was the first... I mean, it wasn't the first one, but it was a significant move towards, yes, we want to build something. And because at the same time my company was acquired, this new division 3D Immersive was created, and I had the chance to lead that. And basically, in that division we gathered all initiatives, almost.

Sébastien Deguy:

Yeah. Almost all, yeah. And then we could hire more people, we could acquire a company called Medium, a small tool that is becoming a modeler. And we could grow from there. And that we working on integrating and then we release something. It has changed very quickly in the last year because a few things happened. One is, we released the first offering that Adobe has done in the world of 3D, it'll be Substance 3D. So we've released something which is already something special. It's it takes so much work to release something in big cooperation like this, and to integrate and connect, and make sure that everybody's aware and working. And when we sell, it's actually bringing money on. And it's legal, et cetera. Right? So there's a lot of work involved there. And then, actually, it went well.

Sébastien Deguy:

That means that we constantly surpassed our objectives, which also is a good thing because that's showing traction in industry, and that's showing that we didn't aim wrong. We aim right, in fact, with our tools and pricing, and way of communicating. And then also, a lot of companies came to us and said, "Adobe, we love you. We'd like to go 3D now. I hear you're doing something, can you tell us more?" When our CEO hears about this, I mean, it is great because he's seeing that there is some momentum again in the industry. And some of these big companies are setting the tone for the future. And not only in entertainment, by the way, it can be in product design and everything. And then the metaverse happened.

Sébastien Deguy:

And then the metaverse, I mean, not happened, but the craziness and the change of name of Facebook from Facebook to Meta. I mean, it puts so many eyeballs on the subject. And then what happened is at the market, Wall Street. And the analysts started to ask companies like Adobe, but not only ours, "What is your play in the metaverse?" And so we needed to have something. So we were in the back waving our hands and saying, "Well, we're ready. We know what we should be doing." And we've been successful, and you've seen traction, et cetera. Anyway, now it's a different time because Adobe sees that as the next big thing. And the same way Adobe's been trying to support creatives and companies embrace this new digital printing revolution, time with PostScript and then PDF. And then digital imaging with Photoshop and more. Illustrator. And then video with Premier and After Effects, et cetera, et cetera. This new space of 3D and the metaverse, call it the way you want, is definitely a new space where Adobe understands, okay, well, there will be a need to go there.

Sébastien Deguy:

Creatives want to embrace that space. We, as Adobe, our DNA is to empower the creatives and the designers of the world who want to embrace that new space. So it's not the only natural evolution for Adobe, it took time. But then, because it took time, it became really something organic and very deep. And now it's very deep and very rooted. And I have the chance to be at the right space at the right time right now. Right spot at the right time because we're supported. We announced an acquisition Monday, a few days ago, and it's hopefully only the beginning. I mean, compared to Epic and other companies, of course we haven't announced that many acquisitions in the past. But it's happening, it's more organic, but it's very rooted now. And so, yeah, I mean, it's great to be in that position now because Adobe is very serious about it.

Marc Petit:

Actually, we can talk about this. And congrats on acquiring Brio VR, a company founded by Dave Cardwell, who was the founder of Mudbox. I know well, I bought his first company Skymatter when I was at Autodesk. Great gang of people. Patrick, it's all web-based technology. I mean, it's all web GPU renderings and product utilization and VR. What are you going to do with this, Sébastien?

Sébastien Deguy:

With this one, it's interesting because it's twofold. I mean, interest is threefold. First, it's amazing talent. And so, Dave, himself, is a very interesting guy. Incredibly interesting, in fact. And the whole team at Brio VR and Jacqueline, they've been building something very special, and they're special. We wanted to grow, and so it's a great way to grow with talent very quickly. And that level, the caliber of the people there is amazing. And so then it's twofold, we've seen a lot of interest for 3D experiences and 3D configuration in the world of commerce. And so we wanted to expand what we were doing on the web. We already have something, but we wanted to go one step further, basically.

Sébastien Deguy:

That's the objective here, is to make sure we have the technology stack and the talent so that we can come up with something that allows for more web-based workflows, and that we can leverage the web and potentially the cloud as well. Which is something that we're not very good at right now, we have desktop applications. And so we needed to expand our knowledge, expand the team, and get more technologies and potentially products. Yeah. I mean, the goal here is to, again, connect the world of 3D creativity, that is very deep and rich and that can use our tools, with more people and more use cases, in fact. And some of these use cases have to be web first, let's put it this way.

Marc Petit:

And so you have a complete set of products now. You've announced modeler, you've announced stager. What's the vision and where do you think this is going, a new creative cloud?

Sébastien Deguy:

Well, yeah. I mean, it's funny because when we started discussing with Adobe a long time, before acquisition even, we were really liking this ecosystem approach of theirs. And of ours now. And something we appreciated was the fact that it was a galaxy of tools instead of a universal tool. And because, both have their advantages. But a universal tool, you can only stretch it so thin, at some point it breaks. And if you wanted to make it too many things, it becomes really complicated to manage both vertical and horizontal at the same time. Well, it's my view anyway, but there are counter examples that work very well. But if you want to go a little bit further than the current community of 3D creators, we feel you have to have potentially more specific, focused, smaller, dedicated tools that connect well with each other, again. And in an ecosystem approach a galaxy of tools.

Sébastien Deguy:

And so, yeah, the creative cloud of 3D is... Yes, because we have many tools already. And you announce, you said it, we have a new modeling application and sculpting modeling application in modeler. Which is going to be, I mean, it's amazing already what the guys have been doing. We have people coming from the Medium team and Dreams also. And with us around, we just watch them and we basically give them food and water. And we just adapt to what they come up with, because it's really amazing. So that will be a good one. But it can do only so much. So if you want to have potentially polygonal modeling, like good always and control to every triangle, maybe it's not the best tool for that. So you still need something else.

Sébastien Deguy:

But then for some people, for some use cases, this might become the best tool-

Marc Petit:

And for the rest you have Blender.

Sébastien Deguy:

And then for the rest you have Blender, exactly. Or all the tools. What we're missing still is animation. We have a tool, but it hasn't been evolving a lot in the past years, unfortunately. Whereas it's used by so many people. It's crazy. Actually, it's one of the biggest success in the 3D world. And so many people use it. It needs a little bit of a refresh when it comes to the technology and things, and so we're looking at it as well. And we're looking into what to do with it. Yeah. Now we have end stager then, which is this simple staging, rendering application. What you guys have been starting doing with... How's it called?

Sébastien Deguy:

Sorry. Your ...Twinmotion, thank you. Yeah. Solution, same idea. How to simplify, how to streamline a staging step like PowerPoint for 3D somehow. Where you can import very quickly, place very quickly, snap objects and have physics, and that kind of thing together to create an image very quickly. All the experience of matter. And so stager is this one, and we developed a very fast path tracer that connects well with all the nice material definitions that we have. And some of the latest development that we did with research, as well as Adobe research on the interactive displacement, micro displacement technique, which is absolutely amazing. Also, one thing that we developed for hair and fur lately that we showcased, I mean, we did a sneak peek last week. Yeah. The idea here is to get to an image that is as realistic, as compelling as possible of a scene or an object. And that's stager. But yeah, for animation, for character animation, we have nothing right now.

Sébastien Deguy:

But the goal, to answer your question, the goal eventually is to get to a point where you have as any options as possible in what we offer. I don't expect it to be anywhere complete in the next very years. Because it, first, takes time. And then again, back to the point I was making before, I think we want to allow people to go somewhere else and come back if they have to, or go somewhere else and then stay somewhere else. But at least at some point they use our tools and that's fine. The goal is to have this complete offering end to end. And one good thing about Adobe is the connection with the creative cloud, so the actual 2D products. So being able to connect very quickly with a Photoshop or Illustrator, which we do already with a tool like Substance sampler. It's amazing that connection when that can happen, because that's bringing a lot to the table.

Sébastien Deguy:

And then on the other side of the spectrum is everything Adobe's doing on the delivery side of thing with OEM, and this type of tools with analytics. For companies like, let's say a product company, willing to have feedback on their offers and their experiences, and the images, or the 3D experiences that they have on the website for their products. It's great if it's 2D or 3D, right? For them, it's actually better if it's 3D as well. There is a connection to be done here. It's a lot of work, but that's the good thing about a company like Adobe, because of creative cloud and the marketing cloud as well.

Patrick Cozzi:

Seb, we appreciate everything you're doing to empower more and more creators. I mean, we agree with you that we want everyone to be a creator. And we see that theme with the metaverse and across the podcast. Your approach of having several products that are focused and interoperable is interesting. I wanted to switch gears just a little. Earlier you explained how you're doing procedural textures in a 3D context. And I wanted to ask, are you applying any procedural to the 3D modeling to creating the actual surfaces, whether it's for the hair you just mentioned, or maybe the displacements?

Sébastien Deguy:

Yes. Absolutely. Procedural techniques are at the heart of everything we've been doing. And we love procedural approaches. A year ago, I think maybe nine month ago we integrated the in Substance designer, the ability to export meshes, procedural meshes as well. And so you can do that to some degree in Substance designer already today, and it's evolving, it's evolving fast. We also have internal research projects that are mind blowing. Along this idea of, again, parametric generation of content and models. Modeler is also using SDF approach, so it's very mathematical, although they're turning the tool into something that is artist first. And so the artist will forget about the fact that they're in 3D, and they're in volume in fact, which is making a lot of sense when you think about it.

Sébastien Deguy:

So they don't have to care about actually the topology and the triangles, et cetera. I mean, the procedural approach to us has many advantages. First, it's one way to scale. So when you have one template definition of one thing, you can generate billions. So that's huge. One, you can also have templates, you can hide the complexity of the thing by exposing only meta parameters that describe some human language described characteristics of a material, or a shape now. Let's say more, I don't know, more ripples or less ripples, or more knitting, or that kind of thing. Or more stains or that kind of thing. And so that's also interesting. When you think about back to the point I was making earlier, bringing access of 3D to more people, there is absolutely a need to have a few step process in between the people willing and capable of producing the templates, describing the very process of creation, the procedure.

Sébastien Deguy:

And then people using these procedures and just tweaking a few parameters, but having an eye for what it should be and how it would fit into a scene. Also, one thing we don't talk about that much, which is very important to procedural texturing and procedural content generation in general, is the idea of consistency. When you generate, for instance, on one of the Star Wars, or whatever. I think it was maybe at ILM. Basically they were saying they needed to generate humongous numbers of textures. So the scale was important here, but when you picture not only this type of material, but this type of material, and this is done by different people. At least if you follow certain rules that are defined by sub patterns, sub graphs, then you can control very easily at the end of the process the consistency of them all.

Sébastien Deguy:

For instance, let's say I will do meta parameter that drives the amount of rust on these metals. Maybe if I have a thousand models or having a thousand different materials, several materials. Then I can have one slider that says, change of rust for all of them, the whole scene now. So it's meta meta, right? So it's very powerful that way. I mean, it's not seeing this actually happening. It's very exciting.

Marc Petit:

It's interesting. And the reason why we like to proceduralize is because we know it's going to play a big role moving forward. But from a standardization perspective it creates a problem because you now have to standardize execution. Guido Quaroni when he was with us a few weeks ago, mentioned that you plan on building all, if not most, if not all of our products on top of the USD Sync graph. I mean, isn't that a bold move from a company like Adobe, given where we are with USD right now?

Sébastien Deguy:

Yes, I agree with you. This is the objective. It is so bold that we might take it step by step. And so, not adopting the sync graph in the first place, but maybe just really ringing in, ring out. And so at least maybe converting into our own data model internally to do our thing, and then exporting to USD and respecting the description model. To your point, the execution side of things, this is where... Back to also, by the way, the question about this VSAR, same thing. If you want to control that you have to have the engineer where... So the first step was to open the SDK. And the SDK, so you have engine that ensures that the generation is the same everywhere can be available everywhere.

Sébastien Deguy:

So as many places as possible. This is important, because there are many things that we do in the engine that would be a waste of time, I think, to just trying to replicate or just use the engine itself. Yeah, to your point, USD, definitely we see that as a big, big opportunity. And Guido being here is important. Obviously he's been driving this, and is the leader of that and one of the founding father of USD. So that's great to have driving this. But it will come in steps. Yeah, some of the newest tools and some of the newest initiatives that we have, they really revolve a lot around USD. Yeah.

Marc Petit:

And so one topic that comes around this podcast regularly is, USD's open source library, but it's guided by a single entity. And basically you've got a single entity who decides what makes it to the library and what does not. First one, are you comfortable with that? And how would you like to see this evolve? We've heard a few different version of that desired evolution, I'm curious to have your take on this. I know it's a little bit of an uncomfortable question.

Sébastien Deguy:

It is a bit. Not only because of the very nature of the question, but also then my position. But also the fact that I'm not that opinionated about it actually. I mean, I would be fine either way, because sometime I know for a fact that de facto standards can work as well. And I know it's uncomfortable for everyone in that case, versus being uncomfortable to only a few companies. I mean, both can work. I know we have very good relationship with Pixar for various reasons. And we work with other companies to make sure that... I mean, to try to enforce one USD that no strong evolution and departure from that very description and that very core idea.

Sébastien Deguy:

So we're working to make sure... We are talking all the time, right? We're talking, trying to make sure that there is a divergence. Yeah. I mean, the short answer is I'm finding a way. I'm not opinionated, maybe I should be more. But we're trying to make sure that it keeps its promise, because it could very quickly die if suddenly there is a branch, there is a version that differs from the others, and then there is separation. Then it's over. Then it's one more format, one more standard.

Marc Petit:

I know. And thanks for your answer. I think it's complex because they should create procedural objects. You would want the procedural reason to be yours because it would support it natively, but then other people may have other ways. And the other conversation we're having here often is at the other end of the spectrum you have an open standard, which is not open source. An open standard like glTF, which seems to work very well and have done wonders in the commerce space. How important is glTF to Adobe?

Sébastien Deguy:

Well, it's super important. We realize that it's spread out and it's used by pretty much everybody. To us, as an export format, yes, absolutely. We support it and we've always... The same thing, you have had a strong relationship with Khronos and Neil [Trevett] and the whole team there. And so, yeah, I mean, it's an important aspect to us. So when we say we want to support USD, it's not at the cost of not supporting anything else. Definitely USD, we see that as an authoring format. And USDZ, or NGLTF definitely as output format for any, in terms of applications. Yes, absolutely. These are the two big ones actually, FBX, we keep supporting FBX as well, because it's -

Marc Petit:

30 years later.

Sébastien Deguy:

Exactly. Right. But it's ours. And by the way, going back also to one of the things I was saying about Adobe, I mean, it's not always been true. But anyway, I like standards. I like to support many standards that help, again, the ecosystem. Again, not only because I feel it's the way we feel in the team, very much, very profoundly. We like to support the ecosystem. But also, because if I want to just wear my businessman hat, right now the state of the industry imposes that we connect well with the rest of the industry. Because we're not in a stage where we can say, well, we take it all and we close. It doesn't work that way. And we would just fail. So to me, it'd be an error basically.

Patrick Cozzi:

Seb, Substance was a pretty early adopter of glTF. It was actually pretty eye opening for us because we designed glTF first to be really efficient at runtime. And then we saw that Substance first added import for glTF. And we said, "Ah, wait a minute. I guess there's so many glTF in the wild that people don't just want to export, but they also want to import."Have you ever thought about it from that angle?

Sébastien Deguy:

Yes. One of the objectives there was to support platforms like Sketchfab. Basically what we want Sketchfab, they're good friends. And so what we always wanted to do, we've seen this number of objects. And we thought, okay, it's actually great. Because one of the hardest thing to do in 3D is to actually model and create something. But one of the funniest thing to do is actually paint. And 3D paint especially with the Substance painter. So one way for us to open up for people to start using Substance painter, having fun very quickly without having to go through the very tedious phase of creating something or finding something, was to connect from within Substance painter to a platform like Sketchfab.

Sébastien Deguy:

And then import a glTF and then paint over it. And so this is also why we developed the auto UV framework, so in case there was no UVs. It was a way for us to bring more people into Substance painter and have fun very quickly. And so that was one of the original thinking behind it. And then we realized, wow, I mean, it's a format, so it should be flowing as easy as possible.

Marc Petit:

Yeah. And one of the point that Patrick and I, we try to propose is that, as you said, glTF has good transmission format, uses good authoring format. Let's at least evolve them in parallel in a way that is compatible and synchronized, so let's make sure that we actually talk. And when we start adding variants and physics and logic, let's make sure we do it in a way that it remains natural and it's easy to generate the glTF from a USD. And try to build on top of that complementary as much as possible. Hopefully we get to show some of that and we get to make some of those things happen. That's one of our goals here.

Sébastien Deguy:

Yeah, I agree. I agree. Simple. And we should make sure that we use our position to somehow voice that loud enough so that it doesn't break and doesn't die.

Marc Petit:

Yeah. No, look, we've talked to a number of people from a number of companies, and I think we all think the same thing. The interdependence, as you said. I mean, nobody can be successful on their own. And I think that guys like Khronos, guys like the Linux foundation, you have a number of people who are pushing for that commonality. And we have places like SIGGRAPH, are neutral places where we all go. I think if it's just one of us sticking our own fate in our own hands and going for it. Well, Sébastien, thank you. On this topic, I think, as you know, it's something dear and near to our heart here on the podcast. Patrick, you want to go through some closing questions?

Patrick Cozzi:

For sure. Seb, we covered a lot of ground today, but was there anything we didn't talk about that you'd like to?

Sébastien Deguy:

There is something you didn't talk about that I'm glad you didn't talk about, so that's good. I will leave you to your imagination. No, but it has to do with ownership and that kind of thing. It's a very interesting subject. Maybe we get that, or whatever. But I'm glad you didn't touch on it because, again, same thing. I don't know, actually. There's your answer, is I don't know. But anyway. No. Let me think.

Sébastien Deguy:

I'm happy to see this type of discussion happening, first. And yeah, I think the metaverse actually is crazy. It's like this curve. As a business owner you're always trying to find, what is the next big wave we need to catch? And so one thing I see in the metaverse is that, well, we've all read the novel. We've all played MMOs, so we're not surprised by it. Right? And so what I'm surprised by is the speed at which people got into... I mean, express interest. And the amount of money that is invested, and the amount of noise it's generating. So that means it will crash at some point.

Sébastien Deguy:

But anyway, and then we come back. What I think is interesting is people now realize that at some point, the same way the internet came and say, it's a fad and it will disappear. And then they came back and now we don't even think about it. I think the metaverse eventually will be something like this, all these expended version of the web. Potentially mostly 3D. I mean, partly in 3D, more potentially interactive, potentially immersive. But that's huge in fact. And yeah, I mean, it's great that we're in the beginning of that. Is potentially a second golden age that we're at the beginning of right now. I mean, we're lucky, Marc.

Marc Petit:

No, I feel the same. We try to remind ourselves it's the beginning, because the past 30 years and how much it has been to get to where we are today from the visual effects industry and the game industry. And we haven't solved... I mean, even on texturing we can barely exchange materials. There's so many problems to solve. So when you see the frenzy from investors. And yes, we know it's big, we know it's going to be important, but it's going to take time too. And then we know that we also have to pay homage to the 30 years of work from those industries, because they are creating the enabling technology that we are there today. That's why we try to invite on the podcast people who actually have made a difference in computer graphics, because they've laid the foundation of the metaverse, and they will play a big role moving forward. Yeah, the amount of work is humbling, but the potential is mind-boggling.

Patrick Cozzi:

Seb, to wrap things up, we talked about how collaborative the industry is. Did you want to give a shout out to anyone or any organization?

Sébastien Deguy:

Oh, there would be too many. Yeah. No, there would be too many-

Patrick Cozzi:

You can do more than one.

Sébastien Deguy:

I don't know. Interesting. Let me... What would I say right now? I have nothing small to say, I'm sorry. Yeah. I mean, I'm pretty excited by the way Adobe is embracing the thing right now. I'm not saying that because I'm there, but it's interesting. And, I don't know. I like what NVIDIA's doing also. I have to say. I'm very impressed by what you guys are doing. I have to say. There are many companies I'm like, "Okay, wow. It's actually pretty impressive and pretty alive." There is a lot of energy and a lot of passion I can see.

Patrick Cozzi:

That's a great sign.

Sébastien Deguy:

Yeah. Yeah. And to Marc's point, I guess it's a lot of people driving this are coming from the same background. And if competing, of course, still love playing the games and just watching the movies. We all have the same pleasure doing that.

Marc Petit:

Yeah. No, you're right, the community is special. And I think it's something we're shooting out too. Well, Sébastien, thank you so much. Dr. Deguy. You said your mom likes to call you Dr. Deguy, so I'll call you Dr. Deguy to close. It was a pleasure to have you. Congrats on the acquisition. Congrats on everything you've been doing at Adobe, because the speed at which you guys come out with things is amazing. You probably have a thing or two to do with that, so congratulation on this as well.

Marc Petit:

And a big thank you to you, Patrick, to be there with us today again, and helping me and helping us with all those technology questions and everything. And a big thank you to our audience. I mean, because we have people like Sébastien, people really like to hear from the folks from the industry, so we are lucky that we have very good guests and so we get very good feedback. But keep telling us what you think and the people you want to hear from. Hit us on social, let us know feedbacks and do all those things you're supposed to do, like subscribe and whatever. Patrick and Sébastien, it was a pleasure. Thank you. And thank you everyone.

Sébastien Deguy:

Thanks everybody.