Making Accessible AI Motion Capture Animation with Tino Millar

Announcer:

Today on Building the Open Metaverse.

Tino Millar:

Just using a single iPhone and soon just any video, we'll be able to take that and then turn that into high-quality 3D animation, making it even more accessible.

Announcer:

Welcome to Building the Open Metaverse, where technology experts discuss how the community is building the open metaverse together.

Hosted by Patrick Cozzi and Marc Petit.

Marc Petit:

Hello, and welcome back metaverse builders, dreamers, and pioneers. You are listening to Building the Open Metaverse, season five, the podcast that is your portal into open virtual worlds and spatial computing.

My name is Marc Petit, and this is my co-host, Patrick Cozzi.

Patrick Cozzi:

Hey, Marc. Great to be here, as always.

Marc Petit:

And as we know, we bring you the people and the projects that are at the leading edge of building the immersive internet of the future, the open and fair metaverse for all.

Patrick Cozzi:

And today, we have a special guest joining us on that mission, Tino Millar. He is the CEO of Move AI, one of the most promising startups at the moment, disrupting 3D animation and motion capture with AI.

Marc Petit:

So, Tino, if you know the podcast, you know we like to start by having you talk about your journey to the metaverse in your own words.

Tino Millar:

Something you should know about me, I've always been a tech nerd most of my life. I've also played a huge amount of sports. I played a lot of games when I was a kid. And then, around six, seven years ago, I found myself in a position where I was doing my PhD, but then my wife and I, we had our first child. This is around six years ago.

I didn't have time to work out anymore and play sports because if you've got kids, it's pretty crazy when you have the first one or if you have multiple. So, I canceled all my gym subscriptions and I started working out from home.

I started to use a bunch of apps that say do press-ups, do squats, do sit-ups. I found it really boring. So, I started experimenting with trying to gamify working out from home; I was trying to gamify capturing my motion in 3D using cameras. Using tech I was working on during my PhD.  I was trying to gamify and capture my motion and put it into 3D. That's the seed of how I got into this space of trying to digitize human motion, and then turn it into a lot of really exciting applications in 3D engines.

Patrick Cozzi:

You mentioned that your PhD–I believe your PhD is in continuum mechanics. I mean, could you tell us a bit about what that field involved and then maybe, ultimately, how it helped inform your work today?

Tino Millar:

I was studying here in London at Imperial College, and I was studying material science, so I was trying to understand how materials fracture in a lot of detail. Typically, when materials break, they'll fracture, so a new surface gets formed. There are some very specific criteria that inform or determine the phenomena of when the material will crack and how far that crack will propagate in different types of materials. That's fundamentally what I was studying.

I was really studying what are the criteria exactly for materials to fracture. I was doing that in three ways. I was working on computer simulations using numerical methods like finite element method to model in some software that was not the easiest to use. You had to program in FORTRAN 77 if you wanted to modify anything.

I was trying to model how these materials behave. I was modeling using the software, also trying to study analytically and then also do experiments. I was trying to use all three methods to really try to triangulate and get to, "Okay, what are the criteria for materials fracturing?"

Yeah, I did that for three to four years. During this period at Imperial, that's where I really learned to code, where I learned a number of other tools across software and computer vision, 3D simulation, 3D modeling, and numerical methods, which I think really laid the foundation to work on a lot of exciting things after that.

Marc Petit:

Right. So, you are officially a nerd.

Tino Millar:

Yeah, I mean, if we go back in the story, I mentioned I played a lot of sports. There was one time in my life when I was in my teens and I was playing a lot of basketball, so much that I was starting to get injured because I really wanted to increase my vertical jump. I was doing a lot of plyometric training, so a lot of really high-impact training. I was doing it quite naively; I would just go to my garden, train on tarmac, and I'd be training every day for a long time.

Then, I started to get a lot of injuries, particularly on my knee. So, I started to get tendonitis in my left knee. I think I was around 15, 16 at the time. I couldn't play basketball anymore, so I couldn't do what I really loved to do.

I wasn't a physiotherapist, I wasn't a sports scientist, but my mum, she used to be a doctor and she had these... There's this publication called the Sports Injury Bulletin. It's a publication where they gather scientific research and viewpoints from physiotherapists and sports scientists. I happened to come across this research journal, and I was reading it and then I started to dig deeper to try to diagnose what's going on with my knee, what do I need to do to rehab? I started to just go into a lot of detail.

So, from an early time, for some reason, I've just really been fascinated with how the human body moves. I'm just really fortunate to be able to combine that love and that interest with interests I've had in games, coding, and technology, to then try to digitize human motion.

Marc Petit:

Many people in the industry view your markerless motion capture technology using consumer cameras, GoPros, or iPhones, to be very disruptive.

In layman's terms, how does your solution work and what does it aim to replace? I'm not sure everybody's familiar with the field of motion capture here.

Tino Millar:

When it comes to creating 3D experiences, animation, how objects and characters move is really essential to these digital experiences that people are creating; think movies, think games. Actually, creating the animation is really hard. You've got to be an expert. Most of this animation is created, I want to say by hand or through key frame animation, that's how a lot of this animation is created. You need to often study this field for a long time.

A lot of people go to college or university to study how to do animation, and then you've got to get access to particular software that helps you modify these animation cards, these rigs, these characters.

It's a very skilled job to be able to animate how objects, people, and characters move. We just wanted to make that process much easier.

What our technology does is we just use consumer-grade cameras; this could be an iPhone, GoPros, iPads, professional-grade cameras, webcams. And we help capture that 2D video and turn it into 3D character animation. We're using our own technology, which uses a combination of computer vision, we use AI, we use understanding of human biomechanics, statistics, to ensure that we can just take these inputs from the video, and then turn it into high-quality 3D character animation to help the lives of people trying to create these 3D digital experiences that need realistic motion.

Marc Petit:

It's all about precision, and I think Tino is being very modest here because the results that people are seeing from the technology are approaching the gold bar of optical motion capture.

Tino Millar:

Marc, as you say, because creating animation is so tough, over the decades people have tried to create solutions to solve this problem; the solution that's gained the most traction is putting suits on people. 

You've got to put on a special suit. That suit has either reflective markers that are tracked by special cameras that then track light bouncing off those markers, or these suits have inertial measurement sensors. Using that information, you can try to recreate what's happening in 3D for the human person. But that also has a lot of challenges.

Firstly, you've got to put a suit on. I don't know if either of you have tried to put on a motion capture suit.

Marc Petit:

You should see me in a spandex suit.

Tino Millar:

A lot of people, maybe you sometimes can be quite self-conscious because you've got to put on this spandex suit, and it gets pretty hot in there. It takes a long time to put on and calibrate. Then, if you have to go to the bathroom, you've got to recalibrate everything. Then, you have to get the actors, the talent, they have to go to a special studio often that has this equipment to be able to do the motion capture. It's also somewhat restrictive, so you have to learn how to move in the suit.

It's hard to capture the true essence and authentic human motion when you put somebody in this kind of unnatural environment because you have to be in a studio. It's a bit darker in there because you've got to make sure the light conditions are right, got to put that suit on people.

You behave slightly differently, which is why often you need to have specialist actors who are familiar with that technology. It does solve some problems, but there's still a huge demand in animation that we want to help through the solutions that we're building.

You mentioned, EA, Electronic Arts; we work a lot with them. They've been an amazing early partner and adopter for the technology. Really, it's been exciting to work with them, and working on some really exciting things for their games coming up.

Marc, as you say, we really try to have the quality as high as possible, whether from a single camera or multiple cameras, to really aid professionals and even a lot of non-professionals who are just wanting to create 3D content.

We have users who are creating memes, they're creating funny dances for their friends, creating YouTube and TikTok shorts. There's a whole new expansion of use cases that we also want to enable by making 3D animation much cheaper, much more accessible, and much easier to use. I think you're going to have a huge explosion of new types of content as different elements of the 3D content creation stack become easier, whether it's creating the avatar, doing the motion, creating the environment.

I think when they're all combined together, you're going to see some really interesting content.

Patrick Cozzi:

I remember being in grad school and I did great on the engineering side, but when it came to our motion capture project and putting on that suit, I did not do well. Then, when it came to manually keyframe a walk cycle, I did even worse.

I see you're totally democratizing the creation of this content and doing so with incredible precision. So, maybe on that note, your company just launched the Move One app. Could you tell us what's new?

Tino Millar:

We really care about helping people with their traditional animation process initially. We started off at the history of the company to really work on using multiple cameras to make sure we can get the best solve, the best quality from using multiple cameras, in order to create high-quality 3D motion that can go into these different digital worlds, to power games, to power movies, architecture models, et cetera. But the holy grail has always been can you get high quality from a single camera? It's something we've been thinking about, something we've been working on for years, and we very recently, just now, got it to a very compelling level of quality.

Using a single iPhone, and soon just any video, our software, Move One, will be able to take that video from a single phone, a single video source, and then turn that into high-quality 3D animation, making it even more accessible. We've been testing and we've just launched it.

I think when this podcast comes out, people should be able to find it in the app store. We've been seeing some really exciting use cases. My mind gets blown every few weeks by a new use case that I would've never thought of when you just make a tool easy and accessible to people. I don't mean to sound grandiose, but I remember when the iPhone came out, and now everyone could take high-quality photos and video, that enabled all sorts of new types of content and platforms and big companies.

I believe when creating 3D animation becomes hyper accessible, when creating avatars becomes hyper accessible, when creating 3D worlds becomes hyper accessible, you're going to have a massive explosion, because now you are increasing the number of creators by 10 to 100X, maybe a 1,000X. 

Traditionally, because it's so complex, as you said, Patrick, it's only accessible to the people willing to put in a huge amount of effort or have big budgets, but very soon, it's becoming accessible to everybody.

Marc Petit:

3D animation, some people may look at this as a niche, but you believe it's a major market.

What do you think is the current size and importance of that discipline?

Tino Millar:

I'm sure a lot of people who listen to the show have watched movies with 3D characters in them, like a Marvel movie, or have played games with 3D characters. Those characters move in 3D, and they need 3D animation to power that. Increasingly, games and these experiences in these movies are becoming high-fidelity and have increasing levels of engaging content. Some games are like semi-movies, they've got cut scenes, multiple characters, all of that needs 3D animation, and creating 3D animation is really tough.

There's a huge industry around creating this 3D content; 3D animation is easily over 10 billion in terms of all the effort that goes into creating this. And so, our goal is really to expand the market and to make it even bigger.

I believe that soon people will be engaging in 3D content creation, but they won't be calling it animation. They won't be calling it rigging because it's all happening behind the scenes. Just like when you take a photo on your phone, you're not thinking about photography like in the classical case. It's just taking a phone, you just press record or capture. All this auto-focusing and all this light balance is happening automatically. People are like professional photographers compared to what you called professional photography a long time ago.

We just want to really expand the market and make it really easy, while also making the best tools to help the traditional 3D content creators, like EA and other games companies, and other film and TV studios, architecture companies. Already, we're seeing the signs of that.

There are some exciting consumer apps, which will be integrating some of this technology for sports and for viral dances, emojis, and different avatars. We're already starting to see that kind of mushroom again and are really excited to see how the market evolves over the next 24 months.

Patrick Cozzi:

I wanted to ask when you're looking at this markerless capture, single-camera, are there any theoretical limits?

Do you think it'll eventually be able to reach things like muscle and cloth?

Tino Millar:

The areas that we're really focused on are, as I mentioned, getting the accessibility even higher, so making it as easy as possible to go from idea to creating this 3D animation. We're beginning with turning images and video into 3D animation. We're going to keep working on pushing the quality, making this real time on device. Just making that whole integration into different characters easier and easier. But there are a lot of different ways to create content. You don't necessarily need to use a phone in the future.

Something we're working on is thinking of a multimodal approach to be able to create animation. Using text prompts and using other modes of inputs, whether it's images, video, text, or different style filters so that you can go from an idea to creating that 3D content as soon as possible.

One requirement for that is a big database of understanding human motion, so it's something we plan to launch next year is just using our technology and going around the world to capture 10 million plus types of human animation. And so, really trying to digitize human motion and code it in an efficient way.

Marc Petit:

Let's talk about a more delicate topic, which is the potential impact if you successfully manage to democratize these very complex techniques and put them in the hands of consumers.

How can professionals embrace these changes and this technology? What's your point of view on that?

Tino Millar:

We really want to expand the market of people creating 3D content; we're all about just making it easier for people. We already now have great adoption in the games market. We have around 10% of games companies adopting us. We're already seeing new use cases, like I mentioned. So, we've got a real-time motion capture system, for example.

We're seeing sports fan engagement activations use cases for it. We're seeing fashion retail use cases for it. We're seeing use cases in theme parks, where you can have these immersive experiences, real-time, location-based experiences, whereby you can puppeteer a Disney character or some other character. We're seeing consumer apps that are able to launch.

I really think it's just going to enable more growth; with AI and a lot of technology, it's quite easy to be very fearful that it takes away jobs. But in so many cases, if not most, there's a lot of value created. There is always naturally a bit of disruption, but overall, a lot of value is created, a lot of use cases are created. It's a really exciting time, I think, for people who are tech-savvy and willing to be open-minded and try new things as well.

Marc Petit:

I'll add a little bit of a follow-up question to that because we've had on the show, Timmu from Ready Player Me, which also is super disruptive technology in terms of avatar creation.

We've had Amit from Luma talking about 3D captures with the NeRFs and now motion splats. So, do you talk amongst yourselves?

Tino Millar:

We do talk. The companies do talk. And, like I mentioned before, I think you're going to have this compounding effect. Creating 3D content is really hard. For people who haven't done it before, I suggest you try and you realize how hard it is. There's a reason why games and movies are so expensive to create; it's because that tech stack is incredibly complex, much more complex than a 2D workflow.

I believe a lot of us, we're working to create tools that integrate with each other; as people start to integrate them, and we're already seeing early versions of that, you start to get really excited use cases, start to reduce costs in production, start to power even higher fidelity content and more exciting use cases.

Patrick Cozzi:

Tino, I wanted to switch gears a little bit.

This morning here at Cesium, we had a bunch of recent software developer graduates from Zip Code Wilmington, and I told them that we were going to talk today on the podcast and that you're a great technologist, but also a great entrepreneur. I wanted to ask you a bit about entrepreneurship.

First, just congratulations on the recent $10 million funding, especially doing that in 2023. Could you share a bit about what the sales pitch was and what the use of proceeds you think will be?

Tino Millar:

On our roadmap, as I mentioned, we are launching Move One, so we're launching the single-camera motion capture tool to market. We have a multi-camera solution as well. We have a real-time version capture system. And then, we are offering APIs so we can integrate this with Ready Player Me and with many other parts of the tech stack.

We're already getting early signs of, okay, this is not just professionals using this, these are not just animation professionals, these are 3D artists in general, these are hobbyists.

We're starting to see consumer companies with hundreds of millions of active users starting to use this for user-generated content. That's something that we're really excited to start seeing our partners launch from early next year.

Marc Petit:

Who are your main competitors, and how do you differentiate yourselves?

Tino Millar:

We're just really focused on our users and helping solve their problems. Right now their problems are not being solved by anyone. A lot of people are still doing a lot of animation painstakingly by hand and using key-frame animation. With no disrespect to people in the animation industry, it feels like more tools can be created to help animators and to be able to help other people do animation. We're really focused on just making the product, making the user experience, making the animation quality, making the integration as high quality as possible.

Our competition really is the old way of doing things and we want to bring more people these tools. We've been really excited with our early community who've been adopting us, and excited to really expand that as we make the product better and better.

Patrick Cozzi:

Tino, given how disruptive your technology is and the growth of the company, I'm super curious: as CEO, where are you devoting most of your time and attention?

Tino Millar:

The filter is you end up working on whatever's failing; then, once you fix that you go to the next thing that's failing. It does tire on you sometimes, but the thing that keeps you going is the mission and seeing your hardcore users who love the product, seeing them gain success, and seeing them value all the blood, sweat, and tears.

Right now, I'm really focused on really digging into the product and making sure you can go from idea to animation and your end content as quickly as possible. At least at this moment in time, we're really focused on the Move One product and products in general.

Marc Petit:

What do you think is your biggest challenge in the role of a leader of a company like Move.AI?

Tino Millar:

It's the mentality to keep going when times are hard. I'm sure both of you, maybe are in this situation now or have been in this situation in the past. You think it's not going to work out, and you’ve just got to push through. You’ve just got to really work with your talented team to fix the problems. Then you might fix it for a while, things are good, and then you come across another big roadblock, and then you come across another roadblock; you just keep going.

I remember in the early days of Move AI, we had this crazy idea to help people generate 3D animation from video. We shared early versions of the product to some very big customers, and you get punched in the face because they say, "The quality's not high enough." Then, you really worked on that for three months, six months, and then you finally think you've cracked it and then show them another version. They say, "The quality is still not high enough." Two years later, we cracked a threshold of quality that can actually truly help people in the content-creation process. So, yeah, at least for me it's you've got to keep going when times are tough.

Patrick Cozzi:

What about culture? I think culture is hugely important to building a great organization.

How would you characterize the culture?

Tino Millar:

A lot of our team actually are not from the animation background, so we try to bring in people who are extremely passionate about what we can potentially build. We've been really focused on bringing these types of people who are really dedicated to the cause. But what has been interesting is we've come as outsiders a little bit into this industry. Very few of us were animators, very few of us came even from the games industry.

I hope with this fresh perspective, we can make the best product for our users because we're not really encumbered by any traditional ideas. I think we really want to leapfrog and go years ahead with this product; not just doing incremental improvements to the old way, but really, make a radically better way of doing this.

Marc Petit:

Another tricky question. Sorry, Tino. I'm going to get you through all of the hard questions today.

As a person of color in technology, have you faced additional challenges due to your ethnicity? And the color question is, what is your approach in your own company to diversity and inclusion?

Tino Millar:

I think I've been very well-respected by people in the industry. I know the culture here in the UK is maybe a bit different to North America over there. The UK's had a lot of diversity for a long time, and so it's a bit more accepted over here. At least in recent years in professional life, I've been very well-respected and not really had issues. But I've had experiences as a kid where perhaps potential violence threatened at you. But I would say in professional industry, being really well-respected.

What we do at our companies, we really just focus on merit and focus on what people can bring. Just naturally, we've got a really talented team of people, all sorts of different backgrounds.

I remember in the early days one of our key employees, I found him on GitHub. I found some code that he had, and I went on his profile, and he said he was looking for a job. We got connected, and he was this smart kid in Moldova. He got impacted by this war that's going on around there. We've moved him across Europe, but we've just tried to find the best people who are really passionate about what we do. 

We had a similar case with an early employee who came to the UK from Egypt. So, the UK also had some really good programs for bringing all sorts of people from around the world who are talented into the country. We really just focus on getting people who are passionate, excited, and just focus on their merits.

Patrick Cozzi:

Tino, it's not your first startup journey. If we look back on your previous entrepreneurial endeavor, any key lessons learned that you want to share?

Tino Millar:

I remember one company I worked on and started, it was a nanotech business. Key learning from that was, it was incredibly deep tech at the complete wrong timing; needed incredible runway to make work. 

Getting the timing right, I've learned, is incredibly important, because as they say, the thing in VC is the wrong timing is the same as having the wrong idea, essentially; the timing's really important.

I think we've been really fortunate with Move AI, and there's a lot of foundational technologies, particularly in AI, that we're just building on top of to bring even more incredible tools to our users.

So, getting the timing right.

Also, not just falling in love with the technology but really caring about our users, our customers, and what they really need. Definitely, with another company I've made the mistake of falling in love with the tech, just because I'm a nerd and I love technology.

There are so many paths to failure with a startup, and so experience in failure does help. Ultimately, you're not trying to fail, but I think all those experiences definitely do help.

Patrick Cozzi:

Tino, we have a tradition on the podcast where the final question is if you want to give a shout-out to any person or organization.

Tino Millar:

Doing a startup, doing an impactful venture of any sort is really tough and needs a lot of sacrifice. I know that I've had to sacrifice, my family's sacrificed a lot.

Definitely, a big shout-out to my wife and our children, who've been incredibly supportive. Also, we've got an incredible team here at Move AI, who also sacrificed a lot and who really bought into the vision.

A huge shout-out to the team that we have here. Also, we've got some really early adopters and customers, without whom we wouldn't have the feedback, and we wouldn't be here today.

Thanks a lot to our early adopters as well.

Marc Petit:

It's clear, at least to me, that Move.AI has enormous potential to shape the future of digital animation, and you are pushing the boundaries of replicating human motion with AI, which, as we discussed, is going to be hugely important. It feels to me like you're an ambitious yet well-grounded CEO. You have a bold vision, and you are on the path for practical execution.

We wish you to fulfill all the promises of all your technology, and we wish you great success.

And, of course, a huge thank you to our ever-growing audience. You can reach us for feedback on our website, buildingtheopenmetaverse.org, and as well as subscribe to our LinkedIn page, our YouTube channels, and on all the podcast platforms.

Thank you very much. Thank you again, Tino. Thank you, Patrick. We'll be back for another episode of Building the Open Metaverse.