The AR Show: Susan Cummings (Tiny Rebel) on AR-First Storytelling and City-Scale Games with Wallace and Gromit

5:58PM Sep 20, 2021

Speakers:

Jason McDowall

Susan Cummings

Keywords:

ar

game

people

glasses

story

metaverse

experience

grand theft auto

lidar

mobile

user

company

project

big

app

susan

welsh government

world

started

thinking

Welcome to the AR show right dive deep into augmented reality with a focus on the technology, the use cases and the people behind them. I'm your host Jason McDowall. today's conversation is with Susan Cummings. Susan is the managing director of tiny rebel games and the founder of fictioneers. Susan and her team were awarded a major UK grant for augmented reality storytelling, which led to Wallace and Gromit the big fix up. This effort was followed by the release, the summer of Wallace and Gromit fix up the city and ambitious city scale AR game that takes place in San Francisco Bristol in Cardiff, including her work at tiny rebel Susan spent more than 20 years in game development in publishing. She was a key player of the origins of Rockstar Games, and held vital roles as the VP of publishing and VP of business development at touquet. Games and parent take two interactive. In this conversation, Susan shares insights into the origins of Grand Theft Auto, and some other big deals and projects in the early days of console and PC games. She goes on to talk about the challenges and telling compelling stories with AR,

it starts with you joining Wallace's company and becoming a member of sticking spanners. And you start doing jobs and you start encountering characters. And over the course of the story, you know, heats up, bad things happen. You save the day, right? But what happened, what would happen if we shut that up, and started with grommet getting kidnapped? And then go back to find out why. And that's what we do in film, right? Film started with traditional stories, and then we started tinkering with it messing with it. And so that I think is the next step is, that was a great experiment. How could we make that better? From a pacing standpoint? How could we start with a conflict? And how can we lead you on cliffhangers every day? So that sort of duration i think is what happens next to tell him a great story.

You also talk about some of the lessons learned and making compelling and accessible city scale AR based games for mobile, and the challenges of immersion in C through AR glasses. A quick note of disclosure, I'm an investor in phantasmal. One of the companies Susan mentions when describing her efforts and fictioneers let's dive in. Susan, I remember, I really got into gaming in the 90s like this was the beginning of the PC gaming era. consoles were really taking off at that time. And one of the most notorious games I think still of all time was Grand Theft Auto. And you have a lot of insight into Grand Theft Auto How did that franchise come to be?

You know, I don't think the whole story has ever really been told. Funny enough. But so grand theft auto I came across because I was working with dmg interactive in the States. This is what I was a recruiter, my earliest days in the game industry. And I was doing a lot of work with them, and BMG decided to get out of the games business, at least in America, it wasn't going so well for them in the States. And so I was introduced to the people running BMG interactive in Europe, which was Sam Houser, his boss with a guy named Gary Dale, and they approached me about signing off the rights the US rights to all of their products, including Grand Theft Auto. And so I spent a fun period of time talking to most of the game publishers in the States about this game that no one had heard of this top down still my favorite GTA I have to say the top down cartoon a Grand Theft Auto. Funny enough, we were about to sell the rights to th Q and th q thought they had the rights to this to the point that it was in their marketing materials to pitch out there to retailers. It turns out that BMG was actually going to sell the company to a company that I don't think anyone will have ever heard of anymore called American soft works, which is a little company in Connecticut, and American softworks was about to buy BMG interactive. And Ryan Brown was running take two at the time, Ryan passed away. But the old friend of mine, he's running Take two. And he used to call me at two o'clock in the morning about deals. And he said, Susan, I heard that BMG is getting out of the business and how do I get involved in this? How do I buy the company because I heard that someone's buying the company and I said, I have no idea what you're talking about, cuz I didn't know I was I was nobody at the time. And they didn't bother to tell me they were looking at doing this. So I found that it was true. And I made this introduction now take two at the time was, you know, $5 a share if that like their stock was in a gumball machine in the lobby, they will do horrible FMV games. And Ryan probably the best person I've learned most about deals from in my career Ryan talks to them and the next thing you know, American software x gets the PC version of Grand Theft Auto and he takes everything else and that over the next few years became Rockstar Games and it took some time for it to gain traction. They did a surfing game that nobody wants to ever heard of. And then and eventually Grand Theft Auto two and then finally Grand Theft Auto three which is the one that really put it on the map and sold the PlayStation but it could have gone so many other ways. You knew Wow.

That's amazing. Two o'clock in the morning. I get this Yeah. What were the magic words that he said to the BMG folks in order to get everything except for the PC rights.

Ryan just knew how to do deals like you know, I

The stuff that we did when we set up to K 2k was set up because what is take two was just grabbed up Dotto. And even though that should be enough, it wasn't enough for wall street at the time. And we were getting hammered for the fact that there was nothing else. And so that's how I ended up setting up to K with Ryan was we needed to expand or IPAs, and Rockstar has always been very focused on the games that Sam Houser wants to make. And they take the time that they take to make even if it's many years like that's, that's just their focus to K was set up with the idea that we need to do more than that. And so we had a couple 100 million at the time to go out there and sign projects. And so my role was to a apologize to a lot of developers because take two had the worst reputation of not paying people and you know, screwing companies over so I do apologize first. And then then then suggest to them look, we want to fund your stuff. What do you want to make, and everything that's to Kato came out of those early conversations we signed Bioshock when nobody else wanted to sign Bioshock we signed it as a $4 million project, which ended up costing far more than that. And we bought the studio but it was a little deal. Borderlands was signed on a verbal pitch at EA three, they pitched us on two games and we signed we signed Borderlands out of it civilization. And that whole four axis thing happens I had a Friday phone call with for access to the first time where they told me Hey, Atari is looking to sell off civilization. And over the course of the next week, we had our lawyers in there. And by the end of the week, we had handed Atari 20 million and got them to throw in XCOM as a freebie. Meanwhile, EA Sega people had been in there for three months poring over things. And at the time, we were just making instinctual decisions, what no other PC games were selling at the time, but you knew Civ was gonna sell and it was an alpha. So there was nothing really to talk about it. It's just a question of how much and it's one of the problems in the game industry, it was that you used to be able to shoot from your hip like that and do a deal instinctually and it's gotten so focused on sequels and trying to look at comparables, the whole point of original IP is there shouldn't be komfortables It should be something new. And that's a really hard thing for wall street to to get their heads around. Yes, I can imagine it's quite challenging to convince a wall street banker type to trust somebody. Just trust us. We know we know how to pick great content, we know how to pick great teams. So for you, how did you find your way into the gaming industry? Why was that a calling for you? You know, I've kind of stumbled into it. I got out of college, I went to Brandeis and I got a job as a recruiter. And the company that I went to work for wanted me to work in multimedia, which which was still a term back then. And my boss threw a book at me of multimedia companies and said, do that that's going to be your specialty. And so I started calling around and it turned out they were mostly video game companies and, and I had no idea. I talked to people like aloe without a time really understanding who who that was and what an icon I was. But talking to people like that, and getting to know people in the industry. It was a much smaller business back then. And it was actually possible to get your arms around it. And and and know everybody in the space. And so I so I was a recruiter and I was decent recruiter. And I did that for a few years and got bored. And then I started doing deals. And that's how I ended up you know, involved in the Grand Theft Auto deal. Other distribution deals, and then I got bored. And so I got into publishing as a deal signer. And then I really wanted to have my hands dirty. I hated the fact that I was involved in the exciting beginning and then I go away. So I got more involved in production. So I think my whole career has been about feeling like I've accomplished all I can in this, I need another challenge. So I've continued to kind of up the ante on the things that I don't really know. But I'm gonna figure it out as I go along. And it's worked so far. And it sounds like with each one of those iterations on your own career, you move a little bit closer to the making of the thing itself. Yeah, yeah, yeah. And I constantly have a this getting pulled back. So I started to k, and the beginnings of 2k, I was running product development. And then very quickly, I started getting pushed the other direction we need to build the product portfolio. So we brought in Greg goby to run product development. And it was really heartbreaking because I love the fact that I was finally getting to be to be in the middle of it creatively and from a production standpoint. So yeah, I said, I eventually started my own company with with my husband. And and so Lee all the while was on the rock star side. And so he very quickly became Sam sort of right hand and all of this stuff that was important to him. So he worked on your autographed daughter products from a production design standpoint. And then we were doing bully and bully was about to get killed. So they moved Lee, my husband up to Vancouver for the better part of the year to sort of save the project, which was great for him. It was the hands on experience that he needed as well for his career. And when we left, realize that between the two of us we've been working alongside each other but not together on different labels. We realized that we kind of knew what we needed to know from production and design standpoint to be able to go out

do stuff together, which is cool. I don't think many people could do that to be married and to work so closely together for for a dozen years now, I guess since we set up time rebel.

That's amazing. So you were focused initially on primarily on console? Yes. And what happened is you kind of got together with your husband to start your own stuff. What was the focus at that point?

So we set up the company to sort of originally be like guns for hire. And we had we had friends that we pull it on projects as well. We worked with Atari, we worked with majesco. And then Paramount hired us to run their games group. I remember the meeting where we were asked if we wanted to work on the next Star Trek video game, I called my husband, he's like, how much do we have to pay that? So so we moved to California, removed out there to run this this Star Trek video game, which should have been the best Star Trek game ever made. Unfortunately, it was probably the worst Star Trek game ever made, for reasons beyond our control. But we moved out there and got to work with Bad Robot and and others on this, this video game that was meant to take place between the first two of the Star Trek movies, unfortunately, paramount. And I got in the way of that vision. But so we moved out there for that. And then, in a couple of we did a War of the Worlds game with Patrick Stewart, which is, which is pretty awesome, too. And then we tried to get funding to do a sort of ambitious shooter. And at the time, mobile games was just heating up. And what seemed like a really small budget coming out of my take to brain was certainly a lot of money considering what you could do on mobile for the same. And so we didn't really want to make mobile games we saw them, as you know, is awful. Time sinks and energy meters. And Lee as a designer is like, why would you tell someone the minute they fall in love with your game to go away and come back, just to seem so counter intuitive. So we got the rights to Doctor Who. And we did our first mobile game hold Doctor Who legacy, which I'm really proud of, we did a few million installs of that game without spending any money on user acquisition in a different time when you could actually do that. But it was all the things that we wanted it to be. It wasn't it was it was called by Kotaku, one of the fairest versions of free to play that ever seen, which probably also means we could have made a lot more money if we hadn't been such a fair game, but you know, became this really loved thing within the doctor who world So, so so proud of that. And then we did another doctor who game and then a few years ago was when we started to jump into mixed reality, when the Welsh Government encouraged us to bid for an immersive, immersive grant on an AR grads, but an immersive, creative, creative, immersive grants from the British government.

Yeah, tell us tell us about that offering. What were they trying to accomplish this Welsh Government? Why were they the ones putting up the money? And what did you end up proposing

the Welsh Government pushed us to do it? It's actually the British government. So they have this a group called innovate UK and innovate, put aside about 30 million pounds of r&d funding. There's a lot of r&d funding in the UK, one of the things I've learned as an American over here is that you can actually get the government to fund creative r&d, which is really exciting, gratifying as a creative person. But the the, it's a variety of challenges that you have to pitch for, but half of the money went to for demonstrators. And so we had to pitch alongside a few 100 consortiums of companies, not even just our company, a consortium of companies to get this money. And so we bid for what was called the Moving Image demonstrator. And we propose that we would tell a story in a new way that we would tell a story in real time and use AR in a more holistic way and use it to bring another world into ours and experiment with mixed reality. But keep in mind, we came into this with no AR experience. This was this was this was a new, this is new grounds to us. And we built a team around it. You know, we started with a small group and it ramped up to probably 3035 people including Aardman over the last couple years to make the big fix up.

How did Wallace and Gromit become part of this

when we got the grant, we had it, we have a different IP involved and when that's sort of what what arrived at the last minute, and so we had the opportunity to actually go out there and and tell people, we had money. And we did an IP and Aardman. We're actually the very first company we sat down with about it, we had plenty of conversations, but they were the first and I was so impressed with how in love they were with playing and experimenting in a way that I've never seen before from IP companies, you know, having worked around Paramount, and having worked with the BBC, I've never seen a company like more willing to take risks and be playful with really valuable IP. And they were from the beginning, they you know, they announced the project in its earliest days where most IP holders would would want to wait for something like that. So yeah, they share it. They shared our our enthusiasm for the idea that a story can be told in real time, and that we could use AR as a as a way of thinking about it.

And so how do you tell a good story using augmented reality in a more holistic way?

Well, in our case, we we started from scratch. We started with the way you would make a movie and we said let's think about this that way. We don't want to tell a story that already exists because we want users to be surprised. And we want to give them some agency and, you know, culpability in the story and to make them feel a part of it. And so, Aardman came up with a few concepts and and pitched us on them. And then our team and their team spent a good couple months, at least in the writers room together, mapping out the story. And nothing existed to do what we needed to do, because we needed to figure out, first of all, what the sort of three act story would be. But then we had to figure out what the sort of beats were that make up that story. And then we had to figure out what the best way was to tell each of those bits of the story. And that was super complex. I mean, we spent months doing that, it was 200 pieces of story delivered through the big fix up. And and then not only that, we had to figure out a way that we could tell that story to everybody at the same time, and how we could deliver that in sort of sort of cloud based authoring tool standpoint. So we've got the mass platform. And that's what the mass platform is, for the entire story lives as beats in the cloud. And we can do all sorts of stuff with that. So we can, we can say it's going to be a 28 day story for everybody, which is what we did when we launched. And that's so that's in real time. So any day during those 20 days, you join and you catch up with the story already in progress, and you progress. After we launched, we started doing staggered timelines where we'd say it's based on you. And it's 10 days long. And over the course of the next 10 days for you, you will experience the story. Now we moved it down to five days. And then we said actually, every day, we're going to deliver you a chunk of content in the morning over the next five days. So we've done all these experiments since we've gone live because it's sort of the showcase of the mass technology. It's an experiment, we didn't have a marketing budget, so it was never meant to be a massive commercial success. It was always meant to be, you know, what's possible. And it's allowed us to, you know, sort of keep playing and tinkering and, and thinking about story with the Aardman team because they're they're just in love with figuring that out. You know, we told it as a three act story. It starts with you joining Wallace's company and becoming a member of statements banners, and you start doing jobs and you started capturing characters. And over the course of the story, you know, it heats up to bad things happen, and then you save the day, right? But what happened, what would happen if we shut that up, and started with grommet getting kidnapped? And then go back to find out why. And that's what we do in film, right? Film started with traditional stories, and then we started tinkering with it messing with it. And so that I think is the next step is, that was a great experiment. How could we make that better? From a pacing standpoint? How could we start with a conflict? And how can we leave you on cliffhangers every day? So that sort of iteration i think is what happens next to telling a great story.

How does the user experience the story ultimately, through their devices?

So they experience it through an app? The app is is your employee app as a as an employee speaking spatters. And so and part of this is because of the limitations of AR right now, while we're holding the phone, right? We spent a lot of time thinking about how long? How long can someone hold her arm out like that before they start to get, you know, fatigue? And you know, what's the best user experience? We started thinking the entire app should be an AR, you know, you should have your AR workshop where you build contraptions and send them off, we very quickly realized that's going to be awful on a mobile phone. And so then it became, this should be a 2d experience. Which of the you know, what, what are the experiences that are sort of a showstopping experiences when did the AR things happen? As a user, you install the app, and it has tabs like you expected, you know, an app to have as an employee. And there's a there's a story feed where the story plays out. And you can swipe through more news sort of Tinder style. There's a jobs board where you can take on jobs and send off your contraptions. There's a sort of social board rich chat where characters chat back and forth with you. And then there's an area that we call the playground, which uses unity is Mars technology, to play stuff and have things move around your space. So you can hang things on walls and put a coffee cup on the table and have your contraptions move around the room. And it's just a little AR playground of sort of fun with stuff from Wallace's world.

How important was place in the telling of the story?

Well, it was meant to be very important before the pandemic happens. The original plan for the story, the three act story was that act three would take place on the streets of Bristol and phantasmal. LIDAR scans Bristol for us. We talked to lots of different companies about slam technologies. And we ended up partnering with phantasma we've been great to work with. And so they'd scan Bristol, and we were all sort of set to go with the notion that you know, at the end the back to grommet gets kidnapped. And there's this countdown to grub fast. And this whole big festival is happening where grub is trying to get elected there. And we're going to tell users to come to Bristol and have this experience. And we always had this notion that not everyone can get to Bristol. And so there was always this plan for an at home experience that we kind of minimized because we wanted people to come in But there was going to be this kind of lowest common denominator, you could have this thing at home and still feel like you knew how the story ends. And then the pandemic happens, of course. And so what we did was we took the data from Bristol, and we turn those into scalable dioramas that you could have within the app. So you could place them on your tabletop, or you could place them in the back garden, and actually have quite a quite large experience, not quite what Bristol would have been but reasonably interesting experience. What happened then was that the government gave us additional funding to sort of for that pivot to fund that and also to agree to still do it when it was feasible this year. So starting January, we started working on something called fix up the city and fix up the city takes place in Bristol and Cardiff and in Yerba Buena gardens in San Francisco. And what we've done is we've scanned locations, and created a new experience that feels a bit like a carnival. And so it's a it's not X ray, it's something all together itself. But the conflict is the same. You have to you have to save the day and save Wallace, but it's a game. So whereas big fix up is a story. And we went to great pains to try to explain to people that it's not a game, don't think of it as a game. It's a story. This is a game, we want this to feel like a game, we want you to go there and have fun not not to not punishing, you know, we don't think that that AR is ready for punishing gameplay. Like we think that it's overwhelming enough as it is that it should make you feel clever. It shouldn't punish you. And so it's so it's not hard gameplay by any means. But it's cool. I haven't seen anything like it, you know, because of the fantasmic scanning, the world really does come to life around you and AR and so it's not about just doing a little job. It's about the billions in the sky and the fake bunting on the walls and Carnival props and stuff like that, that really, you know, give it this richness that makes it feel real, you splatter paint and the paint actually splatters on the wall in front of you. Big lad jumps down off a building, if you're in Bristol, if you're in Cardiff, he punches his way through the front of the Wales Millennium centre. You know, if you if something gets smashed by big lag, you could turn around and you can still see the dent of it in the ground. It's just really, really cool. And you can see that that's it gives you a glimpse of where AR is headed when when we have glasses in the future.

So people need to get themselves to Bristol Cardiff or Yerba Buena gardens in San Francisco, starting this weekend, starting this weekend, Friday, Friday, I think is the official day it'll be available. By the time we publish this, this will be in the past. So here's this this will be it'll be available now. Exactly. Exactly. fix up the city. Yeah, well, let's probably fix that the city, I'd love to maybe dig in a little bit more into the perception you have around creating a good gameplay experience outside of the storytelling experience. But the good game experience around AR, you'd noted that AR is overwhelming. It's tough right now that it shouldn't be a punishing game, the user should feel more more clever as they kind of go through it. How did you come to that conclusion how to kind of think through what's the right sort of gameplay experience for something like this,

we had a heavy focus on user research, because it was an r&d project and University of South Wales brought researchers into the project from the start. And so we started getting things into people's hands. Before we even had an app we had we had, you know paper that we drag people through what the experience would be as a storyboard. And we were really surprised by how hard AR is for people and the things that we assumed people already would know how to do, they don't know. And so for example, if you had the wrong trousers, and you wanted to make them bigger or smaller, and they were anchored in front of you, someone in AR would know to just walk towards it, you know, or to back up. And instead, we found people wanting to pinch and make things bigger and smaller like that. And we found that they didn't necessarily quickly know how to interact with things. And so we call it an AR coaching. And we spent a lot of time on that on thinking about what the right sort of methodology is on screen to show people where they can interact with things to explain without explaining to them how to scan a space to explain how to anchor a space. I mean, the scanning is the most exhausting thing about AR and mobile. Most of us agree it's not fun, and you have to keep doing it. And so looking for ways to make that fun is something in our minds a tiny rebel as well as how do we make that a better experience for people since it's not going away anytime soon?

This is the notion of doing having to move the camera around to understand what it is that's around you. And needs. Yes, phone, the whole system still needs a little bit of time to kind of figure that all out.

Yeah, yeah, I mean, we have, we have a demo that I've shared online of some work we did with LIDAR, on the phone, which has pigeons coming out of the walls and landing on tables and things like that. And it's lovely to look at. It takes a lot of scanning to get it to work that well to try to explain to a user in a that they have to do a lot of scanning and be why they should bother to do it thoroughly. You know what the reward is for the fact that you actually got down and looked at the angles around the couch and things like that. But it's not a great user experiences and nobody wants to do that.

Yeah. So in the case of the the city fix up there that you've already pre scanned these Yeah, good places. Yeah. So now the technical challenge is just about positioning. Okay, given given that we already have the scan, how quickly can get the phone to recognize exactly what it's looking at within this known space?

Correct. And yet, that's not, it's still not perfect either, you know, it's definitely hasn't been quite as easy as we expected it to be. We've gotten there, you know, we've found that, you know, for example, the earliest scans that we had, there was this sort of default of constantly rescanning. And that was frustrating, because sometimes it was perfect. And then suddenly, it was offset because the app decided to rescan so we did away with that, we actually let the user adjust a little bit. It's one of those sort of behind the curtain things that you know, what it actually makes it better to let a user do that. And if you're gonna have a better experience, it's okay to to ask somebody to offset a little bit to make it a better experience, one of the things that you found, but yeah, it's, it's so much better, it's so much better when you don't have to ask the user to do to do anything. You know, the first time you actually see big lad stamp on a contraption, and you see it embedded in the ground, you know, you realize that this is something different from what we've seen before, or you watch big lad smash a building, and you see the rocks pull on the ground, you know, at your feet. It's just it takes it to another level.

As you noted, when you came into this project, originally, this was the he had not done AR before. And so you've been working very closely with artisan and how to tell a great story using this medium, you've been thinking very deeply about how to create great gameplay experience around this medium, you have the benefit of having the resources to do lots of research around it, which is hugely beneficial to you. But also, is there an obligation for you to share that more broadly with the community? It's kind of the research that's being done?

Yeah, the university is doing a lot of that it's doing a lot I did a lot of sort of documenting and output of the of the project. I'm sure it is part of it. Yes.

And you'd also mentioned that part of your learning was that you didn't have a key tool that you needed, which was ultimately some way to manage the multi user real time storytelling experience. What went into this, you call it the must platform m ust what goes into that must is

amazing. So So fictioneers, the group that we assembled as part of this graph is a few companies that came together, no ours being one of them. Potato is one of the partners, and they're they're part of Aqa. And they have a wonderful background in building sort of scalable back end systems. It was one of the reasons why we got involved in the project. And one of their engineers, James Spencer, I remember him coming into the project. And there were a few months that it just felt like he was off intellectualizing and asking lots of questions, and you're wondering where it was all going, I still remember the first sprint, we finally showed us the mus platform, and it suddenly became the coolest part of the project is as excited as you were about AR, it was so cool to see this, this story working, because what's fascinating about the musk platform is the time it would have saved us, because it allows you to simulate a story that almost like you were assembling an orchestra, you know, you have these sort of swim lanes of activity, you have the AR stuff, you have the video animations in gameplay, and then you can hit play and simulate it and watch the story before you ever have to actually deploy anything. So in my mind, that means how we could have created this whole thing as a text based story, just to try it out and see how it worked, and then start building expensive content. So I think that's what's so interesting about the mass platform. But yeah, we needed something that would would enable telling a story in real time. And there was a period of the project where we were so focused on the big fix up. And we kept saying we're going to tell it in real time, but we didn't have the ability to do that. But we have a really great engineering team as part of fictioneers that built this. And they're going forward now and turning that into a software as a service

solution. Oh, amazing. So it'll be available. broader, broader community here soon.

Yes, we hope so. Yeah.

You mentioned Mars, Mars program from unity. What were you using Mars for? How is that better than some other alternatives? Maybe.

So we started talking to Tiffany West and the Mars team early on in the project. You know, the great thing about the graph was, you know, if you go back to 2019, when we started development, was that everybody had cool tech that they wanted people to use, but no one really had funding for it, because no one was funding AR or anything, for the most part back then. And so it was a great calling card to be able to come in and say, Hey, we have Wallace and Gromit. And we have, we have funding and stuff. And we want to we want to check out your tools and see if that can be a part of this thing. And so that's how we got to know unity. And they were super excited about the project and have been so supportive every step of the way, we have a slack with them. And you know, we get to get really quick answers and support from a tech standpoint, we were excited about Mars because of the fact that it could. And we are continuing, you know, within tiny rebel with some of our new activities to push Mars, to use it to give to give things a semantic understanding of your space, because that's where it really comes to life is the idea of something moving around your room and having an understanding of what different surfaces and roles are. It's you know, it's what we're all hoping glasses will have in the future. And so, right now on mobile, it's a great way of iterating and trying to push push forward what's possible But yeah, the the big thing about Mars that we love Is this the rule system that allows you to create so that you can you can give a anchor realistic behavior based on that. And so I think we were one of the first companies to use the Mars technology for this.

Yeah, that's pretty cool. I've seen you get really excited about other types of projects over the years. And the story part of the story, one of the stories I hear is that they've just been a great partner. And it's great to hear that you have the same experience working with Unity. Yeah,

absolutely. They're really excited about fixed up the city to their their sponsor of the the city scale experiences. So they've been helping, you know, with some funding and some PR activity and stuff around around the city scale experience, because, you know, again, it's just a showcase of their stuff. Yeah, you know, we use it we use a our foundation, we use the high definition rendering pipeline, we used for the creation of the of the lovely videos and big fix up and, and we use unity as a library and flutter. So it's just a lot of different sort of lines of unity loves it when you're using lots of their stuff. And so they've been really excited about how many different facets of unity technology brought into this.

The other company you mentioned was phantasma. Yeah, the value add, I guess is they map the city using LIDAR technology in this case, but they map the map the city and then allow you to position exactly figure out where the phone is relative to the real world based on that mapping

Exactly.

Any other notable companies you looked at that are worth

mentioning, we talked to lots of companies we were looking at as your spatial anchors from Microsoft for a while, we looked at strophe. We looked at scape, we talked to 60. So pretty much anyone who was out there at the time, we looked at phantasmal really like the fact that they could also go indoors, because of the backpack solution didn't end up doing anything indoors, but we liked the fact that we could, because they had a background as a developer as well, they were really just sort of the same mindset with us and really sort of pitch Shin when we hit pain points, including when COVID started helping us think about how we can take that data and use it within a home setting. So they they've been they've been really great partners every step of the way.

Amazing. Any other kind of key partners that you will encounter the kind of road made for great experience.

Miro, we use Miro for storyboarding. We absolutely love Miro, they've been so wonderfully supportive of us to adventure. And our we've used mural before, but we use it for all of our storyboarding, we must have made 250 mural boards over the last two and a half years because I've been trying to archive some of this stuff. But we found near to be an indispensable tool for doing figma. You know, so there I've gone through what's wonderful, I guess. And because of the pandemic, it's gotten even more so there are so many wonderful online collaboration tools that it's no longer necessary to be in the same room of people.

Yeah, yeah, absolutely. The idea of doing a city scale game of having kind of users out, kind of in the real world engaging with real world things while also participating in your, in the story itself that you're creating through that through the mobile app, or any other sort of issues you're into around permissions or trespassing or anything else, kind of that are not quite the first order things you consider when making a game like this, or an experience like this.

It's an interesting question, because it's this, it's kind of the Wild West right now, right? In terms of, you know, insurance and public liability, and all these sorts of things. We've had risk assessors involved every step of the way. And thinking about the city scale experiences, Cardiff and the Cardiff City Council and a Welsh Government has worked closely with us in terms of Cardiff, we spent a lot of time just trying to trying to do it the right way and keep people safe. So trying to make sure that nothing is near a body of water that you can fall, fall into, or you know, roads and things like that. So we've taken a lot of care and where we position these experiences. So there's that, you know, there definitely a lot of open questions out there not so relevant to this but broader questions about, you know, who owns the rights to spaces in the virtual world? You know, can you put something on the wall of a building, that's it belongs to somebody else and have it be permanently anchored there? And like, what does that mean? Like? Are they owe money for that? Could you offend somebody? You know, could you could you put an anti Trump billboard on Trump Tower in AR and can he do anything about it? super interesting question. So I don't think anyone knows the answer to yet

yeah, I think we're all gonna stumble our way through it. Yeah, we kind of decide and feel Yeah, what's right or somebody with enough money in power mandates dictates what what they think should be right.

Think about NF T's NFT ownership and if you if you have an NF t standpoint on an anchor, but it's on somebody else's building, like what is what does that mean?

Yeah, yeah, super interesting. fix up the city is out. Now, as of the time people listen to this. Yes. And what's next? What's next for the walls and grommets and kind of the storytelling or gameplay around it or what's next for our tiny robot games?

So fictioneers was an entity that came together for the project, like I said, with a few companies, ours and potato and sugar. Everyone is is sort of also off on to their other stuff. And so for a tiny rebel, we've been working since January, really on we have a few projects in place and our focus case is really on being multi platform and having our hands on everything. Because we think in the future, it's going to be about being able to migrate between different devices. So you know, your AR headset, your mobile phone, your your VR headset, your smartwatch web. And so right now it's about, we spent the last three years gaining a ton of competency in creating unity app, you know, ar experiences, you know, essentially. But what we hadn't done was web xR, we hadn't done social we hadn't, you know, touched a wearable device other than playing with the Magic Leap. And so this year has really been about us sort of spreading out into these other platforms. And so we have a new IP that we're working on, and the focus is really on, how can it be in as many places as possible, as opposed to just how do we create a mobile AR experience?

So your crew starts with introduction to multimedia. And now it's that multi platform. So as you kind of reflect back on the, your period in the PC gaming, the the work that you've done on the mobile side, I think sort of like patterns you see emerging in the early days of AR that you've seen before in these other other cycles?

Oh, well, I think if you, if it's a little bit, it's the Wild West, right? Like in the early days of video games, you know, when you think of when you think back to the arcades and point up games, and how simplistic they were. Part of that was obviously a limitation of the technology. But it was also that it was something new, and someone walked into an arcade and put a quarter in, and they had to be able to quickly figure out what they were meant to do. And I think that's one of the reasons why AR right now needs to be so focused on simplicity, and just sort of getting everybody to the same level that we know what to do when we open an AR experience. And so I think that that's part of it is you look at what's getting people excited online, what was on Twitter, someone doing, Paul, God horizon CEO tweeting about it or something like that. And it was just literally like, you know, monochrome palm. And I think that kind of, and that's cool. It was awesome. But I think that kind of shows where we're at right now is we're still having to just prove out the building blocks of what's possible with the hardcore technologies like Unity and Unreal, but also with, you know, with things like Spark, and eighth wall and stuff. what's possible, there was this wonderful chart that Oscar did online that I'm sure you saw that compared all the different tools and what they can and can't do. There's so much disparity right now. And for developers, it's really daunting, just to get your head around. So who uses LIDAR? Can I use LIDAR with that? Can I can I access the phone? What's how many planes give it? Yes, there's so many questions that it's just really impossible right now to think about, how can we be everywhere? And that's one of the reasons we're spending so much time thinking about it this year. So how do we think about it? Because clearly, in the future, we do need to try to be in in as many places as possible.

So where do you think it goes from here? If If right now we're kind of in this fundamental building box, getting people familiar with the basic types of interactions that are possible? What does that look like? Five years from now? What do you think AR gaming and storytelling go over the next five years? Well, I

think we need to first figure out what is a compelling use case for AR. You know, Pokemon GO was is obviously one of the biggest things ever. But we all know that it wasn't because of the AR, you know, excuse because the math game and Pokemon, obviously. But the AR is something that most people turn off really quickly, which sucks. And so as much as you'd like to use it as a calling card for AR, it isn't so much. And that's partly because it is really hard to find the use cases that make it better. We're doing some work right now with one of the AR tools to try to figure out like, so one of the prototypes that you could do what how do you think about gaming with a tool like a like an eighth wall or a spark, or, you know, the snap lenses lens to do what's possible, and what makes it a better experience than just having a 2d UI. And so I think we need to figure out that I think we need to figure out the mechanics and the sort of verticals, that make sense. And it needs to make it better. It can't just be I can do it in a or it has to be, I prefer to do it in a part of the vaccine to take the glasses. Right. But I think we I think we can't ignore mobile as an AR platform, even if it's not the experience of glasses are going to be because there are tons and tons and tons of mobile devices out there that are AR capable. And so little has been done partly because most game publishers don't want to fund AR games, you know, and this, but there's so much more that can be done. And so that excites us. And I think also mobile AR is so far ahead of what wearables can do right now. Especially like the iPhone 12 Pro, right? There's so much you can do with that phone, that I think it makes sense to iterate there now. And then once the AR glasses start to catch up, you can start to migrate those learnings across. That's our strategy to it anyway.

Do you think there'll be a pretty natural transition from the learnings you have on the mobile side to glasses once they become more pervasive?

I hope so. It's something we're looking at now. There's definitely challenges you know, like AR glasses are so translucent compared to a mobile screen. And that really breaks the illusion for me. I remember seeing someone got part of the big fix up working on a Magic Leap for me and I was I was just put off by how translucent it was. It's so vivid on on an iPhone, and it looks so washed out. And so that's going to get better and better, obviously. But for now, I think it's a, it's a matter of figuring out how to play to its strengths. It's just like a mobile phone, you know, you don't, don't do stuff that breaks the illusion. And so it's a lot right now, it's a lot of trying to figure out how to work around it and make it feel like a feature as opposed to a, you know, a bug. So I do think so I think that, you know, we're still, we still play around with our Oculus quest, and I'm excited about the past year API that's coming. I think we have to think about hand gestures. And that's a harder thing to do on mobile. And so I think, either looking at the early AR glasses, or things like that magically, or looking at the quest for the past, you can help us to start thinking about what the right hand gestures are for control, because I think that's really important, too.

Yeah, I think this, this notion of the see through nature of AR glasses, is probably one of the biggest challenges to overcome for telling a compelling story. We've tried to transport the user, least the mindset of the user from the current location to some alternative reality. Yeah, earlier. Now we've all decided to call the metaverse, although I'm still not comfortable with that, that branding on either I know that that see through nature, really makes it challenging, because that's because you will always have some hints of the real world. And even if you don't have a hit, if you're able to make that lens, right enough for the contrast high enough, you really just focus mentally on the virtual image. What are the safety concerns to the extent that that user is moving through the real world while they're consuming this alternative reality in the course of a story or gameplay? Hmm, what are the safety implications, the practical implications of that sort of thing that's blocking your eyes at that point,

the whole UI that's gonna involve right, I think the UI of Eric glasses has to be fascinating, because it's not going to be like, it can't be like VR, it can't be an experience that takes over everything you're doing. Because you might be sitting in a cafe talking to somebody. And so to me, it almost feels like it's like widgets, right? You're gonna want to be a lot of your map experience open. And, you know, there might be dragon flying rounds, and you know, you're listening to your music, and you do all these things, and it kind of feels, it's almost like a HUD, you're gonna have to be able to do a bunch of things at the same time, as opposed to the Oculus, where you go in and you pick your game and you shut everything off, like it's gonna have to be a different way of thinking about UI and apps can't be traditional apps, right?

That's right. That's right. I can imagine in the early days, we're gonna see lots of stories using ghosts, and some other sort of theory or plan that takes advantage of the fact that everything's translucent. Absolutely. The glasses are with the industry as a whole are very excited about. But clearly, everybody has a mobile phone in their pocket today in the vast majority of them can support a decent AR experience. And so that's, that's probably where the focus is, and should be for a lot of these kind of AR oriented efforts that are going on. But do you think kind of given some of the unique challenges, that air glasses will become a viable platform for storytelling and gameplay?

Oh, I definitely think so when I have no doubt in my mind that they're eventually going to be here. I mean, I don't know if it's three years or five years or 10 years until there's any sort of, you know, ubiquity to it. But I absolutely think that they're coming in, I do think that Well, I do think mobile is viable AR platform for games, I'm probably very, very the small minority for saying that. But I just think it's, it's, it's just been barely tapped. And again, it goes back to figuring out what's a good game experience and what's just using AR, for the sake of using AR, but I absolutely think it's coming with the glasses.

given the opportunity is today in mobile, does the gameplay experience need to be not just AR is AR enough?

I think you need non AR as well, I do I mean, I it feels like it's a cop out. But there are times where it's just not convenient to have AR you know, if you're sitting on an airplane, again, someday we'll do that, if you're if you're in a train, or whatever it is. And you want to be able to have some experience around the app. And I think it's important that there's things that you can do that don't require AR. And I think it's a nice way of sort of bridging people into it. But I think you have to give it a purpose, so that people don't just turn it off. And so it's kind of in my mind, Pokemon Go is that first little bit of AR, I'd like to switch it around and say this is an AR experience. But there are some things that you can do if it's not convenient to be an AR right now. And so I just kind of I feel like that ratio needs to start swapping the other way for anyone to start caring about AR games. And so a little has been done, you know, going back to Pokemon GO came out. Lots of people said, we're gonna make that too, you know, and that was the Ghostbusters was the Jurassic Park one. And they all flopped because it was just the same game. And then no one had any other ideas. You know, there was some there were some, you know, really, you know, by dragon and things like that, but they all came out and sort of the early days of our coordinator kit, so much more possible since this app sort of came out and died. I think the biggest reason is that it's been hard to find investment and I think it's starting to change but last No one wanted to hear that you were an AR company, especially after Magic Leap. You know? It was like, yeah, we're done with AR come back to us in five years. And, you know, in conversations that I've had this year, I found that's changing. And I think that's partially because Facebook is so bullish about AR and Apple, so bullish about AR. And so I think people are finally thinking that even if they won't be perfect that something's coming, you know, it's now putting out their spectacles Niantic announcing their glasses. And so suddenly, it seems like even if it's not a quick win, it feels like investors are finally thinking that this is something that we should be starting to look at. And I hope that bleeds into into games, you know, because there's a lot of people investing in video games right now. But ridiculous amounts of people investing in video games right now. But it seems to me that most of it is focused on very traditional on console games, or mobile games or PC games. They don't really want to touch the other space. And so interestingly, the investment side of AR is more focused more from the investment guys, we're looking at technology are looking at blockchain and NF T's and things like that. They're all starting to want to talk about AR, but I don't see a lot of the game investors quite yet saying that's a valid platform for games. We're happy to invest in that. Yeah. Yeah. has to be a part of something else. You know, you can if you can combine it with some other thing. metaverse blockchain other things, then people are starting to want to finally talk about AR

Yeah, let's wrap up with a few in lightning round questions. Okay. What commonly held belief about spatial computing? Do you disagree with?

Well, the term metal versus a mass as you and I discussed a little bit earlier in touch upon? And I think it's because it seems to mean everything and nothing all at the same time. It's so it just does my head in. Everyone's claiming it as there's you know, fortnight's and metaverse, VR chat and record their meta versus people doing NFT stuff, meta versus AR I don't even think it's a jumble of a term. And I think also when we think about what the metaverse means, I think that there are quite a lot of people out there who see it as like Ready Player waters, no crash, that 10 years from now we're all gonna be playing around the same world. And that's our sort of Second Life. And I don't think it's that I don't think first of all, as a society that we share interests enough that we could ever come up with a world that we'd all be happy to, to inhabit together. And so to me, meta verses are going to be a variety of different experiences. There'll be some some game worlds that we can jump into and chats and stuff like that. But it's not one metaverse, it's going to be lots of them. It is my opinion anyway. And I think right now, is about creating the sort of standards that allow you to move things between those metal versus No, which is why the sort of NF t stuff that's happening is kind of interesting in terms of how that allows us to migrate things between virtual world, right?

Yeah, I haven't quite wrapped my brain around what the term is meant to mean. And I know much less what it actually might mean, you know, that I don't like the connotation to the Snow Crash dystopian sort of world. I completely agree that it's, I find it nearly impossible that we as a human race could agree on a single alternative reality, you can't even agree with current reality, or even close a small geographical area, we don't agree on physical reality. So a virtual reality seems even less likely. Yeah, but whether it's just the 3d version of a website, or meant to represent, you know, what is the website? Is it just a single online page? Or is it you know, a set a full SAS app? With all these kind of tooling behind it? What is what is the definition of, of a website? And does that really mean in practice? Or is it really more about the set of capabilities in the in the screen through which you interact with it? Yeah. And if we decide that okay, if, if, if there is some sort of 3d element in it that we're now going to rebranded as metaverse, then I don't know, maybe that's what what's meant. Anyway, nonetheless, as we have minigames and websites, and the different experiences, movies, whatever you can plug yourself into today, we will have a variety of those things as they are many, many different 3d types of technology. How are you experienced it? Whether it's through, you know, the 2d 3d s two and a half D three, your phone? Yeah, does not truly 3d, right. It's a single view, or it's through a pair of glasses, or whether that glasses is see through words, or it's, you know, video pass through AR VR directly,

I like to hope eventually the glasses will be one thing. And I know that there are a multitude of technical challenges to get there. But you'd like to hope that eventually we'll have glasses that can can handle both of those things. And sometimes they are and sometimes they fall over and you can have your VR experience. You'd like to hope that within decades, we'll we'll get to that.

I think that's probably the right sort of thinking the timeframe to think about that sort of thing. Yeah,

I'm not being an engineer. I know that there are profound reasons why it's hard. So but but I like to hope that we'll get there eventually. And that's because users don't care. They don't want to know if it's AR VR xR better versus they don't understand that people just want to have experiences. And that's right. So

yeah, I think that's exactly right. I think that the current there's so many technical challenges, the current challenges, we can't make a good VR rig yet. We can't make a good AR not even close making a good AR rig. Yeah. So to suggest that we can make one thing that does both of those well Principle. So long ways from that. And there's definitely trade offs. Ultimately, the VR and AR or the AR and VR, depending on the physical environment that you're in which one of those two might make sense? more sense might be the way that the camera phone has gone evolved over time? Right? It's a piece of piece of junk camera initially. But it's the camera you have with you. And so you use it. And then over time we invest enough in it to make it pretty darn good. Yeah, make sense? We shall see. Anyway, onto this next lightning round question. Besides the one you were building, what tool or service Do you wish existed in the AR market?

It's tricky. I think the most frustrating thing for us is the varying performance between mobile devices, it's really hard to think about, you know, something supporting Apple and Google right now. And then even within those things, the various functionalities. So using, you know, LIDAR, for example, on the iPhone 12 Pro, gives you a blindingly good experience. As soon as you go back, just one to iPhone 11. It's not the same experience. And then, you know, Android, I guess, androids announced the pixel six, I'm hoping and praying that it's something like LIDAR is coming in. And I can't imagine how it's not, but they haven't said yet. And so anything that can help to bridge that and make it easier to have an experience where you can think about the multitude of devices, it's why console is so much better to develop for then PC, right? You know exactly what PlayStation five is capable of. And so when you make a game, you're on a fixed format, as soon as you think about PC. And it used to be worse, when machines were slower, you had so many different setups to think about graphics cards, and everything else. mobiles, mobile AR is the same thing. And because the App Store requires you to support everything back to like an iPhone six, with a high end AR experience, it makes it really hard to try to push any bounds because of the fact that you get you'll get nailed by the App Store reviews, as soon as it isn't performance on, you know, a specific person's archaic device. So any anything that would exist in AR foundation tries to do this. But the more we can push towards making that gentler for developers, I'd say the better you know, object recognition stuff and machine learning to so that we don't have to rely so much on LIDAR, you know, anything else that we can do to understand the space is super important. I think fundamental to AR gameplay being good is having some ability to know what's in front of the user. Otherwise, what's the point? It may as well be VR, if we can't have any understanding of the scene, it just seems like AR is worthless.

You're here you're talking about not just having kind of a geometric understanding, but some sort of semantic understanding. Yeah, yeah. as well. Yeah. Yeah. What book Have you read recently that you found to be deeply insightful or profound?

I feel like I'm playing to this conversation. But I would say fall or dodge in hell, I've got I've caught a lot lately. Neal Stephenson's book, have you read that? I've not really good. It's very long. At 600 some odd pages, and it really splits down the center, it becomes a very different book. But it's typical Neil Stevenson sort of future of what we can expect. And everyone's got AR glasses on. And the the implications of it are amazing. He talks about everyone's got the glasses, but it's all filtered to who you are. Right? So if you're conservative, the only stuff that you're seeing on your screen is the stuff that feeds that cycle, right. And you pay people to edit your news feed so that you don't even have to see the stuff that runs sort of contrary to your your opinions. And I thought that was terrifying. I mean, we're already seeing it now. But can you imagine in the future if it's on your head, and all you're seeing is the stuff that reinforces your your stereotypes? But going back to the idea that how could we all live in one metaverse together? You know, it's all about this sort of simulated future. And you can't that I guess the answer is you can't we can't all live together in a world and fighting is going to happen. But yeah, it's a great read.

All right, I'm definitely check that one out. If you could sit down have coffee with your 25 year old self, what advice would you share with 25 year old Susan,

I've given other people younger than me the same advice I would say don't be afraid to take on stuff that you're really not sure you know how to do. And you know, I I've been I've encountered that constantly in my career, the feeling that I'm a little bit out of my depth. And I'm taking something on and I'm not quite sure I even know where to start. And you have to just you have to go with it. You have to surround yourself with great people hire the people who know the things that you don't, you know, that was, I think the reason that the fictioneers stuff went so well was we hired an amazing team. We hired like two two great leaders, Richard sackers, who came out of Vodafone has has been our grant operations for three years. And I've never experienced I've never experienced being around someone who was just so solid, the sort of person who could just, you know, if he said something wasn't a problem, it really wasn't a problem. And it's just so wonderful being around people like that. And so so that's the answer is you if you don't know it, you find out the answers. But I think you really have to be willing to throw yourself into those situations where you feel uncomfortable, and that's the only way you can grow.

Great advice. Great advice. Any closing thoughts you'd like to share?

Oh, The biggest one is I really don't think it's too early for all of this, you know, in terms of if there are investors who are listening to this, I hope people start to take risks. It's not going to be the quick wins. But if this is gonna work like now's the time to I'm so I'm getting so excited and motivated about the stuff that we're doing, even if it's going to take time to sort of realize the results of it and the financial wind of it. It's just so cool. We have this like, you know, once in a generation opportunity to figure out what this this this metaverse stuff, or whatever you want to call it is, and you know, playing an AR playing xR pushing the boundaries. It's just really neat. As a game industry person, it'd be really easy for me to just go back to making traditional games right now. And there's so much money being thrown out there for making traditional games. And so it's taking a lot of courage, I think, to say, that's all fine, but we're doing this new stuff, and we're going to be there. And that's that's something to get out of bed for in a way that I think just going back to sort of safe traditional games wouldn't feel for me right now.

Yeah. Where can people go to learn more about what you're doing that tiny rebel games and more about you generally,

our websites, tiny rebel games.com, which is where you can hear about the stuff that we're doing a tiny rebel and I'm on LinkedIn and on Twitter.

Awesome, Susan, thanks very much of the conversation. Thank you. Before we go, I'm going to tell you about the next episode. In it I speak with Paul powers, the founder and CEO of FISMA. We have caught fizzing in the news recently because they raised $76 million over the last year including 56 million earlier this summer from Tiger global Sequoia and others. Pon his team have created a new way to search and compare 3d models in a way that's faster and more effective than what come before. This ability to recognize and understand physical objects combined with things that are online database of 3d models, creates the potential for them to become the Google search of AR Paul's a great storyteller and an insightful entrepreneur. I think you'll really enjoy the conversation. Please follow or subscribe to the podcast. You don't miss this or other great episodes. Until next time.