The AR Show: Kirin Sinha (Illumix) on Crafting Compelling Content & Experiences for AR

4:27PM Apr 19, 2021

Speakers:

Jason McDowall

Kirin Sinha

Keywords:

ar

game

lumix

people

build

consumers

world

thought

life

experience

company

spent

create

kirin

technology

platform

exist

feel

fundamentally

computer vision

Welcome to the AR show right dive deep into augmented reality with a focus on the technology, the use cases and the people behind them. I'm your host Jason McDowall. Today's conversation is with Kirin Sinha. Kirin is the co founder and CEO of Illumix a company building an AR first mobile gaming platform. You may be familiar with Illumix if you're a fan of the horror franchise Five Nights at Freddy's. The Illumix team has already found a lot of success with their first title Five Nights at Freddy's AR Special delivery.

Prior to founding Illumix, Kirin was the founder and executive director of shine for girls whose mission is to transform the lives of middle school girls, by cultivating a passion for mathematics through a program that incorporates both math and dance. Kirin has a BS in theoretical math and computer science from MIT, as well as three master's degrees spanning mathematics, machine learning in business from Cambridge, the London School of Economics in Stanford. In this conversation, Kirin shares how she thinks about creating compelling stories and gameplay for mobile devices where AR is a central component:

Every other type of content that we've had to date, whether that's, you know, movies, website and app, you know, existing 3d games, even VR games, honestly, it's all built within its own world, you have this one to one relationship as a developer to consumer of, I'm going to put you through this flow or this experience. And you as a user are going to have exactly that. With AR that construct completely breaks down. I have no idea. If you're sitting at home, if you're in an office out in the field, I don't know what your environment is. And it deeply changes how you receive content. And so we felt very strongly from the beginning that in order to create valuable AR content that immersion had to come from content that in some ways adapted to its world, which meant that the way you have to think about content from a creation from a tools from a tech perspective all have to be fundamentally different.

We go on to talk about how the Five Nights at Freddy's franchise is a great fit for AR and also why the Illumix team needed to go beyond AR kit AR core and unity in order to deliver their vision for compelling AR first mobile experience. Kirin talks about her broader ambitions and also has some amazing advice for building and maintaining self confidence. Let's dive in.

Kirin, a lot of college students dream of spending a summer traveling around Europe. But yours is a different sort of experience. What was the motivation to spend a summer in a convent in Italy?

Sure. So I spent after my freshman year of college, I went to a convent in the very northern part of Italy, in the mountains in a place called monado. And there are a bunch of different reasons I think this could have happened in part, I had actually not graduated high school I had gone off to kind of pursue professional dancing, got injured and then ended up going to college. So I felt actually that moment in my life, a little bit of an identity crisis, like I didn't quite have that same amount of clarity. I think one of the incredible things when you grow up and you have such a strong passion for something and for me that was dance is you almost feel like you have a career at a young age, there's incredible amount of focus in the sense of I know exactly what I'm doing. And by the time I got to college, I think like most students, I lost that I didn't have a sense of this is really what why I'm here and what I'm trying to do with this experience. And so I wanted to take a moment, almost a time out from my usual life and my own perspective and do something really, really different. And I was encouraged that, you know, summers of college are a good time to do that, instead of just purely internships and kind of building a resume. So I took advantage of that. And I've watched the sound of music and ungodly number of times, which I am convinced played some kind of mental role in this. And I went off and I did something completely different. And it was actually the most I think, formative experience of my life. It was the first time I was ever truly alone. And I mean truly alone if I tried to learn Italian prior to the summer, and had I actually paid attention and done a little bit more research, I realized I'm in a German speaking town, which so I completely couldn't communicate. I didn't understand. I mean, I have all these crazy stories of the doors being locked very early, which I had no idea because it all the signs were in German that alerted me. So I'm like climbing a tree I mean full sound of music style to try and get into my room at night and dropping in amidst a number of nuns doing some kind of prayer group outside just staring at me like who is this girl? They put me in the highest tower underneath the bell after that as punishment. So I would vibrate awake at 5am it was just so it was a wild summer, but I had an opportunity to just sit with myself and do a lot. I read a ton of books during that summer. And I think that was when I really became passionate about storytelling and thinking about what different worlds could exist in just a little distance from your own world and spending time in other people's worlds, I think actually gives you a really different idea of what could be around people or what kind kinds of experiences people have, or just the types of worlds that we could exist. And I think ultimately a lot of that passion translated into what what's now a Lumix of thinking about the world and looking at spaces and thinking, what kind of stories can I tell? I mean, that one little phrase, like, once upon a time is probably one of the most powerful phrases in the human language, I would argue,

it puts you immediately into a mindset that oh, my God, anything is possible right now, once upon a time.

Yeah, exactly. It's just those little words. So you believe that you can be anything and you could be anywhere. And I think that that essence really grabbed me, I think during my time when I was in the convent, and really spending time on my own, and those were my companions. And that's the kind of spirit like that that really captured me, that's when I kind of knew that I wanted to do something that had some kind of storytelling, some kind of creative aspect to it, ultimately, long term.

Yeah. When you went into this program, was there a curriculum that was intended? Or is it really about being alone, being by yourself with your own thoughts and trying to explore who you are,

it was really about being alone, there was no curriculum, they, the nuns were just as confused by me as I was by everything going around me, they could not figure out why this small little Hindu girl was sitting there, I did try to make them my friends. I did like very publicly read the Bible and things like that. But limited success. It really just was about living a very different type of lifestyle. And, you know, I spent a week and a half in total silence, I really tried to push myself into you spending time cleaning up your own thoughts and getting a sense of who you are separate from all the usual day to day distractions.

Yeah, well, I can only imagine in that my age now, right? There's so many more thoughts, and so much more baggage that I carry with me and to have so much alone time with those might be might be helpful, but uncomfortable.

I always liken it to like deep cleaning a really old rickety house, like, that's kind of how I think about my brain of actually spending some time to really organize and deep clean helps me maintain the regular maintenance work versus kind of constantly trying to take stabs at it. So....

Yeah, that's great. What an amazing, informative experience. As you noted, it said that you weren't a normal high school path, you didn't go through the normal high school graduation in college, you had pursued dance professionally. But when you did get to college, you ended up really focusing on math, you are a math nerd, when I look at your resume, which I have a deeper appreciation for. But how do those two come together for you how to dance and math originate for you.

So math was my first love. I knew from a very young age, and I was accelerated in math and very involved in you know, high school math competitions from a young age. And so there was something really good in math everyone thinks about it is very structured and analytical. And it certainly is, it creates this logic of how to think about problems and break down problems. But it's also deeply creative. When you get really I think, because I was younger and able to become accelerated, I had earlier insight than most into proofs and, you know, analysis. And Kelly, you know, it's so much more than calculus is just the ABCs. You get to read Shakespeare in high school, or even Middle School sometimes. But you've never hit that point with math until you're much much further along. But because I got it younger, I really, I was very passionate about it, I think dance, I mean, originally, I got into it, because I was a huge dork, I would just fall over standing up, and my parents were deeply concerned. And so they put me in dance as a little girl. But I think it's a similar confluence of kind of, there's this creative and this very passionate part of it. And it's also but just as much as about precision and discipline. And I think that those two come together, really in a beautiful way. And I think that both math and dance, let me express that differently. One, you know, very, in the mental space and one on the physical space.

So you brought these passions for math and dance together at an organization you created called shine for girls in service, I guess as an undergrad, right on your MIT, what was the goal of shine?

So the intention was never actually to build what what I guess now looking back I see was my first startup. It was just I very passionate about women in STEM, for obvious reasons. I was accelerated when I was young, even at MIT. The gender balance within the courses I was taking was incredibly lopsided, I think it was probably closer to 8020. And there were several courses where I was the the only girl in my class. And I do believe that fundamentally, that starts young and so I was tutoring in the area and doing after school programs to help teach and volunteer. And what I noticed was the girls would say phrases like, I can't get this question. I can't understand fractions, whereas boys would say, I don't get this. I don't understand fractions. And I thought that single word can't versus don't, was actually kind of underlying In a much bigger issue, the girls didn't fundamentally believe that they could get it, whereas the boys did. And I thought back to just my own life experience and what made me think I could do anything or what made me unafraid to stand out, we did surveys, and one of the most shocking answers for me was something like 70 plus percent of girls said, even when they know they're good at something, they would rather not stand out for it. And that shocked me, because when I was good at something, or you know, when you're doing really well, in dance, did you go in for the solo spot, right? That didn't that excited me. And I thought that dance was really where I had learned that confidence, that willingness to stand out that willingness to think about everything as if you put in the work, and you really throw yourself into it with kind of, again, that discipline and that passion, you can you can do anything. And I think that that is the same thing it takes to succeed in a male dominated field today. And so that's where it just was an experiment to start out with. And then the results were staggering. And next thing I knew I had two or three programs, and then I was raising funding, which means I needed to incorporate a company, and it just grew very organically on its own. And we had a ton of inbound interest in press, because the combination really worked. I mean, it's the, I guess, now looking back, I have all the startup lingo that I didn't know at the time, but it was that product market fit of it was something that actually attracted the right girls, everyone wanted to join an after school dance program, whereas not not everyone wanted to do after school math tutoring.

Yeah. Where did your own confidence come from? You noticed this in other girls that there was this underlying belief that it's not just temporal, it's not I don't understand it yet. It's I won't ever, but you weren't held back by that same belief,

I have to give a lot of credit to my mother here. She never let me stop believing in myself. I think that confidence is a skill, it's a muscle that you exercise, and you eventually build up. It's every time that you think maybe you can't do something. And even if you fail, or even if you succeed, it doesn't matter. The fact that you did it is what builds that muscle. And everybody, every little kid, every little girl, it doesn't matter has moments where they think they can't do something, and my mom would make me do it. And she wouldn't just make me do it. She actually made me get on the piano bench. This was her and she jokes about this today, but she literally would make me stand on my piano bench, and scream, and she would make me scream until it was really screaming, I am the best or I can do anything or you know, some phrase like that, because she really felt like when you stand up and you declare to the world, and when you hear your own voice saying to you, you register it very differently, and you start to believe in it differently. And there have been moments in my embarrassingly my adult life and my startup journey, my college journey, where, you know, I had those crises of confidence. And she made me stand like on a patio or, again, something wildly embarrassing in public and scream something like this. And it does. She is crazy, but maybe a genius, it really did make a big difference. And so I do think ultimately, your confidence, it comes from you pushing through at every single moment. And really, I think you have to view it as a skill or a muscle, it's not something inherent or something you get from one given source.

Do you think that the kind of taking on this analogy of confidence is like a muscle? Does it atrophy as well?

Yes, yes, I think it certainly does. I think if you spend too long in comfort, meaning I think that, you know, one of the things even dance perhaps gave me this is I think any athletic endeavor, or artistic endeavor, anything, there's a willingness to constantly push yourself outside of the comfort zone, you live for that willingness to be uncomfortable. And I think that, that if you sit too long, in a position where you're not pushing yourself, in whatever situation, that muscle does, atrophy becomes harder and harder over time, there's inertia, it's harder and harder to push yourself out. And there's more and more fear that builds up that you can't. And so I think it's really important, throughout everything you do to constantly, maybe not at all at once, and not in every aspect of your life. But I think that the willingness to push yourself and try and make yourself better and build up that confidence again, and it doesn't even matter if you fail, just the fact that you did it, I think is enough to keep that muscle getting stronger.

Yeah, I think this is such an amazing perspective to have and to be able to share, especially with with young girls. This notion that just having gone through it will help you get better and stronger. is so powerful as we've kind of outside of this conversation. I have a couple of daughters also. And I think deeply about how do I instill the sort of confidence that I hope they can have that I wish I had when I was when I was young and going through the same sorts of challenges and opportunities that they face day to day. I might take what your mother's done and make them stand on the piano bench and and scream to the world. What a great idea. But you went on to complete a handful of master's degrees across mathematics which Learning in business, and then you found your way to co founding your second startup. First, maybe when focused on profit. That is an AR FIRST Tech in game company, how did you find your way to specifically an AR first sort of experience.

This was a company, I started out of the technical passion and the potential for what I felt the consumer experience could be. I was very passionate about math. So my freshman year I went and I told the MIT professors, I'm going to do research under you. And they did what you do with every overexcited freshmen, you sent them to stats to kind of tamp down their spirits a little bit until they're tolerable again. And it was just it was incredibly fortunate. For me, it was when before machine learning had all of the hype behind it that it does today. And it put me in the kind of big data space and I got to see, you know, the emphasis around cloud computing happen, I got interested in machine learning when I was in school, because I studied math, Electrical Engineering, and computer science. And it was just really right at the center of all of those fields was AI. And so I went off to graduate school, I knew that that was my interest. And computer vision specifically was what I was most passionate about, and what I was spending the most time thinking about. And when you started to be able to run this type of algorithm on on a mobile device, the idea that every single person has a camera in their pocket that you can put to use, that was my hair on fire moment of, Oh, my God, I have to do something, I thought at that point, I thought I was gonna be a professor for the rest of my life. But that really kind of caused me to drop out of that purely PhD centric route, go and decide to move to Silicon Valley, got a business degree, and actually build something all these worlds that I had spent years now thinking about, you know, from when I was a kid and loved Harry Potter to when I was at that convent, and, you know, reading all of the greats and literature and Moby Dick, and you know, all of that, it just became clear to me that we could use something that everyone already has to bring them into a different world and make them feel powerful, right. One of the most interesting things about AR gaming and experience as an entertainment for me is that you're the protagonist, there's something really exciting about that you're not choosing an avatar and playing through a game, you really become that person. And I think that that was also something that was just thrilling to me, and so that I came to California, and I came to Stanford, and ultimately, you know, built my company with a very clear vision that I was going to leverage the power of these infinite number of mobile devices to bring people into a totally different type of world In reality,

what was that full vision that you had pitched to those early investors and your early teammates? For

the early investors, it's a simple, it's the vision has stayed incredibly stable. I think, ultimately, to make an AR experience compelling, you need a completely different way of thinking about it. If you think of every other type of content that we've had to date, whether that's, you know, movies, website, and app, you know, existing 3d games, even VR games, honestly, it's all built within its own world, you have this one to one relationship, as a developer to consumer of, I'm going to put you through this flow or this experience, and you as a user are going to have exactly that. With AR that construct completely breaks down. I have no idea. If you're sitting at home, if you're in an office out in the field, I don't know what your environment is. And it deeply changes how you receive content. And so we felt very strongly from the beginning that in order to create valuable AR content, that immersion had to come from content that in some ways adapted to its world, which meant that the way you have to think about content from a creation from a tools, from a tech perspective, all had to be fundamentally different, that this was something that wasn't going to fit nicely inside of existing tools and needed its own approach. And that's really, from the beginning what Lumix sought to be, I think, we were very vertically integrated. And so you know, we pitched it's an early field, it's an early space still is, and the strongest companies always come from those who directly touch the consumer and able to build the right set of tools. And so we went out and came out as a gaming company and said, Okay, we're going to touch consumers directly we're going to know what resonated What didn't resonate, you know, what, what can we make that is a compelling experience, does the thesis that we're basing our entire tech platform on actually hold water or does it not and then build backward into the tools that we need to create those types of experiences.

So it's very comprehensive the vision that you had wasn't just simply to take unity and spin out some another mobile game into the world and throw little AR speak little AR on top of it. You know, I think at the point you'd start the company Pokemon GO had already seen its initial success, which was phenomenal. And it's, it's talked about as the the ultimate example of what mobile AR could be. But in practice, there's almost it's mostly geocaching game, not to diminish the the impact that it has on people's lives and the amazing experiences they have within this game, but its actual exposure to augmented reality is incredibly light, as you kind of thought about this different perspective, right? There's this notion of putting somebody in the story making the game player, the protagonist of that story. What are the sorts of new interactions that mobile AR enables? in the entertainment experience? Yes.

So I think that what Pokemon GO did, you're absolutely correct in that it's primarily a GPS geocaching game. But that mechanic was such a good fit for the higher level concept of what is the fantasy of Pokemon, it's that you are a trainer and you know that Bulbasaur outside is real, you're in the Pokemon world. That is what Pokemon GO gave people and why would still consider it a wildly successful mobile AR game, even though it is our light illuminates always had the opposite. In some ways perspective of the point was not to have people go out to the game, which is I think my antics core core thesis is go out to the world. The idea was to transform the world, any world you're in, have the game come to you transform your environment to make it something more than what it was. And so when we thought about what kinds of experiences there were, there was this very clear sense that we wanted the user to be the protagonist, we wanted the phone because you have to think deeply about form factor to be something that inherently made sense for why you were holding it up. So the phone had to be some kind of tool that you were using in your environment. And whatever experience it was, had to be something that was visceral, something that where you could feel in your guts differently than how you feel when you're playing a mobile, you know, traditional mobile game without AR, that something was real and coming for you. And you were going to react as though that thing was actually happening. And that's what made I think five nights at freddy's such a good fit. Because and fundamentally, Five Nights at Freddy's is not about collection. Right? That's not that's the Pokemon world but five nights and Scott who is the owner of the IP knew from the beginning that he didn't want an AR game that was just walk to a place collect, collect an animatronic that didn't make sense. What it really needed to be was a nightmare come to life, right? It needed to be, oh, my god, these things are real. And they're following me around and they're ringing my doorbell and they're at my house that I've never safe again. That was kind of the mental picture we wanted to paint. And I think that core concept is what no matter what we do in our tech world, no matter what we do in our game design, it's always the thing we have to greenlight and come back to is does it fundamentally answer that core user fantasy of these things are real and attacking me and alive and in my world?

Can you take a step back and kind of walk us through the Five Nights at Freddy's franchise that existed before you had met Scott? And why you thought this was such a good fit for you kind of talked about this notion of the game coming to you but kind of walk us through the experience you've created and why it's a good fit for for five nights at freddy's

Five Nights at Freddy's is an indie horror franchise. I came across it via YouTube several years before, Lumix existed. And so it was something I was already familiar with. And when we really thought about what franchises would be a good fit for AR what is something that is unique that people haven't done. And it's an experience where you viscerally feel as though your reality has been augmented as though there's something that is there that isn't that's transforming your actual environment. We thought horror was actually a really interesting fit. We had a lot of pushback actually on this because most people said horror is actually about constraint. And not giving you choices. And doing an open world war game is incredibly difficult an AR doesn't make any sense. What if you're out in the sunshine, but we really felt strongly from the beginning that this franchise, this idea of almost predator prey dynamics of your being hunted, that's a hide and seek, these are things that work. And that made sense that we could really execute very well with AR and that fits no matter where you're playing. So most people will be playing at home certainly during COVID. But even before that most people are in their usual locations when they're playing games. And the idea of taking something that's so familiar like a like your home, and turning it into a haunted house, and actually making these things come alive and be tracking you down and you're trying to survive. We thought that that made a lot of sense. And so the survival horror genre and you know, these these animatronic monsters in particular, we thought it would be a really nice fit

as a game player. In a content consumer horror is probably at the bottom of my list of things that I look forward to. I for whatever reason, don't enjoy a good scare, maybe because it rattles around in my brain for far too long after after the initial experience. But I had a chance to play your five nights at freddy's game. It's really quite compelling. It's a really nice utilization as you noted the phone as a tool that's essential in your pursuit of the of the core game mechanic, which kind of has a bit of collector element to it. But there's also this kind of hide and seek and capture elements to it. That was really I enjoyed it more than I was expecting to enjoy it. So that was really cool. I think about the startups that I engage with and they think about as they kind of begin to resent the monumental vision that they leave companies often have even taken that first bite as often as huge, a huge chunk, but you decide that not only you're going to be creating the content itself, and granted, you're not going to work with a franchise that has a really passionate user base, and is really well formed a good fit for the type of experience you have to make. But not only are you creating this content, you also decided to create all of this underlying technology underneath it all at the same time, that's a really big bite for started to mate. Why not just kind of rely on unity or AI or kidney, our core to at least get started? what compelled you to kind of go beyond what was generally available in the market in terms of technology.

So we did leverage existing tools in order to stand something up initially, over time, we've migrated to our own engine and platform completely, but you know, the initial launch, and we still do leverage parts of AR kit and air core. I think, ultimately, for us, it's the backwards approach, have we thought about it purely from what would make this a great game? Right? What would make this the right product for consumers? And once we have that answer, we went back and said, Okay, what exists today in the market? and what doesn't exist? And how can we fill those gaps. And then over time, as we actually saw that first part of our, of our thesis be validated that first assumption of this is the right game to build. This is the type of immersion and content that makes AR valuable to a consumer, then we were able to go back and, you know, we had to continue to adjust the platform, it wasn't just because we wanted to have our own tech for tech sake, it was that existing platforms had real limitations, both in terms of technical abilities in terms in terms of, you know, we had a lot of our own computer vision, when we had a ton of graphics optimization, there were things that were fundamental to the platform that didn't let us perform the way we needed to, we had to create our own engine to expand compatibility to make it even more mass market, which is, you know, hugely challenging. And as a result of that, and dealing with a very fractured hardware and, you know, operating system ecosystem that exists out there, particularly on Android, we need to have a system that was making smart choices about what to turn on when and how to limit, you know, certain graphics, or how to limit certain types like which which model you're running on computer vision to make it the most optimized experience for everyone without the consumer seeing a glitch. So there were a lot of things that we felt were fundamental to creating great content that the existing platforms just don't have. Because from their perspective, ar is not their full business, a lot of times it's an offshoot of what's a larger business. So it's kind of good enough, is I think, the perspective and all of the big platforms you mentioned, are coming at it from a purely tech platform point of view. So I think that actually, there's a lot that's being lost in translation, between what actually consumers use our AR game, over 70% of all gameplay time is spent actively in camera mode, looking around interacting with their space, I don't know that there's another AR game out there that has that kind of level of interaction, and it creates different opportunities for the company. But it also teaches us very specifically what types of things consumers want and what types of things we have to be able to enable if we're going to continue to scale. And it just necessitates its own technology. It wasn't that I wanted to come in. And we didn't do it the other way around, right? I didn't build a tech platform, and then build the game, I built a game. And then from building the game, we built a tech platform,

there are a couple of keywords that you use when you describe the technology on your website, and I want to pick them apart, and you can translate it for me a little bit. But one of them is this notion of being environmentally intelligent. So I'm guessing it doesn't really have anything to do with climate change, probably more about an understanding of what's happening in the space, what's happening in the image itself. But can you describe what is environmentally intelligent mean?

Yes, sure. Yeah, it is not climate change centric, it is. It is about the computer or the you know, the phone, in this case, itself, being able to glean things about the environment you're in and make choices about the type of content you see. So people in different spaces, not just GPS, people in different physical spaces, will experience different types of content and different game modes based off of those spaces. So it's not, again, that goes back to this kind of core thesis from the company of it not being one to one of everyone is going to follow a specific path. It's going to be based off of your space that we will we mean it and you know, the algorithm will decide what you should be seeing and what makes the most sense. So it's very adaptive, immersive approach to content?

Can you give an example of how it would be different and different depending on the space?

Sure. So if you were in I'm in a very small little office right now, with not a ton of space nearby and certain types of objects that probably would tell tell us that you're in an office setting. And so we might in you know, remnant collection mode, which is a game mode in the game presents different types of experiences, different types of this case, remnant. So it's a different type of kind of game currency that will appear and be attached to different types of objects, right, that's different than if you were out in the middle of the field. There is nothing around you, we don't see any objects for miles, we're going to put you in a different style game mode. That's probably got big animatronics coming at you that you're going to fight. So that's a, that's an example. Got it? Yeah, so AR kit AR core unity, those types of platforms really don't give you any information about the space around you. And I think that that actually is a part of why AR has struggled to take off. A lot of it has been taking the technology and figuring out what can I do with this instead of what should be done? And how do I build the technology for it. So there's been a lot of this sort of random, go find an empty space, or go to a tabletop and plop something down and then hold up your phone where that's not the dream experience. That's not really what's compelling to consumers. And chances are most of those experiences would be just as good if not better when they're just done on the phone, in mobile. And so for us, we think very deeply about how is AR making this experience different? How is it making it valuable? How is this something I couldn't experience without AR,

when it comes to the tech platform itself, specifically as it ties to AR and the computer vision, what's uniquely challenging about operating at world scale beyond what a normal very successful mobile game might have to deal with?

I would argue that the hardware ecosystem and the operating system is actually much, much more challenging for AR because it's more heavy, like we're doing AR is more heavily reliant on those systems. And so as versions change, as different devices come to market, as we try and capture, you know, a lower end of the market, it's actually not just a gameplay question or an optimization graphics question. It's a sometimes it's a very fundamental computer vision or, you know, slam question of how are we going to enable this properly. And so that's all stuff that the engine has to decide and figure out how to do on its own. And I don't think normal mobile games have to contend with that. With that intensity.

Yeah, resource constraint really taken to an extreme and device diversity really taken to an extreme here,

it is extremely, this is a this is a film quality game. Right? If this was one of the this is one of the big challenges is this was one of the flags we planted in the stand was we're going to have this be a beautiful graphic. And if it's a nightmare come to life, it needs to really kind of feel realistic in that way. And I think many AR games have a, you know, lower poly approach in general. And so how do we have something that's running full AR capabilities, our own computer vision algorithms on top of that, and has complete graphics and is a pretty heavy game honestly, in and of itself, the optimization and graphics of that is that it in and of itself could be its own company,

on this note could be its own company. With all of the work that you've done, you've created an amazing game with five nights at freddy's Special delivery. But often for independent studio leveraging, even if it's popular IP, there's a there's a massive challenge to getting the word out. How from a still a young startup early on in the process, how did you get the word out about this sort of experience you've created.

This has been one of the more personally fun chapters of the company, because in some ways, we would never have expected it to have taken off the way it did. We actually did it almost like an Easter egg style hunt, where we have been leaving random Reddit messages with hidden messages inside of them. Over the course of weeks, we had a secret Snapchat AR filter that sort of implied different things. And so if you were following every little hint, you could piece together what was coming. And eventually, that's what the community did, we had no idea that it would blow up. I remember so vividly that day where everyone basically stopped working because we were like, Oh my god, all of this is going viral right now we had a random Reddit post called new phone who dis. And people were like, if this is a fake, it's the most elaborate fake we've ever seen in our lives. And so it just it became one of these things where we got massive, massively talked about on Reddit, on YouTube, on Twitter. And it really took off and it created a huge amount of anticipation. And we really, we leaned into YouTube and Reddit very heavily. So you know, we've never spent on marketing, really, we've never spent on ua, we've never had to do that because people themselves AR is actually I think, almost inherently viral. It makes the world around you more interesting, and especially a younger generation a Gen Z, they loved it. There. They are content creators at heart, they love to post about things that make their world more interesting. And so we had a lot of natural publicity, I think on YouTube and continue to and that's really helped bolster the game as we've continued to evolve it over time. But initially, it was a snapshot AR filter and a single Reddit post and, and we got a lot of excitement and intention. And we had a really I think we have a really amazing trailer also. So I think that that also took us to a whole new level as we put out a really compelling trailer to kind of outline the differences between this experience and what has triggered been an AR game to date, just to really hammer home. This is not the Five Nights at Freddy's version of Pokemon Go. This is our our completely new type of game.

Yeah, the trailer was really awesome. And you expertly masterfully leaned into the fandom around five nights at freddy's was, could not have been better executed as far as far as I can tell, looking back on this, as you think back now, it's been almost a year and a half or so over a year since launching this thing. And we've talked a bit about the the challenges on the tech platform itself, and just huge challenges of making sure that across the huge diversity of devices out there that everybody gets an optimized experience given where they are, and given what they're holding in their hand. Is there more that you've learned about delivering compelling mobile AR experience since watching the game over the last year plus,

when we launched the game, really, our intent was to validate that core loop of the core AR experience was compelling, something that was interesting to people, we didn't expect it to take off and become as mass popular as I think it has become. And so our objective shifted, when it was live into taking this kind of core experience and idea and turning it into a full blown game really became about scale. How do we build this out while still saying really centrally core around AR. And so that has been the challenge that we've really dealt with while dealing with a live product over the last year is how do we continue to build out compelling gameplay modes continue to innovate around AR, but build out more kind of infrastructure and support to enable this to scale to you know, millions and millions of users and have, you know, have our mru and da u increase month over month, because the experience itself is getting better and fans are reacting and staying around longer, versus we're just kind of pumping in more money and into ua and bringing people into an experience and having them churn out. And so that has been a whole new type of learning it is what precipitated some of these tech decisions, like moving completely away from you know, the Unity AR foundation into the Lumix AR engine to increase compatibility to enable different types of game modes to continue to build out what we felt was an AR first game. And so I think that that's, that's definitely been the most challenging. And the most interesting part is building all that out on a live product has been a huge challenge. And in a relatively short period of time. We've built a lot and made a lot of very serious infrastructure transitions on a live product. And you know, a little over a year.

Yeah. Looking back, I can only imagine how much your team has built under the hood, and public view in terms of gaming this experience as you as you think about all that you've created. So far, all the lessons you've learned in how to create a really compelling experience. And as you said, to kind of get people to stick around because they're having such a great time, not just because you're spending marketing dollars on pumping people through through the party itself. Where do you go next with these learnings? Are you planning to release your own IP? What sort of kind of fantasy world do you want to you want to enable your users to interact with next? Yes, I

think that eventually, without committing to any dates, that Lumix will eventually will will release our own IP. It's something we've talked about for a long time. And I think we would take almost the opposite perspective of Five Nights at Freddy's, it's been so fun building a horror game and having a nightmare come to life. But it would be great to have more of a traditional fantasy dream world. And so I know that I and many of you in the studio are inspired by things like Studio Ghibli in that style. And so I think that we'll see something a little bit more fantastical and a little bit more mass market versus horror niche. And so I think that that's what you'll see next out of illuminate studios, when we whenever we do do an original IP,

what sort of role do you imagine a user would play in this sort of experience?

Well, we've toyed around with a number of storylines, I feel like I can't give too much away on this. But we do have a few storylines. And again, you're always the protagonists, but whether that's because you're, you know, someone who's landed there from our world into faraway land, or whether because you know, you are kind of being brought up in that land and making choices. I think there are a lot of very interesting stories to be woven throughout this, and potentially options for the user to pick what kind of character they want.

Yeah, I look forward to this for this. So you noted that the work you've done on the platform itself, just truly recognizing that if you think about the end user experience, and that map that back to the technology and the sort of devices that people actually have with them, there are gaps that go beyond what is available in Eric Eric or unity in your team has filled that gap. And in doing so, you've almost created as you noted, a separate company kind of just just doing this thing. And we've seen companies like epic with their Unreal Engine and Niantic with their real world platform, who have spent a lot have invested a ton in the underlying content creation are the underlying technologies to enable their games, and then they make them available for third parties. Where do you go from here with your platform? And how do you think about enabling others to build on your platform?

I think That is going to be the next step for Lumix moving forward, I think, you know, you nailed it with the epic comparison, that's definitely something we think about a lot in house of someone who's building amazing content for consumers, but able to refine and build a really powerful group of tools to enable a larger ecosystem, ar fundamentally cannot just be about a few small games doing well, or even large games, right? Doing well, it needs to be a more complete ecosystem for you know, all, all ships rise, kind of scenario. And so I think I really do fundamentally believe a part of that challenge is that the right tool sets don't exist, because so many people want to come in and just attack the tool set without touching the consumers. And I think you almost always build the wrong product when you do that. And so we are planning to open up our engine to third parties this year.

Amazing. So this is just just around the corner. Yes, just around the corner. Today, it's really about mobile AR, you'd noted that, you know, every person on the planet just about has has a phone in their pocket. And that creates an amazing base from which to build, you know, some really active amazing experiences and gameplay and communities. But AR glasses around the corner, there's all this talk about what Facebook is doing what Apple's doing what Microsoft and Google have attempted to do. Were the air glasses fit into your plans.

I think about this in two ways. So air glasses have been right around the corner for some time. That's just the reality. And so a Lumix has always from the beginning made it very clear that we were going to be a company that we felt could succeed and scale without the introduction of any new hardware. We're very focused on the mass market, ordinary consumer, the everyday person versus a tech enthusiast. And so I think even if classes appeared tomorrow, we would still have a few years gap before became a mass consumer device. And so we really are in the imminent future much more focused around what can we do with the latest versions of phones, which are becoming more and more interesting, right? latest Apple devices have LIDAR. And that's something that's really exciting and interesting to us more. So we spend more time thinking about that than we do about glasses. That being said, if glasses do appear and do become mass market, I think our fundamental engine is solving problems that aren't getting getting replaced by hardware. So for example, we didn't spend years and years trying to solve for occlusion or some of these very specific technical problems, multiplayer, things like that. We really thought about it from a content creation tools perspective. So what will we need to exist? Regardless of even if glasses were here? What are the problems people are still going to face of? And I think those are going to be? Let's assume you have every crayon in the box, technically, and you look out at the world, it's going to be what do I build? What's interesting? How does this become an immersive experience? How is it adapting to the world around you? So if the hardware on the glasses can tell me, you know, object recognition of everything, that's great, my business isn't based off the fact that we can do object recognition are we have specific computer vision modules, my business is based off of once you have those, I can plug those in and give you the power to build anything you want easily. And you know, well in a way that's going to connect with consumers. And so I think it would be a huge value add to the underlying business. If that came together. It would be my Christmas gift come early. However, I genuinely don't believe that that's going to be something that we have to deal with in the next two to three years.

I agree with you. I don't think it's that soon either. Certainly not at scale. As you think about the kind of ambitions we have that parallel may be epic, and somewhat Niantic Do you do consider Niantic a competitor? as they are also notionally playing in the general area that Linux is playing in as well?

I suppose so. Right? I guess I would consider Niantic a competitor. But I really do feel like AR is at such a nascent space as a as a market that I don't view it as competitors of Oh, I wish they weren't there. Oh, I wish we can beat them. But more I wish there were more companies that were out there and succeeding. So I think I antic going out there and succeeding would be phenomenal for us versus destructive for us. And so I would love to see more and more companies out in the airspace doing well.

If not by antic, then who or what in this industry makes you most nervous?

I would say it's a what? And I think the challenge I see it's it's similar actually in that I think it's around the ecosystem of I hear so frequently at our conferences, I go to our meetups, if only we had x piece of technology then the space will grow up I feel like that sentence itself always scares me I don't care what that axes I feel it very frequently. It's multiplayer for whatever reason where they're like if we had the ability to have everyone interact during synchronously and AR then that changes everything. And I guess this is my uncommon belief is I don't think it changes anything. I think that really people need to focus and we As an industry need to focus on the consumer experience of what is it that makes this technology valuable for consumers versus approaching it from a tech centric of If only we had x if like LIDAR will change everything? I don't know that it will, I think it will enable us to create more experiences. But unless people have really thought about what those experiences are and what it's going to be valuable to consumers, I don't think in and of itself, any one piece of technology is going to unlock a whole different world for the AR ecosystem. And so I think that headspace of once we have headsets that will be different once we have LIDAR once we have multiplayer, whatever. That I think is incredibly pernicious. And I wish people would just focus on what's valuable for consumers

and what's available today. Yes,

I agree, we have crayons, we just have to figure out what to draw.

As you look back over the last three of three and a half, four years or so in your first role as an entrepreneur, CEO for profit, we talked about the shine for girls was really good many ways for for startup experience. But if you look back at this last three or four years, that Olympics, how have you most changed as a as an entrepreneur, as a CEO,

I can handle the bumps much more gracefully than I once did. I remember when the company first started, I don't know if it was even officially a company yet. But every week, it was always Tuesday, I don't know why there I'd see something in the news that would send me into a spiral of panic of Oh my god, someone else is building what I'm building, or Oh, they've so much more funding, or Oh, they're going to be more successful. And it was just, it was this higher level of focus on what was going on on the outside world versus what we were doing. And I think just in general, the ability to handle unexpected situations, crises, things, things like that, really, I can feel my adrenaline spike up when something would happen. And I'd be like, Oh, it's a life or death crisis, I have to do something about it. Now I need to react, everyone needs to pause. And now it's just perspective, and you've been doing it a while, and you realize that happens every day. There is a crisis constantly. And it's just life, you know, I have a joke, I really like jack Sparrow, and Pirates of the Caribbean as well as Harry Potter. And, you know, it's like going up on deck for a while and you think you're seasick and nauseous and can't breathe. And then after a while, you kind of realize, wait, this is what breathing feels like now. And so it's just that reframing of, you know, this feeling of nausea, and like, you might die at any second, like, that's just what breathing feels like. And you just kind of get used to it and you move on. And so I think that level of emotional control, also sort of a muscle. And I think I've definitely evolved leaps and bounds on that regard.

That's a hard one man, that roller coaster and dealing with that effectively. That's a really hard one is you have been thoughtful and reflective maybe in your own growth. Have there been a set of resources or a set of people that you found to be particularly helpful in? Whether it's helping you make an individual decision or helping you just generally be in the right mindset, as you've kind of grown as a CEO?

Of course, I mean, you have people both internally and externally, who you I think rely on for different styles of problems. I have people Marian Dan, who's been with me since I founded the company, and she is the world's most calm, most thoughtful person. And having her around is incredibly valuable to me on a day to day basis. You know, I have incredible investors like David Wu, for man, Ron, who is a real partner, I think in a way that most founders just dream of having an investor partner like that, but he's a real partner and will be there to walk me through and ask the right questions versus tell me anything to do. And I respond very well to that. And so I think a lot of it is just about finding people who push you but push you in the right way where I like it. When people don't agree with me. I like that in my company. I like it out of my company, because it forces me to think more critically. But I like it when it's phrased, almost debate style, or like questions versus statements where I start to feel more attacked. And so I think finding the right group of people around you that are going to support you by questioning you is really valuable. Yeah.

Yeah, amazing. Having this coach slash challenges in your life can be Yeah, usually beneficial. Let's wrap up with a few lightning round questions here. This one, I guess we kind of talked about already, which is what commonly held belief about spatial computing Do you disagree with?

I'm gonna go with that multiplayer AR changes things? I don't think it does.

Got to make it work with what we've got. There's opportunity that's already here. Besides the one you're building, what tool or service Do you wish existed in the AR market?

The ability to take any real life object and create a 3d model or something like a digital twin? If that were easily and cheaply done? I would I think it would really open up potential.

What are the sorts of things you imagined become easier with that sort of capability? What the sort of in terms of experiences that are created?

I mean, I think it has implications in every single sector, whether that's entertainment ecommerce marketing, one of the biggest challenges is part of the scalability question around AR is having the assets to create these worlds and building art is crazy expensive, and especially If you're trying to have versions of things that exist in the real world to populate your digital world, which is frequently what people want, it's either identical or one degree off. It's very expensive to do that you have to either lean and really heavily, I think, to procedural content generation, which I think is, is one thing that you know, we think about or, or you need a way to really easily create these objects. And I don't see one today,

what book Have you read recently that you found to be deeply insightful or profound?

It's a high bar, the way you framed that question, I would have to say the ride of a lifetime with the caveat that I'm Disney obsessed, always have been wrote my business school thesis on the history of Disney, because I think it's an incredible company, and a really beautiful example like that. And Pixar as companies that are centered around being great at storytelling and great at the creative, but are fundamentally, really tech centric. They're all based off of really having better technology. And I think that that's always how I've kind of thought about Olympics. And so the ride of a lifetime, I thought was a wonderful book.

Yeah, that's, you mentioned that is so so fundamentally true, Disney Animation, Pixar, foundational, in terms of the technology breakthroughs that they have pioneered, and, and used leverage to create what is still to this day, the most compelling content experiences that that we have,

I think creativity and guy made everyone who joined our company up to the first 20 people or something, I had them read,

I read creativity, Inc. And I just fell in love with the way with his leadership style and the way that he encouraged collaborative creativity. And kind of going back to this concept, you're talking about this the right sort of challenging environment that brings out the best in people and in doing so in a way that's supportive that people recognize as part of the process. It's not personal. I just thought that was a an amazing book as well.

I agree. I think that was one of the most formative books I read prior to starting the company that really impacted how I thought about building a business. But in terms of recent books, I would still say right of a lifetime, which I thought was also just an incredible amount of management and leadership Insight.

I'm gonna go check that one out to know, thank you. If you could sit down and have coffee with your 20 year old self, what advice would you give to 20 year old Kiran.

So if I could talk to my 20 year old self, I would give them what I call in my head, the Gini test, one of my business school professors did this. And it really stuck with me, I'm not sure if it was just the day or the headspace, but I found it powerful, which was, you know, you hear the everyday question, kind of, if a genie could grant you anything in your life or your career, what would it be? And I always thought it was sort of silly of pick anything. It's unrealistic, what's the value. And I think I would have my 20 year old self truly answer that. And then I would kind of follow it up with what someone once said to me, which is, the reason the genie test is powerful is because it removes your fear of failure, it tells you what you would want if you were truly not afraid of failing. And then they went into kind of, you know, certain types of people. And when you're early in your career, you shouldn't have that fear of failure. And that's when you know, even if you jump in fall, you'll still learn more. And kind of following it up with this idea of, you know, you are the genie at the end, say you have every power to grant yourself what you want. And I think I would have never at a young age expected that my dreams were being held back by any kind of fear, or any kind of sense that I would fail. But they were if I think about what my 20 year old self was thinking, I was thinking, oh, maybe I'll get a nice job. You know, I'll apply for internships and be kind of part of the usual cycle. But if I had really let myself go, I would have never asked the genie for that. And I would have told myself to go for it completely. And that there was no sense being afraid of failure, and that I could myself, give myself everything I needed to take off.

That's a powerful one. That's really powerful. I have some version of that, that I asked myself periodically, just to check to see if I'm, if I'm on track. And I find that some things get crossed off the list. Which is, which is a really amazing feeling. Say, oh my gosh, you know what, 510 years ago, this was this was on the list. And now it's here. And you know, I get to come up with other amazing things to add to the list.

I always think that when I have down moments, and there certainly are some at having lifes and startups and stuff where I have to remind myself, you know, not to hold myself back for fear of failure. And then I actually have, I used to have something in my office up that had like, you are the Genie, you're the genie.

That's amazing. That's the right sort of information. You are the genie or the genie for your own life. Any closing thoughts you'd like to share?

I mean, I think in general, I think AR has the potential to change the way we view information and just change the way we interact with information fundamentally forever. I think the challenge and I always say this because I always feel like I almost come off a little bit like a non believer because so much of our out there I don't believe fundamentally has value to the consumer but I really do believe that if we focus on those pockets of value, those use cases, there'll be so much to be learned from how consumers interact with it and so much growth opportunity off the backs of it, that we'll be able to build a much stronger industry. And when headsets do eventually appear and they will appear I just don't have strong conviction on the next year versus two years versus five years. I think understanding what kinds of experiences were valuable today will only help us build the more valuable experiences for tomorrow.

Where can people go to learn more about you and your efforts there at Lumix?

Well, we're pretty active on social media, our website is unlimited calm. If you follow us on Twitter, at a Lumix official and the same on Instagram, we keep people pretty informed via those channels. And this isn't the same thing Personally, I micro blog on my Instagram at Kiran Sinha if you're interested in the day to day challenges of the entrepreneur.

Awesome. Kirin, thank you very much for this conversation.

Thank you.

Before we go, I want to tell you about the next episode. In it I speak with Japjit Tulsi. Japjit is the CTO at Matterport, a company building a set of technologies to accelerate and simplify the creation of immersive 3d Digital twins for real estate project planning, hospitality, insurance and more. Prior to Matterport Japjit bas been CTO or a senior engineering leader at carta, eBay, Google and Microsoft. In this conversation, we get into the technology at matterport. And how Japjit and his team are leveraging machine learning to deliver greater insights with less data essentially enabling you to teleport into a space. I think you've really enjoyed the conversation. Please follow or subscribe to the podcast so you don't miss this or other great episodes. Until next time.