The AR Show: Damir First (Matterless) on the Art of Storytelling and Shared AR Companions & Toys

10:27PM Jan 16, 2023

Speakers:

Jason McDowall

Damir First

Keywords:

ar

people

create

shared

experience

technology

play

humans

build

toy

game

called

device

story

digital

ideas

communal experience

feel

memes

hand

Welcome to the AR show where I dive deep into augmented reality with a focus on the technology, the use cases and the people behind them. I'm your host Jason McDowall. Today's conversation is with the Damir First, who is the co founder and CEO of Matterrless, a company creating genre defining experiences in shared augmented reality, emphasis on shared early in his career, Damir earned a Master's of Arts in both sociology and art history before pursuing his PhD in anthropology. He has been a lecturer for a number of courses spanning design history, architecture and art. And he has since worked in blockchain technologies, as well as a Narrative Designer for popular video games. While a tech entrepreneur and academic Damir is a d&d dungeon master at heart, who thinks deeply about the role of technology within our human experience. In this conversation, Damir shares his perspective and how AR fits within the spectrum of key enabling technologies.

We believe that AR is the next step in human communication, where we will be able to conjure digital objects interactive objects into the material world around us into the space around us, we will be able to enhance if we will be able to color it to paint it with emotional context through creating interactive madalas objects. And to do that it needs to be a shared experience. So the transformative narrative that we need to bring to AR becoming a mainstream thing is that it needs to be a shared experience for multiple participants can see the same thing in the same place, precisely positioned and be able to use the most natural of all controllers, their own hands, to wield it, to interact with it.

Damir goes on to share what makes their technology unique, how the approach addresses privacy, the types of experiences they're creating, and why and the power of transformative narrative. As reminder, you can find the show notes for this and other episodes at our website, thearshow.com. And please support the podcast@patreon.com slash the AR show. Let's dive in.

Damir, you've done something I admire and aspire to, which is to learn how to tell a great story. And to do so on stage. What was a memorable experience you had there?

Thank you for that. First of all, a couple of years ago, I started doing storytelling as a hobby, which means that I started telling stories on stage. And it was a very long process of learning how to do that, where we were figuring out how to use our memory to figure out the skeleton for what the story is supposed to be like. So we can remember it, using our voice using gestures, using our body language in general to be more enticing on stage and to draw people in. And it was quite a long process. It lasted for a year. And I remember around a month before my final diploma, which was a big show that I was supposed to be in go on stage and tell the story. One month before doing it, I decided to change my story because it didn't feel like the story that I was working on for almost a year was appropriate. So I ended up doing another story. And then quickly, I did all the steps to turn it into a palatable piece of showmanship for five minutes, five to 10 minutes. And I ended up rehearsing it every day, multiple times. And I just couldn't get it right. And every time I would do it, I would mess up some detail. And I remember I was on stage in the backstage and I was pacing around and I was with my notebook trying to remember this thing, which I haven't told properly once sweating thinking about how am I going to do this. And then I went on stage. And a couple of seconds into it. I was in it. And I told the story. And it went pretty well. I even ended up making some people in the audience cry. And they came to me later and they said, How did how did you do this? How did you make us cry? And I said well, because it's something that's very meaningful to me. It's important. And I realized that in that moment, I connected with the people who were listening, I wasn't only performing for them, I was telling them something that I was very passionate about. And it turned out that there was something there that it pushed the buttons of the people in the audience and they felt it and we had this experience a communal experience, which was shared. And it just made me incredibly hooked on doing that in the future. And also it surprised me very much because I've heard it. I've heard people talk about it, but I never experienced it myself.

That's incredible. There's a lot to kind of dig into there a little bit as well. What was the original motivation for you to even pursue a course like this? What was Is it about storytelling? What is it about storytelling that draws you so draws you in? You know,

I grew up playing Dungeons and Dragons. And I've been playing d&d Since I was 13 years old. And I very quickly became a dungeon master. For those of you who don't know, d&d is a tabletop game where you play a bunch of every person plays a different character, and they roleplay it, and you can play it from week to week, and the game really doesn't end. And it's a mixture of communal storytelling, problem solving, and a tabletop game with Ben It's very movement is around. It's pretty insane. I think everybody knows what d&d is. But you know, just in case, I wanted to give a little definition, the person who kind of makes the game who sets the tone, who's kind of like the director is the Dungeon Master and the Dungeon Master is the person who sets up the story, he makes up all the characters that you meet along the way. And he basically play acts hero plays as all these characters, you meet all the monsters there, you're supposed to fight all the puzzles that you're supposed to figure out. And then on the fly, change the story based on the decisions that the characters make. Because no matter how much you prepare, for some situation, let's say a couple of people want to enter a castle. And the warrior wants to fight the guards, the thief wants to steal the key, the wizard wants to cast the spell to make an illusion. And you as the Dungeon Master, you don't know what they're going to pick, right, because every person has their own ideas. So you have to very quickly on the fly, be able to improvise and turn the story in the direction that the players want to go. I mean, you don't have to. But if you're a good Dungeon Master, this is all you think about.

So this has been in your system for a long time, you loved the end as a game that you really enjoy the dungeon master role in particular and all of the storytelling. And it sounds like it was one in which there's a lot of improv so to speak, there's a lot of preparation you could do. But there's a lot of reaction you had to do, based on what the players were choosing,

there's no way to prepare for every situation. So unless you want to kind of railroad the characters to only one thing that you have in mind. And that's not fun, really, for anyone, you kind of have to go with the flow and on the fly, just figure out how to make the story twists and turns the way the characters want to, but then still make them come to the right. Checkpoint, you know. So it's really a skill of sight, I would say it requires psychology, it requires acting abilities that requires creativity and problem solving. It's a, it's a phenomenal, it's a phenomenal thing. It's like a generator for creativity. And I would say that the secret handshake of people in the entertainment industry today is that probably 70% of them played d&d At some point in their lives.

Wow, that's a big percentage, but then you can talk about it as a proving ground and a nurturing environment in which to build a lot of those necessary skills. And you'd also talked about in that story that you really connected with the audience. And part of it was because it wasn't just a story, it was personal to you. But are there are there characteristics or attributes of a really great story that is really effective in communicating and connecting with others? Well,

first of all, if we go a little bit into the science of storytelling, we get to the point where our brain fires up in a certain way when we listen to a story. So when we see or hear or play a story, the neurons in our brain will fire in the same pattern as the speakers. So they it's a process called neural coupling, or some people call it mirroring as well. And according to some research done in the last 15 years, these processes, they occur across many different areas of the brain, and they induce this shared contextual model of the situation. So what that means is that their motor and our sensory cortices, they fire up in the speaker and the listener. So when we are engaged in a story, we're kind of going through the story ourselves, and also anticipating the climax. There is there's secretion of dopamine in our brains really looking for that resolution, you know, and of course, I think anybody who ever watched the TV show, where at the end of the episodes, characters find themselves dangling off from a cliff, and then the story ends, right? It's the literal, literal cliffhanger. And then they're like, Okay, I have to like, let's binge another one, right. So there is something there about the experience of listening to a good story, which is heavy in detail, which is emotive which is relatable, which helps us stand to imagine ourselves in that situation. So, you know, if you ever had a dream that you were very, very taken aback by, you probably still remember it. It's a completely fabricated experience, but you remember it in detail, and you could probably tell it to someone in a way which would make them I really feel at least a part of that feeling that you had.

Yeah, you kind of talk about this power of story to connect to mirrored for us to emote. We can talk a little bit about this idea of a, of a transformative narrative, as kind of a deep storyteller who's, who's been through so many different types of experiences wasted storytelling, and bringing it back to this kind of notion around matter lesson in what you're ultimately creating there, what any entrepreneur is creating and trying to tell a story around their companies? How do you describe like a transformative narrative, something that makes sense, even from a company's perspective, or entrepreneurs perspective?

This is a very, very good question. It's something that we've been really thinking about since the very beginning of madalas. So madalas is a startup, which is creating digital toys and companions in shared augmented reality. And the reason why we wanted to create a shared augmented reality experience is because we believe that currently augmented reality doesn't really exist, it exists as a filter on your phone, maybe you put some bunny ears on your face, or show a friend, some cute animal on screen, but the moment where you give your phone away your device away to somebody else, the spell is broken. And we realize that what that means is, AR doesn't really exist, because AR is this kind of almost as a hallucination that happens on your own device in the moment that you change perspective, and you're left with your hand hovering in the air, trying to touch something that doesn't exist, because you don't really see where the thing in AR is anymore. So we realized that if we really wanted to bring AR to the mainstream, what we need to do is we need to create the transformative narrative for it because it truly we believe, is a transformative technology, we believe that AR is the next step in human communication, where we will be able to conjure digital objects, interactive objects into the material world around us into the space around us, we will be able to enhance it, we will be able to color it to paint it with emotional context through creating interactive madalas objects. And to do that it needs to be a shared experience. So the transformative narrative that we need to bring to AR becoming a mainstream thing is that it needs to be a shared experience for multiple participants can see the same thing in the same place, precisely positioned and be able to use the most natural of all controllers, their own hands, to wield it, to interact with it. This is what we call the transformative narrative. You know, once it's set up in this way, you can really kind of start to see what currently isn't possible to do in AR, which is to have a communal experience, very similar to an experience of telling story together where we can all feel the same thing because we can imagine the same thing.

It's central, I guess, to what you're describing here is that it is shared. The fact that we can you and I both have the same sort of experience makes it more real, because I can talk to you about it in the moment after the moment. We were not individually hallucinating, I guess here we're jointly sharing a story or jointly imagining.

Yeah, and in a way this, this brings us to another scientific definition, something that's known to psychologists and sociologists something called intersubjectivity. And what's intersubjectivity? Is, is the human need to have our version of reality be validated by other people? What do I mean by that? If you ever went to a comedy show, where somebody told the joke, and everybody laughed, and you didn't laugh, because you didn't get it? Well, they had intersubjectivity, and you didn't, like you felt left out, even though you were physically in the same space, you weren't really privy to what was going on. So you weren't mentally in the same room as other people, you have lost intersubjectivity with them. So if you have like inside jokes with friends of yours, and then you bring your new girlfriend to hang out with you, when you say those jokes, you will laugh, but she won't get it. Like she will have lost intersubjectivity. So in a way, what we think is that intersubjectivity is this core human need. That has been around forever. We all want to feel connected, we all want to feel included, but we are constantly finding ourselves in situations where intersubjectivity can be lost. And then we feel like a little pang of discomfort. You know, we feel like we're left out. We're not part of the tribe. We're not part of the people sitting around the campfire, metaphorically, and because of this, like the individual feels like maybe they're not a part of a group. And one of the main reasons why we think this is important for We're designing AR experiences is because there's certainly no intersubjectivity. If two or more people can see the same madalas interactive AR object in the same space, the moment where you have to give away, the phone or the device that you have to see an AR content is where you have a record skipping moment, like you realize it's not real. And we believe that digital things are real, we want to prove to the world Metalist wants to prove to the world that digital things are real, by creating shared experiences. What do I mean by that, I mean, to make something real, it needs to be experienced together. Again, if we go back to storytelling, why can give you another example, which, which probably everybody can relate to. And that is, if you've ever played the game, or read a book, or watch the show, where there was a character that you really love, and something happened to that character, and you had that moment of empathy. And maybe you even wanted to model your behavior on a character whose values you feel are aligned with yours. Isn't that real? Isn't that a real moment in defining your character, but it's made up? You know, it's, it's a story. So, so having that sense of something being real, it doesn't have to refer to material forms, it doesn't have to refer to the material world only. In fact, in sociology, we call it the Thomas theorem. So the Thomas theorem is around 100 years old, and a lot of people know it under the name of the self fulfilling prophecy, meaning that if you define something is real, if you if you find something to influence your behavior, you can define it as real in its consequences. So actually, you believing something going to happen, will influence it happening in that way. So in this way, in our minds are incredibly powerful, we do really create the reality that we live in. And if we create an experience which is communal, then of course, it affects more humans, you know, more, more of the people who are witnessing something, can feel the same way about it.

And in doing so, by this by this notion of not being left out, right, not being left out of the group of feeling connected. How does that ultimately affect? Not maybe necessarily our experience with these, you kind of describe them as digital companions or toys. But how does that affect our relationships with our fellow humans that are in that shared experience? Or who haven't yet maybe had that experience?

Well, I mean, I think it's crucial for us to be able to share experiences with other humans because as an anthropologist, I, I have studied for a long time, the way people behave in groups and individually psychologically, how they feel affected by being in communal experiences. And one of the things that I find is incredible about humanity. And probably one of the reasons why we ended up being so high on the food chain, currently in the world, is our ability to cooperate. So the human superpower is the ability to cooperate, everything about our biology, and physiology is geared towards us wanting to be a part of a group. So for instance, the whites of our eyes are a very interesting indicator on why humans want to work together. Because technically, we are predators, we have eyes in the front of our heads. And herbivores usually have eyes on the sides of their heads so they can have a wider field of view. So predators when they hunt, they want to have their prey in the center of their vision. And most predators, the corneas, the eyes are of the same color. So their irises and their retinas, and their corneas are usually of the same or similar color, except for cats, but that's for reflecting white. And the reason why that is because predators want to hide their gaze. They want to hide where they're looking at, because it's not evolutionary, advantageous. I mean, when I say predators, what I mean evolution, right? So evolutionary, it's more advantageous to mask where you're looking at so you can have a higher chance of getting the prey getting you know, hunting. But humans what we have is an evolutionary, you know, exception because the reason why we have some anthropologists in evolutionary biologists claim have white, white white parts of our eyes so that we can actually show where our gaze is directed at. So by me, looking at something I am letting you know where I'm looking at and it helps us cooperate better. There's another very interesting evolutionary kind of hack that humans have. And that is an involuntary blush response. So when we feel uncomfortable, we blush. And we can't really affect that, like, you know, unless you're a psychopath, maybe you can influence it, but like otherwise, most humans cannot, right. And the reason why that is, is because it's evolutionarily advantageous for us to show discomfort so that we can fire up empathy in other humans so that they know we feel uncomfortable, so they can sue us. So to me, these are two very interesting biological factors, which show that, you know, we're geared towards working together, and we feel the most comfortable working together. In fact, I would say that the original state of being is some sort of community and you can see it now you know, you can see it in the meta, you can hear it everywhere. Everybody talks about communities, online, on web three, community building, this is a buzzword, but I do believe this buzzword has a very deep meaning for us. This is why it's so often repeated, and that is, we feel comfortable being together,

we feel comfortable being together, as you said about kind of implementing these insights into Matterless. And these digital companions, these digital toys. How do you describe the goal of the company? Kind of with that as the focus? How do you describe the goal of Matterrless

we want to prove to the world that digital things can be real by creating shared experiences in AR without the ability to show other people what you're seeing. Virtual Worlds quickly become boring, unsettling and shallow. This revolutionary ability to create shared moments in AR makes augmented reality feel real for the first time. So we could say that it's seeing is believing

why companions and in toys and not games.

This is a very good question. So this is also something we spoke about a lot. And the way we started in 2019, is we built a ruler for games workshops. A tabletop game Warhammer 40k. Are you familiar with Warhammer? Alright, so So Warhammer is this, this game that's made by a UK company, which allows you to create your own armies. So you can buy minis put them together, paint them by yourself, and then build armies and then fight other people who also have armies of their own. So it's an incredibly time demanding hobby, but people are super into it, like a lot of people are really into it. And they spent hours and hours, 10s of hours, hundreds of hours even painting the most beautiful army, making it custom making it unique. And then playing with other people. What we wanted to do is we wanted to create a ruler for this because the way that you move units on a table is by having an actual ruler and then checking, you know, if it's like how far they can move, and like all the time and measuring with this ruler, physical ruler, and we thought, wouldn't it be cool to have an A our ruler, which can recognize does bass recognition for each unit, and then shows you an aura in 3d around it, how far it can move, how far it can shoot, etc. And we tried to solve that problem. And we were we managed to solve that problem. But then we wanted to be able to share it across multiple devices, we ended up realizing this is really difficult to do in AR. It's very hard to do it, but we have cracked it. And once we cracked that we realized that rightful ruler D app for AR measurement, and Warhammer has such a much wider application than just being a tool for a single game. It actually creates a sense of intersubjectivity for any kind of shared experience in augmented reality. And then we thought about what would be the coolest demo we could make that could make people understand how fun this is. And very quickly we realized animals, if we make little digital animals that sit on a table, wag their tail, a cute little Sheba that looks at you. It's tongue wagging out of its mouth hanging out there, it's mouse, and you go and touch it and pet it and it responds to your hand. This immediately intuitively gives people a different type of experience. So using your hand to touch madalas pet was something we realized is going to be so much more intuitively understandable. If we make an animal because people want to play with puppies, and we ended up realizing that this this is a killer Use Case. Of course, this is a very complex task, you know, making a digital companion is quite quite a thing. And we've been working on it very, very hard very, very long. So while we were doing that we thought about some other ways in which we can engage with our users. And we realized one other thing we could try making is a digital toy. So what if we make a toy in AR? And to the question that you asked me earlier, what's the difference between a game and a toy? Well, a game has a set of rules. If we play chess, we know what the wind conditions are. And we're always geared towards finding the most optimum strategy to win. So games are bound by rules, if they have rule sets, toys, on the other hand, don't. So what happens with toys is usually during gameplay, Toy Play, actually, during toy play, you will figure out that you maybe want to change the rules. So you know, anybody who's ever played with a stick will attest to this, right? You know, one moment your stick is a sword. The next is, is a wizard stabbed the next, it's a telescope after that, maybe it's an or did you use the row your boat in, and you kind of switch modes, you switch conditions for how you play with the toy very naturally. And we actually wanted to simulate that in the AR environment. So we created a little hover car in AR that you can raise on any surface on a table on the floor, we call its floor craft. And then we created a game go mode called floor craft arena, where you can race and smash and crash these cars into each other. Again, by creating a shared AR experience, which is something as simple as scanning a QR code on one device, you can immediately jump into an AR session. And you can play with your friends, each one of you having a toy car that's running around on a table, you can put obstacles, landmines, all different kinds of things, kind of you can imagine a little bit like Mario Kart experience, where multiple people can kind of change the conditions of how the game is played, how to play with this particular toy, like on the fly, and then those that are better at it who win more who have more crashed, like we have a crash counter, and those that have a higher score, they can actually change the game modes on the fly. So you can turn it into like a king of the hill type of game, or deathmatch or capture the flag. And actually, it becomes a sort of an infinite game. And I believe that this infinite type of gameplay is something that we all fondly remember, from when we were children, where we could spend, you know, hours playing with a cardboard box or, or a stick, or a ball, or literally anything.

So this this notion that it's not a game, but a toy, makes it more broadly useful and entertaining, creates more opportunities for individual and for shared play. One of the things that you kind of talk through there, as you kind of hinted at, how is it that you initiate a shared experience we've seen, there are other AR apps who attempt to create these sort of shared experiences. And remember, some demos from Pokemon Go, for example, where these two players they walk in, and they kind of scan the same space. And after scanning the same space, which takes some time to kind of have enough of a shared understanding of where those devices are, that you can begin to see the same sort of thing together. So what is it that you're doing so distinctly differently in your technology? So yeah,

one of the things that one of the things that's currently necessary for you to have an AR experience, which can be shared is something called the digital twin. So you need to have a pre scanned 3d version of the environment that you can then use to compare your vision from your device by looking at that digital twin. And then asking the other device where it is, according to this version of the digital twin, we were able to create a ad hoc peer to peer session which does not require you to pre scan the environment by using slam, which is which already exists on on arkit and Arcore. And what we do is we use a QR code to scan into a relative coordinate system, where what we are the only thing we need to know is where the devices are relative to each other. And then we can place 3d assets into that environment based on where these devices are relative to each other.

And so that the process is you scan the code from one device to the next right so once using the camera, he has something on the screen. And and based on that in the Slam Data, the sensor data in the phones, you can kind of figure out where they are relative to each other.

Yes. So what happens is once you scan the QR code, you can have a pose updates pose being In XYZ and the rotation of the device with the camera, right, so you can have a post update 60 times per second on multiple devices. So they all know where they are. And we run this through a local messaging service, which we also invented called huggle. Hoggle means hail in Old Norse. And it's a way for us to distribute our AR sessions locally, so that we can have lower latency and ping times amazing. This was developed by our sister company Aukey labs, on which technology on whose technology we're building the madalas experience,

that's awesome, then you've already you've already described some couple of very rich ideas. One of them is the companion, the digital pet, which you noted to kind of make it a great experience as a huge, huge amount of effort to incorporate to imbue that thing with a sort of interactivity that we would associate with a companion. And then also the set of toys cars, in this case, the ones you've described, describe maybe the process that you go through, from a game testing, toy testing, pet testing, sort of process, how you incorporating users, how you incorporate their feedback, as you continue to make this thing great.

Well, you know, here, here's another kind of highbrow thought that I want to throw into the mix. It's about the centrality of play. Right. So Johan, who is Inga who was an anthropologist and historian. About 100 years ago, he made this book, he wrote this book called Homo Ludens, like the man that plays, and basically he said that civilization comes from the surplus of play. And what does he mean by a surplus of play? Well, he talks about playful pursuit of a craft. So when you have mastery of a craft, you start to improvise, you start to jam. And actually, this playful pursuit of a craft allows you to perfect your skills further than what is just necessary. So mastery, in a way is the consequence of play. And play is so central to the way that we try to develop ourselves that, you know, oftentimes, it's very different from work, work has, or labor, right labor has this instrumental value of, you know, you, you work so you can get paid. So you can do the things you like, you know, unfortunately, this is how most people in general live their lives. But played selfish has some sort of intrinsic value. It's, it's a, it's a, it's a reward in itself. So when we think about creating a shared playful experience, this is what we think about, we think about what is this inherent intuitive, and internal way for you to enjoy this experience from moment to moment. And you know, a lot of Game Dev is exactly about that. It's about finding the fun. Finding fun is something that's quite difficult because you have different types of games, and you don't really know what's fun, until you try it out. So this is why this is why it requires a lot of testing. The way we test our experiences is by people who are on our Discord. And I would love for people who are here who are listening to this to come to come to our Discord and check out what we're doing. And you can find this very, very easily. Our Discord is discord.gg/madalas. If you come there, you will, you will see that we try to engage with everybody who is in our community, and take their feedback into account when when we're designing our experiences. So the way it works is, we create an idea we try to test it, we run it through some some gameplay types, and then we then we test it with our community, and then we take their feedback into account and then try to try to improve it improve on it.

So you've got a very active community who is playing in providing that feedback. Last year,

we had a couple of 100 people who were who were playing our private beta. And we wanted to we wanted to see what they think. And yeah, we got some very good feedback from them. And of course, you know, like, we're trying to build a version of the game that we ourselves find valuable. But of course, you're also developing it for people who want to play it in different types of environments. So of course, this feedback is crucial for us to build a product that we and our backers in our investors can be proud of.

Speaking of investors, who are what is the right type of investor for a company like this, because it's not that quite a mobile game company. AR is definitely critical. You're relying on another company's technology in this case of sister companies technology, what is the profile of the right sort of investor for you?

The type of backer or investor that would be interesting that was interesting to us, was somebody who really Grox what AR might be and And this is why when we spoke to our investors, to our backers, to the people who gave us their trust and their money, we talked a lot about what AR needs to be. And we believe that currently AR is not really that revolutionary, but it could be. And the people that backed us are the people who understand that AR in general, is a field that's only starting to get developed. And that by utilizing the experience of multiple people seeing the same thing, experiencing the same thing in the same place, is something that's really going to change the bandwidth of human communication. And when I say that, what I mean is, imagine a real time experience, which where you can conjure up digital, interactive objects, madalas objects in a conversation with the flick of the wrist, and then have that be seen by multiple people in the same time. This is, this is almost like telepathy, I would say it's the penultimate step in human communication, one step before telepathy, we're talking brain machine interfaces here. So if you have type of experience, where you can color the emotional, you can paint the space with emotional context, the same way you paint space with emotional context. Now with music, where if you listen to a different type of music, it really changes the mood in the room, you know, depending depending on if you're listening to metal or some slow jazz, it really changes how you feel about where you are. Imagine when that happens visually, with objects in real time. So the investors, the backers that we spoke to the VCs that we spoke to are the ones who understand that this is the next step in how we communicate as a species. And, of course, we're not here only talking about doing this on a phone, even though we can actually create these experiences now on devices, which have a single camera, which have monocular vision, meaning that we can simulate depth and have our hand in space is an object without having stereoscopic vision, which is what most VR headsets are using right now. So we are able to create that on a monocular RGB camera. And in the future, of course, we are counting on the mainstream adoption of AR glasses and mixed reality glasses. So in a way, I would say that the people who are interested in joining this quest with us are the people who kind of see this as the next step in how technology will evolve,

given where you are given where the hardware is today, which is primarily a mobile first experience. But we're very excited about what's coming with these headworn devices. Would you think about where we are today with the hardware? What are the biggest hurdles that remain for you in making these companions, these toys generally available?

Well, I mean, you know, there's there's a lot of, you know, immersive technologies still very early. And I would say that the biggest issue right now is the form factor of the devices that you use to access these enhanced experiences of reality. And the other one is privacy concerns. So issues with privacy, are quite a massive chunk of why we also wanted to build what we're building. Because when we started out, we realized if you wanted to have a shared experience in augmented reality, by some of these massive companies that are building digital twins, like Google and Amazon, and meta, all they need for you to give you this AR experiences deep inside of your bedroom, for instance. So they need to know where things are in your bedroom so that they can display something in a digital twin of your bedroom. And we thought about this, and we realized that something that we don't feel very comfortable, it's very black mirror risk for them to have all this data about you. So this is one of the main reasons why we wanted to develop a technology which does not require for us as a company to have any of the data to be able to give you an augmented reality experience. So we ended up creating a decentralized distributed messaging pastor service, which allows us to create local, little local servers for for AR, which we don't own, which you can own yourself on a device that you own by herself. And in fact, the way it works is you need an internet connection and only four megabytes of RAM to be able to run a local Hoggle note. So you can run it on your on your router or you know you can you can install it on your router. If you're a bit tech savvy er you can get like a Raspberry Pi and just install, install huggle node on top of it and you will have a local way to distribute the data that you own yourself into nobody has access to not even us,

beyond simply the person owning the server, having more control of the data or elements to the data itself, that's dissimilar from the kind of other model you describe where you scan the room, I still do the full scan, I guess, yeah,

you know, the slam algorithm is still gonna do the scan. And the way and the way the slam works is like when you turn on slam, what happens is your camera using computer vision, and it looks at feature points around the room. So you know where the floors are, where the tables are, where the walls are, and then reasons about its relative distance from there, and then updates that in a pose, usually around 60 times per second, this is the way for a single device to move around to be able to reason about the world around it. But then what we do is we are able to create a way for multiple devices to talk to each other and ask each other a question, you know, it's T one, you know, during 1/60 of a second, where were you. And then we showed that off to this local server. And then the other device says, I was here, where were you at the time. And then by doing that, we have both of these devices or multiple devices, communicate, talk to each other about where they were in that moment. And that's how we can update in real time. Obviously, for that, we need a very, very low ping time, we need around eight milliseconds for this would be the ideal ping time. Otherwise, if you imagine me holding a digital cup in my hand, and now I move my hand from left to right, the cup is going to be delayed, it's going to lag, it's going to trail behind my hand as they move it so so we kind of tried to figure out a way to solve two problems at once. One problem was the problem of preserving privacy, and the other one was low latency. And we kind of did both in a way by by a single solution.

As you look out over the next year, year and a half or so. What are who concerns are the most?

Well, I mean, you know, one of the companies obviously the you know, the small companies that you perhaps may have heard of Google, Google, Apple, Amazon mitta. Also, one of the companies that we're quite concerned with is Niantic. Because even though Niantic is doing a AR experience, they are harvesting tons of data. And this is something that we feel very uncomfortable about because data harvesting that they do is is quite something. In fact, in terms of invasiveness, they collect more than animal jam, or Roblox or Animal Crossing. So they're

constantly sucking up data about the environment because they're building a full Central 3d model of the world effectively, is that what's happening? Or is

well, yeah, but they also collect, you know, contacting environment and health and fitness and financial info location, like, different different types of diagnostics and usage data. So yeah, it's quite a lot. And of course, in a way, all of these apps, what they ask you to do is wear a camera on your face. So if you are using their apps, they will see what you see. And that's that's quite concerning. This is something that we really do not want as a part of the entire business model that we have the data collection thing, like we really think it's like it's, it's possible, and, in a way even desirable to build a business without that because data is one of these words, which today just sounds like a bad word. You know, data Bad's, it's always data privacy breaches are data harvesting, that all these massive companies are dealing with. In fact, I would say that even telcos telecoms, they're the only ones who still think data is a good word, because that's what they're selling, right? They're selling buckets. And these buckets of data are something that they they're the only ones who see as a positive thing. Everybody else looks at it as information that's being taken away from you for a service that you kind of accept as a as a counterpart to that.

Yeah. And that's when I highlight one one element of the distinction of how you're thinking about data and privacy differently, beyond this idea that it has to be low latency that it's really just about sharing the most recent update of where you are, and where am I? Where are you? Where where are you? And also the the distributed kind of server nature of this this mesh network sort of approach to it. It's also you're not building and retaining a full 3d model of any environment, because you don't need the full model to be able to answer the question, Where were you a moment ago, relative to me?

Exactly, you know, in a way it makes sense for you to try to build a system where or you're observing the world around you, and then trying to reason about your place in it, because this is how humans operate, right? Like with our eyes, we look at where we are, so we don't bump into stuff. So we constantly need to be aware of our surroundings. So it's not surprising to build a model of a world where you're trying to project digital content on top of the world. But it's actually not the only way to build that. For instance, you know, if you wanted to replace the GPS, you could do it be with with AR, you could do it through this collaborative spatial computing, which does collaborative slam, you could have the devices use their cameras to reason about their place in the world, and the distances between them. And in fact, if you if you really want to go far out there, like imagine landing on Mars, and figuring out where you where you are on Mars, how are you going to do that? If you don't have geostationary satellites, like you do on on earth? Like, how are these devices going to reason about their place in the world? Well, actually, they could do it with AR Because with AR, you know, you could even do it indoors, actually, some of the experiments that we're doing is about creating experiences indoors, where you don't have to have line of sight, like in GPS, to know your precise location and position,

as you project out five years from today, or so, who is Matterless,

the biggest digital touring company in the world, because digital to accompany the world. What we want to project into the mainstream is the adjective madalas. So something that people will use as a distinction. So oh, I, you know, I played with my mannerless toy, as opposed to my material toy, or, you know, I was in the office. And we did like we had a matter this brainstorming session without a whiteboard. You know, this is there's so many applications that you can use, AR for education, entertainment, any kind of experience that you can think of which can become a communal experience. So what we want to do is we want to build building seamless, fun experiences where people can feel more embodied, about being in the same place at the same time, we can provide an experience in AR, which is a local one, which happens on the table, when you play with your friends, when you guys raised a bunch of cars, or you pet your cute little madalas companions, or let them play with each other, they're on the table there next to you, you're involved in this experience, you can touch them by hand, and you're using the technology in your hand to be more present and to be more embodied. And this is kind of the dream. We want to be able to create these kinds of digital experiences, play experiences, which allow us to feel like we're here and now. And I don't think that technology right now, the mainstream gaming industry and technology right now is geared towards that. This is what we want to bring into the world. This idea that you can play with your friends on your phone and feel like you're right there. The same way we played this kids, when we didn't have phones in our hands. You know, when we were kids, we wanted to be firefighters and policemen because these were the toys we played with. Chances are that your kid wants to be an influencer, because their toy is the mobile phone in their hand. And all they do is stream and record which which is cool and creative. But chances are they have changed the thing that they think play is, and we believe that there's space for doing that, again, in a different way like reinventing what the communal experience is by creating digital toys that you can play with. Right here and right now.

Fantastic. Extremely well articulated such a beautiful vision. We talked about this idea of transformative narrative. And I wonder if you are able to share some advice with the other entrepreneurs who are listening, and how to how to go about constructing the sort of transformative narrative for their own startups? What are the essential components that any sort of entrepreneur should be thinking deeply about, and be ready to articulate in connecting what it is they're doing to their key audiences?

So one of the things that we are very vocal about and transparent about is the fact that what we did to build our startup was to embrace mimetics. And mimetics is this very interesting branch of we could call it communication theory or evolutionary theory, which talks about memes and I'm not talking here about funny cat photos, that you're circulating online or gifts. What I'm talking about is the principle of using ideas that can self replicate. So, in 1976, Richard Dawkins, the biologist, in his book, The Selfish Gene, talks about something called the Neo Darwinian synthesis. And, you know, this is a lot of big words to explain a concept as a single simple concept. And that is that the evolution doesn't happen on the level of a group, a species, a group, even individual units, but actually, it happens at the level of the gene, the gene wants to eat up all the resources around it, and use up as much as material space as it can. And then he pauses that there's something similar happening in the cultural sphere, or the idea space, or the consciousness space. And that is memes. Memes are basically ideas that can be copied. So a greeting a word, a way to, you know, tie your hair, or how you wear your scarf, or what color is in the season. All of these are memes. All of these are ideas that can be replicated. And the way that mimetics works is it looks at these ideas as wanting to eat up as much of mindspace of consciousness as possible. So genes propagate in physical reality, memes propagate in the consciousness of beings. And when we're able to create an idea that has mimetic fitness, we are able to make something viral. So we have thought thought a lot about how we can craft our own on on memes. And the reason why this is important. This, you know, on another deeper level, madalas is not an AR startup. Matterless is a cultural project. And it's a project which helps humans inhabit a consciousness that digital things are real, because they can accept that memes are real, and that their ideas are real. So we can become the authors of our ideas. And we can become something that we call internally, cultural transhumanists transhumanism, which is usually connected a lot to the community of biohackers, people who are trying out different nootropics, or even actually trying to augment their bodies with certain types of sensors, or chemicals, we're not doing that we're doing cultural, transhumanism, we're like, the reason why transhumanist wants to do this is because they want to break out of the limits of our current physical bodies and our current age and like, how well we can, in a way, fight disease or aging. Cultural, Transhumanism is a way for us to try to author our own culture, our ideas, we can influence on the ideas that we make. And if we embrace this idea that we can build the ideas our culture is based on, then we can actually start creating something that is infused with our values with our idea of what kind of culture we should live in. And, to me, that's very empowering. If we can build ideas, which are long lasting, and which are fit enough so they can propagate, if we can create if we can shoot out memes that are good. That value, the ability for us to teach other people how to teach that value, our curiosity that value, the fact that we can we can build technology in a way where we can preserve privacy, or we can bring back the sense of togetherness and associate these devices that we have with the sense of being more present. I think that's, that's, that's a phenomenal, that's a phenomenal thing to strive for. And the reason why we talk a lot about medics is because it's completely open source. Actually, anyone can do this. The reason why other startups can use the same principle is because it's a it's a transparent principle. So we don't actually hide the fact that we're trying to build memes that can go out there and replicate by themselves. And then people talk about them. In fact, you know, I could even say that this is an exercise in me trying to convince you to talk about our story to your friends, because you think what we do is cool. You know, so if we if if I have done my job correctly, I have tried to engineer behavior where you will feel gratified and awarded by talking about the story to other people. And this is something that we've been thinking about for a very, very long time. And actually, our CEO, Niels has been a behavioral engineer for more than 15 years and he's been working on this mimetic theory, as it is a type of building our organization around so that everything that we do is infused with our values, with our sense of curiosity, and with this incredibly important feat of teaching other peoples how to how to teach. So it is my heartfelt desire that some of the things that I talked about today will make people feel interested enough to go digging about what mimetics are, they try to use it by themselves. So, yeah, this is this is basically our marketing. This is how we create the marketing for metals. We talked a lot about how, you know, we're building pets and companions. Why is that? Well, it's because we realize that if you make a digital puppy, everyone's going to smile, when they see it, they're going to try to touch it with their hand. So there are some intuitive modes of behavior there, there's some emetic scaffolding, we could say, that people will immediately get when we create an AR experience in the right way. So we're always looking for ways to kind of build on top of the things that people already feel comfortable and know about. This is also one of the ways in which we can avoid this black mirror type of future. Because what we want to do is we want to build technology, which is cohesive to connectedness. And that's based on mimetics. And mimetics, are ideas that can go viral. So we're trying to build those ideas

centered to those those the things you just described, there are the right set of values that resonate, right, that you feel are, are good, in that you want to share in that in that goodness, you describe this, this idea of Black Mirror and some of those those concepts are sometimes underhanded, right, this notion of data collection or harvesting, or, or learning and abusing some bit of information to the detriment of maybe the person or the others around you. What you describe is really critical. And and having others want to share this is fundamentally the principles feel good. It feels right.

I believe so. And one of the reasons why we why we want to do this is because we want to, we want to talk to the people who will be interested in what we do, because they're thoughtful, they're not easily impressed. It's the people who understand how big the problem of shear they are, is. And and they know that solving it is a massive challenge. And it requires more than technical know how it requires behavioral engineering, which focuses on a transformative narrative. One of the things that you you asked me before our call is about my background. And actually I'm not a tech person. I'm a sociologist, art historian and an anthropologist. So the way that I think about what technology should be, is very different from some of the engineers that we work with. And you know, by no means I am I knocking engineers, I, what they do is magic to me, like I don't know how to do that. Right? But like, the reason why it's very interesting for us to try to build that is because we can look at technology, not from the point of view, can we build this? But should we build this? And how should we build this? As Frederick Paul, a science fiction writer says, a good science fiction story is not about the automobile, but about the traffic jam. And in this way, I could say that sci fi, in general, is the sociology of the future. What you do is you take the society society as is, and then you place the technology, which changes everything about how people behave, and then you run it as a thought experiment, you know, how is this going to affect humans in the future? Like, what if we had teleportation? How would that work? would then we start commuting, you know, from New York to New Delhi in five minutes and work there? or would there be some sort of limitation on how far you could teleport? Probably yes. Right? Who would? Who would implement that? How would that influence the class structure in society? So actually, it's about taking a technology and then running it through the simulation of what society does with it. And in a way, this is how we try to think about what AR is gonna look like in the future. We're trying to run a simulation about what this technology could be in a very dystopian setting, but also in a very utopian setting. Because if we really are the ones who author our own culture, and you can see it in art happening all the time. Like if you look at for instance, cyberpunk, Cyberpunk is this idea of corporations running rampant and people replacing their limbs with these cheap bionic implants and carbon bones and like, you know, heat sensors and basically augmenting their bodies in a way which is extremely in the time when it was invented in the 80s. It was extremely critical of both technology and corporatism of this surveillance capitalism. But then it turned into a mainstream thing and it lost its edge. It became a mainstream thing. And actually it started to influence this entire idea of what the metaverse is. So the metaverse has this very strong cyberpunk II vibe of you know, dark spaces with neon lights. This is not a future I want to live in, I would rather take a walk in the park instead. And this is why some of the values that we want to infuse into our experiences is sustainability and local action diversity. This is also something that needs to be part of our artistic narrative so that it can become a part of what people think about, as I spoke about the Thomas theorem in in the past the self fulfilling prophecy, like if all we do is vaporware, neon lights, acid rain, cyberpunk storytelling, this is what people are gonna think about. So we want to offer an alternative. Like it really is the fact that it's not that life influences art, art genuinely influences life, way more. So this is why crafting a narrative is so crucially important on so many levels. And apologies if I went super deep into something not so related to AR but like I genuinely think this is incredibly important.

Art influences life because it is a meme, or has the potential to be a meme that influences thought and behavior.

Absolutely. Yeah. You've summarized it really well.

Let's wrap up the last couple of enlightening round questions. This has been very enlightening already. Besides the one that you're building, what tool or service do you wish existed in the AR or VR market,

there are so many things, there are so many cool things that need to be built in, in the in the AR world. For instance, one of the things that I think would be incredibly cool to have are spatial notes, if there were ways for you to basically create some digital posted notes, which you can tag in space, that would be incredibly useful and fun. Imagine walking into an let's say, AirBNB apartment, and you scan a QR code, and you're in an AR session now and you look through the room and you see, okay, this is where the towels are. This is how I turn on the water heater. This is where I have like extra cutlery for guests, et cetera. So imagine how much that would change the way we communicate. If we had a communal experience. In AR, while you're teaching a class, for instance, imagine you're learning about the solar system. So your your students come into the class. And instead of you making you know, like doing doing a projection of PPT on a wall, you have them all scan a QR code and bam, now they can see a massive solar system turning in the middle of the room. And they can reach out with their hands, grab planets, expand them in front of their eyes and get more data about different plans in solar system. You know, these are just some on top of my head and like for gaming experience is something that we want to do with floor craft in the future. Something I think would be very cool is to build communal tracks where you can go and play your with your local floor craft toys. So you build an AR racetrack in a park. And then you bring all the kids from the neighborhood. And some adults, of course, you know, this is not kids exclusive. And they can now race along these racetracks that were built by the community. So yeah, it's it's really, you know, everything that you can build right now, just think about doing that in AR, if you have more than one person being able to see that it's really kind of changes almost anything that you're playing with, or teaching or trying to share.

Next one here, if you could sit down and have coffee with your 25 year old self, what advice would you share with 25 year old Amir?

Oh, wow. Okay, I would probably say you're so interested in so many things. And that's not a bad thing. keep learning and keep being curious about different fields of study different technologies, history, art, anthropology, human behavior, psychology philosophy, because at a certain moment, all of your interests are going to come together. And it's they're going to inform the way you look at the world, and what you think is important, and how you build something with values that have been infused by your experiences. So I would probably say like, don't worry too much like it's everything will eventually fit somewhere. You will you will have built a narrative that you understand.

Yeah. Stay curious. Keep exploring. Don't be too stressed.

Yeah. Oh, definitely stressful.

Any closing thoughts you'd like to share?

I think that if we are able to build the thing that we want to build, it's really well changed the way that humans humans live and communicate. One of the things that we also thought about a lot was how language was the first augmentation of reality it's in a way it's a neural virus. You know, like I can go into your mind I can reach into your mind with my mind language with small mouth noises as Terence McKenna, who is very dear to my heart would say, I can I can reach into your mind and I can convince you of something, I can incept you with something. And language has not changed anything about how we communicate for a very, very long time, we have technology, but the bottlenecks are there, like, you know, you can now type, maybe you can type faster than you talk. But all of these are very minut tweaks in the speed of communication. Like once we can conjure up things in space around us these interactive metal objects, it will really change the way that we communicate. And at a certain point, people will be thinking about like, how did we even how did we even do this before, like it's a massive shift that's going to happen. And I believe because AR is the mainstream of spatial Internet of the future of the Internet, this will be ultimately the thing that people feel most comfortable in the reason why we are developing not for VR, which is a virtual world you need to jump into, which kind of takes you into a different dimension. That's not what most people want. But most people want AR AR experiences. They want to have a local experience when they go and have a cup of coffee with their friend, or they play with their kids on the or they have the play game with their friends. And they're hanging out together in a beautiful green park in a nice part of town where it's sunny, where it feels comfortable. And they just have more digital elements in the world around them the same way they have radio stations today. Like I think this is a future that we want to help build.

That's beautiful. Where can people go to learn more about you and those efforts at medalists

so we are on all the all the socials. So actually, you can download floor craft Arena on iOS to try it out. And we have a bunch of really cool new updates coming out. This month, you can go to www.metals.com. To see our website, you can also check out something that is a passion project of mine and the C RC or nos, which is the cyber Delic podcast, which is something that we started last year. And we currently have six episodes where we talk exactly about the kind of stuff we talked about here. We also have a YouTube channel called madalas Twitter, where we can find this as well. And we have our Discord where you can come and ask us questions because we have the dev team there. You will find me there every day. So this is discord.gg/madalas where you can find us and also all the all the other socials as well. So if they've been marvelous, you will find us but the easiest way to hang out with us is to come to our Discord. And also to support us on Twitter and LinkedIn.

Demand. Thank you so much. This conversation is really enlightening.

Jason has been a pleasure.

Before you go, I want to tell you about the next episode, and I speak with Ari Grobman. He is the CEO of Lumus, a company creating efficient waveguides for AR glasses. As we've chatted in the past combiner optics like these are one of the most critical enabling technologies for widespread Smart Glass adoption, and Lumus has unique approach to providing a high quality image already joins me for the second time on the podcast to talk about their latest 2d expansion technology and just as importantly, the development of supply chain partners who are ready to manufacture at scale. Thanks for listening to the show and please consider contributing to the podcast@patreon.com slash the AR show.