I'd be happy to. Um, so I've been working in so my Hi, my name is Michael Markman. I am I do product design at a company called shapes XR. I've been in the VR industry XR industry now for since I guess like late 2016. So however long that is. I got my start way back when at a hackathon did my first XR project. And I, I just I got my start, I started a company where we were working on XR design and prototyping tool did that for a couple of years, sort of like trying to do something similar to shapes. After that, I worked on a designer for a game called poker VR, which was one of the we released on the first quest. And then after that, I worked on a bunch of different like smaller projects, tried my hand at immersive art, I wanted to do like these, like immersive, like, I wanted to do, like mixed reality art installations, where I was like, Oh, we're gonna bring people like together into like spaces. And then like, you know, that was like February of 2020. And then the world was like, No, we're not going to do that. We're going to do virtual offices. So I worked at a company called Arthur for a bit for a while, for a couple of for about a year or so your two. And then I got my chance to kind of try again at, you know, design and prototyping and making a design and prototyping tool again, shapes when they were like, Hey, we're doing it. You wanna you wanna you want to we remember when you did that other thing? Try again. And I'm like, Oh, yes, yes, I do. A chance at redemption. A second. A second shot? Absolutely. So yeah, so I've been at shapes now for about a year. And you know, it's been a dream. Yeah, that's kind of my, you know, my background in a nutshell, you know, I've been doing, I've been obsessed with spatial prototyping, and specifically, you know, prototyping XR and XR. To do it, the different techniques going from, you know, basically everything like you can do to design VR apps other than like, actually learning unity, right?
That's nice. So yeah, all of you guys, for whoever is here, or whoever is going to just check out this video. Later here. I just want to give a little introduction, also, who we are and what we do in this space. I am Alessio. And I invited Michael because I met him multiple times, and hackathons and other conference about AR shapes. XR is one of the tools that are kind of shaping a little bit the industry trying to, I think that the most contribute the biggest contribute to me is like bringing prototyping and design in 3d space, instead of like, you know, on the usual screen that we are used to work with. So I just want to specify that every time I do this kind of episodes, this is literally like, my personal passion, there is no relationship with my company. Because everyone knows I work at Magic Leap. So I just want to make clear that there is a distinction between simply the effort in creating a community and bringing people in this space to just talk about what they do, and being passionate about what they do. So just making clear. And yeah, I think we can start to talk about also like, in before to just get into the shapes XR route that I also downloaded a project and maybe we can just stream on Oculus and see what actually shapes XR is for other people that eventually they want to use it in the future for the projects. Maybe we can you can just like tell us maybe what are the apps that you think are most significant in the space? What inspired you in what is your vision for, you know, like, I know that shapes XR is like a VR app, but not really, because it can apply to both AR and VR. So definitely, yeah, what kind of apps you prefer that are on the ecosystem now and what what is the vision for what's the ideal one for you?
Yeah, I think like my initial inspiration, I think like you know, it really captured my Attention was probably Tilt Brush, I think was the first thing that really captured my imagination. It was an app created by Patrick Hackett and the other guy and eventually got bought by Google. And that was an app that allowed you to draw with a pen in 3d space. Right. And they, you know, were one of the first apps, you know, eventually they're like, they got the steam, you know, the steam VR trackers, and they were able, they created the first like, you know, app there, you could really, you know, draw around in 3d. And like, I remember I saw this video of a Disney animator, the guy whose name is Glen Keane, who did the he did like the little mermaid and the being created the beast for and he talked about, you know, he was this guy who his entire career, he had been a 2d illustrator and like an animator. But he described the first time he tried Tilt Brush. And for him, he was like, you know, was like, for me, he talked about how it was so natural for him to create with something like Tilt Brush, because he's like I, he was like, I always think about my characters as 3d beings. And then I just draw them from a perspective. And for tilt Tilt Brush allowed him to step into the page, and create them as 3d beings the way he saw them in his head. And for me, that was so powerful, right. And like, this is something that we like, like the creators of masterpiece, which is another their VR sculpting app, something they talk about a semester. PSVR is another app that I think is really powerful. They're like a fully featured like 3d There tool that's trying to replace Maya 3d modeling workflows. And they talk about how the most you know, the, the art of the people who find them the most useful aren't 3d modelers, like who were used to like tools like Maya Houdini, they're talking about, like the people who love them, to people who are like existing 2d artists, people who know how to draw, or like, don't like who like find like a mouse and a keyboard. So like, intuitive, right? Like the beauty of VR, what it does for 3d creation, that it removes the abstraction, right, it makes 3d as intuitive as like a pen and paper. And like that, to me is the beauty of it. Like it makes it so tactile and like human and like immediate, and like that. So that's those are a couple of apps that for me are like those are the things that like, like my spark, I guess.
Yeah, once again, I think that the best like, there is definitely like, an effort in bringing a lot of actually there is not a lot I think around that I see that tries to bring existing 3d modeling tools into VR, there is, surprisingly, the focus is mostly on apps that are very specific, ar, let's say, very specific to, I would say almost like an enterprise use case or gaming in other apps in VR, like definitely the ecosystem matured a lot for gaming. But I see like in all of the spectrum, I see very, like few companies that tries to just port basically like software from computer to VR, and it's not easy. It's definitely not the easiest thing to do. Because everyone trying these apps is gonna feel very uncomfortable at start so there's a lot of learning that you need to go through for for feeling great about you know, that Yeah, I think that in a bit we maybe could just show shapes XR if you want like I prepared myself with an Oculus quest stream so eventually we can do that and you can just give me some direction me well I try I'm going to try to do things live I hope they work there's always a chance that things go wrong. But before to get to that maybe another thing that I want to talk to about about with you is like in this panorama of pass through and see through application, where do you see how do you see changing like prototyping tools? How do you say i How do you see adapting a tool like shapes XR to something like that through and see through for example,
when you say like pass through so when you say pass through and see through you refer to like, like magically being like a see through device right?
Yeah, like eventually like lenses that are just like directly looking outside and you know, video background renders. Yeah,
yeah. Um, so for me the biggest, biggest difference right now at least, like, if we look at the current crop of devices, it really comes down or at least for us at shapes, it comes down to the controls. The like with the current mixed reality devices, they have VR, like controls, they have like the two handed you know, six off track controllers, and they have two of them. And for, for us, because of because of the way we play shapes works, where you can like kind of scale the world up and down, you can move around, we kind of really need the precision of the controllers. And we also use two controllers a lot. A lot of our interactions are two handed. So but then, most of the, let's say, see, through devices like the Magic Leap, or the HoloLens are either hand tracked or have only one controllers, one controller. So that's really been the biggest limitation for us so far. It's so it's not even the fo V so much as it is actually just the controllers, like that said, that said, like, and we've been looking at the Magic Leap a lot more recently, right? Like I would I, you know, I, we I tried the Magic Leap to at CES, I was like, wow, this is pretty freakin crazy. Like it was you guys figured out how to blackout light with a holographic display. I don't know how you did that, but you did it. So that's, that's pretty crazy. We gotta we gotta We gotta do something here. So that's so and then that controller. So there is, you know, potentially, like, there is something we can do with one controller potentially. But for editing we do and just we just, we do so much with like, you know, you have your palette on one hand, and you're operating a menu on the other hand, you know, with it. So, so that's so that's kind of is the biggest difference for us, like, you know, the Magic Leap to had two controllers, we wouldn't really we would, you know, it'd be like a no brainer for us. You know, right now, we'd have to change the UX of the whole app to adapt for you know, a different control scheme.
It's, it's tough. So like, there is a lot of interaction world going on, I feel like there is no really standard that has been defined yet. Unity is trying to do that. And you know, Unity kind of like it's the base platform for really almost everyone. So it's unavoidable that they try to do that. But yeah, I've seen also I don't want to be mistaken maybe if someone knows more about this, but I've seen also some some tools in Unreal Engine to kind of like start to see start to kind of change the scene from the inside. In game something like inside game editing,
Unreal Engine and unity have both had in VR editors for a very long time. Actually, at one point, the predecessor to shapes to vari, actually had an editor XR extension for unity, though there was like one of the unity unites, they did a demo together. But that was way back in the day. Um, so did you think so? Yeah. Unreal, has like an editor. So their own like editor, or editor type of deal. I think Blender has like a VR editor thing to it. There are a lot of them. They're all a kind of like, you know, they're a bit and some of them are better than others. Like I've always been really impressed by like the Unity labs team. Well, they're what used to be the Unity labs team, like they've always put out, like really impressive work. It used to be like, I would sit down for a cup of coffee, I'd be like, what is the unity labs team put out a blog post from the Unity labs team, like Oh, boy. That said, like, I think the biggest thing, I mean, the biggest thing for shapes was just being on a standalone headset and being a collaborative tool. Like that's been, you know, huge. Like, it's such a huge aspect, like being able to, like all be in a scene together with your team is like, such a huge aspect of it. Like, that's one of the main like, biggest selling points for us, for example, is like, Hey, you can all be in there together. That feeling of presence, that feeling of being able to discuss it in real time. It's not just, you know, just hurting myself. It's not it's not just so it's like, it's kind of like it's twofold, right? It's like one it's you know, being able to design VR and VR XR and xr two, it's then being able to do that together with people.
Nice. Nice. Yeah, please. Yeah, well, I set up the Oculus please guys go.
Okay. I will go through what my goal is that like, what? How do you see shapes XR to be to be used in like a workflow with Unity, because, like, I previously worked on a project for my school and didn't use shapes when it comes to like, thinking about UI, but I felt like the primitives were a little bit limited in a way, where I, it's like, Yes, I can get the low fidelity prototype down on paper. But then, in the end, I still have to go back to figma to get that high fidelity UI out. The thing and I, one part that I, like I used to, like about MCAD was that it had more colors and lighting options. So there was much more like, like, you could create more, but she's, I've struggled to, like, find a way, like, a way to incorporate it into my workflow. Because I feel like, like, personally, it felt like there was a little bit like an extra step. It like you said, I felt like it was only good when I was collaborating with my team to show them like, visually, how does this UI how is this you're gonna work by they're gonna look and in, in that sense, but I still, like one of the shapes tends to get that integration from like prototyping and shapes into unity, so that my salaries over role workflow, rather than adding an extra step. That's my question.
Okay, so I want to address a couple a few points there that I can, I'm happy to address. One is a here you want the color picker. And I promise you it is coming, it is coming soon. So yeah, so because it's originally shapes was intended to be like just kind of a rapid prototyping tool. So it was a very limited color palette, you know, so we have, but we have a fuller, you know, fuller material materializer come in, in an update. Um, in terms of like Unity workflow, we have a Unity plugin, actually, that allows you to input a code that will actually pull in your entire space. And so it'll, it'll continuously update. So it'll, you can, you know, your developer can input a code, and it'll pull in, you know, all of the assets from the space at proper, proper scale. And then the other aspect is like, for example, with figma. So let's say you're still doing like your high fidelity 2d UI, as in figma. Also working on a unity of figma integration, you will be able to bring in your figma frames into shapes. And so like, we don't want to replace figma, right? Like, we're not necessarily going to build like, we're not going to, you know, we're not going to build a better 2d editor than figma. But what we can do is we can build you a, you know, we can, we can do is we can bring in linear, or arrange them in space. And then when you make updates in figma, they'll update in shapes. And then the Unity plugin will be able to pull in from the figma frames from shapes will pull in your latest figma frames, from shapes into unity. So, it's a combination of kind of raising the fidelity of what you can create in shapes, as well as increasing both, you know, the inputs and the ease of like, bringing in your kind of like, it's a task of like, you know, making it easier to inputs and outputs, and then also just increasing the fidelity of what you can create in shapes. Yeah, like, I hear you on the color picker. It's like a, there's this, there's this, like, there's this constant struggle and building a product like shapes of like, those, like 15 things you want to build, right? Like, there's always like this big list of things where we're like, okay, we want to build the color picker, and interactivity. And like a figma integration, we want to fix like six different things.
And we'll have sorry, for interrupt.
We want to have like, you know, we have meetings, okay, let's prioritize, like, you know, we have like, Customer A says, We want to build, you know, we want this and B says this since he says this, and then you know, you end up trying to prioritize all the different things. And then you kind of end up in a position where you're like, oh, when do you you only have a limited amount of development resources. And so like, I think that's kind of where my career and that's kind of been like, I think like, you know, in all the different projects I've worked on, like this has been the defining struggle for every single product is like prioritizing features, because this isn't just a problem for shapes right? This is a problem at Arthur poker. Like, anytime you're working on an ongoing product, anytime you're only going on an ongoing project. It's like, what do you build? What do you dedicate resources to like? What is what is like the low you know, what is the thing that's like the relative amount of effort versus like impact that it will have?
In the future, like, what do you think ships is going down? Like in terms of like added features, or warships at more Modeling features where we could create much more like, a little bit more mid fidelity prototypes, or would it be going down to be much more like action based prototyping tool that allows you to do it interactions inside without code? So like, I feel like right now, it's, it's very much of a storyboarding to I just want to know, like, like, where is it headed? Because I, I've talked to the, there's someone's Darwin G. Yeah, I've talked to Gabriel, AWB, and like, we were discussing about hosting a workshop and my school art center as well, on shapes using shapes. The main, the main problem was that it's really hard to like know when to use it, or like, why to use it. That's like the part I want to ask you for, like clarification or like, what is the most like, which part of that that user flow ideas of like people designing for unity, or like VR or AR? So in that workflow, do shapes see that they provide the most value like this is the time to use shapes? This will help you with like, excellent. Well, I feel like that's the question I have. Because I do want to implement shapes into my workflow I just, I've been struggling, struggling with like, when exactly do I pick it.
So the action based prototyping is coming super soon. And then the higher fidelity tools are coming. I mean, it's decent fidelity. Now, it's like, the color picker is probably the last thing that we probably really, it gets, I think the color picker, like the lack of the lack of colors, probably then kind of the last thing that's keeping from everything from looking the way you really want it to look, we have a big update coming to like the text tool that will allow some more fidelity there. But there's a big update with interactivity that allow for like more action based prototyping to happen in terms of where in the process to use shapes. What we've seen is it's used a lot early. So when teams are like kicking off a project, they'll use shapes to basically like figure out like, where things are going to be like where they want to place objects, where they want, like, where they where they want to, like have the user start, where they want UI elements to be, like in the very early steps of where they're like kicking off with their team. And then kind of like, I mean, ideally, we want shapes to be used at all throughout, you know, throughout every step. Initially, we focused on sort of the lower fidelity, right, so right now, it's like, we want to kind of use the shapes earlier on in the process where it's like a more collaborative where the idea is still not fully formed. But what we're working towards, is getting to a higher fidelity prototype that you can actually conduct user tests with it. That's our goal. Like our goal is that like, you can make something in shapes. You can conduct a user test with validate an idea with the you could validate an actual, you can create a prototype that you could test on someone without involving a developer at all right? You can actually validate an idea or a concept, see if it works, or doesn't work without involving a developer, and significantly shorten your iteration cycles. I'm gonna grab my headset Alessio real quick. And then I can join you in the stream. I'll be right back.
So you guys are seeing what I'm doing. Okay, cool. Yeah, so yeah, I mean, well, yeah. Meanwhile, what is happening here? Can you guys hear me? Okay? Yeah, meanwhile, he grabs the headset. I just want to mention again, that is very important for me to put in this videos. This video is not promotional. is really just like exploring tools. I'm not getting paid for doing anything. Yes. So I just like a passion project that I have. So I just want to make it clear, because I don't want anyone to just like bring up things that are not true. But yeah, guys, I think that for I think, Lucas, when when we were at the hackathon also. I think that you made an app in Unity right where that app like for impaired vision which I think I also shared in the if you want to share it later in the in the chat that would be actually very cool project that everyone should check out. But like you had your workflow, mainly in Unity. I think unity is software that is used by most of the people that are creating an AR and VR If there is something to say, unity, it's something needs to be learned. And one of the main reason why I created this discord initially was to teach how to use Unity to people that were just curious about how to use Unity. So it's definitely something that can be picked up. It requires like a learning curve. And usually, it's not just a one or two days thing, it just like, you know, it requires you to get into an environment and understand that environment. Sometimes things go wrong. And you spend a lot of time to bagging in, I love it, like I've been there for a bit, but I understand that if someone else wants to access the media, it might be a barrier that needs to be kind of like, killed or taken. So what do you think Luke? is for your use cases? Do you think you would spend a lot of time in Unity? Or do you think you could achieve something like, Well, why not like with something like chips, etc? Or what would you wish the chips would cover for you?
Actually, you know, I, I actually more interested in the fact that because I just pulled in some of my cost, like my, my peers from Art Center, who doesn't have coding knowledge, I'm curious to see like, when did they find this useful? I'm not sure if they were willing to talk. But like, that's the point. But for myself, I see like the biggest, like, the biggest the only so during the full 14 weeks project that I was taking the mixed reality class, I only put on shapes like twice. The first time was for brainstorming, which is when I invited my friends like, Okay, we're gonna go into this space, and we're just gonna talk while we like those stuff. But even then, one biggest challenge that we had was that the mannequin, so I saw Gabriel, on LinkedIn have his own custom mannequin that you can pose. But the first of all, the mannequin that was in the shapes at that time, I'm not sure you have updated because it was like three months ago, but at that time, we were struggling with just like posing the mannequin, but it was in a way that we could, like the cars were all project was due to do like a mixed reality object. So we were actually using a mannequin to basically, like, show how this human is going to be positioned on like, a product like topic like this, this physical thing that we're tracking in VR space, and was the thing that will show up. And during that time, we thought it was useful, like, oh, yeah, we got the communication across. But I feel like the that's the thing about Oculus quest is that the barrier to going into Oculus quest is I mean, shapes fault. At this point, it just matters both to make like being like, we spent an hour and a half just to get everyone's quest updated, but also to like, connect it. And also to sign up for shapes to log into shape, like to download shapes, to log into shapes, to join the room in shapes and also be able to navigate and it was really hard for me to teach them how to navigate with shapes because I was like, look at like, you can't see my controller because your NVR Codling sounds like it's pressed that button and ended. So they're actually like, what is the x y like, cuz most of them are not used to VR at all in the first place. Even. So there was like some struggles that we had. But the session went well, in a sense that the communication of ideas, even though limited by the mannequin, and the shapes that was present, it was able to communicate that idea, but we I never ended up using Unity plugin to actually win. And that was also the biggest question I have is just like, I built this like environment in shapes. The like upgrade, and everything is not super, like scaled to size. But I build this environment issues. But now I can't. Like I don't really bring it into unity. Because I was too lazy to like import the actual models I want to use into shapes, and then export that import back to you nice. I was like only, you know, this is a great box like, this is where we're gonna put our shelf. Imagine it's a shelf kind of thing. And we never ended up like using it. That's not I felt like, well, it's great. But I brainstorming the use of shapes in my work is less than like an hour. In that 14 weeks that I did end up. That's what I really want to know like, what else like? Well, like what shapes it as a coat because I'm also a designer. I'm just curious, from designer to designer, like what is the use case of this product? Like not
trying to kind of replace we're trying to be like the figma VR basically, right? Like everything you would use figma for so like kind of like right now it's kind of a fig jam, where it's like, oh, it's a great brainstorming tool. That as we build out the tools that allow it to be higher and higher fidelity We'll eventually be you'll be able to vote right now you can import a lot of models that the place that old replacing the workflow is being able to mock up the whole interface and UI is right. So you have like the stages system. So we can kind of show I can show maybe some examples here, we can stream them, I can show some of the different, like use cases of shapes that other people have done. Or I can even show how we use shapes right now in our own workflow. If you're curious, I mean, this could be a really this, this could be a good opportunity, I can show you how we use shapes and shapes of design shapes.
Yeah, I think that would be really interesting. If you could show show like, like, just general use cases, so that like, and like, okay, like, once you're done with like shapes, I hope you guys imported into unity. What are some remapping of textures or shaders? Or do you need to do or like, what extra steps you have to do versus if I just like brought in like SketchUp model models into unity? What those are like some things that I've been curious about, and I hope, like I like I find that it's really a great collaborative brainstorming tool right now. For people who are new to VR, it's just doesn't connect to Unity. That's my biggest question.
So Alessio, I can send you a space code if you want to join me. Okay.
To do that, so please. Yours.
Oh, so if you go to the lobby. So let's do say we have a good Robo Recall space. You guys know the game Robo Recall? Yeah. Cool. Let's do that one. It's like a nice, simple prototype to go. Let's call it a great, thanks. Alessio. This space code is x. So if you go to the lobby, there's a Join button.
The lobby of Pixar or the lobby of Oculus? Shapes. You never know.
It makes it's fair. It's fair. A warm up those shaders.
All right, you guys can see what I see. And just making sure.
Smash that hit that join button. All right. X 5x, D. E, A. C two
Can you guys hear or see? We can see. Nice. All right. Nice.
So um, yourselves in shapes. On your left hand. There's a little button on the side. So if you twist your hand, you see that little microphone? Yeah. Cool. So we let's talk about we so that way, we will just hear each other over discord. So what I'm going to do is I'm going to put us into presentation mode. And what I'm going to do is I'm going to walk Alessio, through this prototype. So like, basically, let's take an example where I'm the designer. And let's say Alessio is a developer or a product manager. And basically, I want to present Alessio my design, like hey, Alessio, welcome to my design, we are working on rubber recall. So we have our first part. Our first part of the design is this little teleportation mechanic. The way it's going to work is that the user will lift their hand up and they're going to flick their joystick forward, and they're going to aim. If they aim, the aim onto the station. Last lighter blue. Then there'll be able to when they press the button, they'll teleport to it. Then they go to the menu and the menu loads. So these are all made with primitives in shapes. So then this part is a menu. Text on Zune. This is kind of like a recreation of the revelry call Main Menu. Yeah, so you got the menu loads in. You got your hand comes in, boom, boom, pointed at the status. You click on it. Click on missions. Then bam. You got this holographic little HUD. Let me take this away from you. Space link. Yeah, so these are all built using like shapes. As you know, these are just basic primitive shapes. Open shapes. Yeah, they can take us out Presentation Mode. And now we can you know, freely explore the space. You can scale ourselves up and down. That's so that you move around with the grips. You can also do it with teleportation What's your you've discovered teleportation? So let's see if you use your middle fingers on the grip buttons, you can move around this
Yeah, with two hands, you can scale up and down.
And I'm skinning myself up and
think of it like pinching and zooming and you can also rotate.
Oh, I see. Can you guys see what I'm doing? Just want to make this is pretty interesting. This kind of, like manipulation of space. Let me see what I'm doing here then. I'm not like I'm still in front of my computer. Yeah, this is pretty interesting. Look at this kind of like, person days or rotation. It's kind of an interesting interaction. Yeah, so
the percentage view is your scale. So you can keep scrolling up. Think of it like pinching like a tablet for example. Oh, exactly. Kind of like godmode
I see. Well, I can not see some architecture application here like I mean I've been in the space in architecture that's where I started everything and I was used to do similar apps not gonna lie where we you would just like move things definitely not not in this way because we're doing more like AR most of the time but this is very interesting.
This is very VR you see what I mean now by like two controllers
Yeah, yeah, yeah, I can tell ya, very interesting. I seems like also now when I hear also other people think in the space, but it seems like there is a chance to do a sort of like oh my gosh seems like there is a chance to kind of like do a flow of spaces that you can check it seems like a very good design review. What a game design review
exactly. What I'm gonna do now is I'm gonna bring a link I'm going to bring in a link to another space that we can go into so I can show you another example. So what I'm going to do now is I'm going to show you guys actually a design for our I want to show you all right Inga This is our this is our CEO actually
not sure what's going on
SEC only some audio stuff going on
Discord were muted Oh yeah, that's that's in good teller so here I'm going to show you guys I've got a another space we've got a bone lab prototype. That's kind of interesting. I want to show you create a link. I'm going to show you a couple of games examples because they're fun
to hear. Okay, there you go.
You're playing with this. And then hold the trigger down
okay, let me just go back live. Just making sure Can you guys see, Lucas? Yeah. All right. Just want to make sure thank you for the feedback. Okay, there is bone lab. So is that a hyperlink something
so point out it and hold the trigger. Nice. Yeah, we had initially where you could just click but then people were accidentally clicking. So we decided to make it not that nice. This is a cool one. So here so here press pulled out your locks on your left hand. Click the joystick in to go into like play mode. So clicking the joystick and then just like hold it down. So hold it to the right. And what will happen is all the slides will basically it'll be like an animation. Yeah. Oh, I see
what we're working on is actually allowing you to in our in our upcoming interaction system, you'll be able to create stop motion animations.
Stop motion, that's very interesting. Yeah.
You can almost think of these like frames
the beauty of being in VR is that you kind of get so this is that poseable this is definitely the poseable mannequin that Luke was talking about.
So this one camera was like Nathan, I was like, wait a minute, I don't have that. Oh, my shapes,
you can send you the model if you'd like.
Yes. We should have these models, like in your workshops.
Yeah, we should include this. thing is this. So there are all separate, they're all separate pieces. These are like the individual. This is like a little table. And then this is like if you want to see it on your own hands. If you come in here, you can, like put it on yourself. Scale yourself to 100%. You can kind of like, like it posture. Yeah, like, Yeah. Take a path through a bubble. Oh, no, I went outside of the world. Okay. You can see just this, this kind of still in play mode, you should click the joystick again. Now you're out. Okay, cool. All right. Let me now show you guys. More real real world example. I want to show you I'm trying to like find a design that isn't that is currently like, I'll find something that's like released already. So not like design that is out? Let's see. Let's go into the archive. Because our whole so everything we do in shapes is all designed in shapes. Let's see, bomb.
But there is a great like, I mean, look at this animation, that you can just go around and just like, take a look and see it in like from many different perspective, almost like, I don't see, it makes me think about a lot like cinematography and sort of like camera motion, you know, something
that's actually the DNA of shapes was. So the the app that came before shapes was to Vory, which was an animation and previsualization app. I see. All right. I'm looking for. I can show you you know, let's try the snapping prototype. I can show you that. That one's interesting. So
all right, but my models look weird. Let me know when yours restarts, and we'll go back to yours.
Yeah, no worries.
I can show you guys. This was my concept for kind of what I call responsive spatial design. But it's like little shape spurts. Built in shapes. Steps stepped out of my stationary guardian. So this traditional wedding I want to show off and shapes as we recorded this feature that we call a hollow notes. Basically what you can do is you can record avatar, you can record yourself as an avatar. And then there's a little play button. It talks to responsive This is me access websites and apps from all sorts of different mobile phones, tablets, and modern designs. This concept of
an assistant
kind of it's more like you know, like Princess Leia leaves Obi Wan. A message from herself. She's like Obi Wan, you're my only hope. We're like, man, wouldn't that be great for like productivity? Anyway, so this is like the equivalent.
Let me check if I can rejoin in here.
One other thing I can show is learning We built into shapes was oh my god way
we also made use of the stylus from Oculus you can draw
oh that's the feature of the workspace no work rooms yeah
it's not working
I loved you can just do it on your table it's pretty cool
that I think sorry say it again.
I was just making a joke send the pens out of ink might be something to do with my guardian
yeah nice you guys ever tried to work rooms
oh I know what's wrong? I know what's wrong. It's good. Snapping is on. Supposed to be off. There we go. Should work now. works like this. Oh, nice. That's cool. Isn't that neat? And then you can grab it, place it in your scene. Look, whatever this is the problem with live TV or one you're live Yeah, we kind of envisioned this to like you can kind of leave notes and be like, oh, you know, I think this is pretty easy. And then what's cool is shapes so you can kind of like see how something looks. So you can see like this interface here. For example, like I have my spatial weather app looks you know, I can see what it looks like in mixed reality. Then I can also see what it looks like in virtual reality. Responsive Design baby. One they think this is a nice little design thing is when you move around, we show you some we show you a little pass through window. So you release that you can make sure you have a clear spot on your desk
it seems very apt to say I was not aware of all these micro features of you know shapes XR it feels like a very well you know designed application. Really know that Well, I'm glad you showed us everything.
Oh, thank you. One nice thing here is one thing to do. One thing to keep in mind when you're designing for AR something like really important is to use shadows. So for example here like I put a custom contact shadow, when you wear this portal touches the wall. It adds a lot to the actual like, you know, like for example like you can tell that my hand is making contact with the wall because of its shadow right? Like this is something that I learned from like watching like Corridor Digital VFX videos. Like if I took away the shadow Portal would look a lot worse. I promise you you have my little avatar here. Yeah, just align this to your wall.
Savage is your voice opens up super aggressive out there. Just goes. So can you get in the portal or? Oh, it's like, sort of like
where the portal works, actually. Oh, yeah, the way the portal works. This is really cool. Or the portal works is it's actually made up of its it's made up of a material in shapes, called a pass through material that we have is actually so what we have is we have this pass through material that basically provides a clusion
reminds me of I don't know if you guys ever used food for you in the past, but there was something called Depth mask. And I was used to always use at school for
very similar. Yeah, I can show you guys another cool thing. So let me show you guys. Another feature that we have been shaped specifically around specifically around reality prototyping. We have like intern mix reality on, I guess it would require so like, for example a could. So like, let's say I create so let's say like I want to create, you know, like a mixed reality I want to have like, you know, I want to have the sphere, right? So what makes the sphere not look realistic is it's not being occluded by my desk, right? So what I do is I want to have a model of my desk
let's just approximate that guy. That's just like, good enough, right? Like that. But I do is a color it like not? Oh, now I look under there is. I see. Oh, nice. You're basically adding inclusion to it. So you can get the ball on? Yeah, exactly. Now does inclusion. Fucking cool. Is that? Pretty sweet. That's pretty sweet. All right, you got really excited about shapes sometimes like
I like to. I like the fact that you can see two things and just design on top of it. I think that in itself is seems like could be a very great use case. Since always, for me, like design, car design. You know, any sort of like, thing that requires like some sort of three dimensional space seems like a very good application of it.
One of the one of the difficulties like I can talk to you about like, design and insight into designing shapes, like one of the things we realize, is like when we were making this past material, like what do you do with the material? Like, once you've made an object invisible? Like? How do I like, move it right? Like now that it's invisible? It's gone, right? Like we were like, so initially, we had a design where like, there was just always like a wireframe on it unless you were in like a very specific mode. But that was also really annoying. So we ended up coming up with this compromise where like, relative to your scale, when you bring your hand very close to it, there is a slight wireframe shader that appears, you can kind of see Gsta see that? Yeah, yeah, very subtle. But you can see it, just enough to like, while you're editing, you can kind of see that it's there. But only when you're testing or you're using it still gives you the illusion.
No, like Alaska, we talk about like the AR for car, like hGD AR AGT for cars. And I know my school, a lot of people work on that like doing like AR or cars. And I'm pretty sure like majority of them doesn't want to go into Unity just to get that feature or function working. So I would love to see like, like this being used in those use cases, where it's like primary to really just two DUIs like expect in a spatial context. And I feel like that will be so much better for like, even user testing and stuff. Because, yeah, because I had a friend that did a unit test. I'm like, which UI should you put in a car that tells you like, when, when unsaved. And when you save and like testing warning signals, and they did it on an iPad, but the iPad wouldn't like stay on like a car dashboard. It is not a Tesla. So so like the position also changed the API user test a lot. I'm like really curious to see that this use this feature being used in those complex of designing for like AR cars and being able to use a test.
I've seen I've seen some videos that came out recently from Mercedes, and an Audi that are kind of like some concepts of UI that applies to you know, like the interior of the car, and also exterior for navigation. It's very interesting space because all On one side, you can think about having really like a UI for navigation inside of the car. But then you have like, you kind of thinking outside. Sorry,
did concept with a car that I'm now trying to find? It's somewhere deep in my spaces. Yeah, yeah. I
mean, like, even if we're not talking about AR or VR, I remember like, design student in my school once told me before, like, oh, they will love to have just a very simple like, XR typing tool that allows them to see like, a big banner, like their design being placed overlay on top of your life banner. And like to see like, at the size that that is. Because I like a lot of them. Like, mentioned, like, oh, you know, like, we're groping nicer. And we're not going to pay for a banner just to see like, on like that size, whether or not it works. And I, I feel like we've been talking about that in the graphic design department as well, like, a lot of people waste money like on print, because of their prototype versions. They want to see like banners size, and you're like, Oh, we were put just like, like, dude, like use the air pass through boat. And just overlaid 3d designs on a place we want it to be, that we're already incredibly helped a lot. The same thing for like, your design, like, a lot of people in my school, just try to design kiosks, but they say it and I pet. I'm like that's not realistic, like, you need to know, like how for, of course, his hand goes across the board to know, like, if your design makes sense. And I'm just like, I feel like my designer brains is thinking like, Oh, like this thing is not just like for each of your even though it's usable, like the classroom was totally usable for anything you want to design,
like envision, right, like the vision is like, eventually it's for all design, right? It's all designers doing all the different things, it's all the, for the chaos designers, it's for all of them, right? And then but it's like right now it's like, who has had sets who has the greatest need for designing in a headset for like spatial prototyping and that's where we start and then if we can build a tool for them, then as like headsets proliferate, you get them all right. And so it's like a but we've seen we have a client like Logitech for example, they built their conference booth for a gaming conference. They use shapes to prototype their entire booth then they built it so they design their booth they made this like giant freaking robot with like this was like a robot was like a robot with like an arm that was like it was like benders like lifting a car up and down. And then they actually brought in different sponsors to like check out the space for to actually sell the sponsorship for that booth did that all in shapes and they did it all collaboratively. And that actually ended up getting featured in Kinect as one of the use cases so I know I can talk about it because it's stuff
it's just really interesting to see like like how like when shapes was purely VR and on the input AR like pasta like the amount of like it changed so much and I feel like this differentiate itself from bezel tools like bezel a lot of what said bezel have like the I don't know if you can move the lie in bezel like while being in VR to know that you can like I saw this I'm like oh this this tool like this part of shapes and this this feature of shapes will be really good in use cases regarding like, just being able to design something like giant that will cost so much money especially for you statistics.
I hear you on the user testing Yeah, the magic is really in like being able to do it in XR right and to be able to do it spatially and it's like it to me and it's like coming back to like where I talk about like the beauty you know where like I kind of came into like Tilt Brush and like seeing like stuff like masterpiece and it's like Like to me like the magic of xr like to me like these headsets like what they allow like the fact that they make 3d less abstract the fact that like you can you break out of a 2d screen like you like they made like that is the magic of these devices to me it I think it makes 3d so much more intuitive I think you know the the ability to scale a world up and down move it around like it makes you feel like I described it earlier today to like we had a meeting and I was like this is like if God was designing the world this is how God would design the world right like you wouldn't sit there and like move it around with like, like it wouldn't like it's not like wouldn't be like blender like my right these are tools like It would be you'd move it around, you scale it up, you move it around like you would on a, you know, 2d, 2d surface on a tablet. This is the 3d version of that. I think it's the beauty is that removal of abstractions make gives it that magic.
Follow up. I love the discussion. And I was also, I think that, like, look, as he told me like, this takes a lot of time to set up right and in for like workshop and things like that. I went through that process, too, when I was a student at CyArk. And I remember that I was also assisting some classes. And there was also always this problem of setting up things. Even at the hackathon. I feel like it was an issue, like the first day for everyone was super tough, because people needed to pick up the tools and all these things. But think about it for a second, like, have you ever done like some design reviews, where they pin up to the wall, all of those drawings, and they just stay in line to look at those stuff. Like, yeah, like I was used to pin those up the day before or two days before, because we were like freaking out at the some corners wouldn't match. Like that was how architecture school was handling that in 2018, which is not too far. So I wonder like, I would have preferred that there was like a massive screen that everyone can see. And then the professor goes into every space that these guys designed. And you have like the possibility to see interior exterior in like 20 minutes review. And that is accessible to everyone through through the very entertaining, very entertaining to through the screen, right? Like you can just stream also to hire people instead of just showing to your students. And that would work also for remote reviews. So we're working from home and things like that, like hybrid work and all that stuff. So I don't know, I feel like a big step forward. I feel like most of now, this kind of AR and VR tools feels a little bit general. But I'm sure that when they will find their niche, they will they will make it they will make it as a standard almost. I have a question. Sorry, I didn't want to cut you look at school. Go for it.
No, I was agreeing to because like I took like a tight last last term. And we had to print like a poster that cost us like 50 bucks just for the teacher to be like oh now that you print it, you realize the type is small now reprinted like after you adjust it and you spend like a lot of one load just correcting stuff because you can't see the scale that it's supposed to be. So I'm like oh my like this might be so useful if I could just like when I'm working on it for Illustrator to just open it up like import it open it up and see that okay is in that skill does my font size look right? Even if it's not an actual print, just being able to get the type skill right would have saved me so much topics in a sense of like this I was like that I feel like that AR feature was the that I do include the past through layer in shapes because I was following shapes since Tamari pie didn't really use it but we were talking about it for like since since like what or it was like two or three years ago. And then shapes came about and and there was a little comparison in our school with like MCAD versus shapes and people ended up going with a cat the biomechanics down, it's gone. So everyone's second Well, there's alternatives right now and shapes are just like going to unity kind of thing but that's like for VR prototyping and like really specifically X or Y just saying looking at shapes or you know, it's like I don't even know how to do it on your I can just like you said like I'm pretty sure that I didn't shapes action based prototyping where you can like click N equals next thing that's available.
DM me if you want to try like a bait.
Okay, yeah, cuz I saw that I was like, Oh, I could just get like a kiosk or like a car you or whatever, with the posture with that, just get that test testing because, like, we had a lot of discussions in especially human computer interaction class where students are like, oh, I want to test to like to be able to use some kind of like hand gesture or like a controller for my TV that's far away to see like what kind of UI is best or and those are just like simple 2d use cases. That's not really intuitive. XRL but like, just imagine being involved in shapes, like using shapes, not just my accent, I read using shape or even like convention to the design, where you don't have the medium that you're actually working with because it's expense So,
there's universality, right? So like designing and production of like 3d things, like you saw with XR, it's like, if you have anything, when you build stuff in 3d for 3d, you have to visualize it. So whether or not you're building a VR app to a lot, or if you're building a building, if you're making a kiosk, if you're making a car, at some point in the process, you're someone, there, it's a stakeholder or it's a designer, or it's an engineer, like someone has to like, see it in person, right? So like a car might make it out of clay, sometimes, you know, you people might, sometimes you make renders, or you might like, actually go ahead and actually or you build it, right. And it's always cheaper to not build the real thing, and make a change in the design process. So like, the biggest problem that we're trying to solve in XR is like so much prototyping is done using production tools. At the end of the day, building something in unity is the same as like building a small house to see if you like the house. And shapes. It's like, oh, let me see if I like, let me build like a not a house. Let me not build a house. Let me like, let me make a mock up, you know what this house will look and feel like and then we will invest the resources into building the house. I know I want to have. And Alessio was like, Okay, leave the architecture to me, buddy. Like, but like, you know, the metaphor. This is sort of the metaphor I go with, but like all design is kind of like has a certain thru line. And, yeah, eventually, we want to be the tool for all designers, anyone who's doing any sort of spatial design, in any kind of design, right? Like, mobile apps are becoming more and more like, when you're ordering an Uber, right, like, the end of the day, I feel like designers at Uber probably go outside and like probably test their figma prototypes, like standing in the street right later, like a compressed thing. And okay, I'm standing in the street. I'm like, pretending to look for my car. Why not do that in VR, right? Like, why don't create a scenario where like, the car comes and like, if it's a high enough resolution headset, you have like, your little interface and like a little phone. Like, eventually, it's all going to be on headsets, right? Like in my head. I'm already living in a world where like, we all have headsets, not world, you're just designing everything, you're designing everything, shapes.
I have, like, I love this, I think the excitement and you know, like I division powers up the excitement to me and what can be used I got, I'm also a little bit like, kinda like when I when I when I was at CES, and I was talking about Magic Leap. I was like, you know, you can do this, you can do that, like, do you get it like you can actually do it. And you know, people get excited with you as well. And I have like a final question to wrap to wrap up the session. Thank thanks, guys, for sticking to this for a bit, it's been more than an hour. So precious time for everyone don't mean to take you more. But this is very interesting. I think we could kind of close thinking like about the scalability, and like the application of design tools on a, on a larger scale. Like, I think that design still has this kind of component of crafting things. And the demonstration is also in a tool like shapes XR, or many other ones that tries to do quite similar things where you are making things with your hand like literally. And the only like, you know, like lately, we saw a lot of big development with AI. So GPT prompt generation, AI image generation, not to fire probably ar 3d generation. So I wonder, like, in this context where things are moving in that direction, how would you see the shapes XR, you know, five years from now.
So you'll be able to like say words, and they'll just like generate a 3d model of whatever, as you want, right? Or, like, eventually, like, you'll be like, Hey, I would like your other cat and you'd like an environment and we'll just like generate a 3d environment for you. Right? Like we're not like, end of the day, like modeling is like, it's a means to an end right? Like we just you're shapes at the end of the day, we it's a tool for you to communicate your idea from point A to point B B being like other people. And so like if you can do that, you can do that by just like speaking it into existence. And if we can like then have like a machine learning model that just like okay, spits out 3d models of a house. Then you gives you the 3d model of an apple and a banana and you put the banana on the table and you arrange it you know, like I think AI assisted buildings. Hopefully the future like, hopefully this isn't all just like hot smoke. Hopefully this AI stuff is real. And like the pace of development continues. I mean, the stuff that like Luma Labs has been doing is just awesome. Like, I can't wait, like I've been playing with some of their imagine models in shapes. And it's awesome. So like, that's something that's really cool. So I can imagine you can just, you know, at some point, you're just like, hey, shapes, I need a model of Ironman for my Iron Man. Prototype, right. And it just like loads and Iron Man. Um, so that's definitely one thing. And then the other thing also is rendering I think, like AI based rendering, like, maybe you're just like you want like an image or something, or sorry, texture at rendering, like, let's say you want some like. Now, like, maybe you're still doing like some T, and maybe you're capturing like an image. Five years from now, five years from now we'll have like realistic shadows on mobile headsets. So like, maybe we don't need that one thing we really would try to play with even now it was like, we like use the AI to render shadows and scenes. But like, our scenes are like, so abstract that like the AI is like, what is a spatial UI? Like the thing it didn't really well, like, we did like some AI tests with like, two guy when you have like two guys just standing there. And I'm like two guys holding a cube. And it's like, two guys holding a cube. And I'm like, Oh, nice. And then I'm like, okay, spatial interface for ordering a pizza. And it's like. But yeah, I think the main thing for us is like, as they get into, like, 3d model generation, and 3d texturing, and that stuff is gonna be really cool for us, right? Like, making that stuff all really streamlined.
Yeah, that's, that's exciting. I think that, like, it's a demonstration of all the AI like startups that are popping out, like, they all have this generative model, but you know, you need to build it. To build your customization to it, you still need to do seems like a lot of work to make it work like you want like, like you said, like, you can say, okay, like just develop the shadows. But how do you apply that in a way that can be, you know, helpful, and like applying it to every situation? It's, it's pretty incredible. Yeah, would like the good thing is that, like, you have like a baseline that you can generate a lot of stuff out of nowhere. Like, for example, we were at this hackathon. And always the MIT and there were like, a team of people that I followed as a mentor, and they were like, We need to create a database of at least 100 When it was like environmental centers and show them on a GIS map, that was like, you know, saying no more like, you can just generate a GIS file, and it just imported in unity in in one go. And it didn't generate, like what they wanted, because they wanted very specific data. But you know, I was like, it's an academic just make it work, right. And they did it. And you weren't just worked. And so the speed that you can do the can, the speed that you can make mistakes is what allows you to succeed my opinion, long term, because to try something new, you need to make a lot of mistakes. So if that mistakes takes one second, instead of one day, can you imagine how many things you can, you can do. Anyone else want to wrap up and say something?
It's really good to have shapes, just so that you can like sometimes just take away step away from unity and just be creative. yourself. These are bogged down by UX UI, and needing to wear the headset because I really don't like wearing the headset. Even though I'm in this space. Yeah.
I'll just say I haven't used shapes before, but that demo was awesome. Really, really cool. And I'm definitely excited to check it out. It's Oh, yeah. Yeah, I guess closing closing words would be I think like the, you know, the big like, the best thing about shapes is it allows you to iterate quickly, right? Like, you know, just like make mistakes really fast, right? Like just throw out a million ideas and see what works and what doesn't work. And like, you know, if we can, if we have made something that can be helpful to y'all in any way then I'm happy. So pizza
Cheers. Thank you so much. Right? Well, and yeah, I think thank you all for joining this happens just to do to catch the opportunity to remain there next to in two weeks, we're going to have Dilma Palacios. That is gonna just take us to his journey and stuff like that. I'm trying to keep these every two weeks. Thanks for you know, great experts to come in and show it in a very animated way. You know, and with great passion, what they do. I think that's exciting for everyone that is part of this space. Thank you so much, everyone, and have a good night.
