The AR Show: Ross Finman (Augmodo) on Customer Discovery and Leveraging AI + AR (Part 2)

4:01PM Oct 16, 2023

Speakers:

Jason McDowall

Ross Finman

Keywords:

ar

store

products

small

problem

founders

tech

customer

company

people

data

years

startup

number

money

picking

find

spending

view

business

Welcome to the AR show where we dive deep into augmented reality with a focus on the technology, the use cases and the people behind them. I'm your host Jason McDowall. Today's episode is a continuation of my conversation with Ross finman. Ross is the CEO of Og Moto, a company utilizing AI and AR to revolutionize grocery store experiences and economics, starting with the personal shoppers who fulfill online grocery purchases. Previously, Ross spent four and a half years at Niantic. There he founded the AR mapping and visual Positioning System effort before becoming the AR strategy lead and then the general manager for the AR headset group. He joined Niantic through the acquisition of his first startup as a reality, which became the foundation for an antics AR platform now called Lightship. He started as reality as he was finishing his PhD in robotics from MIT after previously completing his undergrad in computer engineering from Carnegie Mellon. In this conversation, the second of two parts, Ross digs into his current startup called Moto, we discussed the problem they're solving how AR and AI work together to provide a solution. Why now is the right timing in this the right opportunity, and some recent progress and announcements. Ross goes on to share additional lessons learned and advice for deep tech startup founders. As a reminder, you can find the show notes for this and other episodes at our website, the AR show.com. And please support the podcast@patreon.com slash the AR show, let's dive in is you have now kind of processed all of these opportunities that kind of have the characteristics that you describe what is the one that you are excited about?

Well, I can say that what I'm dedicating my self and my acquisition proceeds towards my company, Okamoto. So for that, it all kind of boiled down to a problem that I actually had after my first kid. So I guess circling back to the beginning, if you remember the baby formula shortage last spring, last summer, that made like national news. And then if you want to talk about personal pain point, feeding your kid, it's pretty high up there. And when you go into a whole bunch of stores, and they're all out of stock, then that was a major problem that I faced. So and then digging into that ended up on finding a very interesting problem. And I swear, it wasn't a me with a hammer looking for a nail, even though once you kind of figure it out, the whole market ends up being like, Oh, that's a very Ross finman problem. But we start to look at augmented reality search for retail. So if you that, hey, AR combines the real and digital world people spend money in the real world that stores How can you make the kind of store experience better. And one of the interesting things that can I ran into is post COVID. Like from an E commerce fulfillment, most stores are thinking of their physical locations as warehouses. So then there's a lot of Instacart DoorDash pickers that are going through their curbside pickup a lot of personal employees doing shopping on there, but fundamentally, like the reason why they list that baby formula is in stock, but then when you order it from five different places, three different zip codes, none of them actually have it. So they just the lack of knowledge of what is actually were in the store, and what are the idea of the different products? And fundamentally, that is a search problem. And how do you search like, Hey, where's it like? Not sure. For you, Jason, but my wife, she'll give me a shopping list. And it's like, Hey, can you go pick up some black lentils? Like, I have no idea where that is that in the canned goods section, and then is that in the bean section, it turns out, it was often like the Indian specialty foods section of my supermarket, and I had no idea and I went through half the store in order to find it. And many, many people run into that search problem. And I view like post COVID. That's now $120 billion ecommerce fulfillment problem. And there's over $10 billion of products that are ordered and not found every year. And it is on average, every order loses money. It's really interesting since Instacart, and last week, just came out with their s one filing as they tried to IPO. Their whole business really depends on their advertising, retail media advertising to actually make a profit on there. So if you can solve the search problem of where our products in store, how do you get people towards there and coming at it, putting it bringing an AR spin to it, doing that all hands free, that is a I think a major or a node as a major opportunity and building towards that. So more specifically, in that we're building software on top of existing pairs of smart classes. So think heads up display. If you're able to see your shopping list, you can use the camera for product verification, then the cool thing that we do, and this gets back into in terms of my background, you can build up an entire like map of the store, not necessarily for the map, but it's more for the products and how do you find real products in there? So how do you index the entire store and make it searchable? Just as people are walking around doing their day jobs, so like, they may have two dozen of your groceries that they're picking, but their eyes are seeing 1000s It's just the human brain doesn't process it but with the camera you instead record on there and actually build up hey, here's the full map of the trajectory that they walked in there. It's not necessarily about the map but about the all the products that they see. Where are they? What's Coca Colas put in a different spot. Now I pick Cheerios is the reason brand knows stock. So just gathering more information about the individual store, and the nice thing, they're getting back to what I said three times, like they're already spending money on physical devices that are handheld. And if your initial value proposition is, you get your right hand back when you're picking so you can push the cart, grab an item, and do that as a two handed problem rather than pushing the cart holding a phone grabbing an item. Like you're not juggling a third device. And if you can get more information about the store, through a lot of the computer vision AI, that is existing budget that they're already spending on post COVID A major problem and really approached it from a okay, and does AR actually help with that. And given my background was like, hey, I want baby formula for my kid and initially was like, Oh, is this a warehouse management problem? You're like, no, actually, that's its own its own problem. But that's its own separate problem. But actually, they think it's in stock and store and they don't know where it is. And what's interesting is like, I think it's three or $4 billion a year of products are in store and not found. So like the picker just didn't know where it was in there. And then they skipped on the item. So there's fascinating audits on there. Sorry, there's a lot to unpack in there. So happy to pause for any questions.

One of the things that stands out is the observation that there is very little known about what is actually in the store by the store, that all of this amazing supply chain that exists for you know, heard and read stories about how you know, Walmart is so intimately integrated with the supply chain, that stuff happens almost automatically as things move into out the front door orders being automatically placed in the back door so that you know the next shipment of whatever arrives. And it's all not only simply demand driven, but all this algorithms about when supplies are needed based on you know, seasonal flows and everything. But you're describing a situation in which a grocery store doesn't even know what they have on the shelves. Yeah. And I find that to be surprising, and a nice opportunity, I guess for you. But why is it they know so little about what's going on inside the store?

It's really well first off the supply chains and everything else like I am now just amazed that in like rural Idaho where I grew up that you can actually have a supermarket with 40,000 products with food from all over the world all over the country, that are all showing up at that time that I can order. So I think it is absolutely amazing that we can have that. And that is a two unique time in history. Like even if you go back, kind of like 7080 years like that really didn't exist, the scale that we have today. So I think that is amazing. But we get down to the in store, there's so much like things can get stocked incorrectly on, there's a concept of plugging that like when people don't like to feel that the shelves are out of stock. So then what people do, which is packed down there and say, hey, I'll take the green beans on here and the corn is sold out. And I'll just spread out the green beans over the corn section as well, too. So it looks visually like everything is filled out, then there's also shoplifting, there's incorrect scanning. So like say that you ordered two small boxes of Cheerios, and one large person at the front might just scan the small box three times and then move it all over in there and not supposed to do that. But there's usually two to 3% errors that just happen at the scanning section. So to put that all into context, from a high level, those are some of the specific reasons but even for some of the largest grocers in there, there can be 30 to 45% screwed up the planograms are the plan of where everything is think of it as the map of where everything should be, or the plan of where everything should be. So if you're thinking about like, hey, in some cases, almost half of the items are not where they're supposed to be according to the plan, due to variety of reasons that the stalker couldn't fit it in there, they got a different order, or they got like double the amount that they should have. So then they have to shove everything in there that displaces other items. Like it's a very dynamic thing. And just from all the data that we've been collecting from my team's like local grocery stores, I was really surprised at just how dynamic and also how out of stock many different things are. And a good thing from a consumer standpoint is any one of your listeners who's ordered online groceries and have them either delivered or picked up on average 10% are not found. So if you order 30 items, and three sometimes four items are not going to or they'll be substituted or not found which okay if it's like small thing like oh, they forgot the saffron on here. But if it's like oh there's chicken or there was a great sale on lobster tail that my wife actually wanted this past weekend it was like oh, probably is too expensive, but then we ordered it to plant and he got everything else for lobster dinner and we got everything but the lobster and that is the number one reason that people Old churn from a lot of these systems because they're like, Oh, I, I know where it is, I can go in the store. And it's not you blame the store, it's okay, I was unable to find it myself. But as consumer behavior starts to move into more preferences, again, hugely accelerated post COVID. They want that channel because it is really convenient when it does work. And people are willing to deal with a lot of the problems associated with that channel for delivery or curbside pickup. But like if you can actually reduce that, and improve the margins, than if you can improve the speed and accuracy of the people going through the store. By just having information of what is where, and doing that all in a hands free way, you can actually save kind of, from an entire market perspective. And again, just in the US and the 100 and $20 billion market, you can have like billions of dollars saved that way.

So the savings are significant. The behavior exists, consumers are already ordering groceries for other people to pick, these pickers exist. Instacart and others are employing them. They wander through a store today, they have some sort of other device, mobile phone, handheld scanner, or something to kind of go through and help guide them. And you're offering a two part alternative as I am understanding that. Part one is it as heads up hands free. So you can use your hands for something else and how they can see the list. And they can do the scanning with the head worn device as part of it. But part of it also is that you are building a real time map of what's actually in the store based on what that camera is seeing as you wander through the the aisles.

Yeah, so our key insight is most people in AR try and do everything on the glasses in real time. Or it's or it's like, they just simplify the features to what the glasses can do. But if you actually record the video and download it and post-process that, then the classes can actually be really small, lightweight, we're using some off the shelf b2b classes at the moment that look pretty good. Or at least, people don't notice it when I'm going around collecting data on there. So it's not like I have an air mountain missile launcher that some of these systems look like. But then there you can post process that data and build up the entire map of the store. And then the cool thing about that is, it's helpful not just for the store employees, but once the apple Google meta Microsoft glasses start to come out eventually, then a lot of the same tech that consumers are going to need. So it's really just solving the utility half of augmented reality search,

why is now the right time to pursue this opportunity.

So post COVID, like ignoring all the AR stuff, you just think post COVID that shifted a lot of consumer behavior and that any of these e grocery that went from like one to 2% of the business to now like 10 to 15%. So when from a here's a small thing that we kind of need to do wasn't a high priority to at the height of COVID was like, Oh my God, if we don't do this, or if we don't have our own curbside pickup our own delivery options, we won't sell enough things because people won't come into the store. And that's kind of settled into anywhere from 10%, sometimes up to 15, maybe 20% on outlier cases. So from a pure market perspective, in the last three or four years that has grown tremendously. And they just threw a whole bunch of operational stuff at it. But on average, they lose money on every order, which they didn't care about when that was like less than 1%. But when you have a red p&l for 15% of your business, that's something that they work on fixing. So right now and kind of the supply chain aftermath post COVID, there's a lot of dynamics. Now in terms of, hey, we need to fix through a lot of our supply chain problems. And this is a new channel that is currently losing money. So then people who go in the store and buy products themselves, they're subsidizing the curbside pickup, because they needed to increase prices on there. So if you can actually improve the curbside pickup operations, you can improve their inventory awareness, help them increase basket size and labor productivity, which they've already been doing inside warehouses. If you can actually bring up the from the store, then that can actually unlock it. So it's a not from an AR tech standpoint, even though there are some advances in there like better Wi Fi chips for downloading data, more storage available. So you can actually record a full day of picking session on there do like enough processing on the device. So like, from a hardware perspective, I don't think you could have done this four years ago. But more importantly, from a pure market perspective. If you think about the number of people who were using delivery or curbside pickup four or five years ago, didn't really move the needle. Now it's like they're bleeding money, and how do they fix that at the moment, which again, gets down into like the don't think about as an AR company, think about it as a like, here's a problem that people have, and that they can relate to. So then how do you solve the problem for them? And if the tech being hands free, and like you're not going to hold up the camera on the phone as you walk through, picking an order that's always going to be down or kind of sitting on the cart as they're going in the air so you're not actually having the right data. So it was a unique connection of the hardware working well with the market, but always be market focused,

always being market focused, are you able to make a dent in these numbers in these costs numbers.

So from a from our own testing, and some stuff we can announce later in the fall, can talk about just yet, there, you can actually improve the picking time even think of it like wave style routing, to get to all the products in there have an idea of, and to give a very concrete example, Coca Cola might be stacked in four different sections. So say there's the Super Bowl section where they have 100 cases. And there's the center aisle where there could be five to eight cases, people going through there, if they're picking, they're told to always go to the center aisle, because they can keep track of the variability of like promotional sections, or, in many cases for direct store delivery, Coca Cola, actually, it's their employees that stocked the shelves for these grocery stores, which is interesting, I never thought about that way. But then they don't know what is inventory or where it is. So then if they're always told to go to the center aisle, then it's out of stock in there, they say hey, out of stock, because they don't remember, they don't know or their app doesn't tell them to go to the promotional section where there's literally 100 them stacked up in the shape of Tom Brady made the go to retire in peace. So but that's kind of an example on there. So then for us, we've done a lot of experiments and gets an over 20% increase in speed. So in terms of time to pick and from a substitution rate, keeping track of that, we can reduce that by over 30%, just by gathering the information about the store, which is not like a Oh, we're better than a small handheld like barcode scanner, which with a small screen on their handheld devices, that is a major step up and not just a oh, we're 5% better. Here's something where even if you multiply it out to the size of grocery numbers, it can be huge, but it's really only incremental, if you can actually get full data of stores and you can prove your speed by 20%. And your substitution rate by 30%. Then that actually is across the market. And even for some of these retailers, hundreds of millions, if not billions of dollars, and you need to keep this in scope of a midsize grocer like this is not like the big ones is 100,000 employees. So like if we think about like that's bigger than meta, like all of us were like all meta the giant, their profit margins are astronomical and amazing. But then you look at like a midsize grocer. 100,000 people, they buy 1000s and 1000s of devices that moves the needle in a b2b context.

Is it enough to move them from the red into the black?

I mean, that is our goal. And it depends on everyone's p&l, but that's the pitch I start off with. It's like you go in there and say, Hey, are you making money on E commerce? And everyone's like, no. Well, I can help you make money on E commerce by improving your operations. Another interesting angle is like the brands also want some of the store data in there. So you can actually get brands to help subsidize it. Like maybe we're getting too much in the weeds for the AR crowd. But it's a fascinating dynamic where, like, if they pay for a promotion to say, let's use the Tom Brady Super Bowl, Coca Cola thing, okay, then that gets stacked up. And they want to make sure that that is set up in every individual store, they'll pay people to go out there and scan literally with a phone, that store and they'll charge they'll pay $150 per visit to check on five items with a two to four week lag time from when they hire the person out there to when they get the information. If you can say okay, rather than you doing that for a sampling of, say, 20% of the stores that you pay, how would you get that 100% And your turnaround is daily. And that's something that existing budget, they're already paying for your senior theme in here. It's like don't try and do something new. It's like how do you do what they're currently doing just better. So that's a way that you can actually improve some of the efficiencies on the labor side and improve the out of stocks and inventory awareness, without the just one change of device, which they're already spending money on existing hardware. And that's a recurring budget, and you can actually get external people to pay money for that. And then if you want to get into really crazy directions, which hedge funds they pay, not saying I'm going after this in the short term, but it was fascinating tidbit that I've never heard hedge funds can sign 10 $20 million contracts for satellite companies to just get data of retailers parking lots, like just to see like, okay, which way is the market going? And technically, anyone can buy that. But like when someone said, oh, yeah, that could be a great revenue stream. I like weight loss stream. So I'm not going down that path. You need to sell off other stuff first, but it's fascinating the amount of people who like to have data of retailers and how can you enable that data to be collected a lot better. And the nice thing about from the data is, I know on your podcast you talk through unlike privacy concerns, this is all private property. The stores own the land or they lease out the land. So then it's all full b2b, even though it's a lot of the consumer goods and same as they go off the shelf

on this notion of privacy See, these cameras are always on as these pickers are wandering the store, and they are capturing other human faces as they go. But how do you then think through the privacy concerns of the other citizens who are being captured in the camera feed as you're doing your processing,

I mean, we don't want that data, it's actually bad data. If there's people in there, then our map is not as good or product recognition is not as good. So literally, step one is once the data hits our servers, we scrubbed it, we remove it, we never touch it. So we pride ourselves on actively not wanting any of that kind of consumer data, none of the faces no one can see that. And that's one of the nice things about what we're going after. It's not like a a LinkedIn for conferences, where you see like people's faces and their LinkedIn profile above their head as you go around in there, which a lot of privacy concerns with that. But like for us, we scrub that all out, we don't want it, we don't need it actively, it'd be nicer from a pure data perspective, to not have anyone in the store. But unfortunately, we have to kind of scrub them all out to make sure our algorithms work on everything else.

Sure. So it's been a year since you started the company been in stealth for the last year? Why come out of stealth? Now, what's the motivation?

Oh, keep in mind with the first like six months of that was like, hey, is this an actual business? Figuring out on their fundamentally like, we are working with a partner, and we're part of their, by the time that this is announced, will be public with them. But a company called story I will be a grocery shop, they do a lot of the E commerce services side. And they also can do the picking, but they prefer to be a pure software company. So then we take over a lot of their in store operations. And for them being a differentiator, on terms of the kind of performance improvements, the out of stock detection, labor force efficiency improvements, then we can help on there. So since we're already going public for the conference, then I thought for the AR community, it'd be good to be like, Hey, here's at least an interesting path that other people can see that hopefully, this can help out my AR brethren out there. But yeah, for us, kind of customers and partners are very interested in having a differentiator. There's no other company like us out there. So then therefore we can be that can help us as part of our partnerships, and make sure we're elevating them as well.

As you studied the market. From a macro perspective, you're engaging directly with store AI and understand the world from their perspective, you're doing your own work out in the stores themselves wearing these glasses in navigating your own local stores, and you're chatting with other prospective customers along the way. Now, you've kind of built up this, this understanding of what's needed. And what are now the big challenges that stand in the way of you delivering a great experience for these customers.

So the hardest thing is that interestingly, for like the retail space, everyone, it's all about the sales process, because they're losing money on orders, then the doors get open initially, it's more a question of the technical robustness, many enterprise b2b companies, it's like, okay, are you solving the problem? are you solving it good enough, but just making sure from a operational perspective that you're actually delivering it in networking, like 99.99% of the time, and granted, like showing the shopping list, having like, checking out the barcode, you're mapping stuff, you can Wizard of Oz, that to a certain degree, if it ever, like blows up horribly, like, the algorithms are pretty good. And since we've been friends for many years, you know, I love my mapping and love going after that problem. So like, we're pretty good on there, but say, on the off chance that things will. So then how do you make sure that that is a good user experience, but number one thing for us is just getting the technical reliability, so that it can go again and again and again. And they can scale because fundamentally, selling into retail, it is a scalable business, because you can start with one store, then you get to the 10 to 20, than 100, and 1000. And once you've proven it out, and 1000, then like, Okay, you're now very tried and true. So then as you're in the early startup phase, just making sure everything is kind of buttoned up as well, so that because if you ever go down for a weekend, and some of our handheld competitors, like they've had weekends where they've had outages, that means that all of these people that ordered online are not able to get their orders. And that is a major pain point. And people will churn as a result of that. So then just making sure you have the technical robustness on there is the number one thing and that's from a mapping and product recognition standpoint, because that's our key differentiator,

technical robustness. That by itself is kind of a hard challenge, because there's always more nines, you can add in terms of uptime, availability, and robustness. Are there other things that look more like science at the moment that are new problems you have to solve?

So for like new problems to solve them there? Again, the nice thing is getting the incremental, like say an average grocery store is 50,000 products, so it can range 40 to 60. Somewhere in that range. Like a Walmart will have 100 and 60,000 products. So then I think even the 10s of 1000s. If you're trying to do product recognition, tell someone Hey, recognize one product, super easy recognize 10, super easy. 100. Okay, you're gonna have to work through some of the edge cases on there. If you say like, Okay, do 50,000, then like, there aren't datasets out there with all of the training data, and everything that you need to get up to that scale. So from a science perspective, like I have, and I have one person designated on the team, and like you are the 50,000, product or condition guy, and he's like, things are awesome. It just means doing a whole bunch of pipelines and labeling boxes for the next three years. And then he's like, I'm like, yep. So from a science perspective, it's just getting everything to work at the scale of the grocery store, which is why like, there is something that, hey, if you just want to recognize kind of 100 products, feel very comfortable in that that doesn't provide enough ROI. So you need to get into the 1000s of products and map them out in the store and be able to update those on a regular basis, which is not impossible, but it's not easy. And then how do you scale it up another order of magnitude or two, to go up from there? So I'd say just working at the scale? This is something like at Niantic. It's like okay, how do you make a game that works for kind of 100,000 players as they're playing Ingress walking around in there? And then when they launch Pokeyman, goes, like, how does it work for 100 million, and then just realize, okay, that is completely different architecture completely a different way of thinking about it. And for all the data flying through in there like that. It's a just a different way of thinking about the problem. So if you're trying to do, how do you improve the personal shopping experience, and make sure that you're moving the metrics on the ROI, okay, you probably don't need to focus on charcoal as a key product and put that in your database on day one. Because like the number of times that gets ordered online, not too often, you want Cheerios boxes, you need to have that down bulletproof. So the nice thing is, is even though like technical robustness is kind of the hard part, there is a gradient towards that. And there's an increasing value proposition where say, the first two to 5000 products are like 8080 to 90% of the business that they do. So then the nice thing about us is because this is a camera connected to people that are picking groceries, or targeting groceries just because of the throughput, you don't order sweaters online too often, but like there's hundreds of orders per store per day. So then you can have the throughput on there. But we're mapping those out, learning them better and better based off of the frequency in which they're ordered. So inherently, we're biased towards the highest velocity products, and learning them the best.

Yeah, that makes perfect sense, is you are a student of a number of markets now, including AR and retail. And you have spent all this time in the AR world, you can appreciate that there are cycles in the fundraising, the ability, the availability of funding from venture capitalists, and we're not at the peak right now. If anything, we're at a bit of a trough of the recent cycles. How do you think about raising money, the money that you need, in order to see this, this company, often to the accelerated growth that you want?

Well, let's just say I have a little bit of a unfair advantage compared to potentially some of your listeners having I don't necessarily need venture capital to get it going and more need it to scale. While many other first time founders or even some of the second time founders are uncomfortable enough, and my wife loves me enough that she can say, okay, I can spend enough money to do fine on precede. She said at some point, we'll have a relationship discussion, but she was vague on exactly what we'll see on that. So I was able can build up the team build out the product. And then once you have kind of customer traction, especially in the new tech space, so like, while you're pitching an AR vision idea right now, doesn't go very far. So then like people are like, Okay, I've been hearing about this for the next five years. Why now and you can have the most compelling story in terms of the why now you can have a great background. But fundamentally, there is that inherent skepticism or anti hype or in the trough, as he called it. I'd say number one gopher like, Okay, if you have metrics on there, and you say, like, Hey, here's what customers are saying, Here's people we signed up, then it's like, okay, in a space where everyone views there's hype, and there's not any traction on there, you actually have traction, oh my god, you stand out. Like, it's like, oh, geez, here's a company that like actually solves a real problem that people are paying money for. So it's a that is step one. And from a founder perspective, thinking about what can you do with less like, how do you get the first customer proof points because like, precede and even concede, one of the big firms are going down into seed, but like, if you're thinking that like, a lot of it is belief in the vision and the founders and can I go after that? But if you're fighting an uphill battle on just people don't believe in the AR vision or they're like, why now? Why not wait a couple of years. For my last company, the line now was hey, in three weeks, Apple is coming out with AR kit that is going to release mobile AR onto hundreds of millions of devices worldwide. We did our fundraising round in 10 days. But if you asked like four months before, that was also a trough and AR fundraising, so you kind of need to wait off of that. So from a for me personally, being in the financially comfortable position, can start to build out, get the metrics on there can be a little bit more opportunistic in terms of one do approach fundraising on there, which, right now, as we are launching with partners and kind of different deals associated, then can start to revisit that and so you don't have to have that relationship conversation with my wife. But I would say for the other founders that are out there, if you can find what is the quickest path Like say, for 100, grand, or 200 grand How do you get some initial customers traction, to get the story in there? Maybe that's not just in the AR case. Maybe that's across the board unless you're in Jenai. Right now, which there they'll still throw I just had dinner with a friend who at the peak in 2021 started to do fundraising and then for a precede got a $45 million valuation and like, was oversubscribed by 2x. And I'm like, oh my god, like that's a decent series A and like. So those days are over and people are getting back into traction. But I would say precede has come down a little bit seed of you, there's less deals, but there's more. So the valuations haven't gone down, but rather than 100 deals in a month, maybe to get down to 50. So therefore there's it's harder to be one of those before. I like other AR companies just focus on the traction. And I'd say probably the other way to think about it, not to be broken record is don't pitch yourself as an AR company. Like AR is a tech, it's two steps away from a customer. And there's kind of this interesting dynamic that I'm not sure it's fully replicated, like maybe in self driving cars do a certain degree, but like, the big players are spending so much money. And there's this vague notion that this is the future, the venture capitalists, they don't know all the different the ins and outs, the details, so they kind of feel like, okay, this will be it. But when everyone's saying now's the time, then you can have the echo chambers. And it becomes very easy to fundraise at that point, like two years ago, there was Metaverse, one meta change their name, and they're just like, oh, okay, this is gonna be the future, this is the now and then kind of goes off from there. But yeah, I would say it's a tough market out there. I would say don't think of yourself as an AR company. Think of yourself as a computer vision company. If you're doing more computer vision with AR, which is what we're doing. Technically, you could say we're a co pilot, even though it would kill my soul to jump on that hype train. But it's at least we're not not fully using LLM. But there is a small chat bot component for substitution communication. But anyways, better think of yourself as a computer vision company, machine learning company, if you're in the hardware space, you just need to find like, who is the right VC that'll go after that, like if you want to do the you can find a better business than I could on like the Kindle reading classes that can be more self sustaining. There are the people out there and fundamentally, raising venture capital is a search problem of like, which investor has the right thesis that matches your product? And it's almost a bad thing. Like I think in two years, we're going to see a lot of Gen AI companies that are very suffering because everyone thinks something is a good idea. Or what's the Mark Twain quote, I, anytime I start to agree with the majority, I start to question my own beliefs. So then, if everyone thinks that going after AR is a good idea, probably a good idea two years before, so I view fundamentally good companies happen during the troughs, but just thinking about it in terms of how do you get customer traction? If you have customers saying, Hey, I like this product for these reasons. And we're paying for it in these ways. Then any argument that they have against Oh, no, AR No one will buy that and anything else you're like, Well, I have someone buying it like what do you like your arguments invalid? Much better conversation rather than believe in the vision believe,

right? For sure. customer attraction is so important. As you kind of look ahead five years. What does outmoded look like at that time?

Well, I think it ties back into the initial framing of AR search for retail because I view right now we're just doing the utility half of kind of what is the shopping experience for personal shoppers, which is the title of like the pickers that are going around in there. But like, for me, I want to have my future Apple, Google meta Microsoft glasses, Samsung glasses, kind of shopping as an experience go in there. So like, Okay, I wear my sunglasses, my smart sunglasses in the car laid out there rather than me taking them off and putting them in sunglass holder. I just keep them on as I walk into the store. My whole shopping list. Like for me, I I tried to get out of the store as quick as possible. Like it's almost an efficiency game. For me. A goal is if I can get out of Costco and 20 to 25 minutes, that's a success.

Oh my gosh, well, you find it out two hours or less. I feel like it's a success.

Yeah, am I My wife is that way where she has to go down every aisle, let's not sail through, do I need that double pack and everything else in there so that for her, it's more of a discovery process. For me, it's like, okay, like, I eat a lot of the same meals, I know where they are, go, go, go. And then she's like, Oh, and go find sesame seeds, like it really frustrated, because like, I've gone three times, I have no idea where the sesame seeds are. But my son loves sesame seeds. So for me, my personalized shopping experience, or my AR search would be like, Okay, how do I navigate the store to get the products that I want? My wife, Carla, then she would love to be like, Oh, I grabbed this product and then see like a coupon come up with like, Hey, I grabbed the Coca Cola Pepsi, it's 50% off, have that kind of personalized shopping experience as you go through there. And fundamentally, that is a where are you in the store? What are the products that are in your field of view? Is there any information about them, like having a vegan filter, a gluten free filter, all of that is just searching over? What am I looking at, and what are the attributes of it. So that's fundamentally a search problem, you get into product recognition, it's actually a view, once you get a lot of integration into the retailers, then you can start to change consumer shopping, because it will be a great idea to go after AR shopping once there are 50 million devices out there being used on a regular basis. But like if you can actually have a business today, you can solve problems today. And then oh, now you can upsell your current solution to be the AR search engine for the Walmart app. Like it's hard for companies to build the maps the store, index all the different products in there, get all the different attributes on there, set up that real world app network. And also, I think, maybe beyond five years, but on the 1010 year horizon, just the cashierless checkout, rather than cameras up in the ceiling, Amazon Go when you're doing picking and I really need to show you this personally, Jason, like you're going through and just grab the item and you just look at it and immediately things off your list that does feel like the future of shopping. So I view fundamentally like, you can have your real world ad network for ads, because people like coupons, they can opt into that if they want, you can navigate your shopping list, you can have product filters as you go through on there. And that's all something that of you is going to be a major driving force for continued usage. Today, Eric classes are smart classes. More specifically,

I want to come back to this one thread around this advice that you have for deep tech founders. While you have a very technical background, the perspective that you've gained through your entrepreneurial journey is very pragmatic and market oriented and customer developer oriented. And you've, you've kind of really honed the opposite end of that spectrum, from the technical founder side to being very much a pragmatic market oriented, customer oriented. Also skill set, I guess in there. And as you have noted before, that you have this, this passion for helping other deep tech founders really kind of navigate the many complexities of the market, very talked about a couple of these pieces of feedback that you would share back around recognizing attackers two steps away, and from the actual customer, you got to tag them in the product and then the customer, the customer problem need to pay and leave in there. And in you talked about this notion of making sure that you're really able to articulate very clearly, you know, what the, the real value is that you're delivering, and and truly honing in on where people are spending money today, where the behaviors are today in order to identify those opportunities. Are there other amazing nuggets of insight that you would like to share to other deep tech founders?

Who would say, well, one, I'm very flattered that I now come across in that way, and some of my mentors from the early days of my last company, which to say I appreciate their patience, because when you initially met us, we were the deepest of tech founders, and then a couple of mentors that would essentially just like verbally abuse us be like, No, the tech doesn't matter. And then they kind of like shove your head dunk your take figuratively in the water, like now the tech matters, like dunk your head and again, you're like, No, no, it doesn't matter anymore. They're like, Are you sure in the dunk even one more time? Like, yes, the tech doesn't matter. So I think that that would be just the lessons learned of I wish I could say that that was intuitive from the start. But like when you come from a like Carnegie Mellon, undergrad, MIT grad school, like, there's a tech oriented ecosystem. So let's just say that it took me seven years to get compliments like that. So thank you. So for other deep tech founders, what I find that's interesting is a lot of the books and literature out there. It's like there's a lot of deep tech that gets mentioned, but a lot of the like, lessons learned, like, oh, how do you do the lean startup? How do you quickly iterate get an MVP out there? Yes, that's great. But like for Airbnb, you could throw up a website, do specific, like search engine targeting, and people would go in there and you could have a spreadsheet on the back end. If you're like, on the extreme opposite end. If you're building a nuclear fusion startup, and you're like, hey, we want to do that. Okay, you're spending the next five to 10 years in like pure r&d. demoed before, like, you get a sign of is a customer vendor that or not. And I view, one of the key differences for deep tech founders is navigating the product need and making sure that you understand the product, maybe because you have a little bit of a larger leap than other startups from when you've identified the customer need. And then you need to build something to address that customer need. And they're kind of in a leap of faith in the between that, so I would say, if you failed, and it's like three months, three, four, or five, maybe even a year of you like correcting that. So putting extra emphasis on understanding every nuance of the customer. So then people say, Oh, okay, do customer discovery on there, I talked to 4050 people, for deep tech founders, you should be talking to like 200, make sure that you understand the market. So when you're building the system internally, you can figure out all the ways to cheat, and I guess would be like nugget number two, find every way that you can cheat as possible to kind of speed up the development, as long as the end customer can't tell the difference. So then, like I talked about, for us on the mapping side, okay, we're building a very robust mapping stack on there. But we're also building manual tooling that like if it ever does, if the map does blow up, okay, we can fix that and overnight, and then maybe like, one or two nights of lost sleep for an engineer on there. But when the store opens again, at 6am, the next morning, everything's fixed, and everything's good to go. So then you can have kind of these fall backs on there, which from a pure technical standpoint, or myself six, seven years ago, I'd be like, No, it's better to spend it up and build it all out. But it's finding what are ways to cheat that don't take on too much tech debt. So there's a constant balance on there of okay, how do you please is for the end customer. And I guess this even gets back to what we were talking about on the hardware side of Niantic, which for your audience, like hey, building AR classes, major deep tech problem, but simplifying it down to like, we could go and talk to the game designers of Pokemon Go of ingress of other Niantic games on there, be like, What do you want from there, they would give you the laundry list, and you'd have to interpret that and you can't please everyone, you need to get back to some of the fundamentals of physics. But if you can have that close iteration loop, now that can work better when you're inside one company. But when you're doing a startup, you need to mentally have the full kind of counter arguments, the points, what are the things that the your customers care about in your head as you're developing the product on there, because again, you have like a multi month technical development jump that you go in there. And that's where you can start to ignore the fusion reactor, because it's so obvious, hey, I put in one watt I get out to, okay, you're now the richest person in the world. But then as you're doing, say, kind of computer vision, deep AI, some of the like nerf stuff, then there you can be building for months, maybe even a year. And if you don't have the customer mindset in your head, for Batum, you can live breathe and drink it just by talking to them understanding and making sure you're actually listening, because that's one of the problems for some of the MIT students that I mentor, love them dearly. And I'm doing to them what I got done to me, it's like, okay, you know, the, the tech doesn't matter, slap, slap, slap, but just making sure that you're listening and being able to reiterate, why do they why do they Why do your customers care about this, because when you're developing it internally, you may only get two shots at building a deep tech solution for your customer base. So therefore, that was one the reasons for my company why I spent the first six months like I had, after about two months, I had a decent idea of going after. And then I just went through, went to different conferences, talk to a bunch of people brought on, I built up my advisory board before I even built out and I kind of engineering team to go after it, just to have that feedback in there. Because for any deep tech founder, you don't have as many shots on goal. But on the other side, it's a lot easier to hire for deep tech stuff. Like I actually have been very fortunate for the team that hired because there's a bunch of AR people that are like, hey, an actual application. And I can see and people get excited about it. Like the founder of boom, supersonic. They're just like, Okay, we want to make supersonic jet, we want you to go to San Francisco to Tokyo in three hours. Like that's something that a lot of like you can NERT snipe a lot of people that way, can get them really excited. So while there's extra risks in deep tech companies I view there are a lot more extra benefits can get people excited about it. People want to help you, you stand out. You're interesting, you don't have as much competition. Like if someone said that they wanted to go and build a full mapping computer vision stack to try and compete with me and run everything on the small pair of classes and doodle. I would say good luck, like I hope you succeed. Other players they're like, Okay, I have five, six different competitors doing almost the same thing. It's almost a marketing game in terms of who succeeds, which you do that to a certain degree, but I would say that's, those are probably the main In high level lessons, and then there's a lot of individual nuances. Sure. For it for particularly companies.

Sure it makes sense. Thanks for sharing that. Let's wrap it a couple last in lightning round questions here. This one we've talked about before, I'm gonna love an updated answer about what commonly held belief within this AR, VR spatial computing world you currently disagree with?

It's not an interesting answer, unfortunately. But it's more of the mindset of like, if you build it, they will come. Yeah, a little bit of broken record in here. But this idea of, okay, I need to change the way that people do their day to day life kind of going after new budgets, I would say, AR needs to solve existing problems before it can open up the door to many new problems. So solving the initial unsexy pain points like of going through it and just focusing on how do you get to a million like, for the first person that gets to a million units sold, have an AR headset, like I'd almost personally fly out there and shake their hand be like you made it on there and celebrate that even though some companies like that pm would get fired, because they wouldn't have enough of the product to justify what they're going after. And that I view is kind of the trap that a lot of AR systems fall into. So just focusing on, like, what is a small thing, being happy about it? That's one of the things I actually even though Mehta gets a lot of flack, and don't talk through all the details on there, but they get a lot of things like, oh, Robin stories didn't do well. And Mark Zuckerberg is like, Okay, what did we learn? It's like, okay, they sold 300,000 400,000 units, like, like, hey, that's like that puts them in like, I mean, it wasn't really an AR System the same way that the Disney Lenovo headset was but that incremental step along the way and celebrating the smaller wins there, which I think the market is starting to come around to, like if you asked three years ago, still Oh, go big go home. The only thing that justifies all the money being spent is replacing the iPhone, which is probably true, but then gets into, okay, is that the way like the first iPhone, I think cost 50 $200 million, or something like was well, less than half a billion dollars. And now like, I think you've published articles on meta spending $10 billion plus a year for multiple years, and they're getting back now to five or 6 billion. Is that the right way to go after it? Or is it doing more of these small bets. But then the large companies they need to carry care about their brand. And if they like, do a little bit more Google, they're known for like punching small things and killing them. So there's pros and cons on that. But I'd say celebrating the small wins. Just keeping in mind, you can boast that you are the number one AR headset in the world if you hit a million units, and the ones here at a million units, then strategically think through like how do you grow to 2 million or 10 million from there. But don't think about 10 million before you hit a million just start basic start small focus on how do you nail one application? And actually, I think probably the first AR headsets that we see that are I don't even say headsets but smart glasses or kind of b2b systems. They're going to be very narrow, very low feature, but they're going to do a couple things very well. And that level of focus is very important.

Yeah, I really like that. What book podcast video have you consumed recently that you found to be deeply insightful, profound?

Well, this gets into deep passion of mine, there's this one YouTube channel called Perun. He does, literally hour plus long PowerPoint lectures on military logistics and strategy, which sounds like the most boring thing in the world. But like, it's this one Australian guy that went viral last year with the Ukraine war, but he just like, there's a certain tranquility and peace of seeing someone who is very, very good at their job, super passionate about it. And then also have a little bit of a sense of humor, but then break down very logically and then cutting through her he focuses a lot of the military stuff sighs the unfortunate war in Ukraine is a main thing that it goes into, but just the level of detail of like, starts off. So for those of you that are excited about 200 Page spreadsheets released for NATO procurement, this one's for you. And you're just like, and you release videos every Sunday, they're usually like an hour to an hour and 20 minutes. And then it's like a loyal following of all of us that just kind of like follow it. And it's just so raw and factual and clear. And like there's not a lot of the propaganda stuff that goes on there. So again, not really for the AR audience, but for those of you that like to go into a lot of the strategy and detail and things highly, highly recommend Perun My wife knows to take take her son like during one hour on a Sunday morning, so as I go to the church of Perun how to spell that P E R U N. So you'll get hooked. I know you Jason you're gonna love it.

It's it's exactly the sort of thing that I actually love to dive into. So cool. If you could sit down and have coffee with your 25 year old self now giving advice to yourself. What advice would you share with 25 year old Ross?

Visual who were responses don't do it? That is a very good question. I think the main thing on there, like when I was in my 20s is all about, like finding your passion, what makes you happy. And then like, especially in a competitive environment, like MIT, it was very, like, there's a lot of stress going on there. And I was faced with the many early 20s or mid 20s People like, Okay, what am I doing? Why am I doing this? So I actually wrote an interesting blog post, then titled, am I the dumb one, saying, metrics of life are the social interactions that I have, but then I'm spending all my life working on fulfillment. But inherently it gets me more and more in a niche that like, there aren't many people that I can talk to like you, Jason that can be like a Hollywood, here's this AR thing. And then now I've been like, okay, AR and then it's like, okay, how do I like search for different products and build an index map of a store and get in the retail space and all the supply chain stuff there. Like it kind of gets narrower and narrower as you go down in there. So I felt I was so focused on what is my passion, what will make me happy that I really discounted idea fulfillment. So I would say, focusing less on the little small things of chasing those little happy moments, because happiness is always fleeting. And we're going for like fulfillment, which is more lasting, which to be clear, this is me talking to my personal self, a few, probably a bunch of therapists would say, this is terrible advice. And your mileage may vary. But I'd say like, my goal in life now is not to live a happy life, but a life worth suffering for. And I view like that as an idea of going after the fulfillment stuff, not like the quick dopamine hits the enjoyment of like small little things like you can cherish those, but looking for that longer term purpose, and what are the things that you can obsess about that? When, like, six, seven months ago, I was like, Oh, my God, is there a real business in here and I was like, All signs were pointing to like, had like five or six bad meetings in a row. And people weren't believing and kind of was going after I'm like, questioning myself. And is this what I've dedicated my life to, maybe I should get out of the whole AR space. But then when you're in it, and you find that the only reason you're continuing to do it is because it's worth it. And it's a very emotional feeling. And you find it very fulfilling, and it's something that you want to obsess over. So well, it's a little tongue in cheek to say, going after a life worth suffering for. But if the only reason you're going after a particular, like startup or project or job, or something is because you just inherently love it. And there are the times when like, Okay, if you did not love it, you would you would have gotten out months ago. Then once you get through that, I find that very fulfilling and my 25 year old self would have been very cheese, if it's ever like too tough and everything else, you just haven't found your passion yet, you need to go to the next thing. When rather, it's like, okay, I still want to continue out this I want to drill down. And yeah, those are the types of things that I find super fulfilling. And, again, if you want to wrap this all the way back to the beginning, first year of having a kid, like, oh, my god, are you happier than you were beforehand? Some moments, yes. But on an average, you have less free time, you're like changing diapers, your sleep has woken up, you get covered in vomit far often than ever in freshman year of college. But like at the end of it, like, Could you do it again? 100% every day, love that kid, and it's absolutely made my life better. So I think thinking about it, in that sense, being a father on the other side. And going back to thinking about that in a career sense of was it worth the suffering? And would I smile and do everything over again? Yes. Wait to

bring it back around. That's really beautiful. Any closing thoughts you'd like to share?

I view perhaps you should have started with this. Some of the best advice I was ever given and of you listening to this podcast is probably a good thing to think through. But the best advice that can ever give you is to learn to filter advice. There's a lot of anecdotes out there. And if you throw away everything I say or or nothing here is helpful for you. I will be almost proud of like the listeners here. And the idea there is just the idea of filtering what makes sense and what doesn't. So from a startup context, then like when you become a world expert in space, like 100 people maybe give you an opinion, three may be useful, it's your job to find the three because those three can be the make or break for the company. So the idea of like filtering advice and taking it in realizing that a lot of stories, they have anecdotes that may not be repeatable, they may be smoothing things over with marketing, for startup founders out there that are thinking through like, oh, okay, this person is so experienced and they're giving me advice on my company, they may think of your company for five to 10 minutes, you've been thinking about your company for six to 12 months. So then making sure that you can filter the advice take it in like don't like a failure cases to reject all of the advice, but a failure case is also to accept all advice. So just as you listen to this, this is a My journey, my thoughts on like being a deep tech founder, my thoughts on the AR market, but like learning to filter how each bit of this and hopefully gather that information is helpful for you, as you're listening, that I view is probably the number one thing that I would say I hope I have crafted and one of the things just to take away from this conversation.

Excellent, excellent additional advice, telling everyone throw away it or throw it all away, throw it away? What can people go to learn more about you and to stay up on the work that you're doing there? acmo

Best thing I publish stuff on LinkedIn. So if you want to connect with me on LinkedIn, happy to also by the time this goes live, we'll have our website publicly available if you want to learn a bit more on there. And if you do add me on LinkedIn, have a personalized message on there. I get probably 10 requests a day from various countries around the world. I don't have time to filter through all of them. So if there's specific things you want to bring up in there rather than generic. Hey, let's connect we have a lot of connections on there.

That's probably the best way. Awesome. Ross, thank you so much for conversations with ketchup day care. Before you go, I'm gonna tell you about the next episode and I speak with Evan rose. Evan is the founder and president of Rose digital a company specializing in using augmented reality to help brands engage and delight their customers. He's also the founder of AR posts an online publication delivering augmented reality news from the front lines. Between these two Evan has been at the leading edge of AR for the last decade as an innovator and insightful observer. In our conversation, we dig into his passion for AR some of the innovative work he and his team are delivering and his perspective on the industry. I think you'll really enjoy the conversation. Please consider contributing to this podcast@patreon.com slash the AR show. Thanks for listening