The AR Show: 2021 Kickoff: Lots of New Devices Coming, None Ready for Mass Adoption

3:53AM Jan 5, 2021

Speakers:

Jason McDowall

Keywords:

ar

glasses

device

vr

smartphone

world

real

display

apple

hardware

focused

hololens

lenovo

optics

guided instruction

year

technology

early

company

incorporate

Welcome to the AR show where I dive deep into augmented reality with a focus on the technology, the use cases and the people behind them. I'm your host Jason McDowall.

Today I'm kicking off 2021 with another annual monologue. I hope you and your family have been able to have a safe and relaxing holiday despite the isolation and other challenges imposed by COVID. I'm going to share my take on some of the highlights from 2020. And what we might see in 2021. As it relates to augmented reality. This is the fourth such annual kickoff episode that I've done. And we're looking back at my notes from last year 2020 played out pretty much as expected, with of course, the glaring exception of the impact of COVID. I'll talk more about that impact later. But one thing COVID did not impact is the pace of progress at the bleeding edge of the technology. The industry continues to make slow progress. Sometimes it feels painfully slow towards addressing the biggest challenges of AR glasses. While the road in front of us is still long. I'm a firm believer that augmented reality in general and wearable AR glasses in particular will have a profound impact on our lives.

For those that are newer to the show, let me reintroduce myself. And for those that are already familiar, go ahead and skip ahead 90 seconds or so. I studied Electrical and Computer Engineering at Carnegie Mellon right earned a BS and Ms. I also study business at UCLA Anderson where I received my MBA. Following my time at CMU, I spent four years as an active duty officer in the US Air Force and a part that is now the US space force. Working on the development and deployment of large scale computer and communication systems and supportive satellites formed by the Department of Defense and other agencies. I got to travel the world and work on some really fun, cool projects. After my service in the Air Force. I began my entrepreneurial journey during the early days of the smartphone revolution. And by early I mean very early, well before the iPhone before Facebook even in that time before the iPhone I was the co founder and CEO of two startups before joining the mobile product management team@salesforce.com. Four years later, I was drawn back to entrepreneurship and co founded a mobile focused enterprise software startup that was acquired by another CRM company. Then almost five years ago, I took a step back and reassessed how I want to spend the back half of my career.

Since my days at CMU, I have been enamored with the idea of having a computer in our pocket connected to the internet. And so much of my career has been about making that device contextually aware and supportive, helping the user become more informed, more effective, or more entertained. When I looked ahead, the next frontier that embodied these same principles was AR glasses. Knowing my proclivity to jump into things way too early, I joined a very promising hardware technology company that is solving one of the biggest hurdles for AR glasses, delivering a high visual quality experience within a truly wearable device. That company is our standard, which I had the chance to talk and write about this past year. On the side. I'm also an angel investor and a venture partner at W xR fund. A female founder focused early stage VC investing in AR VR and AI.

And of course, I also get to do this podcast. I feel so fortunate to engage and learn from the world. In this way. I am grateful you take the time to listen. Rather than the interview. Today I'm going to share my deeply flawed, incomplete and biased perspective on what happened in 2020. And what I expect in 2021. I'll start with restating why AR glasses seem inevitable to me, and touch on some of the hurdles that remain. With that. Let's dive in.

My definition of AR of augmented reality is more broad than sum. For me AR refers to the ability to directly observe the real world while incorporating digital content relevant to that moment in space or time. This content could be basic 2d content, or more immersive 3d experiences tied to the real world which some call mixed reality. As you know, we can experience AR through a handheld device such as a smartphone that captures the real world with its camera and then incorporates digital elements for display on screen. We call this video pass through. The same concept applies to a headworn virtual reality rig that has cameras to perform the same type of data mix. But the real goal is to create a fashionable headworn device that allows us to directly view the real world while also projecting digital content into our field of view. Rather than video pass through. We call this see through video pass through is an easier problem to solve but see through optics allow us to be and feel more natural in our interaction with the real world and with other humans. But why do we need either one? Why does Tim Cook of Apple think AR is as big of an opportunity as smartphones? Why are Facebook Microsoft Google snap and a bunch of startups spending many billions of dollars a year trying to create a headworn device and the complimentary software and experiences. The cynical view is they have to do something new because smartphone sales have plateaued. But I think there's something more fundamental about AR glasses. If you're listening to this, you probably already buy into the potential of AR but I'm going to lay out the argument again in case it helps clarify something for you or it triggers A new idea. We are fundamentally visual creatures who think in three dimensions and live in a physical world. Right now, we are either engaging with that physical world, or our eyes are glued to a 2d screen. But there are times when our engagement with the real world, including the humans within it would be made more efficient, more effective, more safe, more human, if we could receive the right information relevant to that moment, without disengaging with the real world. And there are times when there's even more value if that information can be directly tied or locked to the real world. With the computers and smartphones we use today, the process of gathering and applying the relevant info in these moments is haphazard and inefficient, and often ineffective or unsafe. This is because we have to physically switch between the real world and the device. We often switch back and forth many times as we slowly comprehend and make progress. Before we can act on the information, we have to understand it. And before we can understand it, we have to find it, a system that has enough context to help us quickly find the right information, a system that delivers the information step by step and ties that info to the real world. A system that allows us to keep our hands and our eyes available for engagement is a system that could help in a number of situations. Let's take a look at a couple of examples. Let's say we're trying to assemble something to go through a set of steps in a process. This could be to assemble a subcomponent on an assembly line in a manufacturing plant to cook a new recipe in the kitchen to complete a new exercise routine to repair a lawnmower or maintain an aircraft engine to navigate to an unfamiliar location or to find the right item in a warehouse. If you're already an expert, maybe the availability of hands free heads up contextual information is less helpful. But we don't start out as experts and many of the things we do are done infrequently enough or are different enough that we benefit from some guidance. Although even for the experts, like airline pilots, it checklists ensures higher safety and quality. And that guidance is substantially more helpful if presented in small increments at the right moment with our hands free to take action. The same goes for more complex situations when we need to diagnose and resolve a problem. These are cases where an expert needs to be brought in to assess the problem in guide or help execute a solution. This could be to fix an oil rig to solve a problem in a manufacturing line, or to save a loved one's life after a tragic accident. I found myself in a situation once where I was attempting to perform CPR and receive instruction and other life saving procedures. While on the phone with a 911 dispatcher. It was challenging to juggle everything in that moment. Getting timely access to the right information, while simultaneously being able to act on the information can make a big difference. another batch of situations where mixing the real world with digital info is beneficial is when we want to see something that doesn't currently exist or is hidden from view. This could be to plan and execute a home remodel project with an understanding of the electrical plumbing and framing that sits within the wall or during brain surgery to precisely identify the location, angle and depth to drill a hole. It could also be to see an individual product or a bigger development project in the right location in the real world before we buy it or before we build it. There are many other examples of the value of visualizing data around art or social expression or entertainment. But generally speaking, these concepts around guided instruction, remote collaboration and data visualization are the areas that are most ripe for augmented reality. More pragmatically, the value that AR brings in these situations comes primarily from a reduction in the time from intention to action, meaning smart glasses one prove the time between our desire to know something or do something. And until that action is complete and completed correctly. It will do this by more than an order of magnitude by more than 10 times over today's computers and smartphones. even beyond these core use cases, I can imagine the glasses becoming a window into the information or value provided by all of the computing devices around us not just our phones, but also cars, set top boxes and IoT devices. I also see an opportunity to rethink how personal computing fits into our lives, making it more human centric, conforming better to how we move and engage with the world and relate to each other. As we've heard on this podcast, there's a lot that goes into making AR glasses a reality. On the hardware device, we need a good enough visual experience within a comfortable and socially acceptable form factor. This combination is proven to be exceedingly difficult. I've written a fair amount about this at AR insider and I'll include the links in the show notes. The device also needs to understand enough context to serve up relevant info at the right moment and the right spot within our field of view. It will do this with some local compute, which could be our smartphone or some integrated processing. And it will increasingly rely on computing in the cloud, especially resources at the edge that are accessible with high bandwidth and low latency. Some experiences are dependent upon a precise understanding of where we are in the world and what's around us. So we'll need a new kind of map of the world including a map of our private spaces. There are privacy concerns bound and about 100 other big and small challenges to solve before these devices become mainstream. But I still believe the biggest hurdle is that challenge of getting the display and optics for air glasses to be good enough to deliver decent visual quality and both physical and social comfort. There's no Moore's law when it comes to displays or optics. They progress in fits and starts over a relatively long iteration cycles. And sometimes it feels like we're moving backwards more than we're moving forward. For example, while Microsoft announced and began to produce HoloLens two in 2019, they didn't really distribute in any quantity until 2020. That's when we got an objective look, and the results were disappointing. While there were some significant improvements in the ergonomics and capabilities of the device, the visual quality took a notable step backward. The engineering accomplishments of getting their laser beam scanning display to work with their fancy diffractive waveguides was really impressive, the field of view did improve, but just about everything else about the visual quality got worse. It was a commendable and worthy experiment though, and I can't wait to see what they do with the HoloLens three. If we extrapolate from the release dates of HoloLens one and two, we might expect v3 in late 2022 or early 2023. Until then, I think we'll see continued focus on expanding the Microsoft Azure Cloud and a lot of learnings around ergonomics and user experience. Do you remember that big $500 million US Army contract that Microsoft one beating out Magic Leap after seeing some of the work done for the contract? I don't think Microsoft solution is suitable for soldiers in the field. The current generation of HoloLens technology, even when modified for the military, is unwieldy at best, and a dangerous distraction and safety risk at worst. I don't know what that means for the military contract itself. But I don't think HoloLens two is the answer for soldiers in the field. But I couldn't see the device finding a home for use in simulation training. Speaking of Magic Leap, there are 2020 went pretty much as expected, they were not able to complete an acquisition or raise more money by staying the trajectory they are on. Instead, they shed a lot of jobs, whole teams of employees really, Roni Abbott stepped down as CEO, and they hired Peggy Johnson, who was a former Microsoft executive. Now they are firmly focused on the enterprise. With this new focus, they announced a new $350 million round of financing. I foresee another interesting year for Magic Leap, one that could surprise with a new product announcement or an announcement that they are getting acquired. During last year's annual update, I read magic leaps woes in the mood of the investment community as a sign that we were approaching the bottom of the trough in the hype cycle as it relates to AR glasses. I think we hit bottom in the first half of 2020. With the struggles of Magic Leap, the abandonment of the audio only Bose glasses, and the disappointing acquisition of North by Google assign that North strategy or technology are not working well enough. But heading into 2021. The hype is building again, fueled by the hope that Facebook and Apple will release amazing products in the not too distant future. But it's hard to build a venture backed software company dependent upon hardware that hasn't yet been released. Just because COVID happened and now everybody is aware and excited by the potential for VR and AR doesn't mean the bleeding edge of displays and optics moves any faster. I don't believe the plans or realities of any of the major tech players were significantly altered by COVID. If anything, it's temporarily slowed down some research or development. While progress remains slow. With the bleeding edge. All of the current more mature technologies around AR had an amazing 2020 fueled by the changes in circumstances and mindsets as a result of COVID. For many businesses, a persistent resistance to change was replaced by a desire and need to change. Some of the software and hardware companies that were struggling to get some traction in late 2019 saw an explosion in adoption during 2020. Recent guests on the podcast Justin broad of Osso VR and Angelo struck lattanzio of apprentice.io. Both spoke to this the frictions or inadequate pneus of current hardware were overcome by the real impact that the software solutions can have. Other past guests that had a good 2020 include include a team that leverages the Microsoft HoloLens to enable teams to rapidly co create an AR for use in product development or training for things like manufacturing, spatial, the collaboration tool, built out cross platform extensions for web mobile and VR to complement its AR collaboration experience. Eighth wall sorry, a lot of adoption of its web AR technologies in real where has seen continued growth in the adoption of its head mounted display for enterprise. It has become a platform of choice for many software vendors. The next year, we'll see smaller increments of innovation, with a focus on better integration into existing workflows and infrastructure. The result should be a significant acceleration of revenues. One of the companies that continues to make strides is Lenovo. I started paying more attention to them about a year, year and a half ago or so, when they started hiring several very experienced engineers, product managers and business development executives from odg Epson and elsewhere. Lenovo has the perfect playground within which to make a lot of progress. And they are, they have a bunch of enterprise customers who already buy their hardware and software offerings for data centers and Corporate computers. Now they can deliver AR focused hardware and software solutions to those existing customers in a way that meshes perfectly with support maintenance, deployment and other activities they're already doing with a Novell hardware. For example, when a rack of computers in a data center experiences a problem, a Lenovo customer can pull on a Lenovo AR headset and consult a remote expert using Lenovo or some authorized third party software to resolve the problem. I get a chance to chat with Mike losee and Nathan Pettijohn of Lenovo about their efforts and the episode coming out later this month. When preparing for that interview, I was struck by a slide that I think Bridgestone the tire company had put together on it, they described a full enterprise lifecycle of VR and AR, and it went something like this. early in the process. VR is used for design review, where they collaborate on a concept before it's built for the factory. Then AR is used for design review to visualize the product or modification on the factory floor before they build it or install it. Then back to VR for operator training in the classroom before they get to the factory floor, then back to AR again for operator training using guided instruction with the actual equipment, and then more AR use in the steady flow of operations as a way to visualize real time data from various IoT devices, and possibly as guidance in completing work tasks. one takeaway from that in 2021, enterprise customers will be more sophisticated on the whole, they'll need less education, and more product integration and differentiation. Another takeaway, it's not one or the other. VR and AR are both important in the enterprise. VR is useful early in the product development process. And in in classroom training. AR is useful later in the product development process and in infield training, as well as in real time operations. I think about AR is focused on here in VR is focused on there. Which reminds me putting video cameras on VR devices does not make them AR glasses. It does make them better VR devices though, they become safer to use in our homes or offices. And they're able to bring a bit of the real world into the virtual experience, which can make them better for some training scenarios or for some gameplay or for personal productivity.

These video pass through VR devices can work well as long as motion is kept to a minimum. By this I mean the user should not be expected to move around the room much in the real world objects around the user shouldn't move much or at all. But I do not believe these VR devices will be viable alternatives to AR glasses when we want to actively engage with the real world or with other humans in our presence. And optimizing for a fully immersive VR experience results in a very different device than optimizing for wearability style and context in AR barrio is a good example of this. They recently announced updates to their dedicated VR and video pass their devices along with a $54 million round of new funding. Their products provide some of the best visual quality and video pass through latency of any VR device. But they are not optimized for being worn in social settings or operational environments. They are optimized for training and simulation and visualizing things before they exist. I couldn't see the inverse happening though, I could see a good pair of AR glasses being used for VR by engaging some blackout mechanism over the lenses. They wouldn't be great VR glasses, but they'd be good enough for some situations. And presumably, they'd always be with you. We're already starting to see a clip on visor included with some AR glasses. But before this concept of VR on top of AR glasses even becomes relevant, we first have to make a decent pair of AR glasses, one that consumers will embrace. There was hope that focused by North would be the answer. But the market said that version one, while a great learning experience for the company wasn't good enough. Maybe it's because customers had to put in way too much effort to get a pair that worked for them. Or maybe even then the experience wasn't good enough or the utility not high enough. In my conversation in early 2020. With Stephan Alexandra north, the company was very excited about an improved version too. But the broader market isn't going to see that device because Google bought the company at cost and said they weren't planning to release it. Although in 2021, we might see a new version of Google Glass that incorporate some of the technology and learnings from north. But even though North was focused on consumers, I suspect the next update from Google will continue to focus on the enterprise. On the tails of North's demise, we saw in Rails launch of the real light late in the year and Rio partnered with several mobile carriers to launch their device in Japan and South Korea, typically bundled with a smartphone and use to promote 5g wireless. The same type of rollout is planned for Europe in the first half of 2021. No official word on the US introduction. But as a marketing ploy for 5g an incentive to buy a new smartphone. I expect we'll see it offered by wireless carriers here to this rollout strategy and the specific characteristics of the display and optics in the device means in real has solved for something that North didn't. Their device will be broadly available. Anybody can walk into a nearby store and buy a pair. The question for 2021 is will they shoe the founder and CEO of unreal was on the podcast in early 2020. And he described the primary use case with glasses as a second screen for a smartphone when using entertainment or standard productivity apps such as Microsoft Office or social media, independent of the specific technology choices in the real light. This second screen use case implies sustained interaction with content that has nothing to do with the real world. This second screen use case seems better suited to a video pass through VR device, but maybe the relative lightness of the Unreal light and the slap on VR visor will make it viable here too. We'll see. Of course, it's also an AR device with sensors and processing via the mobile phone to render content that's locked to the real world. It has the potential to be the first experience many people have with AR glasses, but I don't think it hits the marks around visual quality or device comfort at a level necessary to be the breakthrough consumer device for which the world has been waiting. I suspect this will be clear by the end of the year. I'm hopeful the company will have a chance to iterate and incorporate their learnings into the next version in the following years. musics announced they will launch a new pair of glasses in 2021. To me they look like a nice update to the music's blade which will feature a micro led based micro display along with their waveguide optics. This will be one of the first commercially available glasses featuring micro LED display tech and I'm eager to check them out what's available. Epson announced they're rolling out incremental updates to their moverio glasses. You can check out my interview with Remy Del Mar from Epson published a few months ago, and Jeri Ellsworth at tilt five announced they plan to ship their glasses for tabletop gaming experiences in the first half of the year. After working through some COVID induced delays. Jerry gave a fantastic interview on this podcast in late 2019. Another consumer focused AR company is snap. on the software side. They are the poster child of mobile AR for social expression. They have a robust ecosystem and a powerful platform that gets regular updates. New in 2020 was full body tracking is a powerful machine learning library to understand and manipulate images. When I talk about context as the essential third leg in developing mass market consumer grade AR glasses. This is a great example of how significant progress can be made on existing mobile devices. I expect snap to continue to make significant investment in their development platform. Another company that embraced the mobile first approach to AR was sixty.ai they were building software to create an understanding of the surrounding environment for better AR experiences. Their journey as an independent company came to a close when Niantic acquired them in 2020. If you missed it sixty.ai founder Matt Mize, Nick shared a fantastic post mortem during a podcast episode a couple of months back. Niantic has begun the process of integrating the technology into the real world platform, and antic has their eyes on a future with AR glasses. About a year ago, they announced a multi year collaboration with Qualcomm to work on AR glasses, reference hardware, software and cloud components. If they're working on Microsoft timelines, it could be another year or two before we see anything come out of this, but if they're working on snap timelines, we might see some initial something in 2021. Coming back to snap past comments by Evan Spiegel suggests they will keep showcasing new hardware and learning from their customers, and that they see Facebook and Apple as their prime competitors in the long journey to delivering consumer grade AR glasses. Given the 12 to 18 month release cycle of spectacles one two and three. I expect we'll see version four in the first half of 2021. Perhaps they'll surprise us with some new capabilities. taking another page from snaps playbook Facebook is gonna follow suit and start by building glasses that don't include a display in an effort to learn. Project. Aria is the name of their internal research initiative that uses their own glasses that are packed with sensors but no display. And Mark Zuckerberg said in 2021, they plan to release quote unquote smart glasses that are not AR glasses. In partnership with Ray Ban maker luxottica. We'll see how closely they follow the spectacles playbook. One of the comments that Zuckerberg made this past fall, which I agree with is about device diversity. While typically a smartphone maker only sells a couple of different versions. Zuckerberg expects there to be a wide variety of styles and sizes of AR glasses, and I concur. This type of device diversity introduces some logistical challenges, but they are ones the eyewear industry has solved for traditional glasses. I'm confident that once we work out the display and optics challenges for AR glasses, the industry will be able to solve this logistics problem as well. Overall, I'm extremely impressed with the depth and breadth and openness of Facebook's research, I would be even more excited if it wasn't done by a company with a history of aggressively collecting data about us so they can sell it to advertisers. I think we'll see a continued effort by them to change this narrative in 2021. Speaking about large privacy minded companies with AR and VR research efforts, speculation about what and when Apple may introduce headworn hardware is still a favorite conversation topic of many in the industry. Robert Scoble will give his take on Apple's plans in the next episode. Here's mine. Apple will start with a device meant to be used at home. It will be a video pass through VR rig that will incorporate high quality audio and display, probably an OLED and will also incorporate some apple design silicon for local processing and integrate with Apple TV. high bandwidth, low latency wireless connection. Entertainment will be a major focus but solo fitness, it will integrate with the Apple Watch for fitness tracking, and eventually gesture recognition.

I think Apple's recent introduction of fitness plus shows a viable path to consumer adoption of more wearable tech. Today, the Apple TV integrates with the Apple Watch to track progress and trigger timely and relevant information based on the specific exercise content you're viewing. Adding a face one display with additional sensors can significantly elevate the exercise experience to be far more personalized, and more engaging. A reference to the potential AR utility around guided instruction that I talked about before. Apple doesn't need to rush into this. They're playing the long game in the previous cycle. They didn't enter the smartphone wars until there were already three and a half billion mobile subscribers and competitors were already selling 100 million smartphones a year. The VR market still isn't that mature today, but could be in a few years. The AR glasses market is still essentially non existent. If Apple enters the AR glasses market before the end of 2023. I suspect it'll be more focused on tech infused smarter glasses that incorporates sensors and audio similar to Facebook's plans. But Apple will be able to leverage more than 500 retail locations to help people understand and customize their smarter glasses. I can see the early consumer use cases focused on Fitness, Health and audio. These same use cases shine when they incorporate displays, so does navigation in the form of walking or driving directions. I can imagine these AR glasses could introduce additional utility by revealing and allowing us to interact with controls on a home pod or some other homekit device. In addition to having lots of devices and services within its own portfolio. Apple has legions of passionate developers who are familiar with AR kit, notwithstanding the handful that are currently publicly disgruntled. And in 2020, Apple introduced app clips, which is like a slice of an app that focuses on one specific task. This could be ordering food from a specific restaurant or renting out an electric scooter. The concept is perfect for third party apps to push lightweight pieces of functionality to AR glasses, apples lining up all the ingredients to make compelling VR and AR glasses and experiences. However, I don't think 2021 is the year that we will get to play with them. So to summarize what I think will happen in 2021, I think we'll see several new devices introduced this year, enterprises will embrace the new enterprise focused era glasses more quickly than they have in the past. And we'll see notable progress on improving integration into existing workflows and infrastructure. attempts at consumer glasses will continue to be disappointing new devices still won't be good enough for mass adoption. But we will continue to see a trickle of progress in the critical areas of display and combiner optics that will keep alive the hopes that in three to five years, we can have AR glasses and a truly wearable and fashionable form factor. for startups in the space. The biggest risk is being too early. And we're still very early in spatial computing. And of course customers don't buy AR for the sake of AR they buy a better solution to a problem. The cases around guided instruction, remote collaboration and data visualization of where AR can provide a meaningfully better solution because they can offer hands free heads up just in time contextually relevant information. There are some really exciting opportunities that can leverage the hardware available today. But you have to be thoughtful about where you apply your efforts and how you not just survive but thrive as the future slowly unfolds. And now is a great time to learn. I think the lessons learned today can be applied at scale tomorrow. I still remain very enthusiastic about the potential for AR glasses and I look forward to continuing to learn more about where we are and where we're going through the conversations I have with entrepreneurs, executives and investors. I am grateful to be part of this industry. Thank you for listening and Happy New Year.

Before you go, I'm gonna tell you about the next episode. In it I speak with Robert Scoble. Robert is a futurist strategist and the author of four books about technology. His latest book is the infinite retina co authored with arena Cronin, which explores the emerging era of spatial computing, including AR VR, robotics, autonomous vehicles, and more. In this conversation, we talk about his notorious Google glass shower scene, Apple's upcoming AR devices and Tesla. We also talked about his completely unacceptable behavior and the damage that sexual harassment causes to victims, their careers and the companies where they work. We also talk about his efforts to counsel founders to make better choices, and to seek mentorship and help to combat loneliness, mental illness and addiction. Please subscribe to the podcast you don't miss this or other great episodes. Until next time,