Designing the Future - Purpose + Concept (AR conversation volley)
4:18PM Apr 26, 2021
Speakers:
Brendan Langen
Keywords:
conversation
talking
person
design
overlay
today
communication
information
push
technology
volley
create
idea
graphs
life
thoughts
form
path
augmented reality
interact
So I'm sitting up on the roof. First day I'm truly unemployed and I'm thinking a lot about designing on interfaces that don't yet exist. The presentation layer that I am looking for, to design with in the world is still a few years away. I envision this as some sort of - just close your eyes and imagine you have an interface overlay on the right side that provides you with guidance, feedback and insights about the situation in the form that AR promises today.
If you see a map overlay and you're walking to Whole Foods. Perhaps the hours will show up, if you have 15 minutes left before the store closes. Or you may see sales because you wrote a to-do to get groceries today. Perhaps even, you have an optimal path to get into the store and walk around.
Now, a lot of that takes serendipity out so let's not focus too heavily on the specifics. The idea is providing the right context in whatever situation you're in. And as I sit here, I see base principles of how I can design these interfaces today. And truly, it's quite simple. Take a picture of what's in front of you create a 3d overlay in one portion of the screen as if you are looking as a first person gamer, whether this is Call of Duty or career.mode through your eyes.
Now none of this is too abstract, a lot of it is within the realm of our possibility today. It's just actually presenting it in a way without friction is not here. We do not have the Google Glass that is yet accessible and to the quality that people need to experience, augmented reality,
But it doesn't mean I can't design for those things. So the first thing I'm thinking about today is communication. I talked to Afika Nyati over the weekend, who has done really insightful work around things like AR, VR, spatial computing and design, and his mission aligns in some ways to Bret Victor's embedded information that is something you can touch and feel like Vannevar Bush's Memex.
And yet, there are likely basic pieces of groundwork, we have to lay before we would get in any of that, and that's a bit of what I'm hoping to do for myself today.
So nothing of what I will create is necessarily new. Like all things, it is but a remix of what exists in the world and has my unique spin on it. And the redeeming and refreshing, and, I guess comforting, way that we can consider design and development and creation, in general, is that we have all had our own unique experiences, and combined with our genes, we are all meant to create something truly unique.
That means our own path will be unique, just part of what I've done in leaving my job at Capsim, a job that I loved. So even while that might look interesting and perhaps even crazy to some, I know that I am making the decision that is best for my own path. And it's been the work of the last several, several years that I've put into hone that path down into a blend of design and technology and how it helps us interact with information and be our best selves, with others in the real world.
In three to five years from now, the technology will approach what I've discussed here. The idea of the embedded interface on the eye, perhaps glasses or a contact lens. This is coming. And some form of a glove or a wristband will allow us to actually interact with this digital overlay in physical space.
That's so fascinating to me, and there's such an opportunity to create new value for people to make this entire collective world a better place. And that's what I'm skating towards. I know the puck will be in that area, and I know as humanity we've never done the best job of using technology to give the masses, a better life.
It's not to say we haven't tried. Much of what Alan Kay and Doug Engelbart, and Ted Nelson have pushed for has been different than what has been commercialized and brought to the public. That doesn't mean it's gonna stop us in the future.
So the idea of the communication overlay, in conversation. Imagine you're talking to someone that you've never met before, youngish man, intelligent, wide-eyed, curious to hear what you have to say. You've met them through common interests, through some sort of matching engine that identified what the two of you are on similar paths to achieve.
When we're looking at the person and having a conversation, we might think, am I talking too much? Does this person get what I have to say? Am I on the right page at all? What does this person think?
What if we could give ourselves a better idea of how to have a better conversation? By using basic conversational design principles.
A few jumped to mind right off the bat. I often look at conversations like a volley in a game of tennis. You're hitting the ball back and forth. Sometimes leisurely, sometimes pushing each other to the limits, running from side to side or closer to the net. You might even have to jolt forward and reach to make contact, sometimes you'll miss, sometimes you'll return something that continues play in a perfect way. And sometimes you'll make them miss.
The volley is also a back and forth. Inherently, one person talks while the other listens, Then they respond, and back and forth. It at least always looks that way, but in its worst forms, and in its common forms, the two sides aren't necessarily listening to each other. They're waiting for the other person to finish talking so that they can start talking.
Here there's a ton of opportunity to be assist.
Part of it is providing additional context. If you don't quite know what this other person is referring to, or how they've come to this thought, it might be helpful to see a pathway, like Vannevar Bush's Memex. That would show their trail of information that got them to this point.
In Roam, that point would be a block, an evergreen note, as Andy Matuschak would say. And that could enter as an API to get more information about how someone is thinking. Sometimes it's just providing a chart or the facts as someone could see them as they read through an email or a news post.
But I'm more interested in the flow of conversation. One thing that has become a huge problem in the virtual communication from the COVID era has been that technology on the video side limits a proper volleyed back-and-forth conversation. For one, the bandwidth that carries each person's voice - in most products, namely Zoom, Teams, Google does not afford for lossless communication.
If someone else is talking at the same time. I've seen some tools, like Around, push forward on this, where I can be talking to a friend. We can both be talking at the same time and the audio will still carry through to each of our headphones.
There's a lot of research being done around spatial audio and improvement there. Just read a number of things from Facebook. Reality Labs in this area. And the technology will improve.
But our current situation has made it so that most people in a conversation will talk for a very long time to get their entire thoughts out, because the interruption possibility is not quite there, there can't be a quick back-and-forth volley. Instead, words often look like a four paragraph text with one person responding just a few words, and then you repeating another three paragraph text. That idea of the iPhone iMessage metaphor is really apt in our modern day communication.
Now as I said, technology should improve this a touch. But we could also use a little training. I often think in the form of prompts, especially when talking to myself. What would be helpful for me right now? What am I doing that is not helping the situation? Can I remove the misfits?
Imagine a conversation talking to this young gentleman - where you have a visual representation of your communication to the other person in the top right corner. Are you owning the conversation are you allowing for feedback and back and forth?Perhaps, as you've been talking too much you can be triggered to pause and ask a question as that flashes on the right side of your screen.
We often ask questions, though, that aren't as effective as they could be. There's some great research on asking good questions. What if we could bake in some of those principles? To make a specific ask to afford the other person an opportunity to speak openly, candidly, clearly, and push the conversation along.
What if, near the tail end of the conversation, we could see the threads that carried out through our time talking?
A live, evolving, dynamic graph of conversation that tied our thoughts, as they were tagged, to the conversation that we had. This would almost enact a digital summary.
If I'm imagining talking with another Roaman, someone who has created a deep knowledge graph, perhaps our knowledge graphs are intertwined, so that we can actually see the different pieces of our graphs that connect with them.
Now all of this could afford automatic note taking. New evergreen notes and thoughts that come in to our conscious. And like many tools do today, a quick summary follow-up could land in my inbox later in the day.
Now, this is just one concept of the wider possibilities within augmented reality in our lives. Conceptually, this is what I want to design.
We have such an opportunity to enable our interactions to be lightyears better than they are today. When thinking about some of the biggest challenges that we have as humanity in a shared information-heavy, always-on culture. How can we design, interactive solutions to our days that allow us to live in the world and just be a little bit better?
I believe that starts with information. It starts with communication. It starts with an understanding of the spaces and sensory pressures we experience when we're out in the world, interacting with others.
I don't know that I can imagine anything more lofty than trying to take this on. And as I said about the work that I did to understand this, it's still fuzzy. You can hear it. But that clarity and focus has narrowed me to needing to be technically competent. And that's what I'm devoting my time to doing
Within a short time, these tools will be in existence. And I want to help create them, so that we live a better life.
It's often the angle of the unseen that needs to be brought to light, so that the rest of the world can realize there's a better way.
