How to Use Customer Feedback and Data Analysis to Iterate Your Product

9:21PM Sep 23, 2021

Speakers:

Keywords:

customers

feature

product

api

stephanie

people

users

plaid

ebay

roadmap

feedback

data

spotify

envision

set

pandemic

important

startups

changed

partners

Hey, everyone have a great day here to discuss how to trade your product. And it's the third time we do this panel. But I think it's particularly topical this year because I mean, data analytics are more important than ever. And this means that startups need to balance this versus customer feedback and many other sources. So that's what we are going to discuss with our great panelists. We have Johnny gray is from Plaid, who is going to give us his future beer perspective. We have Stephanie, Mike Hurley, from indivision, which has another layer to the topic, because it's also used for people to iterate on design and product. And we have Pete Thompson, from eBay, which obviously is a really big and very diverse customer base. So I'm actually going to start with you, Pete with a question and data. And my question would be, could a start up or a tech company ever be true that that driven?

Me not a trick question, I would just say the answer is absolutely not. It's a matter of how you use the data and how you balance it with with other forms of feedback that you get. But I would say absolutely, I mean, it's just getting more and more important to uncover things within the organization with data that that you can't do with any other means. It identifies things that sort of human curation or manual processing wouldn't uncover. But at the same time, I think you need to be able to also think about sort of other things that you're not doing, or, or features that you could build that the data won't actually tell you. Just to give you a couple examples that eBay, so we have a, you know, global shipping platform. And we're constantly trying to figure out how to tell customers what their estimated delivery date will be, especially because we don't control the all the logistics, and we could be shipping something from Sweden to the US. And so we have hundreds of algorithms that are constantly checking things and looking at all different signals. And we were last year, we uncovered that we were actually, you know, about, you know, over 20% of our packages were actually arriving earlier than what our data was telling us they should have arrived at. And so we were able to, through that, we were able to get some amazing insight and actually set the expectations when people are out shopping on the site, that they'll get the product much faster. So that's just an example of how data can can do something that humans would never be able to come through all of that. On the other hand, we've recently been launching something that we call it searching through images, so not just text queries. And this is a great example where our data would never have told us that the younger audiences just want to be able to, on our site, want to just be able to browse and see, you know, if they're looking at fashion, they just want to see other things that look like the same image, not based on sort of a text based search. And these are things that you have to glean through other forms of feedback loops.

Great. Well, I'm going to ask you, Stephanie, if there were ever instances that that help you find things that you didn't suspect your user wanted.

Yes, definitely. And just to echo everything that Pete just said, data is important. I will slightly disagree maybe and say that there is a point where you can get to data driven, where you're not seeing the what I call the the edges of the innovation bell curve. So it's really important to see what smaller cohorts are doing, because they could be early adopters to a new behavior. And that's something that that we frequently look at and envision and in previous labs for myself. And a lot of times when you look at those early adopters or behaviors in the data that you're seeing that are smaller groups, and you make sure that you're driving features that are driven with hypotheses behind that, you might not get it right all the time, because it might not be an early adopter. It might be someone who just does something really different, like we see at envision, and the way that people work, that everyone is a bit of a snowflake, right? We all have our different workflows. We are all unique humans, and it really matters, the team that comes together and the goals that they have and the intricacies of how they work together. And so we're finding at least in our data, that it's okay that everyone has a unique snowflake, we need to create things that are malleable and are usable that people can like make into their own as part of their workflow.

Right well They'll actually echoed something that Johnny told us when we're preparing for this panel, which was about the concept of representative customers. So Jenny, could you please explain that? And also, maybe tell us how you select those users? Because I'm sure it's not an easy process?

Yeah, sure. So, I mean, this goes beyond data, just because about how you approach you know, building products, you have you have hundreds or 1000s of customers that are telling you they want you to build, you know, different things. How do you? How do you go from that to, you know, a product roadmap, that makes sense. And I think something, something for us has been really important is to identify the customers who we think if we solve their problem, we solve the problems of a host of other of other developers. And the idea is to treat those customers we call representative customers more like partners, right? So the key though, is if you pick the wrong representative customer, you may only solve their problem and not a broader problem that applies to your wide user base. So you have to you have to be very careful. So I'll give you you know, I'll give you an example of one customer that we work closely with, actually over the last nine months this year, that's been very helpful. It's a it's a small PFM. It's not small, it's going quite fascinating, really well called copilot. They're an app to help you manage your finances. And well, we recognize that they were say finally more more support tickets per kind of, per unit of billing that they have for us. And we found interesting that we looked at the tickets, and we thought that their tickets were very sophisticated, meaning they had a really deep understanding of our product and how it worked. And it's contours and realizes that actually like that one customer we had was the best one to get insights from their end users for us, because they were willing to like push that information through us. And then we looked at them a bunch. And we saw that they represented a big category of our customers, maybe like 1015 of our customers have very similar use cases. And so you know, you had that you had a market mapping where you were like someone very demanding and sophisticated. And then you look at new stuff, like their needs, were very similar to a large segment of our customers. So you have those two things in common. And then the question is, how do you turn them into a partner, so that as opposed to just filing support tickets, right, they're willing to talk to you and help you develop the product roadmap, and that works super, super well for us? And, you know, the I really like actually, what, what what Stephanie said, right? When you look at data, or even like product roadmapping at all, there's the danger, if you have too much data is it looks like he's right. It's just, it's like the average of everybody and from the average is difficult actually, to build great products, right. And so you have to find a way to like segment your customers into little buckets. And some of those buckets are like, you know, potential trends that are emerging, like new sets of developers. But there's, there's, there's other buckets that make sense, right? Like market strategically for your business, you think you think three years, four years from now, we're going to be really big. So I really urge people like first, like, kind of narrow down your set of users, and then kind of looked at the data and then really go deep into the the kind of product signals that you're getting.