When I lived in Seattle, a friend bought a boat. He described it as 'a hole in the water that you throw money into.' In the product world, that's a pretty accurate description of Discovery – you never know how long it's going to take or what you're going to get out of it. Even worse, we use the word ‘Discovery’ to mean a lot of things – yet most of the time, we don't discover anything new. Other people (current or potential customers) already know this stuff, we're simply hoping to learn.
← All talks 
Build Turing Fest 2019
Here Be Dragons: Product, Discovery, and How Not to Mess It Up
Auto-generated transcript - may contain errors. Tap a timestamp to jump the video.
Thank you very much. Thank you for coming and staying out. I realize I'm the only thing between you and drinks, and given the politics today, we could all use a drink. So I want to talk to you about a few different things. First off, if my name sounds familiar, like Hilde said, it might be because I've got a very silly name.
It might also be because you've heard this podcast, and if you haven't, it's really good. It gives me an excuse to call up the smartest people I know and ask them how to do my job better. So I want to talk to you about some of the things I've done that brought me here today.
So before I got into product management, I was actually a music journalist for a number of years. I helped start the music store at Amazon, both in the US and the UK. I wrote for an obscure underground country music magazine called No Depression, an obscure underground dance music magazine called Herb.
I was really proud of writing for both at the same time. I did some stuff with Tales by Southwest and the Village Voices Annual Critics' poll. I did pretty much everything you could do as a music journalist at the time. But when I moved to the UK, I spent six months launching Amazon's music store, and then it was up and running, and I was trying to think about, what do I do next?
I'd rather stay in the UK, but I couldn't do my job the same way I'd done it in the States. I had all the subject matter expertise. I could write, I could edit. I knew what I was doing, but I didn't have any contacts in the UK.
I just hadn't spent the time here, whereas I'd spent years and years building that up in the States. So I started thinking about what is it that I actually do? And it turns out working with writers and editors, and developers and designers, and lots of other departments, and helping them all create something that's greater than the sum of the parts and targeted at user delight, It's a highly transferable skill set.
I do pretty much the same thing now as I did as a music editor. So I took that, and I went and I did a bunch of other things. I worked with some radio groups in the US and UK, Sirius Satellite Radio, and Global Radio here in the UK.
Then the financial crisis hit, and there was no jobs in media, and I ended up going and being a head of product with HSBC in Sainsbury's, and now I'm an independent consultant working with a group called Motivate Design, and I'll tell you more about them in the second half of the talk, and also with Mind the Product, working with them as a trainer in doing corporate training and public training.
So if you're interested, we do have brochures about the training up by the stage, and we can come down and visit the booth later today or tomorrow. But it's the background as an editor that led me to the topic for this talk. We have a real problem with terminology, with the way we talk about things.
We're very imprecise a lot of the time. So quick example. If you work with your teams on burn down, you've probably used the word velocity. Right? But if you've ever taken a basic physics course, you know the difference between speed and velocity. Speed is how fast are we going, but doesn't tell you which direction we're going in.
Velocity is how fast you're going in a very specific direction, towards a destination. Now, if you're working in a waterfall way, you know what your destination is. Just get knocked through the tasks, one after another after another. But that's not really what we're doing.
We're trying to go towards a specific outcome. We're not always sure that the tasks that we're doing are going to lead to that outcome. What we're really measuring with our teams most of the time is speed, not velocity. We need to be honest about this.
It would be great if we had confidence, and in advance, we could predict what that velocity was. So let's talk about a few things that are more relevant to the act of discovery, though. Let's talk about what is a product, what's a product team, and then finally, about discovery.
Don't worry, I won't spend too much time on the first half. We'll get to the second half quick enough. So how many people in the room are product managers? Right. And of the rest of you, how many of you have to work with product managers?
I'm so sorry. So if you were a product manager, hopefully you're able to answer this question. Because if you don't, how do you know what it is that you're managing? So there's this business school definition for product for what a product is, and it's this, and it's really long and it's very boring, and we're not going to bother with it.
There is a much better definition, I think. It's something. It's a good start. And that something could be a good idea, a method, an information, an object or a service. And it's that something that serves a need or satisfies a want. Candy Crush is a fantastic product for me.
I'm not sure relieving boredom is a need. It's definitely a want sometimes, and it has to yield enough value to justify its continued existence. Thus, if it doesn't yield value, what's the point? No one's going pay for it. The organization that you work for isn't going to be around.
The person that funds the development and the running of that product isn't going to be around long enough. Value can be profit, but it doesn't have to be. You know, nonprofits, governmental organizations, they all provide products that are very valuable and very good sometimes.
But here's a key thing. How many products is your phone? To me, it's one product. It's one thing. It's this slab that I use and I can't keep get myself detached from. Maybe it's two products because I buy the physical device and I buy a service.
But how many product teams, how many product managers were involved in making the phone? Think about it. There's battery, and there's the chipset, and there's the screen, and there's the case. There's the OS, the system apps, all the other things that come with it.
To somebody, each of these things is a product, but to me, I don't really care. They're components of the product. The definition of a product really is ultimately dependent on what your perspective is. Perspective is really important, because you have to know where you're starting from and where you're going, where you're trying to get to.
So let's keep that in mind as we go through things, because we get perspective wrong a lot. It's really relevant when we talk about what it is that we do. We think that Mario, our customer or potential customer, plus flour, your product, equals awesome person who can do rad ****.
I'll go up this slide. Problem is we focus on the flower. We focus on your product. That's going to change over time. The real product that we have is the customer, the customer who's overcoming their problem. That's what we have to concentrate on.
If you only concentrate on the flower, you'll deliver something that will work for a while, and then their needs will change, or a better flower will come along, and they'll start using that instead. If you focus on the end point, on the end result, on the outcome, you'll always be successful because you'll be able to change your flower, change your product over time to suit their needs, potentially even give them a suite of products.
More product. More profit. Fantastic. So how do we get there? How do we build this? Because as I think it was Gabriela said earlier, product is a team sport. So what's a product team? So based on the definition we had earlier, it's a group of people that fills a need or serves a purpose and delivers value for the organization.
You might be thinking it should be value for customers, and that's absolutely true. But if you don't give value to your organization, then the organization won't be around to give you that job. So you have to keep that in mind. You deliver value to the customers because that delivers value to the organization, and that keeps you employed, you and your team.
So who's on the team? This is a complicated question a lot of times. I worked for a place that had very specific definition. It was four devs, two QAs, half a scrum master, and one product manager on earth. I see you've worked for the same company.
That's not a product team. That's a team that's going to give you presumably functioning code. That's not the end to end experience of a product. So you can throw in other things. You can throw in UX and research and copywriting and DevOps and any number of other departments, and then you can even throw in other specialties.
You can get bankers and lawyers or doctors, and when I was at Sainsbury's, sometimes we'd get store managers involved in the team. But that doesn't really tell you what a product team is, it just tells you who might be on it. So let's go back to this.
I think a better definition of a product team is this. It's a group of people that can do a few things. It can ask a question. That's quite important. Right? Especially if we're talking about discovery, we need to be able to ask questions.
If we're asking, we need to be able to get an answer. More importantly than getting the answer, though, is being able to understand that answer. We miss that out a lot. And even more important than that is being able to take action based on that answer.
If you can't do all of these things, you can't have an outcome focused team. And if you don't have all of these things, then you don't have the right people in the room. So this is an interesting definition, because it doesn't talk about roles at all.
It doesn't talk about who specifically has to be in the room, and it doesn't talk about line management. I think that's really important. You can have a team that has people that don't get Jira tickets, that don't come to daily stand ups. You can have a core team and an expanded team.
The important thing is that everybody on this team is critical in understanding what is the goal you're trying to achieve and helping you to achieve it. And that's cross departmental. That's getting as many different people as you can involved to make sure that you have what you need to achieve all four of these things.
So try and make sure you have all of this, and then you've got a team. That's my core definition. Marty Kagan's got a great line. If you're just using your engineers to code, you're only getting about half their value. It's a great line, and it's about half the truth.
The rest of it is that it's not only true for engineers. It's true for support, and legal, and marketing, and sales. Every department that you work with is more than their job title. They bring their whole self to work as well. You're only getting half their value as well if you don't go any further with them.
And Jenna, who's here in the front row, has a great line at the fact that people are hard. It's hard to get all these people together and to work together. And there's one thing that's even harder than people, and that's lots of people.
Lots of people are really hard. Working at scale is incredibly difficult if you've ever worked in a big company, even a small company with lots of people working on the same thing. It's hard because you run into things like Metcalfe's Law, where you start to get a whole bunch of people, and as you increase people, even very quickly, you start to notice the complexity in the lines of communication.
And that's before you even throw in management structures. Management structures, oh my god. You get Conway's law, the idea that you ship the org chart, and, you know, some very bad org charts. It's really hard. So how do you get to the point of getting everyone to work together, to focus on the right outcome?
That's where discovery comes into play. So let's talk about that. My friend Melissa has a wonderful line, an anecdote, about the difference between a puzzle and a mystery. A puzzle is something where, unless you buy it at a charity shop, you've got a box with a picture of the end result, and you have all of the pieces.
It's really a question of how good are you at assembling it. How can you work with different people? And if you've ever had lots of people trying to work on a puzzle at the same time, they're all reaching over each other and trying to do things.
So it's still complicated, it's still a hard thing to do, but at least you know what you're trying to do. A mystery, that's a total different challenge, because you don't know what the end result is supposed to be. You can't communicate it to everybody.
You might be working across purposes. You might not be working together. You have to turn that mystery into a puzzle. That's not a solitary activity. That should be a team based activity. You should get everyone together working on that to turn the mystery into a puzzle so that you can keep going.
That's what a lot of discovery is about. It's figuring out what is the right problem that we're trying to solve, and what does success ultimately look like. How do we know when we're done with discovery? This is always a fun question to ask a product manager, because we have it's a trick question to a certain degree, because the real answer is we you're never done.
You want to engage in continuous discovery. You always want to keep working on this. But the reality is most of us have a discrete discovery phase built into what we do. And mine usually ends in a room that looks like this, where someone in a more expensive shirt, maybe not a nicer shirt, but a more expensive one, brings me in and says, You've spent a lot of money, you've spent a lot of time.
What am I getting, and when am I getting it? And you have to give them an answer, and that's a terrible way to finish a discovery phase. It's a terrible way to run a process of product development. So let's go back. What the heck is discovery anyway?
I saw this yesterday. This is not unique to us. Science has the exact same problem that we do. You start off with a theory of everything, you're trying to answer it all, and you progressively get bogged down and bogged down, and you end up with.
So let's go into what discovery actually is. So my definition is the processes of not my definition, dictionary definition. The process of finding information, especially for the first time. That's great. We used to represent it like this. When we didn't know what was out there, we'd put dragons and dangers and whirlpools and things like that, and that made a lot of sense when the process of discovery involved being unsure whether you were going to be safe, whether you were going to come home, and made sure that everything was going to be fine,
that you took the proper precautions. I don't really have that problem most of the time. I might be spending too much money, depending on the size of my company, that might put us in danger, But aside from that, I'm pretty sure I'm coming home from the end of a discovery process.
And usually what we do in discovery is go out and explore and ask people questions. We're expecting them to give us answers. So are we using the right word? Is it really discovery if other people already know it? I'm American, so I grew up being taught in fourteen ninety two, Columbus sailed the ocean blue, and he discovered America, which was a huge surprise to the people who were already living there.
He didn't discover anything. He learned that there wasn't a direct route to India. He didn't actually discover America either. He discovered the West Indies, or learned that the West Indies were somewhere in between. But if you take it from a modern product perspective, he failed fast ish.
It took about three years, the first journey. He opened up massively exploitable new markets. It didn't work out so good for the people already living there. But most of the time, we aren't discovering anything. We're learning. And learning has a different definition. Learning is the activity of obtaining knowledge.
That sounds a lot more like what we do most of the time. But obviously, from the design of this slide, I think there's something else involved too. I don't think we can do any of this right unless we first get some insight, and that's a clear, deep understanding of a problem or a situation.
I'm sorry, type broke. But, yeah, this is incredibly important. If we don't have this, if we're not thinking in the right direction, then the rest of this is meaningless. So let's go back to this definition of discovery. I think it breaks down into a few easy to follow things.
There's the first of which problem do we have? Which problem do we need to solve? Next is how do we solve it? This is pretty much the double diamond approach that Janelle was mentioning earlier. And then finally, have we solved it? Not technically part of discovery usually, but really something incredibly important in the product development process.
These things we're really good at. We have lots of tools. We have lots of techniques about learning and doing these things. It's this first part, insight, that we don't have so many tools and techniques on. We kind of wing it a lot of the time.
We try, we mean well, but we're not doing it so well. We're very good at refining a hypothesis. We're bad at figuring out if we're working on the right hypothesis in the first place, because this is not the way we should be working.
And it's important that we do this because it's expensive if we aim x number of teams for y number of sprints. It might be opportunity cost. It might be just expensive teams. And you've probably all seen this in the past, the cost of change versus time.
Typical waterfall goes up like this. This is the risk that you assume. So we move to agile. Agile is the fact that every couple of weeks, or whatever your cadence is, you reduce risk back down to zero because you put really something and you test it.
And James Mays, who's also sitting up here, had a great line a couple of years last year? I think yeah, last year. When executives raise the concern that agile is making it up as you go along, it doesn't hurt to remind them that waterfall is making it up before you even start.
Love that line. Okay. So back to this, though. So there's an assumption baked into here that we don't always address. The assumption is that we're mitigating the risk properly, that we know where zero is on that y axis. What if it's down here?
What if we missed because we didn't do the right kind of discovery because we're not solving a problem that customers actually care about? This is the zone of uncertainty. This is where the dragons are. So let's get into that. Let's bring on the dragons and talk about how to get rid of them.
So I mentioned earlier that I'm working with a group in New York called Motivate Design. I'm working with them because I heard about a technique that they developed called Insider Insight, and I'm going to give the technique away to you, I'm going teach you all how to do it right now.
It's very straightforward, just like anything else. It does take some resources, it does take some work to do it, but it's doable by anybody. The basic conception here is that conversations happen in the wild. People are the gold standard of what we try and do is ethnographic research.
We want to go out and spend time and really observe them. We don't want to go and see them in an artificial environment. We don't want to see them in a zoo. We want to see them doing the real thing. Because if you go see them in the zoo, it's like doing a focus group.
It's, show, I'm here now, do the thing. They don't always want to do that. Animals do their own thing, and so do people. So you don't do this and expect a natural result. They'll tell you what they think you want to hear. They'll tell you what they're comfortable telling you, because they've only just met you.
They're not going to tell you everything. Who will they tell? They'll tell their real life connections. You'll tell things to your friends and your family that you would never tell to me, because why would you talk to me about these things? Why would you open up? You barely know me.
So these are the kinds of people, this kind of connection is important before somebody actually gives you the unvarnished truth, the thing you really want to hear. So the insider insight technique is powered by connection crafted by conversation. It features real people having real conversations.
So we've built up a network you can build up your own. It's got product people and UX people. It's also got yoga instructors. It's got social workers. It's got nurses. It's got cops. It's even got a retired FBI agent. I think that's my favorite one.
But the common thing about these people isn't that they're trained. There's lots of bias involved when they have a discussion. They ask leading questions. Sometimes it gets a bit sweary. They're all people who are interested in having conversations. They like talking to people and ferreting out lots of information, saying, why?
Why? Give me more. Tell me more about this. So this is why we use real people. They have authentic and voluntary conversations, and they unlock what people tell each other. We do pay them, there is an incentive, but it's not much. We pay them enough for dinner or drinks, you know?
The incentive is not the money in this one. The incentive is that they're doing something interesting. So I'm going to give you an example of how we started the process. We were working with the Ad Council in the US, and they were doing something on behalf of the Alzheimer's Society, and they want to know, Why don't people get screened for Alzheimer's?
So we ran a series of focus groups. It was expensive. We ran it in a bunch of cities. And we found this out: if individuals understood what the screening entailed, including details of how long it takes, how much it costs and whether or not it hurts, they'd do it.
So if it's noninvasive, it's free, it doesn't take that much time, hey, I'll probably do it. Solution's easy. Let's run an awareness campaign. We said, let's try our different technique. We have a hypothesis that this is going to work, so I'm just going tell you about the first time we validated this hypothesis many times since then, and we ran this insider insight technique.
The reason we did it is it didn't take months, it didn't take lots of different cities, it wasn't expensive. We ran it in two weeks, had about ten conversations. And we found the same thing. So that's good. At least we can do it quicker and cheaper. Right?
We also found out some more. We found out people telling us stories, things like the symptoms are the screening. It's not a disease, it's a disability, and I don't want to know. No one is going to respond to an awareness campaign about how great it is if this is what they actually feel.
No one in the focus groups told us this because they weren't comfortable talking to us, but they talk to their friends and their family, and they tell them this. And when you hear them telling you these stories or something like this, you know that it's not going to work.
So why spend six, seven, eight figures on an awareness campaign that isn't going to give any results? It isn't solving the problem that people actually want. The hypothesis was that people should get screened. The reality is something totally different. Let's focus on the problems they actually have.
So the technique starts with this: pick one tough, meaty question. And this is hard, because we're good at questions like this. Do people matching up persona like a new financial app? How many mothers purchase baby formula? These are useful at different parts. They're useful when you know what the hypothesis is, when you have something to test, or when you're just starting out and you're doing market sizing and doing some business analysis.
These questions give you an answer, but they're not telling you what the problem is to solve. You want to ask questions like these. What do young adults struggle with when planning for financial success after university? Or does formula mean you failed as a mother?
You want to ask these questions because these generate insight. These tell you what problems people really have. It helps you as a product manager, and as a developer, and as a marketer, and everybody else to really understand what are you trying to do.
How are you going to solve their problems? We've done this with a bunch of people, Facebook, Verizon, Nike, Skip Hop. I'll tweet out a link to this after the talk so you can read all this, but I want to get through everything. So you want to pick one tough, meaty question, and actually, getting the question right is one of the hardest parts That's worth spending some time on, and really workshopping it internally and saying, are we asking the right question?
You'll probably screw it up the first couple of times. That's fine. We do everything iteratively, but there's more. So next thing is activate the network. So we talked about the network that we have earlier. It's going out to that network that you have and saying, right, we have a specific persona that we want to solve a problem for, because we think that there is money to be made in solving their problem.
We have an idea of what the problem is that we want, the space that we want to be in, and we go out to the network and say, Do you know anybody that fits this persona that you want to have this conversation with?
And we give them a couple of days to get back to us, and that's when we start to make sure that we've got the right diversity, that it's not just a bunch of white middle aged people from a specific location, that we have a good range of people to give us that answer.
And then we tell them to have the conversation. So they go out and they talk to their friend, to their family member, people that we would never get access to normally, and in a way that they are speaking honestly about these things. And they record the conversation, and it's great because everyone has this portable recording device in their pocket these days.
It's really easy to do that. And then they send us the recording, and we get it transcribed, and we listen to it, and we read it, and we tease out the common storylines. We do a normal synthesis activity that you would do from any discovery activity.
So we sit down and we do that synthesis, and it does take a few people a couple of days, but it's a really valuable task because you've got a shared understanding among a group of people about where the common stories are, and then we share that with a wider team.
So if we're doing it as an agency, we share it with the client, but as wide a team within the client as possible. If you're doing it internally, you just share it with all your stakeholders, but also your developers and your wider team.
And together, you find the insight. You can't give someone insight. The idea of an insights department that gives you insight is frankly ridiculous to me. Insight is an emergent quality. It's something that you have to understand. It's that sudden clear realization. That's not something someone can tell you.
It has to come from inside. That's the really key thing. What I learned from this was something really interesting. I learned you don't need to talk to customers. Well, you do, but that's marketing. You don't need to talk to customers in discovery, you need to listen to them.
You don't have to be in the room, so long as you get the benefit of what the people are really saying, and you can feel the emotion and hear all that in their voice. So what have we learned through all this? We've learned that we have a problem with terminology.
We can be non very lack specificity a lot of the time. We talked about the definition of a product, we talked about what really is discovery, and why is it really important to do it in the early stages, and we went through the technique of how to do this better.
This is great. I love this because we were able to do it in two weeks, and the idea that you're able to go back and sit in front of your finance board or whoever is giving you funding and give them stories that real customers are saying and help them convince convince them, This is the problem that we're trying to solve, and we know it because they've told us, and you can hear it yourself, that's incredibly beneficial.
That insight, it removes the dragons from this map so that you can then go into the build, learn and succeed phases. But there's one more thing I do want to share, one more definition. This is my favorite one of all. Because we talk about success a lot, but we don't always know what it means.
And we have an awful lot of lagging indicators for success. We have things like dashboards and reports and charts, but they don't tell you in advance whether you're succeeding, and there's only one way you're ever going to find out. And the same holds true for your customers, as for your stakeholders, as for your team.
And it's very simple. When you go talk to them, do they react like this? If they're this happy to see you, then you're doing great. And if they're not this happy, you know you've got to keep working on it. Thank you all very much. Please keep in touch.