Intent-driven tests lead to understanding consumer intent through rapid prototyping. How does one build an organisation where failure is part of its DNA, what practices do they follow and what differentiates this strategy? Supriya shares her experiences from her work at Amazon.com, Priceline.com, Zynga and other startups.
← All talks 
Build Product Turing Fest 2017
Failing Fast: What is Rapid Product Development?
Auto-generated transcript - may contain errors. Tap a timestamp to jump the video.
Hi, everyone. How many of you attended my talk this morning on the strategy piece? Okay, so the first five minutes is gonna be a little bit of a repetition because I'm gonna give some context on myself. But then hopefully it'll get better after that.
How many of you have failed today? How many of you have done something that you contribute as a failure today? Good for you. I mean, there's like, you know, three or four people that raise their hands, right? Kind of goes to say that we don't take failure as a part of our everyday life, right?
We want to succeed. We always want to put our best foot forward. And we don't really think about failure as something that is just part of our everyday life, right? I mean, you know. But if I don't floss my teeth in the morning, that's probably a failure because that's gonna long term impact my teeth.
And so we have to kind of be in the mindset about failing fast. And what does that mean when you think about rapid product development? Okay. So how many of you here are product owners? Please raise your hand. And how many of you are engineers? And designers?
It's fantastic. This is me. I started as a chief product officer of rental cars in October twenty sixteen. This is my dog, Scotch, who's the chief product officer of my household. His major criteria is to ensure the happiness of my household. He's much more successful as a product owner than I am. Okay.
I grew up in India. I arrived in the USA twenty years ago. Did a whole bunch of work there. Got two master's degrees, one from Wharton and one from Michigan Tech. I'm basically a geek at heart. And I then went to Amazon and worked there as an engineer for a long time.
I also worked for a number of years at startups at Amazon in San Francisco. Have co founded three startups. I think like, know, probably failed in all of them. So ten years at Amazon doing product work and started with, as an engineer, kind of helping Amazon move from ten million products to one hundred million products.
Trying to think about how to get sellers on our platform from one million to fifty million. Trying to think about how do we extend our customer base to get to hundreds of millions of users. Trying to figure out, as Amazon Web Services was going to be built, how can we do our own dog fooding exercise and kind of learn from first principles how Amazon Web Services worked.
And then last four years, I've been doing a lot of work on Kindle and Kindle devices in terms of really trying to figure out what these device or what do this family of devices mean for our ecosystem, okay? And what do they mean for our customers?
What does engagement of devices mean? What does loyalty mean? What does it mean for Amazon to be within the house? To have a footprint within the house versus a footprint on the laptop or on the desktop, right? So it led to a bunch of proliferation of many, many devices in our household.
But really trying to figure out how do our customers engage with us differently when we are part of a device family. Also worked at a bunch of different startups in San Francisco and Seattle. How many of you heard about Zynga? It was at Zynga for a while, very early days, kind of helping Farm Bill grow from like fifteen million daily active users to thirty million daily active users.
And trying to wonder where are the sheep gone? Oh, do we need another lonely cow in the mechanic? So a lot of work in that space. I've also been the founder of a microfinance solutions company in India, an online discounted fashion shop. And I ran my own product strategy consulting firm for a few years.
A lot of people ask me this question, why rental cars? Rental cars because I didn't know what I liked more, rainy Seattle or rainy Manchester. And at some point, was like, okay, if it's gonna be rainy, might as well switch. But more importantly, the price line group that rental cars if if managed by and is acquired by, is the third largest online retailer in the world.
It's assisted to Booking dot com, which a lot of you have heard about. It's a very product centric company. It's very, open to change. We're continuously iterating and trying and failing and learning from it. It's truly global, fifty ks different locations, nine hundred suppliers in one hundred sixty three different countries.
And it has fifteen hundred employees and that's a cool place for me to be. That's a sweet spot in terms of like I can still remember people's names and feel like I'm contributing to helping cross the proverbial chasm. To get us from like a two billion pounds to a five billion pound company.
What is product management? Product management, in my opinion, is four basic things that every product owner should kind of learn, internalize, and manage. It's data centricity. Are you able to play around with the numbers? Vision and delivery. Are you able to take different business stakeholder options or create a vision out of nothing?
And then more importantly, are you able to deliver on that vision? If you're great at stakeholder management, but if you cannot deliver anything, that doesn't work, right? Nobody's gonna look at you if you cannot deliver. Are you obsessed with the customers? Are you really getting into the customer mindset and really figuring out what is it that the customer wants, needs, and what is this patch of the blind spot in the customer's life that we cannot, that we have to tap into to figure out what customer
delight actually means? And then the fourth one is an open approach to failure. This is super critical, right? Because even if you have one, two and three but you don't have four, you're probably not going to learn from it. Super critical, and how do you instill this lack of fear to failure and make it a daily practice?
Is everyone aware of this model? Right? As a product owner, is what you breathe. You build a hypothesis, then you experiment with the hypothesis, you evaluate, and then you use that evaluation period to take insights and then build your next hypothesis. And I had a hypothesis today that my talk this morning was excellent.
Okay? That was my hypothesis. I went ahead and gave this thirty minute talk, but at the back of my head I'm like, oh my god, you're rocking it. Okay? Then I asked one of my peers from rental cars and another lady who kindly stood up and asked a few questions.
And he's like, your talk was all fluffy, there was no data. You need to go deeper. And so then, there my hypothesis was totally lost because it's no longer an excellent talk because my audience didn't have any data to kind of capitalize on.
I mean, and shallow with fluffy words. Didn't kind of like give them a chance to reflect through my personal examples. The insight from that is, I can't let that happen in this talk, right? Because if I don't take that insight and I rebuild my hypothesis, then I come in and say, this is a fantastic talk.
And so it's really kind of important that most people don't realize this insight phase and they don't internalize it, right? And if that evaluation and insight phase is super critical and you really don't have to look far for it. Data is always coming to you, right?
So you don't really have to kind of like do tons of market research and do a bunch of customer intent focus groups. Data is always coming to you at all points to help you with your evaluation and insights. So how do we deal with failure?
And most of us do really poorly with failure, right? When I was nine years old, my parents sent me for this public speaking competition and somehow I got picked amongst the top five people from a group of one thousand kids to be able to go and speak to the city of, you know, have a speech in front of the city, in front of the mayor of the city.
So I go in there, I'm like the fourth person in line, and my mom's sitting somewhere out there. And, you know, your eyes kind of wander because you're trying to make eye contact. And when I look at my mom, she starts giggling, but she also hides her face.
And what happens if you're a nine year old kid and you see your mom giggling? You start giggling too. So there I'm on stage trying to giggle twelve minutes into my thirty minute speech. And because my mom was giggling, I forgot my words.
Right? And because I forgot my words, I made a fool of myself in front of the mayor of the city. Now, I could apply that to myself and I could say, I'm never gonna take my mom again to any further speech. Or I could train myself that if someone starts laughing, to not let that affect me.
This process of continuously evaluating and applying it to yourself needs to be a part of a daily regime. If you do that, you become comfortable with failure. Does that make sense? If you don't do that, then you're uncomfortable with failure, right? Because you're like, either you're holding yourself to some crazy standards or you're looking at public perception.
But if you just think about this as a continuous learning mechanism that you have to apply in your personal life, there is no judgment. There is just this continuous evaluation phase of what could have gone better. But never ever do this with your partner, right?
You have a partner and you're like, oh wow, if he or she was this way, things would be a lot better. That's like no go. So you can do it to yourself, but you can never do it to your partner. So let's talk a little bit about what failure means in this iteration, innovation and disruption spectrum, right?
And iteration is like you're trying to look at incremental improvements to a current process. Innovation is you're trying to figure out a new way to look at the same process. And disruption is you're trying to figure out a whole new process for what you look at things.
When you think of back to the slide again, when you think of this slide, what we have to lose spectrum is not a lot, right? Because it's incremental. For example, if you're getting one thousand bookings per day and you run some tests that will give you two or three additional BPD bookings per day.
And if you fail, it's perfectly reasonable. On iteration, the scope of failure or your risk tolerance to failure is really low. But then I was thinking of innovation, right? When you think of innovation, your risk tolerance suddenly becomes a little harder if you put in changes that changes your thousand bookings per day to two thousand bookings per day.
Now you're like more averse to risk because the return of investment seems a lot harder, right? All of this works really well when you think of the disruption spectrum as well. Now, thousand bookings per day and you're trying to get to ten thousand bookings per day.
Now you're gonna be even more risk averse, right? Because the return on investment is higher. So just know that the outcome of a discussion of a thing sometimes changes your internal methodologies of the risk profile that you have. All of this seems to work really well besides the gambling spectrum, right?
Because in the gambling spectrum, don't mind putting ten pounds and ten pounds and ten pounds and ten pounds and hoping for a million pounds and change. But this is something that you as product owners or as co founders of companies have to always think about.
That is the return of investment so high that the risk of failure is making me think biopically? And if you don't think that way, you're probably making the wrong decisions in terms of your failure making criteria. So at Amazon we follow something that is called as the working backwards process.
Does anyone know what that is? Okay, so the working backwards process is really the hypothesis process. And we do a lot of deep learning during the working backwards process. And sometimes we'll say, okay, we wanna go to China, or we want to go to India, or we want to launch a new device, okay?
Really good example. I want to launch a new device that is voice activated. Or I want to launch a new device that is activated just through blinking my eyes. Or I want to launch a new device that is sense activated, right? Sensory, through touch.
And we do this deep analysis and that analysis can go anywhere from like two weeks to nine months. Where we're really trying to internalize and work backwards from what we want from the customer. What is the ideal output that we want from the customer?
And through this practice of working backwards, we inherently figure out our appetite for risk, but also get a deeper understanding of the customer. This by itself is not sufficient, right? If you go walk in a silo and say, okay, this is what it takes for us to get into the Chinese market.
You worked for six months, you've come up with a document of how to go into the China market. You go into the Chinese market and you realize boom, everything's different, right? So your strategy doesn't necessarily define the way you kind of think about your hypothesis intent driven approach.
You have to keep tweaking that to get the best out of it. Here are two common practices that we've that we use even at rental cars. Preotyping versus prototyping. And prototyping is a new concept developed by this really cool leader at Google called Alberto Savoyer.
And he's saying, preotyping is basically prototyping with skin in the game, right? And so what can you do as a product owner or as an engineer or as a co founder that shows skin in the game that I am committed to this, right?
And then a really good example that he gives is for Elon Musk, when he launched his first model of Tesla, that he made sure that everyone got a deposit. Everyone paid one thousand pound deposit even before the car was made, right? And that shows that the customer is showing skin in the game, right?
For us within rental cars, if we are trying to reduce the spectrum of how painful it is to pick up a car, just testing the intent of how painful it is, but also how much customers are willing to pay for that effort to reduce that pickup time is really critical from a pretotyping perspective.
So we kind of deal with both of these. Predotyping seems to take a while for product owners to kind of establish. But once they do, they feel they have skin in the game. And when they have skin in the game, they feel more committed to the outcome of a decision, okay?
Very different than building a prototype. The second concept that we've kind of used is pre mortems versus post mortems. Does everyone know what a post mortem is? Okay, so that post mortem is, you failed. Now let's analyze why you failed, right? Or you succeeded.
Let's analyze what happened and why you succeeded, and let's use that. A pre mortem is an important concept that we've used a couple of times, and I'm gonna give you a few examples on that. A premortem is an assumption of a point in time in future.
So assume I'm launching a voice activated device. A premortem is a discussion that the device launches two years from now and is a disaster. Why is it a disaster and what steps we could have taken to prevent that disaster is really the pre mortem analysis.
So two examples of what we did within Amazon that kind of helped out was we wanted Kindle to be a very social engaged effort. And Amazon doesn't have a lot of foray into social. And so we built what we thought was the product vision for Kindle devices to go social.
And then we ran a pre mortem exercise. And in the pre mortem you bring a bunch of different people with different viewpoints and say, give me all the reasons why this particular product will not succeed. And you say, okay, we launched social within Kindle devices and it's going to fail.
And it's going to fail, and the one recurring theme that we came back over and over and over again is it's going to fail because we do not have the network effects. Amazon inherently isn't what we would consider social product company. And so to be able to build that social product within the transactional context of Amazon is going to be super, super hard.
That reflection along with other things led us to actually make an acquisition of a company that was already inherently social. And so we bought Goodreads and Goodreads is a social community around reading. So sometimes you can make build or buy decisions by running a pre mortem.
We ran a pre mortem just a couple of weeks ago in rental cars. Rental cars has a solution called Rideways. And Rideways is a chauffeur driven service that brings you from point A to point B, okay? So if you land in Amsterdam and you want to go to your hotel and you don't wanna have to deal with trying to figure out where your Uber is or your taxi driver is or trying to figure out whether they'll communicate in English, that Rightways is a good solution for you.
What Rightways does is they'll have a chauffeur or a concierge wait for you right outside your gate or in a common place saying, hey, here's your name, Supriya, you go find that person and they take you wherever you need to be taken. And that works really well for travelers who don't want to engage in a foreign environment.
Also for business travelers that don't necessarily want to wait in a queue, right? So we ran a pre mortem on Rideways and we are like, okay, Rideways is at scale right now. It's doing well. We've had this product on for two years. We ran the pre mortem to say what will prevent rideways to become one hundred exit size, okay?
And we said, rideways is one hundred exit size in two years, but it's a disaster. What are the reasons why it's a disaster? And we came up with the recurrent themes about why it's a disaster. And the common one was that we couldn't figure out how scale worked from a supplier standpoint, right?
How do we get suppliers to come in? And so now, the managing director of right ways is making that his number one focus. And so pre mortems kind of opens up your world into what is the elephant in the room. But not just the elephant, but also say, looks like the legs of the elephant is what everyone is worried about.
So let's try to focus on those two or three things. So it kind of like is a risk mitigation strategy to help you deal with failure. I think the language of failure comes down to two things, risk tolerance versus regret minimization. And risk tolerance is your propensity for risk, right?
How risky are you? If you're a single individual who's starting a company, founding a company, and you saved up enough money, your risk tolerance is like quite high, okay? That risk tolerance changes now if you have a family and someone to feed. That risk tolerance changes if you have like twenty employees.
So your risk tolerance changes based on your life stage, but also your propensity for risk, right? That propensity for risk can be adjusted if you increase the horizon of your thinking. Okay, so the propensity for risking, should I take this risk on this particular venture?
Immediately if the answer is no, and you're like, no, I don't want to take the risk. Ask yourself the question, what is the time horizon for which you're looking at the return of investment? And the time horizon of your return of investment is a year or two years.
That kind of changes your risk portfolio. So if that risk portfolio or the risk, the time horizon becomes seven years or ten years, you're much more likely to take a risk, right? And so just know that that is human behavior, right? If you're looking for a return of investment really, really quickly, and when I say quickly, just could be three months, six months, a year.
Then your risk tolerance changes. The second one is regret minimization. This is something that Jeff Bezos talks about frequently. When asked why he started Amazon dot com, he said he tried to apply the regret minimization framework. And it's basically asking the question, are you going to regret not taking this decision?
Okay, so for me, when I decided I was going to take me and my family away from Seattle and move to Manchester, I thought about, do I really want to do that from a risk tolerance perspective? I mean like in this cushy job in Seattle, in this really nice house, and like you know, I have all my friends and everything set up.
The answer was no. I don't want to take that risk, okay? But when I thought of it from a regret minimization framework, I was like, well, this is an opportunity for an adventure that I may not have again. Will I regret this not taking this opportunity two years or three years down the line?
Okay, will I regret not coming to the UK, working out here, being able to travel, being closer to my family in Dubai? And the answer was a resounding yes. So if you feel like your risk tolerance is failing you, you do have that regret minimization framework that you can apply to yourself, right?
And so coming back to our basic point is like, what does failing fast mean? Failing fast means really the ability to be able to continuously try things, but knowing fully well that as human being your risk tolerance changes based on various criteria. And so if the answer comes a resounding no, I don't want to take that risk.
Then answer the question, is it an iterative, innovative, or a disruptive risk? Is it a risk based on time horizon? And maybe that's why I'm not willing to take the risk. And even if through all those answers of that risk tolerance portfolio says no, don't take the decision, then look at regret minimization and see whether the choice of an adventure is kind of going to lead you elsewhere.
That's all I had. Can I take any questions? Okay. Where's the microphone gone? I can hear people. Does anybody want to ask any questions? Yes. Yeah. How do you fail fast in enterprise organization, but also truly accept that? Yeah. So the question was how do you fail, how do you try to fail fast in an enterprise organization?
And I think when you mean enterprise, you just mean B2B or devices, something with long life cycles, right? Yeah, some area where we just can't afford to fail. Again, rental cars is a one point six billion pound business that sits under a price line which is a one hundred billion pound business.
So our risk tolerance is a lot higher. But if you look at Amazon dot com, we launched like ten or eleven devices in the last seven years. And the phone didn't do really well, right? And I think the quarter after the phone failed, we had to take like a big couple of hundred million hit on our numbers.
So it's perfectly reasonable. It's okay to fail within the enterprise, but you have to figure out what kind of product management methodologies do you have, right? So if you look at a hedge fund and if you think my product management methodology is I'm gonna have a hundred product ideas all working at the same time, right?
And my bet is that one of those product ideas will give me the next billion, okay? If that happens, then you're perfectly willing to take the risk on ninety nine. Even if the time horizon of building the products is a year or two years, right?
And that's very true with the phone. The time horizon of building a phone is not like two minutes or within a day. But because you're hedged so far, right, across so many different product portfolios that failing in a few of them doesn't give you, you're mitigating the risk.
That's exactly how our hedge fund does it, right? Does a bunch of micro investments hoping that you find the next big thing. What happens if it's your only idea if it fails, right? Because assume that you're trying to build an enterprise product and it's the only idea that you're failing.
You have to figure out what you're hedging against, right? And so before building the product, are you doing a bunch of intent tests? Are you prototyping? Are you pre prototyping? Are you faking it, Right? Are you faking it till you get the right customer intent?
Okay, are you doing it in a way that the customer shows skin in the game even before the product is built? And there's several examples where customers can show skin in the game before a product is built. Like I gave you the example of Elon Musk, right?
Mean, my friends, half of, plenty of my friends had already put that one thousand pounds down to get the Tesla just by looking at a picture, right? But then try to think about this. When you buy something off the website or you buy something a service, you don't really know the value that you're going to get from a service.
So you're willing to put money down either on a transaction or a product or a service without completely knowing what that service will give you. So you're already hedging. You're already taking risk in that case. And so there are plenty of ways that B2B businesses by themselves can understand and manage risk.
It's a good question. I've got a question. Have you find any difference in the culture in the UK in working in product teams versus the culture in US? Yeah. It's very big. It's very risk of us. And it's like not as open to failure.
And so that's one of the things that I'm trying to instrument within rental cars and it's hard, right? But I mean, and that's why you see there's certain pockets or regions within, even within UK that have been more successful from an entrepreneurial mindset than others.
Like Edinburgh is a really good example. Good energy for an entrepreneurial mindset, risk taking, ability to fail. But more importantly, the aptitude of never giving up, right? It's not just risk taking. It's the aptitude to keep trying and trying and trying and have that grit till you find something out.
And so, yeah, so I think that's been the biggest challenge. And I think part of that is how do you applaud failure, right? And how do you applaud people saying no? And so that's something that we work on every single day. What I find at rental cars is that we're fairly good at taking incremental risk, what I said as iterative risk.
But we are a little scared of taking innovative and disruptive risk. But if you don't take innovative and disruptive risk, you're not going to get the hockey stick effect, right? I mean, the hockey stick effect doesn't come from ten percent year over year growth.
The hockey stick effect comes from one thousand percent year over year growth. And so I think now that I've identified that it's within the innovative and disruptive spectrum, that I have to help lead my teams kind of towards that thing and make them comfortable with failure.
I'm struggling with that, yeah. Is that the whole team or is it the product manager's responsibility? The question was, is that the whole team or is it product managers? Rentalcars is going through a transition. It was a founder led company that is now moving into a product centric space.
So we have like thirty two product teams. We're at fifteen hundred people, like I said earlier on. And when you have to introduce that risk taking behavior, you find some local heroes, which you can tend to do within product. But then you want to perpetrate it as a cultural concept.
Because you don't want one part of your organization willing to take risks and everyone else being risk averse, right? That never ever works. So you have to kind of build it into the DNA, which means it doesn't happen really quickly. You just can't bring a leader in and say, tomorrow onwards you're going to take risk or you're going to fail a bunch of times.
It takes like two to three years to kind of build that as part of your habits and practices. But you start by finding one individual who shows the temperament and the mannerisms. And then you applaud that individual for every chance you get. So you use the local hero thing to kind of get people to be comfortable around it.
Yeah. So it's like a cultural shift in the org. Brilliant. Any other questions? Excellent. Thank you. Thanks, guys.