Marty Cagan: Customers Are Not The Source Of Innovation

Marty Cagan has worked with some of the biggest tech product companies in the world. He was involved in the early stages of Netscape and eBay, and has since formed SVPG, who work with tech product teams to help them make the most of their products. With SVPG, Marty has worked with teams at Google, Apple, Netflix, Airbnb, Disney, and Amazon – to name just a few household names.

In this talk, Marty hammers home the point that customers are not the source of innovation – it must come from inside your company. Prime and the iPhone would not exist if Amazon and Apple listened to their customers, he argues. Using examples from some of the biggest product companies in the world, Marty shows how using engineers in Product Discovery can lead to innovation in a way that a focus group never could.

Find Marty’s talk slides, video, transcript, and more from Marty below.

Find out more about BoS

Get details about our next conference, subscribe to our newsletter, and watch more of the great BoS Talks you hear so much about.

---

Video

Slides

Transcript

All right. Thanks, everybody for coming.

So it’s a nice sized group, actually, we can do this. Keep it informal and hopefully really interactive. Product is a massive topic. There are so many different sort of topics within the topic. And I kind of picked one that I’ve been kind of interested in lately, I’m always sort of focusing in on different issues or problems I see. And so this is going to drill in on one, it’ll also let me talk about a few others. But I’m gonna save plenty of room afterwards for just open q&a. I love the questions. Let me just ask though, how many of you would call yourself either a product manager or product owner? Okay, almost all of you. How many would call yourself a startup co founder? Okay, and how many would put yourself in the design category? Interaction designer, UX designer. And any engineers. Anybody else? Well, that’s my crowd, for sure. I really, basically just work with product teams, which is Product Design Engineering. And when they’re startups, the product is usually a startup CEO or co founder. So it’s a group I like to talk with and spend my time with.

So I don’t know how many of you know me, but I have worked in this space for a very long time. Started 35 years ago, actually, as a developer for the first 10 years of my career at HP Labs, actually, Judy Kibbutz is in the audience. I don’t know how many of you know her, but you should all know her. She should be the speaker for you, Judy, and I go way back. You were at Microsoft early, right after HP. But I think we first met when I was at HP and newer when HP Labs had a research lab here in Bristol. And so anyway, Judy is on the board of several companies. She just finished up with the Guardian, and is really one of the people that have brought digital here in a serious way. So after about 10 years of engineering, I got very lucky the internet started, I got a chance to work for Marc Andreessen. He was the co founder of Netscape and Netscape was really the birth of the Internet. It’s a while ago now. But we just recently had the 20 year anniversary. It’s hard to believe, but the internet changed everything really, it really did. But I was right at a great spot. I was the head of platform and tools and we put together. Marc Andreessen’s view was that the platform up until the internet was really Microsoft Windows was really the platform that everybody built on, client and server.

And then with the internet, that all changed. And we had to actually figure out what that really meant. We knew it changed. We knew we were going from 1000s of users have a big application to millions. And that was a huge change. But we had to figure out how do people build the kinds of multimillion user applications which should run in the browser? Or what should run on a device? What should run in the server? And anyway, we had all these different teams at Netscape and actually beyond as well, in that sort of broader ecosystem. Like we got, we’ve built things like SSL, we built JavaScript. But Macromedia, if you remember them, they built flash, which was really an integral part to the early internet. We sort of put these all together and I got to work with teams, not just at Netscape, but across the industry, as some of my first trips to like Beatty, showing them how they could use the internet, and of course, many many startups. Anyway, after about six years, we were acquired, we actually lost the browser wars to Microsoft, and we were acquired and then I ended up joining eBay because one of the developers URL in the Developer Program at Netscape plus this amazing co founder named Pierre Omidyar. And I just really liked this guy. And he had already really got traction. He just had developers at eBay then. But I was brought on to build a product and design organizations there. And anyway, eBay was great. And afterwards, I just wanted to work with mostly startups. Silicon Valley product group is only four people. And we just have all sort of been there, done that for a while. And we like working with companies. And now the companies are, you know, early on it was Google and Amazon and Yahoo, and a few years later, Facebook and Twitter, but that now, of course, there’s many, many companies all over and they’re there all over the world.

The other difference, there really isn’t much difference when I’m in London versus San Francisco anymore, there used to be a huge difference, not anymore. I see very little. Some of the best teams in the world are in Shanghai, they’re in Berlin, they’re in London, they’re in Sao Paulo, they’re just all over. So I want to be careful about that too, because I’m gonna pick on. So what I’ve learned is that I don’t see much difference anymore based on where I am in the world. But I see a huge difference anywhere in the world, including San Francisco, between great teams, and most teams. And this just drives me nuts. I continue to see it all over, I’ll go into a company. And even though even though you can often see a Google office through the window, I was at The Guardian. And now the Google office is literally through the window. But companies are often working 10 to 20 years behind the way good companies are working. And so this really bothers me. And today, I wanted to kind of pick on one of those topics. The real point I wanted to focus on here is that in many companies all over the world, I find that the way they think of their engineers, is the root of their lack of innovation. And so that’s what I wanted to focus on today.

The problem with focus groups.

This is Bezos. For those that don’t know, the CEO of Amazon, Amazon is probably the most consistently, truly innovative company in the world, at least in my opinion. There are some other great companies and some of the great leaders but that that company is amazing. Every year he publishes a shareholder letter, he just published the one for this year, you should all read it. Last year’s one was epic. And last year’s one had this line in it, I love this line. He always seems to repeat this refrain. Steve Jobs did too, by the way, he would constantly in fact, I remember him holding up his iPhone and saying, Look, you can conduct 100 focus groups, you will never get an iPhone. He was trying to say the same thing. And it’s true. No customer asked for Prime. Did you see how many people are using prime? I mean, it’s just amazing. Yeah, and these are in the lifetime value of a Prime member versus a non Prime member, it’s just good to see where the money’s coming in. But this is really true, that the customers are not the source of innovation. And yet I see so many companies that either they think it is or the way they’ve set up how they work is like it is and it’s really not now I want to be careful. This is not because customers are any sense dumb or anything. It’s not like that. It’s that there’s two fundamental reasons. The first reason is that our customers don’t know what’s technically possible. This is really what’s different about tech products. If you were doing a new brand of toothpaste, it probably doesn’t matter. You could probably do focus groups and decide like what flavor is going to be good or something. But for tech products, it is not that way. It it is all based on what’s just now possible. And our customers can’t be expected to know that. And the second dynamic with tech products is that it’s really hard for our customers. I’ll go further, it’s really hard for us as well, to know what we want, or what they want until after we show it to them. There’s this chicken and egg thing. In fact, speaking of Apple of all the companies I’ve ever been in, they use more prototypes than anyone I’ve ever see. And it’s because they know this, they totally get this. They think they know what they want. They build a prototype and now they know all the reasons why that won’t work. And then they iterate and they iterate. What Steve Jobs was great at was quickly zeroing in on why this prototype was bad. And his team will get back to work. But when eventually they came to him with a prototype that was good, there’s a good chance they had a multibillion dollar product coming. All right. So this is sort of what’s underneath.

The role of engineers.

Now, what I wanted to talk about is the role of engineers. I’ve, made this point in a lot of my writing and talks, but this idea that if you’re just using your engineers to code, you’re only getting about half their value. So many companies, I mean, they just view their engineers as, not necessarily, literally in the basement. But they just as well could be. John Doerr likes to phrase it as we need teams of missionaries, not teams of mercenaries. But that’s the way so many companies are treating their engineers as mercenaries. They’re saying just shut up and build it. And that’s the root of the issue. Now, what I wanted to do was, first of all, I want to try to convince you that none of the great products out there were built that way. That’s the first thing I want to do. I don’t want you to take my word for it, I want you to see it. And then I want to try to talk to you about like, what do you need to do to get this kind of result out of your engineers? That’s really the heart of this, how do we do it?

Amazon’s Alexa

So let’s first start, I just have a set of examples, I’d like to tell you about the backstory on these. It’s an advantage to know lots of teams you know, a lot of backstories and the backstories are always fascinating. But I will tell you, you appreciate the role of the engineers. I mean, Alexa is off to an amazing start, most people would be thrilled. But what they don’t know is that Alexa was not a grand strategy from Amazon at all. In fact, it started at just a very humble beginnings with just a little echo device. All they wanted was a voice controlled speaker. A lot of people offer a voice control speaker, it’s all they wanted, and fine. But one of the developers on the team was sitting actually right near another team working on a smart version of the TV streaming device. And he just realized, why can’t I just make a prototype that instead of just controlling the speaker, controlled, whatever I want to hook up to it. And he just hacked that together as a prototype, and then showed it around. And it was one of those defining moments for the team, they realized that what he had come up with was way bigger than a voice controlled speaker. In fact, they realized what this really should be is the hub of your home. If you follow Alexa, you now know that now that Amazon believes that also should be the hub of your work your workplace. I mean, they have huge ambitions for Alexa. It’s still just in the infant phase, it’s just getting going. But no grand strategy no grand plan, just engineers playing, creating a prototype realizing something is just now possible. And solving a pretty real need latent need.

Disney’s Magic Bands.

Most people will credit the Apples and the Googles and Amazons as a great tech company, but they don’t really think of older companies that way. And Disney is kind of amazing. Disney is definitely an old company. Disney Parks has been going more than 50 years. And I don’t know how many, I know there’s one outside of Paris or something. But if you’ve ever been to a Disney park, they are their special amusement park. They’re not a typical amusement park. But anyway, they have a whole division called the Imagineering division, which is really where they do the discovery on their rides, their amusements, but separate from that you need to understand, in their business model, the parks are actually the core of their ecosystem. There’s toys, there’s movies, there’s games, there’s all this whole thing, but the park experience kind of gets kids hooked as kids and it’s very powerful. Anyway, the truth is they were a victim of their own success. About seven or eight years ago, they started to see their numbers drop. Now, the numbers were showing up pretty clearly in terms of how much money was being spent on a visit and how often people were coming back. It wasn’t really a big mystery. It got too crowded. The parks were too crowded, people were waiting in line forever. It’s hot as hell in Florida where the Disneyworld is, it’s like: this isn’t fun anymore. And Disney was really becoming a lot less fun. And they knew that they, certainly weren’t willing to give up on this. They could have raised prices more. But if you’ve seen the price is already huge. Instead, it’s like, they want to create some elitist thing. And so they put a team together of some very impressive product design, engineering, they put them together and said: look, it’s time, we need to reinvent the experience. And the charter was basically: make it great again, and not make the country great. I could never say that anymore. They had red hats. Well, don’t get me started. Anyway, they wanted to make that experience like it was. And I don’t want to oversimplify, it was a major effort. They invested over a billion dollars into this by the way they to really invest, they created a prototyping lab. So they’re not just prototyping your app on your phone, which they did. But they also prototyped lines, rides, like when you wait in a line for a ride, you see things and they wanted these things to be relevant to the kids age, the gender, the interest, what are your favorite characters. And so they simulated all this in a prototyping environment, called it a discovery lab. And they were able to reinvent the experience, they ended up creating something called a magic band. A simple RFID device that was that every kid was actually mailed to you at home, you’d buy your tickets online, you describe your family mailed to you the band’s. If any of you’ve done it, you’ve seen this. But now when you go in there, there is no line, there is no line, you walk right through the turnstiles that recognizes your devices and welcomes you. When you want to have lunch, you want your kid to go buy some pizza, they don’t have to ever touch money at all goes through the device, when they want to. They plan their rides in their day, they learned some when to show up where and they don’t. It’s just it got so much better. And you know, that’s if you go, you should if you’re there, you should check it out. But you can see how it’s the foundation for a whole future experiences as well. It’s really a new platform for entertainment. And yeah, the customers again, customers couldn’t tell. I mean, the customers could tell them why they didn’t like it so much anymore. They couldn’t tell them what to do. That’s up to Disney to go do.

Google Translate.

Similarly, Google Translate. Everyone talks shit about translate. But it really is an amazing service. There’s a half a billion people that use Google Translate every month. You know, if it wasn’t Google, anybody else would consider that massive. But, you know, translate is an interesting one. It has been there for over a decade now. Translates been there. And by the way, to be honest, the teams have been working hard on google translate for that entire period. But for really up until last year. Most people would tell you Google Translate is kind of like dank bullets here. It’s better than nothing, but it’s not very good. I get an email in Portuguese, so I can go at least figure out what they’re saying. What they’re asking. So it’s better than nothing but not very good. And by the way, there’s ways to quantify this too. In language translation, they have techniques to actually quantify how much better and after a decade of work, it gotten a little bit better, but not enough for most us to perceive the difference. But last year, they actually realized there was a new technology that could maybe help. And they applied it here in Google Translate. I thought this was very interesting. They rebuilt the back end of Google Translate but didn’t change a pixel of the user experience. You all know what it is. It’s super simple. You put input and there’s the translated. You just pick your language if it can’t detect it. Simple. They didn’t change a pixel of that experience but they changed the entire back end. And they wanted to see if people would notice. So I love this. And by the way, they did in a big way. People were shocked. They were skeptical, like how is this happening? There was a poet in Japan that actually was like this can’t be happening. And it was like, he showed the original translations that used to take me months of work to get it to this stage, what’s going on? What they had done is applied machine learning technology to this problem. And they believe that if they really did a better job at this, then the users would recognize it, which they did. And they made more progress. Literally, quantitatively, we could measure but, they made more progress in one year than the entire I think it was 11 years beforehand. And that was purely by applying an enabling technology, letting the engineers try out this new technology was one of the tests that Google did to help them decide that. I don’t know how many of you’ve heard this. But for years, they have made a big, vocal statement about being a mobile first company, they wanted to really tackle mobile, which is not done, of course, there’s still lots of work. But recently, they made a new announcement. Now they’re a machine learning first company, they are asking every one of their product teams to go see if this technology can really fundamentally improve how they do their jobs, how they actually solve the problems they’re trying to solve. And this is just one of many examples. Actually, there’s some other great stuff that just rolled out actually, on YouTube, you might have noticed, if you browse your videos.

Apple’s iPhone.

Everybody knows the iPhone, it’s 10 years old now. But what a lot of people don’t realize is the iPhone itself was three and a half year effort for the first iPhone, three and a half years of work. And of course, these are devices, this is hard. And what’s amazing to me, and we already kind of know what Apple’s Mo, if you will, it’s not really a secret. But what is amazing to me is at the very time Apple was I mean, literally during the time Apple was discovering, and the solution, which was the first iPhone, the main competitors at the time, well, BlackBerry, Motorola, Nokia. And I actually know people at all four of those areas. And what’s amazing is literally, these companies were conducting focus groups for their next devices, smartphones, if you remember well, at the actual time the iPhone was being designed that the BlackBerry kind of was the leader. And then the trio was kind of like a super Blackberry. If you remember the trio, the trio actually had a keyboard like a Blackberry. But unlike a Blackberry, the trio actually had a touchscreen. And people don’t remember that. But it did. It was a crappy touchscreen. It was really bad. You could barely dial a phone number. But it had a touchscreen. Literally, at these focus groups, the main piece of feedback they got from people was, they were like: well, this touchscreen sucks, lose the touchscreen. That was the main feedback consistently those other companies got. And of course, Apple knows enough not to even ask, or they don’t ask their customers. Do you like a touchscreen? Yeah, the palm TREO was terrible. But you know what, the technology wasn’t right. And it wasn’t ready. But now we think we can. And in fact, not only did the customers not know what could be done with the touchscreen technology. But in truth, I think most of those competitors didn’t realize it either. Anyway, that’s the story.

Workiva.

It’s the engineers and applying technology that is just now possible to solve real customer needs. That’s where these come from. I wanted to share one other example that wasn’t as big, everybody knows consumer devices. I don’t want you to think you have to be an apple or a Google or an Amazon to do something like this. This company is called Workiva. I’d be surprised if anybody here has heard of them. But they are one of the fastest growing companies in the US. They went public after just a few years, they grew from nothing to something like 85% of the Fortune 1000 running their products and just completely disrupted their space. You wouldn’t have heard of them because they basically provide the automated software that helps finance accounting people handle all of the reporting that they do every quarter and every year when you’re a public company. It’s incredibly boring, detailed financial, reporting and accounting. However, these people, the co founders had actually done a company before that in engineering automation space. And they had a friend actually whose company was going public. And the friend was showing them all the unbelievably tedious months and months of work that had to be done to prepare to go public. And, and they were looking at this saying this is just crazy. And the poor people and because they would, you know, they were talking to these finance and accounting people, they often weren’t getting home to their kids. I mean, during the period of month and quarter and year end, many of you have heard this from the accounting people in your own companies. It’s just a thankless, brutal job. And they realize, this is crazy, we can totally reinvent the space, the software that’s there today is terrible, which it was terrible. And they just completely went in a bunch of, really, honestly, six engineers got together now they, they really sat down and learn what they needed to learn from the accounting people. But they committed themselves to really making their lives better. They made some very early bets and cloud technology and rich experience technology. And they wrote that just This is, yeah, one of the things I love about this company is all over the walls are genuine letters from their customers, telling them how genuinely grateful they are, that they can see their kids, you know, the end of the month, and not just so anyway, you can absolutely do that. And this is how the great companies work, I’m happy to see there’s a little trend going on. Now we’re in enterprise software, it’s called consumerization of the enterprise. It’s like applying consumer caliber products to enterprise. To me that’s long overdue, because many of us at work, we have great software at home and terrible software to use at work. Crazy.

All right, so I could honestly keep going. I know so many of the backstories of great technology. In my book, actually, I wrote a bunch more there. There’s some great stories of great initial products, the original Netflix, the original Google AdWords, lots of them. But you’ll see that theme. It’s about unleashing the power of the technology and combining that with a real customer need. What I want to talk about is if you believe or agree that you really need to get engineers to participate in not just building your products, but also inventing your products. Then I wanted to talk about how we make that happen. Because it won’t happen on its own. It won’t happen on its own. And so especially what happens if you treat them the old way where you have your CEO creates roadmaps and throws them over a wall to a bunch of engineers and say build these features.

Don’t Outsource your Engineers.

All right. So there is this notion of table stakes. I only put this in here because I continue to meet companies that think they can actually be a true tech company, yet outsource their engineers to India. So I don’t want to embarrass anybody here. But if any of you are in that boat, where you literally don’t have engineers with you, to be clear, I don’t care if you’re in India, and you have engineers in India. But if you’re here in London, and all your engineers are contractors from Infosys, or something in India, or China, you’ve got almost no chance, just to be honest with you. First of all, you’re going to spend, you’re gonna waste way too much money, and you probably are doing this because you think you’re going to save money, which is the big myth. In this, you’re not going to save money, you’re gonna waste money, it’s gonna take a much bigger team, it’s gonna go way slower, and you’re gonna get no innovation. So table stakes is that you have a team. So if you’re located in Manchester, you’ve got a team in Manchester with you. In fact, if you’re a product manager, and you don’t have at least two engineers to work with on an ongoing basis, you’re not a product manager, You’re something else. It’s some other job. So because we’re there to actually create products and the engineers will help us create those products so. So table stakes is that you have a team of engineers and we want to hire engineers that are passionate about our vision. That’s why the product vision is so important. This talk isn’t about product vision, but I will just point out that our number one recruiting tool is division. That’s what good engineers, that’s what good designers, that’s what good product managers they want to join to work on a great vision. If you wonder why does Tesla share their vision with the world to 10 year vision, they just shared the second 10 year vision. This is their best recruiting tool. So they don’t care if the world knows vision. Now the details of how we solve that, that’s the hard stuff. And that’s not the stuff you’ll find on a website. But the vision is powerful. And, anyway, we want a team of engineers that’s passionate about the vision, that’s table stakes. So without that you’re really not a contender, you just don’t have a chance. Any company that I work with, if they don’t have that, I’m like: Call me when you actually hired a team, because you got nothing to do right now.

Share Context with your Engineers.

All right. So assuming you have that, there are a set of things that I want to encourage. The first one is you have to provide your engineers with the full business context, everything, they need the information. And to be clear, really, everything I’m going to say applies to the designers on the team too. It’s just that the designers more often get access to this product, the engineers often don’t. But the engineers need to know, they need to know the KPIs, they need to know the metrics that matter. They need to know the customer pain, they need to know the sales limitations, they need to know contractual obligations, they need this stuff. They’re the ones it’s amazing, because a lot of product, people think it’s their job, a lot of CEOs think they need to shield or shelter the engineers from this information. But I tell them, This is the worst thing you can do. These are the people that are going to actually save you. So don’t shelter it, they can handle it. This is actually very motivating to good engineers, they’re very motivated by problems to solve hard problems to solve. And that’s what this is. So all the context you can. Now some of the context is pretty obvious. We talk about like requirements, there are some things that are pretty obvious. Things like how performance constraints or scalability estimates, but most things are not as obvious. I was just talking to somebody yesterday, it’s doing another one of those bike sharing systems, right? You’ve seen them all over the place. In fact, in San Francisco, it’s become such a problem. Because of course, the cool thing is bikes, you don’t need to park them anymore, right? They have GPS, isn’t that nice? Problem is people leave them anywhere. And they do. So now they’re littering all over the place. And people are complaining and they’re being impounded. And it’s like, you know what product has to worry about this, we have to worry about compliance about laws about privacy, we’re in the middle of a major privacy shift. Right now product has to worry about privacy constraints and policies. And we have to worry about legal issues, we have to worry about, go to market limitations, our sales force, our marketing channel, product managers have all of these dimensions of the business they need to know so they can share these issues with the team. I’ll get more into that point in a bit. But the critical thing here is you have to share this context with your team. Now the truth is not all your engineers are going to care about all of it, that’s fine. But they need they need to know it. When when they need access to it at the very least from you.

Connect your Engineers with your Customers.

All right. Second, and this is kind of amazing. There’s a psychological thing that happens. But I just call it magic actually. But magic happens when the engineers see the actual users and customers face to face. It’s just, you know, it’s not the same over a video link. It’s not the same in the usability lab with a two way glass stuff. None of that is the same. They need to see it right there, that visceral interaction. And even when by the way, it gets really ugly. A lot of times it does get ugly because the customer is mad. They might be mad at your product. They might be mad at whatever they’re using maybe a competitor’s product, but they’re not happy. Those are the really good ones to have the engineers out because they will get very motivated. And so again, a lot of people don’t want to include engineers of course the most common argument is: how can we take them away for an hour from their coding, right? That’s crazy. We’re not trying to take all six of our engineers to all of our sessions, but we want to take at least one to every session. And again, not all six are going to be crazy about doing this, they don’t need to be. But it’s normal for two or three of them to be very interested in doing this. And so we’ll rotate among them. And if you don’t have at least one engineer on your team that is into this, you need a more senior, serious engineer on the team.

Requirements.

All right. Third, this whole current topic of requirements, we could talk about requirements the fact that they are not talked about like they used to be talked about. But the truth is, most requirements are not really requirements. Somebody in marketing says: we must do this. And, of course, they’re not just making this up, it’s just they think we have to do this, we in product have to go figure out what is the real underlying constraint, maybe what’s going on is they think we have to have this forum because these other sites have it. But what’s really going on is if the user is under 13 years old, we are not allowed to ask certain things. And there’s an underlying constraint that we need to figure out. Because every time we get to those underlying constraints, we give the engineers many more degrees of freedom to solve the problem. The less you can give them in terms of requirements, the better. Mostly, we try to focus on problems to solve if you’ve heard of jobs to be done, and same idea.

Discovery and Delivery.

Okay. In general, for every product team, there’s two things we do, the term I use for it is discovery and delivery. Discovery is when we figure out what to build, like, you might have heard of, build the right product. That’s discovery, figure out the solution. And really, there’s four things we care about, the customer will buy it, or would choose to use it that’s value. The customer can figure out how to use it, that’s usability. We can actually build it, that’s feasibility. And it’s viable for our business, viability. This means it works for the legal constraints, it works within the financial constraints, we can afford it, it works within the go to market constraints. So we’re looking for solutions that do those four things. That’s what discovery is: lean startup techniques. And you mentioned Eric Ries was here. Lean startup is a subset of discovery techniques. There are some great techniques in there. There are many different techniques that contribute to discovery. And we use them all. We have qualitative techniques, quantitative techniques. There’s always one technique. Yeah, I mean, there’s a book out on jobs to be done right now. So a lot of people are talking about, it’s actually an old technique, it’s a good technique, but if that’s all you’re using, you’re in big trouble. Just like design thinking. Design Sprint’s great technique, one of my favorites. But it’s not good for most work, but it’s great for other work. So you should all be skilled, they’re also called discovery Sprint’s you should be skilled in that technique, but just one technique. And we don’t even use it all that much. It’s more like we’ll do one discovery sprint or design sprint per quarter on average, like that. So any of these techniques, you have to be careful, there is no silver bullet in product, we have many techniques and your job is to choose the right one for the job. Now, the other side is delivery work, which is to actually get this, we figured out a product that hopefully we think we have evidence will sell and will work. Now we have to get it to market. That’s delivery. Most companies are much better at delivery than they are at Discovery. Agile is all about delivery. If you have an Agile coach, they’re all about delivery men. In fact, most of the teams that at least that I work with, they’re doing continuous delivery. They’re quite good at this, but discovery is the other half of the problem.

I always frame it as discovery and delivery. The oldest way I’ve seen it framed was build the right product, right. But early Google and still in many parts of Google, they phrase it as fake it before you make it. Fake it is discovery, make it as delivery. Facebook, move fast (discovery), don’t break things (delivery). One of my favorites, Airbnb. I haven’t mentioned them. They’re a great product team, they refer to it as build things that don’t scale that’s discovery, and then build things to do scale. That’s delivery. I like that framing because it really makes people understand that this is a classic. In fact, there’s a backlash going on right now against lean startup. And it really bothers me, because it’s the Lean Startup techniques are great when used appropriately. The problem is, most people don’t use them appropriately, I can’t tell you how many teams I meet that take four months to build an MVP. If you’re taking four months to build an MVP, you are missing the whole point of this, that is what we call product is prototype you have spent. And just to level set you I mean, probably not fair to throw this at you as just a side comment. But in discovery, a competent team product team and discovery is testing on the order of 15 to 30 iterations per week. Okay, 15 to 30 per week, not one performance. Okay. So what’s really going on, when you have one performance is you’re using your engineers to do discovery, which is a really expensive and slow, and I would argue, irresponsible way of working. So that’s a big topic. But that is the book that I guess you all get a copy of is 400 pages. And it mostly talks about how to do that how to do 15 to 30 iterations per week in discovery. And by the way, really good teams can do a lot more than that. It’s not hard to do that, a lot of prototyping, a lot of qualitative techniques, not just quantitative. That’s how we get good stuff done fast. Try out lots of approaches. So anyway, discovery is the product manager, the designer and the engineers. Now, here’s the thing. Discovery is full time really for the product manager and designer. But all we need is about a half an hour a day from your engineers. Now, because most of the day for engineers is doing delivery, that’s their primary job is to build scalable, fault tolerant, reliable software. And so that’s what they spend most of their day doing. But a little bit of time every day to go into figuring out what to build is easily the best investment you can make. Most engineers already insist, they had been complaining that they are not included in this. In fact, I have heard countless times engineers complained to me: if I had just been consulted for even five minutes, we could have saved months of grief. And of course, they didn’t. And I just want to make this real for you. If the first time you’re showing your engineers, what you need them to build is at Sprint Planning, you have screwed up on this. Sprint Planning is way too late. That’s an example of the problem, not what I’m talking about.

Measure Teams Based on Business Results.

Okay, so they need a little time thinking about it, making it better assessing it. The other thing here is that we want to make sure that we measure teams based on the business results. And of course, the challenge is that most of you probably are given roadmaps from your executives or even from yourself. And roadmaps are typically just lists of features and projects. And that output. And I all I can tell you is that most of those things, no matter how smart you are, you’re going to regret doing because they won’t actually work. Most common reasons is that we’re excited, but our customers are not. So they don’t actually use it, or they don’t want to buy it, or they try to use it and they can’t figure out how to use it and then they bail on it for whatever reason, you’ll regret it. And so the question is, you know, that’s all waste. And we want to avoid that. So we want to get some evidence that it’s going to work before that. And so what we’re really measured if we’re a good team is not on whether or not we ship these 25 features this month, but whether or not we actually move the numbers we’re trying to move, like retention rate, or like lifetime value, to customer or engage whatever it is that we’re asked to focus on. That’s how we measure success. And that’s the whole OKR system, if you’ve heard of that, that’s where it came from, was to get the team stop worrying about roadmap, start worrying about business results.

Getting Product Managers Right.

All right. And the last item here is really true. A lot of teams, a lot of engineers complained to me about their product managers, so many of them. That’s, that is the single biggest complaint I get. I mean, many of you may have moved into product from engineering, because you were frustrated with the product managers. I mean, that’s a really common motivation for moving into product. But it is. And this has been another hot button of mine lately, I have a theory about why so many teams feel like they have such a, you know, incompetent is a rough word, but not capable product manager. And my theory is because most product managers, I mean, the only training they’ve actually had is a Certified Scrum Product Owner class cspo. And that, just to be clear, you should have that certification, it’s super easy to get. But that’s like maybe 10% of the job of a product manager. That’s the administrative role. If your team is using Scrum or Kanban, there’s an administrative role that the product manager plays, that’s called the product owner responsibilities. But that is, if that’s what you’re doing, you know, you’re an expensive backlog administrator. And so that is such a problem in our industry. And I try to tell people, no, you know, that’s a small percentage of the job, I started actually going out on a limb, because you really try to get people to understand just how big a difference this is. The product owner responsibilities versus the overall product manager job. And I started advocating the point about a good product manager is truly the CEO of the product. This has been controversial for a long time. But then I went back and forth on it in truth. Of course, it’s controversial, because the product manager is not the boss of anybody, we need to be clear on that; they’re not the boss of anybody. And when they start acting like the boss of, you know, the team, then a lot of things go wrong. So it is not meant to imply that although in a startup where the CEO is the product manager, then they are the boss of everybody. But that’s the startup but that doesn’t scale. But they are like the CEO, because a competent product manager has to understand all these elements of the business, they understand how the product is marketed, how it’s sold, how it’s how the onboarding process goes, how we pay for it, the financials, how much it costs for us to run this product and serve it, how much it cost us to acquire user, how much value we get out of those customers, they understand the legal issues, the compliance issues, they understand GDPR stuff that if that’s going to impact us, they understand contracts with partners, and literally what they’re on the hook for that in the only other role I know of in a company that’s like that as the CEO. So in a very real and meaningful sense. The product manager needs to be the CEO of that product, just not in the boss sets. They’re not an authoritarian role. It’s a knowledge role. And they’re bringing all these things to the team and helping them make a call. So that’s what I mean by a competent product manager. It you know, it takes some real work. I mean, Product Manager is super hard job. I think it’s the hardest job on a team. So no, I don’t apologize for that. It’s a very hard job.

I have this other pet peeve of mine, as many of you have seen that Associate Product Manager title. Now I’m not referring to like Google has an APM program, which is an apprentice product manager. That’s awesome. But this notion of an Associate Product Manager, if it was up to me, that would be gone. We wouldn’t have that notion because no team wants an Associate Product Manager. They want a product manager, somebody who knows this stuff, not somebody that knows 10% of it. And so to me, that’s the table stakes for product manager are pretty significant. It’s a lot of work. But that’s key, and every great team. I know the engineers have contributed in a big way. But they all have a strong product manager. And most of them, once they’ve tasted had had a chance to work with a great product manager, they insist on having somebody like that going forward, because they know now they know what they’re missing.

Question from Audience: Yeah, I have a question. If I may, in your wonderful book, you feature six product managers, from Adobe, and Apple and Microsoft and multiple companies. And the remarkable thing about all six of those there are women. And in a world in which women aren’t very well represented, I think it’s amazing that your six exemplars are all women. And do you want to say anything about that?

Yeah, so I have lots of funny stories that after I gave a talk, actually in London, in mind the product and I prepared this for that talk. And I ended up including it in the book, there’s an article if you want called Behind every great product, and I knew what I wanted to write about, I wanted to write about great product managers. My advantage is I know so many of them. And I put a list together of great product managers. But then I also wanted to pick stories. So because I wanted to tell the backstory, so I wanted them to be iconic products. And I know a lot of great product managers that were just not fortunate enough to work on an iconic product. And I couldn’t help but notice that even though women are not even close to 50% of tech product managers, that they were more than 50% of my list. And I realized that I also wanted to highlight that I did end up I got asked that so much. I did write up my theories about why but I didn’t put this in the book. But if you’re interested, there is an article you can google titled, Why Women make the best product managers. And I do believe there are real reasons why. It’s kind of complicated. I don’t want to not do it justice. But if you’re interested in why I’d encourage you to read that article. That’s just my theories, from a male perspective, obviously, about why women seem to be disproportionately strong in this. And I’ve got many more example. You know, a lot of them, I introduced a lot of great product people to Judy, I want them to know her. And so, yeah, there’s a lot. It’s a great career for women. And by the way, it is the proving ground for startup CEOs. It’s you know, so it’s a great path to getting a lot more women leaders, which I’d love to see. For the same reasons I wrote why women make the best product managers. Okay, good. Well, just about it questions. Anyway, there is the book. And there is my email. You know, I touched on a bunch of topics in there sort of rambled a bit. But are there any questions anybody wanted to ask about?

Mark: I’m just gonna jump in. I want to say firstly, thank you very much indeed, Marty.

Find out more about BoS

Get details about our next conference, subscribe to our newsletter, and watch more of the great BoS Talks you hear so much about.

---

Marty Cagan.

Marty Cagan has worked with some of the biggest tech product companies in the world. He was involved in the early stages of Netscape and eBay, and has since formed SVPG, who work with tech product teams to help them make the most of their products. With SVPG, Marty has worked with teams at Google, Apple, Netflix, Airbnb, Disney, and Amazon – to name just a few household names.

More from Marty.