Matt Lerner: Goals, Metrics and Incentives

Do your teams move in one direction? Single founders struggle to focus on the right things. More people, teams and departments mean exponential complexity.

The truth. A single goal and focusing resources on achieving it is the only way to maximise outcomes.

In this interactive session, Matt takes you through the process of finding, stress-testing and implementing a single North Star Metric to help you align goals, behaviours and incentives across your company. You will learn how to identify your rate-limiting step, the bottleneck where you need to focus to have the biggest impact on your growth. He discusses how to implement your framework across your team so that people are aligned and making good decisions about how to spend time and money.

This session will help you to:

• Simplify prioritization and decision making.
• Focus your team on the most impactful work. Everyone works hard, winners do the right work.
• Develop a metrics-driven culture to reinforce high-impact thinking and decision making.

Slides

Find out more about BoS

Get details about our next conference, subscribe to our newsletter, and watch more of the great BoS Talks you hear so much about.

Transcript

Matt Lerner 

It’s incredible to be here. Mark, thank you for inviting me on this stage.

As for those of you who have never been to BoS before, you’ll see it’s a really, really impressive group of people who will be an attending, will be on stage, will be having conversations for the next few days, and it’s an honor and a privilege to be here.

So I’m going to tell you a story about finding some leverage in a business that is actually a story about my own neuroses, which is actually a story about Stanford University, which is actually a story about the misuse of statistics, which is actually the story about an incredible amount of untapped potential.

On Underutilized Potential in Organizations

Remember, in the old days, people used to sort of say that humans only use 90 or 10% of their brain power, and if we could sort of somehow unlock the other 90% we’d all be super geniuses. So remember that, anyone? Remember then certain famous neuroscientists said that is actually a load of bunk, and there’s no truth to that story. So I think that even though we don’t use only 10% of our brains, I feel like something like that may, in fact, be happening collectively in organizations, that there’s something about the way we run our organizations where we only use like 10% of our collective brain power, and there may be a way to kind of unlock the other 90%.

So let me I’ll quickly introduce myself, and then I’ll kind of get on with the stories. So my name is Matt, Matt Lerner, and I started my career in Silicon Valley long time ago, and I worked for growth stuff for a bunch of startups that you never heard of. One of them got bought for $40 million but you probably never heard of it, Elemental Software, Drum Beat, anyone? One? Alright. Excellent. We’ll talk about that later.

Left the acquiring company, went and joined the PayPal growth team in 2004 and that was a wild ride, and that went really well. And I stayed there for about 11 years in various growth leadership and general management roles, and I moved to the UK, and I became a VC. And as a VC, I mean, that’s a really privileged position, because you get to see so many companies from the inside. And what I saw was this, all these companies making the same preventable mistakes again and again. And that led me to kind of leave the world of VC and start my own company called SYSTM, and I’ve become kind of obsessed with the idea of leverage in business. And I wrote this book called Growth Levers. We have ordered copies for everybody. Hopefully they’ll get here before the event ends, and you can all have them. But I think a lot about how to find leverage in business. And that brings me to the first story.

The PayPal Launch Problem

So early on in my tenure at PayPal, we launched, they created a new product that was basically, this is 2004 or five. It was credit card processing APIs, so something pretty straightforward, but back then it was revolutionary. It was something the world really needed. And so I was in charge of launching it. And I launched it, and lots and lots of people signed up for it. It was $30 a month. 1000s of people signed up right away, but almost none of them ever processed even a single transaction. And my boss came to me and he was like, Matt, no one’s using the products. I’m like, I’m in marketing. That’s not my job, but turns out it was my job. So I was like, just wait. We’ll figure it out. They’ll they’ll start eventually.

Anyways, a few weeks went by and still nobody was using the product, even though they had signed up and they were paying $30 a month for it. So we had some meetings to investigate, and we came up with some really simple fixes, like emailing everyone after they signed up with links to documentation and a phone number they could call if they have questions. Fixed all the obvious stuff. Then we found and fixed a bunch of hard stuff, like releasing a version of the product that worked on web servers other than Apache, integrating with different shopping cart and e-commerce platforms, anyway, a bunch of stuff that was harder, but eventually and by the way, these numbers are completely fictious and made up, but directionally accurate. This is about the where they were.

Anyways, so we eventually just hit a wall. And we kept trying lots and lots of things, and nothing ever worked, and we just couldn’t get it above about 35% and finally, at that point, I did what I should have done all along, which was just to actually go and talk to some customers. And like, that’s the punch line, by the way, of every talk you’re going to hear is like, by the way, you should talk to your customers. It didn’t take very long.

When Talking to Customers Changes Everything

So I flew out to our operations center in Omaha, Nebraska. We started calling people who had signed up and had not yet had any transactions.

The third or fourth call. Very nice lady. Oh yeah, thank you. We love Pay Pal. It’s, it’s up and running on our website. And we look and it’s on our website. I’m curious, why haven’t you had any transactions yet? So we don’t have any customers yet. Fair enough. How long has your business been live? Six months. I see the problem. She doesn’t have a business. Not all of our customers have customers of their own. So I took that nugget home with me that night, and I was thinking about maybe, like distribution of value within our customer base.

Discovering the 10% That Matters

And so I emailed Igor, who was my favorite analyst at PayPal, and I said, Igor, what percentage of our revenue comes from our top 10% most valuable merchants? And he replied, pretty quickly, and he said 102%. Now this isn’t a math error. Igor has an advanced degree in Aeronautical Engineering from Stanford University, literally, a rocket scientist. So there’s supposed to be a quiz. You’re supposed to tell me how is that possible? But the answer is actually up on the slide because of the build.

So the answer is, the bottom decile of sign ups for PayPal accounts are actually going to contain some bad actors, and through fraud, they suck about 4% of the value out of the system in their first couple of months before they are weeks, or however long it takes to shut them down. So the net distribution of value within our customer base was 98%, negative 4% and then the middle 80% only generate 2% of our revenue.

So I realized we don’t actually need to activate 30 or 40 or 50% of our customers. We need to find and activate the right 10%.

High Leverage in Action

So what we did is we developed a program where Igor or his colleagues or whoever built a predictive model and started scoring all new sign ups on their propensity to be high value customers, and anyway, sorry, based on their propensity to potentially be high value customers. And then we took the ones who looked like there were going to be high value customers and we called them and we gave them like a new merchant onboarding experience where we get them through vetting and compliance and get their integration sorted and whatever. So we ran that program for a few months, and then we checked the results, which may be about to appear on this slide.

So we checked the results, and as you can see, the people we called were worth 10x more than the people we didn’t call. Is that good? Is it bad? Does anyone not sure? Why are you not sure? You’re just not sure. Because if I’m asking this question, there must be something.

The cost was negligible, about 5-10, $20 per merchant, but these merchants were worth 1000s per year, so good question. Sorry?

It could be a poor predictive model. Yeah, remember, with this model’s job. Sorry, was there another. Okay, the job of this model was to identify merchants who seemed like they would be high value. So we identified merchants who seemed like they’d be valuable, we called them and then they were valuable. That may be because we called them, or it may be because the model did a good job of finding merchants who are likely to be high value.

So luckily, we didn’t call all it wasn’t luck. We were clever enough not to call all of them. We called half of them, and we ignored half of them in a control group. And what we found, after about six months of cohorts maturing, was the control group was worth about 40% less, or the test group was worth about 40% more. 40% might be a lot, might be a little in some companies. In this company, at this stage, that was an insane amount of money, because we were signing up at that time in Paypal history, about $3 million per week in new annual recurring revenue, and 40% of $3 million of annual recurring revenues, $1.2 million of annual recurring revenue, more per week we were getting because we were making 100 phone calls. So insanely successful leverage program.

Imposter Syndrome & Stanford Lectures

So I should tell you now I had imposter syndrome at PayPal. So when I first got there on my first day, the lady sitting to my left Harvard, undergrad, Stanford MBA. The woman sitting to my right, Stanford, undergrad, Harvard, MBA, and I graduated from a school called Miami University, which is actually in Ohio, for reasons that are beyond the scope of this conversation. But suffice to say, it was not Harvard or Stanford or anything close I already was like.

And then at some point I actually started getting invited up to Palo Alto to give guest lectures, because, like, PayPal became a bit of a thing, and so I started going up there and giving these lectures. And I was always a little bit nervous, because, I mean, I’ve lectured now at a bunch of different business schools and Stanford students, they are smart, but they just have a level of confidence that is unmatched even by their own abilities.

So you know, when you’re up there talking, they’re like, hold on, Mr. Lerner, on slide eight, didn’t you? You know, it’s like, ah, but on the other hand, having smart people looking at being scrutinizing your ideas is, you know, steel sharp and steel, it’s also a good thing. Anyway. So I went up there and I gave my little talk. There’s a picture of me giving a talk. There it is. I gave my little talk. See same slides, different format. I told this exact story, and I got to the end. And, you know, people ask their questions, whatever.

A Smart Question That Stuck

I got to the end, and a woman put her hand up and asked a really, really, really good question that, at the time, kind of threw me.

She said, Matt, it sounds like you discovered concentration in your business. This is kind of a normal thing. You know, in the casino and the gaming industry, they make most of their money from the whales, big, big gamblers. We call them addicts. In the travel industry, they make most of their money from business class, you know, frequent business travelers. How did you guys miss this thing in your business that was so kind of basic and obvious? And I was like, ah, busted, like, but then I just, I taught, you know, something about the fog of war. Everything’s obvious in hindsight, whatever I talked it off, but I take that question with me because it’s a really good question, right?

And I went and asked my friend Mo Sayed, he’s a friend of mine who lives in London, and he said, first of all, Matt, like you calm your ego. He said, You’re the one who figured it out. Like all those people who went to all the Harvards and the Stanfords, they’re not the ones who figured this out. They missed it too. You figured it out, but it is a great question, How did a company full of, frankly, some of the greatest minds, business minds of our generation, miss something so big and so obvious and so important.

And so then I started thinking about this, and I started thinking back, like, how is this going? What was going through my mind at the time? You know, my boss would get in the room and he’d sit down with me for a one to one picture of my boss.

No. Oh, that’s not my boss. I joined PayPal long after Elon had been kicked out.

But the reason he’s there is because this brings us back to that question of untapped potential, right? If a company misses something this big and obvious, what else are they missing? Right? Like this is super important question.

Misaligned Metrics and the Wrong Questions

So I went back through these events in my head, my actual boss, there he is, and he’d sit down and he’d say, Matt, what’s your activation goal for the month? For the quarter? When are we going to get the activation number up? When’s that activation number going up? And he kept asking me, How am I going to get the number up? But, you know, it was the wrong question, right? Because, and the reason he was asking me that, and he wasn’t saying, Matt, what should our strategy be, which what should we be doing, is because he was a political operator in a big company, and I don’t know about what you guys do, but in marketing, the way you get promoted in a big company is to do more things. It’s to do more projects, to take on more budget and more deliverables and more head count and build your little empire. And anytime anyone asks them to do anything and say, Okay, we’ll do that. And someone once told me, there’s three kinds of bosses. There’s a shit shield, a shit funnel and a shit fan.

Well, my boss was a shit funnel, so he would just sign up for everything and then just push it all down to his team. And so someone made a spreadsheet somewhere that said, you know, activate. Get the activation rate up. We’ll make more money. And so he came and told me to do that. But here’s the thing, right? The relationship between like, the amount of work you do, between the relationship between the amount of work you do and the growth of your business is not simple and linear like that. It’s not as simple as like, more work gives you more results.

So the amount of work you do doesn’t necessarily have that linear relationship with growth. And so what he needs to be asking me is, like, really, which work should we be doing? What can have the biggest impact? So when I thought back about my time at PayPal, one thing I remember, and just looking back at the company next slide.

The Leverage Playbook

These, 90% of their growth came from 10% of the stuff. Not only 10%, 90% of their growth came from like five things, getting onto eBay early, getting network effects growth going, reaching out to web developers who were building all the E commerce sites, getting pre integrated with all the shopping carts and E commerce platforms, a bit of sales, and then a bit of international expansion, a lot of international expansion. That was the whole playbook. That’s not the only thing they did. Of course, they did a ton of things, right? They spent money, they built all these products nobody ever used. They ran ad campaigns that never worked until the Howard Will Ferrell, who can’t sing very well, like they did all these things. But all their growth came from just a handful of things.

The Leverage Formula

And then when you look back at all the other like great companies of that generation or this generation, I see the same pattern over and over again, 90% of their growth is coming from 10% of the things that they’re doing. And so that’s where I’ve sort of become obsessed with this idea of leverage, there we go. So if you know that 90% of your growth is going to come from 10% of the stuff you do, then suddenly, which work do you do becomes a super, super important question.

So that begs then the question, how do you get everyone in your company to focus on the most impactful work? Now my day job, I work with startups, and if they make the wrong decisions and do the wrong things, they go out of business because they just have a little bit of money and a handful of people and a little bit of time. But I don’t think there’s these laws the physics of this is any different with big companies. They just have a lot more money so they can throw money at things that don’t end up working and hire people who don’t end up working out, and they just don’t need to have that much discipline. But I think if they’re honest and they look at themselves, they’ll see the difference. You know, that a small the huge amount of growth is coming from a small amount of their work. So let’s talk about how to do this.

Three Steps to Focused Execution

1. Find your North Star
2. Find your big levers
3. Cause mindset change

First and foremost, obviously, show me the incentives. I’ll show you the outcome. So fewer bigger things. Three parts to this. The first one, and we’ll talk about this, we’ll talk about each of these, is to find your North Star.

The next thing is to help your organization use that information to find their big levers.

And the third one, that I’m going to tell you, what I know but I don’t think anyone’s figured out, is to cause this mindset change. But I’ve at least thought a lot about what are the blockers? What are the things that are holding people back from mindset change?

What is a North Star Metric?

Your North Star Metric is a single number that increments when you deliver value to customers. You can align the entire business around it.

When I was in high school, no college, I had a summer job working in an oil refinery, pretty good job in Cleveland in the early 90s. So this was a used oil recycling business, and it’s the greatest business model in the world, because when people have used oil, they can’t just chuck it in the lake. They have to pay you to take it and then you refine it into clean fuel oil and you sell it. So it’s like a marketplace business, but both sides of the marketplace pay you, which is brilliant.

But here’s the rub, all of your revenue is delineated in gallons, dollars per gallon received. All of your costs are delineated in hours, leases on trucks and machinery and equipment, which are the same thing, salaries, all that stuff. So the way you make money is to refine more gallons per hour.

Now, everyone in the company knew this. I’ll tell you that in a refinery in Cleveland, you didn’t have Stanford MBAs, you didn’t have Stanford, you didn’t have MBAs. A lot of these people hadn’t even graduated from college, but everyone understood how the business worked, and everyone knew exactly what they were supposed to do when they came to do when they came to work every day, and so they could make good decisions. They didn’t always do what they were supposed to do, but they understood what they were supposed to do, and that’s the beauty of something like a North Star.

Now, of course, it gets more complicated if you do have MBAs, because investors and finance people seem to have this very simple mental model, in my experience not everyone, of a business, which is, you put in some money, pay a bunch of people, they do a bunch of work, and like, money comes out the other side of it, like some kind of vending machine or something. And unfortunately, or fortunately, it’s kind of a dangerous way to think about it.

Avoiding Empty Revenue Metrics

Could you imagine if I went to that refinery and told all those people, Hey, listen, our job is to make money. So go find a way to make as much money as you can. You would have gotten a lot of strategies. A lot of interesting things would have happened in this refinery. Probably none of them would have involved oil. It gets even worse if you’ve got a bunch of people who have advanced business degrees, because they’re going to come up with all sorts of clever ways to make money, like cracking down on password sharing, or bundling, lock in. Anyways, anti competitive practices and these things, they’re all perfectly good ways to make money. They’re terrible ways to grow.

The only way to really grow a business, if that’s what you need to do, if that’s your mandate, not milking the cash cow is to delight more and more customers. And most of the ways that a business can increase revenue are actually orthogonal to or even work against customer value delivery. So when you’re trying to grow a startup, your north star metric needs to be some measure of value delivered to customers. So how do you measure value delivered to customers?

So nice, these are working. So long time, BoS attendees will certainly recognize Clayton Christensen and his co developer of the jobs to be done theory here in the front row, Bob Moesta. And they give us a very simple but very powerful way to think about this, which is that we don’t, you know, people pull products and services into our lives to help us make progress against a certain goal in a certain situation.

So we deliver the value to customers if we’re helping them make progress in their lives in this certain way. So how do you measure that?

Measuring Value

What measurable you ask yourself, what measurable customer behavior is the closest proxy for value delivery? Or to put that in simpler language, if our customers absolutely loved our product, how would they behave? How many of them are behaving like this? Track it in cohorts, improve it over time.

So there’s examples of this in the consumer it’s often going to be like daily active users or some kind of transactions or a usage metric, like the time spent listening on Spotify. In B2B, it’s going to also could be daily or weekly active users, or it could be a particular action, like records created or weekly active teams, collaboration. In a marketplace is normally the point at which the sides of the marketplace come together. So I’ll leave this with you, but it’s important to figure out, what is your north star metric for your business.

Then under that, and I won’t spend a lot of time on this, but you then can figure out what are the levers, what are the drivers that move this?

At PayPal: Total Payment Volume

So at PayPal, our North Star was total payment volume through the system, the drivers were preference and ubiquity? Are we available everywhere someone could choose to pay? And are consumers choosing us whenever they have the option? I’m saying us, but I no longer work there. I’m not a shareholder or anything. But if you want to map out your whole growth model, I’ve got, I did, like a whole workshop recorded, which you’re welcome to download, about how to map your growth model and fill that in and find your north star metric. So you can go ahead and grab that. You’re welcome to have it, but I’ll leave it there.

So the first thing you’ve got to do is identify, at a minimum, your north star metric, just so that everybody can understand what they’re focusing on. But the next question, and the important one, is how to actually get people to then apply that to their business and prioritize the high impact work, because people out of the box by default when you hire them in your company, oh, this is exciting.

Helping People Find Their Big Levers

When you hire new people by default into your company, what they don’t do is come in, study the business and then reason through from first principles, what’s the best course of action to take nobody I’ve ever met enters the job that way. What they do is they bring best practices, which, you know, best case scenario is the thing that worked for them in their last job. Worst case scenario, it’s something they like read in a blog post or, no offense but heard at a software conference or something. You know, it’s going to be some kind of like, generic thing. And of course, the situation that an intelligent person is trying to face that you’ve hired them for isn’t going to be generic. It’s going to have some complexity to it.

So the next thing we need to do is help them find their big levers. Drives it mindset change. So how do you find your big levers?

Start With a Conversation

So the first thing you want to do, step one, is share your north star metric with everyone in the company, explain it to them, Take time to answer your question, their questions. The next one, and this is super important, is to ask each person to ask, don’t tell, How does your work affect our North Star? Anyone here who has children or employees, you know, if you tell somebody a thing, they may or may not listen. But what happens if you ask somebody a question? You force them to think a little bit, and this is usually going to be an interesting conversation. You’re probably going to you’re guaranteed you’re going to learn something often the person knows some context or piece about the business that you don’t know. Often you know something about the business that the person doesn’t know. And so this ends up being like a pretty fruitful discussion. Minimum worst case scenario, you learn the person’s a moron and that they’re in the wrong job, but often you just best case scenario, you have a lot of context as a leader in the business that your employees don’t have, and if they had that context, they’d make better decisions.

So the epilogue to the story about PayPal that I’ve not told anyone and didn’t put in my book is, so we’re running that program where we’d call these high value merchants and, you know, onboard them. I moved to the UK, and I was kind of looking after the small and medium business segment over there, and I got together with my old boss from California, and she, you know, and are catching up what’s working, what’s not.

When “Best Practices” Backfire

She said, By the way, Matt, we stopped running the high value merchant program. We shut that down. It was only like two head count. I was like, why’d you do that? And she said the magic word. She said, we pulled the data and it wasn’t having a business impact. So that was a very short conversation. Whenever I flew back to London and I was like, well, I should probably shut it down in my region as well, if it’s not a good use of resources. But I still something about the numbers just didn’t sit right with me.

So I emailed Pete, the analyst the Igor of Europe, and I said, Pete, can you pull I didn’t tell him what I was thinking. I wanted to keep it blind, but I just said, Can you pull the results for the last 12 months on the high value merchant program? And he did. And then a week later, he emailed me, Yeah, Matt, it looks like the program is having no impact on the business. I was like, hmm, because it’s just weird, right? Like, the skew of value in our customer base hasn’t changed. I know the customer service reps are doing a good job, because I go and listen to them once a month and like, this doesn’t make sense. Why would this stop working? So I said, Pete, would you mind running me through your analysis. No problem, Matt.

So first thing we do is join from blah, blah, blah tables. I don’t know what he’s talking about. We get all the customers, and there’s a flag here to say if they’re in the test group or the control group, and we throw out any customers younger than three months old, because they need to mature. We chuck out the outliers, we separate them into test and control, and we take the averages. I’m like, wait a sec, we chuck out the outliers. He said, Yeah. He said, It’s we have a new head of marketing analytics. His name’s Steve, and he says it’s the best practice to remove the outliers whenever you produce a piece of marketing analytics, we do that before we send to the business.

And you know, he’s right. It is best practice and outliers do throw off these kinds of things. But what Steve and Pete didn’t know is that PayPal, top 1% most valuable sign ups, generate 54% of their revenue. I don’t know if that’s public, so let’s keep that in here. But that’s your outlier.

Your outliers are the business. So, like, this program can’t work if you chuck out the outliers, you don’t have a business if you check out the outs. So I explained that to him, and I think we may have petitioned the Steve on high and got an exception to this, but this left me with this important message, which is just like, I mean, you may know this feeling if you’ve been in a business where it’s like, I can’t take my eyes off of a single piece of this business, even for a minute, or something’s going to go pear shape, because I’m the only person who understands how this business works. And that was a really disconcerting feeling. And that is why I’m talking about this process. That is why you want to go back and ask each person, how does your work impact the North Star?

And then the next question, there we go, ask people, which work could you do that has the biggest impact on the North Star? And this is also very uncomfortable for people.

Picking the Right Work

People like to do all the work. They don’t like to just think about one particular thing.

And the other problem is often people are going to give you like you’re going to have to spend time, and there’s going to be revs and revs on this. And I’m sorry I don’t like meetings either, but I promise you, this particular one is time well spent. So you’re going to ask people which thing, if it works, can have the biggest impact on our business. If this works, how big can it be? And they’re going to give you a bunch of ideas that are like so-so sized ideas, but they know how to do them, or they can do them with their existing resources, or something like that, rather than the actual things are going to have the biggest impact on the business. And then once you get them to the big things, you need to commit to those big levers.

Now the last piece, and those become your quarterly goals, that becomes the work you’re doing. And that’s actually the easy part. The hard part is then deciding not to do all the other things on the list. And so that brings us to this lovely topic of mindset change.

So as I mentioned, the way you get promoted in marketing, the way you get promoted in lots and lots of jobs, is to make lists and to do all the things on the list. And I think lists give us comfort, because we feel like we’re working hard, we’re busy, so good things are probably going to happen, and even if they don’t happen, we can say we did everything on the list. So we’re good, right? Only we’re not, because the things we were hoping would happen didn’t happen anyways.

So people like lists. The problem is a list is not a strategy. A strategy involves trade offs among multiple good options. The opposite of a good strategy also sounds like a good idea, right? If the opposite of your idea doesn’t sound like a good idea, it’s not a strategy. So, and then so you get your list, you get rid of the list, you get people to do the big things.

And then the other thing I’ve seen, and I’ve done this now with work through this, through my coaching and as an investor with over 200 companies, the best ideas you have are often they don’t look like good ideas at all. Often the good ideas sound bad. And I’ve seen this time and again where the thing that finally works is like, Oh, we tried that three years ago and it didn’t work. Or I don’t like that idea, because it conflicts with cognitive dissonance. It conflicts with this, the vision I sold to my investors. Or we don’t, you know, that’s not how we talk. Those are our customers.

When, before I got to PayPal, the early when they first noticed that they had traction on eBay. They started seeing these comments about eBay, like, in their money transfers, they went and looked at ebay.com and it was like, they were like, this is like an online garage sale. This is rubbish. This isn’t our business. That’s not what PayPal is. I mean, eventually they noticed that the number of eBay transactions was doubling week on week. So they changed their mind. But originally they were quite dismissive of it.

I worked at this company called fat map. It’s 3d high resolution trail maps for back country skiers and trail runners and things like that. So we did our jobs to be done interviews with our customers, and we figured out that this customer journey starts with a Google search for like, a map of a trail, and then they download a PDF to their phone, and then they’re saving the file, and like pinching and zooming and because it needs to work offline, and some giant pain.

Anyways, so I said, Guys, if this starts with a Google search, how many maps do you have in your database? About 300,000 OK, could you expose those as 300,000 search engine optimized pages for the names of a bunch of different trail maps. And they said, Yeah, yeah, yeah, we thought of that. But the problem is, our map rendering engine takes is very heavy. It’s a lot of work. It’s high resolution, so we can’t afford the server time, and Google wouldn’t rank us anyways, because the pages would take too long to load. So they thought of this two years ago and didn’t do it because it wouldn’t work. And I just said, OK, is it possible to just load a static image of the map and then if someone tries to click on it, you load the actual map from the engine in the background? Oh, yeah, we could do that. Yeah, sure, no problem.

So, like, a week later, they did that. And this is a screenshot of SEMrush. As you can see, suddenly their search traffic went up a lot, a lot, a lot. And then about a year later, they got bought by Strava for a lot, a lot of money. And like, the point of the story is, a lot of times we just have these ideas, but for some reason we think they won’t work. We can’t get the resources. The CTO says we can’t do it, so there must be no way to do it. So instead we do the small ideas, and instead that we think we can do.

How do you cause mindset change?

First thing, make a not make a not doing list. These are all the like little things that they’re not going to do. So like, again, like the people I started working with at PayPal, the Harvard Stanford and the Stanford Harvard lady, we’d send a weekly status report to the whole team. And I, you know, I put like five things on my status report because, like, doing five things in a week is good. And this one would have like 40 things, and this one would have like 44 things on her list. And it got to, like, I started making a had a little notebook next to my desk, just so that every time I did the tiniest little thing, I’d write it on there so I could remember to put it on my weekly status report.

My friend Melinda started calling them. I took a shit emails, because it’s like, and then I took a shit, and then I went to another meeting, and it’s just like, got to be ridiculous. So you got to be really deliberate about we’re not trying to do all the things. So start with a not doing list puts and these it’s going to hurt. People are going to every project you’re not doing is going to betray someone. It’s their idea. It’s going to hurt.

The Danger of Sevens

The next thing. One of my favorite quotes Sarah Tavel, partner at Benchmark Capital, sevens kill companies. I think she was talking about talent, but I think the same idea applies to ideas of projects and work. The things we end up doing that aren’t the high impact projects are not stupid, bad ideas. They’re just, like sevens. They’re ideas that kind of work, and we can kind of do them, and they’re kind of profitable and they’re safe, and we know how to do them well, but it’s a case where, like, the opportunity cost of not doing the big things is the problem, and the big things are like the CTO says it won’t work, or I’ve never done this before, or this isn’t what we told our investors anyway. So remember the sevens kill companies in a startup. Don’t waste any time on sevens.

Model the Behavior

Two is, model the behavior as a leader. Talk about the good projects that you’re excited about that you’re not doing. I have a small team now, but when I send my weekly status report, which doesn’t have 40 things in it, it doesn’t talk about my bathroom behaviors, I have a list of things in there that I’m not doing. First of all, just to show the team, like these are all good, important projects, I’m not doing them, but also, just because I don’t want to forget, because someday I hope I will get to do them.

Ask the Right Question

And then the last piece is ask the right question. And there’s one particular question that has really come to guide everything for me. That is what’s important and how’s it going.

So whenever I this is like my quarterly business review cadence. This is my one to one cadence. I don’t want to hear about the 40 things you’re working on. We all agreed the high impact work was like Project X and Project Y. Tell me about how Project X and Project Y are going. And if you know, I got a meeting with Matt next week, and he’s only going to ask me about Project X and project y you’re going to be working on those things. The legend is that when Peter Thiel was running PayPal, he gave each person one job, and he literally refused to talk to you about nothing, about the weather. What did you do this weekend? Any other project? Is someone going to buy us? Is someone going to fire Elon? Nothing. He would only talk to you about the one project that he gave you. So, yeah, oh, it actually works out.

Well, that was those three things were my last slide, anyways. But the wrap up of this is that this idea of leverage, right, corporates should win, but startups do.

Why Startups Win

Corporates with all their resources and money, they don’t win. The startups do. Not because they outwork them, but because they outthink them.

Now this is going to go against most people’s nature and the fiber of their beings, so it’s a constant battle, but focus everybody on the most impactful work. Make sure that your team knows that they’re expected to do fewer, bigger things. That is all.

Q&A

Audience Member 

Thank you. I’m over here. Yeah. Thank you very much. Great presentation. We sort of trying to implement this North Star philosophy at our company as well. But one challenge I’m constantly running into is this thing, which we call business as usual, right? There are many other things, like support engineers, they still need to answer the tickets, right? There are still bugs to fix and stuff like that, right? And so when I try to really push people to focus on, you know, our North Star metric on or innovation. And we identified exactly that, that actually there is, like, you know, some key things which really drive us, drive our company forward. I’m constantly running in this issue that people are just busy with, you know, maintenance work, how I would call it.

Matt Lerner 

That’s exactly the challenge, right? Business as usual. Everyone. What about BAU? It’s like, okay, tell me, what is this BAU and how is it more important than growing this company and not running out of money?

So this process is designed to combat that. Because I don’t know your team, I don’t know your support tickets. I’m sure that some of the stuff they’re doing is really important and critical to the customer experience, or, you know, tech debt you’re really going to have to pay for. And I’m sure some of it is a waste of time, but they think it’s important, because their job is to find and fix bugs. And these are bugs, right? And so that’s where I like in this process. You say, hey, you know, Technical Support Engineer, bug triage or whatever. How does your work affect our North Star?

Well, some of the bugs I fix, you know, hurt our customer experience, and some of them do bubble. Once they think about that, it’s like, okay, now go back to the work you do every day. Which of the work you do has, you tell me, has the biggest impact on our North Star. And therefore, if you had to stop doing some of it to make room for other of it. What would that be? And that conversation will get into the nitty gritty. And the punch line is they’re going to have to end up not doing some things that they really, really, really want to do.

One more PayPal story. So when I joined PayPal in 2004 they’d already IPOed and been acquired by eBay and the entire company, the entire business, was a single cgi bin executable. Like we were doing a promotion with Apple, and Steve Jobs requested a change at the last minute, and I had to stay there overnight while the engineer checked out PayPal. I had to watch him code the thing, and then he checked PayPal back in, and had to regress the entire thing overnight and do a push to change, a sentence on the home page.

This, of course, is a horrible idea, terrible technical debt, and then maybe three months later they pushed an update during Christmas. It didn’t work. It brought the entire thing down. Crash. PayPal, eBay missed its earnings number. It was a disaster. But, you know what those guys are, rich, PayPal, IPOed, eBay did just fine. Like it’s the worst technical debt story ever, because people didn’t do the things they were supposed to do. But you know what? Everything worked out just fine. And whatever bug those engineers on your team are looking at, it’s not that bad.

I can hear you, but nobody else can. Hi, Joe.

Audience Member 

Now, yes. Okay, cool. Thank you. That was amazing. I love when you talk about sevens kill companies. It’s kind of hard still for my team to understand. And then there’s the harder challenge that I found with the marketing side of things, and how you know something’s a seven or an eight with short and long term possibilities, right? So, for example, YouTube, should we be marketing on YouTube? As a question that comes up a lot because it’s a bit of a long game. So I’m wondering what your thoughts are there on like the marketing side of things, particularly, how, for the companies you work with at SYSTM, how do you know? How do you know where to put your marketing resources when some of it won’t pay off in the short term?

Matt Lerner 

Sure thing, just like SEO or any of these. Okay, so the first question we’re asking here is, if it works, how big can it be? Now notice what we did not do. What our friends on the product side will do is they’ll say effort, chance of success and impact is your prioritization. In growth, we don’t do a lot of chance of success because we’re not very good at that, and we don’t do effort because we’re going to find a way around the effort. So first we’re looking at, if this works, how big can it be?

The next question I’ll ask the team is, OK, what are the riskiest assumptions here for this to be true, for this to work, what has to be true? So for the case of YouTube, there has to be volume like people have to be searching for this stuff, or else they’re never going to find us. We have to be good at making really engaging videos. Have we ever made really good, engaging videos before? Do we have a source of content? Do we have a lot of content? If this works, will this turn people into customers or not.

Will they be the customers we want who can actually afford our products and we’ll keep buying them or not? So just go through any whatever proposal with them, identify the riskiest assumptions, and then what is the quickest way that I can just minimum, minimum viable test? How can I test each of those assumptions? Is there volume search volume on YouTube for this thing? Well, that’s easy to find out. Can we create engaging content? Let’s take our best content and see if anyone engages with it, right? Can we get customers? Well, buy some traffic, pay for some traffic, try to get some leads. See if they’re your customers. See if they’re good customers. So just isolate and test each of those things quickly before you decide to make the investment to do the big work.

Audience Member 

So question about the North Star metric, and when you don’t have this insane level of concentration, you know, like PayPal had, where either you have different segments or even customers at different life cycle, where the behavior that you’re trying to get, where they get value, is so different. Trying to wrap that up into one number, it becomes so abstracted. It’s like, net income or something dumb like that, right? That’s not gonna point you in the right direction. Like, how do you deal with that when you have this kind of more heterogeneous behavior from customers?

Matt Lerner 

I will just like, I think you and I, if we sit down for 10 or 15 minutes, we can probably work through a specific business you’re thinking about But in general, that’s why you end up with things like daily active user or weekly active user, because no one’s gonna. You know, there may be a bunch of workflows and a bunch of different ways to use a product and ways to get value from it, but no one’s gonna one’s going to launch and use a piece of software four times a week for six months if they’re not getting value from it in some way, or if their boss isn’t getting value from they’re using it. So we try to abstract it there.

That’s good for the purpose of alignment, of course, then you need to still understand the nuts and bolts of how do people get actually get value from it, what out of box experience drives retention, predicts retention, so you have to get a lot more detailed numbers under that. But for the purposes of the North Star metric, normally it’s you just solve it with abstraction. Great question. Thank you.

Audience Member 

I love the idea the prioritization, I think, in practice, though, felt like a very subjective exercise to say, you know, what do you think your tasks are that most impact the North Star? I’m wondering if you’ve used any prioritization, like frameworks and things that you can be more objective about it and get the team on board with the framework, and not necessarily just what they’re bringing into the conversation their subjectivity.

Matt Lerner 

Yeah, it’s a really good question. I try to I like the beginning of my questions were all very subjective, and I go to them very subjectively, but then I say, if this works, how big can this be? And at that point I am expecting to see some arithmetic, and that could look very different for someone who wants to do YouTube ads, or someone who wants to build a product feature, or someone who wants to open a branch in Luxembourg. So I’m expecting them to understand their business well enough to walk me through that math and however that looks. But that is the point when it’s like, if this works, how big can this be that it’s like, plug in assumptions, right? If only 400 people a month are Googling this search term, then SEO isn’t going to be a very good strategy for us. Or whatever that data is, is that the direction you’re looking?

Audience Member 

Yeah, I think maybe one tag on comment to that is, you know, I think the to focus folks into the one thing or the five things, is a startups, you know, hardest problem, and I think when you if you’re not telling and you’re expecting them to know the business better than you, but maybe there’s a gap. Maybe there’s a chasm there where they don’t know the business as well, or you don’t think or into it that they know the business as well. So what do you do in the interim? And we’ve filled in frameworks in that but I’m curious how you’ve seen.

Matt Lerner 

It sounds like you run a startup. So that was the point of me asking the person, how does your work impact on North Star, and which work should you be doing? Like, if my boss had come to me in the beginning with that question, we probably would have gotten to the right answer a lot sooner, because he knows stuff that I don’t know and I know stuff that he doesn’t know. And that’s not always a simple conversation, but that in that hour, or however long that takes, that’s where you guys sort of will work together to get to the right answer.

Is there another question?

Audience Member 

Thanks Matt for the presentation today. I’m really curious about the do not do list. First off, would love to hear a couple of things that are on your not to do list, and maybe talk a little bit more about, you know, how you can do that with your team as well and working with them to create their do not do lists.

Matt Lerner 

Okay, so obviously, you know, the list of things we’re not doing is potentially infinite. So what I’m really looking for, for that list are things that we all want to do, that we’re excited about doing, that we actually believe in, you know, or they keep coming up. For some reason, there’s a very good reason we’re not doing them. So the biggest thing at the top of my not doing list right now is I have this idea in my business that we’re going to create, like, almost like the Myers Briggs test, but like, a diagnostic for your business. And I’ve got, I’ve already written a thing, like, I got halfway done doing it, which, you know, and it’s all these questions where, if I ask you these questions, I can say, okay, here are the blockers in your business. This is where you need to focus. And I can give you this report. I’m super excited about I think people are going to love it. I’m not doing it because I got, you know, like, three other projects ahead of it.

Audience Member 

Hi. So what advice would you have for managing up or laterally or when consulting with clients, on adopting this mindset? And I’m thinking specifically of working in environments where folks are really driven by sales metrics or marketing or things that maybe are a little off base from the things you’re describing here, of finding what really is driving value, and that there’s maybe a misalignment there, and you’re working in that space, seeing it, but not sure how to get folks on board, especially folks that maybe are in those positions of saying, like, well, this is what’s important.

Matt Lerner 

This. I don’t have a good answer for you. I think you’re you’ve identified the exact problem, right, which is the people in this most senior levels in a business think this way, because they understand that anything their business is going to do that has any meaning works across all these departments. And then they have these department heads who come in, and they spent their whole life building and running sales team or building and running a product team or building. And so they only think in terms of their silos. And then there’s a bunch of people like you who then, I’m assuming that you’re talking from personal first hand experience, but there’s a bunch of people who come in who aren’t yet ingrained in that siloed thinking, and are looking at this from first principles and saying, Hey, this doesn’t make sense, and they’re stuck under that middle layer.

Normally when we work with companies, I mean, it just ends up, like, we come in as outside consultants, so we have a bit of extra credibility, but it does lead to a lot of really hard conversations, and it’s got to just come from the top, from someone saying, like, you know, you guys have to work together to achieve this goal. Some people have this either pragmatic or just open minded mindset, and they’re like, they will take to this, but a lot of people are really threatened by it, because what they’ve been doing has been working and it’s gotten this far. And, you know, there’s newfangled consultants coming in with their crazy ideas are a threat. And I, my honest advice is, if the people you’re working for see new or contradictory information as a threat, find a different role.

Mark Littlewood 

Matt, I can’t thank you enough. That was brilliant.

Matt Lerner

Founder, SYSTM

Matt Lerner is a bestselling author and 15-year Silicon Valley veteran, and three-time BoS speaker. Matt was part of the early growth team at PayPal, and a Partner with the VC fund 500 Startups.

Since founding SYSTM, Matt has worked with over 200 seed-stage startups to help them develop and execute their growth strategies.

He’s been featured in the First Round Review and the Harvard Business School podcast, and he’s guest lectured at Stanford Business School.

You can check Matt’s talks at BoS here.

Next Events