Matt Lerner: Identifying Your Metrics that Matter

As you plan for growth, you need to measure the success of your strategy and actions with clear and unambiguous metrics. Perverse incentives kill your business and you get what you measure. Too many metrics leads to confusion, misalignment and unexpected outcomes.

Matt leads this interactive session by sharing a framework that you will complete to help you identify your core growth metric and the right metrics to measure in your organization to ensure that teams are aligned with your specific growth goals. You’ll learn to how choose the right targets for sales and marketing so your teams can make better decisions, focus, and prioritise the right work.

Slides

Transcript

Matt Lerner 
Put the link in the chat and make a copy. And you can fill that out as we go or just sketch on a piece of paper or wherever it is that you take notes. Show me by raising your actual physical hands, how many of you watch the pre work video? I expected none of you. So okay, I’m going to repeat a little bit of it. And that’s okay, because if you watched it.

My Story
I’m gonna open with a story 2009, I was working at PayPal, in San Jose, California, I was looking after as the General Manager for like the SME business in the US, which was over a billion dollar business at that point. And they it was 2009 strategy season, because it was like February, which is when you come up with your annual strategy for the year. And they came to me and they said, call me into the extra special strategy meeting and said, ‘Okay, Matt, this year, we’re looking for projects to make money.’ I was ‘Like?’, and that’s it, that’s a strategy. In my head, I’m doing the math of like the, I’m reporting to a VP who reports to an SVP reports to CEO and there’s probably like 10, or $15 million of annual salary between me and the board. And the best they can come up with is tell them that we need to make money. All those years in McKinsey, and that’s been and that’s the best you can come up with. So anyways, when you’ve already got a gazillion active customers, the quickest way to make free cash flow in the year is to raise prices. So I’m not proud of this. But we did a strategic price increase. It was kind of, it wasn’t our show rates were just kind of deep in the pricing table, the way we handled some refunds and authorizations and things just to actually bring us on par with industry standards. Anyways, long story short, it only costs each of our merchants about $3 a year. But we had 10 million merchants. So we made an extra $30 million in 2009. And every subsequent year, off the back of that price increase. So that was good. But I don’t think most people realise 2009 was actually a revolutionary year, I mean, history of payments processing. Does anyone else know what exciting things happened in the payments industry in 2009?

I’ll give you a clue. Does anyone recognise these teenagers?

Mark Littlewood 
They’re the Collisons

Matt Lerner 
The Collison Brothers, yeah, interviewing for Y Combinator. They founded Stripe, and Stripe gave customers simple API’s to process credit card transactions. And I knew PayPal customers wanted that. But it would have taken a long time to develop and get it out into the market. And generally card processing is kind of lower margin. So PayPal didn’t build that product in 2009, because that wasn’t part of our make money strategy. Last private round stripe is valued at 95 billion, they may be marked down a little bit since then, but I expect their valuation will return to its lofty highs. This deck is a bit out of date, but also in 2009. Jack Dorsey founded Square, which is now called Block. And that was another product that I knew our customers wanted. They say that your customers don’t know what they want. If you ask your customers what they want, they’ll say faster horses. But our customers were literally calling us and saying can you make a card swiper that plugs into my smartphone, I’m tired of typing in credit card numbers with my thumbs. They were literally describing their product to us. But we didn’t make that either. Because hardware has costed goods. And again, card processing is lower margin. So wasn’t interesting to us. But I don’t know what Block’s valuation is these days. But it’s it’s a healthy, you know, 100 billion dollar plus value company. So our strategy to make money in 2009 kept us from hitting these two gigantic, I mean, massive opportunities so that you know, if you put it think in terms of valuation, I made us 30 million a year in free cash flow, and we were selling it about 40 times earnings. So 1.2 billion in shareholder value. Not bad. But you know, the stuff we didn’t do ended up being worth a lot more.

Takeaway
And so the takeaway from all this is that you get what you measure. And the bigger your company, the more that affects everybody and everything you do. So we’re going to do today is go through a process of really carefully selecting the right metrics for your business to avoid creating perverse incentives. And actually do the best thing for your investors in the long term. Just quickly to introduce myself, this is the intro. So in addition to spending before I joined PayPal, I worked in the Valley for almost 20 years first startup failed second startup failed. The third startup we ended up selling for 40 million bucks to a company called Macromedia. Many of you remember that quit travelled around the world, St. John for a few years came back joined that growth team at PayPal in 2004. That was a fantastic ride, eventually moved to the UK, left PayPal became a VC. And I think the first thing that struck me as a VC was that most of the startups I saw pitching me, were thinking about growth very differently than we were thinking about it in the valley. And I’ll talk about some of the differences later. But it seemed like such a massive opportunity that I left investing and founded Startup Core Strengths, and we run a virtual accelerator online. It’s a 10 week programme. We work with about 60 companies here, we help them with this metrics work and kind of finding, using metrics to find and prioritise the most impactful work. And then run growth sprints, experiment driven growth sprints, and Minimum Viable tests, to kind of speed execution and learning and help the team adopt a growth mindset. We’ve worked with over 100 companies, and we have a founder likely to recommend him 96%. And I’m proud of that. But when companies apply and come into the programme, you know, we we interviewed him first because we’re pretty choosy about who will work with. And one of the things that becomes apparent in every one of these conversations is that people are moving in too many directions. The British are polite about it, they say, We have no shortage of ideas. But what they mean is that it’s, you know, chaos. And if you take any successful startup, this was certainly true at PayPal, and you look backwards, you’ll see that 90% of their growth ended up coming from like 10% of the stuff they did. And so if you assume that most of the stuff you’re doing isn’t going to make a difference, then that means progress is deciding what not to do. Problem is you have to be very, very good at making those decisions about which work you’re doing, and which work you’re not doing, again, is my like story about the Pay Pal strategy in 2009 demonstrates. So how do you decide what to cut? How do you get people? How do you create an organisation that gets people to make better decisions about which work to do?

So does anyone recognise this guy? Warren Buffett’s. Yeah, this is Warren Buffett’s partner. This is Charlie Munger. He’s been running Berkshire Hathaway, one of the most successful investors in the world. And they’ve been managing this fund together for 311 years. So they’re very successful investors. Anyways, the point is, what Charlie says is, show me the assignments. And I’ll show you the outcome. It’s all about the incentive structure you create. So here’s a really simple example, when I was in uni, I had a summer job in a refinery, an oil refinery, it’s a very simple business, dirty oil comes in one end, clean oil comes out the other end, you get paid per gallon of oil refined, but all of your costs increment by the hour, leases, salaries, trucks, things like that. So the more gallons per hour you can refine, the more profit you make. So everything we’re trying to do in there is number of gallons refined per hour or per shift. And so that was like my job was to go around and check all the machines all day and see how fast they were going and measure how much oil is being produced. And or refined and feed that back to the supervisor so they could run the thing. Now, this was not we did not have MBAs in this company. Most of the people in this organisation, you know, maybe graduated high school, but everybody knew what their job was every day, because everybody knew that their job was to run their machine, whatever, centrifuge or unload trucks or load trucks, and just get as many gallons through as possible. And having that really simple metric for everybody helped the business a lot helps me help everyone figure out what to do every day. Unfortunately, software companies are not as simple as that. So the problem is, I think we tend to oversimplify the model a little bit and just say, like a vending machine, like you do work and money comes out. But there’s this step in between, which is super important that sometimes we miss. And that is the reason we get money is because we deliver value to customers. And when I work with teams, and I say What’s your ultimate metric? You know, the reason I push companies away from talking about revenue, like PayPal, is because metric you know, revenue can really create, you know, focusing on revenue, there’s, there’s too many ways to make money that sometimes do not create, but instead erode customer value. So if you start by aligning everyone around, delivering customer value, you should be able to get revenue. So if you can get if you have like a million monthly active users, it’s not that hard. There’s like five or six ways You could turn that into revenue. But getting from zero to a million monthly active users is actually a really hard challenge. So I sort of tried to focus everyone on delivering customer value. So how do you do metrics to do that?

So we’re going to talk about three different types of metrics today, your Northstar metric. And then key drivers, which are the things that move to the North Star. And then nuance metrics, which are these other numbers you’re going to watch because they’re important, but you’re not going to optimise them. So how many people here already have a Northstar metric for their business? Okay, Mark does good. You better I think we’ve done this workshop privately. Just you, me and Kirk? You do. Okay. Excellent. So I guess first of all, what is your Northstar metric if you don’t mind sharing out. Customers numbers? Okay, good

Northstar Metric 
So Northstar metric measures customer value, how do you measure value delivered to customers, I know how to measure revenue. So we’ll do this from first principles. I assume everybody recognises the next member of our old white men quiz name, that old white man quiz. This is Clay Christiansen, and he will tell you that you deliver value to customers, when you help them make progress in their lives. I’m not going to try to convince anyone of jobs to be done theory, I’m just going to take it as first principles at this point. Okay. So what you want to do is figure out what measurable behaviour is a good proxy for customer value delivery? Or in other words, if customers absolutely loved your product, how would they naturally behave? How many of them are behaving like this? Track it in cohorts, and improve it over time. So simple examples here, at PayPal when we were behaving ourselves, our Northstar metric was total payment volume. Amazon. In 1997, Jeff Bezos declared in his letter to shareholders that it would be repeat purchases for their ecommerce business, and Facebook by guest to update this with more relevant company, maybe tick tock is looking for daily active users. So those are examples of Northstar metrics that measure customer value delivery for software companies. Often it’s going to be like daily or weekly or monthly active users. Or sometimes it might be a sort of a more nuanced success metric, like number of records created or a number of queries successfully executed. But I’ll push you to sort of think about your business now. And what would be so you want a metric, specifically, that encompasses the entire funnel, increments when you deliver value to customers, is simple and memorable for the whole team. And something where everybody in the company can connect their work to this number. So what I’d love to do now is if anyone’s got ideas for what could be your Northstar metric, just tell me now and we can kind of stress test them. If that’s helpful. I’ve got a little background on some of your businesses already.

Audience Member
Okay, so number of bookings customers make?

Matt Lerner
Yeah, I was thinking that. That sounded like the right thing. If customers make lots of bookings, are they getting most of the value of the product? Okay, that’s nice and simple. Anybody else?

Audience Member
The churned customers is the one that’s kind of tripping me up when I think about this.

Matt Lerner 
Which one’s your business?

Audience Member
We’re DaVinci education. So our customer, or, you know, our financial customer is the university. We sell an enterprise software platform. So I’m thinking about things and we have multi year contracts. So I’m thinking about things like number of clients, length of contract renewal, something that’s hardest. This is something that we’ve struggled with, I’ve struggled with for a long time. It’s like what is that thing you know? And it’s easy to hide behind the complexity of it being enterprise, right, B2B. And it’s like, well, there’s all these factors and things that we can’t control about university budgets and things like that.

Matt Lerner 
How does universities decide if they’re going to re up?

Audience Member 
How or when?

Matt Lerner 
How?

Audience Member
Well maybe, they find value in the product.

Matt Lerner 
And how do they know if they’re finding value?

Audience Member
It’s they don’t have success metrics really defined very well. So we rely a lot on ad hoc our relationships with them and understanding how things are working. And unfortunately, sometimes even when they’re driving value, that stuff can happen at a leadership or a university level where it can get yanked from them. And it’s, quite honestly very independent of the value that our business users are getting from it. And I don’t want to hide behind that as a defence, it’s just sort of one of the realities of our market.

Matt Lerner 
I mean, if you’ve got usage data, and you can surface that to them, it might be helpful to you know, so they can even make their own business case internally. You know, as you’re talking to lots of universities, you probably get a sense for the ones who really have their shit together, in terms of, you know, tracking the ROI on their software spend, maybe you’ve got some best practices from those universities, you could share with others. I know a guy who runs like an online training, sales training business, and they have dashboards, to see how many people in the company are actually taking the training and how often, and they have account managers who report back to the sales directors and say, hey, you know, these five people in your team haven’t done any of the lessons. Here’s your current utilisation rates, to make sure like, Hey, you guys are actually doing this, right.

So, I mean, there’s probably some, you know, employee or professor or whoever in the university is supposed to use this, who, you know, would be using it or not, and that behaviour would probably be something that they’d want to know.

Audience Member
Yeah, that is something we’re looking into being able to measure. We have multiple stakeholders in different roles in the university, which again, complicates things, but also offers an opportunity, right to be able to tell that story at different levels as well. So that’s, I really appreciate that point. That’s interesting.

Matt Lerner 
No worries. Yeah, I mean, ultimately, you know, jobs to be done, you need it, the person who’s going to make this recommendation to get through the budget has the job, it may be very different than you know, the job your end user has. But it probably has something to do with your end user liking them and thinking they’re great. So helping them understanding the purchase decision makers job and how you can support them in doing that. Let’s start there. All right, any others?

Audience Member
Hi, I am, we seem to be unusually over represented by edtech. Today, it’s not my company, but there’s a company that I advise on work closely. And we’re in this space as well. And I think this is the bit I’m struggling out with this one here. Vanny to the company, you know, is essentially about that sort of commercial progress. And that’s a trajectory that’s there with those decision makers. It’s on a sort of a big annual cycle, there’s big budgets, there’s those kinds of things value to the the user is much easier, more easily understood, I think, for me, because you know, a bunch of the things that we’ve talked about here, we can tangibly put our hands on and say yes, when you do that thing, then you’re clearly sort of we are clearly creating value together. And I think it’s the reconciliation of these two things in these kind of more complex business model environments. That is the bit that we’ve certainly struggled with when we’ve been trying to go around the block on Northstar metric and tattle on something that is going to meet all of those needs.

Matt Lerner 
So if I understand the question, right, you’re saying we can deliver value to the end user, but that’s often not the purchase decision maker. And there’s probably also some other sign offs, you know, IT and finance in there other stakeholders.

Audience Member
I’m saying, you know, and I think there’s also this factor of the value to the procure, it may well be in the space that my clients in, you know, it’s around student retention, cause completion, you know, improvement in grades, outcomes and those kinds of things. So there’s two challenges. One is that’s their data. And it’s private. And, you know, it’s often hard to sort of get that insight in there. But the other thing is, they are, they’re just on a very different cycle, from the user who is onboarding at the start of term using the software getting the value from the software. And, there is a connection, obviously, because the more effective they are doing what they need to do, the more that is going to have an impact on those outcomes that the universities care about. And the more that that a claim is going to come through and influence those purchasing decisions, but they’re just not coupled together on the same cycles. They’re not part of the same feedback loops, that are just, you know, part of a much bigger, complicated set of machine and influence. And it’s, it’s the bit that is hard, like it’s almost, you know, it’s like the it’s hard to get the North Star because there’s two stars in the sky. got both of them are kind of moving at a different rate. And one of them is Halley’s comet and one of them really is the North Star. Right. And, and I think that’s I think that’s the bit certainly with that my folks have have struggled with on on this on this question

Matt Lerner 
When they get into key drivers, see if that helps. Okay. Thank you. I’m not sure I have the answer. And it sounds like it’s probably in the details. But there’s obviously forces at play other than end users using and loving the products that impact the purchase decision and are controllable. Absolutely. Like, fair enough. You know, the state of California cuts it budgets across the university system. You know, you’ll do your best, but ultimately, you know, that’s not a lever, you can pull yourselves.

Audience Member
Right. So okay, sure. Okay, I look forward to the driver selection. Thank you, Matt.

Matt Lerner 
Any other? I mean, I’ll jump into key drivers now. But any other Northstar questions or challenges? Is anybody pre-product market fit?

Mark Littlewood 
Okay, well, some of our companies are building we discussed it, at some point why? It’s quite a different challenge, I guess.

Matt Lerner 
Yeah, I think I mean, the Northstar can move around, because it’s not you don’t know, which drives the Northstar with with pre product market fit. Generally, you’re gonna want a Northstar metric to be an absolute number, rather than a ratio. So that it can, you know, go up 10x in a year or 100x and a year, you know, a conversion rate or net promoter score is never going to be higher than 100%. So,

Mark Littlewood 
Francisco’s free product market fit

Matt Lerner 
Francisco, what’s your country company?

Audience Member
Hi, let me just camera here, so it’s more personable. Hi, hello. My company is doing sports is motion, motion analysis for sports using artificial intelligence. Okay, and we have launched we launched in November last year, we growing we initially are offering is for sports teams and clubs. So initially, obviously, whilst motion tracking is sort of our USP in a company, by offering our solution to clubs and teams, our Northstar really is identifying our key stakeholders and decision makers in sports teams, those are very difficult because some clubs will have a treasury some clubs will be the coach and Gods will be anything in between a player that helps the coach make make those decisions and and controls the budget. To add to sort of the complications, sports teams don’t really see themselves as businesses, although they operate very much like businesses. So that’s a difficult sort of Northstar. And so we, we effectively end up doing a bit of both. So we use, we attract players into the app. So as a product, sort of lead growth hack, where players will tell the coach or the decision making so so that way, we don’t have to worry about that. And we actively go after sort of coaches and head coaches to offer them the platform.

Matt Lerner 
All right, you know, the fact that they don’t see themselves as businesses in this instance, might actually simplify your problem. Because I mean, you haven’t even mentioned what particular sport we’re even talking about. But I already know that your customer wants to win. Yeah, that’s one thing we can count on. And so if they can connect the dots in their brains between your product and improving their chances of winning, that’s a good thing. And so I would guess your Northstar is somewhere around insights delivered. But I do think there is still more of a, like a stakeholder management, like I met the head of quant for Liverpool FC. And we got to talking about this exact subject. And he said, You know, I have all this data. And, you know, I know probabilistically that it’s extremely valuable. But you know, still most football managers are kind of old school, and, you know, trust their gut and trust their instincts and don’t want to listen to data. And I thought, you know, it’s the exact same thing in a corporation. You know, these managers are old school and trust their gut, no one listened to data. There’s all this politics around it, too.

Audience Member
Yeah. So I think one of our strongest sort of promises is that we saved them time. So So 760 7% of all coaches in UK for grassroots are still volunteers. So actually, they value their time more than the data insights that we can put wide. So we sort of flipping there. So they’re saving time is is one of our North Stars. And we tried to measure that increasing engagement is another one that we we, we aim, it works well with academies because they don’t have a way to measure how coaches how well coaches are coaching. So they’re paying all these these people to be coaches, and they don’t know how to how to measure if they had been good coaches. So created a simple way of feedback, and engagement on sort of thumbs up thumbs down hot gloves that are from the player in back and forwards, which sort of gives sort of that metric. And that helps to track and then sort of, in then sort of some kind of an all the way around, then then the performance metrics and the stats and in game stats and in game analysis and the training stats. And using AI then sort of sort of wraps everything around and makes it sort of a unique offering.

Matt Lerner 
In terms of saving time. This is just kind of as a marketer, he’s been, you know, hacking software for the better part of 20 years now. Often saving time is such an abstract thing, that it’s hard for customers to connect that with this specific task. And often, you know, if you’re having fun, like people don’t worry about time, right? It suddenly time matters when it’s a really unpleasant task. And with any software product is usually some really unpleasant, specific, annoying task that you’re getting rid of. So it’s not just saving time, it’s like, you know, do your month end reconciliation in three minutes, or, you know, I worked with photo book making software, and it’s just a pain in the ass trying to assemble photo books. I

Audience Member
Agree. One of the things that helps coaches understand what, instead of us being on the same page in terms of saving time, is asking them how many tools and apps they use to manage that team. And before the chat, we realised that use about five or seven or eight, from YouTube, to spreadsheets to, to various apps, to scheduling to payments, and so on and so on. And so, so that becomes easier to say actually, you’ll save all that time. And then sort of we can wrap that into you actually will save all that money. If you’re paying for some of those services, intercom into a platform that you’ve got your communications, your feedback, your engagement, your scheduling, your game analysis and stats analysis. And by the way, you can remote train your players using our AI technology.

Matt Lerner 
One of those things is probably foremost in their mind and particularly onerous. You know, never do x again, or, you know, track your players without having to why if you can figure out the one thing out of those that that really makes most people wins at the thought of having to do it, I’d start that. Sorry for everyone else. We went super deep on that. But I hope it’s helpful. Alright, so write down what you think will be your Northstar metric. You can also email me after if you want more feedback. Although I apologise I’ll be offline the next two days, but I will reply next week. All right.

Key Drivers
So the next thing I do want to talk about now are our key drivers. So basically, the key drivers are the things that move the Northstar. So this is the work you’re going to do, your team is going to do every day. To move that Northstar metric. Normally, you want to keep it to like one or two or three things. Now when I say you’re tracking controllable customer behaviours, there’s the work and there’s the outcome. And that’s a super important distinction. You know, we could launch five features, we could redesign the onboarding flow, that’s work, if customer behaviour doesn’t improve, you haven’t hit those outcomes is not going to drive your Northstar. So key drivers track customer behaviour, which is the output not the work which is the input. Typically starting with a few kind of high level though it’d be like a tree diagram that nests so you know, new signups activate customers and returning customers. And then you have levers below them that move those what brings new signups, you know, number of leads, number of sales calls, number of demos, you know, close rate, etc. So I’ll sort of do like a tree diagram and start with your first order drivers and move backwards.

Now, with the North Star, I said it’s typically going to be an absolute number. Oftentimes, the key driver could be a ratio like visit to sign up rate or activation rate. And then in terms of key drivers, there’s lots of drivers, not all of them are key drivers. So you want to revisit that every three months and say, Hey, are we still focused on the right part of the business? You know, maybe we didn’t have enough traffic coming to the website. Now we do have enough traffic so we’re gonna focus more on conversion, something like that. And then importantly, any key driver should have an owner and a target. So here’s that, you know that tree diagram I was talking about. So simple one, but weekly active users, it’s going to be driven by the number of visitors, which are going to come from all different places, what percentage of them sign up, what percent of them activate or habituate, and I’ll talk about the difference in a minute. And then this is all the work you can kind of do to drive those things. And some of these are going to be drivers. And some of these are going to be key drivers. So you’re gonna go through and say, right now, which of these, and I’ll show you how you’re going to make that decision. But which of these many things we could be doing? Should we be focused on? And that’s really the problem, right? Is there’s just so much work we could be doing. So normally, I’ll go in and look at a company, and it’s sort of like, you just sort of work it through logically like, Okay, you have terrible retention and terrible conversions and do not pay for more traffic, that makes no sense, you know, or we have no traffic at all. So we don’t know if our landing pages any good. So we need more traffic to get data. So but generally, I’m asking, if this works, how big can it be. So for example, you might do work to shorten your sales cycle. But if you have the same number of leads, and the same sales conversion rate, but you shorten your sales cycle, from 90 days to 30 days, you’re gonna get a little short term boost in customers by moving the deal pipeline up. But ultimately, you’re gonna get the same number of customers, you’re just gonna get them like two months sooner. So that can’t really be that big. You know, if paid advertising is only 3% of your business and the paid opportunity, you could maybe double it, than that could only ever be 6% of your business. So it’s not that big. So just like, if this works, how big can it be? Can it be self sustaining? And by that, I mean, you know, is there a positive feedback loop in there, for example, you know, growth loops, customers, referring customers morality, if you’re doing paid advertising, and the return on investment is, is very high, then those paid ads will fund future acquisitions. Sometimes SEO and inbound marketing can be self sustaining. Anyways, ultimately, can it be profitable on a unit basis? Is it an input or an output? And by that I mean, inputs are the work outputs are the results of that. So typically, a customer behaviour, the key drivers need to focus on the outputs rather than the inputs? And then is it in harmony with your business? And by that, I mean, are you geared up to do this? Well, so for example, if you decide that PR, and content marketing are really important to your business, you need someone who can create content? Do you have a founder, who’s a visionary who loves to go do public speaking? Do you have a bunch of content already in your database? You know, like Francisco, maybe you’ve got average, you know, metrics for, like, what are good rates of, you know, efficiency for players in different sports, and you can publish that on your website. And maybe there’s a bunch of Google searches that are gonna go after that. So like, you know, maybe content marketing is a good strategy for you, or maybe your your co founder loves to write blog articles about data and analytics and sports, right? If you have a source of content, then a content marketing strategy is good for you. If your founder is a salesperson, and loves to hire salespeople, and you have a great sales culture, then sales is in harmony with your business. If you’re a technical founder, and you hate talking to people, and you hired a bunch of nerdy introverts, sales probably isn’t in harmony with your business, right?

So just realistically, there aren’t that many ways to grow a business, choose something, choose drivers, and focus on doing something that’s in harmony with your business. So can you go through now and think about what would then be if that’s your Northstar metric, what would be the key drivers for your business? And then, Gareth, I guess, then my challenge back to you is, are there key drivers that involve the sales cycle, and the stakeholders that will ultimately move your Northstar metric, but that you need to focus on more importantly, to hit your targets?

Matt Lerner 
Any questions on key drivers or numbers? You want me to take a look at metrics you’d want me to take a look at?

All right. So I want to talk about and I can share a copy of these slides. So you can go back and like, look at any of these. But one of the things I want to talk about one of the key drivers is this idea of activation versus habituation. So most people measure activation rate to Who here’s got like an activation rate, they can tell me a thing they measure

Mark Littlewood 
New customers. Oh, sorry, Francisco, you put the hand up

Audience Member
New teams. So, so not just the sign up of a coach into the platform or a play into the app, but when they actually form a team.

Matt Lerner 
Okay, so when you start getting data from multiple players on the same team,

Audience Member
Yeah, that’s correct. Yeah. Okay.

Matt Lerner 
All right, any other activation for enterprise software? Adoption is a bit trickier. But I think I’m like a Pay Pal when I was there. We said if you receive $1 of payment volume you’ve activated. But then we had, I’d say, you know, if 40% of our customers activated, only 20% of them would actually be active three weeks later. And I realised, you know, the first thing you do when you set up a payment system is you, you know, you grab your buddy’s credit card, and you send yourself $1, you do like a $1 or $5 transaction rebuy thing, just to make sure it works. And the test transaction does not an activation make. So when I work with companies now, I encourage them to distinguish between an activation and habituation. And by habituation. I mean, it is kind of what it sounds like, but you know, we’ve got people come in, and what are they going to do, like, they’re going to visit your site, they’re going to take some first action, they’re going to try the product, they’re going to muck about with it, you know, so in, you know, a booking out, they might just try making a booking or, you know, obviously, you know, for sports out there going to, you know, tracking out they’ll put on the kid, and you know, try to get some data, but they’re just playing around with it, it doesn’t necessarily mean that they love it. And then at some point, there’ll be some behaviour threshold, they cross, at which point, their behaviour, because we can’t actually read their minds, we can kind of infer, yeah, this is no longer a decision, this is now part of a standard operating procedure. This is now a habit, and they will keep using the product.

So this is, here’s an example from a company I worked with previously. So you want to get them across that kind of aha moment. Sometimes actually, it’s like a pattern of behaviour. So this is cohort curves from a piece of software. And basically, what they did is they said, here’s all in this, like, doesn’t matter what software was imagined. It’s like QuickBooks zero. There’s a lot of stuff that you can do in there, you can sync your bank account, you can reconcile your month and reconciliation, you can download a p&l statement, you can file a VAT return. So they go in and say, okay, for people who you can add a new user, you can give your accountant an x. So of all the actions they took, on day zero, what is, you know, the month 12, retention or even the month to retention curve look like for people who took that action. And they found this red thing in the yellow thing, maybe that would be like, give your accountant access and file your month and got return. Those things are highly predictive of retention, the other actions are just playing around stuff just getting set up. And so you can go back and look at what behaviours that people need to use inside your product, to really show that they’re likely to keep using it. And I encouraged him to focus more on habituation as an outcome rather than activation, which I find can be a bit of a vanity metric. Because ultimately, if they don’t retain you have a problem. And more often than not, when I see retention problems in software, they’re actually activation problems. It’s more likely that the customer never actually got to a moment of value, aha moment in the first place. And that’s why they didn’t stick around. And the reason I, you know, a churn is not just a churn.

The reason I kind of emphasise that nuance is if you think about, like, if you go on a first date with somebody, and they never call you again, that’s not a divorce, right? That’s an activation problem. And you do different things to solve an activation problem, then you do to solve a retention problem. And that’s why I think it’s having this habituation threshold helps you figure out if you have a really have a retention problem, or if you have an activation problem. So for each of your businesses, once you’ve got your north star and your key drivers, I would go back and ask yourselves what is a habituation threshold for us? What early life behaviour pattern is strongly predictive of You know, long term retention, user retention. And that can be a really sciency analysis, you can do a multivariate regression, if you’re good at that kind of stuff I’m not. So I usually just take a number that I think makes sense. And just, you know, plug it in and run a filter and say, okay, of people who did X, Y, Zed in the first three days with the products, what percent of them are still active after 90 days. And that usually gets me close enough. And it’s just common sense. Just watching a few users use the product. Any questions on activation threshold? Habituation?

Audience Member
I’d like to, Francisco here again. Good. You have multiple activation, sorry, multiple habituation sort of thresholds.

Matt Lerner 
If you’ve got multiple distinct use cases or jobs, so you, for example, you might have a different habituation threshold for a coach versus a player. But a lot of companies, it’s more like, you know, this graph where there’s like, five things you can do with our product, but one of them is this sticky thing that really gets you in there. So, you know, I don’t know, for your business, maybe players are like happy to look at this number and interesting to look at that number, but really depend on this one particular metric. And the ones who discover that and incorporate that into their coaching are the ones who keep using it. So there’s lots of things you can do with Xero, or QuickBooks. But ultimately, if you’re not doing your taxes with them, it’s kind of expensive, a hard and messy software to not give you that value. Early lights, you know, it takes a while to necessarily figure out kind of what is that job to be done. Great question. Thank you. Thank you. Any others on one more key driver thing? After this?

Audience Member
But all right.

Matt Lerner 
I keep freezing. Can you guys still hear me? Okay, great. So the last thing I want to talk to you about in key drivers is this idea of the rate limiting step. So is anyone heard of the theory of constraints? The goal by Le Goldratt. Few of you. So this is an idea I just stole part and parcel from Goldratt. When I joined this refinery, my manager gave me a copy of that book and said read it. And I went home and read it. And basically, somewhere in this refinery. So remember, our job is to refine as much oil as we can as fast as possible, because we get paid by the gallon, and our costs increment by the hour. So somewhere in this refinery, at any given time, there’s a bottleneck, there’s a narrowest point through which the oil refining is the slowest. And it might be literally a pipe that’s too narrow, it might be that the trucks are not loading fast enough because the unloading bays are blocked. It might be that the temperature on a distillation tower is too cold or centrifuges are plugged up. That’s more than you ever needed to know about oil refining. But the point is, at any given time, there’s a bottleneck in this refinery, and it moves around. And that was my job actually, was to go around the refinery with a clipboard, and look at all the metrics and figure out where the bottleneck was. So I can tell the supervisor so he could go yell at the people to make the refinery go faster. So same thing in your business, at any given time, there’s a rate limiting step. And it’s going to be a little bit more metaphorical than the refinery. But it could very well be your retention, it could very well be conversion of visit to sign up. Or it could be the quality of the leads you’re getting. Or it could be the amount of traffic you’re getting. Or it could be a personnel issue, it could be that your team is moving slow, and they’re scared to take risks and make mistakes. It could be a mindset issue, it could be a process issue. Could be you don’t have enough engineers, quite possibly, you don’t have enough engineers. But somewhere in there, you’ve got a rate limiting step. And that’s a very good conversation to have with your leadership team. And the way to find it is just to ask yourselves, which thing, if we can improve it or kind of add resources to it, we’ll make the entire business run faster. And that’s the point of this book, right? If you if, if you put more centrifuges online in the refinery, but you don’t have enough oil to pipe through the centrifuges, it will make a difference. The bottleneck, the rate limiting step is the place where if you add resources there, everything else goes faster. And if you add resources anywhere else in the business, you’re wasting those resources.

So you can see where this is actually, like a super important idea. I’m guessing most of you didn’t know you’re you know, hadn’t didn’t have a sense of where’s your business’s rate limiting step before this call. But now that I’ve said that you can see where like, Okay, this is actually a massively important resource allocation decision and a conversation worth having.

Audience Member
Matt, what this reminds me of two is the flywheel. Right Jim Collins. So, I mean, he didn’t invent it. But the idea of you know, anything on the flywheel if you’ve got your correct flywheel, any one of those elements optimised is going to make the entire wheel turning faster. So this is kind of the flip of that. Right? Which, which ones need to be optimised?

Matt Lerner 
Yeah, and what’s not actually on the flywheel? Yeah. I think, you know, the reason that this idea is powerful is, in my experience, most people who come into a company, they are schooled in whatever their discipline is, they’re good at whatever you hired them to do. So actually, I think we have like, a company on here, who does KYC helps with KYC checks, is, are they um, that’s okay, well, we’ll pick on compliance people, especially since they’re not here. But like, if you work in FinTech, and you hire compliance people, they’re gonna come in, and they’re gonna think doing a good job is doing really good compliance and making sure nobody can use our product, unless we verified their, you know, their background and their identity and run these compliance checks. But in fact, compliance checks are really onerous for customers. So if you said, Well, hold on our bottleneck is monthly active users, and you’re trying to reduce the number of monthly active users we have, then they’re gonna rethink their job. And they’re gonna think, you know, if, if you’re doing this, right, they’re gonna think, okay, how can we onboard customers as efficiently as possible, while ensuring we remain compliant, which is a very different kind of mental orientation than 100%, compliance lawyer robot, you know, sort of default approach.

So everyone’s gonna come in, whether they’re sales or marketing or compliance, product managers are gonna want to do product engineers are gonna want to build stuff. Everyone wants to do the thing they’ve been trained to do, and they’re good at. But in fact, companies don’t need to do all those things equally well at once. And they’re each going to be pulling in slightly different directions. Good product managers, success metrics are different than good finance manager, success metrics, or good compliance person success metrics. So if everyone has a sense of what is this rate limiting step, they can sort of Orient, okay, I need to be a good product manager in service of this short term goal. Now, the other thing is, if you do a good job with it, the rate limiting step will move around, right, you’ll you’ll clear out one bottleneck. And just as soon as you do another one will appear I worked with a team and we improve their homepage visit to sign up conversion rate from 3% to 24%.

With one experiment, it was like holy cow. So then, okay, guess what signup is no longer your rate limiting step now we realise that only 8% of your new signups are activating. So now activation is your rate limiting step. Let’s go work on that. So just keep an eye on the number. And be aware that it might move around. Any questions on the rate limiting step? Alright, this if you do nothing else after this call, I think that’s a great conversation to have with your team, even just by asking him that question. Okay, so we’ve talked about three types of metrics you Northstar, the key drivers that drive it specifically one of those key drivers will be your rate limiting step. The last number I want to talk about is something called a nuanced metric. That’s a term that my friend Andy Young came up with, does anyone know Andy Young, some of these BoS veterans must know him. Anyway, he’s a UK based, brilliant software entrepreneur. And the point of nuanced metrics is they’re very important numbers to watch, but we’re not going to try to maximise them. Now, I think for most people who are competitive and successful, it’s very hard to look at a number and not try to optimise it. But given that 90% of your results are going to come from 10% of the stuff you do, you need to have a little bit of self discipline. So these are numbers you’re gonna want to look at every week. Sometimes they’re check metrics, like grow our acquisition, while keeping our CPA, our cost per acquisition below, you know, $70, or, you know, keeping our onboard as many customers as possible while keeping our customer service response times below two minutes to make sure we’re not overwhelming customer service, or keeping our you know, net promoter score above x. So it might be a check metric to make sure you’re not going off track, it might be a number that gives you a lot more information about the numbers. So for example, your conversion rate might move a lot, depending on your marketing mix. And if you’re getting more SEO traffic that might raise or lower your conversion rate compared to paid traffic. So you know, your traffic mix might be a nuanced metric.

But these are basically numbers that are going to either be like a check metric, or just explain variation in the Northstar or just numbers you want to keep an eye on to make sure that you know they don’t become bad. So I pick a few of these. Often once you start tracking your Northstar and your key drivers, you’ll start to see some nuanced metrics just like oh yeah, we Also every week, we seem to need to know this or that. The funny nuance metric story I have was when I was in the US with PayPal, they were printing out the daily transaction volumes for every major region around the world and putting them on the CEOs, the wall of his conference room physical sheets of paper. And you know, the UK was the second largest market. And what they would do is look at year on year growth by day, so second Tuesday in November, versus the second Tuesday in November of last year. And it was springtime, and they started seeing this huge year on year swings daily in transaction volume from the UK. And they weren’t because they weren’t seeing it anywhere else. So they in a panic, they called the head of the UK business and said what’s going on? Why are why is our business down 30% year on year from, you know, this, this Saturday of last year. And he said because it was sunny outside. The weather is such a big deal in the UK that if it’s sunny, nobody stays home and shops online. And if it’s rainy, everybody’s home buying everything online. says we got to check the weather that became an important number to watch for the UK business so that the executives didn’t panic. You learn crazy stuff when you actually look at the numbers. All right. There’s a quiz I can type your answers in the chat. First question. Should your Northstar metric be an absolute number or ratio?

Matt Lerner 
Great. Okay, that was an easy one. Next question. How often should you change your Northstar metric?

Matt Lerner 
Every few years, maybe roughly annual infrequently not often. So the answer is it depends if you’re Francisco. So if your pre product market fit while you’re still figuring out product market fit, it might move. But once you understand how your business works, unless your business fundamentally changes, your Northstar metric doesn’t change. So Amazon ecommerce marketplace, it’s probably still repeat customers. It probably should be. But Amazon Web Services. AWS obviously has a different Northstar metric. So you may want to revisit it annually. But unless the business fundamentally changes, your Northstar probably won’t change. Okay, next question. Is free cash flow, good Northstar metric?

Matt Lerner 
It depends. It depends if you’re a hedge fund. For a software business, I strongly advise against it. Because there’s just there’s too many ways to generate money revenue, that are not fundamentally going to grow the business. Now if it’s, you know, unlike the BCG grid, if this is like a cash cow business, and you’re just trying to, you know, like squeeze the last bits of juice out of the orange, maybe. But if you’re in a high growth startup, and you’re like really trying to scale customer acquisition, you gotta do that through customer value. All right. How do you find your rate limiting step?

Matt Lerner 
Is anyone typing an answer? We just stumped. Mark, what’s TOC? You’re on mute.

Mark Littlewood 
But Theory of Constraints? Oh?

Matt Lerner 
Yeah, so ask yourselves, which piece? So Tim, you got it? You got it right? Ask yourself which of these numbers if we could improve it would make the entire system run faster? And the other use really useful question on a key driver is if this works, how big could it be? Your marketer is going to want budget for like 26 different things and just say okay, if trade shows work, could we 5x our business? Maybe not? Like what So which of these can really have the biggest impact on our business? All right, next question. Is cost per acquisition a good key driver?

Matt Lerner 
Yeah, most people are saying no, I’m having a hard time imagining a world where if you just nail the heck out of cost per acquisition, your business grows faster. I mean, I think it could be an, it’s probably a very important, I’m sure it’s a very important nuanced metric. It’s definitely something you want to keep an eye on. The other thing is, there’s just so many ways to skin that cat, right? I mean, there’s targeting, there’s different kinds of creative, there’s different offerings, different landing pages, there’s pace of experimentation, like I can think of a dozen things you do affect cost per acquisition, that sort of across so many different pieces of the business. So it’s kind of a messy one. Okay, and I’d rather push people to say, what are the levers we need to pull to effect? cost per acquisition? All right, the last thing I want to talk about, is how to bring these metrics to life in your organisation, because most of you are working in or running big, pretty good sized teams. And, you know, the whole value of this, remember from the beginning, is we know that most of your results are going to come from a small percentage of the stuff you do. And so we need to help everyone in the company every day make really good prioritisation decisions about which work they do. You know, we’ll think about what are we gonna do with our budget, the beginning of the year, we think about headcount planning, but really, every time someone in your company opens their laptop, they’re making a resource allocation decision, especially if it’s an engineer, like what am I going to build today? customer service reps, you know, who am I going to be talking to on the phone today? Those are actually really important decisions for a company. So the goal of these of having these metrics is to help everybody understand the strategy, connect their work to it, so that they can make good resource allocation decisions every day, how they spend their time, their budget, what questions they ask, what numbers are trying to move. So first of all, you’re going to want to track these numbers and share them. If only, you know, there’s just Are there enough analytics tools out there in the world? I don’t know. Like, this is like one of the analytics tools in the world. So what do you use to track these? So the bottom line is like, the best analytics tool is the one you’re actually going to use. If you’re already wired into one tool or another, and people know how to use it, they’re gonna log in and look at it. That’s good. But the truth is, for most of this, our good old friend, Google Sheets will help us. But you got to track these numbers and share them with people. That guy Andy Young that I mentioned, he actually created this Google sheet, which you’re also welcome to grab a copy of, I think I put the link in the chat. And Kurt probably put it on the resources page as well. You fill this in your Northstar metric goes here, your key drivers go here. And then nuanced metrics go here. And then the funnels, these are the bits that lever up to each of the key drivers or layer up to each of the key drivers, you can fill those in below. And it’ll automatically calculate like week on week growth rates for you. So you’re welcome to grab a copy of that one. Wherever you do it. The point is, this only works if everybody actually looks at these numbers and thinks about them and has a conversation about them every week. So step zero is just actually tracking these. Steven, which video from dopamine you’re talking about? He’s done a few.

Mark Littlewood 
One he did at BoS Europe, I think

Audience Member
He did. In your newsletter, you had one comparing Mixpanel and amplitude. And I watched that this morning, is top of mind.

Matt Lerner 
yeah, he’s got Mixpanel amplitude are a good side by side comparison for their marketing, because Mixpanel is clearly a jobs to be done shop. And amplitude just loves putting lists of features on their website. And this is a bronies do exactly the same thing. It’s a great side by side comparison, because Apogee says you can make cohort curves. And Mixpanel says, find out which of your users are retaining. Like, that sounds more useful to me. So alright, right. So once you tracking the numbers, here’s the magic in this whole process. So the leverage is in which work you choose to do. And your goal is to have everyone doing the most impactful work. So make sure that every person understands the Northstar metric. What I literally do is go around to everyone on my team and say, Okay, here’s our Northstar metric. You can you tell me how your work affects this? Don’t I don’t tell them I asked them. And that forces them to think and usually it takes a surprisingly a while I mean, sadly, it takes sales and marketing people a little bit of work to figure this out. And surprisingly, you know, customer service people engineers compliance, it takes even longer, but by asking them this question, and getting them to sort of do this math in their head, you’re you’re fundamentally changing how they think of the business. And often they’re going to tell you stuff you didn’t know that it’s gonna surprise you that they know about their job. So just having that first conversation about how does your work impact our Northstar metric every day, super helpful. Then they’re going to draw those lines and connect those dots. For you, those intermediate dots of their connecting, those can become their OKRs. That becomes the work that they do. So then you say,

Okay, if we need to move our Northstar metric, which work should you be doing, and use that to set their goals. And you can use that information to make better resource allocation decisions. And then I asked people every week or month or whatever, are checking cadences. What’s the most important work you’re doing? And how’s it going? I remember when I first joined PayPal, I don’t know how I snuck in there, everybody else on a team went to Harvard or Stanford, or both. And they were like super overachievers. And we’d send these weekly end of the week emails and all the stuff we did. And it was like this insidious contest to see who could do more stuff. And the emails of list of stuff people were doing, were just getting longer and longer. And I just kept a notepad next to my desk. And every time I did something, I wrote it on the list so that I would remember at the end of the week, all this stuff I did to win this stupid little contest. But again, most of that stuff just didn’t matter. And so I don’t ask people, What did you do last week, I say, what’s the most important stuff you’re working on? And how’s it going? Because that reminds people that Yeah, you did your expense report, I don’t care. Like doesn’t matter. So use the KPI sheet and share it. And then resource at a top level resource your key drivers first. So start with the rate limiting step. And then the other key drivers. And those resources. You know, whatever fungible resources, you have cash headcount, engineering resources, the time and attention of your most talented executives might be an important resource, it might be a huge distraction, it depends on the people. But do think about how you’re going to allocate that time and attention, as well. So as you’re going through your whole list of key drivers, think about how are you going to make sure that you get those resources. Now, it’s coming up on the end of the year. So you’re going to be doing OKRs, which I realise is not to be necessarily uttered in polite company. And that’s not always people’s three favourite letters. But a lot of times the problems people have with OKRs are actually the thinking behind them. It’s actually that they haven’t figured out what is our Northstar, and what are our key drivers. And if you’ve got the top of your metrics, tree mapped neatly, the OKRs kind of write themselves, this whole process becomes a lot more painless. So ultimately, you’ve got your Northstar up here. And then you’ve got an objective under that. And then you’ve got these key results. And then you got the work you’re going to do to move the key results. And the important distinction here is like the key results need to ladder up to the key drivers and the Northstar. If they don’t move it, they shouldn’t make your results. And then the other big distinction is that it was chain work and outcomes. So a key result is not a unit of work. It’s not the number of sales calls I made. It’s not the number of leads I generated, right? It’s going to be some customer behaviour outcome. So how many customers signed up is a key result, you know, or if you’re doing SEO, it’s not the number of backlinks you got. It’s whether you increased your your domain authority, and you’re ranking for key terms. And there’s lots of ways to do that. So you want to push people to achieve outcomes, rather than to do specific increments of work. And hopefully, if you’ve hired smart people, they can figure out what is the right work to do to achieve those outcomes? And if they can’t, you want to know that sooner than later. Right. So this is a good way to find that out. Any questions on tying this back to OKRs?

Mark Littlewood 
I take it you believe in. Okay. As I know that’s one of those ceases to be a debate.

Matt Lerner 
I’ve always used them and not had a problem with them. But I have seen teams tie themselves in knots. I think it’s one of those areas where people can sometimes overthink things. And in my case, I think the problem is the overthinking not the OKRs like often people who overthink OKRs overthink everything. And that’s like that’s a bad culture thing in your organisation.

Mark Littlewood 
How should the incentives the financial incentives, maybe of teams be tied to OKRs versus Northstar metrics? You got an A, so some? Wow. That’s

Matt Lerner 
a great question. There’s a couple of questions. One is, you know, to what extent are people coin operated? Salespeople are very financially motivated. But, you know, there’s a lot of people who, you know, most people most of the time that’s not the main driver of employee engagement. It’s often that, you know, having that as a barometer as a scorecard is a good way to cause a conversation every year about we hit our goals. We did We hit our goals. But like this, I don’t know if you guys have looked at Twitter lately, but it’s all about Twitter. And there was an engineer and talked about, you know, here’s the unethical thing. I was asked to do a Twitter that I can probably finally safely talk about now. And it was, you know, they asked him to share some average tech and some user level tracking data with telcos. And he refused to do it. And he was so annoyed by it, that he quit. And, you know, his manager literally said, if I, you know, back to dump truck full of money up to your house and dumped it on your lawn, would you stick around at Twitter for another year? And he said, Absolutely not. Like, you know, this isn’t about money. This is about integrity, I can get a job anywhere. So I think it’s very important to understand what actually motivates your team first and foremost, and to what extent their coin operated. I like to split incentives and overall rewards kind of somewhat evenly between the overall company objective, and individual or team level objectives, because I want people to keep kind of one eye on each of those two goals.

Mark Littlewood 
Okay, thank you.

Matt Lerner 
For day to day conversations, I find that I don’t know, I have young children. So I find if I tell them things, they don’t listen. But if I ask them questions, that somehow, then that sort of forces him to think because they have to think of an answer. So these are the questions I use to reinforce the focus and prioritisation on the matrix. So how does your work impact our North Star? Which work? You and department head or, you know, manager or whatever? Which work should we do? And that’s often a longer conversation where I challenge their assumptions and the trade offs. And if this works, how big could it be? And is there you know, what are your risky assumptions? And how quickly can we test that? And then the weekly, that’s sort of a goal setting conversation, and then the weekly cadence what’s important, and how’s it going? And what have we learned? What Did We Learn This Week? That didn’t work? Fine. What did we learn? And then daily, because everyone always asks me, you know, okay, what should I do next? Or what should I do in this case? And even if I think I know the answer, I usually push it down and say, What do you think? And at least kind of get their thoughts and encourage them to come to me with ideas, not just questions? So these are the questions I asked to sort of reinforce this decision making. The pitfalls I see. Because I’ve worked with hundreds of companies on this. We talked about focusing on revenue, not customer value chasing too many metrics at once. Analysis paralysis, that’s again, like the companies that have trouble with OKRs, or spend months going around the houses on attribution of marketing spend, like there’s certain topics that I think people overthink, if you’re just like, look, we’re trying to do this stuff that has a big impact. Is this how could this have a big impact? Is it why not? Yes, or why not. And then I also see teams will become obsessed with a local maximum. So they’ll be like optimising conversion, and the winds start getting smaller and smaller and smaller, and they’re just narrowly focused on like the headline, or, you know, managing AdWords campaign spend. And so it’s good every three months to sort of take a step back and say, Okay, are we missing the big levers here? Let’s zoom out. Are we being bold enough? Are we being ambitious enough? Do we have some targets that we’re not sure we’ll be able to hit? Alright, so just a quick recap. Start with a Northstar that measures value delivered to customers, narrow your focus to one to three key drivers that will have the biggest impact on it.

Identify your rate limiting step, assign each of those key drivers to a single accountable person, make sure everyone can connect their worth of work to the Northstar metric, that they’re doing the most impactful work that they’re choosing the right key drivers, and they’re making good trade offs and investment decisions. I really hope this is helpful. The final warning I’ll give you is anyone here recognise this guy? Young kids these days, they don’t haven’t seen The Matrix, BoS matrix. In the immortal words of Morpheus, there’s a difference between knowing the path and walking the path. So I sincerely hope that you can actually take this stuff and measure it and get on the path. And my shameless self promotion. I publish a two minute new email newsletter almost every week, which you know, tech leaders and entrepreneurs find helpful and you’re welcome to subscribe if you’re interested. And then unsubscribe if you don’t like it, but um, you’ll get like an automatic welcome email when you subscribe, and that comes from me and you can actually just reply directly to that and then you’re emailing me, which may be something you might want to do if you have more questions about metrics.

Mark Littlewood 
Brilliant, man, thanks so much. I’d love to open up to a couple of questions. I have a few that To does anybody have anything to?

Matt Lerner 
I’m can look at specific numbers and help challenge them

Mark Littlewood 
Yeah, I mean, I’d really love to dig a little bit into the education piece. And Gareth, you were talking earlier on about not having a North Star and Alison, and Sheila, Angela, and Anna, are all in sort of similar situations. And you can almost abstract that out into other places or other other types of customers where it’s not entirely clear who makes a decision on purchasing or buying. Girls, did the rate limiting conversation, help your thinking?

Audience Member
Do rate limiting thing didn’t because I’ve got a degree in chemical engineering, I’ve spent plenty of time on all refineries. So that particular bit of it was was was was very, I think, yeah, I think the interesting thing here is, you know, it’s a complex system. And it has a number of different influences. And we’re looking for signals from that complexity, to give us something to orient around and engage our organisations with, right, you know, what are you doing getting out of bed every day? What are you doing to think about getting making this number go up into the right, and that is very, very helpful. And I think the challenge comes when we’re in these complex business models, and I think it’s particularly where the sales cycle is very long, or it’s maybe on an annual budgeting cycle. It’s driven by factors, some of which, most of which are not under our control, certainly the factors, but we can influence how we are perceived in the context of those decision making. framework.

So being very specific, the company that I’m thinking of produces tools to, to help learners in the classroom to take notes well, and particularly, this is relevant for students with disabilities, who maybe struggled to take notes and follow the lecture at the same time, or maybe they’re trying to take notes from an online lecture or a video. At the same time, occasionally, the produce really sort of fantastic tools to help, you know, those students to cope in those situations. Now, the majority of their income comes from sort of Disabled Students Allowance disability services within universities, right, their users getting value from the software is something which will influence those people in their purchasing decisions, and their decision to shortlist this product and make it available to the students. But that’s coming from a different sort of pot of money. And the pot of money exists in these departments and exists for something different, right? I mean, the university kind of care, that the universities that the students are getting better at learning, what they really care about, is that the retaining the students and their fees, I mean, that’s very simplistic, but you know, the numbers that those people making the purchasing decisions on a caring about are not about individual students creating a generating value. It’s about how they are perceived as being spending their money well, that has been provided for them to support disabled students. And, you know, the, the total pot of money is established sort of 18 months earlier for a particular cohort. And then individual students arrive and all of these things, right. So the point is, we understand these?

Well, we don’t understand, but we recognise that we’re dealing within these within these complex environments, and trying to understand how to influence them. And I think it just gets really super hard. When we try to boil it down to that, you know that one thing, because you’ve got, you’ve got to go to market from Target sales and marketing function, that trying to do something which is disconnected directly from the process of creating value through the use of that software in the hands of that user. I would love I would really love for these organisations like the one that I’m talking about for them to walk in through the doors every day or fire up their computers every day, see a number on a graph something move up and then say for them to say, yes, we recognise that represents the essence of what progress means for us. And I can do that thing with the OKRs are I can do that thing where somebody can come up to me and say, What are you doing today? And what have you achieved today? And how is that contributed to that? That number improving? And it’s super hard. It’s remained super hard to do that in these environments where you’ve got these kind of complex business models.

Mark Littlewood 
Does that does that ring true with Anna and Alison?

Audience Member
All the A’s in education I think it doesn’t meet. And also what I’ve sort of taken from this talk is maybe how our licencing is in opposition with these key metrics as well. That so if we, you know, we do bookings, well, they may need to pay more for bookings that might cause a disruption with the customer, when really our priority is retention of the of the customer. So it’s kind of given me a new lens to think about those, how complicated we’ve kind of made it for ourselves. And, you know, I’m sort of fit here envious thinking, Oh, I wish I was just in a simple startup, rather than a company. It’s been running for 20 years, and had lots of legacy stuff. So I’m trying to kind of market trying to exit out on so we can get streamlined. But it always feels easy, doesn’t have to tear it all up and start again, from experience, right. But yeah, I kind of, we’re not a simple, have a little go online, sign up, use it, hope you kind of keep those customers, it’s a more complicated sales process. And the retention is, is complicated well, because of kind of our licencing model. And so it’s given me a fresh lens on all of that. But no simple answers, and no numbers really jumping out at me.

Audience Member
Yeah, I would totally agree. It’s retention for us. And for all the reasons you just said, you know, enterprise is hard B2B. And then enterprise education. It’s why there aren’t a lot of investors in this space, right? It’s what there’s not, it just becomes a whole different beast. But I agree being here. And listening to the perspective, it’s a little bit more B2C or more pure SaaS is really fascinating, because I think that’s important to keep our minds open and not think that we know exactly how our business works, maybe there is some opportunity that to kind of have some fresh thinking and identify some rate limiting factors that we might not have considered before. So.

Mark Littlewood 
Alright, thank you, Matt. Would you

Matt Lerner 
Francisco’s got his hand up? I don’t know if he just wanted to say

Audience Member
asked Matt, if you had any thoughts? So obviously obese is a seasonal with all the festive seasons, etc. But we’re selling technology to sports teams, which run on season in how can I can I set drivers knowing that I know that is the middle of the season and the offseason period, where people might not be thinking about any of the stuff that we send it to them.

Matt Lerner 
I think you’ll you’ll start to learn what your seasonality is, once you’re up and running. Typically, in a seasonal business, you’re going to want to look at year over year growth rate to compare to prior period. So like, yeah, in travel, they’ll look at you know, Easter break, you know, half term break this year versus half term break last year and ecommerce, they’ll look at, you know, Black Friday, Cyber Monday, year on year. And then another thing I mean, from startups is they’ll sort of look for, to acquire customers in to sort of counter seasonal markets. So like, you know, maybe football teams in Australia and England because, you know, summer winter are switched and therefore, you know, you’ve got kind of year round customers. I wanted to also respond, I think an Allison and Garrett’s point. I think I forget, he’s I haven’t worked in a big company in a while. But I think most companies, there is a tendency to overcomplicate things. And I don’t mean to say your businesses are not complicated, but and it’s never ever, ever easy to simplify and get people to say, well, this metric doesn’t matter. And I’m gonna stop worrying about this. But I do think there was a lot of work and a lot of hard thinking there is some simplicity to be found. And if you find it, it can be incredibly powerful. I mean, yeah, every time we tried to do it at PayPal, there’d be some, you know, there were always edge cases that didn’t fit. And there were people who get annoyed by it, but most of the time, it helped most people make better decisions. But it’s hard. It’s hard, especially Yeah, like, you know, go with your chemical engineering background. You probably see this as a systems map in your head, in all of its nuance. It’s like how do I Okay, how do I explain this to someone who wasn’t a chemical engineer, and really only can think about three things in a day.

Gareth Marlow 
I think that’s the curse of the of the system thinker, right? I mean, you try and you see all the all of these connections. And it’s just like, Okay, how do I download the matrix for myself for everybody else, because you’re not in my head. And I think just even having artefacts that will help communicate some With this stuff because it’s not necessarily about the complexity of it, it’s about the interdependencies. And the fact that you can’t just reduce it down to A causes B causes C, and it’s just kind of one linear thing, right is there’s a load of feedback loops. And there’s a load of emergent properties that come from these things interacting together. And that’s what makes it really interesting. But it also what makes it difficult to understand how to influence it properly. And also difficult to communicate that stuff to other people. I think the best way I’ve ever found is drawing pictures, right? I mean, the second we start opening a Word document with a blinking cursor in there, it’s going over that it’s very, very, very difficult to communicate. But when we are drawing pictures, at least, there is something in the complexity of what we’re able to represent in the diagram that is just impossible to put into words. Yeah,

Matt Lerner 
I agree. A diagram with feedback loops can be super instructive. I think, Mark, we did that once. We took out Miro and just kind of mapped it all out. Yeah. And the other piece of that, I think, once you’ve got your key drivers in your nuance metrics on a KPI dashboard, if you’re looking at every week, you’ll start to see those things, hey, we did this thing. But there’s other number of changes. I wonder why. And so you know, and it’s one thing for you that, you know, the engineer genius to understand all those complexities, but when everyone sees them every week, then the whole organisation starts to learn that stuff. It can be helpful, it’s not infinitely complex, you know, but it is a learning curve for everybody.

Mark Littlewood 
How much of that? When you’re setting your Northstar metric, and you go through your metrics, and you work out what they are, how conscious should you be of driving external value for the business? Or is this about you, you’re assuming that there’s an understanding of growth is good?

Matt Lerner 
by external value? Do you mean money? Yeah. Okay. I take it as a foregone conclusion that if you can, and I realised this isn’t always true, then if you can get really good customer engagement, you can make money. In SAS, that tends to be true. Ecommerce, it’s a little bit harder in gaming, it can be quite difficult. But ultimately, right, there’s just like five or six business models, right? Subscription marketplace transactional. And so it’s not that hard to connect the dots between like super excited, engaged users, and money. It can be screwed up companies screwed up all the time, especially if you choose a monetization incentive structure that’s orthogonal to customer value delivery, for example, or independent of it. But to me, that just feels like a much simpler challenge. And a much less common mistake that we haven’t figured out how to monetize this. Then we don’t know how to get lots of customers to use and love our product.

Audience Member
I would say for the space that Anna and Annie and I operate in that’s the upside right of these really slow decision, long sales processes. All these things is these folks are very unified in terms of their own community, they talk to each other and once you’re in your sticky, so like you say, if you can get really good and customer engagement, you can make money that becomes at least in our experience, kind of a virtuous viral effect, because sometimes that’s that’s what they’re looking for is what are our peers using? You know, great, well, did they take that that carries a lot of weight. So building that reputation, that reputational currency certainly is isn’t driver for us once we’re in here.

---

---

Next Events