Scott Eblen on Debunking Product Platitudes

At BoS Europe 2017, Scott shows why successful product management requires a degree of lateral thinking and a willingness to challenge or ignore conventional ‘wisdom’. Drawing on his experiences at Google, Nutmeg and Twitter, he will talk about how some of the most successful products are developed from a place where orthodox thinking is abandoned. Just because everything in an industry is done in a certain way, that doesn’t mean it is the right way. Scott shares some actionable ideas to take back to your business that will help you develop a product management framework that works for your situation.

Find Scott’s talk video, slides, transcript, and more from Scott below.

Find out more about BoS

Get details about our next conference, subscribe to our newsletter, and watch more of the great BoS Talks you hear so much about.

---

Video

Slides

Debunking Product Platitudes: Scott Eblen from Business of Software Conference

Transcript

Scott Eblen: All right! On with the show. Just a bit of background. I started my career at Microsoft working on enterprise software building SharePoint portal server. Then I worked with Mark at library house which is a data service in Cambridge. Highlight of my career, definitely! Then I spent 6 years at Google working on some very successful products. I launched the first version of Gmail for IOS and IPAD and some not as successful products like Google offers which was our failed attempts to follow Groupon. Which thankfully we didn’t buy and thankfully we shut down that product.

Previously I was at Nutmeg which hopefully some of you have heard of, but it’s like wealth front or betterment in the US. It’s an online investment manager, it’s a VC backed start-up that’s about 60 people when I joined and 80 people when I left, has about a half a billion pounds under management for the everyday consumer who wants to invest. And currently I’m director of product for a video initiative which started with our effort to broadcast the National Football League in the US and has continued through a variety of premium broadcasts through to just most recently we were broadcasting the BBC and some of their political debates this weekend. So that’s who I am.

Always be shipping

And let me get on to some of the product platitudes that just don’t feel right to me. Always be shipping. This is something I’ve said before and many people believed speed of shipping and execution is absolutely critical but unfortunately, in the pursuit of getting software shipped, sometimes people make mistakes and lose focus on what’s really important.

One example from my experience was Google+ at Google. This was an effort that was initiated who was previously focused who switched to the social initiatives at Google and one of the things we was known for internally was speed. With the launch of Google+, he set out to refocus the company on addressing the social opportunity and he did that, he talked about it in the context of there’s this new wave of social which is coming over the industry, Google needs to adapt and we need to launch something in 100 days. So the initial prototype of this was built in 100 days and in the second 100 days, he said I want to have 100 launches on the back of this. So there was this incredible focus on shipping and as we all know, Google plus doesn’t really exist anymore. I think they lost the focus on having a quality software product because there was an emphasis on speed of shipping. They used this image of the emerald sea painting and Vic was always getting the team to run faster and get in front of that so it doesn’t crash on us.

Always be learning

So if speed of shipping isn’t the priority I think what’s actually important is speed of learning. Let me show you how this was applied in some of my experiences. At nutmeg, we had a relatively small engineering team and it was incredibly important to make sure we were focused on shipping the right things and the only way we could do that was to learn about what our customers were actually doing, how they were using the product. Understanding what they were interested or not interested in. These were some analytics tools we looked at to understand how people moved around the twice. And the mantra measure twice, cut once applies here. If you can measure, you will ship better products in the long term.

Another important thing to understand about learning is you don’t need to look to your industry or product or business to learn. There’s plenty to learn outside of your world. Looking at popular science or academia. The product team at Nutmeg spent a tonne of time pouring through Daniel Kahneman’s behavioural finance and behavioural economics and there are tons of really interesting insights which are applicable to building and shipping great software.

Just to put one little example, this calculation right here, if I ask someone in 10 seconds to throw out a number, what do people guess? 10000 and let’s try this. What do people think this multiplies to? Any suggestions? You guys are a bit smarter. But if you take an average group of people, they estimate oftentimes estimate in about 2000 and the second one they estimate it closer to 500. You see they’re the exact same numbers and multiply to the same thing. The principle here is anchoring and that’s if humans if they see a large number first they assume it’s bigger than if they see a small number first. With a simple insight like this, Nutmeg was able to do some interesting things with the product. So for example, if you gave someone the chance to contribute £400 a month or £100 a week, that came out to the same thing but that anchoring effect of seeing a smaller number upfront made them more likely to contribute at those levels. That’s just one of dozens of insights that are available from behavioural economics and finance.

Can you be learning faster than you’re shipping?

I think a really important thing to look at is if you find yourself in a situation where you are focused on shipping, it’s worth taking a step back and thinking are we learning as fast as we are shipping? If not, take some time to understand the product. Look at the data, look at your users. User research and concept testing are incredibly important. You can get a ton of information out of this and it can be relatively quick. The concept of concept testing was something that was not familiar to me until I started working with a marketer who had done consumer packaged good product testing. And in that case, they had commodities. So like soap or tooth paste. They are all effectively the same, but the product for them is the messaging and how they describe and the words they use to talk about their product. So concept testing is a great, very fast way to try different ways of talking about the product and see how that resonates with customers. Again, it’s a very fast way to learn.

Finally I think if you could look at Academia or popular science, there’s a lot to learn about insights which will be useful to your product or company.

Innovation is key

Innovation, innovation is key. We’re in a hi-tech industry, everyone loves innovation and gets really excited about doing innovating things and everyone wants to be the most innovative company. It sounds wonderful but an overemphasis on innovation can be incredibly ineffective. One example from my career – does anyone remember Google wave? A few hands. This was a product which was very innovative, it was supposed to be a new communication mechanism. There were some interesting and innovative solutions for collaborations. This ended up being the basis for the Google docs collaborative software. But the team that focused on this was so interested in getting out an innovative product that they weren’t able to execute on the really important things about building a good, effective product for customers. And that lack of execution of converting these ideas to something people wanted, I think it was part of its downfall.

Execution is key

Rather than focusing on innovation, I think it’s important to focus on execution. One really interesting example that I’ve seen of this recently about the value of execution came from a book called social physics, by Alex Pentland. In low resolution there. And that was a book that featured a lot of research from MIT and one of the research studies they covered was about the social trading side ‘etoro’. And the basic concept of social trading is you have the option to buy or sell stocks and you can choose to share those with other people and copy what other people’s trades are. And they looked at millions of trades that occurred on the platform over a several month period and one of the observation they made – this is low resolution – on the bottom is the percent of trades that are copied and on the left side are the returns of those investments. What we see here on the left side these are people who are very unique. They are not copying anything, they are being innovative in their investment strategies and we see their returns are below the market returns for that period.

On the right we see people who are copying everything. They’re kind of in an echo chamber. They are doing what everyone is doing and there’s nothing innovative in their investing strategy. In the middle there’s this sweet spot where they’re doing a lot of copying of other people’s trades but they do a few innovative things on their own. And the team that was looking at this ran some analysis on this and some other similar types of datasets and they determined that in a lot of these simulations and situations, a strategy of 90% of copying what other people are doing and 10% of doing something innovative was the optimal solution for getting the best possible returns.

I think this highlights why it’s important to focus on execution. Because if you focus on innovation, you focus on 10% and there’s this 90% of copy/paste straight up execution which isn’t going to be the focus of the business.

One of the ways that – sorry. One way we looked over this focus on execution at Nutmeg was we recognised that we weren’t alone. There were plenty of businesses that are doing exactly what Nutmeg was trying to do and we didn’t need to reinvent the wheel. We spent some of our user research not on our own sites, but we sent user researchers and did user testing on our competitors, and there was a very cost effective way to quickly in parallel understand what was already working in the field and then just spend a little extra effort on innovating on top of that. So figure out what’s the best of what’s already there.

Another thing we did is we spent a lot of time on innovation in execution. Rather than trying to build something radically different, we were thinking how we were building things. For our conversion funnel, getting people to sign-up, we reformed our teams. Rather than having the traditional product team, we had a cross functional group of a single marketer, designer and product manager. They were 100% focussed on specific aspects of the funnel. By having this unconventional focused team, they were able to get 40% uplift in conversion through the funnel during the 12 months we had this implemented.

Hopefully you can see that we’re in an innovative industry and people love innovation but it’s really critical to focus on execution and as you’re executing, think about how you can experiment with that, to try different ways of executing. Be more efficient and get your engineers, product managers and designers to work in new ways to do it more effectively. And finally, part of the execution is looking at how other people are executing. So looking at the competitors and learn from them. There’s no shame in borrowing ideas and concepts.

Ideas come from everywhere

This is something that I’ve spoken about previously and I’ve used in previous talks and it’s only recently I’ve rethought a bit about this mantra. I think that it’s not very effective and useful because everywhere it’s just a throw away concept. That’s not actually a meaningful substance of suggestion for how to come up with ideas for your product.

So how can you come up with ideas? Well, I think to understand this more it’s worth looking at a very old product development team, or product development innovation. It’s a UK innovation but it goes back to the 18th century. And one of the products they had at the time was a shot gun. They had the little metal balls. And the typical product development process at that time was to have moulds or casting of these little lead balls. And it turned out that was error prone because as they were pulling them out, they weren’t spherical or the original mould weren’t perfect so the shot wouldn’t shoot straight. There was a product with a problem and they were looking for ideas to solve it.

And there was one man in particular, William Lots, who wasn’t in the firearm industry or the shot or lead pouring industry, he was a plumber. And he was aware of the fact that shot wasn’t very good at the time and one day when he was outside, or so the story goes, he was outside in a rainstorm and he noticed that the raindrops falling were perfectly spherical. And he was observing nature and said I wonder if we could use that insight about why raindrops fall as perfectly spherical to come up with a perfectly spherical shot piece of lead. So he did some experimentation and he eventually discovered that if he dropped molten lead from an incredibly high height into water to cool it, it would end up as a perfectly spherical shot. This is interesting cause this is his house in Bristol where he added a 3 story extension. No planning permission at the time. You can see a diagram how he built it. He put this huge tower on the back. Built a huge well underneath that and was able to drop shot which created perfectly spherical shot and he ended up making a ton of money out of this. And he got round shot out of that.

So the insight here is that he wasn’t in the industry, he wasn’t expected to be solving this, but he looked around him, he looked to nature and came up with a really good idea about how to solve that problem. So I think a really important thing as you’re thinking about coming up with new ideas is to be structured and looking around you for ideas on how you can improve what you’re building. Thinking back to my previous comment about academia and behavioural science, there’s plenty of examples on how plenty people are looking at and understanding how humans behave, around money. One application of that within Nutmeg was the social proof. Hopefully you heard that but the basic concept is people are more likely to do what other people around them are doing. A simple change to our sign-up page that includes a map that pinpoints where the users were saying joining investors from all over the UK. That simple change had an 8% uplift in the number of people that created accounts on this particular page. So we looked to what other people were saying about how humans behaved and applied that to our product.

Another example was when I was working on Gmail and the first version we had on the iPad. And we did some sort of paper concept testing and we were facing the challenge, figuring out how to do multi-select on the iPad and what we show on the right hand side. And as we were doing user testing with paper cut outs, one of the subjects sort of shuffled a lot of papers together and said I want to grab them all together in this way, and so our designer said that’s something we can do. They took that and made this shuffled card view to collect all the messages which had been selected. So that’s a really simple example but if you look at how people physically react to a proxy for your product, you can get some interesting ideas. This is another example of paper concept testing which is the same kind of thing that yielded that solution for Gmail.

Formalising Idea Capture

So formalising idea capture is really important. This can be a lot more effective at helping your team and your company come up with some ideas. There’s been a lot of research recently about associative thinking so finding ways to apply concepts from one area in another. One of the most effective things that you can do to help encourage that is just to look beyond your industry and look around you. When people ask me for ideas about how can we be more innovative? How can we come up with ideas? What are some books I should read? One of my first suggestions is read some fiction. Get out and think about things unrelated to your industry and that will spur some interesting thoughts.

Own the Roadmap

Owning the roadmap, so this is another concept which I hear a lot. And there’s a lot of emphasis on making sure that you’re articulated an incredibly clear roadmap that everyone can see and understand. Oftentimes, this yields a very exciting gant chart or a huge Trello board. As I spend more time with a variety of different companies, I think this is not terribly effective to focus on a roadmap. I think it’s most important to focus on a story. It may not sound as complete or as complex, but it think it’s a lot more effective to get a team aligned around a particular goal.

To share how I did this with Twitter, when I joined, we were starting the effort to launch a product for the NFL and many people in the company were asking why are we doing live premium professional video? It’s the antithesis of the short, concise brevity of 140 characters. And we had this huge list of things we wanted to do, but we were struggling to get internal buy in and keep the team focused on what was really important. We came up with a story based on the research that we’ve been doing with customers and the user research was unequivocal and it was very clear that watching video is essentially a commodity. Watching videos online is essentially a commodity, you can do that on Facebook, YouTube and any number of different providers. But what people really crave and miss while watching video is the roar of the crowd, the conversation that goes alongside of it. Essentially watching video online now is kind of like watching in an empty stadium. It’s not very exciting and doesn’t feel very authentic. So our pitch was we were gonna bring back the the roar of the crowd, the conversation. We would give you what everyone was saying about that particular event. And this immediately resonated with everyone. They understood what our competitive advantage was, that they needed to think about how the conversation was gonna augment what people were seeing. Very quickly people aligned around that and we extended this and realised that sport alone wasn’t an area where this was needed. So entertainment is a lot of topic where there’s a lot of conversation on Twitter. While it’s great to go watch someone on the red carpet, ultimately what people really crave when they watch these types of events is the hundreds of perspectives of the fans who are around them or the different viewpoints on what’s happening. And that’s something Twitter was uniquely positioned to deliver.

By telling the story of the fact that conversation was a unique competitive advantage and tying it to concrete examples, we were able to get a lot of buy in and get a lot of focus for the engineers. They ended up yielding a wireframe of the product with the video on the top and the conversation alongside of that.

That was back in the autumn of last year and obviously in January of this year I was sort of gratified, sort of mortified to discover that the concept of the story of the conversation and the roar of the crowd resonated with a lot of people. When I saw someone was very concerned about crowd size. That story worked, it does resonate with people.

Own the story

So, if you find yourself in a situation where you’re really focused on the roadmap, I think it’s important to step back and say what is the story behind that? Because ultimately, having a story that everyone can remember, your board member aren’t gonna remember a 300 point list of tasks to look at, but they will remember a story and even more importantly, it’s something they can tell their peers and friends and it makes a great way to explain to everyone what the company is focused on. One other highlight is if you can add or incorporate elements of the rule of 3 of that. Just the aspect to 3 points to what you’re saying, that can oftentimes make it more memorable or more focused for people.

Culture Counts

Culture counts, or any number of culture specific aspects, I think especially in software companies, there’s a lot of focus on what is the culture of the company and what are we doing to foster this? I think people have sort of moved beyond the notion that the culture is about free beers or ping pong, but to get to something substantive it can be difficult and people don’t understand what it takes to get to something meaningful with the culture of the company. One of the things I think can really help with this is to encourage a conversation about culture tradeoffs. So have a specific conversation within the company about what you are willing to sacrifice or aren’t in the culture. That can make a huge impact in terms of defining what makes the culture of the company unique.

Team vs Family

There’s – Netflix is well known for their culture and they’ve got a huge, 200 slide about what makes that culture. But one of the things that really resonates in that and it spurs a really interesting conversation is are we more like a team or are we more like a family? Hopefully many of you heard that conversation before, but this was a conversation at the management team Nutmeg had explicitly. And it was incredibly eye opening to see how different people on the management team felt about this. Everyone is aware of low performers in a company, there’s always someone that needs a bit of help and the question on how long do we work on nurturing or how quickly can we bring someone else in – can really bring to the forefront how people think about culture. That’s one example of an aspect of culture. Thinking about how sort of supportive and how you foster each other.

Another specific example of culture is having hard conversations about trade-offs that you are making. When I joined the product team, it was 10 guys and we had a bunch of headcount open and we had a sparse line of candidates. And we explicitly had the conversation internally about do we want to get someone in ASAP so we are not understaffed or do we want to wait a bit longer, the team will suffer a bit because we’re understaffed, wait until we get more diversity in candidates. These are hard conversations to make, people shy away of putting these things in writing, but ultimately it encouraged some thoughtful discussion within the team.

I think a good way of thinking about culture and making sure you’re focused on important aspects is have a conversation about what kind of trade-offs you’re willing to make. Is it diversity of opinions versus speed of getting additional employees in or any number of factors. A way to think of vetting is, is this substantive enough? Is asking is this something we want to talk to the board about? They don’t care about ping pong or beer but if your team is a family, that makes a bit of a difference on who the company hires and retains and probably is something that a board member could provide some perspective on.

Be data driven

Be data driven. In the big data world, there’s lots of conversation about using data and emphasising data. And in my mind, being data driven is all about recognising the data is objective. It doesn’t lie and gives you a direction to go and numbers to focus on. And the problem with this approach is you lose connection with the underlying people or behaviours of the product.

One of the memorable mistakes we made at Google when I was on the mobile team was we had essentially a weekly major review and then a sort of daily follow up on any metrics that looked like they were significantly off. Very early on, the mobile team started seeing a big decline and everyone scrambled because numbers were always going up at that point and they isolated this to some of the e-station markets and then they saw this was in Japan. Everyone was scared we launched something that was culturally offensive or there was a translation error and we spent a lot of time trying to dig in and find out what was happening. Eventually we spent time talking to people at the Japan office and they said it’s golden week, which is this string of holidays which happened and people just go away. It’s like other holidays in other countries where there’s a natural decline in usage. We were so focused on the data at that point that we lost some cycles because we weren’t connected to what the users were doing. I think it’s important to be data savvy and then that means being willing to ignore data at some points. It also means being really thoughtful about what you’re expecting from the data. Don’t just collect a bunch of numbers and assume that this will tell you where to go. You them mindlessly follow the numbers.

One of the things that drove this home – the final product I worked on at Google was something called financial search ads. So the search module, the results page and this is a screenshot from one of our internal tools called RASTA which measured the success of various experiments. While this looks like a bunch of numbers here, what you can’t tell is there are 100s more metrics that we recorded on all of these experiments. And very quickly I realised that anytime you run an experiment, you could come up with any results out of that. You could tell any story with any of those data points so it was important going in to have specific assumptions about what you expected – to be good or bad – and if it’s drastically different you have to dig in and understand is this behaving dramatically differently or do we not understand something around behaviours on this particular feature?

Another really interesting experience with data and being willing to ignore data came through my efforts to launch the NFL product at Twitter. One of the things I saw early on was that the experiments that we ran for the broadcast of the NFL was that they were all not statistically significant and that was frustrating because we wanted to prove that the $10 million we spent acquiring to this and the advertising was making a big impact for Twitter. If you think about it, it makes sense because the NFL was essentially 2-3 hours a week for these games. A typical experiment at Twitter will run for weeks at a time. The data didn’t add up, but ultimately we had to step back and look at some of the qualitative feedback about these experiences, looked at different ways to measure this. And we made the decision to continue to push forward with the live video initiative even though we weren’t getting statistical significance on some of these experiments.

So in terms of being data savvy, don’t just mindlessly follow the data. Be really thoughtful about understanding the underlying trends, be upfront about predicting how you think things will impact your numbers and there’s certain times when you should be willing to accept a bad outcome or an outcome that is not significant. Just being thoughtful there will make a big impact.

Think about scaling

The final thing I want to talk about is scaling. In software businesses, everyone is focused to make sure you have a scalable outcome, that you can reach the next billion users. And I think this sometimes leads to some blind focus on automation and software solutions over aspects which are more human. The better way to think about this is saying think about scaling later, recognise that’s something the company will have to do it eventually but it’s not something you should focus on upfront.

One example of this was at Nutmeg. This is Nick who was our founder and CEO while I was there. Nick was an incredible spokesman, he was also a very sort of big voice on personal investing, he’d go and talk on CNBC and BCC about various changes in the markets or investment decisions. But what I really loved about Nick was that he would still come back to the office after being on TV, he’d be all dussied up with his suit and tie, and if we were incredibly busy on the customer support lines, he would happily go there pick up the phone and answer customer queries. Which might be something stupid like I’ve forgotten my password and can’t log in and you have to run through some security questions. And what I love about this is he could have spent the time to hire a bunch of additional customer support personnel so we had the extra bandwidth when things got busy, we could have spent time with the software team building more automated solutions but ultimately we realised it was most effective to take advantage of the people we had around and not over-index on scaling too soon.

Now it may be easy to think of course, startups and small companies they have to do things manually at first. But that also applies at a big company like Google. When I was at Google on the Google offers product – this was a screenshot of an internal tool for creating the offer so it essentially was coupons that merchants would come in and create. Just before we launched in our first markets, Portland and San Francisco, we realised there were some quality issues. This was a coupon that some merchant had that said free big lot parking when you save – which I had no idea what they were thinking but this would appear in Google Maps next to the business. Google could have gone ahead and written some sort of error checking into these tools to make sure we didn’t have these crazy messages coming out. But ultimately it made more sense in the early days when we only had a few thousand offers to just manually review. The team on a weekly basis would sit down and manually flag these things that looked like they were totally wrong and that helped us focus on putting the right efforts into building tools, but not building too much so we didn’t waste a bunch of effort on scaling too soon.

Another really interesting anecdote about waiting to scale comes from Brian Chesky of Airbnb. He said that early on, one of the things they struggled with was getting time with customers or being able to talk to customers about what it was like to welcome them into their home. So one of the things he did was he essentially offered free photography of the homes that were put on the site and he would actually go and take these photos. So maybe the first photos weren’t great but they were taken by the CEO of the company. That’s definitely not scalable to have a CEO going out and speaking to individual customers. He said it was incredibly insightful because he built this sort of human connection and when they were scaled, there was no chance he was gonna have any opportunity to go and talk to these hosts but in the early days he did and he felt like he learned a ton more about the customers because of that. So, it’s important to scale gradually. Take advantage of the fact that you can be hands-on, don’t build a bunch of automation upfront and then also make sure that you focus on understanding what you’re automating before you need to actually automate that.

So looking back over some of these ideas about building products and building software teams, I recognise that there are many things up here which look like they’re pretty important and sexy and critical and there’s a lot of discussion of things like shipping really quickly and be data driven and use lots of innovation. But the reality is software has been built for a long time before a lot of these concepts were necessarily packaged in this shiny new way. I think ultimately one way of looking at this is just recognising that building software isn’t sexy. You have to be pragmatic, thoughtful and in some cases, you need to rely on some very simple concepts. Things like storytelling and perhaps being a little more slow and more thoughtful about what you end up doing.

That’s all I got for today and I think I have a bit of time for questions. I’d be happy to follow up on any of those points that I shared.

Q&A

Audience Question: Ok, so I’m sorry but I didn’t fully understand your 90%-10% slide. Could you help clarify that for me please?

Scott Eblen: The point there is that people often times over index on innovation so companies or products say we need to be really innovative. And they might spend 20-50-75% of their time on something really innovative when in fact, executing on the fundamentals is 90% of success. The 90-10 I’m sure varies a bit by industry but that was specific to the financial trading models where they said if you followed what 90% of the other people were doing and only 10% of the time innovated and did something different, that would be the outcome of outcomes.

Audience Question: Thank you! You gave an example for trade-off for culture. You gave one example of slowing down to do more gender balance. Do you have any other examples that you could throw out on trade-offs on culture?

Scott Eblen: I mean the other example before that was the team versus family. And that was a particularly interesting conversation at Nutmeg because there was one family relationship within the team so it was someone’s uncle who was working there as well. That wasn’t the point of contention but there were very different views about – for someone who’s struggling in a particular role, do you spend a month, a quarter, 6 months trying to get them up to speed or do you replace them? I think it’s particularly critical in a start-up environment to have those conversations because the business changes so quickly that you could get someone that is an exceptional person for a certain role. The roles will change and you need to think about as the company is in a different state, how do we want to treat people as the company changes around them? Does that make sense?

Audience Question: You raised a point about copying your competitors and trying to innovate on that. My concern with that is it’s not a really great sense of differentiation. So particularly, if I think about my competitor’s website versus mine, I want mine to look profoundly different because otherwise I feel like I’m making it easier for them to choose someone else instead of me.

Scott Eblen: That’s a great point and maybe I wasn’t as clear. It’s important to copy the underlying insights. I’m not suggesting that you should copy the design and the layout – but by having our – I think it was either usability.com or usertesting.com where you can pay people to try different products like video tapes and you learn insights about what they are thinking. By having customers go through the competitor sites, the kinds of insights we learned were things like oh I feel better putting my money in here because they stay upfront but the cost is this amount. So we copied the insight that was addressing concerns about cost or a trustworthiness in these particular places made an impact. I wouldn’t say just copying designs outright, that would be a mistake. It’s a great suggestion, it’s copying the underlying insights and concerns which is really useful.

Mark Littlewood: It reminds me of a comment from a Des Tranor from Intercom had a talk that he did a few years ago in Boston and he was talking about copying competitor’s features and actually what you end up doing was copying what your competitors were thinking was going to happen 6 months ago and they may be completely wrong so you’re basically making obsolescent potentially into your product.

Scott Eblen: I think one thing that’s particularly interesting in this circumstance because there were so many different competitors and approaches, it’s difficult to – you can’t be launching 5 different features in parallel but if you have competitors launching them at the same time. You can see how people react to those as they happen.

Audience Question: On the point you made, instead of being data driven and be data savvy. You showed us an internal Google tool, an amazing data analysis tool which you could slice the data to prove any point you wanted. Is there an example of Google making a catastrophic mistake when people mis-used the data that you can think of or are allowed to share?

Scott Eblen: Yeah, so the golden week example, that wasn’t a catastrophic mistake, but we certainly burned a lot of resources and it was a week’s worth of time we spent investigating that. If we understood the people upfront, we would save a lot of engineer’s time of going through and seeing if the product had changed or some underlying issue there.

Mark Littlewood: I have a question about how Twitter works, how you manage it. Cause you’re running the video piece and that’s running out of the UK so you made an acquisition to do that. How do you manage, what’s it like working in a satellite office but having that control? What are the kind of goods and bads of it?

Scott Eblen: Yeah, so this is something that I used at both Google and Twitter is being in a remote office between 60-90% of engineering is done in San Francisco. I think one of the key strengths of this is there is isolation to get stuff done, so between 9 am and 4 pm we can work in peace essentially and think and talk without doing lots of meetings and then from 4-7 there’s many collaboration with Seattle in San Francisco. So it’s a bit more challenging from a scheduling and coordination perspective, but we do benefit from the sort of headspace of being in a totally different place. I think it’s also important for a global company, there are a lot of very different behaviours of users around the globe and the UK and US aren’t that different, but we see some significant differences in behaviours in a couple different markets and having people with a different perspective here is helpful to tease those out and share with the extended team.

Mark Littlewood: And when you have conflict or discussions, how do you win being in London and not part of HQ?

Scott Eblen: Arm-wrestling, straight up! That’s a great question and I think definitely traveling back and forth is important so I think build the personal connections so the people do feel like it’s not just a name on email but a person they’ve met with previously. Then kind of going back to 2 of those points, a bit of the storytelling and the data is useful to help convincing others about making a decision.

Audience Question: You mentioned the dangers of having this always be shipping culture and that could potentially lead to a loss of product quality in favour of speed. My question is; there’s obviously a danger if you have people in your team that feel very cautious about that and there’s a worry that maybe you end up being too cautious and you just never ship stuff because you want to test it until you’ve tested everything that’s possible. How can you achieve that sweet spot in the team and come to a consensus on what’s an appropriate amount of testing versus shipping all the time?

Scott Eblen: Yeah. Some of these are a bit exaggerated. If you find yourself to not push out a change every week, that’s probably too slow. But some people just get overly focused on shipping at all costs and they lose any perspective. In terms of the question of quality specifically I think Hoffman of LinkedIn made – he has a great new podcast and I was listening to it recently and he said if you’re not embarrassed by the launches you’ve made, then you probably waiting too long to get this out. Which I think is fair and then he caveats that by saying but if it’s so embarrassing that there’s customer or advertiser impact, then that’s too much. I think it’s being a bit uncomfortable, wishing you could have gotten one more feature or fix in, but knowing that you will ship soon enough, that there will another change within a week or two.

Audience Question: So like many people who’ve used the Twitter API, we’ve had an up and down rollercoaster relationship. But the end users feed the content to Twitter and you guys get to use that so they’re important in your business. When in the development process do you involve the end users to validate some of your assumptions?

Scott Eblen: That’s a great question! Very early on, so we have quite a good and large research team who will go out and start talking to customers early. The other great thing about Twitter is all of our customers are on the platform so it’s easy to see what everyone is saying. For example with the NFL, we were able to look at what the people were saying about the NFL during last season. If we’re going to launch a particular political, like one of the BBC debates, we can look what the people saying about the content, even if they aren’t watching on Twitter. That’s a benefit a lot of other companies don’t necessarily have although people do talk a lot on Twitter about everything. When I was at Gmail, Jonathan Rosenberg, the EVP of product at the time wanted every product manager to have I think 100 interactions with the customers every quarter. And so I would like just tweet out and search daily on Twitter for 1-2 people who made a comment about Gmail and then I’d respond to them. It’s an easy way to interact with people especially at a company like Google to have customers complaining about Gmail and then have a product manager respond kinda blows people’s minds and they’re really excited about that.

Mark Littlewood: You’re all done! Scott, thank you very much!

Find out more about BoS

Get details about our next conference, subscribe to our newsletter, and watch more of the great BoS Talks you hear so much about.

---

Scott Eblen.

Scott has worked in product management roles in some of the largest and smallest tech companies on the planet.

Scott is Director of Product for Twitter Live Video, managed out of Twitter’s London office. His team was responsible for launching NFL football on the platform and has expanded to broadcast a variety of content. Previously, Scott was Chief Product Officer at Nutmeg.com the online investment management service after spending 7 years at Google where he, among other things, was responsible for all mobile Gmail interfaces. Prior to Google Scott led the product management team at Library House, a UK startup that tracked high-tech and high-growth companies across Europe. He started his career as a Program Manager at Microsoft on the SharePoint team. Scott holds a Computer Science degree from Princeton University.

More From Scott.