Derek Sivers has been a musician, circus performer, entrepreneur, TED speaker, and book publisher. Long ago he started CD Baby, and wrote a great little book about it called “Anything You Want”.
In this BoS Talk Derek will offer some perspectives on what we can do to be more mensch and make the world a better place, how can we end up on the right side of history, and why we should keep our eyes on how the game we play is going to end.
Derek breaks the mould in two ways for his 2019 BoS Talk by opening the talk with an auction, and then closing the talk with a Q&A session that sees the audience answer the questions instead of asking them.
Building a company takes an extraordinary amount of effort, no matter your role, and neither life nor competitors hold punches for anyone. CEO and Founder of ProfitWell, Patrick Campbell knows this first hand.
Using his experience bootstrapping ProfitWell to $10M and beyond (while simultaneously undergoing some pretty big personal roadblocks), he talks us through essential frameworks to help you handle problems in the proper context, while also teaching some operational takeaways to keep you ahead of the increasingly competitive landscape.
Lessons from a critique | Spotify’s Failed Squad Model
You know the one, the one that was so eagerly adopted or admired by almost everyone else – including possibly a company near you…? This critique of ‘Spotify’s Failed Squad Model’ is interesting for two reasons:
It’s a great way of adapting a planned conference talk into something else that may have impact.
It’s a warning that critical thinking should never go out of fashion.
Racism, injustice, systemic oppression, and inequality are global problems.
The eruption of state-sponsored violence in the US against its citizens leaves us appalled, sickened and feeling helpless.
What is taking place in the US today is more than protesting the murder of George Floyd. It is the culmination of generations of unjust treatment against black people and other marginalized communities.
This is a big move by one of the leading universities in the world and exemplifies the effect that crises can have in enforcing and driving through change in this case digital transformation. Behind the headline though, there is plenty of nuance, this does not mean the end of education at Cambridge.
This does not mean that students will not attend the University or the end of education.
The Cambridge University teaching model combines lectures with supervisions between in small groups of no more than four people. The intention is that this will continue and students will still be resident in the colleges who all act as independent entities responsible for the welfare of their own students.
This is not the end of education at Cambridge.
It is an organisation thinking radically about how it delivers value to its customers, in this case students, at a time when big thinking and big changes are required.
I remember Clay Christensen talking about disrupting Ivy League education in this talk. He made the point that,
“Harvard Business School… is getting disrupted by crummy low end on the job learning experiences like you are having right now.” Clay Christensen.
Finding & Nurturing Top Talent – Elpie Bannister & Alex Yang BoS Talk
Alex Yang and Elpie Bannister take us through Simprints’ process of finding and nurturing top talent and explain a few of the challenges they had to overcome to hone this skill.
The perfect candidate for a role in your company is out there somewhere; you may need to kiss a lot of frogs, but they do exist. How do you attract good talent, and how do you move an applicant from initial CV submission to long term part of the company?
You will learn how maximising culture, impact, and opportunities for personal development enable Simprints to hire effectively and stand out from the crowd, and that those lessons are applicable to all businesses.
Mark Littlewood: Thank you very much for saving me from having to Rick-roll you using some other devious mechanism this BoS.
Alex Yang: For those of you who are wondering why you just got Rick-rolled, Elpie and I decided to crowdsource our walk on song from our team. Our big mistake was asking on April Fool’s Day and committing to choose the one that received the most votes.
We’re really excited to be here today to share with you how we find and nurture tech talent as Simprints. Everyone knows that hiring engineers is really hard. The best candidates they have their pick of where to work and it’s really hard for small startup to compete. On the one hand you have the tech giants like Google and Facebook. They have no problem competing on compensation. They can always increase the base salary or in a few extra is restricted documents. On the other hand, you have the tech unicorns with billion dollar valuations; like Uber, Stripe, and Dark Trace. And they compete on upside so they can always dangle the prospect of a future IPO. And all of these have brand recognition. So, against this calibre of competition how does a small startup managed to remain competitive? That’s the challenge that we’re facing as Simprints with one extra wrinkle because we’re a not for profit, bootstrapped tech startup
So, this added resource constraint takes an already difficult problem and suddenly makes a lot harder. The real question I’m trying to answer is when you can’t afford to pay a superstar’s salary. How do you still manage to assemble a superstar team before you dive in. Let us first introduce who we are and what Simprints does. I’m Alex and I joined Simpson’s last year as the V.P. of engineering where I lead the people and processes side for the engineering team. Prior to joining Simprints, I’ve worked mostly at the intersection of education and code. Having worked at Google and Code Academy and having founded a startup on my own that has taught web development to over 20,000 entrepreneurs.
Elpie Bannister: I’m Elpie and I’ve led the talent team at Simprints for almost 18 months. I believe organisations are essentially a bunch of people trying to get stuff done. I see my role as helping ensure that we have the right people, the right capabilities, in the right place, at the right time. And hiring is obviously a key part of that. I spent the majority of my professional career in and around the tech ecosystem in Cambridge and I spent a large chunk of my career at Arm where I spent 10 years helping grow the organisation from 750 people 3,500 people. I’ve also got experience of working with stem organisations during the initial scaled up phase.
Alex Yang: The company where we work builds technology to fight global poverty. Our current product is a biometric fingerprint scanner paired with an Android app. Our product needs to work in some pretty extreme environments, so both are pretty robust. The scanner is waterproof, dust proof, and it’s designed to accurately scan the often worn and scarred fingerprints of individuals who’ve spent decades labouring with their hands. The Android app is designed to be used by frontline workers who can often go weeks without an internet connection and for many of them they’re using a smartphone for the very first time. Working primarily in sub-Saharan Africa and South Asia. We partner with some of the biggest NGOs and funders in the world like the Bill and Melinda Gates Foundation, USA aid and Mercy Corps.
What we really help our partners to do, is we help them to demonstrate proof of impact. So, where in the past they can only talk with certainty about the inputs; how much money they’d spent towards solving a specific problem won’t to help them talk about those outputs. So, whether that’s drug delivery in Ethiopia, cash transfers in Kenya, or maternal services in Bangladesh, we want them to be able to say that the amount of money that was spent resulted in exactly how many individuals were treated. How many beneficiaries receive payments and how many pregnant women were visited by health care workers.
Our precision data helps our partners maximise the impact of the resources that they are committing. In a recent project that we didn’t Ethiopia we noticed that drug delivery was happening outside the designated focus area. So, knowing this we were then able to alert the deployment team so they could fix the issue. A lot of the projects that we’ve done started as pilot deployments, but then as our partners developed confidence in our biometric approach, they’ve quickly scaled up to become much bigger. And from a hiring standpoint to keep up with the expanding scope we’ve also had to expand our team as well.
In the last year we grew from 12 to almost 30 and we’re looking to expand to almost 50 by next year. Most of this hiring has been focused on expanding our core team here based in Cambridge. We’re now growing to the point we’re starting to consider bringing on remote workers and contractors. We’ve also received some incredible feedback from our interview process. Just last month we had a candidate who we rejected for an android internship role and afterward he sent us a really long email giving us feedback on our hiring process. I want to highlight something that he said. He wrote
I had an interview with another company on the same day we spoke, and the difference that I felt between the two interviews is what made me want to work at Simprints even more. For comparison I spent nearly five weeks in December for some companies interview process further documentation, snapshot coding challenges, personality tests technical interviews and, in the end, I got to a generic rejection email.
I love what he said because something that we do really well at Simprints is creating an amazing canon experience. The fact that a candidate who we rejected is thanking us for helping him figure out what type of company he wants to work for after he graduates is pretty cool. And it’s this personal approach that lets the candidates stay engaged for the process and from a hiring perspective lets us punch far above our weight. So how do we do it? Elpie is going to share with us the approach that we use.
Elpie Bannister: Throughout the presentation we’ll share our insights in relation to finding talent. Some of these may resonate with you but one thing we know for sure is we don’t have all the answers. We’re continually refining our approach and would like to share some lessons we’ve learned along the way. We’ll be sharing these slides online afterwards and we’ll also share the resources as we go through afterwards.
We’ll focus more on finding talent but our approach to hiring is underpinned by our culture and our values which gives you some insight into the approach we take to nurturing talent once they’ve joined the team. What we’ll do is we’ll start off with a high level strategy in terms of growing our team and give some insight into our candidate value proposition. Then Alex will talk through sourcing candidates and screening candidates. And then I’ll pick up and talk about assessing candidates, closing candidates and then to conclude Alex will share some of the data that underpins our evolving approach.
High-level Strategy
So how do you grow a superstar team when you can’t afford superstar salaries? At Simprints we take a three pronged approach. We know that there are superstars out there that will connect with our purpose. We know it may take some time to find them and say we are patient. I’ll share some more insight into our value proposition in a moment. We also look to find superstars early on in their career and we host summer internships and offer six month internships when we don’t have headcount for an ongoing role. We’ve got a wide network of volunteers as well and we’ve got a very good track record of converting interns into core team members and 40% of our current core team started out either as interns or volunteers. And thirdly we look for talent and superstars in non-traditional places. For example, we’ve developed a very good relationship with the engineering school in France and our two cloud engineers came from that school. They previously interned with us as well. And we got another intern from that school joining us this summer.
The thing with that university is they require their students to have some experience of working abroad and also some experience of working where English is the primary working language. So Simprints is a great opportunity for them because they get both of those things in one experience from the university’s perspective, the internship can be unpaid. However, we’ve chosen to make sure we pay them the same salary as our other interns. In relation to the global talent pool, we also look for superstars in other places as well and we’ve got a strong track record of finding tech talent from San Paolo.
So, once we’ve found some candidates that we hope would like to join us. What can we do to ensure that we have a compelling value proposition to encourage them to apply? We’ve done some research and this is what we’ve come to think that we believe that there are five key factors that attract and motivate top talent and these factors underpin our approach to compensation and align strongly with our values.
We believe individuals care about how much they’re growing personally and professionally and at Simprints, we offer fantastic learning opportunities through stretch assignments, coaching and mentoring, and also, we have an individualised learning and development allowance for each team member. We also have a huge amount of freedom and flexibility in terms of when and where we work. We make sure that there’s the appropriate level of direction and support for each team member so that there’s full autonomy over tasks. We know that culture matters, and we do everything that we can to ensure that when people join, they love the people and the environment around them. Clearly salary matters, and whilst we don’t compete strongly on salary, we know that we need to make sure that the minimum threshold of the financial elements of our package are at a level enough to attract and retain awesome people. And as you would imagine with the social impact organisation, we believe we have a really strong purpose, and this is something that we know that motivated people care about. They want to have meaningful and challenging work. They want to know that what they’re doing makes a difference. And so, we do what we can to ensure that individual personal values align with our organisational values as well.
So, we’ve created our own benchmarking graph for illustrative purposes. And this is based on our knowledge of organisations through our own experience with them, but also just general observation. And you’ll notice that some organisations like McKinsey for example will try to compete on salary. Other organisations, like Google, offer fantastic learning opportunities and growth progression and we believe that most NGOs will also try to differentiate on their organisational purpose. Some organisations try to compete on all of these factors but at Simprints we’ve chosen made a conscious choice to double down on those areas that we believe we really can differentiate on. And so, we believe that we can offer fantastic opportunity in terms of learning, autonomy culture, and purpose. And we’re even managing to close the gap in terms of salary to make sure that we are able to attract and retain awesome talent. We’re closing the gap. We know we’ve got a bit of way to go but we think that’s great in terms of the fact that we’re non-profit. Okay going to hand over now to Alex to talk about sourcing candidates.
Sourcing Candidates
Alex Yang: So, for those of you who aren’t familiar they’re broadly two types of hiring that you can do. There’s inbound hiring and there’s outbound hiring. Inbound hiring is when the candidate comes to you. So, if you put up a job post on LinkedIn then that’s a form of inbound hiring as these candidates need to proactively apply to be considered for the job. Outbound hiring is the opposite. That’s when you go to the company or you go to the candidate. So, if on LinkedIn you are to proactively reach out to individuals, we thought you’d be a good fit for the role. He asked them ‘Hey are you interested in this job?’ That’s a form of outbound hiring. At Simprints, we’ve tried a little bit of both. In particular we found that using external recruiters didn’t work for us. The reason why is because since we pay lower salaries it makes the recruiters job harder because they have to find candidates who are willing to accept less money. At the same time recruiters are paid as a percentage of the hired candidate’s salary. So, it means that if they actually do find a suitable candidate there’s less incentive for them to do so. So, in effect we’re asking them to do more work for less money. And while we did manage to hire one engineer this way, most recruiters that we talked to they try for a little while and eventually they give up.
These days we primarily focus on inbound channels and by far our biggest inbound channel is stack overflow. With our inbound channels it all starts with a job post. This an example of one of the jobs posting we have it’s for senior Android engineer role. So, what is our goal with a job post our first goal is to maximise the pool of applicants that apply. We do this by getting them really excited about the prospect of working with us.
So here are some ways to do that. The first, is to list specific technology in your stack. So, Paul Graham the founder of Y Combinator and former BoS speaker once said
“like all craftsmen hackers like good tools. In fact, that’s an understatement. Good hackers find it unbearable to use bad tools, they’ll simply refuse to work on projects with the wrong infrastructure.”
So, with that in mind you want to showcase the modern technologies that your engineering team is using. This is what we do for that job post. We highlight the Kotlin language which became an official Android development language in 2017 and we and highlight the jetpack library which is a collection of software components that were only released this past September. So, these are pretty new, but the best candidates will recognise them and they’re going to jump at the chance to work with them. So, what do you do if your team isn’t using the latest technologies? Your team is still using outdated technology. Well in that case start by changing the culture of your engineering team. Change doesn’t happen overnight. For example, at Simprints one year ago our code base was 100% of the older Java language, and today it’s 96% of the more modern Kotlin language.
The next thing you can do to really get candidates excited about working with you is to mention books and resources that your team finds really valuable. This demonstrates two things to them; first it means that your organisation really cares about continuous learning and continuous improvement. The second thing is they haven’t even decided whether or not they want to apply for the job and you’ve already demonstrated value to them. So, for this job post we mentioned the book Clean Code which our engineering team uses as a North Star for setting the quality bar for production ready code.
The next thing you can do is to be really specific and quantify where possible. I’ll give a couple examples of how we do that. Lots of companies in their job post mentioned unlimited paid time off. That’s great. But at a lot of these same companies there’s a lot of pressure to not take vacation. So, what ends up happening is that employees end up taking fewer days off than they would have if you had a fixed PTO policy. Simprints used to be one of these companies but then we actually measured how many days people were taking. We found that two years ago people were taking 26 days to include bank holidays and now we’re glad to hear that that number has gone up to 31. The second way to be really specific is we have something at Simprints that we like to call Lego days. These are two days every quarter where employees get to work on a project that’s unrelated to their day to day work in the very beginning. These projects actually used physical Legos because we got a big donation from the Lego Group. But these days they can work on anything they want. We give specific examples of the types of projects that employees have worked on. One of the ones that we mention is a Lego train that delivers beer around the office.
So how could we make this job post even better? We could show a photo of the beer train. This is not a photo of the beer train. Unfortunately, this was a few years ago and so the best we could do was a photo of before they built the beer train. But to give you an idea this is a photo from our recent Lego day that was just a few weeks ago and among other projects we had two teams that competed to build a plane. One plane was powered by a battery powered propeller and the other to the other plane was powered by methanol. Let’s say that we do this really well. If we do a really good job of getting candidates excited, then we end up with a new problem. We end up with our team being overburdened with having to sift through tons of applications trying to find the diamonds in the rough. That’s exactly what happened to us. So, after we did this, we end up getting 450 applications in the span of four months for this one role. So, for our small team it was a huge time sink for them to actually process all these candidates. That’s why writing a job post you should really think about it as a dual optimisation problem. On the one hand you want to make sure you can encourage the strongest candidates to apply by getting them really excited about working with you on the other hand you want to discourage under-qualified candidates from applying because you don’t want to end up with too many applications. So how do you do the latter?
This was our initial requirements section of our job post and the problem is that all of these bullets were pretty bad at filtering out under qualified candidates. For example, if you look at the third bullet
ability to break down complex ideas into simple pieces
Talk to any engineer and they’ll say, ‘yes I can do that’. Look at the fourth bullet.
Understand the importance of automating scalable testing.
This one actually caused a really big problem for our team because basically what happened is, we’d have candidates come in for the interview and the conversation goes something like this.
So, do you understand the importance of automated scalable testing?
Oh yeah absolutely.
So which testing libraries are you familiar with?
X Y and Z.
Okay, so how many for how long you have actually been writing tests at you’re at the companies that you’ve worked for?
Oh, did I say that I’d actually written tests? Actually, all the companies I’ve worked for, none of them believed in testing.
Facepalm.
So, what did we do? We realised we had to be a lot more targeted in our requirements. So, this is the requirement section after we made the change. So where is before we split it into tech skills and human skills. We now split it into about you and bonus points which is another way of saying must haves and nice to have. Now we are a lot more specific. For example, the second bullet says professional experience writing tests and running continuous integration systems. And under the bonus points, we now list the Kotlin language and we list specific Android libraries that we use in our tech stack. It is now a lot harder for under-qualified candidates to convince themselves that they have a shot at getting the job.
Screening Candidates
So, let us say we do this really well. We now have a robust pipeline of not low quality candidates. Our next step is to screen them. So, every candidate that applies to Simprints is sent an automated email that outlines the next steps in the process. Within this email we also include a link to a Google Doc that explains the entire hiring process, so they know what to expect. We copied this approach from Mikey Trafton who talked about in his 2013 BoS talk and it has been really successful for us. This is what that google doc that explains the entire process looks like. You will notice that the very first stage is a combination of a CV screen and a coding challenge. These used to be two individual stages but then we realised two things; first we realised that these aren’t reliable indicators of candidate strength and the second thing we realised was that some candidates who passed the CV screen we then invited them to the coding challenge and they never did it. And that means that we waste their time reviewing their CV.
The way we do it now as we combine the two. Every candidate is invited to complete the coding challenge and we only look at their CV after they have submitted their code. So, let’s say that after two weeks we haven’t heard from them. Then we automatically send them an email and say listen we have not heard from you so we’re going to be closing your application. What is the coding challenge you actually look like? We give candidates three options and we ask that they complete at least one of them in any language they want. Shouldn’t take more than an hour to complete. We also try to tie these coding challenges to Simprints itself. So, this is an example of one of the three options in this challenge. We ask the candidates look at the company values its inference has like relentless commitment to impact and robust as fudge. So, in computing. There is a concept that called an ASCII code. So basically, every character corresponds to a numerical value called an ASCII code. And so, we’re asking candidates to for each row of values, sum up the ASCII values for every single character and sort those values by those sums. So, the thing is that when they submit their code even before opening up the code, we actually already know quite a bit about the candidate. At the highest level. We look did they just finished one challenge, or did they attempt all three? What language do they write it in? Many candidates choose the older Java, but the best candidates choose Kotlin.
Here’s an example when the submissions we received to this candidate had an excellent CV and, in the email, he wrote that he had experience working with both Swift and Kotling. So, it’s great that he has experience working in multiple languages, but his code submission was in Swift and Swift is the language that’s used for iOS development. So, since he’s applying for an Android role that probably wasn’t the best decision and what that teaches us about him is that his decision making is a little suspect.
Here is another example. So, we ask all the candidates when you submit your code, you can either attach it directly to the email but do whatever you like. And so, some candidates do that. A lot of people will put it on github. This candidate sent it to us in a Microsoft Word file. So, word is great for writing essays but it’s basically unusable for writing code. So, in this case we don’t even need to open the file to know that we should reject him. So, when we actually do look at the code, how do we figure out which ones are good and which ones are bad? Well it is more than about whether the solution is correct or incorrect. If it is correct how elegant was the solution. The best ones are usually really easy to read with clear distinctions between the calculating of the ASCII sums, the sorting and then the printing of the result. They also use precise variable naming. You will not see names like I or J or thing 2 or temp. They will be really really specific and only if the code is clean, do we then actually open their CV. And what do we look for in the CV? First, we look for overlap between the libraries that they have experience with and the libraries that we currently use in our tech stack. We also look for side projects and open source contributions as evidence of their passion for technology. And finally, we learned that we should really look for extensive experience with testing.
All of these are much more important than the sheer number of years of experience they have. We get a ton of candidates who have been in the industry for a long time but they’re just coasting on the existing skills that they have. They are not spending the time to actually stay on the edge of the domain that they’re working in. On the other hand, we occasionally get a recent college grad or even a college dropout who doesn’t have that many years of experience but the level of knowledge they have is representative of someone who has much much more experience. From a candidate’s perspective the very worst thing is when you send your CV and you send your code into a black hole and you do it and you never hear back. So, from a candidate’s perspective what do they want? First, they want a job offer. If they can’t get a job offer, they want to get feedback on why they did not get the job. But the problem is that many companies end up giving them number three which is radio silence. Well we aim to do is we aim for 100% response rate.
So what that means is that when candidates first submit their code even before we’ve looked at it we first send them an email saying They just want to let you know we received your code and our engineers are going to take a look and let you know whether we decide to move forward. So, let’s say that we decide not to move forward with this candidate. In that case we let them know and we give them feedback. Let us say that instead we decide to move forward to an interview. We let them know and we give them feedback. So even if you can’t give personalised and detailed feedback at the very least you should be giving them templated feedback that incorporates the most common areas for improvement that you’ve seen of candidates that are applied for the role.
Now a lot of people ask why do you give feedback to candidates who you’re rejecting and it’s a great question. The way to think about it is think about it from a long term perspective. If you’re a candidate and you’ve spent time applying to this job writing code and eventually interviewing for this position by certain point, you’re quite invested. You have done a good job of getting them really excited about working with you. They will be disappointed that they didn’t get the job. But these candidates are going to go on and – the best ones even though they got rejected they’re going to go on to gain more experience and knowledge – the next time they’re in the job market they’re going remember the experience they had with you and they’re going to apply again. I was a couple weeks ago talking to a partner at an early stage firm and he told me that one of his employees had applied three times and the first few times she was rejected. But each time she came back she had improved on the feedback that he had given her until on the third try. She finally did get accepted and now she is one of his best employees. So now that you have candidates going through your screen how do you actually go about assessing them. Elpie is going to hear more about how we do that.
Elpie Bannister: Thank you.
Assessing Candidates
So, as Alex said he’s talked through the sourcing and the screening stages. I am going to now talk through some of the steps involved in our assessing stage. To start with the values interview. So, we start with the 30 minutes values interview which is conducted by a member of our talent team. So, it’s a chance to get to know the candidate. And it’s normally conducted over Hangouts and we ask a series of questions which give us some insight into behaviours which we believe demonstrate whether their behaviours align with our values.
We then move through to the technical interview stage and this is in two parts. The first part digs into a little bit more detail around the non-technical side but the main focus of that interview is obviously around technical expertise. And here our engineers will dig into real life examples of the candidate has said that they’ve experienced on their CV and ask them to give us some concrete examples of that. So, we can check that the skills and experience are they say they have aligned with what we’re looking for. And then we have a trial day. I will tell you a bit more about the trial day in a moment. But just for now I’ll say that prior to trial day we are completely transparent about our salaries. As I mentioned earlier, we don’t compete strongly on salary so it’s important that we have a good open discussion with candidates prior to the final stage in our hiring process. And earlier if they want to ensure that we understand what their expectations are.
Throughout all of the stages we make sure that there’s time for candidates to ask us questions. And in the technical interview a full 30 minutes is set aside for the candidate to ask us questions and also for us to share what our thoughts are about working for Simprints – so essentially to pitch Simprints.
During the CV and CV screen and coding challenge stage – I’m not going to go into as much detail because Alex has covered that already – but just to say in relation to the side projects that is something we do look out for at this stage, because we’re really keen to find candidates who are curious and who love to play and have fun with technology outside of their day job. It’s really hard to know much about the side projects at the CV stage though. So that’s something that we would dig into at the technical interview. For the values interview we asked the same questions for both technical and non-technical roles. And we do this because we’re looking to find people that align with our values particularly, we’re looking for a passion for learning a willingness to confront the grey. And obviously for people who do connect with our core purpose as well. And here are a few example questions that I’ll walk through.
So firstly, we’ll ask them to share a project or piece of work that they’re really proud of and why. We think this is a great question because it gives us insight into what the candidate cares about in relation to their work. And we’re looking for high standards confidence and ambition. We also asked candidates to talk us through a mistake that they made and what happened in that situation. Here, we’re looking for resilience and integrity and we’ll find that perhaps weaker answers might indicate that a candidate is willing to give up at that first hurdle or perhaps seek to assign blame to others in the team. Then we ask about what people might want to be learning in the next two to three years. At Simprints we very much believe in the power of continuous learning both individually and collectively. And so, we’re looking for candidates who are keen to push the boundaries of their learning and their skills and experience both in terms of breadth and depth. One of the things that we know we can improve on in this stage is to have a wide repository of questions that we can dig into with a more rigorous scoring rubric, so that any member of the team can conduct this value stage interview.
In relation to the technical interview. We refine the questions to be aligned to the scope for the technical role. But we also spend some time digging into some of the non-technical aspects as well. And so here we’re looking for understanding a little bit more about the mindset of the candidate in terms of their approach to software design, also looking for some self-awareness. And also, in relation to that passion for learning. So, one of the questions we ask is
‘most products have a warning label, if you had a warning label, what would yours read?’
And we get some very interesting answers on this one, but the reason why we ask it is because it requires a level of self-awareness. It also highlights that as a team we’re really keen to open up the discussion around what it takes to develop effective working relationships. So, if you know what someone’s frustrations or trigger points are then we can build that in to ensure that we have effective working relationships going forwards. We also ask whether they believe it’s better to be perfect and late versus good and on time. This is a classic engineering conundrum and it gives us some insight into the candidate mindset. We also ask them to share some real life examples of when they’ve had to balance the focus on deliverables and milestones and how they manage the interactions between engineering, product, and business which may inevitably lead to some tension. We also ask candidates to take a few moments and to think of something that they can teach us in a five minute timeframe. We’ve heard a ton of answers for this one and so we do give candidates a bit of guidance so they can choose a topic with some complexity but enough to add something that they can give us enough understanding surface level understanding and a five minute period. We’ve had some good and bad examples here. Some of the bad ones perhaps are around ‘let me tell you about the town where I come from’. Or ‘let me teach you how to start a startup’. With the first one about the town the response might turn out to be somewhat unstructured and rambling because they’re just talking rather than it being more structured insight. And then clearly with starting a startup – as you’ll hear from the talks today – it takes more than a five minute talk to be able to understand the complexity and the challenges involved in starting a startup.
Some of the good examples we’ve had have been where candidates have explained to us why the Earth has summer and winter seasons or linking back to the chess game. Some of the starting moves the chess and how to counter them. We even had one candidate teach us some of the basic motorcycle stunts. Clearly this was over hangouts and verbally, so we didn’t have a physical demonstration. But we did get a really good insight into that candidate and the question gives us a glimpse of something that we might not see otherwise. So, this candidate showed us photos of his motorcycle tour across India. And another candidate also shared insights of their three month tour of Europe around heavy metal festivals. So as I said with this section we really dig into the technical expertise and the questions are refined around the scope of the role and so we look at knowledge based questions so more conceptual knowledge based questions but also some more practical situation based questions and both are important to ask because candidates are sometimes better at one or the other. In relation to the conceptual based questions for the Android role, for example, we might ask a candidate about their knowledge around the android architecture components we might ask what experience they have of using them and then also we’ll ask what problem do they think Google was trying to solve in introducing them. And we asked that last point because any candidate perhaps can brush up on their conceptual knowledge but it takes a really good candidate to understand the bigger picture to understand the pros and cons of the different tools to explain what problem they were trying to resolve. For the more practical based questions what we’ll do is we’ll share some insight in terms of real life challenges that our engineering team are facing. And this is a great way to share some insight of what it’s like to work at Simprints whilst feeling less contrived. And even if the candidates may not have had the exact experience in that challenge. Some of the better candidates are able to use their knowledge and experience to at least give a couple of options that we could consider to resolve that problem.
Trial Day
Moving onto the final stage of our hiring process we have what we call a trial day and it’s something that we debated in the team and one of the teams had actually the word trial could be something that they feel people feel like they’re on trial. But we’ve actually phrased it because it’s a trial for them to see us in practice, for them to have a trial and a day in the life of simple prints and here’s a schedule of an example trial day. We refine the schedule based on the role and we also refine the project based on each role as well. So, the day will involve one to one meetings, and also time for project work, will also involve the candidate in the day to day rhythm of life at Simprints – like all team stand up and lunch. With the project it can also be phased direct during the day. So, the first phase could be around the design and planning and then the second phase would be around execution and implementation. When we get to the demo phase at the end of the day, what we’re looking for is obviously technical skills but also some of the non-technical skills around communication and we’re looking for candidates who are able to communicate what they’ve done throughout the day with a clear structure. At the high level, we’re looking to them to explain what the scope of the problem was and what they were trying to achieve with the approach that they set out and do they have a demo to actually show us. At the next level down, we’re looking for a bit more insight in terms of the decisions that they made and perhaps some of the trade-offs that they thought about. So, what architecture did they use and why? And then at the lower level we’re looking for them to walk us through the individual code files that they’ve created to implement their strategy, giving us some insight into whether their code is clean and whether it’s production ready.
So, trial days are a huge investment and we believe they’re something that is of benefit to simple prints and we also believe it gives huge benefit for the individual as well. So here are some stats that we pull together based on a three month period from December 2018 to February 2019. And these covers both technical and non-technical roles. So just want to set the context for the stats there. On average we have five trial days a month during that period. We had a particularly busy period during February where we had multiple concurrent trial days and we even had three trial days for three different roles on the same day. It was a busy day. We also worked out that the average of our average cost of our trial days is £900 for each trial day and that covers international candidates coming over; so, it covers flights, accommodation, visitors’ visas, and other incidental expenses. For any candidates based in Cambridge where we are clearly those expenses aren’t incurred. And across the technical and non-technical roles that we’re hiring for, we’ve had a 50/50 split of U.K. and non UK trial day candidates. In terms of Offer Rate after trial day 62% of the candidates who came to trial day we offered. And that’s okay because actually we sometimes run competitive hiring so we might bring in three candidates for trial day and only want to make one offer. So actually, this is a statistic a metric that we’re not trying to over optimize on. Alex is going to talk more about our offer to acceptance ratio in a moment but it’s interesting to note that by May of this year (2019) 46% of our team will come from outside of the EU and will be on either a Tier 2 or Tier 5 visa. So, as you would expect there are pros and cons with trial days, and this is something that we debated within the team. It’s something that worked very well for us to date. But does it still scale? Will it make sense for us going forward? We believe the pros are that, as I said, it gives candidates to experience a day in the life of Simprints. We very much want people to join us who know what they’re getting into and the way to do that is to give them a day in the life of the organisation. Throughout the hiring process, we’ll have built relationships with the candidates already and the trial date gives a chance to augment that even further by meeting other members of the team outside of the technical team they might be joining. And also, it gives us a great opportunity through the project work to really deep dive into the skills that they have and to make sure that they are a good fit with what we’re looking for. Some of the cons as you can imagine or the cost that’s involved but the cost is both financial and non-financial. They take up a lot of time and we believe it’s time well invested, but it does take time and sometimes the logistics phase of securing visitors visas can be quite long. And so, some candidates do drop out at that stage and we have three really strong technical candidates that disappeared into radio silence through this stage. We have no idea why they dropped out because they didn’t respond to the e-mails. And so, whilst that might be a good thing in the long run, we do feel it’s a bit of a missed opportunity.
Also, when you’ve built up that relationship with people throughout the whole process and you spent a whole day with them, they’ve got to know the team you’ve gone and had dinner at the end of the day. If you then decide after the decision meeting which the whole team can come along to and we decide whether we want to proceed to offer it can feel bad to say no. And so what we do as throughout the process we will make sure that we give detailed feedback to the candidate and we talk this through with them over a Hangout call or in person if they’re in Cambridge but over hangout call normally to give some reasons an explanation as to why we’re not proceeding to offer. So once we’ve invested all that time in building the relationship with the candidate and giving them a chance to get to know Simprints we want to make them an offer and what can we do to make sure that we have a positive outcome. Prior to communicating the decision to the candidate, we spend time with the hiring manager to really tailor that message to the candidate. Throughout the process, we’ve got to know the candidate. And so, we’ve got some insight into what they care about and what some of their fears and concerns might be. For example, if they care about using the latest technology then we’ll emphasise that during the offer call. If they want to use that technology for positive impact, that’s something we’ll mention. And then if they’ve got any fears and concerns about joining Simprints, be that around the salary or the total value proposition that I talked through then we’ll spend time going through that. And also, in terms of international candidates they might have a fear of the process of securing a visa, which believe me can be quite painful at times. Also relocating to the UK so we spend time talking through those during the offer call.
We follow up the offer call with a very personalised offer letter that we’ve drafted with the hiring managers input – again, tailored to what we’ve learned through the hiring process of the candidate. We outline 10 key reasons why we believe they would like to join our team. This example here is for one of our Android engineers who is going to be joining us soon and he, throughout his studies, actually worked full time for the whole five years of his studies and we were super impressed with that in terms of desire and dedication to make it happen. “Make it happen” is one of our values and we wanted to emphasis that in the offer letter. We also supplement the offer letter with the 10 reasons with a handwritten note which has been written by those people that the candidate met during the trial day. Again, we believe this cements the relationships that have been built and also gives that future focus. So, this example circled here is from Rid1 another one of our engineers our Android engineers. He was really pleased to have met Alan during the trial day and is looking forward to building awesome things together. Should he accept our offer and he did. Which is great. So, I’ll pass over to Alex now to conclude.
Be Data Driven
Alex Yang: So so far, we’ve mainly covered the qualitative aspects of how we hire Simprints. But I wanted to spend the last few minutes we had to talk about the quantitative aspects which are equally important. Every hiring process is the product of many gradual changes, the way to think about making your next change is to think about it as an experiment. Look at the data behind it. Did the experiment go well, or did it not go well? The more data you track, in general, the easier it will be to identify opportunities for improvement. It also helps tremendously to use an applicant tracking software or ATS which will automatically collect a lot of this data for you. Some of the ATS to consider are Greenhouse, Applied, and Recruiter box. So here are three key metrics that you should keep your eye on. The first is days to offer. That measures on average for successful candidate. What is the length of time between when they first applied to when they received an offer? You can actually break this down much further to look at each individual stage, but days to offer is a really good high level metric that looks at the speed of your pipeline.
The second metric to look at is qualified candidates for hire. So, if you define a qualified candidate say as someone who passes your CV screen and coding challenge and comes in for an interview. This is then asking how many candidates do you need to interview before you find someone who you extend an offer to and they accept? This is a really good metric to look at if you want to know how long a currently open roll will be open before you’re able to close it.
And finally look at offer acceptance rate. This is really simple. The number of offers that you’ve extended that were accepted over the total number of offers that you’ve actually given. And so, because this is Business of Software, we’ve decided to be completely transparent with our numbers.
For days to offer our current number is 60. And of that 60, logistics for trial day candidates coming from abroad makes up over 30. For qualified candidates for hire. The current number is 12 and that means that we need to interview 12 candidates before we finally get someone who’ll actually be joining the team. That means that every person we interview has around an 8 to 9% chance of being the one that we bring on. And finally, our offer acceptance rate for engineers is 86%.
So, one thing that we’re currently doing is we’re transitioning to a different ATS with stronger analytical capabilities because a great ATS will be it we’ll be able to monitor and analyse all these metrics for you. In our case we’ve got to do a lot of this manually. So even some of these numbers are out of date and we think we’ve improved on them. But I wanted to also share some of the other one off analysis that we’ve done. So first this is a breakdown of the inbound channels that we look, at as you can see Stack Overflow is by far the biggest representing over 90% of the volume of our engineering candidates. So, when we saw this our next question was ‘well how efficient are each of these channels?’. And so, one thing we noticed is that we look on the left stack overflow candidates only one% of them actually make it to trial day. And on the right if you look at some of the other smaller but promising data sources, we have sample size are small but 25% of them make it to trial day.
So, what are the takeaways from this. We had two takeaways. The first takeaways we should experiment more with these smaller channels that seem to be working for us. And the second takeaway is we look at stack overflow numbers, wow they’re really low but that doesn’t mean that we should get rid of stack overflow. Instead that means that we need to do two things we need to bemore targeted with our job posts and we need to reduce the hiring burden. And so, you’ll notice that these charts are a little bit out of date. You’ll see that the first two bars in each graph show a CV screen and coding challenge as two separate phases. So, we actually reacted on this feedback and so, with targeting we update our job posts. And now anecdotally the percentage of candidates who are completing all three of our coding challenges in Kotling has gone through the roof. And then second, by reducing the hiring burden, that was the motivation for why we decide to combine the CV screen the coding challenge. We ran the numbers afterward and we actually found that this reduced our hiring burden by nearly 50%. So massive massive change.
Another thing that we look at as we look at the geographic distribution of candidates. We don’t actually ask this to our candidates, but we do ask them for their phone number so by using the country code of the phone number we’re able to see where they’re actually applying from. And as you can see from the first two bars over 50% of candidates are applying from India and Pakistan alone. We think that’s due to a few reasons. The first reason is stack overflow is user base and how that’s located around the world. And the second reason is the concentration of software development and software engineering in South Asia. And finally, there’s massive demand in Asia for western companies who are willing to sponsor international visas. The other thing you’ll notice is that local candidates from the U.K. is only 5%. And so, given the cost and logistics involved in running trial days we’re really keen to increase this number.
So, to recap some takeaways here when sourcing look at non-traditional channels to find candidates that match your hiring needs. Be really specific with your job postings so you can encourage the strongest candidate to apply. And split your requirements into must haves and nice to have to make sure that under-qualified candidates are discouraged from applying. From a screening standpoint prioritise coding challenge over CVs as they are a more reliable indicator of candidate strength. Consider combining the coding challenge and CVs into a single screen to reduce your hiring workload and finally aim for 100% response rate and give feedback even if templated when you can. When it comes to assessing make sure you test both conceptual and practical knowledge, some candidates will respond more to one or the other and they actually don’t wait until the trial day to line on salary. Finally consider using a trial day as a good way to evaluate international candidates and to pitch the company to them. When closing, go above and beyond with a tailored approach by using personalised offer letters and a signature page signed by every person that that candidate met. And finally be really data driven measure everything you can and use an ATS to capture that data and monitor it for you and finally monitor some key hiring metrics like days to offer, qualified candidates for hire, and offer acceptance rate.
So, before it closed. I wanted to share a personal story about how I joined Simprints. When I first heard about the opportunity. I thought there’s a less than 5% chance that I would actually take it. And in fact, I felt so guilty about it that I asked that they schedule my trial day to work around vacation. I had already planned to Italy and that way they didn’t need to pay for an international flight from New York. Instead they just had to pay for a Ryanair flight from within Europe. And so, when I actually showed up for my trial day. What impressed me the most about Simprints was the team. I’ve worked at a bunch of different organisations now including two of the perennial top five best places to work and what I noticed is that the team was incredibly cohesive. Also even looking at the youngest employees they had experience and knowledge that went far beyond their age. The team was also incredibly diverse. So, we had over 20 nationalities represented among a team of around the same size. And for me the kicker was the signature page. Seeing the signature page signed by every person I’d met that day. So what I believe, is that when you spend as much time at work as we all do it’s really important that work is fun and Simprints team truly is one of the best so I wanted to thank you all for listening.
Hopefully, you’ve taken away a couple of tips they can bring to your own hiring process. If you want to talk about this more. Feel free to email me and Alex@Simprints.com. Shockingly, we’re actually hiring for multiple roles. So, if you’re interested come talk to me afterwards and I can connect you with the hiring manager.
Thank you.
Learn how great SaaS & software companies are run
We produce exceptional conferences & content that will help you build better products & companies.
Join our friendly list for event updates, ideas & inspiration.
In Times of Scarcity, Clear Vision Beats Rapid Iteration – Guest blog post from Radhika Dutt.
Radhika is co-Founder of Radical Product Thinking. She has been based in Singapore for the past two years, will be speaking at BoS Conference USA Online as well as running online product masterclasses in the lead up to the event.
NB, Business of Software is not qualified to offer advice or comment on the political, medical, healthcare, epidemiological or other scientific issues around disease outbreaks, so we won’t. We will cover thoughtful, non-link bait perspectives on lessons that may be of interest and relevance to our audience. We believe you will find this worthwhile.
In Times of Scarcity, Clear Vision Beats Rapid Iteration
Lean Startup changed how we built products when it was published in 2011 – it lowered the risk in innovation by shortening development cycles. In the years that followed, we’ve had the luxury of an abundance of resources – the economy was on an upswing and credit was more easily accessible. Many companies took the Lean methodology too far and began to over-rely on iteration. It was easy to try things in the market and keep iterating to see what works.
Sometimes however, you need more than iteration, you need to rethink.
Want to Entrepreneur like Steve Jobs? How many entrepreneurs do you know who model themselves, consciously or unconsciously, on Steve Jobs? I can name more people I know than I have fingers (8 ✋🤚 at last count).
They all share one common trait and it’s easy for you to copy.
Alex Osterwalder is a leading author, entrepreneur, and renowned speaker whose work has changed the way companies do business and how new ventures get started. In this BoS Talk from 2019, Alex will discuss how you can manage a portfolio of products successfully. How can you invent a new business model, improve existing ones, and manage them all across the organization.
You will learn:
how to map the exploration of new business ideas and test them in a simple and practical way
how to manage and improve the businesses and products you already have
how much profit existing business models generate
how to point out any synergies/conflicts between your models.
Wherever you will be in the world on the 21st-23rd September, you can participate, learn, network and spend quality time with some of the smartest SaaS & software people on the planet. No plane tickets to buy, no hotel rooms to book, and no waiting in a queue for coffee (depending on the size of your family).
We had so much fun at BoS Europe Online back in March, and we learned a lot about what makes a BoS event special.
Just like Dorothy in the Wizard of Oz, event organizers and attendees alike have woken up to the realization that we’re not in that familiar place anymore. And for us all, just as in the Wizard of Oz, what we know worked in Kansas, doesn’t necessarily work online.
For those of you who prefer your literary references to be a little more highbrow…
“The past is a foreign country; they do things differently there.” LP Hartley, The Go Between.
As organizers and participants, we need to recognize that this will not change any time soon (sorry to break it to you if this is news), and work towards building better experiences for everyone that utilize the benefits and advantages of our new virtual venue rather than try to replicate the old model.
We all need to think about designing events from the outset in a different way. In March 2020 we ran Business of Software Conference Europe as an online event. Since then, I’ve also sat through a huge number of other events too, as I’m sure you have.
My main reflection:
Online events will struggle to engage your audience and may seriously harm your reputation as an event organizer if you don’t design your online event in a completely different way.
It doesn’t have to be like this.
How To Run A Successful Online Event
Here are our thoughts on how to think about designing and running events that work online; distilled from hundreds of hours of designing, delivering and watching online conferences.
Lessons Learned Running Our First Online Conference
In March 2020, we ran Business of Software Conference Europe as an online event – our first ever virtual conference. Even with the short timeframes involved in putting this together, and the significant distractions going on all around us, the conference was a resounding success, with attendees noting that the experience of the online event was remarkably close to an in-person event.
As ever, we focused on two principles:
Learning new skills and ideas
Meeting other smart people.
We learned a lot and there are definitely things we’ll iterate on at our upcoming online conference this September, BoS USA Online 2020, to make it even better. We’re also learning every day from other online events that help us all learn the things that work and the things that we can do to create a memorable, fun and valuable experience.
Here’s some of the things we learned that will help us to make BoS USA Online even better:
In this BoS Talk, Claire discusses how no one sets out to be a bad boss, but it’s easy to become one. After years of research from 15,000+ people and 25 countries, Claire shares the biggest, most counter-intuitive mistakes that leaders unintentionally make – and what to do about them.
Claire Lew is the CEO of Know Your Team – a company that helps companies like Airbnb and Kickstarter and also runs a online leadership community, The Watercooler, with 1,000+ leaders and thousands of conversations on hiring, firing, business growth and more. Claire’s mission in life is to help people become happier at work. She is often cited as an expert on the topic of creating more open, honest workplace environments, and has been published in Harvard Business Review, CNBC, Inc, and Fortune among others.
Learn how to ask your team the right questions, why trust is important, and why being nice can be a bad thing.
It takes something special to turn some of the terrible marketing copy being shoved into outbound email into something that is both beautiful and somehow energizing so congratulations Jessica, you win our Lemmy of the Week Award.
There’s an alternative career as a copywriter waiting to happen. Probably. If you wanted it.
Messaging, of course, is incredibly important and no doubt you have seen almost every possible version of these lines land in your inbox over the past few weeks – quite often, followed immediately with a sales pitch, buy-now offer or something similar. The words lose their effect and often seem to stand in stark and hollow contrast to the marketing message or sales pitch that follow.
We’re going to try not to do this at BoS.
We know what’s happening in the world around us already.
We don’t want to offer cod-scientific advice or easy platitudes or fill your inbox with meaningless words.
We want to try to focus our effort on serving and building our community around the world and be a place where you can extract the highest possible value from some of the smartest software people on the planet.
ENJOY:
The First Lines of Emails I’ve Received While Quarantining
At the turn of the century, remote working at scale would have been impossible. Today, more and more teams are discovering the benefits and flexibility that remote working provides; and in an ever changing world, could remote working become the new normal?
Having founded Doist in 2007, Amir Salihefendic is the CEO and founder of the company behind Todoist and Twist. Born in Bosnia, he grew up in Denmark, started the business in Chile and now lives in Barcelona with his family, but has colleagues spread across 25 countries.
In this BoS Talk from Business of Software Europe 2019, Amir will discuss why he believes remote-first could represent a massive paradigm shift with widespread implications for the future of work. You will learn the ten most valuable lessons Amir has learned whilst growing and operating a fully remote company – including why trust is important, why asynchronisation is greater than synchronisation, and why retreats are worth their weight in gold – plus hear his answers to a host of great questions in his Q&A section at the end.
Ayat is a ‘queen of CRO’ with over ten years helping companies create websites that customers love, and that result in increased sales and retention.
Most marketing talk, or business strategy meetings, emphasizes the importance of ‘customer first’. Yet if we take a look at companies, and how they operate, the C-Level exec are still calling the shots without paying heed to the needs and wants of the customer. We have all stumbled across a website which uses terminology and jargon that the average person simply can’t understand or when a design is simply the result of a bunch of executives deciding what they ‘think’ their customer wants. Even when companies embrace a customer first approach, the way they tap into their customers voice doesn’t give them the right result.
In this BoS Talk you will learn:
How to conduct qualitative research to tap into the customer voice more effectively
How to pair quantitative and qualitative research for a more global understanding of the customer journey
How to use what you’ve collected to design a voice that addresses the real motivations, needs, and wants of your customers.
David Cancel is a 5x founder and 2x CEO with four successful exits to date. Currently, he’s co-founder and CEO of Drift, a startup that’s making it easier for businesses to talk to their customers. Previously, he was at HubSpot as Chief Product Officer after they acquired his company Performable.
In this (fireless) Fireside chat, we delve into what inspires and motivates David, how he came to start five different companies, and how he leads a high-growth company. Plus there are some great questions asked by the audience at the end that helps David open up about what he has done and what’s next.
Google famously published a leading reference for establishing Objectives and Key Results (OKRs) as a way to align teams and set short-term goals. In this BoSTalk, Whitney O’Banner (Engineering Manager, Medium) delves into which aspects of Google’s video should be embraced, which should be updated in line with the times, and which should be left in the ’90s.
Whitney leaves you with three ‘top tips’ for incorporating OKRs into your team, plus answers a few of the audiences hard-hitting question in the Q&A at the end of the talk.
This talk touches on how innovation can come from the bottom and that the upper hierarchy shouldn’t be so quick to shoot down ideas that come from grassroots. Marty Cagan (Founder, Silicon Valley Product Group and previous BoS speaker) recently published an article about OKRs which echos some of Whitney’s points about empowering teams and giving team objectives instead of individual objectives – BoS is often ahead of the curve – check out Marty’s post on svpg.com.
Rich Mironov gave a fantastic talk about Software Pricing at BoS Europe 2019. Yesterday, he joined Mark to answer questions from an audience both at our offices in Cambridge, and online.
Questions answered included:
If I know how much money my product makes for each customer, how much of that value should I capture?
How do I understand the pricing units for my software?
Who should own pricing? Product/Marketing/Sales/CEO?
How do I move upmarket from relatively low SaaS pricing to big enterprise sales?
With a background in chemistry and pharmacology and being an MIT MBA with a focus on finance and economics, Isa’s corporate career started at Pfizer where she was a chemist and data scientist before moving to JP Morgan Chase. Whilst VP, Digital Product & Strategy for Small Business at JP Morgan, Isa observed that the bank branches that were most profitable were ones with the most engaged people.
She took the leap into the world of entrepreneurship when she founded Squad in 2015 having seen an opportunity to help companies understand how to do this better but it was a huge career pivot that saw her leave corporate life to go it alone and pursue this.
She will talk about some of the ways that companies can provide a better environment for humans to work and share how she came to realize that in her startup, she had to bring her ‘best self’ to the business every day – no one else in the business has more impact on the morale and well-being of the team than the CEO.
