Finding & Nurturing Top Talent

Finding & Nurturing Top Talent – Elpie Bannister & Alex Yang BoS Talk

Alex Yang and Elpie Bannister take us through Simprints’ process of finding and nurturing top talent and explain a few of the challenges they had to overcome to hone this skill.

The perfect candidate for a role in your company is out there somewhere; you may need to kiss a lot of frogs, but they do exist. How do you attract good talent, and how do you move an applicant from initial CV submission to long term part of the company?

You will learn how maximising culture, impact, and opportunities for personal development enable Simprints to hire effectively and stand out from the crowd, and that those lessons are applicable to all businesses.

Video, Slides, & Transcript below

Video – Finding & Nurturing Top Talent

Slides – Finding & Nurturing Top Talent

Transcript – Finding & Nurturing Top Talent

Mark Littlewood: Thank you very much for saving me from having to Rick-roll you using some other devious mechanism this BoS.

Alex Yang: For those of you who are wondering why you just got Rick-rolled, Elpie and I decided to crowdsource our walk on song from our team. Our big mistake was asking on April Fool’s Day and committing to choose the one that received the most votes.

We’re really excited to be here today to share with you how we find and nurture tech talent as Simprints. Everyone knows that hiring engineers is really hard. The best candidates they have their pick of where to work and it’s really hard for small startup to compete. On the one hand you have the tech giants like Google and Facebook. They have no problem competing on compensation. They can always increase the base salary or in a few extra is restricted documents. On the other hand, you have the tech unicorns with billion dollar valuations; like Uber, Stripe, and Dark Trace. And they compete on upside so they can always dangle the prospect of a future IPO. And all of these have brand recognition. So, against this calibre of competition how does a small startup managed to remain competitive? That’s the challenge that we’re facing as Simprints with one extra wrinkle because we’re a not for profit, bootstrapped tech startup

So, this added resource constraint takes an already difficult problem and suddenly makes a lot harder. The real question I’m trying to answer is when you can’t afford to pay a superstar’s salary. How do you still manage to assemble a superstar team before you dive in. Let us first introduce who we are and what Simprints does. I’m Alex and I joined Simpson’s last year as the V.P. of engineering where I lead the people and processes side for the engineering team. Prior to joining Simprints, I’ve worked mostly at the intersection of education and code. Having worked at Google and Code Academy and having founded a startup on my own that has taught web development to over 20,000 entrepreneurs.

Elpie Bannister: I’m Elpie and I’ve led the talent team at Simprints for almost 18 months. I believe organisations are essentially a bunch of people trying to get stuff done. I see my role as helping ensure that we have the right people, the right capabilities, in the right place, at the right time. And hiring is obviously a key part of that. I spent the majority of my professional career in and around the tech ecosystem in Cambridge and I spent a large chunk of my career at Arm where I spent 10 years helping grow the organisation from 750 people 3,500 people. I’ve also got experience of working with stem organisations during the initial scaled up phase.

Alex Yang: The company where we work builds technology to fight global poverty. Our current product is a biometric fingerprint scanner paired with an Android app. Our product needs to work in some pretty extreme environments, so both are pretty robust. The scanner is waterproof, dust proof, and it’s designed to accurately scan the often worn and scarred fingerprints of individuals who’ve spent decades labouring with their hands. The Android app is designed to be used by frontline workers who can often go weeks without an internet connection and for many of them they’re using a smartphone for the very first time. Working primarily in sub-Saharan Africa and South Asia. We partner with some of the biggest NGOs and funders in the world like the Bill and Melinda Gates Foundation, USA aid and Mercy Corps.

What we really help our partners to do, is we help them to demonstrate proof of impact. So, where in the past they can only talk with certainty about the inputs; how much money they’d spent towards solving a specific problem won’t to help them talk about those outputs. So, whether that’s drug delivery in Ethiopia, cash transfers in Kenya, or maternal services in Bangladesh, we want them to be able to say that the amount of money that was spent resulted in exactly how many individuals were treated. How many beneficiaries receive payments and how many pregnant women were visited by health care workers.

Our precision data helps our partners maximise the impact of the resources that they are committing. In a recent project that we didn’t Ethiopia we noticed that drug delivery was happening outside the designated focus area. So, knowing this we were then able to alert the deployment team so they could fix the issue. A lot of the projects that we’ve done started as pilot deployments, but then as our partners developed confidence in our biometric approach, they’ve quickly scaled up to become much bigger. And from a hiring standpoint to keep up with the expanding scope we’ve also had to expand our team as well.

In the last year we grew from 12 to almost 30 and we’re looking to expand to almost 50 by next year. Most of this hiring has been focused on expanding our core team here based in Cambridge. We’re now growing to the point we’re starting to consider bringing on remote workers and contractors. We’ve also received some incredible feedback from our interview process. Just last month we had a candidate who we rejected for an android internship role and afterward he sent us a really long email giving us feedback on our hiring process. I want to highlight something that he said. He wrote

I had an interview with another company on the same day we spoke, and the difference that I felt between the two interviews is what made me want to work at Simprints even more. For comparison I spent nearly five weeks in December for some companies interview process further documentation, snapshot coding challenges, personality tests technical interviews and, in the end, I got to a generic rejection email.

I love what he said because something that we do really well at Simprints is creating an amazing canon experience. The fact that a candidate who we rejected is thanking us for helping him figure out what type of company he wants to work for after he graduates is pretty cool. And it’s this personal approach that lets the candidates stay engaged for the process and from a hiring perspective lets us punch far above our weight. So how do we do it? Elpie is going to share with us the approach that we use.

Elpie Bannister: Throughout the presentation we’ll share our insights in relation to finding talent. Some of these may resonate with you but one thing we know for sure is we don’t have all the answers. We’re continually refining our approach and would like to share some lessons we’ve learned along the way. We’ll be sharing these slides online afterwards and we’ll also share the resources as we go through afterwards.

We’ll focus more on finding talent but our approach to hiring is underpinned by our culture and our values which gives you some insight into the approach we take to nurturing talent once they’ve joined the team. What we’ll do is we’ll start off with a high level strategy in terms of growing our team and give some insight into our candidate value proposition. Then Alex will talk through sourcing candidates and screening candidates. And then I’ll pick up and talk about assessing candidates, closing candidates and then to conclude Alex will share some of the data that underpins our evolving approach.

High-level Strategy

So how do you grow a superstar team when you can’t afford superstar salaries? At Simprints we take a three pronged approach. We know that there are superstars out there that will connect with our purpose. We know it may take some time to find them and say we are patient. I’ll share some more insight into our value proposition in a moment. We also look to find superstars early on in their career and we host summer internships and offer six month internships when we don’t have headcount for an ongoing role. We’ve got a wide network of volunteers as well and we’ve got a very good track record of converting interns into core team members and 40% of our current core team started out either as interns or volunteers. And thirdly we look for talent and superstars in non-traditional places. For example, we’ve developed a very good relationship with the engineering school in France and our two cloud engineers came from that school. They previously interned with us as well. And we got another intern from that school joining us this summer.

The thing with that university is they require their students to have some experience of working abroad and also some experience of working where English is the primary working language. So Simprints is a great opportunity for them because they get both of those things in one experience from the university’s perspective, the internship can be unpaid. However, we’ve chosen to make sure we pay them the same salary as our other interns. In relation to the global talent pool, we also look for superstars in other places as well and we’ve got a strong track record of finding tech talent from San Paolo.

So, once we’ve found some candidates that we hope would like to join us. What can we do to ensure that we have a compelling value proposition to encourage them to apply? We’ve done some research and this is what we’ve come to think that we believe that there are five key factors that attract and motivate top talent and these factors underpin our approach to compensation and align strongly with our values.

We believe individuals care about how much they’re growing personally and professionally and at Simprints, we offer fantastic learning opportunities through stretch assignments, coaching and mentoring, and also, we have an individualised learning and development allowance for each team member. We also have a huge amount of freedom and flexibility in terms of when and where we work. We make sure that there’s the appropriate level of direction and support for each team member so that there’s full autonomy over tasks. We know that culture matters, and we do everything that we can to ensure that when people join, they love the people and the environment around them. Clearly salary matters, and whilst we don’t compete strongly on salary, we know that we need to make sure that the minimum threshold of the financial elements of our package are at a level enough to attract and retain awesome people. And as you would imagine with the social impact organisation, we believe we have a really strong purpose, and this is something that we know that motivated people care about. They want to have meaningful and challenging work. They want to know that what they’re doing makes a difference. And so, we do what we can to ensure that individual personal values align with our organisational values as well.

So, we’ve created our own benchmarking graph for illustrative purposes. And this is based on our knowledge of organisations through our own experience with them, but also just general observation. And you’ll notice that some organisations like McKinsey for example will try to compete on salary. Other organisations, like Google, offer fantastic learning opportunities and growth progression and we believe that most NGOs will also try to differentiate on their organisational purpose. Some organisations try to compete on all of these factors but at Simprints we’ve chosen made a conscious choice to double down on those areas that we believe we really can differentiate on. And so, we believe that we can offer fantastic opportunity in terms of learning, autonomy culture, and purpose. And we’re even managing to close the gap in terms of salary to make sure that we are able to attract and retain awesome talent. We’re closing the gap. We know we’ve got a bit of way to go but we think that’s great in terms of the fact that we’re non-profit. Okay going to hand over now to Alex to talk about sourcing candidates.

Sourcing Candidates

Alex Yang: So, for those of you who aren’t familiar they’re broadly two types of hiring that you can do. There’s inbound hiring and there’s outbound hiring. Inbound hiring is when the candidate comes to you. So, if you put up a job post on LinkedIn then that’s a form of inbound hiring as these candidates need to proactively apply to be considered for the job. Outbound hiring is the opposite. That’s when you go to the company or you go to the candidate. So, if on LinkedIn you are to proactively reach out to individuals, we thought you’d be a good fit for the role. He asked them ‘Hey are you interested in this job?’ That’s a form of outbound hiring. At Simprints, we’ve tried a little bit of both. In particular we found that using external recruiters didn’t work for us. The reason why is because since we pay lower salaries it makes the recruiters job harder because they have to find candidates who are willing to accept less money. At the same time recruiters are paid as a percentage of the hired candidate’s salary. So, it means that if they actually do find a suitable candidate there’s less incentive for them to do so. So, in effect we’re asking them to do more work for less money. And while we did manage to hire one engineer this way, most recruiters that we talked to they try for a little while and eventually they give up.

These days we primarily focus on inbound channels and by far our biggest inbound channel is stack overflow. With our inbound channels it all starts with a job post. This an example of one of the jobs posting we have it’s for senior Android engineer role. So, what is our goal with a job post our first goal is to maximise the pool of applicants that apply. We do this by getting them really excited about the prospect of working with us.

So here are some ways to do that. The first, is to list specific technology in your stack. So, Paul Graham the founder of Y Combinator and former BoS speaker once said

“like all craftsmen hackers like good tools. In fact, that’s an understatement. Good hackers find it unbearable to use bad tools, they’ll simply refuse to work on projects with the wrong infrastructure.”

So, with that in mind you want to showcase the modern technologies that your engineering team is using. This is what we do for that job post. We highlight the Kotlin language which became an official Android development language in 2017 and we and highlight the jetpack library which is a collection of software components that were only released this past September. So, these are pretty new, but the best candidates will recognise them and they’re going to jump at the chance to work with them. So, what do you do if your team isn’t using the latest technologies? Your team is still using outdated technology. Well in that case start by changing the culture of your engineering team. Change doesn’t happen overnight. For example, at Simprints one year ago our code base was 100% of the older Java language, and today it’s 96% of the more modern Kotlin language.

The next thing you can do to really get candidates excited about working with you is to mention books and resources that your team finds really valuable. This demonstrates two things to them; first it means that your organisation really cares about continuous learning and continuous improvement. The second thing is they haven’t even decided whether or not they want to apply for the job and you’ve already demonstrated value to them. So, for this job post we mentioned the book Clean Code which our engineering team uses as a North Star for setting the quality bar for production ready code.

The next thing you can do is to be really specific and quantify where possible. I’ll give a couple examples of how we do that. Lots of companies in their job post mentioned unlimited paid time off. That’s great. But at a lot of these same companies there’s a lot of pressure to not take vacation. So, what ends up happening is that employees end up taking fewer days off than they would have if you had a fixed PTO policy. Simprints used to be one of these companies but then we actually measured how many days people were taking. We found that two years ago people were taking 26 days to include bank holidays and now we’re glad to hear that that number has gone up to 31. The second way to be really specific is we have something at Simprints that we like to call Lego days. These are two days every quarter where employees get to work on a project that’s unrelated to their day to day work in the very beginning. These projects actually used physical Legos because we got a big donation from the Lego Group. But these days they can work on anything they want. We give specific examples of the types of projects that employees have worked on. One of the ones that we mention is a Lego train that delivers beer around the office.

So how could we make this job post even better? We could show a photo of the beer train. This is not a photo of the beer train. Unfortunately, this was a few years ago and so the best we could do was a photo of before they built the beer train. But to give you an idea this is a photo from our recent Lego day that was just a few weeks ago and among other projects we had two teams that competed to build a plane. One plane was powered by a battery powered propeller and the other to the other plane was powered by methanol. Let’s say that we do this really well. If we do a really good job of getting candidates excited, then we end up with a new problem. We end up with our team being overburdened with having to sift through tons of applications trying to find the diamonds in the rough. That’s exactly what happened to us. So, after we did this, we end up getting 450 applications in the span of four months for this one role. So, for our small team it was a huge time sink for them to actually process all these candidates. That’s why writing a job post you should really think about it as a dual optimisation problem. On the one hand you want to make sure you can encourage the strongest candidates to apply by getting them really excited about working with you on the other hand you want to discourage under-qualified candidates from applying because you don’t want to end up with too many applications. So how do you do the latter?

This was our initial requirements section of our job post and the problem is that all of these bullets were pretty bad at filtering out under qualified candidates. For example, if you look at the third bullet

•  ability to break down complex ideas into simple pieces

Talk to any engineer and they’ll say, ‘yes I can do that’. Look at the fourth bullet.

• Understand the importance of automating scalable testing.

This one actually caused a really big problem for our team because basically what happened is, we’d have candidates come in for the interview and the conversation goes something like this.

So, do you understand the importance of automated scalable testing?

Oh yeah absolutely.

So which testing libraries are you familiar with?

X Y and Z.

Okay, so how many for how long you have actually been writing tests at you’re at the companies that you’ve worked for?

Oh, did I say that I’d actually written tests? Actually, all the companies I’ve worked for, none of them believed in testing.

Facepalm.

So, what did we do? We realised we had to be a lot more targeted in our requirements. So, this is the requirement section after we made the change. So where is before we split it into tech skills and human skills. We now split it into about you and bonus points which is another way of saying must haves and nice to have. Now we are a lot more specific. For example, the second bullet says professional experience writing tests and running continuous integration systems. And under the bonus points, we now list the Kotlin language and we list specific Android libraries that we use in our tech stack. It is now a lot harder for under-qualified candidates to convince themselves that they have a shot at getting the job.

Screening Candidates

So, let us say we do this really well. We now have a robust pipeline of not low quality candidates. Our next step is to screen them. So, every candidate that applies to Simprints is sent an automated email that outlines the next steps in the process. Within this email we also include a link to a Google Doc that explains the entire hiring process, so they know what to expect. We copied this approach from Mikey Trafton who talked about in his 2013 BoS talk and it has been really successful for us. This is what that google doc that explains the entire process looks like. You will notice that the very first stage is a combination of a CV screen and a coding challenge. These used to be two individual stages but then we realised two things; first we realised that these aren’t reliable indicators of candidate strength and the second thing we realised was that some candidates who passed the CV screen we then invited them to the coding challenge and they never did it. And that means that we waste their time reviewing their CV.

The way we do it now as we combine the two. Every candidate is invited to complete the coding challenge and we only look at their CV after they have submitted their code. So, let’s say that after two weeks we haven’t heard from them. Then we automatically send them an email and say listen we have not heard from you so we’re going to be closing your application. What is the coding challenge you actually look like? We give candidates three options and we ask that they complete at least one of them in any language they want. Shouldn’t take more than an hour to complete. We also try to tie these coding challenges to Simprints itself. So, this is an example of one of the three options in this challenge. We ask the candidates look at the company values its inference has like relentless commitment to impact and robust as fudge. So, in computing. There is a concept that called an ASCII code. So basically, every character corresponds to a numerical value called an ASCII code. And so, we’re asking candidates to for each row of values, sum up the ASCII values for every single character and sort those values by those sums. So, the thing is that when they submit their code even before opening up the code, we actually already know quite a bit about the candidate. At the highest level. We look did they just finished one challenge, or did they attempt all three? What language do they write it in? Many candidates choose the older Java, but the best candidates choose Kotlin.

Here’s an example when the submissions we received to this candidate had an excellent CV and, in the email, he wrote that he had experience working with both Swift and Kotling. So, it’s great that he has experience working in multiple languages, but his code submission was in Swift and Swift is the language that’s used for iOS development. So, since he’s applying for an Android role that probably wasn’t the best decision and what that teaches us about him is that his decision making is a little suspect.

Here is another example. So, we ask all the candidates when you submit your code, you can either attach it directly to the email but do whatever you like. And so, some candidates do that. A lot of people will put it on github. This candidate sent it to us in a Microsoft Word file. So, word is great for writing essays but it’s basically unusable for writing code. So, in this case we don’t even need to open the file to know that we should reject him. So, when we actually do look at the code, how do we figure out which ones are good and which ones are bad? Well it is more than about whether the solution is correct or incorrect. If it is correct how elegant was the solution. The best ones are usually really easy to read with clear distinctions between the calculating of the ASCII sums, the sorting and then the printing of the result. They also use precise variable naming. You will not see names like I or J or thing 2 or temp. They will be really really specific and only if the code is clean, do we then actually open their CV. And what do we look for in the CV? First, we look for overlap between the libraries that they have experience with and the libraries that we currently use in our tech stack. We also look for side projects and open source contributions as evidence of their passion for technology. And finally, we learned that we should really look for extensive experience with testing.

All of these are much more important than the sheer number of years of experience they have. We get a ton of candidates who have been in the industry for a long time but they’re just coasting on the existing skills that they have. They are not spending the time to actually stay on the edge of the domain that they’re working in. On the other hand, we occasionally get a recent college grad or even a college dropout who doesn’t have that many years of experience but the level of knowledge they have is representative of someone who has much much more experience. From a candidate’s perspective the very worst thing is when you send your CV and you send your code into a black hole and you do it and you never hear back. So, from a candidate’s perspective what do they want? First, they want a job offer. If they can’t get a job offer, they want to get feedback on why they did not get the job. But the problem is that many companies end up giving them number three which is radio silence. Well we aim to do is we aim for 100% response rate.

So what that means is that when candidates first submit their code even before we’ve looked at it we first send them an email saying They just want to let you know we received your code and our engineers are going to take a look and let you know whether we decide to move forward. So, let’s say that we decide not to move forward with this candidate. In that case we let them know and we give them feedback. Let us say that instead we decide to move forward to an interview. We let them know and we give them feedback. So even if you can’t give personalised and detailed feedback at the very least you should be giving them templated feedback that incorporates the most common areas for improvement that you’ve seen of candidates that are applied for the role.

Now a lot of people ask why do you give feedback to candidates who you’re rejecting and it’s a great question. The way to think about it is think about it from a long term perspective. If you’re a candidate and you’ve spent time applying to this job writing code and eventually interviewing for this position by certain point, you’re quite invested. You have done a good job of getting them really excited about working with you. They will be disappointed that they didn’t get the job. But these candidates are going to go on and – the best ones even though they got rejected they’re going to go on to gain more experience and knowledge – the next time they’re in the job market they’re going remember the experience they had with you and they’re going to apply again. I was a couple weeks ago talking to a partner at an early stage firm and he told me that one of his employees had applied three times and the first few times she was rejected. But each time she came back she had improved on the feedback that he had given her until on the third try. She finally did get accepted and now she is one of his best employees. So now that you have candidates going through your screen how do you actually go about assessing them. Elpie is going to hear more about how we do that.

Elpie Bannister: Thank you.

Assessing Candidates

So, as Alex said he’s talked through the sourcing and the screening stages. I am going to now talk through some of the steps involved in our assessing stage. To start with the values interview. So, we start with the 30 minutes values interview which is conducted by a member of our talent team. So, it’s a chance to get to know the candidate. And it’s normally conducted over Hangouts and we ask a series of questions which give us some insight into behaviours which we believe demonstrate whether their behaviours align with our values.

We then move through to the technical interview stage and this is in two parts. The first part digs into a little bit more detail around the non-technical side but the main focus of that interview is obviously around technical expertise. And here our engineers will dig into real life examples of the candidate has said that they’ve experienced on their CV and ask them to give us some concrete examples of that. So, we can check that the skills and experience are they say they have aligned with what we’re looking for. And then we have a trial day. I will tell you a bit more about the trial day in a moment. But just for now I’ll say that prior to trial day we are completely transparent about our salaries. As I mentioned earlier, we don’t compete strongly on salary so it’s important that we have a good open discussion with candidates prior to the final stage in our hiring process. And earlier if they want to ensure that we understand what their expectations are.

Throughout all of the stages we make sure that there’s time for candidates to ask us questions. And in the technical interview a full 30 minutes is set aside for the candidate to ask us questions and also for us to share what our thoughts are about working for Simprints – so essentially to pitch Simprints.

During the CV and CV screen and coding challenge stage – I’m not going to go into as much detail because Alex has covered that already – but just to say in relation to the side projects that is something we do look out for at this stage, because we’re really keen to find candidates who are curious and who love to play and have fun with technology outside of their day job. It’s really hard to know much about the side projects at the CV stage though. So that’s something that we would dig into at the technical interview. For the values interview we asked the same questions for both technical and non-technical roles. And we do this because we’re looking to find people that align with our values particularly, we’re looking for a passion for learning a willingness to confront the grey. And obviously for people who do connect with our core purpose as well. And here are a few example questions that I’ll walk through.

So firstly, we’ll ask them to share a project or piece of work that they’re really proud of and why. We think this is a great question because it gives us insight into what the candidate cares about in relation to their work. And we’re looking for high standards confidence and ambition. We also asked candidates to talk us through a mistake that they made and what happened in that situation. Here, we’re looking for resilience and integrity and we’ll find that perhaps weaker answers might indicate that a candidate is willing to give up at that first hurdle or perhaps seek to assign blame to others in the team. Then we ask about what people might want to be learning in the next two to three years. At Simprints we very much believe in the power of continuous learning both individually and collectively. And so, we’re looking for candidates who are keen to push the boundaries of their learning and their skills and experience both in terms of breadth and depth. One of the things that we know we can improve on in this stage is to have a wide repository of questions that we can dig into with a more rigorous scoring rubric, so that any member of the team can conduct this value stage interview.

In relation to the technical interview. We refine the questions to be aligned to the scope for the technical role. But we also spend some time digging into some of the non-technical aspects as well. And so here we’re looking for understanding a little bit more about the mindset of the candidate in terms of their approach to software design, also looking for some self-awareness. And also, in relation to that passion for learning. So, one of the questions we ask is

‘most products have a warning label, if you had a warning label, what would yours read?’

And we get some very interesting answers on this one, but the reason why we ask it is because it requires a level of self-awareness. It also highlights that as a team we’re really keen to open up the discussion around what it takes to develop effective working relationships. So, if you know what someone’s frustrations or trigger points are then we can build that in to ensure that we have effective working relationships going forwards. We also ask whether they believe it’s better to be perfect and late versus good and on time. This is a classic engineering conundrum and it gives us some insight into the candidate mindset. We also ask them to share some real life examples of when they’ve had to balance the focus on deliverables and milestones and how they manage the interactions between engineering, product, and business which may inevitably lead to some tension. We also ask candidates to take a few moments and to think of something that they can teach us in a five minute timeframe. We’ve heard a ton of answers for this one and so we do give candidates a bit of guidance so they can choose a topic with some complexity but enough to add something that they can give us enough understanding surface level understanding and a five minute period. We’ve had some good and bad examples here. Some of the bad ones perhaps are around ‘let me tell you about the town where I come from’. Or ‘let me teach you how to start a startup’. With the first one about the town the response might turn out to be somewhat unstructured and rambling because they’re just talking rather than it being more structured insight. And then clearly with starting a startup – as you’ll hear from the talks today – it takes more than a five minute talk to be able to understand the complexity and the challenges involved in starting a startup.

Some of the good examples we’ve had have been where candidates have explained to us why the Earth has summer and winter seasons or linking back to the chess game. Some of the starting moves the chess and how to counter them. We even had one candidate teach us some of the basic motorcycle stunts. Clearly this was over hangouts and verbally, so we didn’t have a physical demonstration. But we did get a really good insight into that candidate and the question gives us a glimpse of something that we might not see otherwise. So, this candidate showed us photos of his motorcycle tour across India. And another candidate also shared insights of their three month tour of Europe around heavy metal festivals. So as I said with this section we really dig into the technical expertise and the questions are refined around the scope of the role and so we look at knowledge based questions so more conceptual knowledge based questions but also some more practical situation based questions and both are important to ask because candidates are sometimes better at one or the other. In relation to the conceptual based questions for the Android role, for example, we might ask a candidate about their knowledge around the android architecture components we might ask what experience they have of using them and then also we’ll ask what problem do they think Google was trying to solve in introducing them. And we asked that last point because any candidate perhaps can brush up on their conceptual knowledge but it takes a really good candidate to understand the bigger picture to understand the pros and cons of the different tools to explain what problem they were trying to resolve. For the more practical based questions what we’ll do is we’ll share some insight in terms of real life challenges that our engineering team are facing. And this is a great way to share some insight of what it’s like to work at Simprints whilst feeling less contrived. And even if the candidates may not have had the exact experience in that challenge. Some of the better candidates are able to use their knowledge and experience to at least give a couple of options that we could consider to resolve that problem.

Trial Day

Moving onto the final stage of our hiring process we have what we call a trial day and it’s something that we debated in the team and one of the teams had actually the word trial could be something that they feel people feel like they’re on trial. But we’ve actually phrased it because it’s a trial for them to see us in practice, for them to have a trial and a day in the life of simple prints and here’s a schedule of an example trial day. We refine the schedule based on the role and we also refine the project based on each role as well. So, the day will involve one to one meetings, and also time for project work, will also involve the candidate in the day to day rhythm of life at Simprints – like all team stand up and lunch. With the project it can also be phased direct during the day. So, the first phase could be around the design and planning and then the second phase would be around execution and implementation. When we get to the demo phase at the end of the day, what we’re looking for is obviously technical skills but also some of the non-technical skills around communication and we’re looking for candidates who are able to communicate what they’ve done throughout the day with a clear structure. At the high level, we’re looking to them to explain what the scope of the problem was and what they were trying to achieve with the approach that they set out and do they have a demo to actually show us. At the next level down, we’re looking for a bit more insight in terms of the decisions that they made and perhaps some of the trade-offs that they thought about. So, what architecture did they use and why? And then at the lower level we’re looking for them to walk us through the individual code files that they’ve created to implement their strategy, giving us some insight into whether their code is clean and whether it’s production ready.

So, trial days are a huge investment and we believe they’re something that is of benefit to simple prints and we also believe it gives huge benefit for the individual as well. So here are some stats that we pull together based on a three month period from December 2018 to February 2019. And these covers both technical and non-technical roles. So just want to set the context for the stats there. On average we have five trial days a month during that period. We had a particularly busy period during February where we had multiple concurrent trial days and we even had three trial days for three different roles on the same day. It was a busy day. We also worked out that the average of our average cost of our trial days is £900 for each trial day and that covers international candidates coming over; so, it covers flights, accommodation, visitors’ visas, and other incidental expenses. For any candidates based in Cambridge where we are clearly those expenses aren’t incurred. And across the technical and non-technical roles that we’re hiring for, we’ve had a 50/50 split of U.K. and non UK trial day candidates. In terms of Offer Rate after trial day 62% of the candidates who came to trial day we offered. And that’s okay because actually we sometimes run competitive hiring so we might bring in three candidates for trial day and only want to make one offer. So actually, this is a statistic a metric that we’re not trying to over optimize on. Alex is going to talk more about our offer to acceptance ratio in a moment but it’s interesting to note that by May of this year (2019) 46% of our team will come from outside of the EU and will be on either a Tier 2 or Tier 5 visa. So, as you would expect there are pros and cons with trial days, and this is something that we debated within the team. It’s something that worked very well for us to date. But does it still scale? Will it make sense for us going forward? We believe the pros are that, as I said, it gives candidates to experience a day in the life of Simprints. We very much want people to join us who know what they’re getting into and the way to do that is to give them a day in the life of the organisation. Throughout the hiring process, we’ll have built relationships with the candidates already and the trial date gives a chance to augment that even further by meeting other members of the team outside of the technical team they might be joining. And also, it gives us a great opportunity through the project work to really deep dive into the skills that they have and to make sure that they are a good fit with what we’re looking for. Some of the cons as you can imagine or the cost that’s involved but the cost is both financial and non-financial. They take up a lot of time and we believe it’s time well invested, but it does take time and sometimes the logistics phase of securing visitors visas can be quite long. And so, some candidates do drop out at that stage and we have three really strong technical candidates that disappeared into radio silence through this stage. We have no idea why they dropped out because they didn’t respond to the e-mails. And so, whilst that might be a good thing in the long run, we do feel it’s a bit of a missed opportunity.

Also, when you’ve built up that relationship with people throughout the whole process and you spent a whole day with them, they’ve got to know the team you’ve gone and had dinner at the end of the day. If you then decide after the decision meeting which the whole team can come along to and we decide whether we want to proceed to offer it can feel bad to say no. And so what we do as throughout the process we will make sure that we give detailed feedback to the candidate and we talk this through with them over a Hangout call or in person if they’re in Cambridge but over hangout call normally to give some reasons an explanation as to why we’re not proceeding to offer. So once we’ve invested all that time in building the relationship with the candidate and giving them a chance to get to know Simprints we want to make them an offer and what can we do to make sure that we have a positive outcome. Prior to communicating the decision to the candidate, we spend time with the hiring manager to really tailor that message to the candidate. Throughout the process, we’ve got to know the candidate. And so, we’ve got some insight into what they care about and what some of their fears and concerns might be. For example, if they care about using the latest technology then we’ll emphasise that during the offer call. If they want to use that technology for positive impact, that’s something we’ll mention. And then if they’ve got any fears and concerns about joining Simprints, be that around the salary or the total value proposition that I talked through then we’ll spend time going through that. And also, in terms of international candidates they might have a fear of the process of securing a visa, which believe me can be quite painful at times. Also relocating to the UK so we spend time talking through those during the offer call.

We follow up the offer call with a very personalised offer letter that we’ve drafted with the hiring managers input – again, tailored to what we’ve learned through the hiring process of the candidate. We outline 10 key reasons why we believe they would like to join our team. This example here is for one of our Android engineers who is going to be joining us soon and he, throughout his studies, actually worked full time for the whole five years of his studies and we were super impressed with that in terms of desire and dedication to make it happen. “Make it happen” is one of our values and we wanted to emphasis that in the offer letter. We also supplement the offer letter with the 10 reasons with a handwritten note which has been written by those people that the candidate met during the trial day. Again, we believe this cements the relationships that have been built and also gives that future focus. So, this example circled here is from Rid1 another one of our engineers our Android engineers. He was really pleased to have met Alan during the trial day and is looking forward to building awesome things together. Should he accept our offer and he did. Which is great. So, I’ll pass over to Alex now to conclude.

Be Data Driven

Alex Yang: So so far, we’ve mainly covered the qualitative aspects of how we hire Simprints. But I wanted to spend the last few minutes we had to talk about the quantitative aspects which are equally important. Every hiring process is the product of many gradual changes, the way to think about making your next change is to think about it as an experiment. Look at the data behind it. Did the experiment go well, or did it not go well? The more data you track, in general, the easier it will be to identify opportunities for improvement. It also helps tremendously to use an applicant tracking software or ATS which will automatically collect a lot of this data for you. Some of the ATS to consider are Greenhouse, Applied, and Recruiter box. So here are three key metrics that you should keep your eye on. The first is days to offer. That measures on average for successful candidate. What is the length of time between when they first applied to when they received an offer? You can actually break this down much further to look at each individual stage, but days to offer is a really good high level metric that looks at the speed of your pipeline.

The second metric to look at is qualified candidates for hire. So, if you define a qualified candidate say as someone who passes your CV screen and coding challenge and comes in for an interview. This is then asking how many candidates do you need to interview before you find someone who you extend an offer to and they accept? This is a really good metric to look at if you want to know how long a currently open roll will be open before you’re able to close it.

And finally look at offer acceptance rate. This is really simple. The number of offers that you’ve extended that were accepted over the total number of offers that you’ve actually given. And so, because this is Business of Software, we’ve decided to be completely transparent with our numbers.

For days to offer our current number is 60. And of that 60, logistics for trial day candidates coming from abroad makes up over 30. For qualified candidates for hire. The current number is 12 and that means that we need to interview 12 candidates before we finally get someone who’ll actually be joining the team. That means that every person we interview has around an 8 to 9% chance of being the one that we bring on. And finally, our offer acceptance rate for engineers is 86%.

So, one thing that we’re currently doing is we’re transitioning to a different ATS with stronger analytical capabilities because a great ATS will be it we’ll be able to monitor and analyse all these metrics for you. In our case we’ve got to do a lot of this manually. So even some of these numbers are out of date and we think we’ve improved on them. But I wanted to also share some of the other one off analysis that we’ve done. So first this is a breakdown of the inbound channels that we look, at as you can see Stack Overflow is by far the biggest representing over 90% of the volume of our engineering candidates. So, when we saw this our next question was ‘well how efficient are each of these channels?’. And so, one thing we noticed is that we look on the left stack overflow candidates only one% of them actually make it to trial day. And on the right if you look at some of the other smaller but promising data sources, we have sample size are small but 25% of them make it to trial day.

So, what are the takeaways from this. We had two takeaways. The first takeaways we should experiment more with these smaller channels that seem to be working for us. And the second takeaway is we look at stack overflow numbers, wow they’re really low but that doesn’t mean that we should get rid of stack overflow. Instead that means that we need to do two things we need to be more targeted with our job posts and we need to reduce the hiring burden. And so, you’ll notice that these charts are a little bit out of date. You’ll see that the first two bars in each graph show a CV screen and coding challenge as two separate phases. So, we actually reacted on this feedback and so, with targeting we update our job posts. And now anecdotally the percentage of candidates who are completing all three of our coding challenges in Kotling has gone through the roof. And then second, by reducing the hiring burden, that was the motivation for why we decide to combine the CV screen the coding challenge. We ran the numbers afterward and we actually found that this reduced our hiring burden by nearly 50%. So massive massive change.

Another thing that we look at as we look at the geographic distribution of candidates. We don’t actually ask this to our candidates, but we do ask them for their phone number so by using the country code of the phone number we’re able to see where they’re actually applying from. And as you can see from the first two bars over 50% of candidates are applying from India and Pakistan alone. We think that’s due to a few reasons. The first reason is stack overflow is user base and how that’s located around the world. And the second reason is the concentration of software development and software engineering in South Asia. And finally, there’s massive demand in Asia for western companies who are willing to sponsor international visas. The other thing you’ll notice is that local candidates from the U.K. is only 5%. And so, given the cost and logistics involved in running trial days we’re really keen to increase this number.

So, to recap some takeaways here when sourcing look at non-traditional channels to find candidates that match your hiring needs. Be really specific with your job postings so you can encourage the strongest candidate to apply. And split your requirements into must haves and nice to have to make sure that under-qualified candidates are discouraged from applying. From a screening standpoint prioritise coding challenge over CVs as they are a more reliable indicator of candidate strength. Consider combining the coding challenge and CVs into a single screen to reduce your hiring workload and finally aim for 100% response rate and give feedback even if templated when you can. When it comes to assessing make sure you test both conceptual and practical knowledge, some candidates will respond more to one or the other and they actually don’t wait until the trial day to line on salary. Finally consider using a trial day as a good way to evaluate international candidates and to pitch the company to them. When closing, go above and beyond with a tailored approach by using personalised offer letters and a signature page signed by every person that that candidate met. And finally be really data driven measure everything you can and use an ATS to capture that data and monitor it for you and finally monitor some key hiring metrics like days to offer, qualified candidates for hire, and offer acceptance rate.

So, before it closed. I wanted to share a personal story about how I joined Simprints. When I first heard about the opportunity. I thought there’s a less than 5% chance that I would actually take it. And in fact, I felt so guilty about it that I asked that they schedule my trial day to work around vacation. I had already planned to Italy and that way they didn’t need to pay for an international flight from New York. Instead they just had to pay for a Ryanair flight from within Europe. And so, when I actually showed up for my trial day. What impressed me the most about Simprints was the team. I’ve worked at a bunch of different organisations now including two of the perennial top five best places to work and what I noticed is that the team was incredibly cohesive. Also even looking at the youngest employees they had experience and knowledge that went far beyond their age. The team was also incredibly diverse. So, we had over 20 nationalities represented among a team of around the same size. And for me the kicker was the signature page. Seeing the signature page signed by every person I’d met that day. So what I believe, is that when you spend as much time at work as we all do it’s really important that work is fun and Simprints team truly is one of the best so I wanted to thank you all for listening.

Hopefully, you’ve taken away a couple of tips they can bring to your own hiring process. If you want to talk about this more. Feel free to email me and Alex@Simprints.com. Shockingly, we’re actually hiring for multiple roles. So, if you’re interested come talk to me afterwards and I can connect you with the hiring manager.

Thank you.