How to run a great technical screening (pre-screening)

John J Masse -

I have been interviewing recruiting and coaching engineers since early 2013. During that time I have developed frameworks, onboarding strategies, and interviewing workshops that help folks succeed in recruiting.

In this post, I wish to share with you one process for the technical screening, commonly known as the phone interview, of engineering candidates. What is difficult to communicate in a simple blog post is the energy it takes to do these things well. But what I hope to impart to you are details that will make a difference in how you interview. By reading this outline of the screening process you will have:

  • A structure to consider when preparing to interview a candidate.
  • An idea of starter questions that help dig around how someone may culturally fit into your team.
  • How to plan technical questions.
  • How to create an ideal scenario that will help candidates you interview succeed.
  • Generally, have more fun doing interviews.

Pre-planning

Before getting into the details it is important to understand first the environment we are going to be working on and the goals we wish to achieve from our meeting with the candidate. The scope of a good technical interview extends beyond asking technical questions.

The closing of a technical screening should conclude with feedback on the following traits:

  • Passion.

  • Innovation.

  • Cultural fit.

  • Current general technical knowledge.

  • Ability to learn and coachability.

  • Interest in continued education.

Later I will describe each section and provide example questions, in the meantime let's look at our setting.

The setting of the meeting

The setting for this meeting is just as important as scheduling it. As humans we communicate in different ways, body language is one of those ways. Not being able to see the person we are speaking with obscures how we naturally interact with one another. As people, we look for visual queues. Signs for when it might be appropriate to help, to move onto another topic completely or when it might be time to dig deeper into a topic.

If at all possible teleconference with the candidate. If all else fails, reach for the phone.

Set your expectations

Positive thinking is a powerful ally. It changes our mood, our perception of those around us and our general perspective on life and problem severity. Go into every interview thinking that the person you are about to meet is the best person they can possibly be.

Go into the meeting looking for reasons to hire this person.

Set the candidates expectations

Work to make the candidate as comfortable as possible. A candidate that is comfortable will be more likely to be themselves and more inclined to respond to your questions with their experience. We do not want candidates in their survival state of mind. Once there it is almost impossible to get anything useful out of them. There are scientific reasons as to why this happens which we won't get into in this post - but a read on self-preservation is a good starting point.

It is no simple task to get a candidate comfortable in an interview and it happens less frequently than we might like. A comfortable candidate will more easily be able to draw upon their own life experiences and knowledge. A candidate focusing on how well they are interviewing is distracted. Candidates distractions cloud our ability to determine their skills and what the candidate is capable of.

Some tips on creating an easier experience for a candidate

One strategy to reduce fear is to reduce "unknowns". Setting expectations is an easy way to start doing that.

For example:

Alight, so over the next 30 minutes to an hour we are going to cover a few topics. First we will go through some introductions, next, we will chat about technology and go through some technical questions, finally, we will have 10 to 15 minutes at the end for you to ask any questions you might have. O.K?

This sets the general structure of the discussion we are about to have with the candidate. We may be working with an outline but the person we are screening has no idea how we are thinking.

This is not a test...

This is a technical interview. The candidate on the other end of the phone knows that their skills are going to be put to the test and people, typically, hate tests - so don't make this a test, make this a conversation. The best conversations I have had were around problems I had been trying to solve or language constructs that I had been bantering with colleagues over. One better than technology banter would be being taught something new.

That is not to say we avoid technical questions. We need them to help set the context and get the ball rolling. More often than not we will fail to make the candidate comfortable enough to relax and participate in a conversation and in those situations, it seems that all we are doing is asking questions. We can improve or otherwise break the barrage of questions by making the conversation educational as well by sharing our own understanding of the questions we are asking.

What about the questions you are asking is important to you?

Now that we have discussed some pre-planning topics of optimizing the environment for success, let's get into the actual conversation. I am going to cover the more complex version of the phone screen, and that is the non-telepresence version - or lack of Google hangouts or Skype.

Keep control of the conversation

This is hard to do and takes a certain amount of confidence and tact. Watch out for where the conversation is headed. If the candidate ends up on a runaway train to somewhere that is not getting you the information that is going to help them earn a role in your company then it might be time to reel them back into scope. Good candidates will appreciate you getting on with the topics that are most important to you.

That is awesome how you started several startups and learned from that experience. I bet there are tons of stories and lessons learned. If you wouldn't mind, however, I am really interested to hear more about your experience with state management.

The previous statement uses summarizing as a technique to show respect for the information you are receiving but at the same time guides the conversation into a more relevant subject.

This is your time as much as it is theirs. Help the candidate succeed.

Screening outline

From here, I am going to detail a 30 to 60-minute technical screening. Originally I had designed this as a 30-minute session, but it also works for up to an hour.

Prepare them for surprises

Often times when in a heated discussion about a given topic the candidate may share something that is particularly interesting and we want to ask more questions. When that happens we may want to interrupt the conversation with a question. It is okay to interrupt, but before the discussion gets too far along warn the candidate that an interruption may occur, and when it does - that you are not trying to be rude, it is just that they had shared something that was particularly interesting.

Interrupting someone when they are talking is jarring.
Explaining your intent up front will help keep the conversation fluid.

This is a simple example of just showing some courtesy and continuing to enable the applicant to be successful.

It's O.K to Google!

Be honest, how much of your day to day comes from the internet? CTO's of companies are not sending their own white papers around - they are reading them from other people and emailing links to blogs and articles other engineers write - often times not even from other CTO's. Don't make the technical exercises about memorizing stuff, make the technical interview about solving real business problems and having fun as a team.

I let the candidate know that Googling is okay with me. There are plenty of strategies we can use in lines of questioning that force the candidate to think critically about what they are saying. A very simple technique I use is:

... and why is that important?

Introductions

Take the first two to five minutes to introduce yourself and your role with the company. Remember that this conversation is also about you selling the company as a fun place to work and grow. Share your most recent work and how it challenged you. Share the last bug you made or production issue you had to work on ( within reason of course ).

An introduction is an easy way to begin breaking the ice with the candidate. Once you have explained your role ask the candidate to introduce themselves. If this feels tough the candidate could be anywhere from mildly to frantically uncomfortable, but do your best to listen.

So my name is { INTERVIEWER_NAME }, I have been with the company for, geez maybe three years now. I have been writing a lot of Javascript and most recently I had to fix a bug in production that I created by overlooking a detail in some promise resolution. Tough, but I learned this interesting thing about Promises from that experience.

{ CANDIDATE_NAME }, tell me about yourself and what you're hoping to do next in your position.

Engage and acknowledge

In coaching and management training we learn this skill called "chunking". Though not the best name for a skill... it is a technique we use to travel around topics in a conversation. Though outside of the goals of this post, to give you a quick intro into how to use Chunking:

Dig deeper into a topic by asking more specific questions about things they are saying. Move up out of the details by asking more broad overview type questions... repeat.

If there is something about what the candidate is sharing, and it is within the boundaries of being a professional, ask a question that would draw more information out of their intro.

- Candidate

And most recently I have been really interested in full stack engineering!

- Interviewer

That's great, me too! What about Javascript has convinced you to study and build stuff with it?

Though the response may be fun, we are already building a conversation around technology which is where we want to be.

Team fit

Just before getting into writing code or working through more technical questions, explore the culture side of the individual first. Doing this first continues to build trust with the candidate. Here is another opportunity for us to share our own opinions about technology or what is important to our team.

Some example questions are:

  • Do you work as part of a team in your current role?

  • What is important about working in teams?

  • Do you guys have a code review?

  • What types of things do you look for in a code review?

  • Have you gotten into any flame wars recently?

  • What do you expect from your leaders?

  • Share with me a time when you taught someone something.

  • Have you paired programmed before?

It is also good to engage by sharing your companies way of working and your thoughts on the importance of topics you discuss.

Code reviews are great here. We have a few rules, for example, each pull request requires at least two approvals from our teammates before the code is merged and considered for production.

A couple of things are happening through this conversation. First, there is some thought sharing happening - this is validating and educational for the candidate. Secondly, we are giving the candidate some information about the organization they are applying for. This helps build enthusiasm within the conversation more so if the stuff you are doing is interesting.

About team culture

It is up to the team to define their culture. Defining the culture is not as easy as it sounds, but a nurturing environment where people are allowed to learn from mistakes and share ideas is a great place to be.

Coming to the end of the first ten or so minutes and chatting up team fit related topics we are looking to gain a sense of what values the candidate has and what it might be like to work with this person on your team. We have also been talking for a little while now and we may have built up a bit more trust in the conversation to help ourselves and the candidate get into the nitty gritty of the technical portion of the interview.

Technical exercises

The goal in the technical portion of the interview is to gain a general sense if this person has been writing code and if they can do what they say they can do. An ideal scenario would be that the person on the phone was able to teach you something that gets you excited to meet them. Before we get into too much detail about the technical portion of the screening run a diagnostic of the types of technical questions we are preparing to ask.

No GOTCHA'S allowed

Save the riddles for the kids, this is about getting work done. Focus on the applied science of the work you are setting out to accomplish.

Asking technical questions is not about getting to say "GOTCHA!".

No gotcha's allowed. None! And if you do use a gotcha make it fun trivia.

An example gotcha could be:

  • What is the typeof NaN?

  • What does the expression [1, 2].hasOwnProperty(0) evaluate to?... How come?

  • What prints to the console and why?

function() {
 var a = b = c = 20;
}

console.log(c);

Keep your questions practical. Practical in the sense of the actual code you will need them to write and actual problems you will need them to solve. If optimizing sort algorithms is in your wheelhouse then ask questions around writing and optimizing sort algorithms. If you need someone to build and iterate on a responsive web page, ask questions about media queries and the box model. Stay out of the weeds of understanding the language intricacies and save those for the next book on "The Best Parts".

Design questions within reason

Make your questions reasonable. Don't make them the most difficult convoluted case possible. And if you do, make it fun! Make it something you do together as an exercise.

A fun way to engineer technical questions is by committee. Work with your colleagues to design questions around challenges you work on every day.

Include some Google'able stuff

Just because folks can Google an answer is not a bad thing. Throw in a few of the common easy ones. Let the candidate be successful. You might actually be surprised as to how often someone might not be able to answer the gimme questions and leaves you with an opportunity to teach something. Since I work a lot with full stack Javascript engineers some of my common goto's are:

  • What does the term "semantic markup" mean to you?

  • What is accessibility as it applies to building software?

  • What is the box model?

  • What tools do you use to author CSS?

  • Can you explain how inheritance works in Javascript?

These are not terribly interesting topics, but they help make the candidate more comfortable. We can also use these topics as conversation starters.

For example:

Did you know there are tools that will emulate the various types of color blindness? We can use these tools to make sure the colors we have chosen for the design create enough visual contrast for these users. Pretty cool huh?

Create an opportunity for even the simplest topics. There is always something neat hidden away in the basics.

Make it interactive

I always keep a library of starter code snippets in a JSFiddle or JSBin handy when I am chatting with someone about writing code. I have used tools like JSFiddle to mock up a scenario and ask questions around it. What I also like about JSFiddle is the collaboration mode in case the candidate gets stuck I can dig them out quickly.

The tricky thing about interactive exercises is that they may drain a lot of time out of your meeting if we are not good at time management.

Stay on top of the candidates progress and use good time management to help them along. Keep notes on the places they got stuck. And sometimes more importantly, how easy was it to teach them and help them along...

Q&A

Always leave 10-15 minutes at the end to answer questions the candidate might have. It is important to keep in mind we want the candidate to be excited about taking a role with the team. While answering these questions be thorough and share personal experiences on the topics where possible.

Closing the discussion

In the closing be courteous and thank the candidate for their time. Typically you will have to have a chat with someone else on your team about the candidate and from there they may invite the candidate in for an in person interview.

Have a bit of caution when sharing your hiring process. Avoid promising things you may not be in control of. It is okay to say that you are unsure and would prefer to refrain from sharing potentially inaccurate information.

Run the best technical phone screen you know how

This post provided details to consider for the next time you need to chat with a candidate. Be motivated about hiring, this is your opportunity to pick your teammates. Find people that will help you be successful, you would be amazed at the excellent folks that get passed on, better your odds with critical thinking and good communication. I couldn't be more thankful for the new opportunities meeting engineers has brought to me. The things I have learned and the friends I have made have made focusing on these skills worthwhile. I hope you get to see that as well.