mevu

How to Interview Entry Level Software Engineers

Clover’s First Associate Cohort

Technical interviewing is hard. The Internet is full of stories on the sorry state of the technical interview process and how it’s difficult to properly evaluate a candidate through white-boarding, homework, or really anything else. Interviewing entry level engineers could be even more difficult because theoretically they lack a basic understanding of writing software.

Last year, Clover engineering created its Associate program and had to figure out how to interview entry level engineers. Does it make sense to use the same interview process we’d used for other engineers? How much would we have to “lower the bar” in order for us to effectively pull in candidates?

Turns out, we didn’t need to change the interview process, or lower the bar much…or at all.

First, we decided what qualities we wanted our Associates to have.

A team of engineers worked on a set of traits that we thought our associates should have. If the program was successful, Associates would graduate to full-time engineers, so we wanted to make sure these traits aligned with the rest of the team. Some of the qualities we looked for in our associates in particular were:

• Resilience: Learning on the job is hard and we assumed that the associates would make mistakes and struggle through difficult concepts. We needed people who could endure these struggles and bounce back ready for the next challenge.
• Willingness to learn and the initiative to do so: Clover would assist the Associates in their growth, and provide teachers and mentors to help along the way, but any incoming Associate would need to be responsible for their own growth.
• Humility: This is an important trait for all Clover engineers, but we paid special attention to it in our Associates. They would have to learn from those around them, be respectful of others, and be able to take difficult feedback with grace.

Second, we decided how to evaluate for each trait.

Once we had our list of Associate traits, we carefully defined each one, as well as the specific ways to assess each trait.

It was also important to define “ways not to assess” because it’s very easy to get incorrect signal with good intentions. For example, when assessing humility, we did not want interviewers to mistakenly look for “submissiveness.” We wanted people who would accept feedback without defensiveness and treat their interviewers as potential collaborators, not commanders. We encouraged them to ask tough questions, and to challenge the interviewer when they did not understand something or if they disagreed with a suggestion.

Similarly, we did not want to evaluate “willingness and initiative to learn” based on the number of independent projects on their personal GitHub profile. This is only an indication that the person has spare time to complete side projects, and doesn’t necessarily mean they’re hungry to take on new challenges in their existing role. We wanted associates to feel like they could live their lives outside of work, while feeling supported to learn in whatever way is best for them.

On each trait, the candidates were rated on a three point scale: Fail, Pass, or Exceed. We chose this system because it is simple, and allowed us to give credit to candidates who were particularly strong in a given area while acknowledging that excellence across the board was not completely necessary. Notably, “potential” is nowhere on the scale. Many posts talk about hiring entry level engineers on their “potential,” but the term is seldom defined. Most people use “potential” to describe a sense that, even though a candidate currently cannot be an autonomous engineer, someday in the future they could be. We did not attempt to predict the future for any of our candidates but rather evaluated them on what they presented through the interviews.

Third, we designed our technical questions

Deciding the technical skills to evaluate was a long process. We expected to teach our Associates most of the technical skills they would need to do their job, but we couldn’t accept candidates that were completely blank slates. We decided the qualities we would look for in associates were:

• The ability to think through a problem logically
• A solid understanding of basic imperative programming and data structures in some language (primitives, arrays, hash tables, loops, if statements)

The thing we decided we would not test, which we do in our other candidates, was:

• The ability to define a problem based on an underspecified problem statement

Given these requirements, we used our standard interview questions, but with some scaffolding to help candidates define the problems space and scope their work.

We kept the standard phone screen question we use for general engineering candidates for our Associates, but provided a sample function signature and basic tests, where a general engineering candidate is given an empty editor.

Also, we chose to completely scrap whiteboard coding and assigned homework instead. Our homework for general engineers mimics a real world problem and expects the candidates to come back with a tested solution to that problem. We didn’t use the same question for Associates because it required the candidate to understand the breadth of the problem, and scope their answer appropriately. Instead, we used an algorithms question that we had previously used as a white-boarding exercise. We created test cases for the problem so that the candidates would know what was expected of them and would not need to worry about setting up a new project and test case.

To get a read on systems design, we present general candidates with a broad problem and expect them to diagram a system as a solution. We used the same question for our Associates, but described the basic components of the system then asked them to describe what one of the components would look like. If there was time, we’d have them move on to other components.

Finally, we interviewed them.

The engineering team was concerned their benchmarks of what a ‘pass’ looked like would not apply, and interviews would be materially different than general engineering candidates. To mitigate these problems, we batched interviews together with a consistent subset of the engineering team to make easier comparisons. We also clearly defined the pass and fail states to reduce any ambiguity.

How did it turn out?

Amazingly well. We were impressed with the overall quality of all our candidates and had a difficult time deciding which of the Associates we would be able to bring on. In the end, we settled on a cohort of seven people, four of whom were selected through the interview process, and all of whom completed program and are now “growing” Clover engineers. Most importantly, we were able to interview, onboard, and grow a group of junior engineers without “lowering our technical bar” or compromising our standards.

UPDATE:

I am so surprised and humbled by the response to this article! Thank you all for reading. Stay tuned for more stories about Clover’s Associate program.

If you like the idea of working on a team with smart, thoughtful people, Clover is hiring in both our San Francisco and Jersey City offices! We’d love to talk to you. Take a look at our careers page to learn more.

Finally, I’d like to make a correction to the article. We ended up with a final cohort of seven associates, four of which came through the interview process. The others were former interns, or transfers from other teams. I’ve updated the original sentence for accuracy.

My profile bct:https://bitcointalk.org/index.php?action=profile