The weird thing hiring managers are really looking at during tech job interviews

Software engineering has quickly emerged as one of the hottest and most in-demand job skills over the past decade or so. A skilled software engineer can be invaluable to a company of any size. From emerging startups to established industry leaders, everyone needs a good software engineer (or a team of them) in 2020. Such positions also offer attractive and lucrative starting salaries.

Of course, just like with any other great job, prospective software engineers must first pass through a rigorous hiring procedure. Most of the time, that process includes a technical interview supposedly designed to gauge the job candidate’s relevant software skills. 

However, a new study from North Carolina State University (and Microsoft) is offering up some advice every software engineer getting ready for a big interview is going to want to read. The research team concluded that technical interviews are really intended to determine if the candidate is prone to on-the-job performance anxiety more than anything related to cut-and-dry software skills.

A typical technical interview for a software engineer usually consists of the candidate being presented with a complex software problem. Then, the candidate is asked to write out an answer to the problem using code on a provided whiteboard, all while explaining their thought process verbally to the interviewer (or team of interviewers). 

The study’s authors contend that this approach to hiring is unfairly eliminating skilled job candidates. While hiring managers may feel justified putting candidates through the interview ringer for such an important position, public speaking and demonstration skills shouldn’t make or break a software engineer’s chances of landing a great job. 

“Technical interviews are feared and hated in the industry, and it turns out that these interview techniques may also be hurting the industry’s ability to find and hire skilled software engineers,” says study co-author Chris Parnin, an assistant professor of computer science at NC State, in a release. “Our study suggests that a lot of well-qualified job candidates are being eliminated because they’re not used to working on a whiteboard in front of an audience.”

This whole project originally started because prior studies had noted that many veteran software engineers and tech employees feel like the entire technical interview structure within their industry is deeply flawed. So, the research team decided to investigate this topic themselves.

For their research, the study’s authors gathered 48 undergraduate and graduate computer science students for an experiment. Half of those students were given a typical technical interview as described above, with an observer watching as they solved and verbally explained a software problem. The other students, though, were allowed to solve their software problem in silence within a private room away from prying eyes. 

Each student’s performance on the provided software problem was measured according to both accuracy and efficiency (does their solution to the problem work? And if so, what resources are needed to run it?).

“People who took the traditional interview performed half as well as people that were able to interview in private,” Professor Parnin explains. “In short, the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding.”

Besides the possibility that fantastic software engineers are being overlooked due to poor public speaking skills, the research team at NCSU also believes that technical interviews can be manipulated to favor certain candidates over others.

“For example, interviewers may give easier problems to candidates they prefer,” Professor Parnin comments. “But the format may also serve as a barrier to entire classes of candidates. For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed. Our study was limited, and a larger sample size would be needed to draw firm conclusions, but the idea that the very design of the interview process may effectively exclude an entire class of job candidates is troubling.”

Additionally, this research begs another question regarding all job interviews in general. What’s more important: the actual job or just the interview? Imagine a candidate, let’s just say a software engineer, spends weeks on end preparing for a technical interview (practicing on a whiteboard, working on public speaking, etc). Well, that person may end up acing the interview and getting the job, but in the meantime, they’ll have become rusty when it comes to actual software skills.

“The technical interview process gives people with industry connections an advantage,” says first study author Mahnaz Behroozi, a Ph.D. student at NCSU. “But it gives a particularly large advantage to people who can afford to take the time to focus solely on preparing for an interview process that has very little to do with the nature of the work itself.

“And the problems this study highlights are in addition to a suite of other problems associated with the hiring process in the tech sector, which we presented at ICSE-SES,” she adds.”If the tech sector can address all of these challenges in a meaningful way, it will make significant progress in becoming more fair and inclusive. More to the point, the sector will be drawing from a larger and more diverse talent pool, which would contribute to better work.”

The full study can be found here. These findings are also set to be presented at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering this November.