I’ve been doing a lot of interviewing lately. I have admittedly not spent much time on the interviewee side of the table over the past decade, so for the last couple of months it has been exciting, fun, and enlightening to experience a range of interview approaches. It’s also given me a chance to reflect on interviewing as a practice, and I thought it’d be worthwhile to capture the things I currently believe about talent acquisition.
There are two thing that I’ve come to learn about humans as it relates to hiring. The first is that, as a species, we are terrible at assessing potential.
Humor me with a quick thought experiment. Imagine that through some wonderful magic, we have found a way to quantify every aspect of an engineer’s individual performance. Not only that, but we have been able to go back in time and quantify the past three or four years of their work. And even better, we can also send people to their current place of work to observe them. Now, having done this, we gather the top 200 engineers into a room and every company that’s interested in hiring someone gets in a line and gets to select whoever they want when it’s their turn. Companies don’t even have to compete with one another. They pick someone out and that person automatically joins their team.
What this looks like is something akin to a professional sports draft. And here’s the kicker: even with all this information and tens of millions of dollars in payroll dedicated to assessing talent and potential, teams still get it wrong. If these folks are getting it wrong, how optimistic should we be that we are getting it right? Given that interviewing is only a fraction of our job responsibility, and we’re only spending four to five hours with a candidate before we have to make a decision, the answer is we’re probably not getting it right.
The second thing I’ve come to understand about humans is that individual performance is not as individual as we like to think.
Back in the 1960s, psychologist Robert Rosenthal and school principal Lenore Jacobson conducted a study on elementary school students in South San Francisco. The study involved the administration of a disguised IQ test at the beginning of the school year. The individual results of the test were not disclosed, but teachers were informed of the names of students who could be considered “intellectual bloomers” for that year and should be expected to perform better than their peers. At the end of the study, the IQ test was re-administered and the “intellectual bloomers” showed a statistically significant increase compared to the non-bloomers despite being randomly selected. This is known as the Pygmalion effect. It turns out that the opposite is also true. When people believe you are an underperformer, you are more likely to underperform. What this means experientially is that our performance is linked to whether our manager believes in us. It turns out these effects are pretty far-reaching and have presented everywhere from military training to mice running mazes.
Two other related studies that bear consideration focus on firm-specific and team-specific performance. The gist here is that individual performance is dependent on the knowledge of the team and company employing an individual. Large changes to those two things can result in degraded performance for significant periods of time.
The first study involves tracking the outcomes for cardiac surgeons as they perform surgical procedures at various hospitals. Basically, it’s a study of knowledge portability. Can a highly skilled knowledge worker be equally successful doing the same job in different contexts? Answer: Not really. From the abstract:
. . . the quality of a surgeon’s performance at a given hospital improves significantly with increases in his or her recent procedure volume at that hospital but does not significantly improve with increases in his or her volume at other hospitals
The second study examines high performers in the securities analyst field and what happens when they switch firms.
Star analysts who switched employers experienced an immediate decline in performance that persisted for at least five years. This decline was most pronounced among star analysts who moved to firms with lesser capabilities and those who moved solo, without other team members. Star analysts who moved between two firms with equivalent capabilities also exhibited a drop in performance, but only for two years. Those who switched to firms with better capabilities and those who moved with other team members exhibited no significant decline in short-term or long-term performance.
So what do we do with all this information? I think two of the immediate takeaways are that it takes more than just talent to succeed, and that performance is a team endeavor. Applying this to interviewing, I think it gives us insight into what the purpose of an interview should be: to build a confidence case for success in a team context.
It is likely that your interview process over indexes on assessing individual performance. The most common version of this is the interview as a test. If you view your interview as a test, you are placing the responsibility of success on the candidate—enforcing an ingroup vs outgroup dynamic and predisposing yourself to finding all the reasons that someone won’t be successful on your team both now and in the future. Put simply, you are preparing for this person to fail.
An alternative is to think of your interview as an exercise in understanding how a candidate can be successful at your company or on your team. What would it take for that specific candidate, and are you capable of providing that support? If you can figure that out and come back with a yes, then you already have a leg up on creating conditions for success—you have conditioned your team and yourself to be on the hook and invested in this candidate’s future. And, if the answer is that you can’t support this candidate, then it’s a recognition that both the team and the candidate fell short together. You can then do the work of reflecting on whether you want to be able to support this type of candidate in the future and take actions accordingly.
Another way to leverage this information is to tailor your interview process to your organization’s values. Interview in a way that is aligned with how you evaluate performance at your company. Play to the rubric. If you have ten points of performance in your annual review process, find ways to explore as much of that as you can with the candidate during the interview. Know why you’re asking the questions you’re asking. If you value communication, find ways to explore the candidate’s communication ability. If you value collaboration, take time to discover how they collaborate. If you value inclusiveness, look for a meaningful history of demonstrated inclusivity. You would be amazed at how many interview processes are a result of survivor bias. “These were the questions people asked me in prior interviews where I got the job, so they must be good questions!” Don’t get caught in that trap. Interview with intention. If you are lucky enough to be at a company experiencing rapid growth, this could be the difference between waking up one day with a bunch of engineers who can’t work together, or an actual high-functioning group of people.
Now that I’m looking to join a new team, it’s been interesting trying to apply these concepts from the opposite end. I know that having a perspective on the purpose of interviews really changed the way I went about making hiring decisions. It gave me confidence and minimized the fears and anxieties I used to experience when bringing someone new on board. Did we still make hires that didn’t work out? Yes. That’s unavoidable. But having an opinion on the purpose interviewing plays in team building and focusing on positives helped us put some stakes in the ground around which we could build toward better outcomes.
P.S. There’s a lot more to talk about when it comes to interviewing, specifically the importance of operationalizing the process. I’d suggest this great conversation with Marco Rogers and these tips from Meetup. Also, the research I cite here is from the book Give and Take: Why Helping Others Drives our Success, by Adam Grant.