A good technical interview process measures 4 things.
Can they decompose an unfamiliar problem. Can they explain a decision to someone who disagrees.
Do they notice what they do not know. Do they finish.
Unstructured interviews explain about 4% of on-the-job performance.
Structured ones explain 26%, from the same hours and the same people.
Most technical interviews measure how well someone performs under artificial pressure.
That is a different skill from doing the job.
I have run over 100 technical interviews and hired more than 10 engineers.
I have also watched strong engineers fail loops they should have passed. The pattern is consistent, and it is fixable.
Why does structure matter this much?
Because unstructured interviews explain about 4% of on-the-job performance and structured ones explain 26%.
Six times more of the outcome, explained. Same time, same people.
An unstructured interview is mostly noise. The interviewer’s mood, the candidate’s nerves, whether the 2 people click.
That gap is the whole argument.
What do most loops actually test?
Four things nobody intended. Recall speed, which is a study-hours proxy.
Stress tolerance, which excellent engineers can lack without it affecting production incidents.
Cultural similarity, because interviewers rate people who sound like them higher.
And interviewer preference. One asks trivia, one asks about vibes, one cares about code style.
Recall speed. Whether the candidate has seen this exact problem before.
Stress tolerance. Some excellent engineers freeze in front of a stranger with a whiteboard.
Your production incidents are stressful, and they are not that kind of stressful.
Cultural similarity. Interviewers rate people who sound like them higher. This is well documented and it feels like judgement.
Interviewer preference. Nobody agreed what the bar is.
What is worth measuring?
Four signals predict performance, and all 4 are visible inside an hour.
Decomposition, defending a decision, noticing your own gaps, and converging.
Can they decompose a problem they have not seen? Give them something unfamiliar and watch the first 5 minutes. Do they clarify, or start typing?
Can they explain a decision to someone who disagrees? Most engineering time goes into convincing other engineers. An undefended choice will not survive the team.
Do they notice what they do not know? The senior signal is one sentence, said before anything breaks. “I would need to check how this behaves under load.”
Can they finish? Plenty of candidates design beautifully and never converge. Shipping is a skill, and it is testable.
Which 3 changes fix most of it?
Write the scorecard before the question. Know what a strong answer looks like before anyone interviews.
Ground the question in the actual job, using something the team shipped last quarter.
And require scorecards before the debrief. Opinions contaminate each other within seconds.
Write the scorecard before you write the question
Write down the evidence you expect at each level.
If you cannot describe a strong answer in advance, the question is not ready.
Ground the question in the actual job
Do not ask for a red-black tree if nobody has touched one in 3 years.
Take something the team shipped last quarter. Strip it to 45 minutes. Now you are testing the work.
Submit scorecards before the debrief
Every interviewer writes their score before anyone speaks. No exceptions.
The loudest person in the debrief sets the anchor, and everyone else drifts toward it.
What changed when I did this
At Zalando I defined the hiring process for a team that now spans 27 markets.
The 2 changes that mattered most were the pre-written scorecard and the silent debrief.
Disagreements stopped being about taste and started being about evidence.
The other effect was quieter. Interviewers got faster, because they knew what they were looking for before the call started.
Run the loop rather than sit in it? The hiring side is in why your loop rejects good engineers.
What does this mean if you are the candidate?
You cannot fix someone else’s broken loop, and you can make yourself easier to evaluate.
Say what you are doing while you do it, because silence gets scored as confusion. Ask clarifying questions before you write code.
State your assumptions out loud. And say when you do not know something, then say how you would find out.
Clarifying questions before you write code are a senior signal. Most candidates skip them.
State an assumption and it turns out wrong? You look thoughtful. Leave it unsaid and you look careless.
Common questions
Is a take-home assignment better than a live interview?
Better signal, worse conversion. A take-home tests the actual work and removes whiteboard stress. It also costs candidates unpaid hours. Your strongest applicants drop out first. Cap it at 3 hours. Say so honestly. Never combine it with a full onsite.
Should interviewers see the candidate’s CV beforehand?
Enough to know the level and the domain, not enough to form a verdict. An interviewer who reads a full CV arrives with a hypothesis. They spend the hour confirming it. Give them the role, the level, and the 2 scorecard lines they own. Nothing else helps.
How do you test whether someone can finish?
Give a problem slightly too large for the time. Score the convergence. Watch whether they cut scope out loud, name what they drop, and land something working. Candidates who design beautifully and never converge are common. They behave the same way on a deadline.
Does interviewing get easier with a structured process?
Yes, and faster. An interviewer who knows their 2 lines stops assessing the whole person in 45 minutes. At Zalando that was the quieter effect. Debriefs shortened. New interviewers were useful in their second loop rather than their sixth. The bar stopped moving between them.
The short version
Structure is worth 6 times the signal of a conversation.
Write the scorecard first. Ask about the actual job. Score before you talk.
Everything else is detail.