Assessing for judgment over skill

At the Director-and-up level, skill just gets you onto the shortlist. Everyone there can do the technical work; the CV proved it and the first screen confirmed it. What actually separates the hire who compounds from the one who stalls is judgment: the ability to decide well under ambiguity, to know which battles are worth fighting, to read an organisation and work inside its grain rather than against it. A competency test won't tell you any of that.

This is why so many senior hires look strong on paper and disappoint in seat. The loop screened for demonstrable ability and never interrogated decision quality. The candidate could describe what good looks like; nobody checked whether they could produce it when the answer wasn't obvious and the cost of being wrong was real. The mismatch rarely shows in the first month. It shows in the second quarter, when the problems come to resemble the ones nobody wrote down.

Judgment tends to surface at the edges of a story. Ask a candidate about a decision that went wrong and listen for whether they can put their finger on their own error, or whether the failure always turns out to be someone else's. Ask what they'd do differently, and watch whether the answer shows they understood the knock-on consequences and not just the obvious first miss. What they decided matters less than how they got there. Good judgment can produce a bad result, and luck can bail out a bad decision, so scoring the outcome alone tells you fairly little about the person who made the call.

The trap is the polished narrative. Senior candidates are fluent, and they can make almost any decision sound inevitable in hindsight. So the interviewer has to push past that fluency to the actual trade-off: what did they give up, what did they get wrong first, who did they have to bring round, what were they still unsure of when they had to move. That's usually where you find out whether you're hiring judgment or a very good account of it.

Part of the appeal of testing skill is that it's legible. A skill has a right answer, a rubric, a pass mark. It produces a clean score a panel can agree on and a committee can sign off. Judgment offers none of that comfort. It resists the rubric, which is one reason most loops end up defaulting to the thing they can grade and calling it rigour. But a signal you can read cleanly isn't automatically a signal that predicts anything. Skill stands in for performance, and the further up you hire, the more that stand-in drifts, because the job stops looking much like the test.

Consider what the seat actually demands. A Director spends little of the week doing the craft that got them promoted and most of it deciding: what to prioritise when everything is urgent, which risk to carry and which to escalate, when to overrule a smart subordinate and when to back off and let them run. None of that shows up on the skills matrix. A loop built around skill is measuring the part of the role that shrinks with seniority and largely ignoring the part that grows. The strongest technical candidate in the room can still be the wrong hire, and a process that only sees skill won't catch it.

If judgment is what you're after, the standard interview is a poor instrument for finding it, since polished candidates are more or less built to win conversations. The methods worth keeping in a Director-and-up loop are the ones that get past presentation. Three tend to hold up in senior hiring, and each reads something the others miss.

A case study or working session shows how someone thinks when the answer isn't rehearsed: how they frame an ambiguous problem, what they ask before they start solving, where they're willing to say “I don't know yet.” It measures live reasoning, which a CV simply can't carry. A back-channel reference gets at something the candidate can't manage at all: how they actually operated, from people who worked alongside them rather than the referees they picked. A discreet, well-placed reference conversation will often surface the pattern the interviews smoothed over. The two together read different things, one the mind in the room, the other the track record outside it, and neither substitutes for the other. Then there's the real-work session, where a candidate spends time on a live problem with the people they'd actually work with. That reads fit under friction: how they behave when the room pushes back.

The mistake is running every method on every candidate as ritual, a 5-stage loop that treats thoroughness as the goal and exhausts strong candidates before it learns anything. Assessment should be pointed at the specific thing you're unsure about, and the doubt is different for every hire. If the risk is whether they can operate at a genuine step up in scope, a working session on a real problem tells you more than any number of conversations, because the past a back-channel reads is a smaller job than the one you're offering. If the risk is how they lead under pressure, the failure mode that breaks senior hires most often, the back-channel is the only method that reliably reaches it, because pressure behaviour is exactly what a candidate manages away in an interview.

Chosen this way, a Director-and-up loop tightens: fewer stages, each doing more work. Each method earns a distinct verdict on a distinct doubt, and a method that isn't answering a live question doesn't run. What survives contact with senior hiring is the loop where every assessment is aimed at a question you genuinely can't answer any other way. The design question is “what am I actually afraid of with this person, and which instrument reads it?”

Put together, this resolves into a way of building the loop rather than a longer checklist. Start from the CV as settled evidence of skill: the technical bar is a gate you pass once. Then name the 2 or 3 doubts that would actually make this hire fail: the step up in scope, the leadership under load, the fit with a specific room. Assign one method to each doubt and stop there. A loop with 3 sharp instruments beats a loop with 6 dull ones, and the candidates you most want, the ones with options, will notice the difference between a process that respects their time and one that pads it.

The discipline is to keep asking what each stage is for. If a round can't name the doubt it's resolving, it's ritual, and ritual is how good candidates get lost and weak ones get waved through on polish. Skill is easy to test and comfortable to score, which is exactly why loops over-index on it. Judgment is harder to read and it's the thing that decides whether the hire holds. Assess for it on purpose, with the case, the back-channel and the session, each pointed at a real question, and you hire judgment itself, where a loop built around skill hires the best account of it.