Structured interviewing
Hiring on vibes: what the research says about gut feel in interviews
The Itya team · Updated · 5 min read
The short answer
Gut feel is the least reliable part of most interviews. In the largest re-analysis of hiring methods, structured interviews predicted job performance about twice as well as unstructured ones (.42 against .19). The fix is not to remove judgment. It is to write down what you are judging before the first candidate walks in.
What "hiring on vibes" means
Every recruiter has read this feedback: "Great energy. Strong culture fit. Lean hire." It arrives three days after the interview, from memory, and it says almost nothing about the job. That is hiring on vibes: a decision formed from an impression and justified afterwards.
Researchers call it the unstructured interview. Its markers are easy to spot:
- Each interviewer picks their own questions, so no two candidates answer the same thing.
- Nobody wrote down, before the interview, what a strong answer looks like.
- Scores are given after the panel has talked, so the loudest opinion sets the anchor.
- Feedback describes the person ("confident", "sharp", "not a fit") instead of the evidence ("described how she cut the release time from two weeks to three days").
What the research says
For decades the standard reference was Schmidt and Hunter (1998), a meta-analysis of 85 years of selection research. In 2022, Sackett and colleagues re-ran the numbers with a correction that earlier work had over-applied. Most estimates fell. The ranking at the top changed: structured interviews came first.
| Method | Sackett et al. (2022) | Schmidt and Hunter (1998) |
|---|---|---|
| Structured interview | .42 | .51 |
| Unstructured interview | .19 | .38 |
Read the left column as the current best estimate. A structured interview carries roughly twice the predictive signal of an unstructured one. The Society for Industrial and Organizational Psychology summarises the shift for practitioners.
Why vibes feel accurate anyway
If unstructured interviews predict so poorly, why do experienced interviewers trust them? Because confidence and accuracy are different things, and an interview produces a great deal of the first.
- A story forms early. Once an interviewer likes or dislikes a candidate, later answers are heard through that story.
- Familiar feels good. A candidate who went to the same college or talks the same way is easy to like, and liking is not evidence.
- Memory edits. Feedback written days later records how the interview felt, not what was said.
- Nobody sees the misses. A team never learns how the rejected candidates would have done, so its gut is never corrected.
What vibes cost
Weak signal does not stay inside the interview room. When a panel cannot agree on what it saw, it adds another round. In Gem's 2025 recruiting benchmarks, interviews per hire rose from 14 to 20 between 2021 and 2024, and time to hire grew from 33 to 41 days. Structure is not the only cause of that drift, but extra rounds are what a team buys when it does not trust its own evidence.
Candidates pay too. In a Greenhouse survey, 61% of job seekers said they had been ghosted after an interview. Late, vague feedback is the same habit seen from the other side of the table.
How to replace vibes with evidence, in five steps
- Write four to six competencies before the job is posted. Each one should be observable: "breaks an ambiguous problem into steps", not "smart".
- Ask every candidate the same core questions, in the same order. Follow-ups can vary; the questions you score cannot.
- Anchor the scale in words. Describe what a 1, a 3 and a 5 sound like for each competency, so two interviewers mean the same thing by "4".
- Score each answer straight after the interview, alone. Submit the scorecard before the debrief, so no one anchors on the most senior voice.
- Debrief on quotes, not adjectives. For every rating, ask: what did the candidate say that earned it? If nobody can quote it, the rating is a vibe.
Ready-made competencies, questions and anchored scales for common roles are in our interview kits. The scorecard guide covers how to make interviewers actually fill one in.
Where AI fits, and where it does not
Software can do the tedious half of structure well: record and transcribe the conversation, check which competencies were covered, and draft a scorecard that quotes the moments behind each rating. That removes the memory problem and the three-day delay.
It should not make the decision. Candidates agree: Pew Research found 71% of Americans oppose AI making the final hiring decision. And in a Greenhouse survey from May 2026, 70% of job seekers who had faced an AI interview were not told upfront that AI would evaluate them. Tell candidates when AI is involved, offer a person, and keep a person accountable for the decision.
The AI listens. People decide.
A test for your next debrief
Take the written feedback from your last five interviews. Count the adjectives about the person, then count the quotes from the candidate. If the adjectives win, your team is hiring on vibes. That is fixable in a week, with a rubric and the discipline to score before you talk.
Questions people ask
- Is gut feel ever useful in an interview?
- It is useful as a prompt, not a verdict. If something feels off, turn the feeling into a question and probe it, then score the answer against the rubric. The research problem is not intuition itself; it is deciding on intuition alone.
- Does a structured interview feel robotic to candidates?
- It need not. Structure fixes the questions you score and the scale you use; tone, warmth and follow-ups stay human. Candidates often find it fairer, because everyone is asked about the job.
- What does a validity of .42 mean?
- It is a correlation between interview scores and later job performance. No hiring method comes close to 1, so the useful comparison is relative: .42 for structured interviews against .19 for unstructured ones, in the 2022 re-analysis.
- Can AI remove bias from interviews?
- No honest vendor can promise that. What structure and AI can do is make every rating inspectable: a written rubric, the quoted evidence behind each score, and a named person who made the call.
Sources
Every source was opened and checked on 10 October 2026.
- Revisiting meta-analytic estimates of validity in personnel selection (Sackett, Zhang, Berry and Lievens, 2022), Journal of Applied Psychology
- Is cognitive ability the best predictor of job performance?, Society for Industrial and Organizational Psychology
- The validity and utility of selection methods in personnel psychology (Schmidt and Hunter, 1998), Psychological Bulletin
- 10 takeaways from the 2025 recruiting benchmarks report, Gem
- Greenhouse 2024 State of Job Hunting report, Greenhouse
- AI in hiring and evaluating workers: what Americans think, Pew Research Center
- 63% of job seekers have faced an AI interview. Most haven't had a good one yet, Greenhouse
Run every interview on one rubric.
Itya turns a job into competencies, questions and a scored rubric, then holds every interviewer to it. Free plan, unlimited teammates.