Most companies treat interview skill as something engineers bring with them and candidates figure out on their own. In practice it is a skill like any other: it fades without use, and it is unevenly distributed across a team. This article looks at where technical interview preparation breaks down inside organizations, what AI interview practice is genuinely good for, where it is not, and how to run a small pilot that tells you whether it helps.

It is written by the team behind Mentara, so we describe our own product in places. We have tried to keep those parts factual and to be clear about the limits.

The problem: interview skill decays, and good mocks are expensive

An engineer who last interviewed four years ago has not forgotten how to build software. They have forgotten how to explain it under a clock, to a stranger, while being interrupted with “why not the other approach?”. That gap matters in two places:

Internal mobility and promotion. Many companies run structured technical conversations for promotions, team changes or tech-lead roles. People who have not practiced explaining their work underperform relative to what they actually know.
Hiring. Interviewers are engineers too. The consistency of a hiring loop depends on people who rarely practice interviewing, and it drifts from one interviewer to the next.

The traditional fix is a mock interview with a senior engineer. It works, and it does not scale. An hour of a staff engineer’s time per mock, per person, per topic adds up quickly, and the quality varies with whoever is available that week.

Why the usual approaches don’t scale

Question banks. Reading a list of common questions produces recognition, not recall. People feel prepared because the questions look familiar, then struggle to produce a structured answer out loud. Nobody asks a follow-up on a vague answer, so the vagueness never shows.

Peer mocks. Valuable, and the best way to get human judgment on how someone comes across. But they are hard to schedule, easy to turn into a friendly chat, and almost nobody will run five mocks on the same narrow topic so a colleague can fix one recurring mistake. We compared the two formats in more detail in peer mock interviews vs. AI mock interviews.

Take-home exercises. Useful for hiring, less so for practice. They take hours, candidates increasingly resent them, and the feedback, if any, arrives days later.

A general-purpose chatbot. An engineer can ask a chatbot for interview questions. They will get questions. What they will not get is a fixed structure, a time limit, a follow-up that builds on their own answer, code that is actually executed, or a record of what went wrong last time. The engineer ends up running the interview and grading it at the same time.

What AI interview practice is good for – and where it isn’t

A dedicated AI interview practice tool earns its place when it does the parts that are tedious or expensive for humans:

Availability and repetition. Practice at 7 a.m. or between meetings, and repeat the same topic until it is solid, without asking anyone for their time.
Structure. The interview moves through stages on a clock. In Mentara that is warm-up, technical questions, a deep dive and a wrap-up, for a chosen role (frontend, backend or algorithms), level (junior, middle or senior) and up to six technologies.
Follow-ups on the actual answer. The useful pressure in an interview is the second question, not the first. A good tool asks it, narrows the question when the candidate stalls and pushes further when the answer is strong.
Feedback with evidence. A report should tie each score to something the person said, mark the skills it could not assess instead of guessing, and turn the gaps into a plan.

It is not good for everything, and it is worth being explicit:

It is not a person. It does not judge culture fit, motivation or how someone works in a team. Behavioral judgment still belongs to people.
It does not make hiring decisions. At most, it gives a human a consistent, comparable first signal. The decision, and the accountability for it, stays with your team.
It is only as good as its rubric. An AI score without evidence behind it is noise. Ask any vendor how scores are produced, and whether you can see why a score is what it is.

Conversation and code are different skills – assess them separately

Explaining a design and writing working code under time pressure are related but distinct. When both happen in one chat, it is hard to tell which one needs work, and a model that “reads” code is guessing at whether it runs.

That is why Mentara keeps them apart. The interview is a spoken or typed technical conversation. The coding assessment is a separate, timed exercise: one to five problems, an editor, runnable examples, a choice of JavaScript, TypeScript, Python, Java, Go or C++, and grading against hidden tests that never leave the server. Only after grading does an AI review comment on code quality, efficiency and readability, per problem. The overall result combines the test score with that review, so “it looks right” never counts as right.

For a team, this separation is what makes results comparable: two candidates who took the same assessment were graded by the same tests.

Consistency: company interviews and evidence-based scoring

The main reason organizations adopt structured interview tooling is consistency. In Mentara this takes the form of company interviews: a template that fixes the role, level, stack, length and language, plus up to 30 seed questions, each with the concepts your team expects a good answer to cover.

Every person who takes that interview gets the same structure and the same starting questions. Follow-ups still adapt to what each person says, which is the point of an interview rather than a quiz.

Scoring works from evidence. During the interview, answers are labelled against the rubric, and the scores in the report are computed from that evidence rather than taken from a number the model produces. Skills the interview did not reach are reported as “not assessed”, not given an invented score.

Team development without surveillance

The fastest way to kill a practice program is to make people feel watched. Engineers practice honestly when their mistakes stay private; they stop practicing, or start gaming it, when a manager can read every stumble.

So the limits have to be in the product, not in a policy document. In a Mentara organization:

For each member, the organization sees only the join date, the number of interviews in the last 30 days, and the last activity.
Team skill averages appear only when at least five people contributed, so no average can be traced back to one person.
No role in the organization – owner, admin, recruiter or member – can read an employee’s transcript or individual report.

This is a deliberate trade-off. You get enough signal to know whether the program is used and where the team as a whole is weaker, and not enough to evaluate individuals with it. If your goal is individual performance evaluation, a practice tool is the wrong instrument.

Screening candidates fairly

Used for hiring, the same structure becomes a consistent first-round interview. What matters here is consent and a clear line on what the company sees:

A candidate assignment is a single-use link that opens only for the invited email address, and expires.
Before starting, the candidate sees a disclosure of what will be shared and consents.
The recruiter sees the report for an interview, or the test results and scores for a coding assessment – not the transcript and not the submitted code.
The assignment does not consume the candidate’s own practice allowance.

Treat the result as one structured input to a human conversation, not as a pass/fail gate.

A security and privacy checklist for any vendor

Whatever tool you evaluate, these questions separate a careful product from a demo:

Who can see what? Ask for the exact list per role, and whether it is enforced on the server or only hidden in the interface.
How is tenant isolation done? Can a user of one organization ever address another organization’s data by changing an ID?
Where do model API keys live? They should stay on the server, never in a browser or app.
What is the data used for? Reports and progress, or also sold or used for advertising?
How are scores produced? From evidence you can inspect, or from a single model number?
What happens when the AI fails? Is a failed analysis flagged, or silently replaced with a plausible-looking score?

Mentara’s answers: role permissions and privacy thresholds are enforced by the server on every request; the organization is derived from the signed-in user’s membership, never from an ID in the URL; model keys stay server-side; session data powers reports and progress and is not sold; and a report whose AI analysis failed is marked as such, with the evidence-based scores kept.

What is not available yet, so you can plan around it: SSO and SCIM provisioning, ATS integrations, custom branding, company-authored coding problems and self-serve checkout.

Getting started: a two-week pilot

A pilot should answer one question: does structured practice change anything for your team? A simple plan:

Pick one use case. Upskilling or hiring, not both at once.
Start small. A trial organization in Mentara is 14 days with 5 seats and 10 candidate assignments – enough for one team or one open role.
Build one company interview for the role and stack that matters most, with a handful of seed questions your senior engineers actually ask.
Measure what you can see. For upskilling: how many interviews people chose to do, and how the team averages moved. For hiring: whether candidates completed the assignment, and whether your interviewers found the reports useful in the next conversation.
Ask the participants. Whether it felt useful is the leading indicator of whether they will keep using it.

If you want to try this with your team, tell us about it. For individual engineers, the home page explains how a single interview works, and inside an AI interview simulator goes one level deeper into the architecture.

AI interview practice for engineering teams: what it can and can’t do