AI Engineer interview loops

An AI engineer is a software engineer who builds products and tools on foundation models, called through a provider's API or run as open models, rather than training them. Latent Space's essay The Rise of the AI Engineer (June 2023) put training and fine-tuning on the machine learning engineer's side of the line, and the product's own data, its evaluations and the shipping on the AI engineer's.

As of October 2026 the title covers three different jobs, and a job ad should say which. On a product team it means model-backed features that customers use: a Brex posting has its AI engineer building the workbench where finance teams work with its agents, and owning those features through production (Brex). On an internal team it can mean applying models to the company's own work: GitLab's Senior AI Engineer posting has them diagnose a business problem first and be ready to say it does not need AI (GitLab), and a Stripe posting embeds its AI engineer with the solutions architects, building their tools (Stripe). At a model company an applied AI engineer can work with customers: Anthropic's postings have them helping enterprises deploy its models into their operations and products (Anthropic), which is closer to a forward deployed engineer. The loop below is written for the first job, an engineer who owns a model-backed feature in front of customers.

Nobody should be hired into the role for knowing which models exist. The work is a feature that is wrong some of the time by construction, and the job is to know how often, to make the failure cheap, and to keep the rest of the product standing when the model is unavailable or has been talked into something. That is ordinary engineering judgment applied to a component nobody can make deterministic, and it is what the rounds below are written to read.

Two things make this role hard to interview for. The first is that almost every question about it has a fluent answer available from a blog post, so a round built on vocabulary measures reading rather than experience. The second is that the failures worth hiring against are quiet: a cache that serves a confident wrong answer, an eval suite that has been green for a month while support escalations climb, a tool that acts on instructions written by a stranger. None of those announce themselves, so each round here hands the candidate a real artifact — a pack of traces, a pack of prompt pairs, a repository, a ticket — and watches what they notice in it.

Employers disagree about whether a candidate may use AI in the interview. Anthropic asks candidates to do take-home assessments and live interviews without it unless told otherwise (Anthropic, updated July 2025). Canva wrote in June 2025 that it expects backend, frontend and machine learning candidates to use AI coding tools in its technical interviews, and that it assesses how they direct them, break down an unclear problem and catch flaws in what the tools produce (Canva). Either is defensible; decide before the first round and tell the candidate which it is. None of the plans below says either way. The coding round lets the candidate look anything up, and the other three mostly credit what a candidate names unprompted, which an assistant could name for them, so allowing one there changes what those rounds measure.

A loop is a set of round plans in order. Each round scores one to three competencies, every must-have is scored in at least two rounds, and the debrief is held competency by competency rather than round by round, so no single interview decides the hire. How these plans work covers the rest of the method.

Further reading, as of October 2026. Chip Huyen's AI Engineering (O'Reilly, December 2024) is a book-length treatment of the job, with two of its chapters on evaluation. Hamel Husain's Your AI Product Needs Evals (March 2024) makes the case the evals round tests. The OWASP Top 10 for LLM applications (2025 edition) opens with prompt injection, the subject of the untrusted-input round.