AI Engineer interview questionsSystem design — untrusted input and tool permissions round

Purpose

Establishes three text sources with different trust properties without characterising any of them, which is the thing the candidate is being asked to notice.

Purpose

Shows the artifact rather than describing the category, so the candidate reacts to text rather than to a term they may have prepared an answer about.

Purpose

The first half is a comprehension check and most candidates pass it. The second half is where they separate, entirely on whether their fix is an instruction or a boundary.

Signals to score

• Says the model has no way to distinguish its instructions from the text it was given, because both arrive as tokens in one context
• Names the knowledge base as a second untrusted source, not just the ticket, and asks who can write to it
• Marks or fences the untrusted spans, and says explicitly that this reduces the rate and is not a boundary
• Does not claim a system prompt, a delimiter or a "never follow instructions in the ticket" line solves it
• Moves the defence to what the system is able to do rather than to what the model is asked to do
• Asks what the assistant can currently reach, before proposing anything
• Distinguishes the model being fooled from the system being harmed, and says only the second is worth engineering against
• Notices the "summarise the last five tickets from this email address" request is asking for other people's data
• Proposes a detector or a classifier as defence in depth while saying what its false-negative rate costs
• Would test this deliberately rather than waiting to see it in production

Follow-up questions

• You have added a line to the system prompt telling it to ignore instructions in the ticket. Is that done?
• Who can write into the knowledge base? Does that change your answer?
• What is the worst outcome here, given only what this assistant can do today?
• If the model does get fooled, what did we lose?
• How would you find out whether this is already happening?

Purpose

Presents the refund as already shipped and unremarkable, so objecting to it is a choice the candidate has to make rather than a prompt they have been given.

Purpose

Reads whether authorization is expressed in the tool or expected of the model. The order lookup is the discriminator: almost everyone objects to the refund, and far fewer notice that a customer id parameter is the actual hole.

Signals to score

• Scopes the order lookup to the session's own customer rather than to an identifier the model supplies
• States the general rule: a tool acts with the caller's authority, never with the model's
• Recognises that validating the id against a list the model can also see does not help
• Objects to the unattended refund, and objects on the grounds of reversibility rather than of amount
• Treats approval as a property of the tool's declaration rather than as something the interface decides
• Separates reads from writes, and outward-facing writes from internal ones
• Wants a record of what the assistant did and on whose behalf, distinct from the application's own logs
• Asks what the fifty dollar limit is per — a call, a ticket, a customer, a day — and notices it is per call
• Says the knowledge base search is fine and says why, rather than treating every tool as equally risky
• Would bound the blast radius of the tools in aggregate, not only each one on its own

Follow-up questions

• Where does that customer id come from?
• Suppose the model passes a valid id belonging to someone else. What in your design refuses it?
• Fifty dollars per what?
• Which of these three would you gate on a human, and is that a property of the tool or of the screen?
• A month from now someone adds a fourth tool. What stops them getting this wrong?

Purpose

Makes the incident the consequence of the system they have just criticised, so the reasoning is about consequence rather than about design.

Purpose

Removes every available deflection. What the candidate does in the next sentence is the read.

Purpose

Deliberately not a technical question. Order of operations, who they tell, what they say about cause, and whether the account survives contact with the fact that it was their code.

Signals to score

• Stops the bleeding before investigating, and says what stopping it means concretely
• Establishes the total — how many, how much, which accounts — before communicating anything
• Tells someone early, and names who rather than saying "stakeholders"
• Owns it in plain language without performing contrition or hunting for a share of the blame
• Refuses the explanation that the model misbehaved, and locates the cause in a permission the system granted
• Separates what is recoverable from what is not, and knows a sent refund is mostly the second
• Checks whether the same ticket pattern hit anything else, rather than treating the refunds as the whole event
• Distinguishes the immediate fix from the permanent one and does not ship the permanent one at nine in the morning
• Writes it up in a way that changes how the next tool is reviewed, not only how this one works
• Says what would have caught it — and is honest about whether anything in the system was watching

Follow-up questions

• What is the first thing you actually do, before anyone is told?
• Who do you tell, by name or by role, and what do you say in the first sentence?
• Someone in the room says the model was jailbroken. Do you agree with that framing?
• How much of this comes back?
• What goes in the write-up that would stop the next person building the same thing?

Purpose

Reframes the model's output as an input to three other systems, which is the shift the section is testing.

Purpose

Most defensive thinking stops at what goes into the model. This reads whether the candidate carries the same suspicion outward, and whether they see that a chain of assistants is a trust boundary nobody drew.

Signals to score

• Treats the model's output as untrusted input to whatever consumes it, in those terms or their own
• Names rendering as the risk in the console, not just the wording of the text
• Identifies that a rendered image or link can carry conversation content to somewhere else in its URL
• Restricts what the markdown may contain rather than trying to sanitise arbitrary output
• Sees that the second assistant reads the first one's output as if it were trustworthy, and that an injection therefore survives the hop
• Says the chain does not multiply authority — the second assistant should hold no capability the first was denied
• Notices that storing the answer makes one bad answer reusable, and connects it to anything that serves stored answers
• Asks what else reads that store, rather than assuming the two assistants are the only consumers
• Proposes a human check at the point the text leaves the company, and says why that boundary and not another
• Would not let the model's output become a parameter to anything without validating it as a parameter

Follow-up questions

• The console renders markdown. What can markdown do that plain text cannot?
• What does the second assistant believe about the text it was handed?
• Does the second assistant need anything the first one was not allowed?
• One manipulated answer goes into that store. How long does it keep answering?
• Where is the last point in this chain where a person could still stop it?