DevOps Engineer interview questionsPipeline review — a 38-minute build nobody trusts round

Purpose

Frames the pack as an inheritance rather than a quiz, so the conversation is about what to change first and what it costs, not about which features the candidate has heard of.

Purpose

Separates reading the numbers from reaching for a faster machine. The cache key and the Dockerfile order are in the files, the long pole is in the data, and the target can be checked with arithmetic; the read is whether the candidate uses all three.

Signals to score

• Starts from the per-step medians and names install and image builds, 16.5 of the 38 minutes, as work a working cache should mostly remove
• Explains the 4%: a key containing the commit SHA is new on every commit, so only a re-run of the same commit hits, and with no fallback a miss starts from nothing
• Keys the cache on the lockfile's hash with a fallback, and switches to an install that fails rather than rewrites the lockfile
• Moves `COPY . .` below the dependency install in the Dockerfiles, and knows a fresh runner has no layer cache unless the build exports and imports one
• Splits lint, unit tests and image builds into parallel jobs, and says what that adds in runner minutes
• Uses the 61% figure to build and test only the services a change touches, with `packages/shared`, the lockfile and root files triggering everything
• Keeps a full run on `main` or in a merge queue to catch what the affected-only rule misses
• Names integration tests as the long pole no cache touches, and proposes splitting or sharding them with the cost
• Leaves alone what already works or barely matters — the concurrency setting and the 0.7-minute Postgres wait
• Adds up the plan against 15 minutes instead of asserting that it gets there

Follow-up questions

• Why does the cache hit 4% of the time rather than never?
• You fixed the cache key. Why are image builds still slow on a fresh runner?
• A change touches only `web`. What could it break that your rule would not test?
• Which of your changes makes the CI bill bigger?
• Add it up. Does a typical run now finish under 15 minutes?

Purpose

The honest answer starts with what a red run costs the person waiting on it. Many candidates go straight to flaky-test tooling or to removing the bypass; the read is whether they see the pipeline from the side of the engineer who clicked merge.

Signals to score

• Starts from what a red run costs its author — another 38 minutes, maybe 51 — and reads the bypass as a rational response to that
• Notices that 51 of the 66 retried runs are one test, so this is mostly one problem rather than general flakiness
• Asks whether `applies a discount code once` fails because of the test or because checkout sometimes applies a discount twice
• Says a retry that turns a failure into a pass hides exactly the bugs that only happen sometimes
• Quarantines that test with a named owner and a date, runs it outside the required check, and keeps its results visible
• Tracks a failure rate per test from run history, so the next flaky test is found by data rather than by complaint
• Narrows or removes the retry only after quarantine exists, and says why the order matters
• Treats the administrator bypass as evidence about the pipeline before treating it as a permissions problem
• Explains the change to the team where they merge, including what to do when a quarantined test fails
• Lets an engineer re-run only the job that failed rather than the whole pipeline

Follow-up questions

• Why would a careful engineer merge past a red check?
• 51 of the 66 retries are the same test. What do you want to know about it first?
• If you delete the retry script tomorrow morning, what happens by lunchtime?
• Who owns a quarantined test, and what happens if they never fix it?
• In a month, how will you know people trust a red run again?

Purpose

Tests build reproducibility as a mechanism. A candidate who has been caught by it names the differences — a build argument, a moving base image, dependencies resolved again, a night between the builds — before being asked, and knows that identical inputs do not give an identical image by default.

Signals to score

• Names the `APP_ENV` build argument, which makes the production image a different build by construction
• Names `FROM node:22` as a tag that can point at a newer base image by the time of the second build
• Names `npm install` resolving dependencies again, where an install from the lockfile would not
• Notices the required reviewer can put hours or days between the two builds, which widens every difference
• Knows that building one commit twice does not by default give the same image, even with identical inputs
• Builds once per commit, pushes by commit SHA, records the digest, and deploys that digest to both environments
• Moves environment differences to deploy time — environment variables or mounted configuration — and says what to do about a value that must be known at build time
• Pins the base image by digest and lets an automated pull request propose updates through CI
• Records which commit and workflow run built each digest, so a production image can be traced to its source

Follow-up questions

• The production build ran the morning after staging's. What could have changed overnight?
• You build the same commit twice, an hour apart, with every input pinned. Is it the same image?
• `web` needs its API URL at build time. Now what?
• Once the base image is pinned, how do security patches reach you?
• Should the image built for the pull request be the one that ships?

Purpose

Tests whether the candidate knows how the trigger works rather than having heard that it is dangerous: what code runs, with which token and secrets, and what that code can reach. Plenty of candidates can name the problem and cannot say which lines make it exploitable.

Signals to score

• Explains that this trigger runs the base repository's workflow with its secrets and a write token, and that checking out the fork's head commit runs a stranger's code in that context
• Names the way in: lifecycle scripts in the fork's `package.json` run during `npm install`, before anything is built or reviewed
• Says what that code can reach: the token checkout leaves for later git commands, the files and environment later steps inherit, and on hosted runners the secrets the job holds
• Names the second-order risk: a run in the default branch's context can write cache entries that later builds on `main` restore
• Splits the work: build the fork's code with no secrets and a read-only token, then deploy the output from a separate privileged workflow that never executes it
• Sets `permissions` per job to the minimum and stops checkout persisting credentials
• Pins third-party actions to full commit SHAs, because a tag can be moved to other code by whoever controls that repository
• Rotates the preview token now, scopes the replacement to previews, and treats the old one as exposed since the workflow was added
• Checks whether it was used: fork pull requests since the file was added, their run logs, the token's activity at the provider, pushes made by the workflow's identity
• Replaces the long-lived cloud keys in the deploy jobs with short-lived credentials issued to the workflow

Follow-up questions

• The deploy token is only passed to the last step. Is it out of reach of `npm install`?
• What in a fork's `package.json` runs, and when?
• A fork opened a pull request four months ago and closed it an hour later. What do you look at?
• The action is from a vendor you trust. Why is `@v2` still a problem?
• Contributors still want previews. How does your version give them one?

Purpose

Not scored. The close is for the candidate to find out what they would be walking into; anything notable goes in the notes.

Purpose

A specific, true, unflattering fact about your own pipeline shows the candidate that this round describes the job they would do, and tells one who wanted it already tidy that it is not. Make sure it is still true.