Senior (L5) · Staff (L6) · System design

AI Infrastructure Engineer

AI infrastructure loops test whether you can run models in production: admission control and cost governance in front of providers, GPU-efficient serving, retrieval that does not hallucinate, and agent workflows that survive crashes.

How the loop runs

Roughly 37 minutes across three rounds.

1. 1

   High-level design

   15 min

   Data plane vs control plane, request lifecycle, where state lives

2. 2

   Chaos injection

   12 min

   Provider outages, KV-cache pressure, retry storms, noisy tenants

3. 3

   Cost & edge cases

   10 min

   Token spend governors, eval gates, rollout safety

Where candidates lose points

• Treating the LLM provider as infinitely available
• No hard per-tenant budget, only soft alerts
• Checking ACLs after retrieval instead of inside it
• Ignoring time-to-first-token while optimizing throughput