01What does the platform accept — a prompt, or a goal?
A user goal with success criteria: the platform decomposes it into a plan, runs a step loop, and keeps a run state the user can watch, pause, and cancel.
Premium preview
Design a production AI agent application platform that accepts user goals, manages context and memory, calls tools safely, pauses for human review when needed, and improves through traces and evals.
The requirements are open as a taste. From the numbers onward, the full guide opens in the app.
Do not only state requirements. Ask for them. Each card pairs the design constraint with a clarification question you can say out loud before drawing the architecture.
01What does the platform accept — a prompt, or a goal?
A user goal with success criteria: the platform decomposes it into a plan, runs a step loop, and keeps a run state the user can watch, pause, and cancel.
02How does the agent touch the outside world?
Only through registered tools with typed schemas — every call is validated before execution and recorded after, so the set of possible side effects is enumerable, never ad hoc.
03What actually enters the model on each step?
A curated context pack, assembled fresh each step: instructions, the goal, the plan, compact summaries of prior steps, and just-relevant memory — never the raw append-everything transcript.
04A step needs human sign-off — what happens to the run?
The run checkpoints its full state and parks; a human approves hours or days later and the run resumes from durable state exactly where it stopped — an approval pause is a checkpoint, not a held process.
05Does the platform remember users across runs?
Yes, as governed memory: facts and preferences carry provenance and expiry and can be corrected or deleted — a stale preference or a revoked permission must not steer a future run.
06How do behavior changes ship?
Through the eval harness: every prompt, tool, or model change re-runs the regression suites — built partly from traced production failures — before rollout, so behavior ships with evidence, not vibes.
Out of scopeTraining or fine-tuning foundation models (the platform orchestrates hosted ones) · Vertical agent products (support, coding — they run as tenants on this platform) · Tool transport internals (the MCP — Model Context Protocol — gateway layer below; this platform consumes it)
01What stops a run from spiraling?
Hard budgets checked before every step — step count, token cost, wall clock — so a stuck loop trips a circuit breaker and halts with a partial-result report, never an unbounded bill.
02A worker dies mid-run — what does the user lose?
Nothing completed is lost: run state persists after every step. External side effects use a stable idempotency key plus an execution record or outbox; a checkpoint alone cannot prevent the sent-email-but-not-checkpointed failure window.
03A run did something odd — can we reconstruct why?
Every step is traced — context pack contents, prompt and model versions, tool inputs and outputs, budget state, the decision taken — replayable end to end.
04Many tenants, shared platform — where are the walls?
Tenants are strictly isolated: memory, tool credentials, and run state are scoped per tenant, and each tool call carries the end user’s own permission, never a platform-wide credential.
05What keeps a long run from getting expensive AND dumb?
Per-run cost is bounded by design: context packs cap per-step tokens so cost grows linearly with steps, and large tool outputs are processed outside the context, not appended into it.
Real interviews probe far more than a tidy list. These are the scope questions that separate candidates who interrogate the problem from those who recite it.
Numbers, architecture diagram, API and data model, deep dives, expected topics, self-check, whiteboard starter, and common mistakes unlock inside the app.
Locked in the app
Locked in the app
Locked in the app
Locked in the app