Online Judge

Design a LeetCode-style online judge that safely runs untrusted user code at scale.

00

Practice checkpoints

The requirements are open as a taste. From the numbers onward, the full guide opens in the app.

  1. 01
    Clarify scope
  2. 02
    Requirements + scale
  3. 03
    API + data modelUnlocks in the app
  4. 04
    Draw architectureUnlocks in the app
  5. 05
    Deep diveUnlocks in the app
  6. 06
    Trade-off decisionUnlocks in the app
01

Requirements that shape the design

Do not only state requirements. Ask for them. Each card pairs the design constraint with a clarification question you can say out loud before drawing the architecture.

Functional requirements

01What does the user see the instant they hit Submit?

The API accepts the code, returns a submission id immediately, and judging happens asynchronously — the verdict streams back seconds later, never inline in the request.

02How many languages, and who guarantees the toolchain?

A fixed language matrix (say C++, Java, Python, Go) with pinned compiler versions baked into sandbox images — the platform owns the toolchain, not the worker host.

03What exactly comes back as a verdict?

Per-test results rolled into one verdict — Accepted, Wrong Answer, Time Limit, Memory Limit, Runtime Error, Compile Error — plus measured CPU time and peak memory.

04Who sets the time and memory limits?

The problem does: each problem declares per-language CPU-time and memory limits, and the judge enforces them as hard caps, not suggestions.

05Is output always compared byte-for-byte?

No — each problem picks a checker: exact match with whitespace normalization, floating-point tolerance, or a special-judge program for problems with many valid answers.

06Can users ever see the hidden test data?

Never: users get the verdict and which test failed, but test inputs and expected outputs stay server-side — leaking them destroys the problem for everyone.

Out of scopeProblem authoring and test-data generation tooling · Plagiarism and code-similarity detection between contestants · Contest scoreboard math (penalties, ratings, tie-breaks)

Non-functional requirements

01Assume the code is actively malicious — what must hold?

Total containment: no network egress, no reading outside its own scratch directory, dangerous syscalls filtered — a hostile submission can kill its sandbox, never the host or another run.

02Same code submitted twice — same verdict, guaranteed?

Deterministic verdicts: one pinned core per run, CPU-time (not wall-clock) as the limit, no oversubscription — the same code must never get Time Limit on one run and Accepted on the next.

03How fast must verdicts land during a contest spike?

P95 verdict under about 30 seconds even when 10,000 submissions arrive in minute one — the queue absorbs the burst and the fleet drains it.

04One user fires 50 submissions — do others wait?

Scheduling is fair per user — round-robin across users, not FIFO across submissions — so one flooder queues behind their own work, not behind everyone else’s.

05What state may survive from one run to the next?

Nothing survives between runs: every submission gets a fresh sandbox from a clean image and a full teardown after — leftover processes or files corrupt both security and verdicts.

Keep asking — the interview is a conversation

Real interviews probe far more than a tidy list. These are the scope questions that separate candidates who interrogate the problem from those who recite it.

  • Is this steady practice traffic or contest mode — do we design for a minute-one spike of 10,000 submissions?
  • What is the threat model — merely buggy student code, or actively malicious code hunting the host and the hidden tests?
  • What verdict latency is acceptable at P95 during a contest, and does a visible queue position buy us patience?
  • Must verdicts be strictly reproducible — same code, same verdict, every time — because ratings and prizes hang on them?
  • Which languages at launch, and do we pin toolchain versions so an old submission re-judges identically years later?
01

Unlock the full playbook for Online Judge

Numbers, architecture diagram, API and data model, deep dives, expected topics, self-check, whiteboard starter, and common mistakes unlock inside the app.

02

Numbers that force architecture decisions

Locked in the app

03

Architecture path

Locked in the app

04

API and data model

Locked in the app

05

Deep dive directions

Locked in the app

+Series

Practice the related series