Skip to content
Internals

Architecture

The pieces of the application, its four execution paths, and what each leaves on disk.

The pieces

The pieces of the application The browser talks to the Flask routes, the backend reads and writes the host’s disk, and reaches a run host, GitHub and PyPI, the model provider and the judge. BROWSER · LOOPBACK NULLIUS · FLASK · APP/WEB ON THE APPLICATION HOST ELSEWHERE REGISTRY REDACTED SSH · RSYNC HARBOR RUN Pages runs, job, trial, compare, builder, settings Routes app/web, one module per capability Backend app/nullius, imports no Flask settings.json user configuration directory workspaces/<name>/ parse cache, imports, judgements runs/<name>/ run record, logs, jobs harbor-pins/<pin>/ built wheels, run with uvx, detached GitHub · PyPI refs, releases Judge TypeSafe.ai SSH run host harbor run Model provider called by the run

app/web/, the Flask side, one module per capability; app/nullius/, the backend, one package per purpose, no Flask; app/templates/ and app/static/, the interface.

Root pyproject.toml packages the backend as nullius and the web layer and its assets as the private _nullius_app package. The unified CLI loads Flask only for serve; backend commands do not import it.

Bundled resources are read-only. User config, data and cache roots come from nullius.paths (locations); the user catalogue overrides bundled entries by whole folder. No personal data lives in the wheel.

Path 1: open results that exist

Path 1: open results that exist Roots given to the runs pages are recognised, converted when foreign, parsed into the workspace cache, and the pages read that cache back. /IMPORT · ROOTS RECOGNISED? INTO IMPORTED/ PARSE EACH JOB PARSE CACHE /RUNS /JOB /TRIAL READ · RE-PARSE You Runs pages web/runs Importers nullius/importers Parser parse, classify Workspace workspaces/<name>/ LEGEND Call Write
Step Detail
Recognised Nullius and Harbor jobs (one shape); ATIF trajectories; OTLP/JSON span exports
Folder picker Matching folders appear as you type, including mounted disks and linked directories
Refused Any other root, with the reason
From a shell nullius import <tree> -o <root> (Import a run)
Refresh Refresh runs re-parses and re-converts

Path 2: build a recipe and run it here

Path 2: build a recipe and run it here The builder posts the recipe, execution records the run and starts each part through the pinned Harbor, which calls the model, and the job page follows it live. LOOP [nop, oracle, then the agents] EACH STEP, REVIEW RUN POST /RUN RUN.JSON · KEY RUN-PART.SH MODEL CALLS JOBS/, EVERY 10 S JOB PAGE · LIVE You Builder page, recipe core Execution nullius/execution Harbor pinned, via uvx Model provider LEGEND Call Return What you see Loop
Recipe Runs
Nullius The standard: agents or capabilities that proved the benchmark's ports, several per run, or its own scripts as Original
Harbor Harbor's task environments through the Harbor it pins (Which Harbor)

Path 3: the same on an SSH host

Path 3: the same on an SSH host Execution probes the host, sends the part and the key, starts it detached, then tails the log and pulls the jobs back every ten seconds. LOOP [every 10 s] PROBE: DOCKER, UV (INSTALLED IF MISSING) SEND WHEELS, TASKS, RUN-PART.SH · KEY OVER STDIN START RUN-PART.SH, DETACHED TAIL THE LOG, READ THE EXIT FILE RSYNC JOBS/ INTO RUNS/<NAME>/JOBS/ Execution nullius/execution SSH host ~/.nullius/ Nullius closed: the part keeps running, and a restart picks it up.

nop, oracle, then the agents; a failed part pauses the run: Continue or Stop. Stop, restart, credentials and what cannot run from the page: Run.

For the standard, the host receives a deterministic snapshot of the running backend and effective catalogue, addressed by its contents. Source edits and user entries travel together; configuration, credentials and licence stores are excluded. Local source runs use the current checkout.

What a run leaves behind

What a run leaves behind A run directory holds its record and one job directory, whose trials each keep the agent’s output, the verifier’s and the artifacts. runs/<name>/ run.json: plan and state run-part.sh, log, exit file jobs/<job-name>/ config.json, result.json, events.jsonl <trial>/ result.json, trial.log, exception.txt agent/ trajectory.json, logs, traces verifier/ reward, test output artifacts/
File Read for
config.json Agent, model, dataset, job name
Job result.json Totals, start and end
events.jsonl The live board
Trial result.json Outcome, reward, exception, phase timings
agent/trajectory.json Conversation, tool calls, tokens
agent/gateway.log, agent/direct.log Trace evidence, provider errors
Other agent/*.json Trace documents, saved trace views
verifier/ Reward, test output
lock.json Which Harbor ran

What the workspace keeps

These paths are below the user data directory, or WORKSPACES_DIR for the workspace root. nullius paths shows the active location.

Path Holds
workspaces/<name>/workspace.json Roots, label, opened time
workspaces/<name>/jobs/<hash>.json Parse cache
workspaces/<name>/imported/ Converted foreign roots
workspaces/<name>/judgements/<job>/<trial>.json Judge records

Page by page

Page by page The runs page opens a job, a job opens its trials, live board, reconcile and feasibility views, and the runs page also opens the comparison. Runs /runs Compare /compare Job /job/<id> Live board while it runs Reconcile /job/<id>/reconcile Feasibility /job/<id>/feasibility Trial /trial/<job>/<name> conversation, trace, artifacts, judgement
Page Shows
/runs Runs by launch; ?view=all the flat table
/job/<id> Trials, or the live board while it runs
/trial/<job>/<name> Conversation, trace, artifacts, judgement
/job/<id>/reconcile Outcome versus judged failure cause
/job/<id>/feasibility Oracle and nop brackets, broken tasks
/compare Runs, whole or matched by task
/benchmarks, /agents, /capabilities, /kits Catalogue cards and entry pages; each section expands into a list in the left navigation