Skip to content
Release notes

v0.1.0rc1

v0.1.0rc1 · First release candidate

Bring anything. Run it. Prove it. Nullius brings AI evaluations onto your machines and opens each score into the evidence behind it.

  • Bring benchmarks, agents and capabilities. Add catalogue entries and match them by proven ports.
  • Run on your machines. Review a recipe, run locally or over SSH, and merge work across hosts.
  • Open every score into evidence. Inspect verifier reports, model calls, conversations and artifacts.
  • Reuse and compare results. Read Harbor jobs, import ATIF or OpenTelemetry traces, and compare shared tasks.
  • Check the benchmark too. Use controls and optional model judgement to investigate failures.

Install

Install the latest version with uv and Python 3.12+:

uv tool install nullius-eval
nullius serve

Open http://127.0.0.1:5057.

Full changelog

What's Changed

  • Initial import of Nullius by @aitianshi, @vultkayn, @yiannislamprou, @42int42 and @opPO333 in cd08443
  • Bump the python group across 3 directories with 2 updates by @dependabot[bot] in #2
  • Bump actions/setup-python from 7.0.0 to 7.1.0 in the github-actions group by @dependabot[bot] in #1
  • Use white transparent icons for documentation tabs by @aitianshi in #5

Full Changelog: All commits in v0.1.0rc1

Installation options, downloads and validation

Installation options

The package is nullius-eval; the application and command remain nullius. To select this specific candidate, use uv tool install nullius-eval==0.1.0rc1. Alternatively, download the wheel attached to this release and run uv tool install ./nullius_eval-0.1.0rc1-py3-none-any.whl.

To replace an existing Nullius tool installation, add --force to the install command. Settings, catalogue additions and runs are stored outside the installed package; nullius paths shows their locations.

Reading existing results does not need a model key. Running benchmarks needs the appropriate provider credentials and benchmark prerequisites; Docker is needed for Harbor and the standard's docker backend.

Downloads and validation

  • The .whl is the installable Python package, including the local web UI.
  • nullius_eval-0.1.0rc1.tar.gz is the source distribution for building the package.
  • SHA256SUMS records the checksums of both files. After downloading all three files into one directory, verify them on Linux with sha256sum --check SHA256SUMS.

These are Python packages, not standalone desktop installers. Release validation targets Linux with Python 3.12 and checks the backend, web routes, installed commands and package resources. It also checks that replacing an installation preserves user storage. This does not establish macOS or Windows support, or validate every benchmark against live model providers.

Feedback

Report installation problems, missing assets, incorrect results or confusing workflows through the issue tracker. Include the version, operating system, reproduction steps and relevant errors, with credentials and personal data removed. GitHub downloads and issue access follow the repository's access permissions.