BaaRai d2f4697b2b feat(bench): add the DrostBench corpus with a flag oracle ...

DrostBench v1 ships 100 docker-compose web challenges, each with one
flag-bearing vulnerability plus decoys that do not reach the flag, so
flag capture is an objective win condition that no judge decides.

scripts/benchmark/drost.py drives one challenge: compose build with an
explicit per-build random flag (the suite's deterministic sha256 default
is never used, so a memorized flag cannot be replayed), up --wait, host
port discovery, down --rmi local. Ground truth is derived from the
challenge benchmark.json; the flag is scanned over the fetched findings
first and over full details, facts and the report only before declaring
a miss.

run_bench.py gains --corpus-type drost (one challenge per invocation),
flag capture in every drost report, a flag-only mode when no judge is
configured (gates reported as not measured, never as zero), and
--deploy-only for install validation. Image mode is unchanged.

test_drost.py covers the pure functions (flag formats, challenge id
validation, port parsing, ground-truth derivation incl. its refusals);
the lifecycle was exercised end to end on DBEN-v1-001: build with a
random flag, reachable app, compiled flag matching the override,
teardown without leftovers.

Closes #94

2026-10-16 11:27:54 +02:00

..

benchmark

feat(bench): add the DrostBench corpus with a flag oracle

2026-10-16 11:27:54 +02:00

submission_review

feat: pre-submission report review (validate against program rules)

2026-10-12 08:09:57 +02:00

backup.sh

feat: built-in backup script (pg_dump with rotation, optional data archive)

2026-10-11 22:46:47 +02:00

posture_check.py

feat: refusal-posture probe for LLM profiles

2026-10-12 07:01:40 +02:00