Commit Graph
325 Commits
Author SHA1 Message Date
Chubboy Ren c0c568b31f Merge remote-tracking branch 'upstream/main' into codex/fix-record-fact-input-contract 2026-11-18 01:34:26 +00:00
BaaRai 6568a8eb44 fix(llm): record breaker health on every task LLM path
The /system/llm/ badge read llm_profile_health, which only the failover
Pool wrote. A task pinned to an explicit chain, a bound agent with
failover off, and the global provider all call the provider directly, so
a dead backend stayed "Healthy".

One policy now lives in llmpool.Registry.RecordCall, used by the Pool and
by the task runtimes: a success clears the state, a failover-eligible
failure trips it, context cancellation and deterministic 400s do not, and
a stream that already delivered output is left alone. The selection
carries the profile whose breaker the call updates, and a provider
already wrapped in the Pool reports nothing to avoid double counting.

The LLM recording page also explains an empty list while the llm_record
toggle is off, after the settings call resolves.

Originally reported in #257.
2026-11-18 01:31:18 +01:00
Coff0xc 6a0b79d34f feat(findings): add report_state separate from triage 2026-11-18 00:07:50 +01:00
protonmatter 95bd9c24f0 Merge commit 'eddf03ce61f0a506fede039ea8936709b84ade8e' into protonmatter/session-epoch
# Conflicts:
#	CHANGELOG.md
2026-11-17 16:56:29 -05:00
protonmatter 134562c3d9 Drop the unused database import and align the password constants. 2026-11-17 22:54:16 +01:00
protonmatter 7cac9c240b Retry the password version on login when startup could not read it. 2026-11-17 22:54:16 +01:00
protonmatter ebc54b8751 Bind sessions to a password version so revocation cannot tie inside one second.
The hash and version commit together, a failed read rejects tokens, and the reset script tells the operator to restart.
2026-11-17 22:54:16 +01:00
protonmatter 26a7bbec37 Reject sessions issued before a password change.
A failed password read no longer mints a setup token, and filling the login limiter no longer drops an active lockout.
2026-11-17 22:54:16 +01:00
BaaRai eddf03ce61 Merge pull request #304 from Protonmatter/protonmatter/llmpool-health-queue
Coalesce circuit-breaker health writes
2026-11-17 22:52:27 +01:00
BaaRai 9bc5856565 Merge pull request #302 from Protonmatter/protonmatter/budget-restore
Restore task spend from the lanes the live ledger counts
2026-11-17 22:52:23 +01:00
protonmatter c9d81a453f Align the Server literal so gofmt accepts it. 2026-11-17 22:39:53 +01:00
protonmatter b2a2f95e8c Release the circuit-breaker queue lock around the database write. 2026-11-17 22:39:53 +01:00
protonmatter 1fbf2263a1 Keep circuit-breaker writes in state order and flush them before the database closes.
A later update can no longer be queued behind an older one, a failed save is retried, and shutdown waits for the last cooling window.
2026-11-17 22:39:53 +01:00
protonmatter 847e32407a Coalesce circuit-breaker writes onto one queue.
A newer state for the same profile replaces one that has not been written yet, and a failed write is logged instead of discarded on a detached goroutine.
2026-11-17 22:39:50 +01:00
BaaRai 7bd9bf6e3e Merge pull request #309 from BaaRai/feat/traffic-rewrite-p33b1
feat(traffic): apply rewrite rules in the capture proxy (P3.3b1)
2026-11-17 22:38:59 +01:00
BaaRai f8bc3b3b6d Merge pull request #305 from Protonmatter/protonmatter/json-body-cap
Cap JSON request bodies
2026-11-17 22:29:08 +01:00
protonmatter 074962a351 Say that a revived task keeps the spend already recorded. 2026-11-17 16:25:16 -05:00
BaaRai b57bafe74a feat(server): upgrade the built-in traffic_get description once
It now says that traffic_get shows the originals of a rewritten exchange
and that traffic_search never finds a value a rewrite rule wrote. As for
traffic_search, only the original built-in text is replaced.
2026-11-17 22:23:12 +01:00
BaaRai 5fdf8eb075 fix(server): keep the MaxBytesError so a caller's 413 mapping still works
A handler that installs its own MaxBytesReader and then calls decode lost the 413: decode converted the error to a plain one and the middleware's limitedBody had been replaced. Return the original *http.MaxBytesError and update the cap test to errors.As.
2026-11-17 22:19:59 +01:00
BaaRai 03ac1c7d47 Merge pull request #300 from Protonmatter/protonmatter/shutdown-outbox
Drain task goroutines on shutdown and replay unsaved activities
2026-11-17 22:08:25 +01:00
protonmatter 186594ed3a Cap every non-multipart body and answer an oversized one with 413.
Handlers that decode JSON directly were still unbounded, and a body past the cap was reported as a bad request.
2026-11-17 15:57:43 -05:00
protonmatter 40de6ba898 Say what a restored task budget still allows, and test the lanes that count.
A task already inside the wrap-up reserve only has that reserve left. The worker estimate starts from the defaults because usage rows have no run boundary.
2026-11-17 15:50:00 -05:00
protonmatter a1c6103cbf Stop admitting work before the drain and replay each outbox row once.
The listener closes before Wait, and a shutdown refuses new task operations. Wait also stays until detached runs finish. The activity insert and the outbox delete commit together, and a row that keeps failing is dropped after five attempts so it cannot block the rest.
2026-11-17 15:41:56 -05:00
BaaRai d4c1a5cca1 Merge pull request #298 from BaaRai/feat-bulk-finding-status
feat(findings): bulk status update from the findings table
2026-11-17 19:58:49 +01:00
protonmatter 0234025d28 Drain task goroutines on shutdown and replay activities that could not be stored.
The outbox migration is 0006 because 0005 is already traffic rewrite rules. A shutdown cancel leaves an in-flight intent claim running, so the next start can reset it to open instead of recording the interrupt as done or blocked.
2026-11-17 13:05:23 -05:00
protonmatter 69bdd94d13 Cap JSON request bodies and bound how long a body may arrive.
decode stops at 32 MiB. ReadTimeout limits header and body arrival without cutting a handler that is already running.
2026-11-17 13:04:50 -05:00
protonmatter be25f14970 Restore a task spend cap from the lanes the live ledger counts.
Planner, worker, judge, and compactor rows rebuild the cap. Main-agent chat, goal decomposition, and side questions stay out, so a restart does not count turns the running process never metered.
2026-11-17 13:04:36 -05:00
Coff0xc cc0082cfda fix(db): scope RSI lessons from task_scope and stop splitting UTF-8
Lessons were scoped on tasks.company_id, which create never writes.
Generic rows also sorted first on Postgres.
2026-11-17 18:03:10 +00:00
BaaRai 75efc29323 feat(findings): bulk status update from the findings table 2026-11-17 17:52:09 +01:00
BaaRai b452b13989 feat(web_search): wire the opt-in Keenable backend 2026-11-17 17:22:35 +01:00
BaaRai ea59a8aea5 Merge pull request #252 from renaesop/codex/upload-500mib
fix: support 512 MiB task and chat attachments
2026-11-17 16:22:43 +01:00
BaaRai f33e3d14f2 Merge pull request #286 from Coff0xc/fix/llmpool-window-tokens
fix(server): use the real context window for llmpool and sidequestion
2026-11-17 15:45:50 +01:00
Coff0xc 14b93cecab fix(server): use the real context window for llmpool and sidequestion
8K/16K profiles were treated as 32K because WindowTokens came from
CompactionWindow(). They 400 and don't fail over.
2026-11-17 12:56:13 +00:00
Chubboy Ren 70d6833f35 fix: repair record_fact input contracts and asset aliases 2026-11-17 06:14:31 +00:00
dach-a fa33d4dac0 test: assert compression by size, not by looking for a word
The plaintext-substring guard encoded one encoder's output shape: it only held
while a small input compressed to fewer bytes than it started with, and Go
1.27 stores instead. Comparing sizes states the same invariant without
assuming how flate picks between stored and compressed blocks: a stored block
is larger than its input by construction, so "output is under half the input"
is true for any working encoder on a repetitive body and false for a body that
was never compressed.

Checked against the old 60-byte fixture on go1.27.4: it fails with "fixture is
not compressed: 60 bytes in, 85 bytes out", so the guard still catches the
case it was written for.
2026-11-17 03:49:25 +00:00
dach-a 25816c7fd6 test: make the gzip fixture compressible on every toolchain
TestDecodeEvidenceBodyGzipQuoteFound built its body from the 60-byte
decodeHTML constant and guarded with "the stream must not contain admin".
That assumes the encoder always deflates, and Go 1.27's flate does not: for
this input it emits a stored block, the stream is 85 bytes, and the raw bytes
are the plaintext the guard rejects. The test then fails on the encoder's
choice rather than on the decode it is meant to cover.

Repeat the body so it has something to compress, and assert the decoded
output equals it, so the guard measures the round trip instead of one
encoder's block selection. The quote and the body's content are unchanged,
and the sibling truncated/deflate tests already use repeated input.
2026-11-17 03:31:36 +00:00
Chubboy Ren 99a0d27f88 fix: enforce 512 MiB uploads across client and proxy 2026-11-17 02:03:57 +00:00
Chubboy Ren d41a5232ab Merge remote-tracking branch 'upstream/main' into codex/upload-500mib 2026-11-17 01:55:45 +00:00
BaaRai 278de8b2e0 Merge pull request #268 from liuxiaobo0920/fix/report-finding-quality
feat(report): surface finding proof strength in the report and the export
2026-11-17 02:09:01 +01:00
Xiaobo a0adc209ad feat(report): surface finding proof strength in the report and the export
The task report and the findings export asserted every finding with no
qualification: nothing said how far exploitation actually reached (PoE level),
whether the claim was machine-verified (verification status and note), whether
counterevidence was recorded, or whether any traffic was bound. The web UI
badges all of it (VERIFICATION_META, the detail page), so the report — the
artifact a client actually reads — was the last surface still asserting bare
claims.

Render it from one place:

- report/quality.go (new): two constructors and two pure renderers, no I/O, no
  DB handle, so the block is unit-tested without a database.
- Both surfaces call the same helpers, which is the point: the task report and
  the per-finding export cannot drift apart the way duplicated formatting does.
- Missing is explicit: a finding node with no standalone row prints
  "not assessed (no finding record)" instead of an invented zero — the honest
  gap the existing coverage work (#207) also insists on.
- Labels match the shipped UI wording (Verification, Confidence rationale,
  Counterevidence) and states are humanised ("Not assessed", "False positive"),
  with an unrecognised value printed verbatim so a new state cannot silently
  vanish from a report.
- Bound-traffic count appears once per document: as a Proof bullet in the task
  report (it has no traffic section), and in the export only when nothing is
  bound — otherwise the traffic section already states it.
- The detailed report is capped at 8 KiB in the task report only, rune-safe, with
  the cap derived into the truncation message; the export is the canonical
  per-finding document and is never silently shortened.
- The export keeps its document structure: the single-finding document gets
  "## Detailed Report" (a section like its siblings), the multi-finding summary
  gets a bold label.

Read paths only: no change to the write side of poe_level or verification_status,
no schema change (all five columns already exist), no API/DTO change (the widened
FindingMeta stays internal to the report path, and the findings list keeps
selecting only what it renders). One extra task-scoped query on GET /api/report,
because findings bodies stay off the list path.

docs/vulnerability-traffic-evidence.md §Export describes the new Markdown.
2026-11-17 08:40:37 +08:00
BaaRai 8474131ba2 fix(server): match a kept sensitive value on type, header name and regex
A sensitive action sent without its value took the stored value at the same
position, so two sensitive actions of the same type and header name swapped
their values when reordered. The stored action is now found by type,
canonical header name and regex, wherever it stands, and exactly one must
match: none or several is refused (400, send the value).

The CRUD test no longer expects a refusal for a reordered action.
2026-11-17 00:12:09 +01:00
BaaRai 97f081ed63 fix(server): refuse a host change while a stored sensitive value is kept
An update that keeps the stored value of a sensitive action can no longer
change the rule's host matcher (compared case-insensitively): the value
would otherwise be written to a new destination. Same invariant as the
custom HTTP tools. The url_regex matcher is not compared, an accepted and
documented limit.
2026-11-17 00:12:09 +01:00
BaaRai 7a9b4fdf76 feat(server): rewrite-rule API and compiled snapshot for the proxy
GET, POST, PUT and DELETE /api/traffic/rewrite-rules and
POST /api/traffic/rewrite-rules/{id}/toggle manage the rules behind
requireAuth. A rule is validated on save (400), created disabled, and
validated again before the toggle enables it. The value of a sensitive action
is never returned; an update sent without it keeps the stored value when the
stored rule has a sensitive action of the same type and header name at the
same position.

The manager compiles the enabled rules after traffic.Open and after every
change, and publishes the snapshot with Traffic.SetRewriteRules, an atomic
swap. An enabled rule the validation refuses is left out and reported as
load_error.

The proxy stores the snapshot and does not apply it: a test sends the same
request with and without matching rules and compares what the target
receives, what the client receives and what is recorded.
2026-11-17 00:12:09 +01:00
Toys0125 d18b841a70 feat: expose concurrent multi-LLM worker rotation 2026-11-16 15:31:47 -06:00
BaaRai 1263cdc185 fix(traffic): replay reports recorded and headers, drops Connection-named headers, 413 for large bodies
The Host refusal now explains that go-mitmproxy rebuilds the upstream request from the URL and drops the Host override. ReplayResult gains Recorded (false for an https passthrough host) and the sorted response Headers, both exposed by the route. Headers named in Connection and Proxy-Authorization are dropped. A body over the limit returns ErrReplayTooLarge, mapped to 413.
2026-11-16 16:17:55 +01:00
BaaRai e770caa28e feat(traffic): replay an edited request through the recording proxy
Traffic.Replay sends an edited request (method, URL, headers, body) through the local proxy, which records the exchange like any other, and returns the response bounded to 2 MiB. POST /api/traffic/replay exposes it to operators only (no agent tool). Invalid input is rejected with ErrReplayInvalid; hop-by-hop headers are dropped, redirects are not followed, and a Host that differs from the URL is refused because the proxy routes by Host.
2026-11-16 15:32:02 +01:00
BaaRai dcdc0652ad fix(traffic): bound stream message pages, spill before the transaction, seconds timestamps
Stream payloads over 4 KiB now spill to the blob store (spillAt with maxStreamInline; spill keeps maxInlineBody for HTTP bodies) and StreamMessages clips every payload to 4 KiB, reading a spilled binary preview from its blob. The byte cap counts the inline preview. persistStream spills before opening the transaction, as record() does. Without the library lock nothing is read. ts is unix seconds. Adds the final-message-on-close and route tests.
2026-11-16 08:21:38 +01:00
BaaRai d9df84d2e5 feat(traffic): record WebSocket and SSE stream messages
The recording proxy now stores WebSocket and Server-Sent Events streams: one exchange per stream (101 websocket, or the real text/event-stream response) and its messages in stream_messages, with per-message and per-exchange limits and blob spill. Messages are readable through Traffic.StreamMessages, GET /api/traffic/stream and the traffic_messages agent tool; deletion and blob GC cover them.
2026-11-16 07:47:25 +01:00
BaaRai 2c514a4cd9 perf(server): look up only the requested task archive blocker in getTask
getTask aggregated task_relations for the whole database on every 5-second detail poll to read one entry. TaskArchiveBlocker scopes the same predicate (shared with TaskArchiveBlockers through archiveBlockerScope) to one source task; listTasks keeps the global map.
2026-11-16 06:46:52 +01:00
Chubboy Ren 202de31b5b fix: support 500 MiB chat attachments 2026-11-16 10:29:14 +08:00