GET, POST, PUT and DELETE /api/traffic/rewrite-rules and
POST /api/traffic/rewrite-rules/{id}/toggle manage the rules behind
requireAuth. A rule is validated on save (400), created disabled, and
validated again before the toggle enables it. The value of a sensitive action
is never returned; an update sent without it keeps the stored value when the
stored rule has a sensitive action of the same type and header name at the
same position.
The manager compiles the enabled rules after traffic.Open and after every
change, and publishes the snapshot with Traffic.SetRewriteRules, an atomic
swap. An enabled rule the validation refuses is left out and reported as
load_error.
The proxy stores the snapshot and does not apply it: a test sends the same
request with and without matching rules and compares what the target
receives, what the client receives and what is recorded.
traffic/rewrite holds the rule model of the capture proxy's rewrite rules,
their save-time validation, an immutable compiled snapshot and the
transformation of one request or response. It imports the standard library
only, so both server and traffic can use it.
Rules run by priority ascending then id ascending, as a pipeline; within a
rule the actions run in a fixed order. A rule is atomic and fails open: a
failing action or a panic cancels that rule only. A rule that needs a body it
cannot touch (binary, over 1 MiB, compressed, streamed, absent by protocol)
is skipped whole. The engine never writes to the URL, headers or bodies it is
given.
Apply returns the trace lines of the phase: the rule snapshot once per rule,
the body of the last body action only, and no value for a sensitive action or
for a field one wrote earlier in the phase. A sensitive value is fingerprinted
with HMAC-SHA256 truncated to 128 bits, or not at all without a key.
Nothing calls the engine from the proxy yet.
For plain http the client puts the Host in the absolute-form request target, so the proxy dials it; for https go-mitmproxy records the exchange under the inner Host. Comment and error message only.
The Host refusal now explains that go-mitmproxy rebuilds the upstream request from the URL and drops the Host override. ReplayResult gains Recorded (false for an https passthrough host) and the sorted response Headers, both exposed by the route. Headers named in Connection and Proxy-Authorization are dropped. A body over the limit returns ErrReplayTooLarge, mapped to 413.
Traffic.Replay sends an edited request (method, URL, headers, body) through the local proxy, which records the exchange like any other, and returns the response bounded to 2 MiB. POST /api/traffic/replay exposes it to operators only (no agent tool). Invalid input is rejected with ErrReplayInvalid; hop-by-hop headers are dropped, redirects are not followed, and a Host that differs from the URL is refused because the proxy routes by Host.
Stream payloads over 4 KiB now spill to the blob store (spillAt with maxStreamInline; spill keeps maxInlineBody for HTTP bodies) and StreamMessages clips every payload to 4 KiB, reading a spilled binary preview from its blob. The byte cap counts the inline preview. persistStream spills before opening the transaction, as record() does. Without the library lock nothing is read. ts is unix seconds. Adds the final-message-on-close and route tests.
The recording proxy now stores WebSocket and Server-Sent Events streams: one exchange per stream (101 websocket, or the real text/event-stream response) and its messages in stream_messages, with per-message and per-exchange limits and blob spill. Messages are readable through Traffic.StreamMessages, GET /api/traffic/stream and the traffic_messages agent tool; deletion and blob GC cover them.
With traffic capture on, the Bash proxy URL carries a marker userinfo. The recording proxy drops Proxy-Authorization from every request before it is forwarded or recorded; when the value is the marker it marks the client connection, and every later request on it, including those inside its CONNECT tunnel, waits for the per-host token and jitter of the current noise profile (egress.Wait) and is counted, with a bounded wait. WebFetch, the http custom tools and enrich keep the unmarked URL, which egress already paces, so nothing is paced twice. Off paces nothing. Best effort: only clients that honour the proxy are covered, with capture on.
Closes#33. The MITM proxy only sees the agent's own actions, so
anything behind a human gate (signup captcha, email OTP, 2FA, SSO)
left the authenticated surface dark. The operator's browser becomes a
capture source instead:
- traffic: exported RecordExternal storing an externally captured
exchange through the same spill/blob/FTS path as proxy traffic
(insert logic shared with record()); rejects unparsable URLs and
empty methods
- server: POST /api/traffic/ingest inside the authenticated API group,
batch capped at 50 entries, bodies truncated at 256 KiB, per-entry
ingested/failed counts
- extension/: Manifest V3 Chrome/Edge extension driving the
chrome.debugger Network domain on the chosen tab, batching text
exchanges to the ingest endpoint with the operator's URL + JWT in
chrome.storage.local; binary bodies skipped; flush on batch/stop
- tests for both layers, README section
Comments, doc comments, UI labels, log and error messages, LLM prompt
strings, markdown docs, test data and assertions. Identifiers, enum
keys, routes, placeholders and code structure are unchanged.
Notes:
- docs/漏洞流量证据.md renamed to docs/vulnerability-traffic-evidence.md
- tests that exercised CJK specific handling (GBK zip names, Chinese
full text search) now use ASCII equivalents; go build passes and the
test suite matches the pre translation baseline (remaining failures
need a PostgreSQL DSN and fail identically on the original)
- frontend mention syntax now uses English kinds (finding, asset, ...)
matching the backend, chat mentions tests pass
The manual v0.3.6 sync (3675f7e) was incomplete, and the follow-up merge
(fa7b3cd) resolved conflicts in favour of that copy, silently reverting three
shipped changes. main is already an ancestor of this branch, so re-merging
cannot bring them back; restore them explicitly.
- schema.sql: re-add the idempotent llm_profiles_format_check migration.
CREATE TABLE IF NOT EXISTS leaves an existing table alone, so databases
created before openai-responses kept the old two-value CHECK and rejected
every openai-responses profile.
- orchestration.go: re-add the reseedPlannerPrompt/reseedWorkerPrompt calls,
dropped by a comment-alignment hunk while the functions stayed behind as
dead code. Without them existing installs never receive the quantified
acceptance check or the negative-conclusion evidence threshold.
- CHANGELOG.md: restore the 0.3.6 entries, the @neouks credit and the correct
norma version range.
Also close three recovery gaps in the new cold-archive path:
- traffic: write the staging journal even when no legacy host directory
exists. Modern installs keep traffic in SQLite only, so a crash between the
PostgreSQL commit and the SQLite commit left the archived task's exclusive
traffic in the hot store with nothing for recovery to act on.
- recoverTaskArchiveStages: treat a missing journal.json as a discardable
directory instead of an error, which used to set recoveryFailed and stop the
archive worker from starting for the rest of the process lifetime.
- stageTaskArchiveFiles: replay the journal's renames when resuming an
existing staging root. The journal is written before the first rename, so
its presence did not guarantee a complete payload, and a resumed round could
package an archive missing transcripts or workspace files.
Add delete controls to the Traffic and LLM Records pages so users can
purge recorded data without hand-editing the data volume.
Traffic page:
- Host target picker (Popover with checkboxes + select-all toggle)
- Exact-match batch delete for multiple hosts (DELETE /api/traffic/hosts)
- Substring delete for the host-filter input (DELETE /api/traffic?host=)
- Orphaned content-addressed blob garbage collection after deletes
- Distinct-hosts endpoint (GET /api/traffic/hosts) for the picker
LLM Records page:
- Task picker dropdown (distinct task_id + record counts)
- Per-task delete (DELETE /api/llm/records?task=)
Other:
- Add /dist/ to .gitignore (build artifact)
- Extend traffic_test.go with DeleteHost, DeleteHostsExact, Hosts tests
- Fix Popover render bug (replace ScrollArea that broke height calc)
- Remove fade/zoom animation on the host picker Popover