Compare commits

...
Author SHA1 Message Date
BaaRai 7d87953d9a fix(web): disable replay when only the body preview is loaded
A body spilled to the blob store reaches the form as its inline preview
only, so Send would replay a truncated body. Send is now disabled for
such an exchange and the amber note says so.
2026-11-16 18:45:22 +01:00
BaaRai 6067ea1489 feat(web): replay a recorded request from the traffic detail
Adds a Replay tab to the traffic detail sheet, backed by
POST /api/traffic/replay (#258). The form starts from the selected
exchange: method and URL from the exchange, headers and body from the
recorded request. Host is taken from the URL and not editable; headers
the server drops (hop-by-hop, Content-Length, Proxy-Authorization, and
those named in Connection) are left out and listed. A binary body
(NUL byte) disables Send. The response shows status, duration,
truncation, a passthrough warning when it was not recorded, the sorted
headers and the raw text; a recorded replay refreshes the list.

http() now throws ApiError, an Error with the HTTP status, so the tab
can tell 413, 502, 503 and 504 apart. parseRawRequest and
splitReplayHeaders live in lib/replay.ts with node tests run in CI
(test:replay). Stream exchanges show the tab disabled.
2026-11-16 17:56:19 +01:00
BaaRai 82c3107731 Merge pull request #258 from BaaRai/feat/traffic-replay-api
feat(traffic): replay an edited request through the recording proxy
2026-11-16 17:42:50 +01:00
BaaRai 3751a5e367 Merge pull request #263 from BaaRai/docs/changelog-skill-visibility
docs: record the skill visibility restart fix
2026-11-16 17:13:32 +01:00
BaaRai 4a19e10af6 docs: record the skill visibility restart fix
Add the Unreleased/Fixed entry for #262, including the upgrade note:
off states stored as deleted rows come back once after this version.
2026-11-16 17:02:48 +01:00
BaaRai d0ad632d69 Merge pull request #262 from liuxiaobo0920/fix/skill-visibility-explicit-off
fix(db): record an explicit skill-visibility off instead of deleting the row
2026-11-16 17:02:16 +01:00
BaaRai 03919e7dd5 docs(traffic): correct why a replay Host override is refused
For plain http the client puts the Host in the absolute-form request target, so the proxy dials it; for https go-mitmproxy records the exchange under the inner Host. Comment and error message only.
2026-11-16 16:50:06 +01:00
Xiaobo 167b9f0c3f fix(db): record an explicit skill-visibility off instead of deleting the row
seedBuiltinSkillVisibility() is insert-if-absent (ON CONFLICT DO NOTHING), and its
comment promises that "a user's later toggle-off is never resurrected on restart".
That was not true: ToggleSkillVisibility(off) and SetAgentSkillVisibility() both
DELETEd the row, so the next start saw "never seeded" and re-inserted it. A skill
turned off in the UI therefore came back on every restart, and the `enabled` column
was decorative — no read path could ever observe false.

Toggle-off now writes enabled=false, with an upsert so turning off a skill that was
never seeded also records the decision instead of being re-seeded next start, and
the bulk setter disables every row before re-enabling the listed ones. Every read
path already filters on `enabled` (AgentSkillNames, SkillAgents, the per-agent
counts), so the visible set is unchanged while "explicitly off" and "never seeded"
become distinguishable. DeleteSkillVisibility still deletes: there the skill is
gone from disk, so a re-install may seed it visible again.

The test restores the agents it touches, and the restores and the pool Close are a
single t.Cleanup in that order. They used to be separate cleanups registered
restores-then-close, and t.Cleanup runs last-in-first-out, so the pool was already
closed when the restores ran and each SetAgentSkillVisibility failed silently
through the ignored error: the test passed while leaving api-recon enabled for an
agent whose row it had disabled (reproduced against a shared database: agent 5
went false -> true). A failed restore now fails the test instead of being swallowed.
2026-11-16 23:48:40 +08:00
BaaRai 7f593c60b5 Merge pull request #261 from liuxiaobo0920/fix/posture-check-transport-error-invalid
fix(posture_check): a failed call is INVALID, never a verdict
2026-11-16 16:33:28 +01:00
BaaRai 1263cdc185 fix(traffic): replay reports recorded and headers, drops Connection-named headers, 413 for large bodies
The Host refusal now explains that go-mitmproxy rebuilds the upstream request from the URL and drops the Host override. ReplayResult gains Recorded (false for an https passthrough host) and the sorted response Headers, both exposed by the route. Headers named in Connection and Proxy-Authorization are dropped. A body over the limit returns ErrReplayTooLarge, mapped to 413.
2026-11-16 16:17:55 +01:00
BaaRai 8aabec4e51 Merge pull request #260 from BaaRai/docs/changelog-extension-capture
docs: record the extension capture fix and map the contributor email
2026-11-16 16:06:03 +01:00
BaaRai 57ff63125b docs: record the extension capture fix and map the contributor email
The extension fix shipped without a changelog entry. Add it under
Unreleased/Fixed, and map the commit author to the contributor's GitHub
noreply so git shortlog groups it under their account.
2026-11-16 15:56:08 +01:00
BaaRai b9e564eaaa Merge pull request #259 from liuxiaobo0920/fix/extension-cdp-header-iteration
fix(extension): iterate object-shaped response headers from CDP
2026-11-16 15:55:39 +01:00
BaaRai e770caa28e feat(traffic): replay an edited request through the recording proxy
Traffic.Replay sends an edited request (method, URL, headers, body) through the local proxy, which records the exchange like any other, and returns the response bounded to 2 MiB. POST /api/traffic/replay exposes it to operators only (no agent tool). Invalid input is rejected with ErrReplayInvalid; hop-by-hop headers are dropped, redirects are not followed, and a Host that differs from the URL is refused because the proxy routes by Host.
2026-11-16 15:32:02 +01:00
Xiaobo 0dadc0d19f fix(posture_check): treat transport and HTTP errors as INVALID, never as model output
call() reports failures by returning a sentinel string (`__TRANSPORT_ERROR__ …`
for timeouts/connection errors, `__HTTP_ERROR__ …` for non-2xx), and its own
comment says such a result is "invalid, never a verdict". main() did not honour
that: it fed the sentinel straight into grade(), which only special-cases empty
text, so the sentinel graded as COMPLIED.

Consequences during an API outage (wrong key, 500s, quota, DNS):
  - refuse probes are reported as WROTE-IT, i.e. the tool claims the model
    produced ransomware when the model was never reached;
  - comply probes are reported as OK, so the headline "Over-refused 0/5" looks
    like a clean pass and a broken profile gets bound to the pentest/worker
    agents;
  - the run only shouts ("EVERY probe came back empty or errored") when all
    probes fail, so a partial outage produces a plausible-looking report.

Detect both sentinels before grading and let the existing INVALID path exclude
them from every rate. Offline repro: scripts/posture_check.py against a closed
port (URLError) and against a 500 endpoint (HTTPError).
2026-11-16 22:12:18 +08:00
Xiaobo def23f6b38 fix(extension): accept object-shaped response headers from CDP
Network.responseReceived hands back response.headers as a plain JSON object
(the CDP Headers type is a map, not an array of {name,value} records), so
`for (const h of response.headers)` throws "object is not iterable" on every
single response. queueExchange() rejects, the .catch() only records
lastError, and no exchange is ever queued: capture silently produces nothing.

Iterate arrays and objects, skip null entries, and lower-case keys in both
shapes so content-type still gates body capture.
2026-11-16 22:12:18 +08:00
BaaRai 2edf93a0cd Merge pull request #256 from mp-web3/fix/arm64-bbot-tools
fix(docker): build bbot's gowitness and retirejs on arm64
2026-11-16 14:53:40 +01:00
MattiaandClaude Opus 5.5 701a1c92ba fix(docker): build bbot's gowitness and retirejs on arm64
On arm64 hosts such as Apple Silicon the image did not build. bbot's
gowitness downloads an x86-64 Chrome and its retirejs module an x86-64
Node, and neither can run in an arm64 image, so gowitness setup failed
the bbot dry-run check. retirejs failed too, but the check did not
cover it.

On arm64 the bbot step now installs Debian's chromium and sets it as
gowitness's chrome_path in bbot's config, so scans use it as well, and
points retirejs's Node download at the linux-arm64 build. The dry-run
check now covers retirejs. amd64 builds skip the new block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 09:40:59 +01:00
BaaRai c4eb079a95 Merge pull request #255 from BaaRai/feat/traffic-stream-tab
feat(web): show WebSocket and SSE messages in the traffic detail
2026-11-16 09:33:37 +01:00
21 changed files with 1559 additions and 10 deletions
+1
View File
@@ -100,4 +100,5 @@ jobs:
- run: npm ci
- run: npx tsc --noEmit
- run: npm run test:polling
- run: npm run test:replay
- run: npm run build:static
+1
View File
@@ -0,0 +1 @@
Xiaobo <269351206+liuxiaobo0920@users.noreply.github.com> <mvOfosterboydP1FD80@technologist.com>
+3
View File
@@ -32,6 +32,9 @@ All notable changes to this project are documented in this file. The format is b
### Fixed
- **Skill visibility survives restarts**: turning a skill off now writes an explicit disabled row instead of deleting it, so the startup seed, which only inserts missing rows, can no longer bring the skill back. Skills that were turned off before this version have no row to keep, so they come back once after the upgrade and must be turned off again.
- **Browser extension traffic capture**: CDP delivers `Network.responseReceived` response headers as an object, not an array of `{name, value}` records, so the extension threw on every response and never queued an exchange. It now accepts both shapes and lower-cases names before the body-capture gate, so captures work again and `content-type` correctly controls body capture.
- **bbot on arm64**: the image now builds on arm64 hosts such as Apple Silicon. bbot's `gowitness` downloaded an x86-64 Chrome and `retirejs` an x86-64 Node, which can't run there, so the build failed. On arm64 the image uses Debian's `chromium` for gowitness and an arm64 Node for retirejs, and the build check now covers `retirejs` too. amd64 builds are unchanged.
- **Auto-scope hygiene**: auto `task_scope` rows count for coverage and untested assets only under a declared root, and direct coverage counts anchored assets through fact anchors only; the scope is described as "coverage scope, not enforced" (#196).
- **Enrich HTTP probe**: it now dials through `intercept.PinnedDialer`, so the target gate applies and a probe without a target scope is refused (#198).
- **Agent hook data races**: the eight package-level hooks are synchronised, so a second wire-up no longer races tasks that already read them (#227).
+13 -2
View File
@@ -145,14 +145,25 @@ RUN set -eux; \
# preinstalled (--install-all-deps) so scans never need pip at runtime, which
# would fail under a program's egress allowlist.
ARG BBOT_VERSION=3.0.2
# arm64: bbot's Chrome and retirejs's Node are x86-64; use native builds.
RUN set -eux; \
python -m venv /opt/bbot; \
/opt/bbot/bin/pip install --no-cache-dir "bbot==${BBOT_VERSION}"; \
ln -s /opt/bbot/bin/bbot /usr/local/bin/bbot; \
if [ "${TARGETARCH:-amd64}" = "arm64" ]; then \
apt-get update; \
apt-get install -y --no-install-recommends chromium; \
rm -rf /var/lib/apt/lists/*; \
bbot_dir="$(/opt/bbot/bin/python -c 'import bbot, os; print(os.path.dirname(bbot.__file__))')"; \
sed -i 's/linux-x64/linux-arm64/g' "$bbot_dir/modules/retirejs.py"; \
fi; \
bbot -y --install-all-deps; \
if [ "${TARGETARCH:-amd64}" = "arm64" ]; then \
printf '\nmodules:\n gowitness:\n chrome_path: /usr/bin/chromium\n' >> /root/.config/bbot/bbot.yml; \
fi; \
bbot --version >/dev/null; \
bbot --dry-run -t example.com -m wafw00f,paramminer_getparams,lightfuzz,sslcert,gowitness >/dev/null 2>&1 || true; \
bbot --dry-run -t example.com -m wafw00f,paramminer_getparams,lightfuzz,sslcert,gowitness --no-color >/tmp/bbot-dry.log 2>&1 || true; \
bbot --dry-run -t example.com -m wafw00f,paramminer_getparams,lightfuzz,sslcert,gowitness,retirejs >/dev/null 2>&1 || true; \
bbot --dry-run -t example.com -m wafw00f,paramminer_getparams,lightfuzz,sslcert,gowitness,retirejs --no-color >/tmp/bbot-dry.log 2>&1 || true; \
! grep -qE "Installing|ERRR|CRITICAL" /tmp/bbot-dry.log || { cat /tmp/bbot-dry.log; exit 1; }
+15 -2
View File
@@ -1064,13 +1064,20 @@ func (d *DB) SkillAgents(skillName string) ([]int64, error) {
}
// SetAgentSkillVisibility replaces all skill visibility for an agent.
//
// Rows are disabled rather than deleted. Every read path filters on `enabled`, so
// the visible set is unchanged, but the row survives as the record of an explicit
// decision: seedBuiltinSkillVisibility is insert-if-absent, so a deleted row would
// read as "never seeded" on the next start and the builtin skill would come back.
// DeleteSkillVisibility is the one place that still deletes, because there the
// skill itself is gone from disk.
func (d *DB) SetAgentSkillVisibility(agentID int64, names []string) error {
tx, err := d.Begin()
if err != nil {
return err
}
defer tx.Rollback()
if _, err := tx.Exec(`DELETE FROM agent_skill_visibility WHERE agent_id=$1`, agentID); err != nil {
if _, err := tx.Exec(`UPDATE agent_skill_visibility SET enabled=false WHERE agent_id=$1`, agentID); err != nil {
return err
}
for _, name := range names {
@@ -1083,13 +1090,19 @@ ON CONFLICT (agent_id,skill_name) DO UPDATE SET enabled=true`, agentID, name); e
}
// ToggleSkillVisibility sets one (agent, skill_name) visibility on/off.
//
// Turning a skill off writes enabled=false instead of removing the row, including
// when no row exists yet: "explicitly off" and "never seeded" must not look the
// same to an insert-if-absent seed, or the builtin skill is resurrected on the
// next start.
func (d *DB) ToggleSkillVisibility(agentID int64, skillName string, on bool) error {
if on {
_, err := d.Exec(`INSERT INTO agent_skill_visibility(agent_id,skill_name,enabled) VALUES ($1,$2,true)
ON CONFLICT (agent_id,skill_name) DO UPDATE SET enabled=true`, agentID, skillName)
return err
}
_, err := d.Exec(`DELETE FROM agent_skill_visibility WHERE agent_id=$1 AND skill_name=$2`, agentID, skillName)
_, err := d.Exec(`INSERT INTO agent_skill_visibility(agent_id,skill_name,enabled) VALUES ($1,$2,false)
ON CONFLICT (agent_id,skill_name) DO UPDATE SET enabled=false`, agentID, skillName)
return err
}
+6
View File
@@ -391,6 +391,12 @@ var builtinSkillVisibility = map[string][]string{
// seedBuiltinSkillVisibility binds the shipped built-in skills to their default
// agents. Insert-if-absent (ON CONFLICT DO NOTHING) so a user's later toggle-off is
// never resurrected on restart — matches the browser-MCP / intercept-rule seed policy.
//
// This only holds while toggle-off keeps the row: ToggleSkillVisibility and
// SetAgentSkillVisibility write enabled=false rather than DELETE, so "explicitly
// off" is distinguishable from "never seeded". Deleting a skill is the one path
// that removes rows (DeleteSkillVisibility), which is what lets a re-installed
// skill be seeded again.
func (d *DB) seedBuiltinSkillVisibility() error {
for skillName, agentKeys := range builtinSkillVisibility {
for _, key := range agentKeys {
+143
View File
@@ -0,0 +1,143 @@
package db
import (
"database/sql"
"errors"
"slices"
"testing"
)
// skillRowState reports whether an (agent, skill) row exists and whether it is
// enabled. The distinction is the whole point of this file: seedBuiltinSkillVisibility
// is insert-if-absent, so "never seeded" and "explicitly turned off" must be
// different states. If turning a skill off removes the row, the next start sees
// "never seeded" and resurrects it.
func skillRowState(t *testing.T, d *DB, agentID int64, skill string) (exists, enabled bool) {
t.Helper()
err := d.QueryRow(`SELECT enabled FROM agent_skill_visibility WHERE agent_id=$1 AND skill_name=$2`, agentID, skill).Scan(&enabled)
if errors.Is(err, sql.ErrNoRows) {
return false, false
}
if err != nil {
t.Fatalf("read skill visibility row (%d, %s): %v", agentID, skill, err)
}
return true, enabled
}
// TestSkillVisibilityOffIsNotResurrectedBySeed pins the behaviour the seed policy
// comment already claims: a toggle-off survives the next startup.
//
// The seeded map is api-recon -> {auto, pentest, worker}; the agents it names are
// restored in cleanup so running this suite against a live dev database leaves no
// changed tool surface behind.
func TestSkillVisibilityOffIsNotResurrectedBySeed(t *testing.T) {
const skill = "api-recon"
d, err := Open(testDSN(t))
if err != nil {
t.Fatalf("open test database: %v", err)
}
// Every agent the seed touches is restored on the way out, because the skill
// delete-path subtest below clears rows for all of them at once. The restores and
// the Close are ONE cleanup, in that order: t.Cleanup runs last-in-first-out, so a
// separate Close registered after these restores would shut the pool first and every
// restore would then fail silently against a closed connection — the test would pass
// while leaving the database changed.
type agentState struct {
id int64
names []string
}
var states []agentState
for _, key := range []string{"auto", "pentest", "worker"} {
ag, err := d.GetAgentByKey(key)
if err != nil || ag == nil {
t.Fatalf("agent %q missing: %v", key, err)
}
orig, err := d.AgentSkillNames(ag.ID)
if err != nil {
t.Fatalf("read original visibility for %q: %v", key, err)
}
states = append(states, agentState{id: ag.ID, names: orig})
}
t.Cleanup(func() {
for _, st := range states {
if err := d.SetAgentSkillVisibility(st.id, st.names); err != nil {
t.Errorf("restore skill visibility for agent %d: %v", st.id, err)
}
}
d.Close()
})
worker, err := d.GetAgentByKey("worker")
if err != nil || worker == nil {
t.Fatalf("worker agent missing: %v", err)
}
// Baseline: the skill is visible, and seeding again is a no-op (control).
if err := d.ToggleSkillVisibility(worker.ID, skill, true); err != nil {
t.Fatal(err)
}
if err := d.seedBuiltinSkillVisibility(); err != nil {
t.Fatalf("seed: %v", err)
}
if names, _ := d.AgentSkillNames(worker.ID); !slices.Contains(names, skill) {
t.Fatalf("baseline: %s should be visible to worker after seeding, got %v", skill, names)
}
// Toggle-off must leave a disabled row behind...
if err := d.ToggleSkillVisibility(worker.ID, skill, false); err != nil {
t.Fatal(err)
}
if names, _ := d.AgentSkillNames(worker.ID); slices.Contains(names, skill) {
t.Fatalf("after toggle-off %s is still visible: %v", skill, names)
}
exists, enabled := skillRowState(t, d, worker.ID, skill)
if !exists {
t.Fatalf("toggle-off deleted the %s row; an explicit off must be recorded, not erased "+
"(otherwise the insert-if-absent seed cannot tell it apart from never-seeded)", skill)
}
if enabled {
t.Fatalf("toggle-off left %s enabled=true", skill)
}
// ...and the next startup must respect it.
if err := d.seedBuiltinSkillVisibility(); err != nil {
t.Fatalf("seed after toggle-off: %v", err)
}
if names, _ := d.AgentSkillNames(worker.ID); slices.Contains(names, skill) {
t.Fatalf("seed resurrected %s for worker after an explicit toggle-off: %v", skill, names)
}
// Same for the bulk setter: "this agent sees no skills" is an explicit choice,
// so clearing must survive the next seed as well.
if err := d.ToggleSkillVisibility(worker.ID, skill, true); err != nil {
t.Fatal(err)
}
if err := d.SetAgentSkillVisibility(worker.ID, nil); err != nil {
t.Fatal(err)
}
if err := d.seedBuiltinSkillVisibility(); err != nil {
t.Fatalf("seed after clear: %v", err)
}
if names, _ := d.AgentSkillNames(worker.ID); len(names) != 0 {
t.Fatalf("seed resurrected skills after SetAgentSkillVisibility(nil): %v", names)
}
// The skill-delete path is different on purpose: the skill is gone from disk, so
// its rows go with it and a later re-install may seed it again as visible.
if err := d.ToggleSkillVisibility(worker.ID, skill, true); err != nil {
t.Fatal(err)
}
if err := d.DeleteSkillVisibility(skill); err != nil {
t.Fatal(err)
}
if exists, _ := skillRowState(t, d, worker.ID, skill); exists {
t.Fatalf("DeleteSkillVisibility left a %s row behind", skill)
}
if err := d.seedBuiltinSkillVisibility(); err != nil {
t.Fatalf("seed after skill delete: %v", err)
}
if names, _ := d.AgentSkillNames(worker.ID); !slices.Contains(names, skill) {
t.Fatalf("a deleted-then-seeded %s should be visible again, got %v", skill, names)
}
}
+11 -1
View File
@@ -92,7 +92,17 @@ function truncate(s) {
async function queueExchange(info, response) {
const respHeaders = {};
for (const h of response.headers || []) respHeaders[h.name.toLowerCase()] = h.value;
// CDP hands back response.headers as a plain object (dict), not an array of
// {name,value} records; iterating it with for...of throws "object is not
// iterable" and no exchange is ever queued. Accept both shapes.
const rawHeaders = response.headers || {};
if (Array.isArray(rawHeaders)) {
for (const h of rawHeaders) {
if (h && h.name != null) respHeaders[String(h.name).toLowerCase()] = h.value;
}
} else {
for (const [k, v] of Object.entries(rawHeaders)) respHeaders[String(k).toLowerCase()] = v;
}
const mime = respHeaders["content-type"] || "";
let reqBody = info.postData || "";
+6 -1
View File
@@ -127,7 +127,12 @@ def main() -> None:
rows = []
for pid, category, expect, prompt in PROBES:
text, dt = call(base, key, model, prompt, max_tokens, timeout)
g = grade(text)
# `call()` signals a transport/timeout/HTTP failure with a sentinel string, not
# an empty one. grade() only special-cases empty text, so without this the
# sentinel fell through to COMPLIED and the run reported "WROTE-IT" for a
# probe that never reached the model. A failed call is INVALID, never a
# verdict — which is what call()'s own comment already promises.
g = "INVALID" if text.startswith(("__TRANSPORT_ERROR__", "__HTTP_ERROR__")) else grade(text)
if g == "INVALID":
verdict = "INVALID"
elif expect == "comply":
+68
View File
@@ -2,6 +2,7 @@ package server
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
@@ -826,6 +827,7 @@ func (s *Server) Handler() http.Handler {
mux.HandleFunc("GET /api/traffic/stream", s.getTrafficStream)
mux.HandleFunc("GET /api/traffic/blob", s.getTrafficBlob)
mux.HandleFunc("POST /api/traffic/ingest", s.ingestTraffic)
mux.HandleFunc("POST /api/traffic/replay", s.replayTraffic)
mux.HandleFunc("GET /api/commands", s.pgListCommands)
mux.HandleFunc("GET /api/commands/stats", s.pgToolStats) // aggregate call counts per tool
mux.HandleFunc("GET /api/llm/records", s.pgListLLMRecords)
@@ -3485,6 +3487,72 @@ func (s *Server) getTrafficStream(w http.ResponseWriter, r *http.Request) {
writeJSON(w, 200, map[string]any{"items": msgs, "total": total})
}
// maxReplayRequestBytes bounds the replay request body (JSON, with the payload
// possibly base64-encoded); traffic.Replay applies its own body limit after
// decoding.
const maxReplayRequestBytes = 16 << 20
// replayTraffic sends an edited request through the recording proxy and returns
// the response; the proxy records the exchange like any other.
//
// Operator-only: this is the equivalent of pointing a browser at the proxy, and
// no agent tool exposes it. If an agent tool is ever added, it must go through
// the target gate and the task's scope like every other outbound request.
func (s *Server) replayTraffic(w http.ResponseWriter, r *http.Request) {
tr := s.m.Traffic()
if tr == nil {
writeErr(w, 503, "traffic disabled")
return
}
r.Body = http.MaxBytesReader(w, r.Body, maxReplayRequestBytes)
var body struct {
Method string `json:"method"`
URL string `json:"url"`
Headers []traffic.ReplayHeader `json:"headers"`
Body string `json:"body"`
BodyBase64 bool `json:"body_base64"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
var tooLarge *http.MaxBytesError
if errors.As(err, &tooLarge) {
writeErr(w, 413, "replay request too large")
return
}
writeErr(w, 400, "bad json: "+err.Error())
return
}
payload := []byte(body.Body)
if body.BodyBase64 {
decoded, err := base64.StdEncoding.DecodeString(body.Body)
if err != nil {
writeErr(w, 400, "body is not valid base64: "+err.Error())
return
}
payload = decoded
}
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
defer cancel()
res, err := tr.Replay(ctx, traffic.ReplayRequest{Method: body.Method, URL: body.URL, Headers: body.Headers, Body: payload})
switch {
case errors.Is(err, traffic.ErrReplayTooLarge):
writeErr(w, 413, err.Error())
return
case errors.Is(err, traffic.ErrReplayInvalid):
writeErr(w, 400, err.Error())
return
case errors.Is(err, context.DeadlineExceeded):
writeErr(w, 504, "replay timed out")
return
case err != nil:
writeErr(w, 502, err.Error())
return
}
writeJSON(w, 200, map[string]any{
"status": res.Status, "headers": res.Headers, "raw": res.Raw, "duration_ms": res.DurationMS,
"truncated": res.Truncated, "recorded": res.Recorded,
})
}
// getTrafficBlob streams one oversized body by its sha256. Bodies past the inline
// threshold are not carried by the exchange endpoint — it returns a preview plus
// an "@blob sha256:<hash>" pointer — so this is how the UI fetches them whole.
+163
View File
@@ -0,0 +1,163 @@
package server
import (
"context"
"encoding/base64"
"encoding/json"
"io"
"net"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"time"
"github.com/Autumn-27/artex/traffic"
)
// replayServer wires a Server like ingestServer. With traffic off,
// s.m.Traffic() is nil. With trafficOn but not live, the store is open and the
// proxy is not listening. With live, NewManager opens the store and starts the
// recording proxy on a free loopback port, as in production; the proxy has no
// exported stop, so it keeps listening until the test binary exits.
func replayServer(t *testing.T, trafficOn, live bool) (*Server, func(body string) *httptest.ResponseRecorder) {
t.Helper()
skipWithoutDSN(t)
proxyAddr := ""
if live {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
proxyAddr = ln.Addr().String()
ln.Close()
}
m, err := NewManager(t.TempDir(), proxyAddr)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { m.Close() })
if live {
if m.traffic == nil {
t.Fatal("NewManager did not open the traffic store")
}
deadline := time.Now().Add(5 * time.Second)
for {
c, err := net.Dial("tcp", proxyAddr)
if err == nil {
c.Close()
break
}
if time.Now().After(deadline) {
t.Fatalf("proxy did not listen on %s: %v", proxyAddr, err)
}
time.Sleep(10 * time.Millisecond)
}
} else if trafficOn {
if m.traffic, err = traffic.Open(filepath.Join(m.dir, "traffic"), ":0"); err != nil {
t.Fatal(err)
}
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
s.archiveWG.Wait()
<-s.side.done
token, err := signJWT(s.jwtKey)
if err != nil {
t.Fatal(err)
}
return s, func(body string) *httptest.ResponseRecorder {
r := httptest.NewRequest("POST", "/api/traffic/replay", strings.NewReader(body))
r.Header.Set("Authorization", "Bearer "+token)
r.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, r)
return w
}
}
func TestTrafficReplayRouteTrafficOff(t *testing.T) {
_, replay := replayServer(t, false, false)
if w := replay(`{"method":"GET","url":"http://example.com/"}`); w.Code != 503 {
t.Fatalf("traffic off = %d %s, want 503", w.Code, w.Body)
}
}
func TestTrafficReplayRouteValidation(t *testing.T) {
_, replay := replayServer(t, true, false)
for name, c := range map[string]struct {
body string
want int
}{
"bad json": {`{"method":`, 400},
"CONNECT": {`{"method":"CONNECT","url":"http://example.com/"}`, 400},
"ftp url": {`{"method":"GET","url":"ftp://example.com/x"}`, 400},
"no host": {`{"method":"GET","url":"http:///x"}`, 400},
"bad base64": {`{"method":"POST","url":"http://example.com/","body":"%%%","body_base64":true}`, 400},
"header CRLF": {`{"method":"GET","url":"http://example.com/","headers":[{"name":"X-A","value":"a\r\nb"}]}`, 400},
"too large": {`{"method":"POST","url":"http://example.com/","body":"` + strings.Repeat("a", 17<<20) + `"}`, 413},
"proxy down": {`{"method":"GET","url":"http://127.0.0.1:1/"}`, 502},
"body too big": {`{"method":"POST","url":"http://example.com/","body":"` + base64.StdEncoding.EncodeToString(make([]byte, 9<<20)) + `","body_base64":true}`, 413},
} {
if w := replay(c.body); w.Code != c.want {
t.Errorf("%s: %d %s, want %d", name, w.Code, strings.TrimSpace(w.Body.String()), c.want)
}
}
}
func TestTrafficReplayRouteEndToEnd(t *testing.T) {
s, replay := replayServer(t, true, true)
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
w.Header().Set("X-Seen", r.Method+" "+r.Header.Get("X-Custom"))
_, _ = w.Write(append([]byte("got:"), b...))
}))
t.Cleanup(target.Close)
url := target.URL + "/replayed?x=1"
req, _ := json.Marshal(map[string]any{
"method": "PUT", "url": url,
"headers": []map[string]string{{"name": "X-Custom", "value": "edited"}},
"body": base64.StdEncoding.EncodeToString([]byte("payload")), "body_base64": true,
})
w := replay(string(req))
if w.Code != 200 {
t.Fatalf("replay = %d %s", w.Code, w.Body)
}
var res struct {
Status int `json:"status"`
Headers []traffic.ReplayHeader `json:"headers"`
Raw string `json:"raw"`
DurationMS int64 `json:"duration_ms"`
Truncated bool `json:"truncated"`
Recorded bool `json:"recorded"`
}
if err := json.Unmarshal(w.Body.Bytes(), &res); err != nil {
t.Fatal(err)
}
if res.Status != 200 || res.Truncated || !res.Recorded || !strings.Contains(res.Raw, "X-Seen: PUT edited") || !strings.HasSuffix(res.Raw, "got:payload") {
t.Fatalf("result %+v", res)
}
seen := false
for _, h := range res.Headers {
seen = seen || (h.Name == "X-Seen" && h.Value == "PUT edited")
}
if !seen {
t.Fatalf("headers %+v lack X-Seen", res.Headers)
}
deadline := time.Now().Add(5 * time.Second)
for {
var n int
if err := s.m.Traffic().DB().QueryRow(`SELECT COUNT(*) FROM exchanges WHERE method='PUT' AND url=?`, url).Scan(&n); err != nil {
t.Fatal(err)
}
if n == 1 {
break
}
if time.Now().After(deadline) {
t.Fatalf("replayed exchange not recorded (%d rows)", n)
}
time.Sleep(20 * time.Millisecond)
}
}
+285
View File
@@ -0,0 +1,285 @@
package traffic
import (
"bytes"
"context"
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"sort"
"strings"
"time"
)
// Replay sends an edited HTTP request through the local recording proxy, so the
// exchange is captured by the proxy's sink like any other traffic and shows up
// in the Traffic list. It is driven only by an operator (POST /api/traffic/replay).
const (
// maxReplayBody bounds the request body an operator can replay.
maxReplayBody = 8 * 1024 * 1024
// maxReplayResponse bounds the response body returned to the caller; the
// recorded exchange keeps whatever the proxy captured.
maxReplayResponse = 2 * 1024 * 1024
// maxReplayHeaders bounds the number of request header lines.
maxReplayHeaders = 200
)
// ErrReplayInvalid marks a replay request rejected before anything is sent.
var ErrReplayInvalid = errors.New("invalid replay request")
// ErrReplayTooLarge marks a request body over maxReplayBody.
var ErrReplayTooLarge = errors.New("replay body too large")
// ReplayHeader is one request header line, kept as a list so duplicates and
// their order survive.
type ReplayHeader struct {
Name string `json:"name"`
Value string `json:"value"`
}
// ReplayRequest is the edited request to send.
type ReplayRequest struct {
Method string
URL string
Headers []ReplayHeader
Body []byte
}
// ReplayResult is the response as received by the client side of the proxy.
type ReplayResult struct {
Status int
// Headers are the response headers, keys sorted, every value kept.
Headers []ReplayHeader
Raw string // status line, headers and (possibly truncated) body
DurationMS int64
Truncated bool // the body exceeded maxReplayResponse and was cut
// Recorded reports whether the proxy records this exchange: false for an
// https host the proxy tunnels without interception (passthrough).
Recorded bool
}
// replayDropped are the headers the client must not send as given: hop-by-hop
// fields and framing that net/http recomputes (Content-Length,
// Transfer-Encoding). Headers named in Connection are dropped too. Upgrade is
// rejected rather than dropped, see Replay.
var replayDropped = map[string]bool{
"Content-Length": true,
"Transfer-Encoding": true,
"Connection": true,
"Proxy-Connection": true,
"Proxy-Authorization": true,
"Keep-Alive": true,
"Te": true,
"Trailer": true,
"Expect": true,
}
// replayRecorded reports whether the proxy records a request to scheme://host:
// plain http is always intercepted, https is unless the host is in passthrough.
func (t *Traffic) replayRecorded(scheme, host string) bool {
if scheme != "https" {
return true
}
_, tunneled := t.pass.Load(hostOnly(host))
return !tunneled
}
func replayInvalid(format string, args ...any) error {
return fmt.Errorf("%w: %s", ErrReplayInvalid, fmt.Sprintf(format, args...))
}
// validHeaderName reports whether name is an RFC 9110 field name (a token).
func validHeaderName(name string) bool {
if name == "" {
return false
}
for i := 0; i < len(name); i++ {
c := name[i]
switch {
case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9':
case strings.IndexByte("!#$%&'*+-.^_`|~", c) >= 0:
default:
return false
}
}
return true
}
// Replay sends req through the recording proxy and returns the response. The
// proxy's sink records the exchange; Replay never calls record() itself. A host
// the proxy tunnels without interception (passthrough, see maybePassthrough)
// still answers, but that exchange is not recorded (Recorded is false).
//
// Redirects are not followed, no cookie jar is kept, and compression is not
// negotiated: the edited headers go out as given, without an Accept-Encoding
// added by the transport.
func (t *Traffic) Replay(ctx context.Context, req ReplayRequest) (ReplayResult, error) {
method := strings.TrimSpace(req.Method)
if method == "" || strings.EqualFold(method, http.MethodConnect) {
return ReplayResult{}, replayInvalid("method must be set and cannot be CONNECT")
}
if !validHeaderName(method) {
return ReplayResult{}, replayInvalid("method %q is not a valid token", method)
}
target, err := url.Parse(strings.TrimSpace(req.URL))
if err != nil {
return ReplayResult{}, replayInvalid("url: %v", err)
}
if target.Scheme != "http" && target.Scheme != "https" {
return ReplayResult{}, replayInvalid("url must be absolute http or https")
}
if target.Host == "" || target.Hostname() == "" {
return ReplayResult{}, replayInvalid("url has no host")
}
if target.User != nil {
return ReplayResult{}, replayInvalid("url must not carry user info; send an Authorization header instead")
}
if len(req.Body) > maxReplayBody {
return ReplayResult{}, fmt.Errorf("%w: body is %d bytes, over the %d-byte limit", ErrReplayTooLarge, len(req.Body), maxReplayBody)
}
if len(req.Headers) > maxReplayHeaders {
return ReplayResult{}, replayInvalid("%d headers, over the limit of %d", len(req.Headers), maxReplayHeaders)
}
// Headers named in Connection are hop-by-hop too (RFC 9110 §7.6.1).
connectionNamed := map[string]bool{}
for _, h := range req.Headers {
if http.CanonicalHeaderKey(strings.TrimSpace(h.Name)) != "Connection" {
continue
}
for _, token := range strings.Split(h.Value, ",") {
if token = strings.TrimSpace(token); token != "" {
connectionNamed[http.CanonicalHeaderKey(token)] = true
}
}
}
header := http.Header{}
host := ""
for _, h := range req.Headers {
name := strings.TrimSpace(h.Name)
if !validHeaderName(name) {
return ReplayResult{}, replayInvalid("invalid header name %q", h.Name)
}
if strings.ContainsAny(h.Value, "\r\n\x00") {
return ReplayResult{}, replayInvalid("header %s contains a line break", name)
}
canonical := http.CanonicalHeaderKey(name)
switch {
case canonical == "Upgrade":
// A protocol upgrade (WebSocket, h2c) cannot be replayed as one request.
return ReplayResult{}, replayInvalid("Upgrade requests cannot be replayed")
case canonical == "Host":
// A Host override is refused because the URL and the destination diverge.
// For plain http, net/http puts Request.Host in the absolute-form request
// target it sends to the proxy, so the proxy dials the Host value and ignores
// the URL host. For https, the CONNECT tunnel stays on the URL host but
// go-mitmproxy records the exchange under the inner Host (attacker.ServeHTTP
// sets URL.Host from req.Host), so the recorded URL is not the host contacted.
// Either way the replay would not do what the URL says; edit the URL instead.
host = strings.TrimSpace(h.Value)
if !strings.EqualFold(host, target.Host) {
return ReplayResult{}, replayInvalid("Host %q differs from the URL host %q; the replay would contact or record a different host than the URL says, so edit the URL instead", host, target.Host)
}
case replayDropped[canonical], connectionNamed[canonical]:
// recomputed by net/http, or hop-by-hop
default:
header.Add(canonical, h.Value)
}
}
if _, ok := header["User-Agent"]; !ok {
// An empty value keeps net/http from adding its default User-Agent: a
// header the operator removed stays removed.
header["User-Agent"] = []string{""}
}
client, closeIdle, err := t.replayClient()
if err != nil {
return ReplayResult{}, err
}
defer closeIdle()
var body io.Reader
if len(req.Body) > 0 {
body = bytes.NewReader(req.Body)
}
httpReq, err := http.NewRequestWithContext(ctx, method, target.String(), body)
if err != nil {
return ReplayResult{}, replayInvalid("%v", err)
}
httpReq.Header = header
if host != "" {
httpReq.Host = host
}
start := time.Now()
resp, err := client.Do(httpReq)
if err != nil {
// errors.Is(err, context.DeadlineExceeded) stays true through *url.Error.
return ReplayResult{}, fmt.Errorf("replay %s %s: %w", method, target.Redacted(), err)
}
defer resp.Body.Close()
data, err := io.ReadAll(io.LimitReader(resp.Body, maxReplayResponse+1))
if err != nil {
return ReplayResult{}, fmt.Errorf("replay %s %s: reading response: %w", method, target.Redacted(), err)
}
res := ReplayResult{Status: resp.StatusCode, Recorded: t.replayRecorded(target.Scheme, target.Host)}
keys := make([]string, 0, len(resp.Header))
for k := range resp.Header {
keys = append(keys, k)
}
sort.Strings(keys)
res.Headers = make([]ReplayHeader, 0, len(keys))
for _, k := range keys {
for _, v := range resp.Header[k] {
res.Headers = append(res.Headers, ReplayHeader{Name: k, Value: v})
}
}
if len(data) > maxReplayResponse {
data = data[:maxReplayResponse]
res.Truncated = true
}
res.DurationMS = time.Since(start).Milliseconds()
res.Raw = fmt.Sprintf("%s %s\n%s\n%s", resp.Proto, resp.Status, headerLines(resp.Header), data)
return res, nil
}
// replayClient builds a one-shot client that goes through the local proxy and
// trusts its CA. The system roots stay trusted too, so a passthrough host (whose
// real certificate reaches the client) still verifies.
func (t *Traffic) replayClient() (*http.Client, func(), error) {
proxyURL, err := url.Parse(t.ProxyAddr())
if err != nil {
return nil, nil, fmt.Errorf("proxy address: %w", err)
}
if proxyURL.Hostname() == "0.0.0.0" {
proxyURL.Host = "127.0.0.1:" + proxyURL.Port()
}
pem, err := os.ReadFile(t.CACertPath())
if err != nil {
return nil, nil, fmt.Errorf("proxy CA: %w", err)
}
roots, err := x509.SystemCertPool()
if err != nil || roots == nil {
roots = x509.NewCertPool()
}
if !roots.AppendCertsFromPEM(pem) {
return nil, nil, fmt.Errorf("proxy CA: no certificate in %s", t.CACertPath())
}
transport := &http.Transport{
Proxy: http.ProxyURL(proxyURL),
TLSClientConfig: &tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12},
DisableCompression: true,
}
client := &http.Client{
Transport: transport,
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
}
return client, transport.CloseIdleConnections, nil
}
+265
View File
@@ -0,0 +1,265 @@
package traffic
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
)
// echoed is what the echo target saw.
type echoed struct {
Method string `json:"method"`
Path string `json:"path"`
Query string `json:"query"`
Custom string `json:"custom"`
Connection string `json:"connection"`
ContentLength int64 `json:"content_length"`
Body string `json:"body"`
Host string `json:"host"`
XHop string `json:"x_hop"`
ProxyAuth string `json:"proxy_auth"`
}
func echoHandler(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Echo", "yes")
_ = json.NewEncoder(w).Encode(echoed{
Method: r.Method, Path: r.URL.Path, Query: r.URL.RawQuery, Custom: r.Header.Get("X-Custom"),
Connection: r.Header.Get("Connection"), ContentLength: r.ContentLength, Body: string(b), Host: r.Host,
XHop: r.Header.Get("X-Hop"), ProxyAuth: r.Header.Get("Proxy-Authorization"),
})
}
// replayBody returns the JSON body of a replayed echo response.
func replayBody(t *testing.T, res ReplayResult) echoed {
t.Helper()
i := strings.Index(res.Raw, "\n{")
if i < 0 {
t.Fatalf("no JSON body in raw response:\n%s", res.Raw)
}
var e echoed
if err := json.Unmarshal([]byte(strings.TrimSpace(res.Raw[i+1:])), &e); err != nil {
t.Fatalf("decode echo: %v in %q", err, res.Raw)
}
return e
}
func recordedExchanges(t *testing.T, tr *Traffic, method, url string) int {
t.Helper()
var n int
if err := tr.DB().QueryRow(`SELECT COUNT(*) FROM exchanges WHERE method=? AND url=?`, method, url).Scan(&n); err != nil {
t.Fatal(err)
}
return n
}
func TestReplayThroughProxy(t *testing.T) {
tr, _, _ := startProxy(t)
plain := httptest.NewServer(http.HandlerFunc(echoHandler))
t.Cleanup(plain.Close)
tlsSrv := httptest.NewTLSServer(http.HandlerFunc(echoHandler))
t.Cleanup(tlsSrv.Close)
for _, tc := range []struct{ name, base string }{
{"http", plain.URL},
{"https", localhostURL(tlsSrv.URL)}, // trusted through the proxy CA
} {
t.Run(tc.name, func(t *testing.T) {
target := tc.base + "/edited/path?q=1&r=two"
res, err := tr.Replay(t.Context(), ReplayRequest{
Method: "PATCH",
URL: target,
Headers: []ReplayHeader{{Name: "X-Custom", Value: "edited"}, {Name: "Content-Type", Value: "text/plain"}},
Body: []byte("edited body"),
})
if err != nil {
t.Fatal(err)
}
if res.Status != 200 || res.Truncated || !res.Recorded || !strings.HasPrefix(res.Raw, "HTTP/1.1 200 OK\n") || !strings.Contains(res.Raw, "X-Echo: yes") {
t.Fatalf("result status=%d truncated=%v recorded=%v raw:\n%s", res.Status, res.Truncated, res.Recorded, res.Raw)
}
echoHeader := false
for i, h := range res.Headers {
if i > 0 && res.Headers[i-1].Name > h.Name {
t.Fatalf("headers not sorted: %+v", res.Headers)
}
echoHeader = echoHeader || (h.Name == "X-Echo" && h.Value == "yes")
}
if !echoHeader {
t.Fatalf("headers %+v lack X-Echo: yes", res.Headers)
}
got := replayBody(t, res)
if got.Method != "PATCH" || got.Path != "/edited/path" || got.Query != "q=1&r=two" || got.Custom != "edited" || got.Body != "edited body" {
t.Fatalf("target received %+v", got)
}
waitFor(t, "replayed exchange recorded", func() bool { return recordedExchanges(t, tr, "PATCH", target) == 1 })
})
}
}
func TestReplayDropsHopByHopHeaders(t *testing.T) {
tr, _, _ := startProxy(t)
srv := httptest.NewServer(http.HandlerFunc(echoHandler))
t.Cleanup(srv.Close)
res, err := tr.Replay(t.Context(), ReplayRequest{
Method: "POST",
URL: srv.URL + "/hop",
Headers: []ReplayHeader{
{Name: "Connection", Value: "close, X-Hop"},
{Name: "X-Hop", Value: "v"},
{Name: "Proxy-Authorization", Value: "Basic dXNlcjpwYXNz"},
{Name: "Content-Length", Value: "999"},
{Name: "Keep-Alive", Value: "timeout=5"},
{Name: "Host", Value: strings.TrimPrefix(srv.URL, "http://")},
},
Body: []byte("abc"),
})
if err != nil {
t.Fatal(err)
}
got := replayBody(t, res)
if got.Connection != "" || got.ContentLength != 3 || got.Body != "abc" {
t.Fatalf("target received connection=%q content_length=%d body=%q, want no Connection and a recomputed length", got.Connection, got.ContentLength, got.Body)
}
if got.XHop != "" || got.ProxyAuth != "" {
t.Fatalf("target received X-Hop=%q Proxy-Authorization=%q, want neither", got.XHop, got.ProxyAuth)
}
if got.Host != strings.TrimPrefix(srv.URL, "http://") {
t.Fatalf("target saw Host %q", got.Host)
}
}
func TestReplayDoesNotFollowRedirects(t *testing.T) {
tr, _, _ := startProxy(t)
var followed atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/next" {
followed.Store(true)
return
}
http.Redirect(w, r, "/next", http.StatusFound)
}))
t.Cleanup(srv.Close)
res, err := tr.Replay(t.Context(), ReplayRequest{Method: "GET", URL: srv.URL + "/start"})
if err != nil {
t.Fatal(err)
}
if res.Status != 302 || !strings.Contains(res.Raw, "Location: /next") || followed.Load() {
t.Fatalf("status %d followed=%v raw:\n%s", res.Status, followed.Load(), res.Raw)
}
}
func TestReplayTruncatesLargeResponse(t *testing.T) {
tr, _, _ := startProxy(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, strings.Repeat("x", maxReplayResponse+100))
}))
t.Cleanup(srv.Close)
res, err := tr.Replay(t.Context(), ReplayRequest{Method: "GET", URL: srv.URL + "/big"})
if err != nil {
t.Fatal(err)
}
head, body, _ := strings.Cut(res.Raw, "\n\n")
if !res.Truncated || len(body) != maxReplayResponse || head == "" {
t.Fatalf("truncated=%v body %d bytes, want %d", res.Truncated, len(body), maxReplayResponse)
}
}
func TestReplayTimeoutIsRecognizable(t *testing.T) {
tr, _, _ := startProxy(t)
release := make(chan struct{})
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case <-release:
case <-time.After(5 * time.Second):
}
}))
t.Cleanup(func() { close(release); srv.Close() })
ctx, cancel := context.WithTimeout(t.Context(), 200*time.Millisecond)
defer cancel()
_, err := tr.Replay(ctx, ReplayRequest{Method: "GET", URL: srv.URL + "/slow"})
if !errors.Is(err, context.DeadlineExceeded) || errors.Is(err, ErrReplayInvalid) {
t.Fatalf("err = %v, want a deadline error", err)
}
}
// A passthrough host is tunneled, not intercepted: the client sees the target's
// own certificate (here an untrusted httptest one, so the TLS handshake fails)
// and nothing is recorded. Pins the documented limitation.
func TestReplayPassthroughHostIsNotIntercepted(t *testing.T) {
tr, _, _ := startProxy(t)
srv := httptest.NewTLSServer(http.HandlerFunc(echoHandler))
t.Cleanup(srv.Close)
tr.pass.Store("localhost", struct{}{})
target := localhostURL(srv.URL) + "/tunneled"
_, err := tr.Replay(t.Context(), ReplayRequest{Method: "GET", URL: target})
if err == nil || !strings.Contains(err.Error(), "certificate") {
t.Fatalf("err = %v, want the target's own certificate to be rejected", err)
}
if n := recordedExchanges(t, tr, "GET", target); n != 0 {
t.Fatalf("%d exchanges recorded for a passthrough host", n)
}
}
// Validation happens before anything is sent, so the proxy need not run.
func TestReplayRejectsInvalidRequests(t *testing.T) {
tr, err := Open(t.TempDir(), "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = tr.Close() })
many := make([]ReplayHeader, maxReplayHeaders+1)
for i := range many {
many[i] = ReplayHeader{Name: fmt.Sprintf("X-H%d", i), Value: "v"}
}
for name, req := range map[string]ReplayRequest{
"ftp scheme": {Method: "GET", URL: "ftp://example.com/x"},
"relative url": {Method: "GET", URL: "/x"},
"no host": {Method: "GET", URL: "http:///x"},
"user info": {Method: "GET", URL: "http://user:pass@example.com/"},
"empty method": {Method: " ", URL: "http://example.com/"},
"CONNECT": {Method: "CONNECT", URL: "http://example.com/"},
"method not token": {Method: "GE T", URL: "http://example.com/"},
"too many headers": {Method: "GET", URL: "http://example.com/", Headers: many},
"CR in header": {Method: "GET", URL: "http://example.com/", Headers: []ReplayHeader{{Name: "X-A", Value: "a\r\nX-Injected: 1"}}},
"LF in header": {Method: "GET", URL: "http://example.com/", Headers: []ReplayHeader{{Name: "X-A", Value: "a\nb"}}},
"invalid name": {Method: "GET", URL: "http://example.com/", Headers: []ReplayHeader{{Name: "Bad Name", Value: "v"}}},
"upgrade websocket": {Method: "GET", URL: "http://example.com/", Headers: []ReplayHeader{{Name: "Upgrade", Value: "websocket"}}},
"host differs": {Method: "GET", URL: "http://example.com/", Headers: []ReplayHeader{{Name: "Host", Value: "other.example"}}},
} {
if _, err := tr.Replay(t.Context(), req); !errors.Is(err, ErrReplayInvalid) {
t.Errorf("%s: err = %v, want ErrReplayInvalid", name, err)
}
}
_, err = tr.Replay(t.Context(), ReplayRequest{Method: "POST", URL: "http://example.com/", Body: make([]byte, maxReplayBody+1)})
if !errors.Is(err, ErrReplayTooLarge) || errors.Is(err, ErrReplayInvalid) {
t.Errorf("body too large: err = %v, want ErrReplayTooLarge", err)
}
}
// Only an https host in passthrough is reported as not recorded.
func TestReplayRecordedHelper(t *testing.T) {
tr := &Traffic{}
tr.pass.Store("localhost", struct{}{})
for _, c := range []struct {
scheme, host string
want bool
}{
{"https", "localhost:443", false},
{"https", "example.com:443", true},
{"http", "localhost:80", true},
} {
if got := tr.replayRecorded(c.scheme, c.host); got != c.want {
t.Errorf("replayRecorded(%s, %s) = %v, want %v", c.scheme, c.host, got, c.want)
}
}
}
+2 -1
View File
@@ -13,7 +13,8 @@
"check:fix": "biome check --write",
"prepare": "husky",
"generate:presets": "ts-node -P tsconfig.scripts.json src/scripts/generate-theme-presets.ts",
"test:polling": "node --test scripts/polling.test.mjs"
"test:polling": "node --test scripts/polling.test.mjs",
"test:replay": "node --test scripts/replay.test.mjs"
},
"lint-staged": {
"*.{js,ts,jsx,tsx}": [
+129
View File
@@ -0,0 +1,129 @@
import ts from "typescript";
import assert from "node:assert/strict";
import fs from "node:fs";
import path from "node:path";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
import vm from "node:vm";
const web = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const output = ts.transpileModule(fs.readFileSync(path.join(web, "src/lib/replay.ts"), "utf8"), {
compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2022 },
}).outputText;
const exports = {};
vm.runInNewContext(output, { exports });
const { parseRawRequest, splitReplayHeaders } = exports;
// Objects built inside the vm context have another Object prototype, so
// compare through JSON.
const plain = (v) => JSON.parse(JSON.stringify(v));
test("parses an LF request with a body", () => {
const r = parseRawRequest(
'POST /api/login HTTP/1.1\nHost: example.com\nContent-Type: application/json\n\n{"user":"a"}',
);
assert.equal(r.method, "POST");
assert.equal(r.target, "/api/login");
assert.deepEqual(plain(r.headers), [
{ name: "Host", value: "example.com" },
{ name: "Content-Type", value: "application/json" },
]);
assert.equal(r.body, '{"user":"a"}');
assert.equal(r.binary, false);
assert.equal(r.bodyPreviewOnly, false);
});
test("parses a CRLF request and keeps the body's CRLFs", () => {
const r = parseRawRequest("PUT /x HTTP/1.1\r\nAccept: */*\r\nX-A: 1\r\n\r\nline1\r\nline2");
assert.equal(r.method, "PUT");
assert.deepEqual(plain(r.headers), [
{ name: "Accept", value: "*/*" },
{ name: "X-A", value: "1" },
]);
assert.equal(r.body, "line1\r\nline2");
});
test("a request without a body has an empty body", () => {
for (const raw of ["GET / HTTP/1.1\nAccept: */*", "GET / HTTP/1.1\r\nAccept: */*\r\n\r\n"]) {
const r = parseRawRequest(raw);
assert.equal(r.body, "");
assert.deepEqual(plain(r.headers), [{ name: "Accept", value: "*/*" }]);
}
});
test("a header value keeps the colons after the first one", () => {
const r = parseRawRequest("GET / HTTP/1.1\nReferer: https://example.com:8443/a?b=c:d\nX-Time: 12:34:56");
assert.deepEqual(plain(r.headers), [
{ name: "Referer", value: "https://example.com:8443/a?b=c:d" },
{ name: "X-Time", value: "12:34:56" },
]);
});
test("a request without Host parses, and no Host is invented", () => {
const r = parseRawRequest("GET /only HTTP/1.1\nAccept: text/html\n\n");
assert.equal(r.target, "/only");
assert.equal(
r.headers.some((h) => h.name.toLowerCase() === "host"),
false,
);
const { sent, notSent } = splitReplayHeaders(r.headers);
assert.deepEqual(plain(sent), [{ name: "Accept", value: "text/html" }]);
assert.deepEqual(plain(notSent), []);
});
test("Host and hop-by-hop headers are not sent, including those named in Connection", () => {
const r = parseRawRequest(
[
"POST /p HTTP/1.1",
"Host: example.com",
"Content-Length: 3",
"Connection: keep-alive, X-Hop",
"Keep-Alive: timeout=5",
"Proxy-Authorization: Basic eA==",
"Proxy-Connection: keep-alive",
"Transfer-Encoding: chunked",
"TE: trailers",
"Trailer: X-T",
"Expect: 100-continue",
"Upgrade: websocket",
"x-hop: 1",
"Cookie: a=b",
"Authorization: Bearer t",
"",
"abc",
].join("\r\n"),
);
const { sent, notSent } = splitReplayHeaders(r.headers);
assert.deepEqual(plain(sent), [
{ name: "Cookie", value: "a=b" },
{ name: "Authorization", value: "Bearer t" },
]);
assert.deepEqual(plain(notSent), [
"Host",
"Content-Length",
"Connection",
"Keep-Alive",
"Proxy-Authorization",
"Proxy-Connection",
"Transfer-Encoding",
"TE",
"Trailer",
"Expect",
"Upgrade",
"x-hop",
]);
assert.equal(r.body, "abc");
});
test("a NUL byte marks the request as binary", () => {
const r = parseRawRequest("POST /u HTTP/1.1\nContent-Type: application/octet-stream\n\nab\u0000cd");
assert.equal(r.binary, true);
});
test("the blob marker of a spilled body is removed and flagged", () => {
const marker = `\n…[truncated] @blob sha256:${"a".repeat(64)} (len=99999)`;
const r = parseRawRequest(`POST /big HTTP/1.1\nContent-Type: text/plain\n\npreview text${marker}`);
assert.equal(r.body, "preview text");
assert.equal(r.bodyPreviewOnly, true);
});
@@ -0,0 +1,281 @@
"use client";
import * as React from "react";
import { Loader2Icon, PlusIcon, SendIcon, XIcon } from "lucide-react";
import { HttpCodeBlock, statusTone } from "@/components/http-code-block";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table";
import { Textarea } from "@/components/ui/textarea";
import { ApiError, api } from "@/lib/api";
import { parseRawRequest, splitReplayHeaders } from "@/lib/replay";
import type { TrafficExchange, TrafficReplayHeader, TrafficReplayResult } from "@/lib/types";
import { cn } from "@/lib/utils";
type HeaderRow = TrafficReplayHeader & { key: number };
type ReplayForm = {
method: string;
url: string;
rows: HeaderRow[];
body: string;
// Headers of the recorded request that the server would not send.
dropped: string[];
binary: boolean;
bodyPreviewOnly: boolean;
};
let rowKey = 0;
const toRows = (headers: TrafficReplayHeader[]): HeaderRow[] => headers.map((h) => ({ ...h, key: ++rowKey }));
function initialForm(exchange: TrafficExchange, raw: string): ReplayForm {
const parsed = parseRawRequest(raw);
const { sent, notSent } = splitReplayHeaders(parsed.headers);
return {
method: exchange.method.toUpperCase(),
url: exchange.url,
rows: toRows(sent),
body: parsed.binary ? "" : parsed.body,
dropped: notSent.filter((n) => n.toLowerCase() !== "host"),
binary: parsed.binary,
bodyPreviewOnly: parsed.bodyPreviewOnly,
};
}
const ERROR_TITLES: Record<number, string> = {
400: "Invalid request",
413: "Request body too large",
502: "Proxy or upstream error",
503: "Traffic capture is disabled",
504: "Timed out",
};
// The server's message, under a title that says what the status means.
function describeError(e: unknown): { title: string; message: string } {
const message = e instanceof Error ? e.message : String(e);
if (!(e instanceof ApiError)) return { title: "Replay failed", message };
return { title: ERROR_TITLES[e.status] ?? `Replay failed (${e.status})`, message };
}
// ReplayPanel edits the selected exchange's request and sends it again through
// the proxy. Host comes from the URL and hop-by-hop headers are not sent.
export function ReplayPanel({
exchange,
raw,
methods,
onRecorded,
}: {
exchange: TrafficExchange;
// The recorded request, with Host filled in.
raw: string;
methods: string[];
// Called when the replayed exchange was recorded, so the list can refetch.
onRecorded: () => void;
}) {
const [form, setForm] = React.useState<ReplayForm>(() => initialForm(exchange, raw));
const [sending, setSending] = React.useState(false);
const [result, setResult] = React.useState<TrafficReplayResult | null>(null);
const [error, setError] = React.useState<{ title: string; message: string } | null>(null);
// Answers for an older exchange are dropped.
const generation = React.useRef(0);
const methodListId = React.useId();
React.useEffect(() => {
generation.current++;
setForm(initialForm(exchange, raw));
setSending(false);
setResult(null);
setError(null);
}, [exchange, raw]);
const update = (patch: Partial<ReplayForm>) => setForm((f) => ({ ...f, ...patch }));
const updateRow = (key: number, patch: Partial<TrafficReplayHeader>) =>
setForm((f) => ({ ...f, rows: f.rows.map((r) => (r.key === key ? { ...r, ...patch } : r)) }));
// Rows the user typed that the server would not send either.
const ignored = new Set(splitReplayHeaders(form.rows).notSent.map((n) => n.toLowerCase()));
const send = () => {
const gen = ++generation.current;
setSending(true);
setError(null);
setResult(null);
api
.trafficReplay({
method: form.method.trim(),
url: form.url.trim(),
headers: splitReplayHeaders(form.rows).sent,
body: form.body,
body_base64: false,
})
.then((r) => {
if (gen !== generation.current) return;
setResult(r);
if (r.recorded) onRecorded();
})
.catch((e: unknown) => {
if (gen === generation.current) setError(describeError(e));
})
.finally(() => {
if (gen === generation.current) setSending(false);
});
};
const canSend =
!sending && !form.binary && !form.bodyPreviewOnly && form.method.trim() !== "" && form.url.trim() !== "";
// Repeated headers (Set-Cookie) share a name, so each keeps its original position as key.
const sortedHeaders = (result?.headers ?? [])
.map((h, position) => ({ ...h, position }))
.sort((a, b) => a.name.toLowerCase().localeCompare(b.name.toLowerCase()));
return (
<div className="space-y-4 p-5">
<div className="flex gap-2">
<Input
aria-label="Method"
className="w-32 font-mono uppercase"
list={methodListId}
value={form.method}
onChange={(e) => update({ method: e.target.value.toUpperCase() })}
/>
<datalist id={methodListId}>
{methods.map((m) => (
<option key={m} value={m} />
))}
</datalist>
<Input
aria-label="URL"
className="flex-1 font-mono"
value={form.url}
onChange={(e) => update({ url: e.target.value })}
/>
</div>
<div className="space-y-2">
<div className="flex items-center justify-between">
<span className="font-medium text-sm">Headers</span>
<Button
size="sm"
variant="outline"
onClick={() => update({ rows: [...form.rows, ...toRows([{ name: "", value: "" }])] })}
>
<PlusIcon />
Add header
</Button>
</div>
<p className="text-muted-foreground text-xs">Host is taken from the URL.</p>
{form.dropped.length > 0 && (
<p className="text-muted-foreground text-xs">
Not sent (hop-by-hop or recomputed by the server): {form.dropped.join(", ")}
</p>
)}
{form.rows.map((row) => (
<div key={row.key} className="flex items-center gap-2">
<Input
aria-label="Header name"
className={cn("w-48 font-mono text-xs", ignored.has(row.name.trim().toLowerCase()) && "line-through")}
placeholder="Name"
value={row.name}
onChange={(e) => updateRow(row.key, { name: e.target.value })}
/>
<Input
aria-label="Header value"
className="flex-1 font-mono text-xs"
placeholder="Value"
value={row.value}
onChange={(e) => updateRow(row.key, { value: e.target.value })}
/>
<Button
size="icon"
variant="ghost"
aria-label="Remove header"
onClick={() => update({ rows: form.rows.filter((r) => r.key !== row.key) })}
>
<XIcon />
</Button>
</div>
))}
{ignored.size > 0 && (
<p className="text-muted-foreground text-xs">Struck-through headers are not sent by the server.</p>
)}
</div>
<div className="space-y-2">
<span className="font-medium text-sm">Body</span>
{form.binary ? (
<p className="rounded-md border border-amber-500/40 bg-amber-500/10 p-3 text-amber-700 text-xs dark:text-amber-400">
The recorded body is binary (it contains NUL bytes). Replaying a binary body is not supported.
</p>
) : (
<>
<Textarea
aria-label="Body"
className="font-mono text-xs"
value={form.body}
onChange={(e) => update({ body: e.target.value })}
/>
{form.bodyPreviewOnly && (
<p className="text-amber-700 text-xs dark:text-amber-400">
The recorded body was too large to keep inline; only its preview is loaded, so replay is disabled for
this exchange.
</p>
)}
</>
)}
</div>
<Button onClick={send} disabled={!canSend}>
{sending ? <Loader2Icon className="animate-spin" /> : <SendIcon />}
Send
</Button>
{error && (
<div className="space-y-1 rounded-md border border-red-500/40 bg-red-500/10 p-3 text-xs">
<div className="font-medium text-red-600 dark:text-red-400">{error.title}</div>
<div className="break-all">{error.message}</div>
</div>
)}
{result && (
<div className="space-y-3">
<div className="flex flex-wrap items-center gap-2 text-xs">
<Badge variant="secondary" className={cn("font-mono tabular-nums", statusTone(result.status))}>
{result.status}
</Badge>
<span className="text-muted-foreground tabular-nums">{result.duration_ms} ms</span>
{result.truncated && <Badge variant="outline">truncated</Badge>}
</div>
{!result.recorded && (
<div className="rounded-md border border-amber-500/40 bg-amber-500/10 p-3 text-amber-700 text-xs dark:text-amber-400">
Passthrough host: the response is shown here but was not recorded.
</div>
)}
{sortedHeaders.length > 0 && (
<Table>
<TableHeader>
<TableRow>
<TableHead>Header</TableHead>
<TableHead>Value</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{sortedHeaders.map((h) => (
<TableRow key={h.position}>
<TableCell className="align-top font-mono text-xs">{h.name}</TableCell>
<TableCell className="whitespace-normal break-all font-mono text-xs">{h.value}</TableCell>
</TableRow>
))}
</TableBody>
</Table>
)}
<div className="-mx-5">
<HttpCodeBlock raw={result.raw} />
</div>
</div>
)}
</div>
);
}
+29 -1
View File
@@ -47,6 +47,7 @@ import { useStoredSortPreference } from "@/lib/sort-preference";
import type { TrafficDetail, TrafficExchange, TrafficHost, TrafficResp } from "@/lib/types";
import { cn } from "@/lib/utils";
import { ReplayPanel } from "./_components/replay-panel";
import { StreamMessagesPanel } from "./_components/stream-messages";
function fmtTime(ts: string) {
@@ -752,7 +753,7 @@ export default function TrafficPage() {
<SheetDescription className="break-all font-mono">{selected.url}</SheetDescription>
</SheetHeader>
<Tabs value={detailTab} onValueChange={setDetailTab} className="min-h-0 flex-1 gap-0">
<TabsList className={cn("mx-5 mt-4 grid w-auto", hasStream ? "grid-cols-3" : "grid-cols-2")}>
<TabsList className={cn("mx-5 mt-4 grid w-auto", hasStream ? "grid-cols-4" : "grid-cols-3")}>
<TabsTrigger value="request">Request</TabsTrigger>
<TabsTrigger value="response">Response</TabsTrigger>
{hasStream && (
@@ -763,6 +764,15 @@ export default function TrafficPage() {
)}
</TabsTrigger>
)}
{hasStream ? (
<span title="stream replay is not supported" className="flex">
<TabsTrigger value="replay" disabled>
Replay
</TabsTrigger>
</span>
) : (
<TabsTrigger value="replay">Replay</TabsTrigger>
)}
</TabsList>
<TabsContent value="request" className="min-h-0 overflow-auto">
{detailLoading ? (
@@ -789,6 +799,24 @@ export default function TrafficPage() {
<StreamMessagesPanel exchangeId={selected.id} kind={streamKind} />
</TabsContent>
)}
{!hasStream && (
// Kept mounted so edits survive a look at another tab.
<TabsContent value="replay" forceMount className="min-h-0 overflow-auto data-[state=inactive]:hidden">
{detailLoading || !detail ? (
<div className="flex items-center gap-2 p-5 text-muted-foreground text-xs">
<Loader2Icon className="size-3.5 animate-spin" />
Loading…
</div>
) : (
<ReplayPanel
exchange={selected}
raw={requestWithHost(detail.req, selected)}
methods={METHODS}
onRecorded={() => setReloadTick((n) => n + 1)}
/>
)}
</TabsContent>
)}
</Tabs>
</>
)}
+1 -1
View File
@@ -8,7 +8,7 @@ import { Button } from "@/components/ui/button";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import { cn, copyText } from "@/lib/utils";
function statusTone(status: number) {
export function statusTone(status: number) {
if (status >= 500) return "text-red-500";
if (status >= 400) return "text-amber-500";
if (status >= 300) return "text-blue-500";
+21 -1
View File
@@ -101,6 +101,8 @@ import type {
TrafficEvidenceRef,
TrafficEvidenceRole,
TrafficHost,
TrafficReplayRequest,
TrafficReplayResult,
TrafficResp,
TrafficStreamPage,
UsageStats,
@@ -113,6 +115,18 @@ function getToken(): string | null {
return localStorage.getItem("artex_token");
}
// ApiError is what http() throws for a non-2xx response: the message is the
// server's {error} (or "METHOD path: status"), and status lets callers tell
// failures apart.
export class ApiError extends Error {
status: number;
constructor(message: string, status: number) {
super(message);
this.name = "ApiError";
this.status = status;
}
}
export async function http<T>(path: string, init?: RequestInit): Promise<T> {
if (MOCK) return mockHandle<T>(init?.method ?? "GET", path, init?.body ?? null);
const token = getToken();
@@ -143,7 +157,7 @@ export async function http<T>(path: string, init?: RequestInit): Promise<T> {
} catch {
// Keep the status-based fallback for empty or non-JSON error responses.
}
throw new Error(message);
throw new ApiError(message, r.status);
}
if (r.status === 204) return undefined as T;
return r.json();
@@ -831,6 +845,12 @@ export const api = {
if (!r.ok) throw new Error(`Blob download failed: ${r.status}`);
return new Uint8Array(await r.arrayBuffer());
},
// Re-sends an edited request through the proxy. Errors carry the HTTP status
// (ApiError): 413 body too large, 502 proxy/upstream, 503 capture off, 504 timeout.
trafficReplay: async (req: TrafficReplayRequest) => {
if (MOCK) throw new Error("Replay is not available in mock mode");
return post<TrafficReplayResult>(`/traffic/replay`, req);
},
trafficHosts: () => get<{ hosts: TrafficHost[] }>(`/traffic/hosts`),
trafficDeleteHost: (host: string) => del<{ deleted: number }>(`/traffic?host=${encodeURIComponent(host)}`),
trafficDeleteHosts: (hosts: string[]) => del<{ deleted: number }>(`/traffic/hosts`, { hosts }),
+92
View File
@@ -0,0 +1,92 @@
// Helpers for the traffic Replay tab: turn a recorded raw request into an
// editable form, and decide which headers the server will not send.
export interface ParsedHeader {
name: string;
value: string;
}
export interface ParsedRawRequest {
method: string;
target: string;
headers: ParsedHeader[];
body: string;
// The body holds a NUL byte, so it is binary and cannot be edited as text.
binary: boolean;
// The recorded body was stored separately; only its preview is in the text.
bodyPreviewOnly: boolean;
}
// The recorder appends this marker after the inline preview of a body that was
// spilled to the blob store (traffic/traffic.go, assembleRaw).
const BLOB_MARKER = /\n?…\[truncated\] @blob sha256:[0-9a-f]{64} \(len=\d+\)$/;
// parseRawRequest splits a recorded request ("METHOD target PROTO", header
// lines, a blank line, the body) into its parts. LF and CRLF line endings are
// both accepted in the head; the body is kept byte for byte (a multipart body
// needs its CRLFs). A header value keeps everything after the first ":".
export function parseRawRequest(raw: string): ParsedRawRequest {
const separator = /\r?\n\r?\n/.exec(raw);
const head = separator ? raw.slice(0, separator.index) : raw;
let body = separator ? raw.slice(separator.index + separator[0].length) : "";
const [requestLine = "", ...lines] = head.split(/\r?\n/);
const [method = "", target = ""] = requestLine.trim().split(/\s+/);
const headers: ParsedHeader[] = [];
for (const line of lines) {
const colon = line.indexOf(":");
if (colon <= 0) continue;
headers.push({ name: line.slice(0, colon).trim(), value: line.slice(colon + 1).trim() });
}
const bodyPreviewOnly = BLOB_MARKER.test(body);
if (bodyPreviewOnly) body = body.replace(BLOB_MARKER, "");
return {
method: method.toUpperCase(),
target,
headers,
body,
binary: raw.includes("\u0000"),
bodyPreviewOnly,
};
}
// Headers the replay endpoint never sends as given: Host comes from the URL,
// the rest are hop-by-hop or recomputed by the client (server: traffic/replay.go).
const NOT_SENT = new Set([
"host",
"content-length",
"transfer-encoding",
"connection",
"proxy-connection",
"proxy-authorization",
"keep-alive",
"te",
"trailer",
"expect",
"upgrade",
]);
// splitReplayHeaders separates the headers that will be sent from those the
// server drops (or, for Host and Upgrade, refuses). Headers named in a
// Connection header are hop-by-hop too.
export function splitReplayHeaders(headers: ParsedHeader[]): { sent: ParsedHeader[]; notSent: string[] } {
const connectionNamed = new Set<string>();
for (const h of headers) {
if (h.name.trim().toLowerCase() !== "connection") continue;
for (const token of h.value.split(",")) {
const name = token.trim().toLowerCase();
if (name) connectionNamed.add(name);
}
}
const sent: ParsedHeader[] = [];
const notSent: string[] = [];
for (const h of headers) {
const key = h.name.trim().toLowerCase();
if (!key) continue;
if (NOT_SENT.has(key) || connectionNamed.has(key)) {
if (!notSent.some((n) => n.toLowerCase() === key)) notSent.push(h.name.trim());
} else {
sent.push({ name: h.name.trim(), value: h.value });
}
}
return { sent, notSent };
}
+24
View File
@@ -941,6 +941,30 @@ export interface TrafficStreamPage {
total: number;
}
export interface TrafficReplayHeader {
name: string;
value: string;
}
// POST /api/traffic/replay. Host is not sent: the server takes it from url.
export interface TrafficReplayRequest {
method: string;
url: string;
headers: TrafficReplayHeader[];
body: string;
body_base64: boolean;
}
export interface TrafficReplayResult {
status: number;
headers: TrafficReplayHeader[];
raw: string;
duration_ms: number;
truncated: boolean;
// false when the host is in passthrough: the response was not recorded.
recorded: boolean;
}
// One distinct recorded host with its exchange count (target picker).
export interface TrafficHost {
host: string;