Compare commits

..
Author SHA1 Message Date
BaaRai 64b90a746a Merge pull request #292 from henefesto/docs/local-systemd-deployment-32ebb38
docs: add local systemd deployment guide and verified reboot results
2026-11-20 07:46:02 +01:00
BaaRai dbbc287cd8 Merge pull request #395 from BaaRai/security/agent-exec-isolation
security: contain agent execution (exec-uid drop, run dirs, spills, Norma temp)
2026-11-20 07:42:38 +01:00
BaaRai 72fe3c3da1 Merge pull request #394 from BaaRai/feat/replay-rewrite-toggle
feat(traffic): per-replay rewrite-rule toggle and rewrite stats (P3.3b2b1)
2026-11-20 06:02:23 +01:00
BaaRai 15ed120758 test(traffic): adversarial coverage for the replay rewrite toggle
Duplicate and lower-case nonce headers are stripped without bypassing,
the registry is single-use and rejects multi-valued fields, a forged
caller header cannot steer either mode, concurrent bypass and apply
replays do not share state, SSE streams bypass by default, a cancelled
replay leaves no nonce or marker, and the route rejects a non-boolean
apply_rules.
2026-11-20 05:48:34 +01:00
BaaRai 221ad67549 feat(traffic): per-replay rewrite-rule toggle and RewriteStats API (P3.3b2b1) 2026-11-20 05:25:35 +01:00
BaaRai dbf5cc3186 docs: generalize systemd example unit and deployment guide (#292) 2026-11-20 03:26:50 +01:00
ParallelsandWarp 32ebb38e0f docs: document local systemd deployment and verified reboot persistence
Add the ARTEX systemd unit and document PostgreSQL setup, application deployment, operations, and successful automatic startup after reboot.

Co-Authored-By: Warp <agent@warp.dev>
2026-11-17 09:20:13 -04:00
16 changed files with 1092 additions and 18 deletions
+1 -1
View File
@@ -24,7 +24,7 @@ All notable changes to this project are documented in this file. The format is b
- **Opt-in update banner**: `ARTEX_UPDATE_CHECK` makes the backend check the latest GitHub release at most once per 24 hours and the UI show a banner; off by default (#202).
- **Release provenance**: release archives carry GitHub build provenance attestations and a CycloneDX SBOM (`gh attestation verify <zip> --repo BaaRai/ARTEX`); release notes are the matching `CHANGELOG.md` section (#219, #201).
- **Capture-proxy rewrite rules, storage and API**: `GET`, `POST`, `PUT` and `DELETE /api/traffic/rewrite-rules` and `POST /api/traffic/rewrite-rules/{id}/toggle` manage rules that match a request or a response (method, host, URL, headers, body, status) and describe a rewrite (method, path and query, headers, body, status). Rules are validated on save, limited to 100 of which 50 enabled, created disabled, and stored in the new table `traffic_rewrite_rules` (migration `0005_traffic_rewrite_rules.sql`) with `matchers` and `actions` encrypted at rest. The value of an action marked `sensitive` is never returned, and an update sent without it keeps the stored one. The server compiles the enabled rules at startup and after every change and hands the snapshot to the proxy. The proxy applies them from the next entry. See `docs/traffic-rewrite-rules.md`.
- **Capture-proxy rewrite rules applied**: the capture proxy applies enabled rules to each HTTP request before it is sent to the target and to each response before it reaches the client. Each action that ran is recorded in the new table `exchange_rewrites` in the same transaction as the exchange, `traffic_get` shows a `REWRITTEN` line, and `traffic_search` never finds a value a rule wrote. SSE status and headers are rewritten at the start of the stream. **WebSocket handshakes are not rewritten**: the capture library runs its hook after the upgrade, so the status and headers cannot be changed, and frames pass unchanged. **Responses of 5 MiB or more** are forwarded unchanged and not recorded in full, because the library does not call the response hook for them. Snapshots of original requests are capped at 1024 entries and 64 MiB. Replays go through the proxy, so the rules apply to them. Archives, a per-replay rule toggle, the web UI and the proof flag come later.
- **Capture-proxy rewrite rules applied**: the capture proxy applies enabled rules to each HTTP request before it is sent to the target and to each response before it reaches the client. Each action that ran is recorded in the new table `exchange_rewrites` in the same transaction as the exchange, `traffic_get` shows a `REWRITTEN` line, and `traffic_search` never finds a value a rule wrote. SSE status and headers are rewritten at the start of the stream. **WebSocket handshakes are not rewritten**: the capture library runs its hook after the upgrade, so the status and headers cannot be changed, and frames pass unchanged. **Responses of 5 MiB or more** are forwarded unchanged and not recorded in full, because the library does not call the response hook for them. Snapshots of original requests are capped at 1024 entries and 64 MiB. Replays (`POST /api/traffic/replay`) bypass capture rewrite rules by default via an internal `X-ARTEX-Replay-Nonce` header stripped unconditionally by the proxy, and apply the rules when opted in with `apply_rules: true` (the **Apply capture rewrite rules** checkbox in the Replay panel). `GET /api/traffic` exposes the proxy's `rewrite` counters (`large_responses`, `capacity_refusals`, `tombstone_drops`, `conversions`, `sweeps`). Archives and the rule-management / traffic-list web UI come later.
- **Bulk finding status** (#298): select findings in the findings table and apply one disposition to all of them, up to 500 per call, in one transaction; each row still goes through the single-decision notification semantics and the change is mirrored onto its exploration node (#289).
- **Stream message direction filter** (#379): `GET /api/traffic/stream` and the `traffic_messages` agent tool take an optional `direction` (`c2s` or `s2c`) that lists only one side of a WebSocket or SSE conversation. `total` and `offset` apply to the filtered list, each row keeps its `seq` in the whole stream, and the `meta` row (messages dropped past the recorder limits) stays in a filtered page. Any other value is a 400 (API) or a tool error.
- **Traffic replay** (#266): replay a recorded request from the traffic detail with its method, URL, headers and stored body; replay is disabled when only the body preview is loaded.
+25
View File
@@ -0,0 +1,25 @@
[Unit]
Description=ARTEX application
After=network.target
StartLimitIntervalSec=60
StartLimitBurst=5
[Service]
Type=simple
User=artex
Group=artex
WorkingDirectory=/opt/artex
Environment=ARTEX_JWT_KEY_DIR=/opt/artex/keys
ExecStart=/opt/artex/artex -addr 127.0.0.1:8787 -proxy 127.0.0.1:8788
Restart=on-failure
RestartSec=5s
TimeoutStopSec=30s
KillSignal=SIGTERM
KillMode=control-group
UMask=0002
StandardOutput=journal
StandardError=journal
SyslogIdentifier=artex
[Install]
WantedBy=multi-user.target
+178
View File
@@ -0,0 +1,178 @@
# Local ARTEX systemd deployment
This guide describes how to deploy ARTEX locally under `systemd` using the example unit at `docs/examples/artex.service`.
## Placeholders and defaults
The example unit uses these defaults; replace them if your host uses different paths or names:
- Install directory (`<install-dir>`): `/opt/artex` (executable at `/opt/artex/artex`).
- System user and group (`<artex-user>`): `artex` (non-root).
- PostgreSQL service unit (`<pg-unit>`): distribution-specific — for example `postgresql.service` on RHEL/Fedora/Arch or `postgresql@<pg-version>-<pg-cluster>.service` on Debian/Ubuntu (where `<pg-version>/<pg-cluster>` comes from `pg_lsclusters`, such as `18/main`).
- Configuration: `<install-dir>/config.json`, ignored by Git, owner-only mode `0600`.
- Web UI: `http://127.0.0.1:8787`; recording proxy: `127.0.0.1:8788`.
- Example unit file: `docs/examples/artex.service`; installed at `/etc/systemd/system/artex.service`.
## System user and directory permissions
Create a dedicated non-login system user and group before installing the binary:
```bash
sudo useradd --system --user-group --home-dir /opt/artex --shell /usr/sbin/nologin artex
```
Create the install, runtime data, and key directories with explicit ownership and modes:
```bash
sudo install -d -o root -g artex -m 0755 /opt/artex
sudo install -d -o artex -g artex -m 0770 /opt/artex/data
sudo install -d -o artex -g artex -m 0700 /opt/artex/keys
```
Directory ownership and modes:
- `<install-dir>` (`/opt/artex`): `root:<artex-user>`, mode `0755` (executable `/opt/artex/artex` mode `0755`).
- `<install-dir>/data` (`/opt/artex/data`): `<artex-user>:<artex-user>`, mode `0770` so the service user (and, in a root-run deployment using an executor group, that group) can write task run directories while blocking world access.
- `<install-dir>/keys` (`/opt/artex/keys`): `<artex-user>:<artex-user>`, mode `0700`. Holds `jwt.key` and `secret.key` (mode `0600`) outside the browsable `data/` directory. `docs/examples/artex.service` sets `Environment=ARTEX_JWT_KEY_DIR=/opt/artex/keys` so the non-root service writes key files here rather than into `root`-owned `/opt/artex` (if `ARTEX_JWT_KEY_DIR` were left unset, ARTEX would fall back to `<install-dir>` and fail to create `jwt.key` and `secret.key` when `<install-dir>` is `root:<artex-user>` `0755`).
- `<install-dir>/config.json` (`/opt/artex/config.json`): `<artex-user>:<artex-user>`, mode `0600`.
## Database and credentials
Install PostgreSQL on the host (or use an existing reachable PostgreSQL instance) and create a dedicated non-superuser login role and database named `artex` with no role-creation or database-creation privileges.
On Debian/Ubuntu, confirm the cluster name (`<pg-version>/<pg-cluster>`) with `pg_lsclusters`, start `<pg-unit>`, and create the role and database if absent:
```bash
sudo systemctl start <pg-unit>
sudo -u postgres createuser --pwprompt artex
sudo -u postgres createdb --owner=artex artex
```
Enter a unique password at the prompt. In `<install-dir>/config.json`, set `database.host` to `127.0.0.1`, `port` to `5432`, `user` and `dbname` to `artex`, `password` to that secret, and `sslmode` to `disable` (disabling TLS is appropriate only for loopback traffic). Preserve other settings and restrict access:
```bash
sudo chown artex:artex /opt/artex/config.json
sudo chmod 0600 /opt/artex/config.json
```
`ARTEX_PG_DSN`, if set in a systemd drop-in override, takes precedence over `config.json`. Never commit configuration, credentials, encryption/signing keys, or startup logs.
## Build and install
The binary embeds the UI when built with `-tags embedui`. To rebuild the UI from a source checkout, run `npm ci` and `npm run build:static` in `web/`, create the parent directory with `mkdir -p server/webui`, and replace `server/webui/dist` with `web/out`. Build the binary with:
```bash
CGO_ENABLED=0 go build -tags embedui -trimpath -o artex ./cmd/artex
```
Install the binary into `<install-dir>`:
```bash
sudo install -o root -g artex -m 0755 artex /opt/artex/artex
```
For a running deployment, build into a separate staging path and stop the service before replacing `/opt/artex/artex` or generated assets. Avoid rerunning `install.sh` without reviewing its prompts: it regenerates `config.json` and starts a separate process.
Review `User`, `Group`, `WorkingDirectory`, `Environment=ARTEX_JWT_KEY_DIR=...`, and `ExecStart` in `docs/examples/artex.service` before installing it on your host:
```bash
systemd-analyze verify docs/examples/artex.service
sudo install -o root -g root -m 0644 docs/examples/artex.service /etc/systemd/system/artex.service
sudo systemctl daemon-reload
```
### Database dependency and environment overrides
`docs/examples/artex.service` intentionally declares only `After=network.target` and does not hardcode a distribution-specific PostgreSQL unit (such as Debian's `postgresql@<pg-version>-<pg-cluster>.service`), so it works across Linux distributions and with remote PostgreSQL servers.
When PostgreSQL runs locally on the same host, add a systemd drop-in (`sudo systemctl edit artex.service`) so ARTEX starts after and depends on your local PostgreSQL unit:
```ini
[Unit]
Requires=<pg-unit>
After=<pg-unit>
```
For example, set `<pg-unit>` to `postgresql.service` on RHEL/Fedora/Arch, or to `postgresql@<pg-version>-<pg-cluster>.service` on Debian/Ubuntu (and verify `/etc/postgresql/<pg-version>/<pg-cluster>/start.conf` is `auto`, not `disabled`). `ARTEX_JWT_KEY_DIR=/opt/artex/keys` is already set in `docs/examples/artex.service`; override it in the drop-in only if you install ARTEX under a different `<install-dir>`. Run `sudo systemctl daemon-reload` after editing the drop-in.
### Enable and start
If ARTEX is running outside `systemd`, identify its PID/executable and send that verified process `SIGTERM`. Wait for shutdown and release of ports `8787` and `8788` before enabling the service:
```bash
ss -ltnp '( sport = :8787 or sport = :8788 )'
sudo systemctl enable --now artex.service
systemctl status artex.service --no-pager
curl --fail --silent --show-error http://127.0.0.1:8787/api/health
```
The unit runs `/opt/artex/artex` directly without an extra supervisor script. It starts after `network.target`, restarts on failure after five seconds, allows 30 seconds for graceful shutdown, logs to the journal, and uses `UMask=0002` (rather than `0077`, which would block group writes in a root-run deployment where executor privilege drop sets umask `002` for the executor group; sensitive files such as `config.json`, `keys/jwt.key`, and `keys/secret.key` are protected explicitly with `0600`/`0700` permissions). Five starts within 60 seconds trigger the start limit.
### Agent isolation
Because `docs/examples/artex.service` runs the server as the non-root `<artex-user>` (`User=artex`), the `-exec-uid`/`-exec-gid` privilege drop cannot be applied: `ResolveExecPrivilege` disables the drop when the server is not root (and errors if those flags are passed explicitly). The example unit deliberately omits `-exec-uid`/`-exec-gid`, so agents and spawned tools run under the `<artex-user>` service identity rather than a separate executor account. The root-owned run-directory layout and executor-group containment exist only when the server runs as root with explicit `-exec-uid`/`-exec-gid`; if you require unprivileged-executor containment, use the Docker deployment, which is the supported path for that isolation model.
## Web setup
On a new database, visit `http://127.0.0.1:8787/setup`, provide the one-time setup token from the private startup log or journal (`journalctl -u artex.service`), and choose a unique password of at least 12 characters. The username is `ARTEX`. Store the password in your password manager rather than in a plaintext file on disk.
## Reboot persistence verification (example)
Pre-reboot checks confirm persistent ARTEX enablement, active PostgreSQL and ARTEX services, HTTP 200 health, and initialized authentication, but do not by themselves prove survival across a reboot.
To verify reboot persistence on a host, record the current boot ID and timestamp in an ignored runtime file such as `<install-dir>/data/reboot-verification-baseline.json` before rebooting, and do not overwrite that baseline until the test is complete:
```bash
sudo -u artex python3 -c 'import json, time; from pathlib import Path; Path("/opt/artex/data/reboot-verification-baseline.json").write_text(json.dumps({"boot_id": Path("/proc/sys/kernel/random/boot_id").read_text().strip(), "ts": time.time()}))'
```
Save other work before rebooting; this disconnects local and remote sessions:
```bash
sudo systemctl reboot
```
After reconnecting, do not manually start ARTEX before checking automatic startup. Compare the current boot ID with the saved baseline and check the services:
```bash
cd /opt/artex
python3 -c 'import json; from pathlib import Path; old=json.loads(Path("data/reboot-verification-baseline.json").read_text())["boot_id"]; new=Path("/proc/sys/kernel/random/boot_id").read_text().strip(); print("PASS: a reboot occurred" if old != new else "FAIL: no reboot occurred"); raise SystemExit(0 if old != new else 1)'
systemctl is-enabled artex.service
systemctl is-active <pg-unit> artex.service
curl --fail --silent --show-error http://127.0.0.1:8787/api/health
python3 -c 'import json, urllib.request; r=json.load(urllib.request.urlopen("http://127.0.0.1:8787/api/auth/status", timeout=5)); print("Authentication initialized:", r.get("initialized")); raise SystemExit(0 if r.get("initialized") else 1)'
systemctl show artex.service --no-pager -p ActiveState -p SubState -p MainPID -p ActiveEnterTimestamp
```
Success requires a changed boot ID, enabled ARTEX, both services active without manual startup, HTTP 200, and authentication still initialized. A new PID alone is not proof of reboot. If the system has just booted, allow startup time and repeat health checks. Refresh the baseline before running a future independent reboot test.
### Example reboot verification record
Example of a completed post-reboot verification (recorded from the original Debian/PostgreSQL 18 reference deployment on 2026-11-17, without manually starting or restarting either service):
- Current boot ID differed from the saved pre-reboot baseline, confirming an actual reboot.
- `artex.service` remained enabled and was `active (running)` under its non-root service user with `NRestarts=0`.
- The local PostgreSQL cluster service was `active`.
- `/api/health` returned HTTP 200.
- `/api/auth/status` confirmed initialized authentication was preserved.
Observed timestamps, PIDs, and restart counts are host-specific examples, not fixed configuration values.
## Operations and troubleshooting
```bash
sudo systemctl restart artex.service
sudo systemctl stop artex.service
sudo systemctl start artex.service
journalctl -b -u artex.service --no-pager
journalctl -u artex.service -f
```
Treat journal output as potentially sensitive; never share unredacted setup tokens or credentials. If startup fails, check PostgreSQL status, database settings and permissions on `<install-dir>/config.json`, `<install-dir>/data`, and `<install-dir>/keys`, occupied ports (`8787`/`8788`), and the journal. After a PostgreSQL dependency failure, restart ARTEX explicitly once the database is healthy. To recover from the start limit:
```bash
sudo systemctl reset-failed artex.service
sudo systemctl start artex.service
```
Back up PostgreSQL, `<install-dir>/config.json`, `<install-dir>/data`, and `<install-dir>/keys` (`jwt.key` and `secret.key`) securely before upgrades. The application stays loopback-only; remote access requires a separately reviewed tunnel or reverse proxy.
+18 -4
View File
@@ -4,8 +4,9 @@ A rewrite rule matches a request or a response that goes through the capture pro
how to change it: the method, the path and query, headers, the body, the status code.
**Status: the capture proxy applies enabled rules to HTTP traffic (request and response phases),
and records what it did.** Not yet in this change: archive format, a per-replay rule toggle, the web UI,
and the proof flag. See [Application in the capture proxy](#application-in-the-capture-proxy) for
records what it did, tags bound evidence snapshots, supports per-replay rule opt-in, and exposes
rewrite counters on `GET /api/traffic`.** Not yet in this change: archive format and the traffic-list /
rule-management web UI. See [Application in the capture proxy](#application-in-the-capture-proxy) for
what is covered and what is not.
Rules are a convenience for the operator, not a security control: a rule that fails leaves the
@@ -207,5 +208,18 @@ the slot may be converted into a tombstone if a new reservation needs the room.
### Replay
Replay goes through the recording proxy like any other traffic, so the enabled rules apply to a
replay and the resulting exchange is traced like a captured one. There is no per-replay toggle yet.
`POST /api/traffic/replay` sends the edited request through the recording proxy so the exchange is
captured in Traffic, but **bypasses capture rewrite rules by default** (`apply_rules: false`). To
run enabled capture rules against a replay and trace them like captured traffic, opt in with
`"apply_rules": true` (or check **Apply capture rewrite rules** in the Replay panel).
When `apply_rules` is false, `traffic.Replay` registers a single-use random 32-byte hex nonce in an
in-memory registry, attaches the internal header `X-ARTEX-Replay-Nonce: <nonce>`, and removes the
nonce when the call returns. The capture proxy strips `X-ARTEX-Replay-Nonce` unconditionally in the
request hook so the target never sees it; only a value matching a registered nonce bypasses both
request and response rule phases, while any forged header is stripped and ignored.
### Stats
`GET /api/traffic` includes a `rewrite` object exposing the proxy's `RewriteStats` counters since
start: `large_responses`, `capacity_refusals`, `tombstone_drops`, `conversions`, and `sweeps`.
+10 -2
View File
@@ -3535,7 +3535,7 @@ func taskActivityDetail(t *Task, seq int64) (string, error) {
func (s *Server) getTraffic(w http.ResponseWriter, r *http.Request) {
tr := s.m.Traffic()
if tr == nil {
writeJSON(w, 200, map[string]any{"enabled": false, "exchanges": []any{}})
writeJSON(w, 200, map[string]any{"enabled": false, "exchanges": []any{}, "rewrite": traffic.RewriteStats{}})
return
}
q := r.URL.Query()
@@ -3562,6 +3562,7 @@ func (s *Server) getTraffic(w http.ResponseWriter, r *http.Request) {
"page": page,
"size": size,
"exchanges": trafficDTOs(ex),
"rewrite": tr.RewriteStats(),
})
}
@@ -3740,6 +3741,7 @@ func (s *Server) replayTraffic(w http.ResponseWriter, r *http.Request) {
Headers []traffic.ReplayHeader `json:"headers"`
Body string `json:"body"`
BodyBase64 bool `json:"body_base64"`
ApplyRules bool `json:"apply_rules"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
var tooLarge *http.MaxBytesError
@@ -3761,7 +3763,13 @@ func (s *Server) replayTraffic(w http.ResponseWriter, r *http.Request) {
}
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
defer cancel()
res, err := tr.Replay(ctx, traffic.ReplayRequest{Method: body.Method, URL: body.URL, Headers: body.Headers, Body: payload})
res, err := tr.Replay(ctx, traffic.ReplayRequest{
Method: body.Method,
URL: body.URL,
Headers: body.Headers,
Body: payload,
ApplyRules: body.ApplyRules,
})
switch {
case errors.Is(err, traffic.ErrReplayTooLarge):
writeErr(w, 413, err.Error())
+21
View File
@@ -0,0 +1,21 @@
package server
import (
"net/http"
"testing"
)
// apply_rules is a boolean in the API contract: a string or a number is a 400,
// not a silent false or true.
func TestAdversarialReplayRouteRejectsNonBoolApplyRules(t *testing.T) {
_, replay := replayServer(t, true, false)
for _, body := range []string{
`{"method":"GET","url":"http://example.com/","apply_rules":"yes"}`,
`{"method":"GET","url":"http://example.com/","apply_rules":1}`,
} {
w := replay(body)
if w.Code != http.StatusBadRequest {
t.Fatalf("%s = %d %s, want 400", body, w.Code, w.Body)
}
}
}
+78
View File
@@ -14,6 +14,7 @@ import (
"time"
"github.com/Autumn-27/artex/traffic"
"github.com/Autumn-27/artex/traffic/rewrite"
)
// replayServer wires a Server like ingestServer. With traffic off,
@@ -161,3 +162,80 @@ func TestTrafficReplayRouteEndToEnd(t *testing.T) {
time.Sleep(20 * time.Millisecond)
}
}
func TestGetTrafficExposesRewriteStats(t *testing.T) {
// Traffic disabled: rewrite object still present with zero counters.
sOff := &Server{m: &Manager{}}
wOff := httptest.NewRecorder()
sOff.getTraffic(wOff, httptest.NewRequest(http.MethodGet, "/api/traffic", nil))
if wOff.Code != http.StatusOK {
t.Fatalf("getTraffic(off) status = %d", wOff.Code)
}
var bodyOff map[string]any
if err := json.Unmarshal(wOff.Body.Bytes(), &bodyOff); err != nil {
t.Fatal(err)
}
rwOff, ok := bodyOff["rewrite"].(map[string]any)
if !ok {
t.Fatalf("getTraffic(off) missing rewrite object: %s", wOff.Body.String())
}
for _, k := range []string{"large_responses", "capacity_refusals", "tombstone_drops", "conversions", "sweeps"} {
if _, exists := rwOff[k]; !exists {
t.Fatalf("getTraffic(off) rewrite object missing key %q: %v", k, rwOff)
}
}
// Traffic store open: rewrite counters mirror tr.RewriteStats().
tr, err := traffic.Open(t.TempDir(), ":0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { tr.Close() })
sOn := &Server{m: &Manager{traffic: tr}}
wOn := httptest.NewRecorder()
sOn.getTraffic(wOn, httptest.NewRequest(http.MethodGet, "/api/traffic", nil))
if wOn.Code != http.StatusOK {
t.Fatalf("getTraffic(on) status = %d", wOn.Code)
}
var bodyOn struct {
Rewrite traffic.RewriteStats `json:"rewrite"`
}
if err := json.Unmarshal(wOn.Body.Bytes(), &bodyOn); err != nil {
t.Fatal(err)
}
if bodyOn.Rewrite != tr.RewriteStats() {
t.Fatalf("getTraffic rewrite = %+v, want %+v", bodyOn.Rewrite, tr.RewriteStats())
}
}
func TestTrafficReplayRouteApplyRulesToggle(t *testing.T) {
s, replay := replayServer(t, true, true)
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Seen-Rule", r.Header.Get("X-Rule-Tag"))
_, _ = io.WriteString(w, "ok")
}))
t.Cleanup(target.Close)
set := rewrite.Compile([]rewrite.Rule{{
ID: 7, Name: "tag", Enabled: true, Phase: rewrite.PhaseRequest,
Actions: []rewrite.Action{{Type: rewrite.ActionSetHeader, Name: "X-Rule-Tag", Value: "on"}},
}}, rewrite.Options{})
if len(set.LoadErrors()) != 0 {
t.Fatalf("load errors: %v", set.LoadErrors())
}
s.m.Traffic().SetRewriteRules(set)
// Default (apply_rules absent -> false): bypasses rule.
reqOff, _ := json.Marshal(map[string]any{"method": "GET", "url": target.URL + "/toggle-off"})
wOff := replay(string(reqOff))
if wOff.Code != 200 || strings.Contains(wOff.Body.String(), `"X-Seen-Rule","value":"on"`) {
t.Fatalf("default replay applied rewrite rule: %d %s", wOff.Code, wOff.Body.String())
}
// Opt-in (apply_rules: true): applies rule.
reqOn, _ := json.Marshal(map[string]any{"method": "GET", "url": target.URL + "/toggle-on", "apply_rules": true})
wOn := replay(string(reqOn))
if wOn.Code != 200 || !strings.Contains(wOn.Body.String(), `"value":"on"`) {
t.Fatalf("apply_rules=true did not apply rewrite rule: %d %s", wOn.Code, wOn.Body.String())
}
}
+71 -4
View File
@@ -3,8 +3,10 @@ package traffic
import (
"bytes"
"context"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"errors"
"fmt"
"io"
@@ -13,6 +15,7 @@ import (
"os"
"sort"
"strings"
"sync"
"time"
)
@@ -28,8 +31,61 @@ const (
maxReplayResponse = 2 * 1024 * 1024
// maxReplayHeaders bounds the number of request header lines.
maxReplayHeaders = 200
// replayNonceHeader is the internal header Replay attaches when ApplyRules
// is false (the default). The capture proxy strips it unconditionally
// before forwarding and bypasses rewrite rules only when the value matches
// a live registered nonce.
replayNonceHeader = "X-ARTEX-Replay-Nonce"
)
// replayNonces holds live single-use nonces for in-flight Replay calls that
// bypass capture rewrite rules (ApplyRules == false).
var replayNonces struct {
mu sync.Mutex
live map[string]struct{}
}
func registerReplayNonce() (string, error) {
var raw [32]byte
if _, err := rand.Read(raw[:]); err != nil {
return "", fmt.Errorf("replay nonce: %w", err)
}
nonce := hex.EncodeToString(raw[:])
replayNonces.mu.Lock()
if replayNonces.live == nil {
replayNonces.live = map[string]struct{}{}
}
replayNonces.live[nonce] = struct{}{}
replayNonces.mu.Unlock()
return nonce, nil
}
func deleteReplayNonce(nonce string) {
replayNonces.mu.Lock()
delete(replayNonces.live, nonce)
replayNonces.mu.Unlock()
}
func consumeReplayNonce(nonce string) bool {
if nonce == "" {
return false
}
replayNonces.mu.Lock()
_, ok := replayNonces.live[nonce]
if ok {
delete(replayNonces.live, nonce)
}
replayNonces.mu.Unlock()
return ok
}
func replayNonceCount() int {
replayNonces.mu.Lock()
n := len(replayNonces.live)
replayNonces.mu.Unlock()
return n
}
// ErrReplayInvalid marks a replay request rejected before anything is sent.
var ErrReplayInvalid = errors.New("invalid replay request")
@@ -45,10 +101,11 @@ type ReplayHeader struct {
// ReplayRequest is the edited request to send.
type ReplayRequest struct {
Method string
URL string
Headers []ReplayHeader
Body []byte
Method string
URL string
Headers []ReplayHeader
Body []byte
ApplyRules bool
}
// ReplayResult is the response as received by the client side of the proxy.
@@ -78,6 +135,7 @@ var replayDropped = map[string]bool{
"Te": true,
"Trailer": true,
"Expect": true,
http.CanonicalHeaderKey(replayNonceHeader): true,
}
// replayRecorded reports whether the proxy records a request to scheme://host:
@@ -217,6 +275,15 @@ func (t *Traffic) Replay(ctx context.Context, req ReplayRequest) (ReplayResult,
httpReq.Host = host
}
if !req.ApplyRules && t.replayRecorded(target.Scheme, target.Host) {
nonce, err := registerReplayNonce()
if err != nil {
return ReplayResult{}, err
}
defer deleteReplayNonce(nonce)
httpReq.Header.Set(replayNonceHeader, nonce)
}
start := time.Now()
resp, err := client.Do(httpReq)
if err != nil {
+334
View File
@@ -0,0 +1,334 @@
package traffic
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
"github.com/Autumn-27/artex/traffic/rewrite"
)
func advHeaderValue(hs []ReplayHeader, name string) string {
for _, h := range hs {
if strings.EqualFold(h.Name, name) {
return h.Value
}
}
return ""
}
// The internal header is stripped whatever its casing and however many
// occurrences are sent, and it never bypasses rules without a live nonce.
func TestAdversarialReplayNonceHeaderDuplicatesStripped(t *testing.T) {
var mu sync.Mutex
var seen []string
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
seen = append([]string{}, r.Header.Values(replayNonceHeader)...)
mu.Unlock()
w.Header().Set("X-Seen-Rule", r.Header.Get("X-Rule"))
_, _ = io.WriteString(w, "ok")
}))
t.Cleanup(target.Close)
tr, _, proxyURL := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule", Value: "applied"},
}},
rewrite.Rule{ID: 2, Phase: rewrite.PhaseResponse, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Resp", Value: "applied"},
}},
))
h := http.Header{}
h[replayNonceHeader] = []string{"aaaa", "bbbb"}
h["x-artex-replay-nonce"] = []string{"cccc"}
resp, _ := sendThrough(t, tr, proxyURL, http.MethodGet, target.URL+"/dup", "", h)
mu.Lock()
got := append([]string{}, seen...)
mu.Unlock()
if len(got) != 0 {
t.Fatalf("target saw nonce header values %v", got)
}
if resp.Header.Get("X-Rule-Resp") != "applied" {
t.Fatalf("duplicate forged nonce bypassed the response rule: %v", resp.Header)
}
if resp.Header.Get("X-Seen-Rule") != "applied" {
t.Fatalf("duplicate forged nonce bypassed the request rule: %v", resp.Header)
}
id := waitExchangeIDBySuffix(t, tr, "/dup")
reqText, _, err := tr.Get(id)
if err != nil {
t.Fatal(err)
}
if strings.Contains(strings.ToLower(reqText), strings.ToLower(replayNonceHeader)) {
t.Fatalf("recorded request kept a nonce header:\n%s", reqText)
}
}
// A nonce is single use, and takeReplayNonce refuses to return a value when
// more than one occurrence is present.
func TestAdversarialReplayNonceRegistrySingleUse(t *testing.T) {
n, err := registerReplayNonce()
if err != nil {
t.Fatal(err)
}
if len(n) != 64 {
t.Fatalf("nonce length = %d, want 64 hex chars", len(n))
}
if !consumeReplayNonce(n) {
t.Fatal("first consume did not match")
}
if consumeReplayNonce(n) {
t.Fatal("second consume matched: nonce is not single use")
}
if consumeReplayNonce("") {
t.Fatal("empty nonce matched")
}
h := http.Header{}
h.Set(replayNonceHeader, n)
if got, present := takeReplayNonce(h); !present || got != n || len(h) != 0 {
t.Fatalf("single occurrence = (%q, %v, %d), want (%q, true, 0)", got, present, len(h), n)
}
h2 := http.Header{replayNonceHeader: {"a", "b"}, "x-artex-replay-nonce": {"c"}}
if got, present := takeReplayNonce(h2); !present || got != "" || len(h2) != 0 {
t.Fatalf("duplicates = (%q, %v, %d), want (\"\", true, 0)", got, present, len(h2))
}
h3 := http.Header{replayNonceHeader: {}}
if got, present := takeReplayNonce(h3); !present || got != "" || len(h3) != 0 {
t.Fatalf("empty values = (%q, %v, %d), want (\"\", true, 0)", got, present, len(h3))
}
}
// A caller-supplied nonce header is dropped in both modes and never wins over
// the real nonce: default still bypasses, apply_rules still applies.
func TestAdversarialReplayForgedNonceHeaderCannotSteerReplay(t *testing.T) {
var mu sync.Mutex
var seen []string
var rule string
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
rule = r.Header.Get("X-Rule")
seen = append([]string{}, r.Header.Values(replayNonceHeader)...)
mu.Unlock()
w.Header().Set("X-Seen-Rule", r.Header.Get("X-Rule"))
_, _ = io.WriteString(w, "ok")
}))
t.Cleanup(target.Close)
tr, _, _ := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule", Value: "applied"},
}},
))
forged := ReplayHeader{Name: replayNonceHeader, Value: strings.Repeat("ab", 32)}
res, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodGet,
URL: target.URL + "/forged-bypass",
Headers: []ReplayHeader{forged},
})
if err != nil {
t.Fatalf("default replay: %v", err)
}
mu.Lock()
if rule != "" || len(seen) != 0 {
t.Fatalf("default replay: target saw X-Rule=%q nonce=%v", rule, seen)
}
mu.Unlock()
if advHeaderValue(res.Headers, "X-Seen-Rule") != "" {
t.Fatalf("default replay with forged header did not bypass: %v", res.Headers)
}
res2, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodGet,
URL: target.URL + "/forged-apply",
Headers: []ReplayHeader{forged},
ApplyRules: true,
})
if err != nil {
t.Fatalf("apply replay: %v", err)
}
mu.Lock()
if rule != "applied" || len(seen) != 0 {
t.Fatalf("apply replay: target saw X-Rule=%q nonce=%v", rule, seen)
}
mu.Unlock()
if advHeaderValue(res2.Headers, "X-Seen-Rule") != "applied" {
t.Fatalf("apply replay did not apply the rule: %v", res2.Headers)
}
if replayNonceCount() != 0 {
t.Fatalf("nonce registry not empty: %d", replayNonceCount())
}
}
// A bypassing replay and an applying replay to the same target at the same
// time must not share state, and a normal request after a bypass must still be
// rewritten.
func TestAdversarialReplayConcurrentBypassAndApply(t *testing.T) {
var mu sync.Mutex
seen := map[string]string{}
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
seen[r.URL.Path] = r.Header.Get("X-Rule")
mu.Unlock()
w.Header().Set("X-Seen-Rule", r.Header.Get("X-Rule"))
_, _ = io.WriteString(w, "ok")
}))
t.Cleanup(target.Close)
tr, _, pxy := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule", Value: "applied"},
}},
))
var wg sync.WaitGroup
var offRule, onRule string
wg.Add(2)
go func() {
defer wg.Done()
res, err := tr.Replay(context.Background(), ReplayRequest{Method: http.MethodGet, URL: target.URL + "/conc-off"})
if err != nil {
t.Errorf("bypass replay: %v", err)
return
}
offRule = advHeaderValue(res.Headers, "X-Seen-Rule")
}()
go func() {
defer wg.Done()
res, err := tr.Replay(context.Background(), ReplayRequest{Method: http.MethodGet, URL: target.URL + "/conc-on", ApplyRules: true})
if err != nil {
t.Errorf("apply replay: %v", err)
return
}
onRule = advHeaderValue(res.Headers, "X-Seen-Rule")
}()
wg.Wait()
if offRule != "" {
t.Fatalf("bypass replay saw X-Rule=%q", offRule)
}
if onRule != "applied" {
t.Fatalf("apply replay saw X-Rule=%q", onRule)
}
if replayNonceCount() != 0 {
t.Fatalf("nonce registry not empty after concurrent replays: %d", replayNonceCount())
}
// A plain request after the bypass is still rewritten: the marker is per flow.
resp, _ := sendThrough(t, tr, pxy, http.MethodGet, target.URL+"/after", "", nil)
if resp.Header.Get("X-Seen-Rule") != "applied" {
t.Fatalf("normal request after a bypass was not rewritten: %v", resp.Header)
}
mu.Lock()
defer mu.Unlock()
if seen["/conc-off"] != "" || seen["/conc-on"] != "applied" || seen["/after"] != "applied" {
t.Fatalf("target saw %v", seen)
}
}
// An SSE response is bypassed by default and rewritten when opted in: the
// stream phase is the path the plain response test cannot reach.
func TestAdversarialReplayBypassCoversSSE(t *testing.T) {
tr, _, _ := startProxy(t)
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(200)
fl := w.(http.Flusher)
for _, ev := range []string{"data: one\n\n", "data: two\n\n"} {
_, _ = io.WriteString(w, ev)
fl.Flush()
time.Sleep(10 * time.Millisecond)
}
}))
t.Cleanup(srv.Close)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseResponse, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Resp", Value: "applied"},
}},
))
res, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodGet,
URL: localhostURL(srv.URL) + "/sse-off",
})
if err != nil {
t.Fatalf("bypass SSE replay: %v", err)
}
if got := advHeaderValue(res.Headers, "X-Rule-Resp"); got != "" {
t.Fatalf("bypass SSE replay applied header rule: %q", got)
}
if !strings.Contains(res.Raw, "data: two") {
t.Fatalf("bypass SSE body changed: %s", res.Raw)
}
idOff := waitExchangeIDBySuffix(t, tr, "/sse-off")
if rewritten, ids, err := tr.RewriteInfo(idOff); err != nil || rewritten || len(ids) != 0 {
t.Fatalf("bypass SSE RewriteInfo = (%v, %v, %v)", rewritten, ids, err)
}
res2, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodGet,
URL: localhostURL(srv.URL) + "/sse-on",
ApplyRules: true,
})
if err != nil {
t.Fatalf("apply SSE replay: %v", err)
}
if got := advHeaderValue(res2.Headers, "X-Rule-Resp"); got != "applied" {
t.Fatalf("apply SSE replay header rule = %q, want applied", got)
}
idOn := waitExchangeIDBySuffix(t, tr, "/sse-on")
if rows := traceRows(t, tr, idOn); len(rows) == 0 {
t.Fatal("apply SSE replay recorded no rewrite trace")
}
}
// A cancelled replay leaves no live nonce and no permanent bypass marker.
func TestAdversarialReplayCancelCleansUp(t *testing.T) {
block := make(chan struct{})
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
<-block
_, _ = io.WriteString(w, "late")
}))
t.Cleanup(target.Close)
tr, s, _ := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule", Value: "applied"},
}},
))
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if _, err := tr.Replay(ctx, ReplayRequest{Method: http.MethodGet, URL: target.URL + "/cancel"}); err == nil {
t.Fatal("cancelled replay returned no error")
}
close(block)
if replayNonceCount() != 0 {
t.Fatalf("nonce registry leaked after cancel: %d", replayNonceCount())
}
waitFor(t, "bypass marker cleanup", func() bool {
empty := true
s.bypass.Range(func(_, _ any) bool {
empty = false
return false
})
return empty
})
}
+271
View File
@@ -0,0 +1,271 @@
package traffic
import (
"context"
"io"
"net/http"
"net/http/httptest"
"slices"
"strings"
"sync"
"testing"
"time"
"github.com/Autumn-27/artex/traffic/rewrite"
mproxy "github.com/lqqyt2423/go-mitmproxy/proxy"
)
func hasNonceHeader(h http.Header) bool {
for k := range h {
if strings.EqualFold(k, replayNonceHeader) {
return true
}
}
return false
}
func waitExchangeIDBySuffix(t *testing.T, tr *Traffic, suffix string) string {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for {
var id string
err := tr.DB().QueryRow(`SELECT id FROM exchanges WHERE url LIKE ?`, "%"+suffix).Scan(&id)
if err == nil {
return id
}
if time.Now().After(deadline) {
t.Fatalf("exchange %s not recorded: %v", suffix, err)
}
time.Sleep(15 * time.Millisecond)
}
}
func TestReplayBypassesRewriteRulesByDefault(t *testing.T) {
var mu sync.Mutex
var gotMethod, gotBody, gotOrig, gotRuleReq string
var sawNonce bool
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
mu.Lock()
gotMethod = r.Method
gotBody = string(b)
gotOrig = r.Header.Get("X-Orig")
gotRuleReq = r.Header.Get("X-Rule-Req")
sawNonce = hasNonceHeader(r.Header)
mu.Unlock()
w.Header().Set("Content-Type", "text/plain")
_, _ = io.WriteString(w, "target-resp")
}))
t.Cleanup(target.Close)
tr, s, _ := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Req", Value: "applied"},
{Type: rewrite.ActionSetBody, Value: "rewritten-req"},
}},
rewrite.Rule{ID: 2, Phase: rewrite.PhaseResponse, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Resp", Value: "applied"},
{Type: rewrite.ActionReplaceBody, Regex: "target-resp", Value: "rewritten-resp"},
}},
))
// ApplyRules omitted (false by default) -> bypass both phases.
res, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodPost,
URL: target.URL + "/default-bypass",
Headers: []ReplayHeader{{Name: "X-Orig", Value: "1"}},
Body: []byte("orig-body"),
})
if err != nil {
t.Fatalf("Replay: %v", err)
}
if replayNonceCount() != 0 {
t.Fatalf("nonce registry not empty after Replay: %d", replayNonceCount())
}
mu.Lock()
if gotMethod != http.MethodPost || gotBody != "orig-body" || gotOrig != "1" || gotRuleReq != "" || sawNonce {
t.Fatalf("target received method=%q body=%q X-Orig=%q X-Rule-Req=%q sawNonce=%v", gotMethod, gotBody, gotOrig, gotRuleReq, sawNonce)
}
mu.Unlock()
if res.Status != 200 || !res.Recorded || strings.Contains(res.Raw, "X-Rule-Resp") || !strings.HasSuffix(res.Raw, "target-resp") {
t.Fatalf("client received modified response: %+v", res)
}
id := waitExchangeIDBySuffix(t, tr, "/default-bypass")
if rewritten, ids, err := tr.RewriteInfo(id); err != nil || rewritten || len(ids) != 0 {
t.Fatalf("RewriteInfo = (%v, %v, %v), want (false, nil, nil)", rewritten, ids, err)
}
var traceCount int
if err := tr.DB().QueryRow(`SELECT COUNT(*) FROM exchange_rewrites WHERE exchange_id=?`, id).Scan(&traceCount); err != nil {
t.Fatal(err)
}
if traceCount != 0 {
t.Fatalf("exchange_rewrites rows = %d, want 0", traceCount)
}
reqText, _, err := tr.Get(id)
if err != nil {
t.Fatal(err)
}
if strings.Contains(strings.ToLower(reqText), strings.ToLower(replayNonceHeader)) {
t.Fatalf("recorded request leaked internal nonce header:\n%s", reqText)
}
// Response-only rules must also be bypassed and the nonce header stripped.
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 3, Phase: rewrite.PhaseResponse, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Resp", Value: "applied"},
}},
))
res2, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodGet,
URL: target.URL + "/resp-only-bypass",
})
if err != nil {
t.Fatalf("Replay response-only: %v", err)
}
mu.Lock()
if sawNonce {
t.Fatal("target saw nonce header when only response rules were active")
}
mu.Unlock()
if strings.Contains(res2.Raw, "X-Rule-Resp") {
t.Fatalf("response-only rule was not bypassed: %s", res2.Raw)
}
id2 := waitExchangeIDBySuffix(t, tr, "/resp-only-bypass")
if rewritten, _, err := tr.RewriteInfo(id2); err != nil || rewritten {
t.Fatalf("response-only bypass RewriteInfo = %v %v", rewritten, err)
}
// RequestError clears the bypass marker.
f := withID(&mproxy.Flow{}, 99)
s.bypass.Store(keyOf(f), struct{}{})
s.RequestError(f, io.EOF)
if _, ok := s.bypass.Load(keyOf(f)); ok {
t.Fatal("RequestError did not clear bypass marker")
}
// Failed Replay (unreachable target) still leaves nonce registry empty.
_, _ = tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodGet,
URL: "http://127.0.0.1:1/unreachable",
})
if replayNonceCount() != 0 {
t.Fatalf("nonce registry leaked after failed Replay: %d", replayNonceCount())
}
}
func TestReplayApplyRulesTrue(t *testing.T) {
var mu sync.Mutex
var gotBody, gotRuleReq string
var sawNonce bool
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
mu.Lock()
gotBody = string(b)
gotRuleReq = r.Header.Get("X-Rule-Req")
sawNonce = hasNonceHeader(r.Header)
mu.Unlock()
w.Header().Set("Content-Type", "text/plain")
_, _ = io.WriteString(w, "target-resp")
}))
t.Cleanup(target.Close)
tr, _, _ := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Req", Value: "applied"},
{Type: rewrite.ActionSetBody, Value: "rewritten-req"},
}},
rewrite.Rule{ID: 2, Phase: rewrite.PhaseResponse, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Resp", Value: "applied"},
{Type: rewrite.ActionReplaceBody, Regex: "target-resp", Value: "rewritten-resp"},
}},
))
res, err := tr.Replay(context.Background(), ReplayRequest{
Method: http.MethodPost,
URL: target.URL + "/apply-rules",
Body: []byte("orig-body"),
ApplyRules: true,
})
if err != nil {
t.Fatalf("Replay(ApplyRules=true): %v", err)
}
if replayNonceCount() != 0 {
t.Fatalf("nonce registry not empty after Replay(ApplyRules=true): %d", replayNonceCount())
}
mu.Lock()
if gotBody != "rewritten-req" || gotRuleReq != "applied" || sawNonce {
t.Fatalf("target received body=%q X-Rule-Req=%q sawNonce=%v", gotBody, gotRuleReq, sawNonce)
}
mu.Unlock()
if !strings.Contains(res.Raw, "X-Rule-Resp: applied") || !strings.HasSuffix(res.Raw, "rewritten-resp") {
t.Fatalf("response rule did not apply on Replay(ApplyRules=true): %s", res.Raw)
}
id := waitExchangeIDBySuffix(t, tr, "/apply-rules")
rewritten, ids, err := tr.RewriteInfo(id)
if err != nil || !rewritten || !slices.Equal(ids, []int64{1, 2}) {
t.Fatalf("RewriteInfo = (%v, %v, %v), want (true, [1 2], nil)", rewritten, ids, err)
}
if rows := traceRows(t, tr, id); len(rows) == 0 {
t.Fatal("expected exchange_rewrites trace rows when ApplyRules=true")
}
}
func TestForgedReplayNonceHeaderStrippedAndIgnored(t *testing.T) {
var mu sync.Mutex
var gotRuleReq string
var sawNonce bool
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
gotRuleReq = r.Header.Get("X-Rule-Req")
sawNonce = hasNonceHeader(r.Header)
mu.Unlock()
w.Header().Set("Content-Type", "text/plain")
_, _ = io.WriteString(w, "target-resp")
}))
t.Cleanup(target.Close)
tr, _, proxyURL := startProxy(t)
tr.SetRewriteRules(rules(t, nil, nil,
rewrite.Rule{ID: 1, Phase: rewrite.PhaseRequest, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Req", Value: "applied"},
}},
rewrite.Rule{ID: 2, Phase: rewrite.PhaseResponse, Actions: []rewrite.Action{
{Type: rewrite.ActionSetHeader, Name: "X-Rule-Resp", Value: "applied"},
}},
))
resp, _ := sendThrough(t, tr, proxyURL, http.MethodGet, target.URL+"/forged", "", http.Header{
replayNonceHeader: {"0000000000000000000000000000000000000000000000000000000000000000"},
})
mu.Lock()
if sawNonce {
t.Fatal("forged replay nonce header reached target")
}
if gotRuleReq != "applied" {
t.Fatalf("forged nonce bypassed request rule: X-Rule-Req=%q", gotRuleReq)
}
mu.Unlock()
if resp.Header.Get("X-Rule-Resp") != "applied" {
t.Fatalf("forged nonce bypassed response rule: X-Rule-Resp=%q", resp.Header.Get("X-Rule-Resp"))
}
id := waitExchangeIDBySuffix(t, tr, "/forged")
rewritten, ids, err := tr.RewriteInfo(id)
if err != nil || !rewritten || !slices.Equal(ids, []int64{1, 2}) {
t.Fatalf("forged nonce RewriteInfo = (%v, %v, %v), want (true, [1 2], nil)", rewritten, ids, err)
}
reqText, _, err := tr.Get(id)
if err != nil {
t.Fatal(err)
}
if strings.Contains(strings.ToLower(reqText), strings.ToLower(replayNonceHeader)) {
t.Fatalf("recorded request kept forged nonce header:\n%s", reqText)
}
}
+46
View File
@@ -6,6 +6,7 @@ import (
"log"
"net/http"
"net/url"
"strings"
"sync"
"sync/atomic"
"time"
@@ -420,6 +421,32 @@ func engineRequest(r *mproxy.Request) rewrite.Request {
return rewrite.Request{Method: r.Method, URL: r.URL, Header: r.Header, Body: r.Body}
}
// takeReplayNonce removes every occurrence of X-ARTEX-Replay-Nonce from h
// unconditionally and returns its value only when exactly one header value was
// present.
func takeReplayNonce(h http.Header) (nonce string, present bool) {
if h == nil {
return "", false
}
count := 0
for k, vs := range h {
if strings.EqualFold(k, replayNonceHeader) {
present = true
for _, v := range vs {
count++
if count == 1 {
nonce = strings.TrimSpace(v)
}
}
delete(h, k)
}
}
if count != 1 {
return "", present
}
return nonce, present
}
// Request runs the request phase. The result goes on the flow only once the
// request's snapshot slot is reserved; without one the request leaves as the
// client sent it. Rules fail open: a panic leaves the request unchanged.
@@ -427,6 +454,12 @@ func (s *sink) Request(f *mproxy.Flow) {
if s.t == nil || f.Request == nil || f.Request.URL == nil {
return
}
// Strip the internal replay bypass header unconditionally before forwarding
// or running rules; honour it only when its value matches a registered nonce.
if nonce, present := takeReplayNonce(f.Request.Header); present && consumeReplayNonce(nonce) {
s.bypass.Store(keyOf(f), struct{}{})
return
}
set := s.t.rewriteRules.Load()
if set.Len(rewrite.PhaseRequest) == 0 {
return
@@ -459,6 +492,10 @@ func (s *sink) Response(f *mproxy.Flow) {
if f.Request == nil || f.Response == nil {
return
}
if _, bypassed := s.bypass.LoadAndDelete(keyOf(f)); bypassed {
s.t.record(f)
return
}
snap := s.t.snaps.take(keyOf(f))
set := s.t.rewriteRules.Load()
if snap != nil {
@@ -506,6 +543,13 @@ func (s *sink) applyResponse(f *mproxy.Flow, set *rewrite.Set, in rewrite.Input)
// upgrade, with no response on the flow: nothing of the handshake can change
// there, so every candidate rule is traced "not rewritten: stream".
func (s *sink) rewriteStream(f *mproxy.Flow, kind string) (rq requestView, rs responseView, lines []rewrite.TraceLine, res rewrite.Result) {
if _, bypassed := s.bypass.LoadAndDelete(keyOf(f)); bypassed {
rq = flowRequestView(f.Request)
if kind == "sse" {
rs = flowResponseView(f.Response)
}
return rq, rs, nil, rewrite.Result{}
}
snap := s.t.snaps.take(keyOf(f))
set := s.t.rewriteRules.Load()
if snap != nil {
@@ -568,6 +612,7 @@ func (s *sink) applyStreamResponse(f *mproxy.Flow, res rewrite.Result) {
// and the rules, never the URL or the error text.
func (s *sink) RequestError(f *mproxy.Flow, err error) {
if f != nil {
s.bypass.Delete(keyOf(f))
if snap := s.t.snaps.take(keyOf(f)); snap != nil && snap.requestRewritten() {
ids := snap.appliedRuleIDs()
for _, id := range ids {
@@ -593,6 +638,7 @@ func (s *sink) StreamResponseModifier(f *mproxy.Flow, in io.Reader) (out io.Read
if s == nil || s.t == nil || f == nil || !f.Stream || f.SSE != nil {
return in
}
s.bypass.Delete(keyOf(f))
snap := s.t.snaps.take(keyOf(f))
if snap == nil {
return in
+1
View File
@@ -469,6 +469,7 @@ type sink struct {
t *Traffic
paced sync.Map // ClientConn.Id of a marked connection -> struct{}
streams sync.Map // *mproxy.Flow of a live WebSocket/SSE stream -> *streamState (stream.go)
bypass sync.Map // flowKey of a replay flow bypassing rewrite rules -> struct{}
}
// maybePassthrough marks a host to be tunneled transparently on the next
@@ -7,6 +7,7 @@ import { Loader2Icon, PlusIcon, SendIcon, XIcon } from "lucide-react";
import { HttpCodeBlock, statusTone } from "@/components/http-code-block";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import { Checkbox } from "@/components/ui/checkbox";
import { Input } from "@/components/ui/input";
import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table";
import { Textarea } from "@/components/ui/textarea";
@@ -23,6 +24,7 @@ type ReplayForm = {
url: string;
rows: HeaderRow[];
body: string;
applyRules: boolean;
// Headers of the recorded request that the server would not send.
dropped: string[];
binary: boolean;
@@ -40,6 +42,7 @@ function initialForm(exchange: TrafficExchange, raw: string): ReplayForm {
url: exchange.url,
rows: toRows(sent),
body: parsed.binary ? "" : parsed.body,
applyRules: false,
dropped: notSent.filter((n) => n.toLowerCase() !== "host"),
binary: parsed.binary,
bodyPreviewOnly: parsed.bodyPreviewOnly,
@@ -83,6 +86,7 @@ export function ReplayPanel({
// Answers for an older exchange are dropped.
const generation = React.useRef(0);
const methodListId = React.useId();
const applyRulesId = React.useId();
React.useEffect(() => {
generation.current++;
@@ -111,6 +115,7 @@ export function ReplayPanel({
headers: splitReplayHeaders(form.rows).sent,
body: form.body,
body_base64: false,
apply_rules: form.applyRules,
})
.then((r) => {
if (gen !== generation.current) return;
@@ -230,10 +235,20 @@ export function ReplayPanel({
)}
</div>
<Button onClick={send} disabled={!canSend}>
{sending ? <Loader2Icon className="animate-spin" /> : <SendIcon />}
{t("Send")}
</Button>
<div className="flex flex-wrap items-center gap-4">
<Button onClick={send} disabled={!canSend}>
{sending ? <Loader2Icon className="animate-spin" /> : <SendIcon />}
{t("Send")}
</Button>
<label htmlFor={applyRulesId} className="flex cursor-pointer items-center gap-2 text-xs select-none">
<Checkbox
id={applyRulesId}
checked={form.applyRules}
onCheckedChange={(checked) => update({ applyRules: checked === true })}
/>
<span>{t("Apply capture rewrite rules")}</span>
</label>
</div>
{error && (
<div className="space-y-1 rounded-md border border-red-500/40 bg-red-500/10 p-3 text-xs">
+1
View File
@@ -275,6 +275,7 @@
"Appended {count} scopes": "{count} kapsam eklendi",
"Applies only to this profile, overriding the global defaults from \"Retry & backoff\". Each field left empty = follow the global; attempts -1 = disable this layer; a filled interval replaces the exponential backoff with a fixed one. The breaker and intent rerun are process-level and can only be tuned on the global page.": "Yalnızca bu profile uygulanır ve \"Yeniden deneme ve geri çekilme\" sekmesindeki genel varsayılanları geçersiz kılar. Boş bırakılan her alan = genel ayarı izle; deneme sayısı -1 = bu katmanı devre dışı bırak; doldurulan aralık üstel geri çekilmeyi sabit bir aralıkla değiştirir. Devre kesici ve niyet yeniden çalıştırma süreç düzeyindedir ve yalnızca genel sayfada ayarlanabilir.",
"Applies to this task only and is not written to global rules. Evaluation order: \"Block\" rules (including global ones) match first, and a hit forbids testing; with no hit and \"Allow\" rules configured for this task, an asset must match an allow rule to be permitted, otherwise testing is likewise forbidden; with no allow rules configured, no whitelist is active.": "Yalnızca bu göreve uygulanır ve genel kurallara yazılmaz. Değerlendirme sırası: önce \"Engelle\" kuralları (genel olanlar dahil) eşleştirilir ve eşleşme testi yasaklar; eşleşme yoksa ve bu görev için \"İzin ver\" kuralları yapılandırılmışsa, bir varlığa izin verilmesi için bir izin kuralıyla eşleşmesi gerekir, aksi halde test yine yasaktır; izin kuralı yapılandırılmamışsa beyaz liste etkin değildir.",
"Apply capture rewrite rules": "Yakalama yeniden yazma kurallarını uygula",
"Approval calls": "Onay çağrıları",
"Approval decision": "Onay kararı",
"Approval details #{id}": "Onay ayrıntıları #{id}",
+7 -3
View File
@@ -874,11 +874,15 @@ export const api = {
if (!r.ok) throw new Error(t("Blob download failed: {error}", { error: r.status }));
return new Uint8Array(await r.arrayBuffer());
},
// Re-sends an edited request through the proxy. Errors carry the HTTP status
// (ApiError): 413 body too large, 502 proxy/upstream, 503 capture off, 504 timeout.
// Re-sends an edited request through the proxy (bypasses capture rewrite rules
// unless req.apply_rules is true). Errors carry the HTTP status (ApiError):
// 413 body too large, 502 proxy/upstream, 503 capture off, 504 timeout.
trafficReplay: async (req: TrafficReplayRequest) => {
if (MOCK) throw new Error(t("Replay is not available in mock mode"));
return post<TrafficReplayResult>(`/traffic/replay`, req);
return post<TrafficReplayResult>(`/traffic/replay`, {
...req,
apply_rules: Boolean(req.apply_rules),
});
},
trafficHosts: () => get<{ hosts: TrafficHost[] }>(`/traffic/hosts`),
trafficDeleteHost: (host: string) => del<{ deleted: number }>(`/traffic?host=${encodeURIComponent(host)}`),
+11
View File
@@ -905,6 +905,14 @@ export interface TrafficExchange {
resp_len: number;
}
export interface TrafficRewriteStats {
large_responses: number;
capacity_refusals: number;
tombstone_drops: number;
conversions: number;
sweeps: number;
}
export interface TrafficResp {
enabled: boolean;
proxy?: string;
@@ -913,6 +921,7 @@ export interface TrafficResp {
page?: number;
size?: number;
exchanges?: TrafficExchange[];
rewrite?: TrafficRewriteStats;
}
// Full raw request/response of one exchange (lazy-loaded on row select).
@@ -955,12 +964,14 @@ export interface TrafficReplayHeader {
}
// POST /api/traffic/replay. Host is not sent: the server takes it from url.
// apply_rules defaults to false (bypasses capture rewrite rules).
export interface TrafficReplayRequest {
method: string;
url: string;
headers: TrafficReplayHeader[];
body: string;
body_base64: boolean;
apply_rules?: boolean;
}
export interface TrafficReplayResult {