The status line said the proxy does not apply rules yet. Replace it with the request and response phases, what exchange_rewrites records and how the hashes are formed (plain SHA-256, keyed HMAC for sensitive actions), the SSE and WebSocket behaviour, the 5 MiB response pass-through, the snapshot limits (live slots are never deleted by the TTL), and the fact that replays go through the proxy and are rewritten like captures.
11 KiB
Capture-proxy rewrite rules
A rewrite rule matches a request or a response that goes through the capture proxy and describes how to change it: the method, the path and query, headers, the body, the status code.
Status: the capture proxy applies enabled rules to HTTP traffic (request and response phases), and records what it did. Not yet in this change: archive format, a per-replay rule toggle, the web UI, and the proof flag. See Application in the capture proxy for what is covered and what is not.
Rules are a convenience for the operator, not a security control: a rule that fails leaves the message unchanged (fail open).
Rule
| Field | Meaning |
|---|---|
name |
Required. |
enabled |
A rule is created disabled, whatever the request says. Enable it with the toggle or an update. |
phase |
request or response. |
priority |
Rules run by priority ascending, then id ascending: the highest priority runs last and wins. Each rule sees the output of the previous one. |
matchers |
Object, see below. |
actions |
Array, at least one, see below. |
note |
Free text. |
Matchers
All matchers of a rule must hold. An absent matcher is true.
| Matcher | Phase | Meaning |
|---|---|---|
methods |
both | List of methods, case-insensitive. |
host |
both | Host name without port, case-insensitive: an exact name, or a suffix glob *.example.com (which does not match example.com). No other wildcard. |
url_regex |
both | Applied to the path and query. |
headers |
both | List of {name, value_regex} on the request headers. The name is case-insensitive; an empty value_regex tests presence. |
body_regex |
both | Applied to the request body. |
status |
response | List of codes (404) or classes (4xx), any of. |
resp_headers |
response | Like headers, on the response. |
resp_body_regex |
response | Applied to the response body. |
In the response phase, the request matchers see the request as it was sent to the target, that is, after the request-phase rules.
Regular expressions use Go's regexp syntax (RE2).
Actions
| Action | Phase | Fields | Effect |
|---|---|---|---|
set_method |
request | value |
Replaces the method. CONNECT is refused. |
set_path_query |
request | value |
Replaces the path and query. Must start with /. |
replace_url |
request | regex, value |
Regex replacement on the path and query only. |
delete_header |
both | name |
Removes every value of the header. |
set_header |
both | name, value |
Replaces every value of the header with one. |
replace_header |
both | name, regex, value |
Regex replacement on each value. |
set_body |
both | value |
Replaces the body. |
replace_body |
both | regex, value |
Regex replacement on the body. |
set_status |
response | value |
Replaces the status code: 200 to 599, except 204 and 304. |
In a replacement, value is a template: $1 or ${name} insert a capture group.
Within a rule, actions run in a fixed order whatever the order they are listed in: method and URL,
delete_header, set_header and replace_header (in listed order), body, status. A rule is
atomic: if one action fails, none of the rule's actions applies.
A rule can never change the scheme, the host or the port. These header targets are refused: Host,
Content-Length, Transfer-Encoding, Connection, Keep-Alive, TE, Trailer, Expect,
Upgrade and every Proxy-* header. A header value cannot contain a line break or a NUL byte.
A rule that reads or rewrites a body is skipped whole, its header, URL and status actions included,
when that body is binary, larger than 1 MiB, compressed (Content-Encoding other than identity),
streamed, or absent by protocol (response to HEAD, status 1xx, 204 or 304).
Sensitive actions
An action marked "sensitive": true keeps its value out of every API response. A read returns
the action without value and with "value_set": true.
On update, a sensitive action sent without value keeps the stored value, provided the stored
rule has exactly one sensitive action with the same type, header name (case-insensitive) and
regex, whatever its position: actions can be reordered without sending their values. When the
stored rule has no such action, or several (same type, same name, same regex), the update is
refused with HTTP 400 and the value must be sent again. An explicit "value": "" is a value.
A stored sensitive value is not written to a new destination. When an update keeps at least one
stored value, the host matcher must stay the same (compared case-insensitively); otherwise the
update is refused with HTTP 400 and the value must be sent again or cleared. The url_regex
matcher is not part of this comparison, an accepted limit: a kept value follows a rule whose
url_regex changes on the same host.
The flag covers the action's value only. Matchers are not masked. A value an action places in the URL, marked sensitive or not, is sent to the target as part of the URL.
Limits
| Limit | Value |
|---|---|
| Rules | 100, of which 50 enabled |
| Regular expression | 1 KiB |
| Value or replacement template | 1 MiB |
A request over a limit is refused with HTTP 400.
API
All routes require the administrator token, like the rest of /api.
| Route | Effect |
|---|---|
GET /api/traffic/rewrite-rules |
{"rules": [...]} in execution order. |
POST /api/traffic/rewrite-rules |
Creates a rule (disabled). |
PUT /api/traffic/rewrite-rules/{id} |
Replaces a rule. |
DELETE /api/traffic/rewrite-rules/{id} |
Deletes a rule. |
POST /api/traffic/rewrite-rules/{id}/toggle |
Body {"enabled": true}. |
A rule the validation refuses, or a limit reached, answers 400; an unknown id answers 404.
{
"name": "staging token",
"phase": "request",
"priority": 10,
"matchers": {"host": "*.staging.example.com", "methods": ["GET", "POST"]},
"actions": [
{"type": "set_header", "name": "Authorization", "value": "Bearer ...", "sensitive": true},
{"type": "replace_url", "regex": "^/v1/", "value": "/v2/"}
],
"note": ""
}
A rule in a response also carries load_error when it is enabled but left out of the running
snapshot (for example a rule saved by an older version that the current validation refuses), and
last_error (class, at, count) for its most recent run-time error. last_error is kept in
memory and lost at restart.
Rules are never exposed as an agent tool.
Storage
Rules live in the PostgreSQL table traffic_rewrite_rules (migration
0005_traffic_rewrite_rules.sql). They are global configuration and are not part of a task
archive. The matchers and actions columns are encrypted at rest with the instance secret key,
like the other stored secrets: a database backup without keys/secret.key (or ARTEX_SECRET_KEY)
cannot restore them, and ARTEX refuses to start under a key that cannot open them.
The server compiles the enabled rules into an immutable snapshot at startup and after every rule change, and hands it to the proxy without a restart.
Application in the capture proxy
The proxy applies the snapshot to each exchange in two phases. The request phase runs before the request is sent to the target; the response phase runs before the response is returned to the client. Each rule sees the output of the previous one. Only rules whose phase matches the step are run, and a rule that fails leaves the message as it was (fail open).
What is recorded
For a rewritten exchange, the record keeps the request as the client sent it and the response as the target sent it. The rewritten values are not stored as the exchange's original data.
- Each action that ran (applied, skipped or failed) is written in the SQLite table
exchange_rewrites, in the same transaction as the exchange itself, with the rule id, a copy of the rule as it was applied, the action, the outcome and the reason. - The before value is kept as its SHA-256. The after value is kept as the value itself, inline or in the blob store when it is large, with its SHA-256. A sensitive action stores neither value nor plain hash: its after value is not stored, and the hashes it leaves are a keyed HMAC when a key is available (empty otherwise). The same HMAC applies to later actions on a field a sensitive action wrote.
traffic_getshows a=== REWRITTEN ===line naming the rule ids when the exchange was changed. The line says that the shown values are the originals.traffic_searchsearches only the original values. It never finds a value that a rule wrote.
Reasons a rule is not applied are recorded in the trace, for example not rewritten: binary,
not rewritten: stream, or not rewritten: snapshot capacity.
Streams: SSE and WebSocket
- SSE (
text/event-stream): status and headers are rewritten at the start of the stream (SSEStart), since they are sent once. Later event frames pass unchanged. - WebSocket: not rewritten. The capture library calls its start hook after the upgrade is
already done, and it short-circuits the upgrade before the request and response hooks run, so
the handshake status and headers cannot be changed. Frames pass unchanged. The exchange is
traced
not rewritten: stream.
Large responses
go-mitmproxy does not call the response hook for a body of 5 MiB or more (its StreamLargeBodies
default). Such a response is forwarded to the client unchanged and is not recorded in full, and no
response rule runs on it.
Snapshot limits
Each exchange with a rewrite trace (at least one rule matched and produced a line: applied, skipped or failed) keeps an original copy in memory until it finishes. Limits:
| Limit | Value |
|---|---|
| Entries holding an original request | 1024 |
| Bytes held by those entries | 64 MiB |
| Tombstones (slots whose data was released) | 4096 |
| Time after which the slot of a live flow may be turned into a tombstone (see below) | 15 minutes |
When there is no room, the request is not rewritten. The exchange is traced not rewritten: snapshot capacity when a tombstone slot is free; when the tombstone table is full, the request is refused with no trace line at all (the refusal is counted and logged, at most once a minute).
The slot of a finished flow is swept when its connection closes (at most once per second) and at
least once a minute while slots exist. The 15-minute TTL never deletes a live flow's slot: past it,
the slot may be converted into a tombstone if a new reservation needs the room.
Replay
Replay goes through the recording proxy like any other traffic, so the enabled rules apply to a replay and the resulting exchange is traced like a captured one. There is no per-replay toggle yet.