In a gated scope, the two HTTP clients the platform owns connect only to
addresses that a pin or a stored exemption covers at the moment of the
connection, on every hop.
- intercept.PinnedDialer(tool, scoped): the DialContext of both clients.
It reads the target scope from the context net/http passes to the dial.
Gate off, or no scope in a client built without one: plain dial, as
today. A client built for a scope whose scope is missing at the dial
fails closed with TARGET_GATE_UNAVAILABLE.
- dialTarget runs the gate decision on the dialled host:port with a fresh
lookup (block rules first, exemptions, pins, port precedence) and
connects only to the addresses it compared, never to the name. Derived
exemptions and name-only pins dial the address as received. An address
no pin covers finds or opens the request (new_ip / no_pin, wait 0) and
fails at once with TARGET_APPROVAL_PENDING. Redirect hops are dialled
against the hop's own (host, port).
- WebFetch (withWebFetch, now given the run context): DialContext set on
every agent kind; DisableDirectRetry when the run's scope is gated.
- Custom http tool: a direct call in a gated scope gets a DefaultTransport
clone with the pinned dialer, no environment proxy and keep-alive off.
Proxied calls are unchanged (the recording proxy is step 2.2).
- Name-only: request reason says the platform does not bind the
connection; the first-use pin_match row records the effective state.
- Derived exemptions: a proxy URL without a port is listed with the port
net/http dials (1080 for socks5) (OPEN-9).
- go.mod: Norma v0.4.8 (DialContext, DisableDirectRetry).
WebFetch and the custom http tool followed redirects with net/http's
default policy, so a pinned host could send the client to a target nobody
approved. RedirectCheck is now the CheckRedirect policy of both clients.
- Gate off, or no scope (operator test endpoint): only the 10-hop limit,
same error text; nothing is parsed, resolved, read or written.
- Gate on: each hop runs through the call's decision code (URL rules on
the hop URL, block rules before any lookup, exemptions, resolution,
pins). A hop no pin covers is not followed and never waits: its request
row is found or created and the call fails with TARGET_APPROVAL_PENDING.
http -> https asks again for port 443 unless a (host, 0) pin exists. An
unreadable hop is blocked with TARGET_GATE_UNAVAILABLE.
- Rows raised by a hop record redirect_from in the audit targets.
- Norma v0.4.7 has WebFetchConfig.CheckRedirect but no agentcore option,
so the four agent kinds build WebFetch themselves (same proxy, CA and
position in the tool list) instead of EnableWebFetch. Norma carries the
policy to the direct retry client.
Point the Norma replace at v0.4.7, which brings the thinking replay fix
(#4) and the WebFetch CheckRedirect hook (#3). go.sum updated by
go mod tidy. No ARTEX code changes.
The target gate resolved each name through the cache before it matched the
asset block rules, so a call to a name that a name rule blocks still cost a
DNS lookup, and that query reached the name servers of the blocked domain.
URL rules (exact_url, fuzzy_url) were not evaluated at dispatch at all: the
gate passed no URL to the matcher, so they only applied to asset and intent
writes.
targetCheck.run now decides in this order, for a gated call:
1. URL rules, once per decision, on the URLs the call declares, as written.
2. Every block rule on the declared form of every target: the name as
written, the IP literal, the range.
3. Only then resolution (names a derived exemption covers are still never
resolved), and the block rules again on the resolved addresses.
4. Exemptions, pins and requests, unchanged.
A hit in 1 or 2 blocks with TARGET_RULE_BLOCKED and the same decided row as
any block, costs no lookup for any target of the call, and opens no request.
- intercept.extractDeclared returns the targets (ExtractTargets is now a thin
wrapper, unchanged for its callers) and the URLs that name a target: the
WebFetch url, and every URL token the command-string rules read in Bash,
shell_open, Monitor, shell_send and the string leaves of other tools,
including ExecuteExtraTool's inner call.
- The custom http tool's rendered URL and the URLs of a rendered command go to
the same URL rules (gateRendered takes the URLs).
- Known limit, stated in the comments next to the matcher (db.matchOne) and
at the dispatch match: URL rules match the declared string. exact_url misses
any other spelling (case, percent-encoding, trailing slash, added query,
/./), fuzzy_url misses an encoded substring, and a command can build its URL
at run time. For a host, a domain, IP or CIDR rule is the one to write.
Unchanged: a tool-rule or judge deny still blocks before the gate is reached
(no extraction, no lookup, no request); asset block rules still win over an
allow and an ask; the gate off path does nothing; scripts stay advisory.
Tests:
- intercept: TestBlockRuleOnDeclaredNameNeedsNoLookup (exact_domain and
fuzzy_domain; zero lookups, also for an unblocked name earlier in the same
call), TestExtractDeclaredURLs, TestURLBlockRuleAtDispatch (WebFetch, Bash,
MCP, ExecuteExtraTool, a pinned and an exempt host, a disabled rule, the
rendered URL and command; zero lookups), TestExactURLRuleMatchesOnlyAsWritten
(documents the limit).
- guard: TestBlockRuleWinsOverAllowAndAsk (name rule and URL rule, each over a
rule allow, judge allow, rule ask and judge ask, real interceptor: blocked,
no allow row, nobody asked, zero lookups). TestDenyRuleWinsBeforeTargetRequest
now also covers a judge deny and asserts zero lookups with the real gate.
- server: TestBypass_CustomScriptToolTargetNotGated documents that a script
tool's target, written in its stored code, is not visible to the gate: in a
gated task the call is allowed and reaches the target with no lookup, no
request and no row.
spawn_task now reads the target-gate scope of the calling session from the
tool context. When the caller is gated, the child task is created with the
gate on (D23) and with a copy of the calling task's block rules (D24). Pins
and allow rules are never copied, parent_ref is not consulted, and spawn_task
gets no new parameter. A gate-off caller, or a call with no scope, creates the
child exactly as before.
A conversation started by a trigger of a gated task is created with the gate
on (D25). A merged run is gated if any of its fires came from a gated task.
Pins are never copied. Fires of a gate-off task and fires with no task are
unchanged.
TestSpawnedTaskStartsWithGateOff becomes TestSpawnedTaskInheritsGateFromCaller.
New: TestSpawnTaskSeesCallerScopeThroughHarness and
TestTriggeredConversationInheritsGate.
The hook reads the parameters of a custom tool call, not the URL or the
command they render, and not a target written in the template itself. The
custom command and http tools now run the target gate inside the tool, on what
they are about to run, before anything is sent or started. Gate off (the
default) is unchanged: with no scope or the flag off nothing is parsed,
resolved, read or written.
- intercept.GateRenderedURL: the target is read as net/http dials it (host of
the parsed URL, explicit port, else the scheme default), then goes through
the same decision as at the hook (asset block rules, exemptions, resolution,
pin lookup) and the same request-and-wait, at most three passes. A URL whose
target cannot be read is blocked with TARGET_GATE_UNAVAILABLE and logged as
a decided row; no request, no lookup.
- intercept.GateRenderedCommand: the rendered command is read with the
command-string rules of Bash, and so is the rendering without the shell
quoting of the parameters (a quoted parameter inside a host name hides the
name from the rules). A command that names no target passes, as in Bash.
- A gated scope without the interceptor that owns the gate blocks
(TARGET_GATE_UNAVAILABLE), it never lets the call through.
- runCommandTool and runHTTPTool take the tool key, which names the request
rows. The gate runs before the tool's own timeout starts.
- guard.TargetBlockMessage (was targetBlockMessage) is exported so the tools
return the same model-facing text as the hook.
- Unchanged: the script tool (its target is in stored code; advisory, D9), the
custom-tool test endpoint (no scope: the operator's own action, D14), the
hook order (a rule or judge deny blocks first; the tool then never runs, so
its rendered target is never read), the evaluation of asset rules.
Tests: TestRendered* in intercept (no database); in server, against
PostgreSQL: TestCustomHTTPRenderedTargetGated,
TestCustomCommandRenderedTargetGated, TestCustomToolTestEndpointUngated,
TestTargetGateEndToEnd (a scripted worker's WebFetch waits, a click pins, the
next call passes, revoke, it asks again), TestDenyRuleWinsBeforeRenderedTarget
and TestGateOffTaskIsUnchanged.
Not in this change: redirect targets of WebFetch and of the custom http tool
(1.9).
The approval gate for asset targets now decides. It is opt-in per task and
per conversation (target_gate, default off). With the flag off nothing is
extracted, resolved, read or written, and every branch of the hook behaves as
before. No UI.
Extraction and normalisation (1.5)
- intercept.NormalizeTarget: lower case, trailing dot, userinfo, IPv6
brackets, IPv4-mapped IPv6, decimal / hex / octal / short IPv4, percent-
encoded host, punycode; port explicit, else the scheme default, else 0.
- intercept.ExtractTargets: WebFetch url exactly; Bash, shell_open, Monitor
(command) and shell_send (text) with the command-string rules (URL with a
scheme, IP, IP:port, host:port, CIDR; a bare name is not a target); every
string of the input for MCP tools, custom tools and any tool the gate does
not know. ExecuteExtraTool is unwrapped to the inner tool. The platform's
domain tools, the file tools and web_search have no target.
- The fields are decoded as the tools decode them (key case, last duplicate),
and a URL authority that reads differently quoted and unquoted yields both
readings.
Resolver, exemptions, gate decision (1.6)
- Resolver interface, system implementation (A and AAAA), 3 s per lookup,
positive answers cached 10 s per host with collapsed lookups, failures and
empty answers never cached, always fresh when a request is raised.
- Exemptions are exact host + port. Derived from the configuration (LLM
gateways, search backend in use, platform proxies): computed on every call,
never resolved, never voided. Stored (settings key
target_gate_exempt_hosts, seeded once with the package sources): resolved,
and void when any address is private, loopback, link-local, unspecified,
unique-local, multicast or a metadata address. An entry in both origins
counts as derived. A stored IP literal that can never be exempt does not
apply.
- GET / PUT /api/target-gate/exemptions and GET .../history. A change is
validated (ValidateExemptionHost on the normalised host) and appended to
target_gate_exemption_log in the transaction that stores it.
- Interceptor.CheckTargets: asset block rules (global and of the task) on the
name and its addresses, exemptions, resolution, pin lookup. A (host, 0) pin
covers every port; a (host, 443) pin covers 443 only and never a target
without a declared port; two pins are never merged. At most 5 unpinned
targets per call. Codes: TARGET_RULE_BLOCKED, TARGET_RESOLVE_FAILED,
TARGET_TOO_MANY, TARGET_GATE_UNAVAILABLE; none opens a request.
- With the upstream egress proxy set every pin is name-only at call time,
whatever name_only_effective says in its row.
- An approval never pins zero addresses: the pin writer refuses a request
that displays none (db.ErrTargetPinEmpty, 422 on the decide endpoint, the
row stays pending) and RequestTargets does not open one.
- Every block, and the first use per process of each exemption and of each
pin version, is a decided row (decided_by system, decision_source target).
The gate in the hook (1.7)
- guard.applyIntercept: a deny by a rule or by the judge blocks first, before
any target request exists; then CheckTargets; an allow is logged as before
and does not satisfy the gate; an ask and an unpinned target share one
request, and the classic ask is raised only when it was not merged. The
call is checked again after the decisions, at most three passes.
- The gate applies to every tool of a gated session, enabled for rules or
not, and to the target-only guard of the planner and the main agent, which
still records blocks only.
- Target blocks have their own model-facing text, built from the code and the
reason of the result.
Not in this change: rendered targets of custom tools (1.8) and redirect
targets (1.9). Subprocess paths stay advisory; the TestBypass_* tests document
what the gate does not see.
Storage, flag and request path of the approval gate for asset targets. The
gate itself is not wired into the tool hook yet (steps 1.5 to 1.7): with this
change nothing is extracted, resolved or blocked, whatever the flag says.
Schema and store (1.2)
- tasks.target_gate, conversations.target_gate (default false).
- intercept_pending.target_key and a partial unique index: one open request
per scope (task or conversation) and host:port.
- target_pins (no foreign keys: a pin outlives its task, its conversation and
its request row) and target_gate_exemption_log. All DDL is idempotent; no
CHECK constraints; the cascade on intercept_pending.conversation_id stays.
- db/target_pins.go: pin lookup with port precedence, list, revoke,
find-or-create of a request, and ResolveTargetRequest, which settles the
row, supersedes the old pin and writes the new version in one transaction.
- Deleting a task or an archive stub revokes its pins and settles its open
target requests in the delete transaction. DeleteConversation and
DeleteConversations revoke the pins of the removed conversations in the
same transaction as the delete.
Opt-in flag (1.3)
- Per task through the create and metadata-patch paths, per conversation
through create and patch; exposed in the task DTO. A retest conversation
takes the flag of the finding's task, never its pins. A spawned task starts
with the gate off.
- The task flag is atomic and read on every call through TargetScope.Enabled.
TargetScope now carries the Interceptor that owns the gate.
Request, wait, decide (1.4)
- pendingManager keeps a list of waiters per request; resolve wakes all.
- RequestTargets: find-or-create per target, one card per request, wait on
channel, poll, context and wait bound. A plain request is never settled by
a call that leaves. A request merged with an ask is bound to its creating
call and a timeout never allows it.
- Decide records the approver (subject of the session token) and the remote
address for every human decision, classic asks included. On a target
request "allowed" pins the displayed addresses; that is the only writer of
pins.
- A request raised while the global egress proxy is set says the approval is
name-only and its pin is stored with name_only_effective.
- Routes: GET/POST .../target-pins and .../target-pins/{pid}/revoke under
/api/tasks/{id} and /api/conversations/{id}; decision_source=target filter.
- ValidateExemptionHost: an exemption written as an IP literal may be private
unicast, never loopback, link-local or metadata, unspecified or multicast.
Step 1.1 of the S3b phase 1 plan. Wiring only: the approval gate for asset
targets is not enforced yet and no behaviour changes.
- Planner.Plan and MainAgent.Chat take a hooks parameter, like Worker.Execute.
Their sessions had no hooks, so nothing saw their tool calls before execution.
- guard.NewTargetOnly: the guard for those two agent kinds. It runs the
target-gate seam only: no intercept rule, no fallback judge, and no entry in
the audit ring (Guard.record is untouched and not called for these calls).
- Task.TargetGuard, built beside Task.Guard.
- intercept.WithTargetScope / TargetScopeFromContext: task id or conversation
id, an Enabled func and a wait bound, set at the five session entry points
(planner round, final planner round with wait bound 0, main-agent turn,
worker intent, conversation turn).
The seam allows every call and reads and writes nothing. No task or
conversation has an opt-in flag yet, so Enabled is nil (off) everywhere.
An intent claim was atomic but had no heartbeat, no expiry and no owner
token: startup reset every running intent to open unconditionally, so a
second instance on the same database stole live claims, and the worker
that lost its intent could still write the result.
- schema: last_heartbeat_at and unclaim_token on exploration_nodes
(idempotent ADD COLUMN IF NOT EXISTS, nullable, no CHECK, no backfill).
- db: every claim (open->running and the paused->running message path)
gets a fresh crypto/rand token and starts its lease in the same UPDATE.
HeartbeatIntentClaim renews it for that token only.
- TTL: db.IntentClaimTTL = 90s, heartbeat every 15s
(db.IntentClaimHeartbeatInterval), both on the database clock.
- takeover: ResetRunningIntents now reopens only claims whose heartbeat
is older than the TTL, or NULL (rows claimed before the columns
existed). The engine runs it per task every 15s as well as at startup.
- fencing: a heartbeat or result write whose token is not the current
one returns db.ErrIntentClaimLost and writes nothing. All intent state
writes in runIntent go through the fenced SettleIntentClaim /
SettleIntentClaimBlocked; a rejected heartbeat cancels the stale run
with the new work_claim_lost cause.
Behaviour change: after a restart, intents that were in flight are
resumed once their claim expires (at most 105s) instead of immediately.
Not in this change: fencing of the facts, findings and activity a worker
writes during its run; they are not guarded by the token.
A Feishu digest sends every item as its own card element. The packing
counted the text of each line only, not the element around it (about 52
bytes of JSON per item, plus whatever the encoding escapes) nor the rest
of the body beyond a fixed 320-byte reserve. A digest of 500 short lines
therefore left as a single request of 41846 bytes, while Feishu rejects
a request body over 20 KB as a whole.
feishuCard now adds whole items while the request body, measured as it
is serialized, stays within feishuMaxCardBytes: each item element by its
JSON encoding, the card around the items (header, intro, platform link)
by encoding it for the number of items kept, and what Send adds around
the card (msg_type, timestamp, sign) as feishuEnvelopeBytes, which the
new test pins to the bytes on the wire.
Unchanged: the first item of a digest is always kept and its text is
truncated to the same budget, single pushes are built as before, and
the other channels still pack with packItemCount.
Without ARTEX_PG_DSN and without a config file, go test ./server/ failed
44 tests with "no database config found" instead of skipping them. The
tests go through NewManager and t.Fatal on its error: 35 through the
shared fixture trafficEvidenceServer (directly, or through
verificationFixture and newNotifyFixture), 2 through ingestServer, and 7
call NewManager themselves. The 7 already carry a skip on m.pg == nil,
but it sits after the t.Fatal and is never reached.
Add skipWithoutDSN, which calls db.DSN() and skips with the message the
db package and assembly_test.go already use ("no database config (%v) —
skipping"), and call it before NewManager at those 9 sites.
Nothing changes when a database is configured: db.DSN() succeeds, the
helper returns, and a NewManager error (for example an unreachable
Postgres) still fails the test.
PR #172 made the summary length and the number of assets listed in an IM
message configurable, but only through the API. The Notifications page now
shows both next to the other global settings:
Summary length (characters) notify_summary_max_runes 0-4000
Assets listed per finding notify_assets_max_shown 0-50
The values are read from GET /api/settings (they are not part of
/notify/meta) and saved with the existing "Save global settings" PUT.
Each field carries a help text stating that 0 means no truncation.
A value that is not a whole number within the bounds marks the input
invalid and blocks the save with a toast. The check runs before the
request because the backend applies the fields of one PUT in order, so a
400 on a limit would leave the URL and digest interval already saved. An
empty field is not submitted and keeps the stored value.
The IM renderers cut the summary of a single push at 120 characters and
listed at most 3 assets, as fixed constants. Both are now global settings
in the settings table, read by the delivery engine when it renders:
notify_summary_max_runes 0-4000, default 120
notify_assets_max_shown 0-50, default 3
0 means no truncation and an unset key keeps 120 / 3. PUT /api/settings
answers 400 for a value outside the bounds. The limits travel on
notify.Message, so the notify package still does not touch the database.
With no truncation, nothing bounded a single item in a Feishu card, and
Feishu rejects an oversized request as a whole. feishuCard now truncates
an item's text as the final backstop, the role TruncateBytes plays in
markdownBody for DingTalk and WeCom. feishuMaxCardBytes goes from 24000
to 16000: Feishu custom bots cap the request body at 20 KB, which the old
value exceeded.
Refs #168.
malformedIPAssetWarning appended "and N more" with N = total, the number
of offending assets, instead of the number left out of the sample. With 8
bad rows and 5 named it read "... and 8 more"; it now reads "... and 3
more". Same bug as the one fixed in notify/render.go.
Move the wording into malformedIPAssetWarningText so it can be tested
without Postgres, and add a test for the 8-of-5 and 5-of-5 cases.
assetLine appended "and N more" with N = len(assets), the total, instead
of the number of assets left out. With 5 assets and a limit of 2 it read
"a, b and 5 more"; it now reads "a, b and 3 more".
Fix the test that pinned the old value and add an 8-assets/limit-3 case.
A response captured with Content-Encoding: gzip and no body (HEAD, 304) is
decoded to an empty body without error. Pin that behaviour, and pin that an
empty body, compressed or not, matches no non-empty quote through
readBoundResponse and quoteInBodies, so it can never lead to a verified
finding.
Compressed HTML responses are stored as captured, so the S1 receipt check saw them as binary and returned not_assessed. readBoundResponse now undoes the Content-Encoding declared in the snapshot's response head before the text/binary detection and the quote search.
decodeEvidenceBody handles identity and a single layer of gzip or deflate (zlib-wrapped or bare), bounded by verificationReadCap on the decoded side. Any other coding, stacked codings, or a stream that fails to decode is an error, which the caller reports as not_assessed (evidence body unreadable). The stored evidence bytes are unchanged.
The previous commit stops list_task_credentials from building its schema
with obj(map[string]any{}, ""). Two gaps remained:
- obj still emitted whatever it was given, so the same typo on another tool
would again produce "required": [""]. obj now drops names that are blank
after strings.TrimSpace and omits the "required" key when nothing is left.
- Tool schemas are cached in the tools table and SeedTool is insert-only, so
the code fix only reached fresh installs. Existing databases kept the
malformed row and strict providers kept rejecting every worker run.
healListTaskCredentialsSchema runs at startup next to the other tool
seeds and removes the "required" key from that one row, only when it is
exactly [""]:
UPDATE tools SET schema = schema - 'required', updated_at=now()
WHERE key='list_task_credentials' AND schema->'required' = '[""]'::jsonb
It is idempotent (a healed row no longer matches), needs no settings flag,
and leaves the description, bindings, enabled flag and every other tool
untouched. tool_schema_refresh_v7 is not bumped.
Tests: TestObjDropsBlankRequired and
TestListTaskCredentialsSchemaHasNoBlankRequired (agent),
TestHealListTaskCredentialsSchema (server, Postgres).
Norma v0.4.6 frames tool output between nonce-tagged markers, so the model
reads it as data. ARTEX passes that option from the environment: 1, true or
on enable it (case and surrounding spaces ignored); unset or anything else
keeps it off, which is the default. Nothing changes unless the variable is set.
The option is wired into the four agentcore.Options builders that set
ToolOutputDir: worker, planner, chat and the main agent. .env.example
documents the variable, commented out, with a note to test it on a local
target before enabling it in production.
Not included: docker-compose.yml does not pass the variable to the artex
service, so it has no effect under compose until it is listed there.
The option filters decided_by = model, which includes denials the model judge
made, so "AI-approved" was inaccurate. Approved-by-AI is the combination with
status allowed.
The approval list and detail return decided_by exactly as stored, and the
decided_by filter matches that same value. Rows decided before the column
existed show "Legacy" and match no decided_by filter; their decision_source
stays in its own column and nothing is rewritten.
UI: a "Decided by" column (Human, Rule, Model, System) and a matching filter
beside the decision-source filter, including "AI-approved only" for model.
Types and the API query carry decided_by the same way decision_source does.
Add intercept_pending.decided_by (human | rule | model | system; empty while
pending or for rows decided before the column existed). Approval semantics are
unchanged: only the actor is recorded.
- rule-matched and model-judged rows written by CreateDecidedIntercept record
rule or model;
- ResolveIntercept takes the actor explicitly and rejects any other value
before touching the database;
- Decide (human allow/deny) records human; timeouts and work-cancelled
denials record system. Work-cancelled is a system refusal, not a human one.
- decided_by is exposed on the pending and approval records and filterable
with decided_by=human|rule|model|system (anything else is a 400).
- Legacy rows show decision_source as their actor when decided_by is empty.
The filter uses the stored column only, so decided_by=model cannot match
older human decisions or timeouts.
- Archive restore fills decided_by for archives that predate the column, and
marks a restored pending row as system.
A completed retest with a reproduced or fixed verdict is an independent proof,
so it sets verification_status to verified with a note naming the retest, its
verdict and the start of its summary. The upgrade applies only from unverified,
not_assessed or blocked: a verified finding keeps its POE level and note.
Inconclusive, failed and stopped retests never change verification_status; their
verdict stays in the retest history. The human status is untouched by this path.
The existing fixed path (status set to Fixed with notifications) is unchanged.
Tests cover reproduced, fixed (status stays Fixed), inconclusive on a verified
finding, failed and stopped, and the upgrade from not_assessed and blocked.
Show the automated verification outcome next to the human triage status:
a badge on the findings list and detail page ("Verified L3", "Unverified",
"Blocked", "Not assessed", "False positive"), a verification-state filter
next to the status filter (persisted with the other list preferences), and
the verification note on the detail page when the state is not verified.
API: the findings list, groups, asset tree and export accept a
verification_status filter. Without it nothing changes.
Not included: a verified count in the statistics.
The quote x7 is now rejected as too short before the store is read, so the
test never reached the not_assessed path. Use a quote long enough to pass
the minimum; the store removal is unchanged.
A two-character quote such as "x7" can exist in any body and proves
nothing. Normalized quotes under 8 runes are now ignored; if none remain
the finding is unverified with the note "evidence quote too short".
Check order is unchanged: binding, POE, quote presence, then body.
BaaRai