Commit Graph
279 Commits
Author SHA1 Message Date
BaaRai e770caa28e feat(traffic): replay an edited request through the recording proxy
Traffic.Replay sends an edited request (method, URL, headers, body) through the local proxy, which records the exchange like any other, and returns the response bounded to 2 MiB. POST /api/traffic/replay exposes it to operators only (no agent tool). Invalid input is rejected with ErrReplayInvalid; hop-by-hop headers are dropped, redirects are not followed, and a Host that differs from the URL is refused because the proxy routes by Host.
2026-11-16 15:32:02 +01:00
BaaRai dcdc0652ad fix(traffic): bound stream message pages, spill before the transaction, seconds timestamps
Stream payloads over 4 KiB now spill to the blob store (spillAt with maxStreamInline; spill keeps maxInlineBody for HTTP bodies) and StreamMessages clips every payload to 4 KiB, reading a spilled binary preview from its blob. The byte cap counts the inline preview. persistStream spills before opening the transaction, as record() does. Without the library lock nothing is read. ts is unix seconds. Adds the final-message-on-close and route tests.
2026-11-16 08:21:38 +01:00
BaaRai d9df84d2e5 feat(traffic): record WebSocket and SSE stream messages
The recording proxy now stores WebSocket and Server-Sent Events streams: one exchange per stream (101 websocket, or the real text/event-stream response) and its messages in stream_messages, with per-message and per-exchange limits and blob spill. Messages are readable through Traffic.StreamMessages, GET /api/traffic/stream and the traffic_messages agent tool; deletion and blob GC cover them.
2026-11-16 07:47:25 +01:00
BaaRai 2c514a4cd9 perf(server): look up only the requested task archive blocker in getTask
getTask aggregated task_relations for the whole database on every 5-second detail poll to read one entry. TaskArchiveBlocker scopes the same predicate (shared with TaskArchiveBlockers through archiveBlockerScope) to one source task; listTasks keeps the global map.
2026-11-16 06:46:52 +01:00
BaaRai 09c9e6cfc2 fix(server): do not rerun intents refused by the NOA budget
The NOA budget refusal now wraps an exported ErrNoaBudget sentinel, which survives the Norma harness unchanged; retryableWorkerModelError excludes it, so a deterministic refusal is no longer replayed twice by the engine.
2026-11-15 22:26:21 +01:00
BaaRai e6ab0f34a7 fix(server): sum task LLM usage across retries and failover
The logical call now returns the usage of every physical attempt (atomic and stream), keeps the usage of failed uncommitted stream attempts without delivering their events, normalizes per-attempt echoes with MergeEcho, stops with ctx.Err() when cancelled during backoff, and treats signed/redacted thinking and tool-use signatures as committing output. The llm_usage ledger stays at one row per physical call.
2026-11-15 16:14:35 +01:00
BaaRai fdba29f1a9 ci: run gofmt in the go job, and format the 11 files it flags
Mechanical: gofmt -w on the 11 files gofmt -l listed on main. Besides spacing and alignment, gofmt sorted the imports inside one group in two test files and removed a trailing blank line in a third. The go job now fails, listing the files, when gofmt -l reports any tracked Go file.
2026-11-15 13:38:35 +01:00
BaaRai c2d1f557c3 perf(db): per-task metrics for the task detail and an index-served latest activity
getTask computed the list metrics of every task to read one; it now calls TaskListMetrics(expID), which runs the same laterals as TaskListMetricsAll with a WHERE on the exploration. listTasks keeps the global query. The latest_activity lateral ordered by its EXTRACT(EPOCH) output alias, so PostgreSQL read and sorted every activity row of each task; it now orders by activity.created_at and idx_act_latest serves the LIMIT 1. Both queries share one SQL constant.
2026-11-15 08:20:08 +01:00
BaaRai ac6e2b3c44 feat(egress): change the noise profile at runtime
The egress governor is now a current governor that SetProfile swaps: new requests use the new one, requests in flight finish on the old one, and one package-level ticker logs the counters of the current governor. WebFetch, the http custom tools and the enrich probe resolve it per request, and the agent prompt block reads the current profile. The noise_profile setting (off, quiet, balanced, aggressive) is applied at startup and wins over ARTEX_NOISE_PROFILE, which stays the fallback; PUT /api/settings validates, stores and applies it without rebuilding agents, and GET /api/settings/noise returns the per-host counters without resetting them. System settings gets a Target noise card.
2026-11-14 23:30:06 +01:00
BaaRai 8f5a67d10c fix(intercept): destructive commands score high; chunk batch decisions to the server limit
The destructive shell signal is worth 60 instead of 50, so a destructive command alone scores 10 + 60 = 70, high. The level thresholds are unchanged. The approvals page sends the selection in sequential calls of at most 100 ids, the server limit (maxBatchDecide), instead of one call that the server would refuse with 400 when more than 100 rows are selected. If a call fails, sending stops: its ids and the later ones stay selected with the error, and ids already decided stay decided. The final toast gives the totals.
2026-11-14 22:46:18 +01:00
BaaRai d8f3c7aec2 feat(intercept): deterministic risk score and batch approval
ScoreRisk scores a pending tool call from fixed, additive signals (no model): shell destructive, privilege, secret, exfiltration, reverse shell, scanner and file-write patterns; file writes with a sensitive-path bonus; web reads with a mutating-method bonus; MCP, unknown tools and unreadable input never score low. The score is computed when a pending row is read and returned as risk next to the unchanged row fields. POST /api/intercept/pending/batch-decide decides up to 100 ids, each through Interceptor.Decide with the same approver and address as a single decision, and reports a result per id. The approvals page gets row selection, a confirmation that counts high-risk rows, per-id errors, and a risk badge with its reasons.
2026-11-14 22:19:09 +01:00
BaaRai 8c8ae97400 test(server): join every live test server before its TempDir cleanup
New restores every unfinished task in the shared database and runs its planner, worker and deadline loops on the server context. Tests that passed context.Background() never stopped or joined them, so they kept writing under the manager TempDir after the test returned and the TempDir cleanup failed with directory not empty. Add stopTestServer, which cancels the context, waits for each task to go quiescent, joins its loops, the archive worker and the side-question writer, and defer it right after New in every test that started a live server.
2026-11-14 18:21:52 +01:00
BaaRai 149151919f test(server): join restored task loops before TestTaskDetailReturnsListMetrics cleanup
New restores every unfinished task in the shared database and runs its planner and worker loops on the server context. They kept writing transcripts under the TempDir after the test returned, so the TempDir cleanup failed with directory not empty. Cancel the context, wait for each task to go quiescent, and join its loops before m.Close and the TempDir cleanup.
2026-11-14 16:21:31 +01:00
BaaRai 5cdb720122 test(server): stop leaked fixtures from polluting later tests
TestMgmtAPI never closed its manager, so the notifier and task-archive workers started by New(context.Background()) kept claiming other tests digest batches and archive jobs. TestTaskOutcomeCard left a skill_usage miss row behind (the table has no foreign keys), which outranked TestSkillGapsReport fixture from the fourth -count iteration on.
2026-11-14 15:44:52 +01:00
BaaRai 54ff51177d agent: guard prompt and wrap-up override hooks against concurrent access
PromptOverride, WrapupOverride, WrapupMaxTurnsOverride,
WrapupTaskTimeoutOverride and WrapupTaskTimeoutTurnsOverride are wired
in server.New while restored tasks may already render prompts and build
settlement configs on other goroutines. go test -race ./server reported
races between server.New (server.go:195/207/215) and agent.renderSystem /
resolveWrapup / resolveWrapupTurns.

Same treatment as the tool hooks: unexported variables behind a
sync.RWMutex (one for the prompt hook, one shared by the four wrap-up
hooks), Set* setters and getters, and readers that copy the function
value under the lock and call it outside. A nil hook still means the
built-in default.

TestAgentHooksConcurrentSwap now covers all eight hooks.
2026-11-14 14:19:13 +01:00
BaaRai 37f2573a53 agent: guard ToolAugment/ToolResolve hooks against concurrent access
server wiring (wireAgentAugment, wireTools) assigned the package-level
hooks agent.ToolAugment and agent.ToolResolve while restored tasks could
already be running AugmentTools on other goroutines. go test -race ./server
reported this as a DATA RACE between agent.AugmentTools and
server.wireAgentAugment.

Each hook is now an unexported variable behind a sync.RWMutex, with a
setter (SetToolAugment, SetToolResolve) and a getter (ToolAugment(),
ToolResolve()). AugmentTools copies the function value through the getter
and calls it outside the lock. A nil hook still means no augmentation /
no resolution, as before.

FindingTrafficBindingEnabled is wired in the same server/assembly.go code
and read on the same AugmentTools path, so it gets the same treatment.

Adds TestAugmentToolsHooksConcurrentSwap, which runs AugmentTools while
the setters swap the hooks, so -race catches an unsynchronised access
without depending on test ordering.
2026-11-14 13:46:03 +01:00
BaaRai fcee6af45e secrets: encrypt the evidence snapshot heads at rest
traffic_evidence_snapshots.req_head and resp_head join the secret column registry (AAD per column, enc:v1:). Heads are sealed at insert and opened on every read; the startup key check, the no-key refusal and the idempotent backfill cover them. Archives keep the heads sealed; the import opens them to check the snapshot ID and the insert seals them once.
2026-11-13 22:56:52 +01:00
BaaRai 7899662aee secrets: keep a masked tool secret only for its saved destination
As for the MCP env (#213), a placeholder in a custom tool's exec keeps the
stored header, proxy or body only while the url (exact string) and the kind
are unchanged. A changed or emptied url, or a kind change, with a resolved
placeholder is refused (400 on the API and the test endpoint, a tool error
for update_custom_tool) before any write and before the test request is
sent. An explicit value with a new url is allowed.
2026-11-13 21:58:19 +01:00
BaaRai 68976a7680 secrets: encrypt the exec spec of custom tools at rest
Phase 3 of secrets at rest (#213, #216), same AES-256-GCM scheme and key.

- tools.exec joins the secret columns for system=false rows (AAD
  tools.exec): the whole JSON is sealed in the {"__artex_enc":"enc:v1:..."}
  envelope, so headers, body, proxy, script code and command are covered;
  {} stays {}
- CreateCustomTool and UpdateCustomTool seal it; scanTool opens it, so
  GetTool, ListTools and ListCustomTools (and through them customTools,
  the platform tools and the test endpoint) see the clear JSON in memory
- the startup backfill, the no-key refusal and the wrong-key check cover
  it, for system=false rows only; built-in rows stay in clear for the seeds

API: GET /api/tools returns the header values, proxy and body of a custom
tool masked, with headers_set, proxy_set and body_set. On update (API and
update_custom_tool) the placeholder keeps the stored value, an explicit
value replaces it and "" clears it; on create (API and create_custom_tool)
a placeholder is refused. The test endpoint takes an optional key: the
saved tool's stored values replace the placeholders; the editor sends it.
2026-11-13 21:52:27 +01:00
BaaRai dc169b074f secrets: encrypt search keys, proxies and the LLM profile proxy at rest
Phase 2 of secrets at rest (#213), same AES-256-GCM scheme and key.

- llm_profiles.proxy joins the secret columns (AAD llm_profiles.proxy);
  scanProfile decrypts it for both column lists, SaveProfile encrypts
  it, an empty proxy stays NULL
- brave_search_api_key, tavily_search_api_key, web_search_proxy and
  global_proxy are stored encrypted (AAD settings.<key>) through
  GetSetting, SetSetting and InsertSettingIfAbsent; other settings stay
  in clear
- the startup backfill, the no-key refusal and the wrong-key check
  cover the new values; legacy plaintext is read as-is until backfill

API: the proxies are returned masked as a whole (placeholder plus a
*_set flag) by GET /api/settings, GET /api/llm/profiles and GET /api/llm.
On write the placeholder keeps the stored value, an explicit value
replaces it and "" clears it; the placeholder is never stored. The test
and model-list endpoints resolve the placeholder to the stored proxy.
2026-11-13 21:33:27 +01:00
BaaRai 4f785a8b4f Guard masked MCP env against a destination change
A masked env value keeps the stored secret only while transport, command, args (canonical JSON) and url are unchanged, as notify.PrepareConfigUpdate does for channels. A changed destination with a masked value is refused (400). The credentials tab drops reveal/copy and shows a stored password as "stored, hidden".
2026-11-13 15:02:47 +01:00
BaaRai 07cc87c1a3 Encrypt stored secrets at rest in PostgreSQL
Seal the four credential columns with AES-256-GCM (random nonce per value,
AAD "<table>.<column>"): llm_profiles.api_key and task_credentials.password
store "enc:v1:<base64(nonce||ciphertext)>"; mcp_servers.env and
notification_channels.config store {"__artex_enc":"enc:v1:..."} over the
canonical JSON. The db layer encrypts on write and decrypts on read, so agent
tools (LLM provider, MCP, notifier, test credentials) get plaintext
server-side. Decryption failure is an error, never a plaintext fallback.

Key: ARTEX_SECRET_KEY (32 bytes, base64 or hex) or keys/secret.key next to
jwt.key, generated with crypto/rand at first start (0600). Startup refuses
when encrypted values exist and no key is available, and when the key does
not decrypt them. An idempotent backfill under the schema advisory lock
encrypts legacy plaintext rows and logs only a count.

API: MCP env is masked on read and masked placeholders on write keep the
stored value; task credential passwords are masked server-side and a masked
password on write is refused. ARTEX_SECRET_KEY is scrubbed from the process
environment with the other startup secrets.
2026-11-13 14:42:56 +01:00
BaaRai 62e65f1235 Intercept: always allow a pending Bash prefix (argv allow rule, code-owned denylist)
POST /api/intercept/pending/{id}/always-allow derives an argv allow rule from the stored pending row (never from the body), at priority -1000, reusing an identical enabled rule, and leaves the request pending. A code-owned denylist of interpreters, launchers, package managers, orchestration and destructive programs refuses such rules at creation, in the manual rule form, and neutralises argv allow rules at matching; argv deny and ask rules still match.
2026-11-13 11:24:25 +01:00
BaaRai 2457a1f103 Intercept: argv prefix rules (match_type=argv) 2026-11-13 10:14:39 +01:00
BaaRai 06808396cc Intercept judge: review fixes for the closed-question mode
- A destructive or operator_policy unknown asks, even when read_only is yes.
- An enabled closed judge with no completer wired asks (judge_not_wired)
  instead of falling through to allow.
- The argument cap is checked before the review input is built.
- MaxTokens for the closed reply raised to 1536.
- The operator_policy question also covers a whitelist policy that fails to
  authorize the action; golden digest updated.
2026-11-13 08:40:02 +01:00
BaaRai 9a927846ef Intercept judge: closed-question mode, fail-closed to ask
The LLM fallback judge gets a closed mode (default when the judge is on):
the model answers eight code-owned questions with yes|no|unknown and one
evidence line each, in one call; the decision is taken in code. Every
failure (invalid output, missing key, value outside the vocabulary, a
failed call, arguments above the 256 KiB cap) is ask. The custom judge
prompt becomes bounded operator-policy data (8000 runes). The answers are
stored in the intercept audit and shown in the approval detail.

freeform keeps the previous behaviour; defaultJudgeFailAction changes
from allow to ask.
2026-11-12 22:35:11 +01:00
BaaRai 101c92d516 Coverage ledger: review fixes (one-line capped fields, framed gap lines, per-run end_reason)
Normalise and length-check ledger fields in db.RecordCoverage, frame the planner gap lines in the data envelope, cap the wrap-up (25) and report (200) gap lists, cap current rows at 2000 per task. Replace the activity budget marker with tasks.end_reason, written with the guarded terminal status and cleared on revive. Generic 500 message in taskCoverage; early return on an empty ledger in the planner block.
2026-11-12 20:48:34 +01:00
BaaRai 6f3a59e205 Coverage ledger: record examined, ruled-out and untested risk classes
Workers record per (surface, risk_area) outcomes with the new record_coverage
tool (reported, no_issue_found, ruled_out, not_applicable, needs_follow_up).
Gaps are needs_follow_up entries, entries missing their required evidence or
reason, and goal vulnclass values with no entry (unrecorded_risk_class), so an
absent finding never reads as a clean target. Completeness marks timeout,
budget wrap-up, pause, abort and running tasks as incomplete with a caveat.

Surfaced in the planner context (one line per round, full gap list on the
timeout/budget wrap-up round), GET /api/tasks/{id}/coverage, the coverage card
and the Markdown report. Follows coverage_enabled; no new env var.
2026-11-12 20:16:23 +01:00
BaaRai 4cde1a889f Budgets: revive clears the settle state, document config, allow update_finding_report in wrap-up
- resetSettledRun clears settling and the budget ledger; a done/failed task
  revived to running now plans and claims again (was gated by the old wrap-up).
  resetTimeoutRevival calls it. Test: TestResetSettledRunRestartsBudget.
- docker-compose.yml passes the four ARTEX_*_BUDGET_* variables through.
- .env.example and README document the variables, the wrap-up, the in-memory
  ledger and the practical floor.
- llmrec: the usage sink runs even without a PostgreSQL recorder; comment fixed.
- BudgetWrapupBlocks allows update_finding_report.
2026-11-12 18:48:41 +01:00
BaaRai 16ddfc7779 Hierarchical run budgets: min(child, parent) with forced wrap-up (off by default)
Per-level token/turn budgets: task -> intent (worker run) -> planner round
(counted inside the task). A child's effective budget is
min(own budget, parent's remaining above the wrap-up reserve). When the task
can no longer afford one worker, workers stop claiming intents and the task
settles through one forced wrap-up planner round (the existing settle
sequence, budget words instead of the timeout words).

Config: ARTEX_TASK_BUDGET_TOKENS, ARTEX_TASK_BUDGET_TURNS,
ARTEX_INTENT_BUDGET_TOKENS, ARTEX_INTENT_BUDGET_TURNS. 0 or unset = unlimited;
with all four unset nothing changes.
2026-11-12 17:27:28 +01:00
BaaRai d116517c64 Egress: return large unshared responses without an error, namespace the transport cache per client, pace WebFetch only 2026-11-12 16:44:08 +01:00
BaaRai 0b59f4670b Egress governor: noise profiles (off by default), per-host rate limit, dedupe and short cache
ARTEX_NOISE_PROFILE = off | quiet | balanced | aggressive drives one shared
governor used by WebFetch, the http custom tools and the enrich probe: token
bucket, bounded jitter and concurrency per host, a global cap, backoff on
429/503 honouring Retry-After with bounded retries, per-task coalescing and
a short cache of identical GETs (never POST/PUT/PATCH/DELETE), and a
per-host noise log line each minute. Off (the default) is a strict
passthrough.

Removes the fixed artex-enrich/1.0 User-Agent; ARTEX_TARGET_USER_AGENT sets
one under a profile (empty = unchanged).
2026-11-12 16:30:56 +01:00
BaaRai e59db80173 Opt-in update banner, off by default
ARTEX_UPDATE_CHECK=1 lets the backend check the public GitHub releases API
for the latest tag at most once per 24 h (in-memory cache; failures are not
retried in the window) and report it in GET /api/health when it is newer
than the build version. The web UI shows a dismissible strip linking to the
Releases page. Unset (the default), no update request is made.
2026-11-12 12:54:25 +01:00
BaaRai a7e91c81e6 fix(ask_operator): seed the goal targets with bare goal domains (F12)
The goal gate accepts a goal that names its target as a bare domain (goalDomainRE), but the in-goal set was built only with commandTargets, which does not read bare hosts. A question citing the operator's own target as a URL was then tagged target-derived. goalTargets adds the goalDomainRE matches, widened to the whole dotted name, lowercased, port and path dropped. The question-side parser is unchanged.
2026-11-12 11:51:38 +01:00
BaaRai 6dbcd33552 feat(ask_operator): tag requests that name hosts outside the goal (F12)
An ask_operator question that names a host or URL not among the task's
goal targets is tagged "contains target-derived text": the hosts go into
the transcript activity's Detail payload as target_derived_hosts, and the
transcript shows the tag on the request card when the field is set.

Hosts are read with intercept.ExtractTargets (the target gate's
command-string rules) from both the question and the task goal, and
compared without the port. The text the model reads, the pause, and the
operator's decision are unchanged: the tag is a visible hint only.

Audit F12, 0.7.0.
2026-11-12 11:12:22 +01:00
BaaRai 88306a0567 fix(tools): migrate the add_task_scope description on existing databases (F8)
add_task_scope is seeded through BuiltinToolSeeds, SeedTool inserts only
once, and ToolResolve serves the stored row, so existing installs kept the
old "this is the task's authorization boundary" text.

seedTaskScopeDescription runs from seedOrchestrationTools and updates the
description only:

  UPDATE tools SET description=$new, updated_at=now()
  WHERE key='add_task_scope' AND system AND description=$old

$old is the description as seeded at 0dc8bdf, kept verbatim; $new is the
code description from BuiltinToolSeeds. It is idempotent, a no-op when the
row is absent, and leaves an operator-edited description alone. It does not
use RefreshToolDefaults, which overwrites without condition.

TestTaskScopeDescriptionUpgrade (server, PG) covers the replacement, the
preserved custom text, an unchanged updated_at on a second run, and the
description ToolResolve serves to the planner.
2026-11-12 09:38:52 +01:00
BaaRai bc33db2735 server: add the record_fact source enum to existing databases
SeedTool is first-insert only and ToolResolve serves the stored tools row,
so on an existing database the record_fact schema never gained the source
property: the model never saw the enum, every fact was stored as inference,
and prove_goal's target_response refusal could not fire.

seedFactSourceSchema copies properties.source from the record_fact seed into
the stored schema, only when the stored schema lacks it (same pattern as
seedFindingEvidenceSchema). Operator edits to the description and to other
properties are preserved, and a second run changes nothing. It runs from
seedOrchestrationTools next to the other schema seeders.

The PG test edits the stored row (no source, custom description and
parameter text), runs seedOrchestrationTools twice, and checks the added
enum, the preserved operator text, the unchanged row on the second run,
and that ToolResolve serves the source property to the worker.

Refs #124 (F4).
2026-11-12 01:09:30 +01:00
BaaRai 8c9c1fc0ab feat(findings): optional counterevidence and confidence_rationale
Add two optional free-text fields to findings so a reporter can record
what was tested and did not work (counterevidence) and why the
conclusion holds (confidence_rationale).

- db: idempotent ADD COLUMN IF NOT EXISTS, TEXT NOT NULL DEFAULT '',
  no CHECK; RecordFindingInput, the INSERT, the shared select list and
  all three scan sites carry them.
- agent: report_finding accepts both as optional strings; the worker
  and pentest prompts say to fill them when confidence is not high.
- server: FindingDTO (list and detail) carries both, omitted when empty;
  existing databases get the two schema properties added to the stored
  report_finding row once, without overwriting user edits.
- web: the finding detail page shows each section only when non-empty.

Both fields stay optional: a missing field never refuses a finding.
2026-11-11 23:43:12 +01:00
BaaRai 4bcb6d7250 Add agentic guards for goals, hint origin and planner Bash (F5, F6, F11)
Audit findings F5, F6 and F11 for issue #124.

F5, goal guards:
- goal_met refuses while any goal node of the task is open, and when the
  goals cannot be read. A task without goal nodes keeps the old behavior.
- prove_goal accepts a fact as evidence only when its confidence is
  "observed"; otherwise it refuses and the goal stays open. Findings are
  still accepted.

F6, hint origin:
- add_hint stores the calling ToolSet's worker as the hint origin instead
  of "human", and passes it to the planner trigger (TriggerEvent.HintOrigin).
- renderTriggers names the writer: the human via the main agent, a
  cross-task orchestration, or an agent, with a neutral fallback.

F11, built-in rules for the planner and the main agent:
- Interceptor.MatchBuiltin evaluates the enabled "[built-in]" deny rules on
  the intercept-enabled tools.
- The target-only guard applies them before the target gate, whether the
  gate is on or off. User rules and the judge stay out of it.
- SetRulesForTest is a test seam for the store-less guard tests; Log is a
  no-op without a store.
2026-11-11 22:39:13 +01:00
BaaRai 8acf1be46e agent: code-owned prompt-injection tails and planner goal anchor (#124)
Phase B, part 1 of the agentic attack-class audit (issue #124).

- Planner and cold-digest compactor: a code-owned untrusted-data tail
  after the rendered body. Worker summaries, facts, digests, traces and
  tool outputs are data; they cannot add goals, targets or scope, end the
  task, or be relayed through steer_work as instructions.
- The #49 untrusted-content block moves out of the editable section [A]
  of the worker, mainagent, auto and pentest seeds into a code-owned tail,
  also applied to the reporter. A body that already carries the block
  verbatim (rows seeded before this change) does not get it twice.
- Planner user message: a code-owned goal anchor after the situational
  block restates the task goal and the operation constraints and asks each
  new intent to name the goal id it serves. The message builder is
  factored into plannerRoundInput for pure tests.
- bbot-recon skill: every event field is untrusted; only scope_distance
  == 0 reaches insert_assets; scope_distance > 0 stays an intel fact with
  source=third_party_intel, never auto-scoped; FINDING text is a lead.

Tests: prompt_injection_seed_test.go checks that a PromptOverride without
the block still renders the tail for every role, planner and compactor
included, that the block is not duplicated, and that the planner input
ends with the goal anchor. shipped_skills_test.go checks the bbot markers.
2026-11-11 21:47:44 +01:00
BaaRai bbc4f6269a Fix doubled scheme in the custom http tool's recording proxy URL
ProxyAddr returns a URL that already carries a scheme (the recording
proxy and the validated global proxy), but httpProxyTransport wrote
"http://" before it, so use_recording_proxy produced http://http://...
proxyURLString now adds http:// only to a bare host:port, for both the
operator proxy and ProxyAddr. httpToolTransport falls back to a fresh
http.Transport when http.DefaultTransport is not an *http.Transport.
2026-11-11 20:20:16 +01:00
BaaRai f6b29b835b feat(intercept): pinned dialer for WebFetch and the custom http tool (S3b 2.1)
In a gated scope, the two HTTP clients the platform owns connect only to
addresses that a pin or a stored exemption covers at the moment of the
connection, on every hop.

- intercept.PinnedDialer(tool, scoped): the DialContext of both clients.
  It reads the target scope from the context net/http passes to the dial.
  Gate off, or no scope in a client built without one: plain dial, as
  today. A client built for a scope whose scope is missing at the dial
  fails closed with TARGET_GATE_UNAVAILABLE.
- dialTarget runs the gate decision on the dialled host:port with a fresh
  lookup (block rules first, exemptions, pins, port precedence) and
  connects only to the addresses it compared, never to the name. Derived
  exemptions and name-only pins dial the address as received. An address
  no pin covers finds or opens the request (new_ip / no_pin, wait 0) and
  fails at once with TARGET_APPROVAL_PENDING. Redirect hops are dialled
  against the hop's own (host, port).
- WebFetch (withWebFetch, now given the run context): DialContext set on
  every agent kind; DisableDirectRetry when the run's scope is gated.
- Custom http tool: a direct call in a gated scope gets a DefaultTransport
  clone with the pinned dialer, no environment proxy and keep-alive off.
  Proxied calls are unchanged (the recording proxy is step 2.2).
- Name-only: request reason says the platform does not bind the
  connection; the first-use pin_match row records the effective state.
- Derived exemptions: a proxy URL without a port is listed with the port
  net/http dials (1080 for socks5) (OPEN-9).
- go.mod: Norma v0.4.8 (DialContext, DisableDirectRetry).
2026-11-11 19:19:55 +01:00
BaaRai 1c3a5f4ffa feat(intercept): check every redirect hop against the target gate (S3b 1.9)
WebFetch and the custom http tool followed redirects with net/http's
default policy, so a pinned host could send the client to a target nobody
approved. RedirectCheck is now the CheckRedirect policy of both clients.

- Gate off, or no scope (operator test endpoint): only the 10-hop limit,
  same error text; nothing is parsed, resolved, read or written.
- Gate on: each hop runs through the call's decision code (URL rules on
  the hop URL, block rules before any lookup, exemptions, resolution,
  pins). A hop no pin covers is not followed and never waits: its request
  row is found or created and the call fails with TARGET_APPROVAL_PENDING.
  http -> https asks again for port 443 unless a (host, 0) pin exists. An
  unreadable hop is blocked with TARGET_GATE_UNAVAILABLE.
- Rows raised by a hop record redirect_from in the audit targets.
- Norma v0.4.7 has WebFetchConfig.CheckRedirect but no agentcore option,
  so the four agent kinds build WebFetch themselves (same proxy, CA and
  position in the tool list) instead of EnableWebFetch. Norma carries the
  policy to the direct retry client.
2026-11-11 17:07:35 +01:00
BaaRai c5e12541bd Merge pull request #183 from BaaRai/fix/target-gate-declared-first
fix(intercept): match block rules on the declared targets before any lookup
2026-11-11 16:05:15 +01:00
BaaRai ca15063db7 fix(intercept): match block rules on the declared targets before any lookup
The target gate resolved each name through the cache before it matched the
asset block rules, so a call to a name that a name rule blocks still cost a
DNS lookup, and that query reached the name servers of the blocked domain.
URL rules (exact_url, fuzzy_url) were not evaluated at dispatch at all: the
gate passed no URL to the matcher, so they only applied to asset and intent
writes.

targetCheck.run now decides in this order, for a gated call:

1. URL rules, once per decision, on the URLs the call declares, as written.
2. Every block rule on the declared form of every target: the name as
   written, the IP literal, the range.
3. Only then resolution (names a derived exemption covers are still never
   resolved), and the block rules again on the resolved addresses.
4. Exemptions, pins and requests, unchanged.

A hit in 1 or 2 blocks with TARGET_RULE_BLOCKED and the same decided row as
any block, costs no lookup for any target of the call, and opens no request.

- intercept.extractDeclared returns the targets (ExtractTargets is now a thin
  wrapper, unchanged for its callers) and the URLs that name a target: the
  WebFetch url, and every URL token the command-string rules read in Bash,
  shell_open, Monitor, shell_send and the string leaves of other tools,
  including ExecuteExtraTool's inner call.
- The custom http tool's rendered URL and the URLs of a rendered command go to
  the same URL rules (gateRendered takes the URLs).
- Known limit, stated in the comments next to the matcher (db.matchOne) and
  at the dispatch match: URL rules match the declared string. exact_url misses
  any other spelling (case, percent-encoding, trailing slash, added query,
  /./), fuzzy_url misses an encoded substring, and a command can build its URL
  at run time. For a host, a domain, IP or CIDR rule is the one to write.

Unchanged: a tool-rule or judge deny still blocks before the gate is reached
(no extraction, no lookup, no request); asset block rules still win over an
allow and an ask; the gate off path does nothing; scripts stay advisory.

Tests:
- intercept: TestBlockRuleOnDeclaredNameNeedsNoLookup (exact_domain and
  fuzzy_domain; zero lookups, also for an unblocked name earlier in the same
  call), TestExtractDeclaredURLs, TestURLBlockRuleAtDispatch (WebFetch, Bash,
  MCP, ExecuteExtraTool, a pinned and an exempt host, a disabled rule, the
  rendered URL and command; zero lookups), TestExactURLRuleMatchesOnlyAsWritten
  (documents the limit).
- guard: TestBlockRuleWinsOverAllowAndAsk (name rule and URL rule, each over a
  rule allow, judge allow, rule ask and judge ask, real interceptor: blocked,
  no allow row, nobody asked, zero lookups). TestDenyRuleWinsBeforeTargetRequest
  now also covers a judge deny and asserts zero lookups with the real gate.
- server: TestBypass_CustomScriptToolTargetNotGated documents that a script
  tool's target, written in its stored code, is not visible to the gate: in a
  gated task the call is allowed and reaches the target with no lookup, no
  request and no row.
2026-11-11 15:47:58 +01:00
BaaRai f07e84ac7a feat(intercept): spawned tasks and triggered conversations inherit the target gate
spawn_task now reads the target-gate scope of the calling session from the
tool context. When the caller is gated, the child task is created with the
gate on (D23) and with a copy of the calling task's block rules (D24). Pins
and allow rules are never copied, parent_ref is not consulted, and spawn_task
gets no new parameter. A gate-off caller, or a call with no scope, creates the
child exactly as before.

A conversation started by a trigger of a gated task is created with the gate
on (D25). A merged run is gated if any of its fires came from a gated task.
Pins are never copied. Fires of a gate-off task and fires with no task are
unchanged.

TestSpawnedTaskStartsWithGateOff becomes TestSpawnedTaskInheritsGateFromCaller.
New: TestSpawnTaskSeesCallerScopeThroughHarness and
TestTriggeredConversationInheritsGate.
2026-11-11 15:47:21 +01:00
BaaRai c0b741bcf6 feat(intercept): gate the rendered targets of custom tools (S3b 1.8)
The hook reads the parameters of a custom tool call, not the URL or the
command they render, and not a target written in the template itself. The
custom command and http tools now run the target gate inside the tool, on what
they are about to run, before anything is sent or started. Gate off (the
default) is unchanged: with no scope or the flag off nothing is parsed,
resolved, read or written.

- intercept.GateRenderedURL: the target is read as net/http dials it (host of
  the parsed URL, explicit port, else the scheme default), then goes through
  the same decision as at the hook (asset block rules, exemptions, resolution,
  pin lookup) and the same request-and-wait, at most three passes. A URL whose
  target cannot be read is blocked with TARGET_GATE_UNAVAILABLE and logged as
  a decided row; no request, no lookup.
- intercept.GateRenderedCommand: the rendered command is read with the
  command-string rules of Bash, and so is the rendering without the shell
  quoting of the parameters (a quoted parameter inside a host name hides the
  name from the rules). A command that names no target passes, as in Bash.
- A gated scope without the interceptor that owns the gate blocks
  (TARGET_GATE_UNAVAILABLE), it never lets the call through.
- runCommandTool and runHTTPTool take the tool key, which names the request
  rows. The gate runs before the tool's own timeout starts.
- guard.TargetBlockMessage (was targetBlockMessage) is exported so the tools
  return the same model-facing text as the hook.
- Unchanged: the script tool (its target is in stored code; advisory, D9), the
  custom-tool test endpoint (no scope: the operator's own action, D14), the
  hook order (a rule or judge deny blocks first; the tool then never runs, so
  its rendered target is never read), the evaluation of asset rules.

Tests: TestRendered* in intercept (no database); in server, against
PostgreSQL: TestCustomHTTPRenderedTargetGated,
TestCustomCommandRenderedTargetGated, TestCustomToolTestEndpointUngated,
TestTargetGateEndToEnd (a scripted worker's WebFetch waits, a click pins, the
next call passes, revoke, it asks again), TestDenyRuleWinsBeforeRenderedTarget
and TestGateOffTaskIsUnchanged.

Not in this change: redirect targets of WebFetch and of the custom http tool
(1.9).
2026-11-11 15:04:42 +01:00
BaaRai 78f92f44fc feat(intercept): target extraction, resolver, exemptions and the gate in the hook (S3b 1.5-1.7)
The approval gate for asset targets now decides. It is opt-in per task and
per conversation (target_gate, default off). With the flag off nothing is
extracted, resolved, read or written, and every branch of the hook behaves as
before. No UI.

Extraction and normalisation (1.5)
- intercept.NormalizeTarget: lower case, trailing dot, userinfo, IPv6
  brackets, IPv4-mapped IPv6, decimal / hex / octal / short IPv4, percent-
  encoded host, punycode; port explicit, else the scheme default, else 0.
- intercept.ExtractTargets: WebFetch url exactly; Bash, shell_open, Monitor
  (command) and shell_send (text) with the command-string rules (URL with a
  scheme, IP, IP:port, host:port, CIDR; a bare name is not a target); every
  string of the input for MCP tools, custom tools and any tool the gate does
  not know. ExecuteExtraTool is unwrapped to the inner tool. The platform's
  domain tools, the file tools and web_search have no target.
- The fields are decoded as the tools decode them (key case, last duplicate),
  and a URL authority that reads differently quoted and unquoted yields both
  readings.

Resolver, exemptions, gate decision (1.6)
- Resolver interface, system implementation (A and AAAA), 3 s per lookup,
  positive answers cached 10 s per host with collapsed lookups, failures and
  empty answers never cached, always fresh when a request is raised.
- Exemptions are exact host + port. Derived from the configuration (LLM
  gateways, search backend in use, platform proxies): computed on every call,
  never resolved, never voided. Stored (settings key
  target_gate_exempt_hosts, seeded once with the package sources): resolved,
  and void when any address is private, loopback, link-local, unspecified,
  unique-local, multicast or a metadata address. An entry in both origins
  counts as derived. A stored IP literal that can never be exempt does not
  apply.
- GET / PUT /api/target-gate/exemptions and GET .../history. A change is
  validated (ValidateExemptionHost on the normalised host) and appended to
  target_gate_exemption_log in the transaction that stores it.
- Interceptor.CheckTargets: asset block rules (global and of the task) on the
  name and its addresses, exemptions, resolution, pin lookup. A (host, 0) pin
  covers every port; a (host, 443) pin covers 443 only and never a target
  without a declared port; two pins are never merged. At most 5 unpinned
  targets per call. Codes: TARGET_RULE_BLOCKED, TARGET_RESOLVE_FAILED,
  TARGET_TOO_MANY, TARGET_GATE_UNAVAILABLE; none opens a request.
- With the upstream egress proxy set every pin is name-only at call time,
  whatever name_only_effective says in its row.
- An approval never pins zero addresses: the pin writer refuses a request
  that displays none (db.ErrTargetPinEmpty, 422 on the decide endpoint, the
  row stays pending) and RequestTargets does not open one.
- Every block, and the first use per process of each exemption and of each
  pin version, is a decided row (decided_by system, decision_source target).

The gate in the hook (1.7)
- guard.applyIntercept: a deny by a rule or by the judge blocks first, before
  any target request exists; then CheckTargets; an allow is logged as before
  and does not satisfy the gate; an ask and an unpinned target share one
  request, and the classic ask is raised only when it was not merged. The
  call is checked again after the decisions, at most three passes.
- The gate applies to every tool of a gated session, enabled for rules or
  not, and to the target-only guard of the planner and the main agent, which
  still records blocks only.
- Target blocks have their own model-facing text, built from the code and the
  reason of the result.

Not in this change: rendered targets of custom tools (1.8) and redirect
targets (1.9). Subprocess paths stay advisory; the TestBypass_* tests document
what the gate does not see.
2026-11-11 00:34:55 +01:00
BaaRai 1f4cb65ca6 feat(intercept): target-gate schema, opt-in flag and approval requests (S3b 1.2-1.4)
Storage, flag and request path of the approval gate for asset targets. The
gate itself is not wired into the tool hook yet (steps 1.5 to 1.7): with this
change nothing is extracted, resolved or blocked, whatever the flag says.

Schema and store (1.2)
- tasks.target_gate, conversations.target_gate (default false).
- intercept_pending.target_key and a partial unique index: one open request
  per scope (task or conversation) and host:port.
- target_pins (no foreign keys: a pin outlives its task, its conversation and
  its request row) and target_gate_exemption_log. All DDL is idempotent; no
  CHECK constraints; the cascade on intercept_pending.conversation_id stays.
- db/target_pins.go: pin lookup with port precedence, list, revoke,
  find-or-create of a request, and ResolveTargetRequest, which settles the
  row, supersedes the old pin and writes the new version in one transaction.
- Deleting a task or an archive stub revokes its pins and settles its open
  target requests in the delete transaction. DeleteConversation and
  DeleteConversations revoke the pins of the removed conversations in the
  same transaction as the delete.

Opt-in flag (1.3)
- Per task through the create and metadata-patch paths, per conversation
  through create and patch; exposed in the task DTO. A retest conversation
  takes the flag of the finding's task, never its pins. A spawned task starts
  with the gate off.
- The task flag is atomic and read on every call through TargetScope.Enabled.
  TargetScope now carries the Interceptor that owns the gate.

Request, wait, decide (1.4)
- pendingManager keeps a list of waiters per request; resolve wakes all.
- RequestTargets: find-or-create per target, one card per request, wait on
  channel, poll, context and wait bound. A plain request is never settled by
  a call that leaves. A request merged with an ask is bound to its creating
  call and a timeout never allows it.
- Decide records the approver (subject of the session token) and the remote
  address for every human decision, classic asks included. On a target
  request "allowed" pins the displayed addresses; that is the only writer of
  pins.
- A request raised while the global egress proxy is set says the approval is
  name-only and its pin is stored with name_only_effective.
- Routes: GET/POST .../target-pins and .../target-pins/{pid}/revoke under
  /api/tasks/{id} and /api/conversations/{id}; decision_source=target filter.
- ValidateExemptionHost: an exemption written as an IP literal may be private
  unicast, never loopback, link-local or metadata, unspecified or multicast.
2026-11-10 23:05:47 +01:00
BaaRai 135b083b9a feat(guard): planner and main-agent hooks, target-gate scope in every session (S3b 1.1)
Step 1.1 of the S3b phase 1 plan. Wiring only: the approval gate for asset
targets is not enforced yet and no behaviour changes.

- Planner.Plan and MainAgent.Chat take a hooks parameter, like Worker.Execute.
  Their sessions had no hooks, so nothing saw their tool calls before execution.
- guard.NewTargetOnly: the guard for those two agent kinds. It runs the
  target-gate seam only: no intercept rule, no fallback judge, and no entry in
  the audit ring (Guard.record is untouched and not called for these calls).
- Task.TargetGuard, built beside Task.Guard.
- intercept.WithTargetScope / TargetScopeFromContext: task id or conversation
  id, an Enabled func and a wait bound, set at the five session entry points
  (planner round, final planner round with wait bound 0, main-agent turn,
  worker intent, conversation turn).

The seam allows every call and reads and writes nothing. No task or
conversation has an opt-in flag yet, so Enabled is nil (off) everywhere.
2026-11-10 22:14:24 +01:00