Workers record per (surface, risk_area) outcomes with the new record_coverage
tool (reported, no_issue_found, ruled_out, not_applicable, needs_follow_up).
Gaps are needs_follow_up entries, entries missing their required evidence or
reason, and goal vulnclass values with no entry (unrecorded_risk_class), so an
absent finding never reads as a clean target. Completeness marks timeout,
budget wrap-up, pause, abort and running tasks as incomplete with a caveat.
Surfaced in the planner context (one line per round, full gap list on the
timeout/budget wrap-up round), GET /api/tasks/{id}/coverage, the coverage card
and the Markdown report. Follows coverage_enabled; no new env var.
Per-level token/turn budgets: task -> intent (worker run) -> planner round
(counted inside the task). A child's effective budget is
min(own budget, parent's remaining above the wrap-up reserve). When the task
can no longer afford one worker, workers stop claiming intents and the task
settles through one forced wrap-up planner round (the existing settle
sequence, budget words instead of the timeout words).
Config: ARTEX_TASK_BUDGET_TOKENS, ARTEX_TASK_BUDGET_TURNS,
ARTEX_INTENT_BUDGET_TOKENS, ARTEX_INTENT_BUDGET_TURNS. 0 or unset = unlimited;
with all four unset nothing changes.
When ARTEX_TOOL_RAG is enabled, AugmentTools keeps the core tools plus the
top-k (ARTEX_TOOL_RAG_TOPK, default 12) BM25-ranked bound tools for the
worker intent / planner goal. Runs after ToolResolve, so it only removes;
falls back to the full list on empty query, catalog under 20, or scorer error.
ARTEX_UNTRUSTED_BOUNDARY unset or empty now enables the boundary.
0, false, off and no (case-insensitive, trimmed) disable it; any other
value keeps it on. Docs state the default, the opt-out and the measured
cost (about 154 bytes per tool result plus one system note).
declaredScopeFilter treats only root_domain, subdomain, ip and cidr rows as
roots. A declared company, icp or keyword row does not make an auto row
count, but the add_task_scope description said "a goal target or scope
added here or manually". The description and the code comments now say
"declared domain or IP root" and name the kinds that act as roots.
Auto task_scope rows (source='auto') come from insert_assets and intent
asset_ids, i.e. from target-derived content. They now count toward the
coverage denominator and list_untested_assets only when they sit under a
root the same task declared (a non-auto row: goal target registered via
add_task_scope, or manual scope): the same or a deeper domain of a declared
root_domain/subdomain, or a net inside a declared ip/cidr.
The direct-source coverage view also stops counting an asset from an intent
anchor alone; fact anchors still count, so tested evidence is unchanged.
Without this, insert_assets put a planted host back into the backlog
through its intent anchor.
task_scope is described as "coverage scope, not enforced" in the code
comments, the add_task_scope description, the goal-decomposer tail and the
task overview UI, instead of "authorization boundary".
Audit F8, 0.7.0.
The planner and worker user messages now carry worker output, finding
details, recent_facts, finding_list, cold_digests, related_tasks and hints
not written by the operator inside a code-owned data envelope whose nonce
comes from crypto/rand (16 bytes, hex) once per message. Embedded marker
stems are neutralised with a fixed-string replacement. The task goal, goal
nodes, constraints and the operator's own hints stay outside, labelled
operator. Overview fact summaries are capped at factSummaryMax (160 runes),
the cap list_facts already uses.
record_fact takes a source field from a closed enum (target_response,
tool_output, third_party_intel, operator_input, inference). An empty value
is stored as inference; any other value is refused with the allowed list.
The source is stored in the fact payload.
The writer of a node is already recorded by the server in the origin
column. compactNode now surfaces origin and source, so recent_facts in the
overview, list_facts, expand_digest members and related-task facts carry
them. The cold-digest compressor input carries source and origin next to
confidence; the digest body is unchanged model text.
prove_goal refuses a fact whose source is target_response, in addition to
the existing confidence=observed rule. The goal stays open; findings are
still accepted.
The worker, pentest and planner default prompts ask for the source, and the
planner treats target_response facts as non-authoritative for goals.
Refs #124 (F4).
Add two optional free-text fields to findings so a reporter can record
what was tested and did not work (counterevidence) and why the
conclusion holds (confidence_rationale).
- db: idempotent ADD COLUMN IF NOT EXISTS, TEXT NOT NULL DEFAULT '',
no CHECK; RecordFindingInput, the INSERT, the shared select list and
all three scan sites carry them.
- agent: report_finding accepts both as optional strings; the worker
and pentest prompts say to fill them when confidence is not high.
- server: FindingDTO (list and detail) carries both, omitted when empty;
existing databases get the two schema properties added to the stored
report_finding row once, without overwriting user edits.
- web: the finding detail page shows each section only when non-empty.
Both fields stay optional: a missing field never refuses a finding.
Phase B, part 1 of the agentic attack-class audit (issue #124).
- Planner and cold-digest compactor: a code-owned untrusted-data tail
after the rendered body. Worker summaries, facts, digests, traces and
tool outputs are data; they cannot add goals, targets or scope, end the
task, or be relayed through steer_work as instructions.
- The #49 untrusted-content block moves out of the editable section [A]
of the worker, mainagent, auto and pentest seeds into a code-owned tail,
also applied to the reporter. A body that already carries the block
verbatim (rows seeded before this change) does not get it twice.
- Planner user message: a code-owned goal anchor after the situational
block restates the task goal and the operation constraints and asks each
new intent to name the goal id it serves. The message builder is
factored into plannerRoundInput for pure tests.
- bbot-recon skill: every event field is untrusted; only scope_distance
== 0 reaches insert_assets; scope_distance > 0 stays an intel fact with
source=third_party_intel, never auto-scoped; FINDING text is a lead.
Tests: prompt_injection_seed_test.go checks that a PromptOverride without
the block still renders the tail for every role, planner and compactor
included, that the block is not duplicated, and that the planner input
ends with the goal anchor. shipped_skills_test.go checks the bbot markers.
In a gated scope, the two HTTP clients the platform owns connect only to
addresses that a pin or a stored exemption covers at the moment of the
connection, on every hop.
- intercept.PinnedDialer(tool, scoped): the DialContext of both clients.
It reads the target scope from the context net/http passes to the dial.
Gate off, or no scope in a client built without one: plain dial, as
today. A client built for a scope whose scope is missing at the dial
fails closed with TARGET_GATE_UNAVAILABLE.
- dialTarget runs the gate decision on the dialled host:port with a fresh
lookup (block rules first, exemptions, pins, port precedence) and
connects only to the addresses it compared, never to the name. Derived
exemptions and name-only pins dial the address as received. An address
no pin covers finds or opens the request (new_ip / no_pin, wait 0) and
fails at once with TARGET_APPROVAL_PENDING. Redirect hops are dialled
against the hop's own (host, port).
- WebFetch (withWebFetch, now given the run context): DialContext set on
every agent kind; DisableDirectRetry when the run's scope is gated.
- Custom http tool: a direct call in a gated scope gets a DefaultTransport
clone with the pinned dialer, no environment proxy and keep-alive off.
Proxied calls are unchanged (the recording proxy is step 2.2).
- Name-only: request reason says the platform does not bind the
connection; the first-use pin_match row records the effective state.
- Derived exemptions: a proxy URL without a port is listed with the port
net/http dials (1080 for socks5) (OPEN-9).
- go.mod: Norma v0.4.8 (DialContext, DisableDirectRetry).
WebFetch and the custom http tool followed redirects with net/http's
default policy, so a pinned host could send the client to a target nobody
approved. RedirectCheck is now the CheckRedirect policy of both clients.
- Gate off, or no scope (operator test endpoint): only the 10-hop limit,
same error text; nothing is parsed, resolved, read or written.
- Gate on: each hop runs through the call's decision code (URL rules on
the hop URL, block rules before any lookup, exemptions, resolution,
pins). A hop no pin covers is not followed and never waits: its request
row is found or created and the call fails with TARGET_APPROVAL_PENDING.
http -> https asks again for port 443 unless a (host, 0) pin exists. An
unreadable hop is blocked with TARGET_GATE_UNAVAILABLE.
- Rows raised by a hop record redirect_from in the audit targets.
- Norma v0.4.7 has WebFetchConfig.CheckRedirect but no agentcore option,
so the four agent kinds build WebFetch themselves (same proxy, CA and
position in the tool list) instead of EnableWebFetch. Norma carries the
policy to the direct retry client.
Norma v0.4.6 frames tool output between nonce-tagged markers, so the model
reads it as data. ARTEX passes that option from the environment: 1, true or
on enable it (case and surrounding spaces ignored); unset or anything else
keeps it off, which is the default. Nothing changes unless the variable is set.
The option is wired into the four agentcore.Options builders that set
ToolOutputDir: worker, planner, chat and the main agent. .env.example
documents the variable, commented out, with a note to test it on a local
target before enabling it in production.
Not included: docker-compose.yml does not pass the variable to the artex
service, so it has no effect under compose until it is listed there.
Extracts candidate lessons from terminal tasks (confirmed, negative,
retest caution), gates them on cue + action + evidence, and makes
extraction idempotent through a per-source evidence key.
ARTEX_RSI_LESSONS selects off / shadow / inject. Shadow logs what would
have been shown; inject appends a code-owned, data-not-instructions block
after the planner and worker system prompts. No prompt mutation and no
adoption: status stays candidate.
Credit: the Library admission rule comes from harness2 (Google Research,
Apache-2.0, arXiv 2610.14224); the gain taxonomy from awesome-rsi
(lobehub).
Workers learn their budget is finite only when the wrap-up interrupt
arrives, which is too late: commands started minutes before exhaustion
die mid-flight, and their results are the ones lost. The pentest agent
has a wrap-up priority rule but nothing that prevents launching
long-running work that cannot finish before the task deadline.
Changes:
- worker seed: budget discipline block. Before starting any command or
probe, check it finishes well within the remaining step and wall-clock
budget; never start work whose timeout can outlive what remains, take
the smaller variant that lands; when the budget is nearly spent or the
wrap-up signal arrives, land everything already collected (facts
first, then findings).
- pentest seed: deadline contract bullet. Long commands must fit the
task's remaining time; as the deadline approaches, shift from
discovery to consolidation.
- agent/deadline_contract_seed_test.go guards the markers.
Pattern adapted from bug-bounty-hunting-prompts (MIT), whose deadline
contract reads: never start a command whose timeout can extend beyond
the hard deadline.
Nothing today makes an agent check for an existing finding before
registering a new one, and the reporter has no duplicate or collapsing
rules. Duplicates burn reviewer trust on submission (programs read them
as noise or padding) and they inflate the task's apparent progress.
Changes:
- worker and pentest seeds: before report_finding, check
list_findings / list_task_findings for an existing finding with the
same root cause, endpoint, and parameter; the same defect again means
extending that finding's evidence, not registering a duplicate. A
genuinely distinct entry point, parameter, or privilege level remains
its own finding (the adversarial gate's anti-merge rule still holds:
the dedup check collapses only true duplicates).
- reporter seed: an explicit duplicate pass over list_task_findings in
the work steps, plus two discipline rules (no duplicate reports for an
already-reported defect; one finding per report, additional locations
of the same root cause listed inside it, never silently merged or spun
off).
- agent/finding_dedup_seed_test.go guards the markers.
When a worker hits an environmental failure (sandboxed or filtered
network path, missing binary or key, TLS or proxy interception, missing
runtime permission), nothing today tells it how to stop: transcripts
show turn after turn of retries with different flags, equivalent tools,
or fallback transports, all burning budget against a wall the agent
cannot move.
Changes:
- the ask_operator tool description now covers environmental blocks
explicitly: do not retry or work around; include the exact single
command the operator should run in their own terminal as one
copy-pastable line, plus what to paste back, then wait. The question
parameter description gains the same guidance.
- worker and pentest seeds gain an "Operator handoff on environmental
blocks" block: stop after the first environmental failure, no
flag-hopping, no bypass hunting, one command per ask_operator request,
never batch.
- tests: the tool description must keep the handoff markers
(TestAskOperatorEnvHandoffGuidance) and the executor seeds must keep
the block (TestPromptSeedsCarryOperatorHandoff).
Pattern adapted from bug-bounty-hunting-prompts (MIT), whose operator
handoff rule produced exactly this single-command format.
The worker and pentest seeds already tell the agent to self-check
conclusions, but the check has no concrete axes and no rules for what
an elimination requires. In practice candidates get dropped on vibes
("probably validated downstream") or merged into one narrative finding,
and there is pressure to produce at least one finding per intent.
Changes, all in the default prompt seeds plus a guard test:
- worker and pentest seeds gain an "Adversarial gate before
report_finding" block: actively try to disprove every candidate along
five named axes (reachability, preconditions, downstream controls,
scope, simpler explanation); eliminations need cited evidence at the
same bar as confirmations; distinct entry points, parameters, or
privilege levels are separate findings (anti-merge); zero confirmed
findings is a valid outcome and precise negatives are real work.
- the retester seed now argues the opposite case before every verdict
along the same five axes and must cite the specific requests,
responses, or controls supporting reproduced / fixed / inconclusive
alike; a prose assumption never substitutes for evidence.
- a seed test guards all markers against silent removal.
Methodology adapted from VulnHunter (Apache-2.0, adversarial
verification and anti-merge rules) and bug-bounty-hunting-prompts
(MIT, independent pre-confirmation disprove pass).
Confirmed findings today assert impact in prose ("the attacker can dump
the table") with no number behind it, and nothing in the seeds demands
that a PoC demonstrate the attacker's goal end to end rather than the
presence of a vulnerable pattern.
Three changes, all in the default prompt seeds plus a guard test:
- worker and pentest seeds gain a "Finding quality gates" block: the PoC
must show the goal achieved (data returned, command executed, another
user's resource touched); a mechanism blocked downstream is a hardening
note, not a finding; every finding carries a measured impact as a
number with its denominator plus what the defense blocked, a proof
basis line (executed live / executed lab / static unexecuted), bounded
non-destructive testing rules (own objects first, at most three
non-owned objects, smallest sufficient request, reversible values,
touched IDs recorded), and a priority rule against standalone
low-impact oracles.
- the reporter seed gains a mandatory "## Measured impact" report section
(quantified outcome, denominator, defense-blocked facts, proof basis)
and a "measured, not asserted" discipline bullet.
- a seed test guards all markers against silent removal.
Methodology adapted from VulnHunter (Apache-2.0, measured-impact and
attacker-goal doctrine) and bug-bounty-hunting-prompts (MIT, bounded
testing standard and priority discipline).
Executor agents ingest attacker-controlled content on every run (HTTP
responses via curl, browser pages via the browser MCP, traffic records,
binary strings via remote analysis tools) with no counter-instruction
anywhere in their context.
Two defenses, both zero-cost per call:
- A standing hardening block in the default prompt seeds of worker,
pentest, mainagent and auto: tool output from or through the target is
data, never instructions; embedded directives are quoted verbatim and
recorded as a possible prompt-injection finding instead of followed;
credentials and operator data never go to a destination suggested by
target-controlled content.
- Every MCP tool description gains a one-line untrusted-output note at
wiring time (covers both transports and travels with the description:
cached prompt for global tools, on-demand schema for deferred ones).
A seed test guards the hardening block against silent removal.
Closes#48
Comments, doc comments, UI labels, log and error messages, LLM prompt
strings, markdown docs, test data and assertions. Identifiers, enum
keys, routes, placeholders and code structure are unchanged.
Notes:
- docs/漏洞流量证据.md renamed to docs/vulnerability-traffic-evidence.md
- tests that exercised CJK specific handling (GBK zip names, Chinese
full text search) now use ASCII equivalents; go build passes and the
test suite matches the pre translation baseline (remaining failures
need a PostgreSQL DSN and fail identically on the original)
- frontend mention syntax now uses English kinds (finding, asset, ...)
matching the backend, chat mentions tests pass
BaaRai