As for the MCP env (#213), a placeholder in a custom tool's exec keeps the
stored header, proxy or body only while the url (exact string) and the kind
are unchanged. A changed or emptied url, or a kind change, with a resolved
placeholder is refused (400 on the API and the test endpoint, a tool error
for update_custom_tool) before any write and before the test request is
sent. An explicit value with a new url is allowed.
Phase 3 of secrets at rest (#213, #216), same AES-256-GCM scheme and key.
- tools.exec joins the secret columns for system=false rows (AAD
tools.exec): the whole JSON is sealed in the {"__artex_enc":"enc:v1:..."}
envelope, so headers, body, proxy, script code and command are covered;
{} stays {}
- CreateCustomTool and UpdateCustomTool seal it; scanTool opens it, so
GetTool, ListTools and ListCustomTools (and through them customTools,
the platform tools and the test endpoint) see the clear JSON in memory
- the startup backfill, the no-key refusal and the wrong-key check cover
it, for system=false rows only; built-in rows stay in clear for the seeds
API: GET /api/tools returns the header values, proxy and body of a custom
tool masked, with headers_set, proxy_set and body_set. On update (API and
update_custom_tool) the placeholder keeps the stored value, an explicit
value replaces it and "" clears it; on create (API and create_custom_tool)
a placeholder is refused. The test endpoint takes an optional key: the
saved tool's stored values replace the placeholders; the editor sends it.
Phase 2 of secrets at rest (#213), same AES-256-GCM scheme and key.
- llm_profiles.proxy joins the secret columns (AAD llm_profiles.proxy);
scanProfile decrypts it for both column lists, SaveProfile encrypts
it, an empty proxy stays NULL
- brave_search_api_key, tavily_search_api_key, web_search_proxy and
global_proxy are stored encrypted (AAD settings.<key>) through
GetSetting, SetSetting and InsertSettingIfAbsent; other settings stay
in clear
- the startup backfill, the no-key refusal and the wrong-key check
cover the new values; legacy plaintext is read as-is until backfill
API: the proxies are returned masked as a whole (placeholder plus a
*_set flag) by GET /api/settings, GET /api/llm/profiles and GET /api/llm.
On write the placeholder keeps the stored value, an explicit value
replaces it and "" clears it; the placeholder is never stored. The test
and model-list endpoints resolve the placeholder to the stored proxy.
Extends ingest_scan_output with three formats:
- nuclei-jsonl: one tool_output fact per result and an asset for the
scanned target when it is valid; no automatic finding
- httpx-jsonl: one HTTP service asset and one fact per URL
- ffuf-json: one URL asset and one fact per result, read token by token
with encoding/json; a file cut short keeps the complete results
JSONL lines go through readJSONLines, taken from the bbot ingester
(64 KiB per line; longer lines and invalid JSON are skipped and
counted). bbot now uses the same reader, with no behaviour change.
One file yields at most 50000 assets and facts; records past the cap
are counted in over_cap and the result is marked truncated.
A worker can write nmap output with -oX and call ingest_scan_output with
the format and path. The file is parsed deterministically: assets go
through the insert_assets handler, each host becomes one tool_output
fact listing its open ports, and the model gets only counters and at
most 20 notable items per type.
- path is resolved under the run working directory; "..", an absolute
path outside it and a symlink leading outside are refused; 256 MiB cap
- the XML is streamed host by host with encoding/xml; a file cut short
keeps the complete hosts and returns truncated: true
- target text (banners, products, hostnames) is kept only as a
Go-quoted string in the fact detail
- re-ingesting a file upserts the same assets and skips identical facts
Worker-only, like ingest_bbot. nuclei, httpx and ffuf follow separately.
A masked env value keeps the stored secret only while transport, command, args (canonical JSON) and url are unchanged, as notify.PrepareConfigUpdate does for channels. A changed destination with a masked value is refused (400). The credentials tab drops reveal/copy and shows a stored password as "stored, hidden".
Seal the four credential columns with AES-256-GCM (random nonce per value,
AAD "<table>.<column>"): llm_profiles.api_key and task_credentials.password
store "enc:v1:<base64(nonce||ciphertext)>"; mcp_servers.env and
notification_channels.config store {"__artex_enc":"enc:v1:..."} over the
canonical JSON. The db layer encrypts on write and decrypts on read, so agent
tools (LLM provider, MCP, notifier, test credentials) get plaintext
server-side. Decryption failure is an error, never a plaintext fallback.
Key: ARTEX_SECRET_KEY (32 bytes, base64 or hex) or keys/secret.key next to
jwt.key, generated with crypto/rand at first start (0600). Startup refuses
when encrypted values exist and no key is available, and when the key does
not decrypt them. An idempotent backfill under the schema advisory lock
encrypts legacy plaintext rows and logs only a count.
API: MCP env is masked on read and masked placeholders on write keep the
stored value; task credential passwords are masked server-side and a masked
password on write is refused. ARTEX_SECRET_KEY is scrubbed from the process
environment with the other startup secrets.
Dalfox v3.2.4 (Rust) has no -w and exposes a global --rate-limit plus --workers (docs/content/reference/cli.md). Subfinder -t only bounds concurrent DNS lookups in active mode, so the passive line is removed; the generic fallback covers it.
When ARTEX_NOISE_PROFILE is set, the planner, worker, main agent and chat system prompts gain a code-owned block with the profile per-host limits and exact rate/delay flags for the scanners installed in the image. With no profile the block is empty, so every prompt is byte-for-byte unchanged.
README documents the block.
POST /api/intercept/pending/{id}/always-allow derives an argv allow rule from the stored pending row (never from the body), at priority -1000, reusing an identical enabled rule, and leaves the request pending. A code-owned denylist of interpreters, launchers, package managers, orchestration and destructive programs refuses such rules at creation, in the manual rule form, and neutralises argv allow rules at matching; argv deny and ask rules still match.
- A destructive or operator_policy unknown asks, even when read_only is yes.
- An enabled closed judge with no completer wired asks (judge_not_wired)
instead of falling through to allow.
- The argument cap is checked before the review input is built.
- MaxTokens for the closed reply raised to 1536.
- The operator_policy question also covers a whitelist policy that fails to
authorize the action; golden digest updated.
The LLM fallback judge gets a closed mode (default when the judge is on):
the model answers eight code-owned questions with yes|no|unknown and one
evidence line each, in one call; the decision is taken in code. Every
failure (invalid output, missing key, value outside the vocabulary, a
failed call, arguments above the 256 KiB cap) is ask. The custom judge
prompt becomes bounded operator-policy data (8000 runes). The answers are
stored in the intercept audit and shown in the approval detail.
freeform keeps the previous behaviour; defaultJudgeFailAction changes
from allow to ask.
Normalise and length-check ledger fields in db.RecordCoverage, frame the planner gap lines in the data envelope, cap the wrap-up (25) and report (200) gap lists, cap current rows at 2000 per task. Replace the activity budget marker with tasks.end_reason, written with the guarded terminal status and cleared on revive. Generic 500 message in taskCoverage; early return on an empty ledger in the planner block.
Workers record per (surface, risk_area) outcomes with the new record_coverage
tool (reported, no_issue_found, ruled_out, not_applicable, needs_follow_up).
Gaps are needs_follow_up entries, entries missing their required evidence or
reason, and goal vulnclass values with no entry (unrecorded_risk_class), so an
absent finding never reads as a clean target. Completeness marks timeout,
budget wrap-up, pause, abort and running tasks as incomplete with a caveat.
Surfaced in the planner context (one line per round, full gap list on the
timeout/budget wrap-up round), GET /api/tasks/{id}/coverage, the coverage card
and the Markdown report. Follows coverage_enabled; no new env var.
- resetSettledRun clears settling and the budget ledger; a done/failed task
revived to running now plans and claims again (was gated by the old wrap-up).
resetTimeoutRevival calls it. Test: TestResetSettledRunRestartsBudget.
- docker-compose.yml passes the four ARTEX_*_BUDGET_* variables through.
- .env.example and README document the variables, the wrap-up, the in-memory
ledger and the practical floor.
- llmrec: the usage sink runs even without a PostgreSQL recorder; comment fixed.
- BudgetWrapupBlocks allows update_finding_report.
Per-level token/turn budgets: task -> intent (worker run) -> planner round
(counted inside the task). A child's effective budget is
min(own budget, parent's remaining above the wrap-up reserve). When the task
can no longer afford one worker, workers stop claiming intents and the task
settles through one forced wrap-up planner round (the existing settle
sequence, budget words instead of the timeout words).
Config: ARTEX_TASK_BUDGET_TOKENS, ARTEX_TASK_BUDGET_TURNS,
ARTEX_INTENT_BUDGET_TOKENS, ARTEX_INTENT_BUDGET_TURNS. 0 or unset = unlimited;
with all four unset nothing changes.
ARTEX_NOISE_PROFILE = off | quiet | balanced | aggressive drives one shared
governor used by WebFetch, the http custom tools and the enrich probe: token
bucket, bounded jitter and concurrency per host, a global cap, backoff on
429/503 honouring Retry-After with bounded retries, per-task coalescing and
a short cache of identical GETs (never POST/PUT/PATCH/DELETE), and a
per-host noise log line each minute. Off (the default) is a strict
passthrough.
Removes the fixed artex-enrich/1.0 User-Agent; ARTEX_TARGET_USER_AGENT sets
one under a profile (empty = unchanged).
When ARTEX_TOOL_RAG is enabled, AugmentTools keeps the core tools plus the
top-k (ARTEX_TOOL_RAG_TOPK, default 12) BM25-ranked bound tools for the
worker intent / planner goal. Runs after ToolResolve, so it only removes;
falls back to the full list on empty query, catalog under 20, or scorer error.
ARTEX_UNTRUSTED_BOUNDARY unset or empty now enables the boundary.
0, false, off and no (case-insensitive, trimmed) disable it; any other
value keeps it on. Docs state the default, the opt-out and the measured
cost (about 154 bytes per tool result plus one system note).
ARTEX_UPDATE_CHECK=1 lets the backend check the public GitHub releases API
for the latest tag at most once per 24 h (in-memory cache; failures are not
retried in the window) and report it in GET /api/health when it is newer
than the build version. The web UI shows a dismissible strip linking to the
Releases page. Unset (the default), no update request is made.
The release job now checks out the repo and extracts the CHANGELOG.md
section matching the pushed tag (leading v stripped) with
scripts/extract-changelog-section.sh, and passes it as body_path. If the
section is missing it logs a warning and falls back to a generic body
naming the tag and pointing at CHANGELOG.md.
scripts/test-extract-changelog-section.sh covers the found and missing
cases against a fake CHANGELOG in a temp dir; the CI shell job runs it
and shellchecks both scripts.
ingest_bbot reads a bbot output file by path and ingests it through
the typed parser from #199 (parseBbotJSONL), one line at a time:
- Lines are read with a fixed cap (bbotMaxLineBytes = 64 KiB, newline
included). A longer line is skipped and counted, never buffered whole.
- scope_distance == 0 events that are assets (assetItems) go through
the existing insert_assets handler; no asset SQL is written here.
- Every other event is recorded as a fact with source
third_party_intel. The summary is built from typed fields only; the
free text is stored in detail as the inert Go-quoted string.
- Intel never reaches insert_assets or task_scope.
The tool is in WorkerTools, so the existing BuiltinToolSeeds/SeedTool
loop inserts its row (a new key, inserted on fresh and existing DBs).
It is listed in the target gate's noTargetTools: its input is a local
path, not a network target (TestDomainToolsDeclareNoTarget).
Parse bbot output.json reading only the typed fields (type,
scope_distance, host, url, port, module, module_sequence), validated.
The free text (data, or data_json with the FINDING description) is kept
only as one Go-quoted inert string, never used for scope or an action.
Unknown keys are ignored. Events at scope_distance 0 are the only ones
mapped to insert_assets items, from their typed host; any other distance
becomes a third_party_intel item that is never auto-scoped.
doHTTP now connects through intercept.PinnedDialer under the target scope of the request's context, so the probe uses the same pinned-address dial as the target gate. ProbeSite takes the context of the call that asks for the probe. A probe without a scope is refused; with the gate on, the probe bypasses the recording proxy so the dialer decides on the target itself. The path stays dormant: nothing calls ProbeSite on main.
The goal gate accepts a goal that names its target as a bare domain (goalDomainRE), but the in-goal set was built only with commandTargets, which does not read bare hosts. A question citing the operator's own target as a URL was then tagged target-derived. goalTargets adds the goalDomainRE matches, widened to the whole dotted name, lowercased, port and path dropped. The question-side parser is unchanged.
BaaRai