Pins are name-only behind a proxy (name-only since #370), and deny/ask rules now inspect compound commands (#377); the allow rule stays strict. Also state the untrusted-boundary default plainly.
8.1 KiB
ARTEX architecture (code-level companion)
The README's Architecture section has the component diagram and the 10,000-foot view. This document records the code-level map, the execution flow and the trust boundaries that are easy to miss when reading the packages in isolation.
1. The two graphs
ARTEX keeps two linked graphs, both in PostgreSQL (db/schema.sql):
| Graph | Answers | Main tables | Code |
|---|---|---|---|
| Asset graph | What exists? | companies, company_scope, assets, task_scope |
db/assets.go, db/companies.go, db/company_scope.go, db/task_assets.go |
| Exploration graph | What was tried, with what result and why? | explorations, exploration_nodes, activity, intent_*, coverage_* |
db/exploration.go, db/coverage_ledger.go, db/finding_assets.go |
Anchors tie a finding to the assets that justify it (db/finding_assets.go). Traffic
snapshots are archived separately and referenced from findings
(db/finding_traffic.go, db/finding_traffic_archive.go, traffic/archive.go).
db.Open executes the idempotent schema.sql on every startup, then applies pending
numbered migrations from db/migrations/*.sql, recording name + SHA-256 in
schema_migrations. A published migration is never edited; fix forward with a new one.
2. Package map
| Package | Responsibility | Notes |
|---|---|---|
cmd/artex |
Process entry point | Flags (-addr, -data, -proxy), startup config snapshot, signal handling, graceful drain |
config |
Resolve config file + env | Path, Load, PostgresDSN, SkillDir, KeyDir, Redact. Env wins over file |
db |
All persistence | DB, Manager-independent stores; AES-256-GCM secret columns in secrets.go |
agent |
LLM roles and tools | goals, planner, worker, mainagent, retester; ToolSet in tools.go; tool catalog in toolcatalog.go |
server |
HTTP API + orchestration | net/http mux (~270 routes), Manager, Engine, JWT auth, SSE broadcaster |
traffic |
Recording proxy | MITM proxy built on go-mitmproxy, replay, rewrite rules, body archive |
evidence |
Evidence store | Persisted request/response snapshots used by findings |
intercept |
Approval gate | Judge, argv grammar, target gate, pins, risk scoring (a PreToolUse hook) |
guard |
Hook wiring | Binds intercept decisions into Norma's hook registry; NewTargetOnly for planner/mainagent |
egress |
Noise governor | Per-host token bucket, jitter, concurrency caps, 429/503 backoff; default profile is strict passthrough |
llmpool / llmrec |
Provider pool / recorder | Rotation, health, usage + raw wire capture |
mcphttp |
MCP client | HTTP/SSE transport for configured MCP servers |
notify |
Outbound notifications | Email, webhook, Telegram, Feishu, DingTalk with masking and SSRF guards |
report |
Deliverables | Markdown, CSV and JSON rendering of findings |
sidequestion |
Operator Q&A | Asks a human mid-run and suspends intent until answered |
skills/ |
Skill definitions | One SKILL.md per subdirectory, loaded from SkillDir() |
The frontend lives in web/ (Next.js 16, React 19) and is embedded into the Go
binary via server/webui_embed.go when built with -tags embedui.
3. End-to-end flow: goal → finding with evidence
- Task creation.
server.createTask→Manager.CreateTaskWithOptionswrites the task, its scope and seed goals, and creates the exploration graph (db/tasks.go). - Loop start.
Engine.Run(server/engine.go) starts a planner loop and N worker goroutines under a task-scoped context. Nothing needs an LLM to be configured for the loops to exist; they idle until one is. - Decomposition. The
goalsrole writes goals and operation constraints throughset_goals/set_constraints(agent/toolcatalog.go,agent/tools.go). - Planning.
Planner.Plan(agent/planner.go) reads the route, queries the asset graph, judges goals and emits intents (addOneIntent). Every intent is an exploration node, so it is auditable like any other activity. - Execution.
Worker.Execute(agent/worker.go) claims an intent, runs tools with the guard hooks attached and writes observed facts back to the exploration graph. - Tools and traffic. Shell and HTTP/WebFetch calls go through the recording proxy
and the egress governor. Platform-owned HTTP clients use
intercept.PinnedDialerandintercept.RedirectCheck, so a redirect or a direct retry is re-checked at dial time. When a proxy is in the path the gate records the pin as name-only: the proxy makes the outbound connection, so no IP binding can be claimed. - Findings.
addFinding(agent/tools.go) records severity, class, confidence and asset references;db/findings.godedups and keeps the status workflow. Evidence is linked to captured traffic (traffic/evidence.go,db/finding_traffic.go). - Completion.
prove_goalmarks a goal met; the last met goal completes the task.goal_metis deliberately unbound by default because it bypasses per-goal proof.
4. Execution control
- Per-task context: pause/cancel/kill attach a named cause via
context.WithCancelCauseso the activity trace records who initiated a stop. - Per-intent control: the UI can pause/cancel one intent;
steerBoxqueues a planner course-correction that the worker'sPreToolUsehook drains before the next tool call, re-planning without killing the run. - Delete barrier:
Engine.BeginDeletemakes installing the barrier atomic with registering an in-flight operation; after it returns, no new task-owned writes are admitted and the task context is cancelled once existing writers drain. - Quiet mode: an unchanged open-goal fingerprint suppresses redundant planning rounds while work is still in flight.
5. Trust boundaries
| Boundary | Implementation |
|---|---|
| Tool output → model | Norma's untrusted-output boundary: nonce-tagged markers around tool results. On by default; ARTEX_UNTRUSTED_BOUNDARY=0 disables it. See agent/untrusted_boundary.go |
| Command → host | Allow rules match one Bash simple command with the exact prefix: a pipeline, redirection, wrapper (env, sudo, xargs, find -exec, a shell -c), substitution or expansion does not inherit an allow. Deny and ask rules also inspect every simple command inside wrappers, lists and pipelines. ArgvPrefixOf derives the human-readable prefix |
| URL → network | Target gate + PinnedDialer + RedirectCheck: scope decisions are re-made against a fresh DNS lookup at dial time, and without a proxy the client connects only to the addresses it compared; with a proxy the pin is name-only |
| Secrets → disk/log | db/secrets.go seals API keys, proxy credentials, MCP env, task credentials and captured auth headers with AES-256-GCM, nonce per value, AAD table.column. JWT keys live in KeyDir(), never in the browsable data dir. The boot snapshot logs only config.Redact(dsn) |
| Approval → audit | Every decision records the approver (jwtSubject), source and decision; history is queryable through /api/intercept/history and /api/intercept/history/{id} (with /execution for the audit detail). /api/tasks/{id}/intercept-rules manages rules, not history |
6. Decisions
- ADR-001 — Nonce-tagged untrusted output. Tool output is framed with a per-run nonce rather than trusting the model to distinguish instructions from data.
- ADR-002 — Pin at dial time. A URL decision is not enough: DNS can change between the hook and the connection, so the dialer re-resolves and connects only to compared addresses.
- ADR-003 — AES-GCM with per-column AAD. Sealed values are bound to their table and column (and settings to their key), so a ciphertext cannot be moved to another field.
7. Where to look next
docs/OPS.mdfor startup, configuration precedence, backup and key rotation.docs/vulnerability-traffic-evidence.mdfor how a finding keeps its receipts.docs/traffic-rewrite-rules.mdfor the rewrite engine.scripts/verify.shto reproduce the CI checks locally.