Compare commits

...
Author SHA1 Message Date
BaaRai fdba29f1a9 ci: run gofmt in the go job, and format the 11 files it flags
Mechanical: gofmt -w on the 11 files gofmt -l listed on main. Besides spacing and alignment, gofmt sorted the imports inside one group in two test files and removed a trailing blank line in a third. The go job now fails, listing the files, when gofmt -l reports any tracked Go file.
2026-11-15 13:38:35 +01:00
BaaRai a59d430175 Merge pull request #236 from BaaRai/perf/task-metrics
perf(db): per-task metrics for the task detail and an index-served latest activity
2026-11-15 08:40:39 +01:00
BaaRai c2d1f557c3 perf(db): per-task metrics for the task detail and an index-served latest activity
getTask computed the list metrics of every task to read one; it now calls TaskListMetrics(expID), which runs the same laterals as TaskListMetricsAll with a WHERE on the exploration. listTasks keeps the global query. The latest_activity lateral ordered by its EXTRACT(EPOCH) output alias, so PostgreSQL read and sorted every activity row of each task; it now orders by activity.created_at and idx_act_latest serves the LIMIT 1. Both queries share one SQL constant.
2026-11-15 08:20:08 +01:00
BaaRai e636408f66 Merge pull request #235 from BaaRai/docs/upstream-changelog
docs: credit the upstream and complete the unreleased changelog
2026-11-15 02:01:55 +01:00
BaaRai fcbd87445e docs: credit the upstream and complete the unreleased changelog 2026-11-15 01:54:50 +01:00
BaaRai 59f768f1ef Merge pull request #234 from BaaRai/feat/proxy-pace-bash
feat(traffic): pace Bash requests at the capture proxy
2026-11-15 01:41:56 +01:00
BaaRai 7f3edf9bfe fix(agent): noise prompt states what the capture proxy paces
The low-and-slow block said that commands launched through Bash are not paced. With traffic capture on, Bash commands that use the capture proxy are now paced by rate and jitter; raw scanners and commands that bypass the proxy are not, so the rate flags remain the brake.
2026-11-15 01:23:56 +01:00
BaaRai 2e55615713 feat(traffic): pace Bash requests at the capture proxy
With traffic capture on, the Bash proxy URL carries a marker userinfo. The recording proxy drops Proxy-Authorization from every request before it is forwarded or recorded; when the value is the marker it marks the client connection, and every later request on it, including those inside its CONNECT tunnel, waits for the per-host token and jitter of the current noise profile (egress.Wait) and is counted, with a bounded wait. WebFetch, the http custom tools and enrich keep the unmarked URL, which egress already paces, so nothing is paced twice. Off paces nothing. Best effort: only clients that honour the proxy are covered, with capture on.
2026-11-15 00:57:13 +01:00
BaaRai 251ab680cc Merge pull request #233 from BaaRai/feat/noise-profile-runtime
feat(egress): change the noise profile at runtime
2026-11-15 00:14:23 +01:00
27 changed files with 858 additions and 107 deletions
+4
View File
@@ -29,6 +29,10 @@ jobs:
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: gofmt
run: |
files=$(gofmt -l $(git ls-files '*.go'))
if [ -n "$files" ]; then echo "gofmt required:"; echo "$files"; exit 1; fi
- run: go build ./...
- run: go vet ./...
# Skipped tests, tracked in issue #9:
+31 -2
View File
@@ -6,11 +6,40 @@ All notable changes to this project are documented in this file. The format is b
### Added
- **Noise profile at runtime**: System settings has a **Target noise** card. It sets the profile (`off`, `quiet`, `balanced`, `aggressive`) through the `noise_profile` setting, which takes precedence over `ARTEX_NOISE_PROFILE`. The change applies to the next WebFetch, http custom tool and enrich request, and to the next agent prompt, without a restart. The card also shows the per-host counters, which `GET /api/settings/noise` returns.
- **Target noise governor**: one shared egress governor paces ARTEX's own HTTP requests (WebFetch, the http custom tools and the enrich probe) against a noise profile (`off` by default, `quiet`, `balanced`, `aggressive`): a per-host token bucket with jitter, per-host and global in-flight caps, 429/503 backoff, GET deduplication with a short cache, and a configurable target User-Agent (`ARTEX_TARGET_USER_AGENT`). Off is a strict passthrough (#205).
- **Low-and-slow prompt block**: when a noise profile is set, the planner, worker, main agent and chat prompts carry the per-host limits of the profile and the exact rate or delay flags for nmap, nuclei, httpx, katana, ffuf, dalfox and sqlmap; with the profile off the prompts are unchanged (#212).
- **Noise profile at runtime**: System settings has a **Target noise** card. It sets the profile (`off`, `quiet`, `balanced`, `aggressive`) through the `noise_profile` setting, which takes precedence over `ARTEX_NOISE_PROFILE`. The change applies to the next WebFetch, http custom tool and enrich request, and to the next agent prompt, without a restart. The card also shows the per-host counters, which `GET /api/settings/noise` returns (#233).
- **Bash requests paced through the capture proxy**: with traffic capture on, the Bash proxy URL carries a marker userinfo. The recording proxy paces each request of a marked connection by the noise profile's per-host rate and jitter, and counts it, before it goes upstream. This applies inside CONNECT tunnels too. WebFetch, the http custom tools and enrich keep the unmarked URL and are not paced twice. The proxy now drops `Proxy-Authorization` from every request it forwards or records. This is best effort: only clients that honour the proxy are covered, with capture on (#234).
- **Hierarchical budgets**, off by default: a task budget and a per-intent budget take the minimum of their own budget and the parent's remaining budget above the wrap-up reserve, planner calls count inside the task, and reaching the reserve forces a wrap-up (#206).
- **Coverage ledger**: the worker records what it examined, ruled out or left open for each surface and risk class with `record_coverage`; gaps, unexamined classes and completeness appear in the planner context, `GET /api/tasks/{id}/coverage`, the coverage card and the Markdown report (#207).
- **Closed-question judge**: the LLM fallback judge answers eight fixed questions that the code owns and the code takes the decision; every failure now leads to ask instead of allow, and the previous behaviour stays available as `llm_judge_mode = freeform` (#208).
- **argv interception rules**: a new `match_type = argv` matches the parsed prefix of a Bash command exactly, and a pending Bash approval can create such a rule in one click; a code-owned denylist refuses programs where a prefix would allow arbitrary code (#210, #211).
- **Scanner output ingestion**: `ingest_scan_output` parses nmap XML, nuclei JSONL, httpx JSONL and ffuf JSON, and writes assets and facts deterministically, so the model never reads the raw output (#214, #215).
- **bbot ingestion**: a typed parser splits bbot JSONL events by `scope_distance`, only in-scope events go through `insert_assets`, and `ingest_bbot` reads the file line by line with a 64 KiB cap (#199, #200).
- **Tool RAG**, off by default: when the bound tool list of an agent is large, only the subset relevant to the current intent is injected, as a projection that never adds or edits a tool (`ARTEX_TOOL_RAG`) (#204).
- **Fact provenance**: `record_fact` takes a closed `source` enum (default `inference`) and every fact carries the server-set `origin`, both shown in `recent_facts`, `list_facts` and `expand_digest` (#194).
- **Per-message nonce envelope**: agent- and target-derived fields reach the planner and worker user messages inside a code-owned data envelope with a random nonce per message; operator input stays outside it (#195).
- **Intercept risk score and batch approval**: a deterministic risk score (low, medium, high) is shown on each pending approval, and up to 100 pending decisions can be settled in one call, each through the existing single-decision path (#232).
- **`ask_operator` target warning**: a request whose question names a host or URL outside the goal targets is tagged as containing target-derived text (#197).
- **Opt-in update banner**: `ARTEX_UPDATE_CHECK` makes the backend check the latest GitHub release at most once per 24 hours and the UI show a banner; off by default (#202).
- **Release provenance**: release archives carry GitHub build provenance attestations and a CycloneDX SBOM (`gh attestation verify <zip> --repo BaaRai/ARTEX`); release notes are the matching `CHANGELOG.md` section (#219, #201).
### Changed
- **Untrusted-output boundary is now on by default**: Norma frames tool results for the worker, planner, chat and main agent unless `ARTEX_UNTRUSTED_BOUNDARY` is set to `0`, `false`, `off` or `no`. It was opt-in via `ARTEX_UNTRUSTED_BOUNDARY=1` in 0.6.0 (#166).
- **Untrusted-output boundary is now on by default**: Norma frames tool results for the worker, planner, chat and main agent unless `ARTEX_UNTRUSTED_BOUNDARY` is set to `0`, `false`, `off` or `no`. It was opt-in via `ARTEX_UNTRUSTED_BOUNDARY=1` in 0.6.0 (#166); the default flips here (#203).
- **Versioned migrations**: the `db` package applies numbered `db/migrations/NNNN_slug.sql` files once, in order, under the schema advisory lock, and records a SHA-256 checksum in `schema_migrations`; missing, renamed, changed or out-of-order migrations fail closed (#226).
- **Coverage graph performance**: the coverage graph batches its connector lookups (three queries instead of one per root and per company) and adds the partial index `idx_ts_icp`; on a task of about 3,000 assets the p50 dropped from 126 ms to 27 ms and the queries per call from 221 to 3 (#230).
### Fixed
- **Auto-scope hygiene**: auto `task_scope` rows count for coverage and untested assets only under a declared root, and direct coverage counts anchored assets through fact anchors only; the scope is described as "coverage scope, not enforced" (#196).
- **Enrich HTTP probe**: it now dials through `intercept.PinnedDialer`, so the target gate applies and a probe without a target scope is refused (#198).
- **Agent hook data races**: the eight package-level hooks are synchronised, so a second wire-up no longer races tasks that already read them (#227).
### Security
- **Secrets at rest**: the credential-bearing columns and settings (LLM API keys and proxy, task credentials, MCP server env, notification channel config, search and proxy settings), the exec spec of custom tools and the evidence snapshot heads are encrypted in PostgreSQL with AES-256-GCM (`enc:v1:`) using `ARTEX_SECRET_KEY`, else `keys/secret.key`; existing plaintext rows are backfilled once at startup, and a missing or wrong key while encrypted data exists refuses to start. This protects database dumps and backups, not a running instance (#213, #216, #217, #218).
- **URL credentials are dropped before storage**: scan and bbot ingestion, the asset store and archive restore strip URL userinfo, so `https://user:pass@host/app` is stored and matched as `https://host/app` (#222, #223, #224).
## [0.6.0] - 2026-11-12
+15 -2
View File
@@ -6,6 +6,8 @@
**Autonomous penetration testing that keeps its receipts.**
(upstream: [Autumn-27/ARTEX](https://github.com/Autumn-27), AGPL-3.0)
Self-hosted multi-agent offensive security. A planner explores, workers run real tools,
and every finding stays bound to the assets and HTTP traffic that prove it.
@@ -405,8 +407,13 @@ flight), `aggressive` = 20 req/s (burst 40, 4 in flight). A profile adds jitter,
429/503 (honours `Retry-After`, bounded retries, GET/HEAD/OPTIONS only), deduplicates identical
in-flight GETs and caches them briefly (60/30/10 s; never POST/PUT/PATCH/DELETE), and logs a
per-host noise line each minute. `ARTEX_TARGET_USER_AGENT` sets a User-Agent on target requests
that carry none. Bash-launched tools are not paced yet; the low-and-slow prompt block supplies
their rate flags.
that carry none. With traffic capture on, Bash commands that use the proxy (curl, wget,
python-requests) are paced too, per host by the same rate and jitter, and counted in the same
per-host lines. The in-flight caps, retries and cache apply only to ARTEX's own requests. This is
best effort, not a security boundary. Raw scanners such as nmap are not paced, and neither is a
command that bypasses the proxy (`--noproxy`, a removed proxy variable). With capture off,
nothing launched from Bash is paced. The low-and-slow prompt block supplies the rate flags in
every case.
The **Target noise** card in System settings sets the profile at runtime (no restart) and shows the
per-host counters; once saved, that setting takes precedence over `ARTEX_NOISE_PROFILE`.
@@ -688,6 +695,12 @@ Operational security properties:
---
## About this fork
This repository is the English translation and continuation of ARTEX, originally by [Autumn-27](https://github.com/Autumn-27). The upstream repository is no longer public, so the link points to the author profile. Changes in this fork are tracked in [CHANGELOG.md](CHANGELOG.md), under the same AGPL-3.0 license.
---
## References and prior art
- [Cairn](https://github.com/oritera/Cairn)
+1 -1
View File
@@ -64,6 +64,6 @@ func renderNoiseProfile(p egress.Profile) string {
fmt.Fprintf(&b, "- sqlmap: --delay=%s --threads=1\n", delaySec)
fmt.Fprintf(&b, "For any other tool, stay at or below %d requests per second per host, at most %s in flight, and at least %d ms between requests, using the tool's own rate or delay flag. Keep bbot within the same limits with its own rate settings, and do not enable active modules the program policy does not allow.\n", rps, inFlight, delayMS)
b.WriteString("Prefer passive and read-only checks; test one host at a time; do not run two scanners against the same host in parallel; on 429 or 503 stop and back off; never raise the rate to meet a deadline.\n")
b.WriteString("ARTEX paces its own WebFetch, http custom tool and enrich traffic with this profile, but commands launched through Bash are not paced: these flags are the only brake.\n")
b.WriteString("ARTEX paces its own WebFetch, http custom tool and enrich traffic with this profile. With traffic capture on, Bash commands that use the capture proxy (curl, wget, python-requests) are paced too, by rate and jitter; raw scanners such as nmap and commands that bypass the proxy are not. These flags remain the brake: do not rely on the proxy.\n")
return b.String()
}
+1 -1
View File
@@ -41,7 +41,7 @@ func TestRenderNoiseProfileTable(t *testing.T) {
fmt.Sprintf("- dalfox scan: --rate-limit %d --workers %d", c.rps, c.conc),
fmt.Sprintf("- sqlmap: --delay=%s --threads=1", c.sql),
fmt.Sprintf("at least %d ms between requests", c.delay),
"commands launched through Bash are not paced",
"ARTEX paces its own WebFetch, http custom tool and enrich traffic with this profile. With traffic capture on, Bash commands that use the capture proxy (curl, wget, python-requests) are paced too, by rate and jitter; raw scanners such as nmap and commands that bypass the proxy are not. These flags remain the brake: do not rely on the proxy.",
} {
if !strings.Contains(got, want) {
t.Errorf("%s: missing %q in:\n%s", c.name, want, got)
+23
View File
@@ -49,3 +49,26 @@ func TestProxyEnvInjectsCAWhenRecording(t *testing.T) {
}
}
}
// Only the recording proxy (a CA is present) gets the pace marker; the global
// proxy of the capture-off path is passed as configured.
func TestProxyEnvMarksOnlyTheRecordingProxy(t *testing.T) {
proxyVars := map[string]bool{"HTTP_PROXY": true, "HTTPS_PROXY": true, "ALL_PROXY": true}
check := func(env []string, want, what string) {
t.Helper()
n := 0
for _, e := range env {
if k, v, _ := strings.Cut(e, "="); proxyVars[strings.ToUpper(k)] {
n++
if v != want {
t.Errorf("%s: %s, want %s", what, e, want)
}
}
}
if n != 6 {
t.Errorf("%s: %d proxy variables, want 6", what, n)
}
}
check(proxyEnv("http://127.0.0.1:8788", "/data/ca.pem"), "http://artex-pace:artex-pace@127.0.0.1:8788", "capture on")
check(proxyEnv("http://u:p@10.0.0.1:3128", ""), "http://u:p@10.0.0.1:3128", "capture off")
}
+4 -4
View File
@@ -46,9 +46,9 @@ const stagnationMinRepeats = 3
func detectStagnation(done, findings []*db.Node) []map[string]any {
type dir struct {
firstID int64
repeats int
summary string
firstID int64
repeats int
summary string
}
dirs := map[string]*dir{}
order := []string{}
@@ -551,7 +551,7 @@ func (t *ToolSet) graphOverviewData() map[string]any {
// with the data so no prompt-template change is needed on deployed installs.
if stale := detectStagnation(recentDone, vulnNodes); len(stale) > 0 {
out["stagnation_warning"] = map[string]any{
"directions": stale,
"directions": stale,
"instruction": "these directions were dispatched repeatedly with zero findings produced since their first attempt — do NOT re-dispatch the same approach again; open a materially different direction, or judge the goal on the existing evidence instead of grinding",
}
}
+10
View File
@@ -11,6 +11,7 @@ import (
"time"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/egress"
"github.com/Autumn-27/artex/intercept"
"github.com/Autumn-27/norma/agentcore"
"github.com/Autumn-27/norma/harness"
@@ -212,10 +213,19 @@ func (w *Worker) SetWebSearch(o WebSearchOpts) { w.webSearch = o }
// honor the proxy vars. ALL_PROXY is set too so a socks5 egress proxy (which curl
// only reads from ALL_PROXY, not HTTP(S)_PROXY) works in the capture-off path.
// Empty proxyAddr → nil (direct, unchanged env).
//
// With a MITM CA (capture on) the proxy URL carries the pace marker userinfo
// (egress.MarkProxyURL): the recording proxy then paces this connection's
// requests by the noise profile. Only Bash gets the marker; WebFetch, the http
// custom tools and enrich keep the plain URL, since egress paces them already.
// The global proxy (capture off) is left as configured.
func proxyEnv(proxyAddr, caCert string) []string {
if proxyAddr == "" {
return nil
}
if caCert != "" {
proxyAddr = egress.MarkProxyURL(proxyAddr)
}
env := []string{
"HTTP_PROXY=" + proxyAddr, "HTTPS_PROXY=" + proxyAddr,
"http_proxy=" + proxyAddr, "https_proxy=" + proxyAddr,
+6 -6
View File
@@ -7,12 +7,12 @@ import "time"
// these match the *target asset*: an exact/fuzzy domain·ip·url, or a CIDR range.
// This layer only stores rules; the matching/enforcement logic lives elsewhere.
type AssetInterceptRule struct {
ID int64 `json:"id"`
Enabled bool `json:"enabled"`
Kind string `json:"kind"` // exact_domain|exact_ip|exact_url|fuzzy_domain|fuzzy_ip|fuzzy_url|cidr
Pattern string `json:"pattern"`
Note string `json:"note"`
Builtin bool `json:"builtin"`
ID int64 `json:"id"`
Enabled bool `json:"enabled"`
Kind string `json:"kind"` // exact_domain|exact_ip|exact_url|fuzzy_domain|fuzzy_ip|fuzzy_url|cidr
Pattern string `json:"pattern"`
Note string `json:"note"`
Builtin bool `json:"builtin"`
// Action is only used by task-level rules: 'block'=intercept, 'allow'=allow (whitelist).
// Global rules (asset_intercept_rules) do not have this column; it is always empty and treated as intercept.
Action string `json:"action,omitempty"`
+6 -6
View File
@@ -1168,11 +1168,11 @@ func TestQueryDSLInScopeMembership(t *testing.T) {
}
stamp := time.Now().UnixNano()
root := fmt.Sprintf("sc%d.invalid", stamp) // in-scope root domain (task)
srcRoot := fmt.Sprintf("src%d.invalid", stamp) // in-scope via source task
out := fmt.Sprintf("out%d.invalid", stamp) // out of every scope
marker := fmt.Sprintf("mk%d", stamp) // bare-text token present in all rows
foreignTask := stamp + 777 // asset produced by an unrelated task
root := fmt.Sprintf("sc%d.invalid", stamp) // in-scope root domain (task)
srcRoot := fmt.Sprintf("src%d.invalid", stamp) // in-scope via source task
out := fmt.Sprintf("out%d.invalid", stamp) // out of every scope
marker := fmt.Sprintf("mk%d", stamp) // bare-text token present in all rows
foreignTask := stamp + 777 // asset produced by an unrelated task
// Scope: task owns root; source task owns srcRoot; task owns an IP /24.
for _, sc := range []struct {
@@ -1200,7 +1200,7 @@ func TestQueryDSLInScopeMembership(t *testing.T) {
{"subdomain", "api." + root, root, "", "", "", marker}, // under task root
{"service", "www." + root, root, "https://www." + root + "/", "", "", marker}, // service under task root
{"endpoint", "www." + root, root, "https://www." + root + "/a?" + marker + "=1", "GET", "", ""},
{"subdomain", "dev." + srcRoot, srcRoot, "", "", "", marker}, // under source-task root
{"subdomain", "dev." + srcRoot, srcRoot, "", "", "", marker}, // under source-task root
{"endpoint", "198.51.100.9", "198.51.100.9", "http://198.51.100.9:8080/" + marker, "GET", "", ""}, // IP-literal host, ip col empty
{"subdomain", "x." + out, out, "", "", "", marker}, // out of scope
}
+50 -26
View File
@@ -1486,11 +1486,13 @@ type TaskListMetrics struct {
Findings FindingSeverityCounts // this task's vulnerability count in the findings table (bucketed by severity)
}
// TaskListMetricsAll returns list aggregates for every live task in one query.
// The lateral lookups use the per-exploration indexes instead of grouping the
// complete activity history on every poll.
func (d *DB) TaskListMetricsAll() (map[int64]TaskListMetrics, error) {
rows, err := d.Query(`
// taskListMetricsQuery is the select shared by TaskListMetricsAll and
// TaskListMetrics; each appends its own WHERE on task. The lateral lookups use
// the per-exploration indexes instead of grouping the complete activity history
// on every poll. latest_activity orders by the column activity.created_at, so
// idx_act_latest (exploration_id, created_at DESC) serves the LIMIT 1; ordering
// by the output alias would sort every activity row of the task.
const taskListMetricsQuery = `
SELECT task.exploration_id,
COALESCE(token_metrics.input_tokens,0),
COALESCE(token_metrics.output_tokens,0),
@@ -1514,10 +1516,10 @@ func (d *DB) TaskListMetricsAll() (map[int64]TaskListMetrics, error) {
WHERE exploration_id=task.exploration_id AND kind='result'
) token_metrics ON true
LEFT JOIN LATERAL (
SELECT EXTRACT(EPOCH FROM created_at)::bigint AS created_at
SELECT EXTRACT(EPOCH FROM activity.created_at)::bigint AS created_at
FROM activity
WHERE exploration_id=task.exploration_id
ORDER BY created_at DESC
WHERE activity.exploration_id=task.exploration_id
ORDER BY activity.created_at DESC
LIMIT 1
) latest_activity ON true
LEFT JOIN LATERAL (
@@ -1539,30 +1541,41 @@ func (d *DB) TaskListMetricsAll() (map[int64]TaskListMetrics, error) {
FROM findings
WHERE task_id=task.id
) finding_metrics ON true
WHERE task.deleted_at IS NULL`)
WHERE task.deleted_at IS NULL`
// scanTaskListMetrics reads one row of taskListMetricsQuery.
func scanTaskListMetrics(row interface{ Scan(...any) error }) (int64, TaskListMetrics, error) {
var explorationID int64
var metrics TaskListMetrics
err := row.Scan(
&explorationID,
&metrics.Tokens.InputTokens,
&metrics.Tokens.OutputTokens,
&metrics.Tokens.CacheReadTokens,
&metrics.Tokens.CacheWriteTokens,
&metrics.LastActivity,
&metrics.Goals.Total,
&metrics.Goals.Met,
&metrics.RunningIntents,
&metrics.Findings.Critical,
&metrics.Findings.High,
&metrics.Findings.Medium,
&metrics.Findings.Low,
)
return explorationID, metrics, err
}
// TaskListMetricsAll returns list aggregates for every live task in one query.
func (d *DB) TaskListMetricsAll() (map[int64]TaskListMetrics, error) {
rows, err := d.Query(taskListMetricsQuery)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[int64]TaskListMetrics{}
for rows.Next() {
var explorationID int64
var metrics TaskListMetrics
if err := rows.Scan(
&explorationID,
&metrics.Tokens.InputTokens,
&metrics.Tokens.OutputTokens,
&metrics.Tokens.CacheReadTokens,
&metrics.Tokens.CacheWriteTokens,
&metrics.LastActivity,
&metrics.Goals.Total,
&metrics.Goals.Met,
&metrics.RunningIntents,
&metrics.Findings.Critical,
&metrics.Findings.High,
&metrics.Findings.Medium,
&metrics.Findings.Low,
); err != nil {
explorationID, metrics, err := scanTaskListMetrics(rows)
if err != nil {
return nil, err
}
out[explorationID] = metrics
@@ -1570,6 +1583,17 @@ func (d *DB) TaskListMetricsAll() (map[int64]TaskListMetrics, error) {
return out, rows.Err()
}
// TaskListMetrics returns the same aggregates for the one live task of
// exploration expID, with the same laterals as TaskListMetricsAll. A missing or
// deleted task gives zero metrics, as a missing key of TaskListMetricsAll does.
func (d *DB) TaskListMetrics(expID int64) (TaskListMetrics, error) {
_, metrics, err := scanTaskListMetrics(d.QueryRow(taskListMetricsQuery+` AND task.exploration_id=$1`, expID))
if errors.Is(err, sql.ErrNoRows) {
return TaskListMetrics{}, nil
}
return metrics, err
}
// GoalCountsAll returns goal totals (total / met) per exploration id, one query
// for all tasks — used by listTasks so the task list shows progress for every task.
func (d *DB) GoalCountsAll() (map[int64]GoalCounts, error) {
+130
View File
@@ -117,3 +117,133 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExplorationID); er
t.Fatalf("unexpected goal metrics: %+v", metrics.Goals)
}
}
// TaskListMetrics(expID) is exactly the row TaskListMetricsAll returns for that
// exploration, on every counter, and TaskListMetricsAll still lists every task.
func TestTaskListMetricsMatchesAll(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
}
t.Cleanup(func() { d.Close() })
newest := time.Now().Add(-time.Minute).Truncate(time.Second)
seed := func(name string, n int) *Task {
t.Helper()
task, err := d.CreateTask(name, "goal", nil, 0, 0)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
_, err := d.DeleteTaskCascade(task.ID, false, true)
cleanupErr(t, "task "+name, err)
})
exp := task.ExplorationID
// n result rows with tokens; the newest activity first, then an older row
// with a higher id, so ordering by id instead of created_at would fail.
for _, q := range []struct {
sql string
args []any
}{
{`INSERT INTO activity(exploration_id,kind,input_tokens,output_tokens,cache_read_tokens,cache_write_tokens,created_at)
SELECT $1,'result',10,5,3,1,$3::timestamptz - interval '1 hour' FROM generate_series(1,$2::int)`, []any{exp, n, newest}},
{`INSERT INTO activity(exploration_id,kind,created_at) VALUES ($1,'text',$2::timestamptz)`, []any{exp, newest}},
{`INSERT INTO activity(exploration_id,kind,created_at) SELECT $1,'text',$3::timestamptz - interval '2 hours' FROM generate_series(1,$2::int)`, []any{exp, n, newest}},
{`INSERT INTO exploration_nodes(exploration_id,kind,payload,state)
SELECT $1,'goal','{}',CASE WHEN g<=$2::int THEN 'met' ELSE 'open' END FROM generate_series(1,$2::int+1) g`, []any{exp, n}},
{`INSERT INTO exploration_nodes(exploration_id,kind,payload,state)
SELECT $1,'intent','{}',CASE WHEN g<=$2::int THEN 'running' ELSE 'done' END FROM generate_series(1,$2::int+2) g`, []any{exp, n}},
} {
if _, err := d.Exec(q.sql, q.args...); err != nil {
t.Fatal(err)
}
}
if _, err := d.Exec(`INSERT INTO findings(task_id,severity)
SELECT $1, s FROM unnest(ARRAY['critical','high','high','medium','low','info']) s, generate_series(1,$2)`, task.ID, n); err != nil {
t.Fatal(err)
}
return task
}
a := seed("task-list-metrics-a", 2)
b := seed("task-list-metrics-b", 3)
all, err := d.TaskListMetricsAll()
if err != nil {
t.Fatal(err)
}
for _, c := range []struct {
task *Task
n int
}{{a, 2}, {b, 3}} {
row, ok := all[c.task.ExplorationID]
if !ok {
t.Fatalf("TaskListMetricsAll misses task %d", c.task.ID)
}
one, err := d.TaskListMetrics(c.task.ExplorationID)
if err != nil {
t.Fatal(err)
}
if one != row {
t.Fatalf("task %d: TaskListMetrics=%+v, TaskListMetricsAll=%+v", c.task.ID, one, row)
}
n := c.n
want := TaskListMetrics{
Tokens: TokenUsage{InputTokens: 10 * n, OutputTokens: 5 * n, CacheReadTokens: 3 * n, CacheWriteTokens: n},
LastActivity: newest.Unix(),
Goals: GoalCounts{Total: n + 1, Met: n},
RunningIntents: n,
Findings: FindingSeverityCounts{Critical: n, High: 2 * n, Medium: n, Low: n},
}
if one != want {
t.Fatalf("task %d metrics=%+v, want %+v", c.task.ID, one, want)
}
}
if none, err := d.TaskListMetrics(-1); err != nil || none != (TaskListMetrics{}) {
t.Fatalf("missing exploration: %+v, %v; want zero metrics", none, err)
}
}
// BenchmarkTaskListMetrics compares the per-exploration metrics query with the
// all-tasks one on three tasks of 40,000 activity rows each. No time assertion.
func BenchmarkTaskListMetrics(b *testing.B) {
dsn, _, err := DSN()
if err != nil {
b.Skipf("no database config (%v) - skipping", err)
}
d, err := Open(dsn)
if err != nil {
b.Skipf("postgres unavailable (%v) - skipping", err)
}
b.Cleanup(func() { d.Close() })
var exp int64
for i := range 3 {
task, err := d.CreateTask(fmt.Sprintf("bench-task-list-metrics-%d", i), "goal", nil, 0, 0)
if err != nil {
b.Fatal(err)
}
b.Cleanup(func() { _, _ = d.DeleteTaskCascade(task.ID, false, true) })
if _, err := d.Exec(`INSERT INTO activity(exploration_id,kind,input_tokens,created_at)
SELECT $1, CASE WHEN g%20=0 THEN 'result' ELSE 'tool_use' END, 100, now() - (g||' seconds')::interval
FROM generate_series(1,40000) g`, task.ExplorationID); err != nil {
b.Fatal(err)
}
exp = task.ExplorationID
}
if _, err := d.Exec(`ANALYZE activity`); err != nil {
b.Fatal(err)
}
b.Run("one", func(b *testing.B) {
for range b.N {
if _, err := d.TaskListMetrics(exp); err != nil {
b.Fatal(err)
}
}
})
b.Run("all", func(b *testing.B) {
for range b.N {
if _, err := d.TaskListMetricsAll(); err != nil {
b.Fatal(err)
}
}
})
}
+8 -8
View File
@@ -6,14 +6,14 @@ import "testing"
// ip/cidr scope path so a target like "10.0.188.136:3000" no longer 404s.
func TestStripHostPort(t *testing.T) {
cases := []struct{ in, want string }{
{"10.0.188.136:3000", "10.0.188.136"}, // the reported IP case
{"10.0.188.136", "10.0.188.136"}, // bare IPv4 unchanged
{"[2001:db8::1]:8080", "2001:db8::1"}, // bracketed IPv6 + port
{"2001:db8::1", "2001:db8::1"}, // bare IPv6 unchanged (has colons)
{" 1.2.3.4:80 ", "1.2.3.4"}, // trims surrounding space
{"example.com:443", "example.com"}, // domain + port → bare host
{"api.example.com:8080", "api.example.com"}, // subdomain + port
{"example.com", "example.com"}, // bare domain unchanged
{"10.0.188.136:3000", "10.0.188.136"}, // the reported IP case
{"10.0.188.136", "10.0.188.136"}, // bare IPv4 unchanged
{"[2001:db8::1]:8080", "2001:db8::1"}, // bracketed IPv6 + port
{"2001:db8::1", "2001:db8::1"}, // bare IPv6 unchanged (has colons)
{" 1.2.3.4:80 ", "1.2.3.4"}, // trims surrounding space
{"example.com:443", "example.com"}, // domain + port → bare host
{"api.example.com:8080", "api.example.com"}, // subdomain + port
{"example.com", "example.com"}, // bare domain unchanged
}
for _, c := range cases {
if got := stripHostPort(c.in); got != c.want {
+11 -11
View File
@@ -28,19 +28,19 @@ type Task struct {
LLMProfileID *int64 `json:"llm_profile_id,omitempty"`
// Task-level ordered LLM chain. LLMProfileID remains the compatibility alias
// for ActiveLLMProfileID while older API clients still send one profile id.
LLMProfileIDs []int64 `json:"llm_profile_ids,omitempty"`
ActiveLLMProfileID *int64 `json:"active_llm_profile_id,omitempty"`
LLMChainRevision int64 `json:"-"`
LLMFailoverState string `json:"llm_failover_state,omitempty"`
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
LLMProfileIDs []int64 `json:"llm_profile_ids,omitempty"`
ActiveLLMProfileID *int64 `json:"active_llm_profile_id,omitempty"`
LLMChainRevision int64 `json:"-"`
LLMFailoverState string `json:"llm_failover_state,omitempty"`
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
// LLMRotate: per-intent chain-head rotation switch (#85); false = classic
// ordered failover from the active head.
LLMRotate bool `json:"llm_rotate"`
SourceTaskIDs []int64 `json:"source_task_ids,omitempty"`
CompanyIDs []int64 `json:"company_ids,omitempty"`
ParentRef string `json:"parent_ref,omitempty"` // parent task id (recorded by orchestration spawn; empty=top level)
CreatedAt time.Time `json:"created_at"`
CompletedAt *time.Time `json:"completed_at,omitempty"` // when it entered a terminal state (done/failed/timeout); nil for non-terminal
LLMRotate bool `json:"llm_rotate"`
SourceTaskIDs []int64 `json:"source_task_ids,omitempty"`
CompanyIDs []int64 `json:"company_ids,omitempty"`
ParentRef string `json:"parent_ref,omitempty"` // parent task id (recorded by orchestration spawn; empty=top level)
CreatedAt time.Time `json:"created_at"`
CompletedAt *time.Time `json:"completed_at,omitempty"` // when it entered a terminal state (done/failed/timeout); nil for non-terminal
// Task-level timeout (see the task-level timeout and wrap-up design doc).
TimeoutSeconds int `json:"timeout_seconds"` // 0=no time limit
FirstRunAt *time.Time `json:"first_run_at,omitempty"` // when it first actually started running (not created_at); nil=not yet run
+67
View File
@@ -0,0 +1,67 @@
package egress
import (
"context"
"encoding/base64"
"net/url"
"strings"
)
// Bash commands reach targets through the recording proxy, outside the clients
// this package governs. With traffic capture on, the proxy URL in their
// environment carries a marker userinfo (MarkProxyURL). The proxy paces each
// request of a marked client connection with Wait, and leaves unmarked
// connections alone: WebFetch, the http custom tools and the enrich probe use
// the plain proxy URL and are already paced by their governor, so no request
// is paced twice. Best effort, not a security boundary: a command that ignores
// the proxy, or drops the marker, is not paced.
// PaceUser is the user and password of the marker userinfo.
const PaceUser = "artex-pace"
// MarkProxyURL returns raw with the marker userinfo. raw is returned unchanged
// when it does not parse or already carries a userinfo.
func MarkProxyURL(raw string) string {
u, err := url.Parse(raw)
if err != nil || u.Host == "" || u.User != nil {
return raw
}
u.User = url.UserPassword(PaceUser, PaceUser)
return u.String()
}
// IsPaceMarker reports whether a Proxy-Authorization value is Basic
// credentials for PaceUser.
func IsPaceMarker(proxyAuthorization string) bool {
scheme, enc, ok := strings.Cut(strings.TrimSpace(proxyAuthorization), " ")
if !ok || !strings.EqualFold(scheme, "Basic") {
return false
}
dec, err := base64.StdEncoding.DecodeString(strings.TrimSpace(enc))
if err != nil {
return false
}
user, _, _ := strings.Cut(string(dec), ":")
return user == PaceUser
}
// Wait paces one request to hostKey under the current profile. See
// Governor.Wait.
func Wait(ctx context.Context, hostKey string) error { return Current().Wait(ctx, hostKey) }
// Wait takes a token of the host's bucket plus the jitter, as a governed request
// does, and counts the request in the host's noise counter. It takes no
// in-flight slot, and nothing is cached, coalesced or retried. Off, it returns
// at once and counts nothing. An error means ctx ended first; the request is
// counted either way, since the caller sends it regardless.
func (g *Governor) Wait(ctx context.Context, hostKey string) error {
if g.off() {
return nil
}
h := g.host(hostKey)
err := sleep(ctx, g.reserve(h)+g.jitter())
g.mu.Lock()
h.sent++
g.mu.Unlock()
return err
}
+85
View File
@@ -0,0 +1,85 @@
package egress
import (
"context"
"encoding/base64"
"errors"
"testing"
"time"
)
func TestMarkProxyURL(t *testing.T) {
for in, want := range map[string]string{
"http://127.0.0.1:8788": "http://artex-pace:artex-pace@127.0.0.1:8788",
"http://u:p@127.0.0.1:8788": "http://u:p@127.0.0.1:8788",
"": "",
"127.0.0.1:8788": "127.0.0.1:8788",
"socks5://proxy.example.test:1080": "socks5://artex-pace:artex-pace@proxy.example.test:1080",
} {
if got := MarkProxyURL(in); got != want {
t.Errorf("MarkProxyURL(%q) = %q, want %q", in, got, want)
}
}
}
func TestIsPaceMarker(t *testing.T) {
enc := func(s string) string { return base64.StdEncoding.EncodeToString([]byte(s)) }
for v, want := range map[string]bool{
"Basic " + enc("artex-pace:artex-pace"): true,
"basic " + enc("artex-pace:x"): true,
"Basic " + enc("user:pass"): false,
"Bearer " + enc("artex-pace:artex-pace"): false,
"Basic !!!": false,
"Basic": false,
"": false,
} {
if got := IsPaceMarker(v); got != want {
t.Errorf("IsPaceMarker(%q) = %v, want %v", v, got, want)
}
}
}
func TestWait(t *testing.T) {
if err := New(Profile{Name: "off"}).Wait(t.Context(), "a.example"); err != nil {
t.Fatalf("off Wait: %v", err)
}
off := New(Profile{Name: "off"})
_ = off.Wait(t.Context(), "a.example")
if s := off.Snapshot(); len(s) != 0 {
t.Fatalf("off counted: %v", s)
}
// 10 req/s, burst 1, no jitter: the second Wait takes ~100 ms.
g := New(Profile{Name: "test", RPS: 10, Burst: 1, MaxPerHost: 1, MaxGlobal: 1})
start := time.Now()
for range 2 {
if err := g.Wait(t.Context(), "a.example"); err != nil {
t.Fatal(err)
}
}
if d := time.Since(start); d < 80*time.Millisecond {
t.Fatalf("two Waits at 10 req/s took %v", d)
}
if s := g.Snapshot(); len(s) != 1 || s[0].RPM != 2 || s[0].Inflight != 0 || s[0].PeakInflight != 0 {
t.Fatalf("snapshot after Wait = %+v, want rpm 2 and no in-flight slot", s)
}
// A wait past the context's end returns its error and still counts.
slow := New(Profile{Name: "test", RPS: 0.5, Burst: 1, MaxPerHost: 1, MaxGlobal: 1})
_ = slow.Wait(t.Context(), "b.example")
ctx, cancel := context.WithTimeout(t.Context(), 20*time.Millisecond)
defer cancel()
if err := slow.Wait(ctx, "b.example"); !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("bounded Wait: %v, want DeadlineExceeded", err)
}
if s := slow.Snapshot(); len(s) != 1 || s[0].RPM != 2 {
t.Fatalf("snapshot after a timed-out Wait = %+v, want rpm 2", s)
}
// The package Wait uses the current profile.
keepProfile(t)
SetProfile(Profile{Name: "off"})
if err := Wait(t.Context(), "c.example"); err != nil || len(Current().Snapshot()) != 0 {
t.Fatalf("package Wait under off: err=%v snapshot=%v", err, Current().Snapshot())
}
}
+1 -1
View File
@@ -6,9 +6,9 @@ import (
"path/filepath"
"testing"
actool "github.com/Autumn-27/norma/tool"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
actool "github.com/Autumn-27/norma/tool"
)
// hasSkillTool reports whether the packed tool set contains the Skill meta-tool.
+9 -9
View File
@@ -7,19 +7,19 @@ import (
func TestGoalGateCheck(t *testing.T) {
good := map[string]string{
"url target": "Audit " + strings.Repeat("x ", 60) + " at https://example.com/app every module for injection issues and record one finding per issue with evidence.",
"bare domain": strings.Repeat("probe ", 25) + "the api at target.example.org for unauthenticated data exposure, one record proof, then stop.",
"ipv4": strings.Repeat("recon ", 25) + "10.0.0.5 and map every listening service before any test.",
"localhost port": strings.Repeat("verify ", 25) + "localhost:3000 health endpoints respond identically before and after config reload.",
"path": strings.Repeat("review ", 25) + "the module at /app/data/core for hardcoded secrets and report each hit.",
"targetless opt-out": "[targetless] " + strings.Repeat("summarize ", 20) + "the attached material into a prioritized hypothesis list.",
"url target": "Audit " + strings.Repeat("x ", 60) + " at https://example.com/app every module for injection issues and record one finding per issue with evidence.",
"bare domain": strings.Repeat("probe ", 25) + "the api at target.example.org for unauthenticated data exposure, one record proof, then stop.",
"ipv4": strings.Repeat("recon ", 25) + "10.0.0.5 and map every listening service before any test.",
"localhost port": strings.Repeat("verify ", 25) + "localhost:3000 health endpoints respond identically before and after config reload.",
"path": strings.Repeat("review ", 25) + "the module at /app/data/core for hardcoded secrets and report each hit.",
"targetless opt-out": "[targetless] " + strings.Repeat("summarize ", 20) + "the attached material into a prioritized hypothesis list.",
}
good["targetless short but marked"] = "[targetless] analyze nothing"
bad := map[string]string{
"no target long": strings.Repeat("do a thorough security review of everything and report anything interesting that comes up during the exploration phase of this audit, thanks. ", 2),
"too short": "audit the site",
"short and bare": "find bugs",
"no target long": strings.Repeat("do a thorough security review of everything and report anything interesting that comes up during the exploration phase of this audit, thanks. ", 2),
"too short": "audit the site",
"short and bare": "find bugs",
}
for name, g := range good {
if name == "targetless short+:url" {
+7 -7
View File
@@ -55,13 +55,13 @@ type Task struct {
LLMChainRevision int64 `json:"-"`
LLMFailoverState string `json:"llm_failover_state,omitempty"`
// LLMRotate (#85): per-intent round-robin over the task's LLM chain heads.
LLMRotate bool `json:"llm_rotate,omitempty"`
rotateMu sync.Mutex
rotateNext int
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
SourceTaskIDs []int64 `json:"source_task_ids,omitempty"`
CompanyIDs []int64 `json:"company_ids,omitempty"`
Status string `json:"status"` // persisted lifecycle status (done/failed/timeout are terminal; empty/other is derived from runtime state)
LLMRotate bool `json:"llm_rotate,omitempty"`
rotateMu sync.Mutex
rotateNext int
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
SourceTaskIDs []int64 `json:"source_task_ids,omitempty"`
CompanyIDs []int64 `json:"company_ids,omitempty"`
Status string `json:"status"` // persisted lifecycle status (done/failed/timeout are terminal; empty/other is derived from runtime state)
// Task-level timeout (see docs/task-timeout-and-settle-design.md). DeadlineAt/FirstRunAt are unix seconds, 0 = unset/not yet run.
TimeoutSeconds int `json:"timeout_seconds"`
PlanHeartbeatSeconds int `json:"plan_heartbeat_seconds"` // planner heartbeat trigger interval (seconds)
+7 -7
View File
@@ -1609,12 +1609,12 @@ type createTaskReq struct {
CategoryID *int64 `json:"category_id,omitempty"`
Description string `json:"description"`
Goal string `json:"goal"`
LLMProfileID *int64 `json:"llm_profile_id,omitempty"` // the LLM profile to run this task on; omitted/null = use the active profile
LLMProfileIDs []int64 `json:"llm_profile_ids,omitempty"` // ordered task-level profile chain; the first entry takes effect initially
SourceTaskIDs []string `json:"source_task_ids,omitempty"` // direct, read-only inherited source tasks
CompanyIDs []int64 `json:"company_ids,omitempty"` // links the company scope and snapshots its current assets; does not copy assets or force intent generation
LLMProfileID *int64 `json:"llm_profile_id,omitempty"` // the LLM profile to run this task on; omitted/null = use the active profile
LLMProfileIDs []int64 `json:"llm_profile_ids,omitempty"` // ordered task-level profile chain; the first entry takes effect initially
SourceTaskIDs []string `json:"source_task_ids,omitempty"` // direct, read-only inherited source tasks
CompanyIDs []int64 `json:"company_ids,omitempty"` // links the company scope and snapshots its current assets; does not copy assets or force intent generation
TimeoutSeconds int `json:"timeout_seconds"`
LLMRotate *bool `json:"llm_rotate"` // task-level timeout (seconds); 0/omitted = no limit
LLMRotate *bool `json:"llm_rotate"` // task-level timeout (seconds); 0/omitted = no limit
PlanHeartbeatSeconds int `json:"plan_heartbeat_seconds"` // planner heartbeat interval (seconds); 0/omitted = default 600 (10min); floor = default = 600, lower values are raised to 600
PlanQuietMode *bool `json:"plan_quiet_mode,omitempty"` // planner quiet-mode switch (#86); omitted = default on; false = edge rounds always plan
TargetGate *bool `json:"target_gate,omitempty"` // approval gate for asset targets, opt-in; omitted/null = off
@@ -1927,11 +1927,11 @@ func (s *Server) getTask(w http.ResponseWriter, r *http.Request) {
return
}
dto := taskDTO(t, s.resolvedTaskStatus(t))
metrics, err := s.m.PG().TaskListMetricsAll()
metrics, err := s.m.PG().TaskListMetrics(t.ExpID)
if err != nil {
log.Printf("[tasks] detail metrics query failed for %s — metrics zeroed: %v", t.ID, err)
}
s.applyTaskMetrics(&dto, t, metrics[t.ExpID])
s.applyTaskMetrics(&dto, t, metrics)
archiveBlockers, _ := s.m.PG().TaskArchiveBlockers()
applyTaskArchiveBlocker(&dto, archiveBlockers)
writeJSON(w, 200, dto)
+47 -10
View File
@@ -38,6 +38,21 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExpID); err != nil
t.Fatal(err)
}
// A second task, so the detail must pick its own row and the list must keep
// serving every task.
other, err := m.CreateTask("detail metrics other", "goal", nil, 0, 0)
if err != nil {
t.Fatal(err)
}
otherID, _ := strconv.ParseInt(other.ID, 10, 64)
defer func() { _, _ = m.pg.DeleteTaskCascade(otherID, false, true) }()
defer func() { _, _ = m.pg.Exec(`DELETE FROM findings WHERE task_id=$1`, taskID) }()
if _, err := m.pg.Exec(`
INSERT INTO findings(task_id, severity) VALUES ($1,'critical'), ($1,'high'), ($1,'high'), ($1,'low'), ($2,'medium')`,
taskID, otherID); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
defer stopTestServer(t, s, cancel)
@@ -45,7 +60,19 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExpID); err != nil
if err != nil {
t.Fatal(err)
}
// After New: restoring the task resets running intents to open.
if _, err := m.pg.Exec(`
INSERT INTO exploration_nodes(exploration_id, kind, payload, state)
VALUES ($1,'intent','{}','running'), ($1,'intent','{}','done')`, task.ExpID); err != nil {
t.Fatal(err)
}
type findingCounts struct {
Critical int `json:"critical"`
High int `json:"high"`
Medium int `json:"medium"`
Low int `json:"low"`
}
var probe struct {
Tokens struct {
InputTokens int `json:"input_tokens"`
@@ -53,8 +80,10 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExpID); err != nil
CacheReadTokens int `json:"cache_read_tokens"`
CacheWriteTokens int `json:"cache_write_tokens"`
} `json:"tokens"`
GoalsMet int `json:"goals_met"`
GoalsTotal int `json:"goals_total"`
GoalsMet int `json:"goals_met"`
GoalsTotal int `json:"goals_total"`
InFlight int `json:"in_flight"`
Findings findingCounts `json:"findings"`
}
req := httptest.NewRequest(http.MethodGet, "/api/tasks/"+task.ID, nil)
req.Header.Set("Authorization", "Bearer "+token)
@@ -73,6 +102,9 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExpID); err != nil
if probe.GoalsMet != 1 || probe.GoalsTotal != 2 {
t.Fatalf("detail goal progress not filled from metrics: %d/%d", probe.GoalsMet, probe.GoalsTotal)
}
if probe.InFlight != 1 || probe.Findings != (findingCounts{Critical: 1, High: 2, Low: 1}) {
t.Fatalf("detail in-flight/findings not filled from this task's metrics: %d %+v", probe.InFlight, probe.Findings)
}
// The list endpoint must report the same numbers for the same task.
listReq := httptest.NewRequest(http.MethodGet, "/api/tasks", nil)
@@ -89,27 +121,32 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExpID); err != nil
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
} `json:"tokens"`
GoalsMet int `json:"goals_met"`
GoalsTotal int `json:"goals_total"`
GoalsMet int `json:"goals_met"`
GoalsTotal int `json:"goals_total"`
InFlight int `json:"in_flight"`
Findings findingCounts `json:"findings"`
} `json:"tasks"`
}
if err := json.Unmarshal(listRec.Body.Bytes(), &listed); err != nil {
t.Fatal(err)
}
found := false
found, foundOther := false, false
for _, lt := range listed.Tasks {
if lt.ID == other.ID {
foundOther = lt.Findings == (findingCounts{Medium: 1})
}
if lt.ID != task.ID {
continue
}
found = true
if lt.Tokens.InputTokens != probe.Tokens.InputTokens ||
lt.Tokens.OutputTokens != probe.Tokens.OutputTokens ||
lt.GoalsMet != probe.GoalsMet || lt.GoalsTotal != probe.GoalsTotal {
t.Fatalf("list and detail metrics disagree: list=%+v/%d detail=%+v/%d",
lt.Tokens, lt.GoalsMet, probe.Tokens, probe.GoalsTotal)
lt.GoalsMet != probe.GoalsMet || lt.GoalsTotal != probe.GoalsTotal ||
lt.InFlight != probe.InFlight || lt.Findings != probe.Findings {
t.Fatalf("list and detail metrics disagree: list=%+v detail=%+v", lt, probe)
}
}
if !found {
t.Fatalf("task %s missing from list response", task.ID)
if !found || !foundOther {
t.Fatalf("list misses a task or its metrics: task %s found=%v, task %s with its findings=%v", task.ID, found, other.ID, foundOther)
}
}
-1
View File
@@ -77,4 +77,3 @@ func TestEscalationAdvancesAfterConsecutiveFailures(t *testing.T) {
t.Fatalf("exhaust calls=%d want 2", exhaustCalls)
}
}
+1 -1
View File
@@ -5,9 +5,9 @@ import (
"encoding/json"
"testing"
actool "github.com/Autumn-27/norma/tool"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
actool "github.com/Autumn-27/norma/tool"
)
func names(tools []actool.CoreTool) map[string]actool.CoreTool {
+63
View File
@@ -0,0 +1,63 @@
package traffic
import (
"context"
"log"
"net/http"
"time"
"github.com/Autumn-27/artex/egress"
mproxy "github.com/lqqyt2423/go-mitmproxy/proxy"
)
// paceWaitMax bounds the pacing wait of one request. Past it the request is
// sent unpaced: pacing never blocks a command indefinitely.
const paceWaitMax = 30 * time.Second
// Requestheaders runs for every request the proxy sees: the CONNECT that opens
// a tunnel, each request inside an intercepted tunnel, and each plain-HTTP
// request. The recording proxy takes no credentials, so a Proxy-Authorization
// header is only ever meant for it (clients send the proxy URL's userinfo
// there): it is dropped here, before anything is forwarded, and never reaches
// the target or the recording. When it carries the pace marker
// (egress.PaceUser), the client connection is marked; every request on it,
// including those inside its tunnel, is then paced by the current noise
// profile before it goes upstream. Unmarked connections are not paced here.
func (s *sink) Requestheaders(f *mproxy.Flow) {
if f.Request == nil || f.ConnContext == nil || f.ConnContext.ClientConn == nil {
return
}
id := f.ConnContext.ClientConn.Id
if egress.IsPaceMarker(f.Request.Header.Get("Proxy-Authorization")) {
s.paced.Store(id, struct{}{})
}
f.Request.Header.Del("Proxy-Authorization")
if f.Request.Method == http.MethodConnect || f.Request.URL == nil {
return
}
if _, marked := s.paced.Load(id); marked {
pace(f)
}
}
// ClientDisconnected forgets a closed connection's mark.
func (s *sink) ClientDisconnected(c *mproxy.ClientConn) {
if c != nil {
s.paced.Delete(c.Id)
}
}
// pace waits for the request's host under the current noise profile, bounded
// by paceWaitMax and by the request's own context. Off returns at once.
func pace(f *mproxy.Flow) {
ctx := context.Background()
if raw := f.Request.Raw(); raw != nil {
ctx = raw.Context()
}
ctx, cancel := context.WithTimeout(ctx, paceWaitMax)
defer cancel()
host := egress.HostKey(f.Request.URL)
if err := egress.Wait(ctx, host); err != nil {
log.Printf("[traffic] pacing a Bash request to %s stopped waiting (%v); sending it unpaced", host, err)
}
}
+262
View File
@@ -0,0 +1,262 @@
package traffic
import (
"crypto/tls"
"crypto/x509"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"strings"
"sync"
"testing"
"time"
"github.com/Autumn-27/artex/egress"
mproxy "github.com/lqqyt2423/go-mitmproxy/proxy"
)
// startProxy runs a real recording proxy on a free loopback port and returns it
// with its sink and URL. extra addons run after the recording sink.
func startProxy(t *testing.T, extra ...mproxy.Addon) (*Traffic, *sink, string) {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := ln.Addr().String()
ln.Close()
tr, err := Open(t.TempDir(), addr)
if err != nil {
t.Fatal(err)
}
s := tr.proxy.Addons[0].(*sink)
for _, a := range extra {
tr.proxy.AddAddon(a)
}
go func() { _ = tr.Start() }()
t.Cleanup(func() {
_ = tr.proxy.Close()
_ = tr.Close()
})
deadline := time.Now().Add(5 * time.Second)
for {
c, err := net.Dial("tcp", addr)
if err == nil {
c.Close()
break
}
if time.Now().After(deadline) {
t.Fatalf("proxy did not listen on %s: %v", addr, err)
}
time.Sleep(10 * time.Millisecond)
}
return tr, s, "http://" + addr
}
// proxyClient is a curl-like client: it sends the proxy URL's userinfo as
// Proxy-Authorization on CONNECT and on plain-HTTP requests, and trusts the CA.
func proxyClient(t *testing.T, tr *Traffic, proxyURL string) *http.Client {
t.Helper()
pem, err := os.ReadFile(tr.CACertPath())
if err != nil {
t.Fatal(err)
}
pool := x509.NewCertPool()
if !pool.AppendCertsFromPEM(pem) {
t.Fatal("no CA in", tr.CACertPath())
}
u, _ := url.Parse(proxyURL)
return &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(u), TLSClientConfig: &tls.Config{RootCAs: pool}}}
}
// probe records the ClientConn of each request the proxy sees, after the sink.
type probe struct {
mproxy.BaseAddon
mu sync.Mutex
conn map[string]string // "METHOD url" -> ClientConn.Id
}
func (p *probe) Requestheaders(f *mproxy.Flow) {
p.mu.Lock()
defer p.mu.Unlock()
if p.conn == nil {
p.conn = map[string]string{}
}
p.conn[f.Request.Method+" "+f.Request.URL.String()] = f.ConnContext.ClientConn.Id.String()
}
func get(t *testing.T, c *http.Client, u string) {
t.Helper()
resp, err := c.Get(u)
if err != nil {
t.Fatal(err)
}
_, _ = io.Copy(io.Discard, resp.Body)
resp.Body.Close()
if resp.StatusCode != 200 {
t.Fatalf("GET %s: %d", u, resp.StatusCode)
}
}
func markedCount(s *sink) int {
n := 0
s.paced.Range(func(any, any) bool { n++; return true })
return n
}
func hostRPM(host string) int {
for _, h := range egress.Current().Snapshot() {
if h.Host == host {
return h.RPM
}
}
return 0
}
// A marked client (the Bash proxy URL) is paced and counted, through a CONNECT
// tunnel and on plain HTTP; an unmarked one (WebFetch, custom tools, enrich)
// is not paced here. No Proxy-Authorization reaches a target or the recording,
// every exchange is recorded, and a closed connection loses its mark.
func TestBashPacingThroughProxy(t *testing.T) {
prev := egress.CurrentProfile()
t.Cleanup(func() { egress.SetProfile(prev) })
// 2 req/s, burst 1, no jitter: from the second request on, each waits 500 ms.
egress.SetProfile(egress.Profile{Name: "test", RPS: 2, Burst: 1, MaxPerHost: 1, MaxGlobal: 8})
var mu sync.Mutex
leaked := map[string]string{}
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if v := r.Header.Get("Proxy-Authorization"); v != "" {
mu.Lock()
leaked[r.URL.Path] = v
mu.Unlock()
}
_, _ = io.WriteString(w, "ok")
})
marked := httptest.NewServer(handler)
defer marked.Close()
unmarked := httptest.NewServer(handler)
defer unmarked.Close()
tlsSrv := httptest.NewTLSServer(handler)
defer tlsSrv.Close()
// The MITM leaf for a bare IP has no IP SAN: address the TLS target by name.
tlsURL := strings.Replace(tlsSrv.URL, "127.0.0.1", "localhost", 1)
p := &probe{}
tr, s, proxyURL := startProxy(t, p)
bash := proxyClient(t, tr, egress.MarkProxyURL(proxyURL))
plain := proxyClient(t, tr, proxyURL)
start := time.Now()
for range 3 {
get(t, bash, marked.URL+"/bash")
}
if d := time.Since(start); d < 900*time.Millisecond {
t.Fatalf("3 marked requests to one host took %v, want >= 1s of pacing", d)
}
if n := hostRPM(strings.TrimPrefix(marked.URL, "http://")); n != 3 {
t.Fatalf("egress counter for the marked host = %d, want 3", n)
}
for range 3 {
get(t, plain, unmarked.URL+"/plain")
}
if n := hostRPM(strings.TrimPrefix(unmarked.URL, "http://")); n != 0 {
t.Fatalf("unmarked requests were paced by the proxy: counter %d, want 0", n)
}
get(t, bash, tlsURL+"/tls1")
get(t, bash, tlsURL+"/tls2")
tlsHost := strings.TrimPrefix(tlsURL, "https://")
if n := hostRPM(tlsHost); n != 2 {
t.Fatalf("egress counter for the tunnelled host = %d, want 2", n)
}
p.mu.Lock()
connectID := p.conn["CONNECT //"+tlsHost]
inner1, inner2 := p.conn["GET "+tlsURL+"/tls1"], p.conn["GET "+tlsURL+"/tls2"]
p.mu.Unlock()
if connectID == "" || inner1 != connectID || inner2 != connectID {
t.Fatalf("tunnel flows do not share the CONNECT's connection: connect=%q inner=%q,%q", connectID, inner1, inner2)
}
mu.Lock()
if len(leaked) != 0 {
t.Fatalf("Proxy-Authorization reached the target: %v", leaked)
}
mu.Unlock()
rows, err := tr.Search("", 0, 100)
if err != nil || len(rows) != 8 {
t.Fatalf("recorded %d exchanges (err %v), want 8", len(rows), err)
}
for _, r := range rows {
req, _, err := tr.Get(r.ID)
if err != nil || strings.Contains(strings.ToLower(req), "proxy-authorization") {
t.Fatalf("recorded request %s keeps Proxy-Authorization (err %v):\n%s", r.ID, err, req)
}
}
if markedCount(s) == 0 {
t.Fatal("no connection marked")
}
bash.CloseIdleConnections()
plain.CloseIdleConnections()
deadline := time.Now().Add(5 * time.Second)
for markedCount(s) != 0 {
if time.Now().After(deadline) {
t.Fatalf("%d marks left after the clients closed", markedCount(s))
}
time.Sleep(10 * time.Millisecond)
}
}
// Off: a marked request is neither paced nor counted.
func TestBashPacingOff(t *testing.T) {
prev := egress.CurrentProfile()
t.Cleanup(func() { egress.SetProfile(prev) })
egress.SetProfile(egress.Profile{Name: "off"})
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
defer srv.Close()
tr, _, proxyURL := startProxy(t)
bash := proxyClient(t, tr, egress.MarkProxyURL(proxyURL))
for range 3 {
get(t, bash, srv.URL+"/")
}
if snap := egress.Current().Snapshot(); len(snap) != 0 {
t.Fatalf("off counted marked requests: %v", snap)
}
}
// The sink drops Proxy-Authorization from every request and marks only on the
// pace marker; a disconnect clears the mark.
func TestSinkStripsAndMarks(t *testing.T) {
prev := egress.CurrentProfile()
t.Cleanup(func() { egress.SetProfile(prev) })
egress.SetProfile(egress.Profile{Name: "off"})
s := &sink{}
flow := func(method, auth string, conn *mproxy.ClientConn) *mproxy.Flow {
u, _ := url.Parse("http://a.example/")
h := http.Header{}
if auth != "" {
h.Set("Proxy-Authorization", auth)
}
return &mproxy.Flow{Request: &mproxy.Request{Method: method, URL: u, Header: h}, ConnContext: &mproxy.ConnContext{ClientConn: conn}}
}
f := flow(http.MethodGet, "Basic dXNlcjpwYXNz", &mproxy.ClientConn{}) // user:pass, not the marker
s.Requestheaders(f)
if f.Request.Header.Get("Proxy-Authorization") != "" || markedCount(s) != 0 {
t.Fatalf("foreign credentials: header %q, marks %d", f.Request.Header.Get("Proxy-Authorization"), markedCount(s))
}
conn := &mproxy.ClientConn{}
conn.Id[0] = 1
f = flow(http.MethodConnect, "Basic YXJ0ZXgtcGFjZTphcnRleC1wYWNl", conn) // artex-pace:artex-pace
s.Requestheaders(f)
if f.Request.Header.Get("Proxy-Authorization") != "" || markedCount(s) != 1 {
t.Fatalf("marker on CONNECT: header %q, marks %d", f.Request.Header.Get("Proxy-Authorization"), markedCount(s))
}
s.ClientDisconnected(conn)
if markedCount(s) != 0 {
t.Fatal("mark kept after disconnect")
}
}
+4 -2
View File
@@ -383,10 +383,12 @@ func (t *Traffic) stopping() bool {
}
func (t *Traffic) DB() *sql.DB { return t.db }
// sink is the go-mitmproxy addon that records completed exchanges.
// sink is the go-mitmproxy addon that records completed exchanges, and paces
// the requests of client connections marked for pacing (see pace.go).
type sink struct {
mproxy.BaseAddon
t *Traffic
t *Traffic
paced sync.Map // ClientConn.Id of a marked connection -> struct{}
}
func (s *sink) Response(f *mproxy.Flow) {
@@ -78,8 +78,11 @@ export function NoiseProfileCard() {
</CardTitle>
<CardDescription>
Paces the requests ARTEX itself sends to targets (WebFetch, http custom tools, the enrich probe) and sets the
low-and-slow guidance in agent prompts. Commands run through Bash are not paced. This setting takes
precedence over ARTEX_NOISE_PROFILE and applies to the next request.
low-and-slow guidance in agent prompts. With traffic capture on, Bash commands that use the proxy (curl,
wget, python-requests…) are paced too, by rate and jitter only: the in-flight, cache and retry limits apply
to ARTEX's own requests. Raw scanners such as nmap, and commands that bypass the proxy, are not paced; this
is best effort, not a security boundary. This setting takes precedence over ARTEX_NOISE_PROFILE and applies
to the next request.
</CardDescription>
</CardHeader>
<CardContent className="flex flex-col gap-4">