Compare commits

...
Author SHA1 Message Date
BaaRai 15ed120758 test(traffic): adversarial coverage for the replay rewrite toggle
Duplicate and lower-case nonce headers are stripped without bypassing,
the registry is single-use and rejects multi-valued fields, a forged
caller header cannot steer either mode, concurrent bypass and apply
replays do not share state, SSE streams bypass by default, a cancelled
replay leaves no nonce or marker, and the route rejects a non-boolean
apply_rules.
2026-11-20 05:48:34 +01:00
BaaRai 221ad67549 feat(traffic): per-replay rewrite-rule toggle and RewriteStats API (P3.3b2b1) 2026-11-20 05:25:35 +01:00
BaaRai 90c002befc Merge pull request #392 from BaaRai/feat/evidence-rewrite-flag
feat(evidence): record capture-proxy rewrite provenance on traffic snapshots
2026-11-20 04:33:29 +01:00
BaaRai 9562248508 feat(evidence): record capture-proxy rewrite provenance on traffic snapshots (P3.3b2a)
Add rewritten and rewrite_rule_ids provenance columns to traffic_evidence_snapshots (migration 0008), enrich prepared snapshots from traffic.RewriteInfo outside the ReadEvidence callback, exclude both fields from TrafficSnapshotID so snapshot IDs remain compatible with older binaries, and display applied rule IDs in TrafficEvidenceViewer (with tr.json i18n entries).

Note: snapshots created before migration 0008 keep rewritten=false because InsertEvidenceSnapshotTx uses ON CONFLICT DO NOTHING and does not backfill provenance on existing rows; this is accepted and intentionally not backfilled here.
2026-11-20 04:00:34 +01:00
BaaRai bd95c2669f Merge pull request #375 from Alexandre200-lab/main
Refactor PostgreSQL test skipping logic and improve documentation
2026-11-20 03:33:50 +01:00
BaaRai 0a968d71ae docs: correct the proxy pin and argv descriptions
Pins are name-only behind a proxy (name-only since #370), and deny/ask rules now inspect compound commands (#377); the allow rule stays strict. Also state the untrusted-boundary default plainly.
2026-11-20 03:18:21 +01:00
BaaRai 705ac80128 Merge pull request #380 from Amineharrabi/feat/stream-messages-direction-filter
feat(traffic): filter stream messages by direction
2026-11-20 03:06:08 +01:00
BaaRai ba84c374a3 Merge pull request #390 from BaaRai/security/llm-models-ssrf
security(server): block SSRF in POST /api/llm/models
2026-11-20 02:58:03 +01:00
unknownandClaude Opus 5.5 d22e9e0e58 feat(traffic): filter stream messages by direction
GET /api/traffic/stream and the traffic_messages tool take an optional
direction (c2s or s2c) to list one side of a WebSocket or SSE
conversation. total and offset apply to the filtered list, seq stays the
position in the whole stream, and the meta limit marker is kept in a
filtered page so dropped messages are never hidden. Any other value is
rejected (400 / tool error).

Part of #379 (traffic inspection: filter by direction).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:37:23 +01:00
Alexandrelab-200 33684f378b test(cmd): make snapshot test independent of Redact internals 2026-11-19 14:55:18 -03:00
Alexandrelab-200 1f0b30cb6a fix(test): assert redaction without pinning the mask marker 2026-11-19 14:48:46 -03:00
Alexandrelab-200 cb0915319c fix(test): use pgdb alias for skip helper in aliased imports 2026-11-19 14:43:29 -03:00
Alexandrelab-200 f081449d58 fix(db): declare Skip on the test skipper interface 2026-11-19 14:38:23 -03:00
Alexandrelab-200 f85eb86103 Merge remote-tracking branch 'refs/remotes/origin/main' 2026-11-19 14:25:18 -03:00
Alexandrelab-200 db0aae3657 Refactor PostgreSQL test skipping logic
- Introduced uniform skipping functions in the db package: SkipPGUnavailable and SkipNoPG.
- Updated all relevant test files to utilize these new functions for better consistency and readability.
- Added new db/testpg.go file to house the skipping logic.
- Created architecture and operations documentation to provide a comprehensive overview of the ARTEX system.
- Added a verification script to streamline local testing and ensure code quality.
2026-11-19 14:23:29 -03:00
133 changed files with 2422 additions and 323 deletions
+1 -1
View File
@@ -125,7 +125,7 @@ jobs:
# shellcheck is preinstalled on the ubuntu runners. -S warning: the scripts
# currently have only info-level notes (intentional A && B || C chains in
# update.sh, an unquoted glob in backup.sh's prune), so fail on warnings.
- run: shellcheck -S warning install.sh update.sh start.sh scripts/backup.sh reset-password.sh scripts/extract-changelog-section.sh scripts/test-extract-changelog-section.sh .github/scripts/bot.sh scripts/test-bot-context.sh
- run: shellcheck -S warning install.sh update.sh start.sh scripts/backup.sh scripts/verify.sh reset-password.sh scripts/extract-changelog-section.sh scripts/test-extract-changelog-section.sh .github/scripts/bot.sh scripts/test-bot-context.sh
- run: bash scripts/test-extract-changelog-section.sh
# The triage bot's repository summary must expose every top-level
# directory; a truncated file list once made it deny that docs/ and
+2 -1
View File
@@ -24,8 +24,9 @@ All notable changes to this project are documented in this file. The format is b
- **Opt-in update banner**: `ARTEX_UPDATE_CHECK` makes the backend check the latest GitHub release at most once per 24 hours and the UI show a banner; off by default (#202).
- **Release provenance**: release archives carry GitHub build provenance attestations and a CycloneDX SBOM (`gh attestation verify <zip> --repo BaaRai/ARTEX`); release notes are the matching `CHANGELOG.md` section (#219, #201).
- **Capture-proxy rewrite rules, storage and API**: `GET`, `POST`, `PUT` and `DELETE /api/traffic/rewrite-rules` and `POST /api/traffic/rewrite-rules/{id}/toggle` manage rules that match a request or a response (method, host, URL, headers, body, status) and describe a rewrite (method, path and query, headers, body, status). Rules are validated on save, limited to 100 of which 50 enabled, created disabled, and stored in the new table `traffic_rewrite_rules` (migration `0005_traffic_rewrite_rules.sql`) with `matchers` and `actions` encrypted at rest. The value of an action marked `sensitive` is never returned, and an update sent without it keeps the stored one. The server compiles the enabled rules at startup and after every change and hands the snapshot to the proxy. The proxy applies them from the next entry. See `docs/traffic-rewrite-rules.md`.
- **Capture-proxy rewrite rules applied**: the capture proxy applies enabled rules to each HTTP request before it is sent to the target and to each response before it reaches the client. Each action that ran is recorded in the new table `exchange_rewrites` in the same transaction as the exchange, `traffic_get` shows a `REWRITTEN` line, and `traffic_search` never finds a value a rule wrote. SSE status and headers are rewritten at the start of the stream. **WebSocket handshakes are not rewritten**: the capture library runs its hook after the upgrade, so the status and headers cannot be changed, and frames pass unchanged. **Responses of 5 MiB or more** are forwarded unchanged and not recorded in full, because the library does not call the response hook for them. Snapshots of original requests are capped at 1024 entries and 64 MiB. Replays go through the proxy, so the rules apply to them. Archives, a per-replay rule toggle, the web UI and the proof flag come later.
- **Capture-proxy rewrite rules applied**: the capture proxy applies enabled rules to each HTTP request before it is sent to the target and to each response before it reaches the client. Each action that ran is recorded in the new table `exchange_rewrites` in the same transaction as the exchange, `traffic_get` shows a `REWRITTEN` line, and `traffic_search` never finds a value a rule wrote. SSE status and headers are rewritten at the start of the stream. **WebSocket handshakes are not rewritten**: the capture library runs its hook after the upgrade, so the status and headers cannot be changed, and frames pass unchanged. **Responses of 5 MiB or more** are forwarded unchanged and not recorded in full, because the library does not call the response hook for them. Snapshots of original requests are capped at 1024 entries and 64 MiB. Replays (`POST /api/traffic/replay`) bypass capture rewrite rules by default via an internal `X-ARTEX-Replay-Nonce` header stripped unconditionally by the proxy, and apply the rules when opted in with `apply_rules: true` (the **Apply capture rewrite rules** checkbox in the Replay panel). `GET /api/traffic` exposes the proxy's `rewrite` counters (`large_responses`, `capacity_refusals`, `tombstone_drops`, `conversions`, `sweeps`). Archives and the rule-management / traffic-list web UI come later.
- **Bulk finding status** (#298): select findings in the findings table and apply one disposition to all of them, up to 500 per call, in one transaction; each row still goes through the single-decision notification semantics and the change is mirrored onto its exploration node (#289).
- **Stream message direction filter** (#379): `GET /api/traffic/stream` and the `traffic_messages` agent tool take an optional `direction` (`c2s` or `s2c`) that lists only one side of a WebSocket or SSE conversation. `total` and `offset` apply to the filtered list, each row keeps its `seq` in the whole stream, and the `meta` row (messages dropped past the recorder limits) stays in a filtered page. Any other value is a 400 (API) or a tool error.
- **Traffic replay** (#266): replay a recorded request from the traffic detail with its method, URL, headers and stored body; replay is disabled when only the body preview is loaded.
- **Finding proof strength in reports** (#268): the Markdown report and the export show each finding's proof strength next to its verification status.
- **Opt-in Keenable web search backend** (#295): the search tool gains Keenable alongside Brave and Tavily; the key is stored encrypted and can be cleared independently.
+5
View File
@@ -659,6 +659,7 @@ CGO_ENABLED=0 go build -tags embedui -o artex ./cmd/artex
- Backend: `go run ./cmd/artex` (without `-tags embedui`, the frontend is not embedded)
- Frontend: `cd web && npm run dev` (`/api` is reverse-proxied to the backend, with hot reload)
- Tests: `go test ./...`
- Reproduce all CI checks locally: `./scripts/verify.sh` (add `--full` for `-race` and the static web build)
- Mock preview (no backend): `cd web && NEXT_PUBLIC_MOCK=1 npm run dev`
---
@@ -726,6 +727,10 @@ flowchart TB
| **db** | PostgreSQL persistence (pgx); embedded schema and numbered migrations applied at startup |
| **Supporting** | Recording MITM proxy, approval gate, async enrichment, MCP/skills/memory/report |
For the code-level map (execution flow, trust boundaries, decisions) see
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md); for startup, configuration precedence,
backup and key rotation see [docs/OPS.md](docs/OPS.md).
---
## Security
+9 -1
View File
@@ -26,7 +26,15 @@ func callReadJSON(t *testing.T, tool actool.CoreTool, input string) any {
}
func TestGraphOverviewExpandsAssociatedCompanyScope(t *testing.T) {
t.Skip("known failure with a fresh database, pre-existing on unmodified upstream; tracked in BaaRai/ARTEX#9")
// Known failure with a freshly seeded database (BaaRai/ARTEX#9). Repro:
// ARTEX_PG_DSN=postgres://artex:testpass@localhost:5432/artextest_agent \
// go test ./agent -run TestGraphOverviewExpandsAssociatedCompanyScope -count=1 -v
// On a fresh DB the company-scope expansion in graphOverviewData is not
// visible to the task: the t.Fatalf checks after QueryByTask and
// graphOverviewData (company asset linkage, company_scope rows and
// company_keywords) fail. Keep the skip until seeding is ordered correctly;
// do not delete it.
t.Skip("known failure with a fresh database; repro and failing assertions documented above; BaaRai/ARTEX#9")
d := testDB(t)
defer d.Close()
+9 -2
View File
@@ -17,10 +17,17 @@ import (
// Exercise the actual SDK event -> hook -> execution -> result path. No real
// model or command is used; the probe tool only returns a fixed string.
func TestCaptureApprovalLifecycle(t *testing.T) {
t.Skip("known failure with a fresh database, pre-existing on unmodified upstream; tracked in BaaRai/ARTEX#9")
// Known failure with a freshly seeded database (BaaRai/ARTEX#9). Repro:
// ARTEX_PG_DSN=postgres://artex:testpass@localhost:5432/artextest_agent \
// go test ./agent -run TestCaptureApprovalLifecycle -count=1 -v
// The checks after ListTaskIntercepts / GetInterceptDetail and the
// "unexpected review" t.Fatalf below fail before any other test has seeded
// intercept state. Keep this skipped until the fresh-DB setup is fixed;
// do not simply delete the skip.
t.Skip("known failure with a fresh database; repro and failing assertion documented above; BaaRai/ARTEX#9")
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+2 -2
View File
@@ -15,11 +15,11 @@ func testDB(t *testing.T) *db.DB {
t.Helper()
dsn, _, err := db.DSN()
if err != nil {
t.Skipf("no database config (%v)", err)
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
return d
}
+1 -1
View File
@@ -19,7 +19,7 @@ import (
func TestWorkerReviewContextAcrossToolCalls(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+5
View File
@@ -5,6 +5,11 @@ import (
"strings"
)
// UntrustedBoundaryEnabled reports whether the untrusted-output boundary is on.
// Exported for the startup configuration snapshot; the agents themselves keep
// calling the private helper below.
func UntrustedBoundaryEnabled() bool { return untrustedBoundaryEnabled() }
// untrustedBoundaryEnabled reports whether Norma's untrusted-output boundary
// (agentcore.Options.UntrustedOutputBoundary: tool results framed between
// nonce-tagged markers) is on. It is on by default: unset, empty or any other
+82 -28
View File
@@ -5,6 +5,7 @@ package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"log"
@@ -18,9 +19,41 @@ import (
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/config"
"github.com/Autumn-27/artex/egress"
"github.com/Autumn-27/artex/server"
)
// configSnapshot is the single startup log line describing the effective
// configuration. It carries no credential material: the DSN is passed through
// config.Redact, and no key or token value is ever included.
type configSnapshot struct {
Version string `json:"version"`
ConfigFile string `json:"config_file"`
ConfigFileFound bool `json:"config_file_found"`
PGDSN string `json:"pg_dsn"`
PGDSNSource string `json:"pg_dsn_source"`
DataDir string `json:"data_dir"`
SkillDir string `json:"skill_dir"`
KeyDir string `json:"key_dir"`
Addr string `json:"addr"`
Proxy string `json:"proxy"`
NoiseProfile string `json:"noise_profile"`
UntrustedBoundary bool `json:"untrusted_boundary"`
TrustedProxies []string `json:"trusted_proxies"`
}
// logConfigSnapshot emits one structured line so an operator can reproduce the
// exact configuration of a run from the logs. A serialization failure is
// reported but never aborts startup.
func logConfigSnapshot(s configSnapshot) {
b, err := json.Marshal(s)
if err != nil {
log.Printf("[config] snapshot unavailable: %v", err)
return
}
log.Printf("[config] snapshot %s", b)
}
// version is the build version, injected at release time via
// -ldflags "-X main.version=<tag>". Defaults to "dev" for local builds.
var version = "dev"
@@ -65,17 +98,59 @@ func run() int {
// page can show a live log stream. Do this first, to catch startup logs too.
server.StartLogCapture()
// surface which config file the binary reads (absolute, so `go run`'s relative
// "config.json" — resolved against the CWD — is unambiguous).
cfgPath := config.Path()
// Resolve every effective setting before starting the stores, then emit a
// single structured snapshot line. The old code logged the same facts in
// five scattered lines, which made a run hard to reproduce from logs.
cfgPath := config.Path() // absolute, so `go run`'s relative CWD is unambiguous
if abs, e := filepath.Abs(cfgPath); e == nil {
cfgPath = abs
}
if _, e := os.Stat(cfgPath); e == nil {
log.Printf("[config] config file: %s", cfgPath)
} else {
log.Printf("[config] config file: %s (not found — will only try the environment variable ARTEX_PG_DSN)", cfgPath)
_, statErr := os.Stat(cfgPath)
cfgFound := statErr == nil
skillDir, err := config.SkillDir()
if err != nil {
log.Printf("[config] %v", err)
return 1
}
if abs, e := filepath.Abs(skillDir); e == nil {
skillDir = abs
}
// JWT signing key directory (also holds secret.key, see server.NewManager).
keyDir := config.KeyDir()
// Proxies whose X-Forwarded-For the login rate limit reads; empty trusts none.
trusted, err := server.ParseTrustedProxies(os.Getenv("ARTEX_TRUSTED_PROXIES"))
if err != nil {
log.Printf("[config] ARTEX_TRUSTED_PROXIES: %v", err)
return 1
}
trustedAddrs := make([]string, 0, len(trusted))
for _, p := range trusted {
trustedAddrs = append(trustedAddrs, p.String())
}
// NewManager resolves the DSN too, but does so internally. Calling
// PostgresDSN here is idempotent (no connection, no side effects) and lets
// the snapshot show the source and a redacted view even on a failed start.
dsn, dsnSource, dsnErr := config.PostgresDSN()
if dsnErr != nil {
dsn, dsnSource = "", "unavailable"
}
logConfigSnapshot(configSnapshot{
Version: version,
ConfigFile: cfgPath,
ConfigFileFound: cfgFound,
PGDSN: config.Redact(dsn),
PGDSNSource: dsnSource,
DataDir: *dataDir,
SkillDir: skillDir,
KeyDir: keyDir,
Addr: *addr,
Proxy: *proxy,
NoiseProfile: egress.ProfileFromEnv().Name,
UntrustedBoundary: agent.UntrustedBoundaryEnabled(),
TrustedProxies: trustedAddrs,
})
sigCtx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
@@ -88,27 +163,6 @@ func run() int {
}
defer mgr.Close()
skillDir, err := config.SkillDir()
if err != nil {
log.Printf("[config] %v", err)
return 1
}
if abs, err := filepath.Abs(skillDir); err == nil {
skillDir = abs
}
log.Printf("[config] skill directory: %s", skillDir)
// JWT signing key directory (also holds secret.key, see server.NewManager).
keyDir := config.KeyDir()
log.Printf("[config] JWT key directory: %s", keyDir)
// Proxies whose X-Forwarded-For the login rate limit reads; empty trusts none.
trusted, err := server.ParseTrustedProxies(os.Getenv("ARTEX_TRUSTED_PROXIES"))
if err != nil {
log.Printf("[config] ARTEX_TRUSTED_PROXIES: %v", err)
return 1
}
if len(trusted) > 0 {
log.Printf("[config] trusted proxies: %v", trusted)
}
srv := server.New(ctx, mgr, skillDir, *dataDir, keyDir, server.WithSecretScrub(), server.WithTrustedProxies(trusted))
// ReadTimeout bounds how long the request headers and body may take to arrive.
// It does not limit how long a handler runs, so an event stream stays open.
+24
View File
@@ -2,11 +2,35 @@ package main
import (
"context"
"encoding/json"
"strings"
"testing"
"github.com/Autumn-27/artex/agent"
)
// The snapshot must be serializable and must not add fields. Password masking
// itself is config.Redact's contract, covered by config/config_test.go;
// cmd/artex only passes its result through to the log line.
func TestConfigSnapshotSerializesRedactedDSN(t *testing.T) {
redacted := "postgres://artex:***@localhost:5432/artex?sslmode=disable"
s := configSnapshot{
Version: "test",
ConfigFileFound: true,
PGDSN: redacted,
PGDSNSource: "environment variable ARTEX_PG_DSN",
TrustedProxies: []string{},
}
b, err := json.Marshal(s)
if err != nil {
t.Fatal(err)
}
out := string(b)
if !strings.Contains(out, `"pg_dsn":"`+redacted+`"`) {
t.Fatalf("snapshot did not serialize the redacted DSN: %s", out)
}
}
// The shutdown cause must survive the parent context being cancelled first;
// losing it would drop the diagnostic on every graceful stop.
func TestShutdownContextPreservesNamedCause(t *testing.T) {
+2 -2
View File
@@ -20,7 +20,7 @@ func TestActivityOutboxKeepsDetail(t *testing.T) {
func TestFlushActivityOutboxReplaysOnce(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("outbox", "replay")
@@ -63,7 +63,7 @@ func TestFlushActivityOutboxReplaysOnce(t *testing.T) {
func TestFlushActivityOutboxDropsStuckRow(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
const missing = int64(9_000_000_000_001)
+2 -2
View File
@@ -8,7 +8,7 @@ import "testing"
func TestActivityPageSessions(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -129,7 +129,7 @@ func TestActivityPageSessions(t *testing.T) {
func TestListByKindPage(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -14,7 +14,7 @@ func testSetup(t *testing.T) (*DB, *AssetStore, *CompanyStore) {
t.Helper()
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
return d, d.Assets(), d.Companies()
}
+1 -1
View File
@@ -168,7 +168,7 @@ func TestRewriteStockNpxBrowserMCPDoesNotAliasInput(t *testing.T) {
func TestSeedAndMigrateBrowserMCP(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
origCmd, _, origRaw := readBrowserMCP(t, d)
+12 -12
View File
@@ -17,7 +17,7 @@ func cleanupCompany(d *DB, id int64) {
func TestCompanyUpsertAndGet(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -64,7 +64,7 @@ func TestCompanyUpsertAndGet(t *testing.T) {
func TestCreateCompanyWithScopeRejectsNormalizedDuplicate(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -103,7 +103,7 @@ func TestCreateCompanyWithScopeRejectsNormalizedDuplicate(t *testing.T) {
func TestCreateCompanyWithScopeRollsBackOnScopeWriteFailure(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -128,7 +128,7 @@ func TestCreateCompanyWithScopeRollsBackOnScopeWriteFailure(t *testing.T) {
func TestCompanyScope(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -166,7 +166,7 @@ func TestCompanyScope(t *testing.T) {
func TestCompanyScopeInvalid(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -195,7 +195,7 @@ func TestCompanyScopeInvalid(t *testing.T) {
func TestResolveCompany(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -236,7 +236,7 @@ func TestResolveCompany(t *testing.T) {
func TestUpdateScope(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -281,7 +281,7 @@ func TestUpdateScope(t *testing.T) {
func TestUpdateScopeRollsBackOnInsertFailure(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -316,7 +316,7 @@ func TestUpdateScopeRollsBackOnInsertFailure(t *testing.T) {
func TestDeleteCompany(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -344,7 +344,7 @@ func TestDeleteCompany(t *testing.T) {
func TestDeleteCompanyWithAssetsDeletesBoth(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -391,7 +391,7 @@ RETURNING id`, domain, id).Scan(&assetID); err != nil {
func TestRecomputeAttribution(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
@@ -433,7 +433,7 @@ func TestRecomputeAttribution(t *testing.T) {
func TestListCompanies(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
cs := d.Companies()
+1 -1
View File
@@ -233,7 +233,7 @@ func TestCompanyScopeLimitsAndCheckedErrors(t *testing.T) {
func TestCompanyScopeCheckedReturnsSystemErrorSeparately(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
companies := d.Companies()
if err := d.Close(); err != nil {
+2 -2
View File
@@ -233,7 +233,7 @@ func TestParseStructuredCompanyScope(t *testing.T) {
func TestCompanyICPAttribution(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Closing the connection must go through t.Cleanup and be **registered before
// the data cleanup**: t.Cleanup is last-in-first-out, so registering Close
@@ -322,7 +322,7 @@ func TestCompanyICPAttribution(t *testing.T) {
func TestCompanyScopeAttribution(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
as := d.Assets()
cs := d.Companies()
+2 -2
View File
@@ -9,7 +9,7 @@ import "testing"
func TestProfileMaxTokensRoundTrip(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Close via Cleanup, registered FIRST so it runs LAST: cleanups are LIFO, and a
// plain `defer d.Close()` would fire before them — the row-deleting cleanups
@@ -77,7 +77,7 @@ func TestProfileMaxTokensRoundTrip(t *testing.T) {
func TestProfileMaxTokensDefaults(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Close via Cleanup, registered FIRST so it runs LAST: cleanups are LIFO, and a
// plain `defer d.Close()` would fire before them — the row-deleting cleanups
+2 -2
View File
@@ -17,7 +17,7 @@ import (
func TestPoolProfilesOrder(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Close via Cleanup, registered FIRST so it runs LAST: a plain defer would
// close the pool before the row-deleting cleanups below run.
@@ -83,7 +83,7 @@ func TestDeleteProfileContextHonorsCancellation(t *testing.T) {
func TestConfigStores(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -9,7 +9,7 @@ import (
func TestConversationPinOrderingAndPatch(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
+1 -1
View File
@@ -87,7 +87,7 @@ INSERT INTO exploration_anchors(node_id,asset_id) SELECT fn.id, an.id FROM fn JO
func TestBuildCoverageGraphLargeInvariants(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
f := seedLargeCoverageFixture(t, d)
+3 -3
View File
@@ -100,7 +100,7 @@ func TestCoverageJSONCarriesLedger(t *testing.T) {
func TestRecordCoverageUpsertHistory(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -157,7 +157,7 @@ func TestRecordCoverageUpsertHistory(t *testing.T) {
func TestCoverageCompletenessBudgetThenReviveDone(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -221,7 +221,7 @@ func TestCoverageCompletenessBudgetThenReviveDone(t *testing.T) {
func TestRecordCoverageRowCap(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
+1 -1
View File
@@ -10,7 +10,7 @@ import (
func TestCustomToolCRUD(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
+2 -2
View File
@@ -75,7 +75,7 @@ func testDSN(t *testing.T) string {
t.Helper()
dsn, _, err := DSN()
if err != nil {
t.Skipf("no database config (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
return dsn
}
@@ -85,7 +85,7 @@ func testDSN(t *testing.T) string {
func TestOpenSeed(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+239
View File
@@ -0,0 +1,239 @@
package db
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/hex"
"encoding/json"
"io/fs"
"slices"
"strings"
"testing"
"testing/fstest"
"time"
)
func TestTrafficSnapshotIDExcludesRewriteFlag(t *testing.T) {
base := TrafficEvidenceSnapshot{
SourceTrafficID: "tr-100",
CapturedAt: 1700000000,
URL: "https://target.example/login",
Method: "POST",
Status: 200,
ContentType: "application/json",
ReqHead: "POST /login HTTP/1.1\nHost: target.example\n",
RespHead: "HTTP 200\nContent-Type: application/json\n",
ReqHash: strings.Repeat("a", 64),
RespHash: strings.Repeat("b", 64),
ReqLen: 42,
RespLen: 128,
}
idPlain := TrafficSnapshotID(base)
withFlag := base
withFlag.Rewritten = true
withFlag.RewriteRuleIDs = []int64{1, 2}
if got := TrafficSnapshotID(withFlag); got != idPlain {
t.Fatalf("TrafficSnapshotID changed with Rewritten=true/ids=[1,2]: got %s want %s", got, idPlain)
}
withOtherRules := base
withOtherRules.Rewritten = true
withOtherRules.RewriteRuleIDs = []int64{99, 42, 7}
if got := TrafficSnapshotID(withOtherRules); got != idPlain {
t.Fatalf("TrafficSnapshotID changed with different rule ids: got %s want %s", got, idPlain)
}
// Simulate an older binary whose TrafficEvidenceSnapshot struct did not have
// Rewritten or RewriteRuleIDs: it must marshal to the exact same hash.
type legacyTrafficEvidenceSnapshot struct {
ID string `json:"id"`
SourceTrafficID string `json:"source_traffic_id"`
CapturedAt int64 `json:"captured_at"`
URL string `json:"url"`
Method string `json:"method"`
Status int `json:"status"`
ContentType string `json:"content_type"`
ReqHead string `json:"req_head,omitempty"`
RespHead string `json:"resp_head,omitempty"`
ReqHash string `json:"req_hash"`
RespHash string `json:"resp_hash"`
ReqLen int64 `json:"req_len"`
RespLen int64 `json:"resp_len"`
CreatedAt time.Time `json:"created_at"`
}
norm := base.Normalize()
legacy := legacyTrafficEvidenceSnapshot{
SourceTrafficID: norm.SourceTrafficID,
CapturedAt: norm.CapturedAt,
URL: norm.URL,
Method: norm.Method,
Status: norm.Status,
ContentType: norm.ContentType,
ReqHead: norm.ReqHead,
RespHead: norm.RespHead,
ReqHash: norm.ReqHash,
RespHash: norm.RespHash,
ReqLen: norm.ReqLen,
RespLen: norm.RespLen,
}
raw, err := json.Marshal(legacy)
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256(raw)
wantLegacy := hex.EncodeToString(sum[:])
if idPlain != wantLegacy {
t.Fatalf("TrafficSnapshotID = %s, old-version hash = %s", idPlain, wantLegacy)
}
}
func TestEvidenceRewriteFlagMigrationAndRoundTrip(t *testing.T) {
dsn := scratchDSN(t, "ev_rewrite")
// Build an FS containing migrations 0001..0007 (pre-0008 database).
pre0008 := fstest.MapFS{}
entries, err := fs.ReadDir(migrationsFS(), ".")
if err != nil {
t.Fatal(err)
}
for _, e := range entries {
if e.IsDir() || strings.HasPrefix(e.Name(), "0008_") {
continue
}
data, err := fs.ReadFile(migrationsFS(), e.Name())
if err != nil {
t.Fatal(err)
}
pre0008[e.Name()] = &fstest.MapFile{Data: data}
}
d0, err := openWithMigrations(dsn, pre0008)
if err != nil {
t.Fatalf("open pre-0008: %v", err)
}
task, err := d0.CreateTask("ev-rewrite-mig", "verify 0008", nil, 0, 0)
if err != nil {
d0.Close()
t.Fatal(err)
}
legacySnap := TrafficEvidenceSnapshot{
SourceTrafficID: "tr-pre-0008",
CapturedAt: 1700000001,
URL: "https://legacy.example/a",
Method: "GET",
Status: 200,
ContentType: "text/plain",
ReqHead: "GET /a HTTP/1.1\nHost: legacy.example\n",
RespHead: "HTTP 200\n",
ReqHash: strings.Repeat("1", 64),
RespHash: strings.Repeat("2", 64),
}
legacySnap.ID = TrafficSnapshotID(legacySnap)
var legacyFindingID int64
if err := d0.WithEvidenceTx(context.Background(), func(tx *sql.Tx) error {
rec, err := RecordFindingTx(context.Background(), tx, RecordFindingInput{
TaskID: task.ID, ExplorationID: task.ExplorationID, Worker: "test",
VulnClass: "TEST", Name: "pre-0008 finding", Severity: "low", Summary: "legacy row",
}, nil)
if err != nil {
return err
}
legacyFindingID = rec.FindingID
if _, err := tx.Exec(`INSERT INTO traffic_evidence_snapshots
(id,source_traffic_id,captured_at,url,method,status,content_type,req_head,resp_head,req_hash,resp_hash,req_len,resp_len)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13)`,
legacySnap.ID, legacySnap.SourceTrafficID, legacySnap.CapturedAt, legacySnap.URL, legacySnap.Method,
legacySnap.Status, legacySnap.ContentType, legacySnap.ReqHead, legacySnap.RespHead,
legacySnap.ReqHash, legacySnap.RespHash, legacySnap.ReqLen, legacySnap.RespLen); err != nil {
return err
}
_, err = tx.Exec(`INSERT INTO finding_traffic_bindings(finding_id,snapshot_id,role,note,position) VALUES($1,$2,'proof','pre-0008',0)`,
legacyFindingID, legacySnap.ID)
return err
}); err != nil {
d0.Close()
t.Fatalf("seed pre-0008 snapshot: %v", err)
}
d0.Close()
// Re-open with full embedded migrations: 0008 must apply cleanly.
d, err := Open(dsn)
if err != nil {
t.Fatalf("Open with 0008: %v", err)
}
defer d.Close()
assertEmbeddedApplied(t, d)
// Existing pre-0008 row reads rewritten=false and empty rewrite_rule_ids.
ftLegacy, err := d.GetFindingTraffic(context.Background(), legacyFindingID)
if err != nil {
t.Fatal(err)
}
if len(ftLegacy.Bindings) != 1 {
t.Fatalf("legacy bindings = %d, want 1", len(ftLegacy.Bindings))
}
gotLegacy := ftLegacy.Bindings[0].Snapshot
if gotLegacy.Rewritten || len(gotLegacy.RewriteRuleIDs) != 0 {
t.Fatalf("pre-0008 snapshot = rewritten:%v ids:%v, want false/empty", gotLegacy.Rewritten, gotLegacy.RewriteRuleIDs)
}
// New snapshot with Rewritten=true and unsorted/duplicate rule IDs round-trips cleanly
// through InsertEvidenceSnapshotTx and FindingTrafficTx while preserving sealed heads.
rewrittenSnap := TrafficEvidenceSnapshot{
SourceTrafficID: "tr-rewritten-1",
CapturedAt: 1700000002,
URL: "https://rewritten.example/item",
Method: "POST",
Status: 201,
ContentType: "application/json",
ReqHead: "POST /item HTTP/1.1\nAuthorization: Bearer topsecret\n",
RespHead: "HTTP 201\nSet-Cookie: sid=topsecret\n",
ReqHash: strings.Repeat("3", 64),
RespHash: strings.Repeat("4", 64),
ReqLen: 10,
RespLen: 20,
Rewritten: true,
RewriteRuleIDs: []int64{9, 2, 9, 5},
}
rewrittenSnap.ID = TrafficSnapshotID(rewrittenSnap)
var rewrittenFindingID int64
if err := d.WithEvidenceTx(context.Background(), func(tx *sql.Tx) error {
rec, err := RecordFindingTx(context.Background(), tx, RecordFindingInput{
TaskID: task.ID, ExplorationID: task.ExplorationID, Worker: "test",
VulnClass: "TEST", Name: "rewritten finding", Severity: "medium", Summary: "post-0008",
}, []PreparedTrafficEvidence{{
Ref: TrafficRef{TrafficID: rewrittenSnap.SourceTrafficID, Role: "proof", Note: "rewritten"},
Snapshot: rewrittenSnap,
}})
if err != nil {
return err
}
rewrittenFindingID = rec.FindingID
return nil
}); err != nil {
t.Fatalf("record rewritten snapshot: %v", err)
}
ft, err := d.GetFindingTraffic(context.Background(), rewrittenFindingID)
if err != nil {
t.Fatal(err)
}
if len(ft.Bindings) != 1 {
t.Fatalf("rewritten bindings = %d, want 1", len(ft.Bindings))
}
got := ft.Bindings[0].Snapshot
if !got.Rewritten || !slices.Equal(got.RewriteRuleIDs, []int64{2, 5, 9}) {
t.Fatalf("round-tripped snapshot = rewritten:%v ids:%v, want true [2 5 9]", got.Rewritten, got.RewriteRuleIDs)
}
if got.ID != rewrittenSnap.ID || TrafficSnapshotID(got) != got.ID {
t.Fatalf("snapshot ID mismatch: got=%s want=%s recomputed=%s", got.ID, rewrittenSnap.ID, TrafficSnapshotID(got))
}
if got.ReqHead != rewrittenSnap.ReqHead || got.RespHead != rewrittenSnap.RespHead {
t.Fatalf("heads mismatch after round-trip: req=%q resp=%q", got.ReqHead, got.RespHead)
}
}
+3 -3
View File
@@ -92,7 +92,7 @@ func assertClearHeads(t *testing.T, d *DB, findingID int64, want TrafficEvidence
func TestEvidenceHeadsSealedAtInsert(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
_, fid, s := evidenceFixture(t, d, "insert", false)
@@ -106,7 +106,7 @@ func TestEvidenceHeadsSealedAtInsert(t *testing.T) {
func TestEvidenceHeadsBackfillAndKeyChecks(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
_, fid, s := evidenceFixture(t, d, "legacy", true)
@@ -171,7 +171,7 @@ func TestEvidenceHeadsBackfillAndKeyChecks(t *testing.T) {
func TestEvidenceHeadsArchiveRoundTrip(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
if err = d.EnsureLLMRecordsTable(); err != nil {
+3 -3
View File
@@ -8,7 +8,7 @@ import (
func TestInheritedActivityReadsRequireTerminalIntent(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -92,7 +92,7 @@ func TestInheritedActivityReadsRequireTerminalIntent(t *testing.T) {
func TestExplorationDirectSourceReadView(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -315,7 +315,7 @@ func TestExplorationDirectSourceReadView(t *testing.T) {
func TestTaskAssetContextUsesDirectSourceScopeAndAnchors(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
+3 -3
View File
@@ -8,7 +8,7 @@ import (
func TestExplorationFlow(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -119,7 +119,7 @@ func TestExplorationFlow(t *testing.T) {
func TestIntentPauseResumeAndCancelCleanup(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -252,7 +252,7 @@ func TestIntentPauseResumeAndCancelCleanup(t *testing.T) {
func TestNodesPageQueryMatchesID(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -8,7 +8,7 @@ import (
func TestTokenStatsBySessionUsesCompleteIntentHistory(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+2 -2
View File
@@ -53,7 +53,7 @@ func nodeByKey(tree *FindingAssetTree, key string) *FindingAssetNode {
func TestBuildFindingAssetTree(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -164,7 +164,7 @@ func TestBuildFindingAssetTree(t *testing.T) {
func TestFindingAssetScopeFilter(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -10,7 +10,7 @@ import "testing"
func TestFindingQualityByNodeID(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
tk, err := d.CreateTask("finding quality lookup", "goal", nil, 0, 0)
+1 -1
View File
@@ -10,7 +10,7 @@ import (
func TestFindingReasoningFieldsRoundTrip(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
tk, err := d.CreateTask("finding reasoning fields", "goal", nil, 0, 0)
+30 -3
View File
@@ -8,6 +8,8 @@ import (
"encoding/json"
"errors"
"fmt"
"slices"
"strconv"
"strings"
"time"
@@ -85,6 +87,8 @@ type TrafficEvidenceSnapshot struct {
RespHash string `json:"resp_hash"`
ReqLen int64 `json:"req_len"`
RespLen int64 `json:"resp_len"`
Rewritten bool `json:"rewritten,omitempty"`
RewriteRuleIDs []int64 `json:"rewrite_rule_ids,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
@@ -131,11 +135,34 @@ func TrafficSnapshotID(snapshot TrafficEvidenceSnapshot) string {
snapshot = snapshot.Normalize()
snapshot.ID = ""
snapshot.CreatedAt = time.Time{}
snapshot.Rewritten = false
snapshot.RewriteRuleIDs = nil
raw, _ := json.Marshal(snapshot)
sum := sha256.Sum256(raw)
return hex.EncodeToString(sum[:])
}
func sortedUniqueInt64(ids []int64) []int64 {
if len(ids) == 0 {
return nil
}
out := slices.Clone(ids)
slices.Sort(out)
return slices.Compact(out)
}
func formatBigintArray(ids []int64) string {
ids = sortedUniqueInt64(ids)
if len(ids) == 0 {
return "{}"
}
parts := make([]string, len(ids))
for i, id := range ids {
parts[i] = strconv.FormatInt(id, 10)
}
return "{" + strings.Join(parts, ",") + "}"
}
// LockTaskEvidenceTx serializes writes with archive queueing (which locks the
// same task row). Once queued, its snapshot must not acquire new evidence.
func LockTaskEvidenceTx(tx *sql.Tx, taskID int64) error {
@@ -205,9 +232,9 @@ func InsertEvidenceSnapshotTx(tx *sql.Tx, s TrafficEvidenceSnapshot) error {
return err
}
_, err = tx.Exec(`INSERT INTO traffic_evidence_snapshots
(id,source_traffic_id,captured_at,url,method,status,content_type,req_head,resp_head,req_hash,resp_hash,req_len,resp_len,created_at)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14) ON CONFLICT(id) DO NOTHING`,
s.ID, s.SourceTrafficID, s.CapturedAt, s.URL, s.Method, s.Status, s.ContentType, reqHead, respHead, s.ReqHash, s.RespHash, s.ReqLen, s.RespLen, s.CreatedAt)
(id,source_traffic_id,captured_at,url,method,status,content_type,req_head,resp_head,req_hash,resp_hash,req_len,resp_len,created_at,rewritten,rewrite_rule_ids)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16::bigint[]) ON CONFLICT(id) DO NOTHING`,
s.ID, s.SourceTrafficID, s.CapturedAt, s.URL, s.Method, s.Status, s.ContentType, reqHead, respHead, s.ReqHash, s.RespHash, s.ReqLen, s.RespLen, s.CreatedAt, s.Rewritten, formatBigintArray(s.RewriteRuleIDs))
return err
}
+7 -7
View File
@@ -35,7 +35,7 @@ func TestValidFindingReportState(t *testing.T) {
func TestDeleteFinding(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -81,7 +81,7 @@ func TestDeleteFinding(t *testing.T) {
func TestSetFindingStatusMirrorsNodePayload(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -144,7 +144,7 @@ func TestSetFindingStatusMirrorsNodePayload(t *testing.T) {
func TestFindingReportState(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -256,7 +256,7 @@ func TestFindingReportState(t *testing.T) {
func TestFindingsPageAndStats(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -414,7 +414,7 @@ func TestFindingsPageAndStats(t *testing.T) {
func TestListFindingsPageUsesStableIDTieBreaker(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -462,7 +462,7 @@ func TestListFindingsPageUsesStableIDTieBreaker(t *testing.T) {
func TestFindingGroupsAndUnassignedPaging(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -606,7 +606,7 @@ func TestFindingGroupsAndUnassignedPaging(t *testing.T) {
func TestAddFindingFollowUpIntent(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+2 -2
View File
@@ -16,7 +16,7 @@ func openClaimInstances(t *testing.T) (*DB, *ExplorationStore, *ExplorationStore
dsn := testDSN(t)
a, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { a.Close() })
b, err := Open(dsn)
@@ -341,7 +341,7 @@ func TestIntentClaimTakeoverRacesHeartbeatAtomically(t *testing.T) {
func TestIntentClaimColumnsMigrateIdempotently(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+3 -3
View File
@@ -11,7 +11,7 @@ import (
func TestCompareAndSetIntentStateAllowsSingleWinner(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("intent CAS", "only one controller wins")
@@ -61,7 +61,7 @@ func TestCompareAndSetIntentStateAllowsSingleWinner(t *testing.T) {
func TestCancelIntentPreservesTokenMeteringWithoutDoubleCount(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("cancel token rollup", "preserve consumed tokens")
@@ -229,7 +229,7 @@ func assertDailyTokenBucketsEqual(t *testing.T, got, want map[string]DailyTokenB
func TestCancelIntentPreservesOutputsYieldedByAnotherIntent(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("shared intent output", "preserve shared facts and findings")
+3 -3
View File
@@ -41,7 +41,7 @@ func gone(t *testing.T, es *ExplorationStore, id int64) bool {
func TestSoftDeleteIntent(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("soft delete", "soft delete")
@@ -89,7 +89,7 @@ func TestSoftDeleteIntent(t *testing.T) {
func TestHardDeleteCascadesExclusiveDescendants(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("hard cascade", "cascade delete")
@@ -127,7 +127,7 @@ func TestHardDeleteCascadesExclusiveDescendants(t *testing.T) {
func TestHardDeletePreservesSharedAndGoal(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
expID, err := d.CreateExploration("hard preserve", "keep shared/goal")
+12 -12
View File
@@ -76,7 +76,7 @@ func TestRenderLessonsUsageOmitsTruncated(t *testing.T) {
func TestTaskLessonsUsesTaskScopeAndRanksCompanyFirst(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -174,7 +174,7 @@ func TestTaskLessonsUsesTaskScopeAndRanksCompanyFirst(t *testing.T) {
func TestExtractLessonsScopesSingleCompanyFromTaskScope(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -226,7 +226,7 @@ func TestExtractLessonsScopesSingleCompanyFromTaskScope(t *testing.T) {
func TestExtractLessonsSkipsMultiCompanyTask(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -273,7 +273,7 @@ func TestExtractLessonsSkipsMultiCompanyTask(t *testing.T) {
func TestExtractLessonsGenericWhenTaskHasNoCompany(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -311,7 +311,7 @@ func TestExtractLessonsGenericWhenTaskHasNoCompany(t *testing.T) {
func TestTaskLessonsRanksCategoryBeforeGeneric(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -377,7 +377,7 @@ func TestTaskLessonsRanksCategoryBeforeGeneric(t *testing.T) {
func TestLogLessonUsageRecordsOnlyRendered(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -457,7 +457,7 @@ func insertLessonFinding(t *testing.T, d *DB, taskID int64, status string) int64
func TestExtractLessonsRetractsReversedDisposition(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -569,7 +569,7 @@ func TestExtractLessonsRetractsReversedDisposition(t *testing.T) {
func TestExtractLessonsRetireOnlyWhenLastEvidence(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -660,7 +660,7 @@ func TestExtractLessonsRetireOnlyWhenLastEvidence(t *testing.T) {
func TestExtractLessonsRetractsAllRetests(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -731,7 +731,7 @@ func TestExtractLessonsRetractsAllRetests(t *testing.T) {
func TestExtractLessonsDoesNotUnretireWithEvidence(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -777,7 +777,7 @@ func TestExtractLessonsDoesNotUnretireWithEvidence(t *testing.T) {
func TestExtractLessonsLeavesAdopted(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -820,7 +820,7 @@ func TestExtractLessonsLeavesAdopted(t *testing.T) {
func TestExtractLessonsReopenRetractsNothing(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
+1 -1
View File
@@ -12,7 +12,7 @@ import (
func TestLLMRecordsMigrateAddsRawColumnsToOldTable(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
if err := d.EnsureLLMRecordsTable(); err != nil {
+1 -1
View File
@@ -12,7 +12,7 @@ import (
func TestLLMUsageTsIndex(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
+3 -3
View File
@@ -13,7 +13,7 @@ import (
func TestProfileRetryRoundTrip(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -71,7 +71,7 @@ func TestProfileRetryRoundTrip(t *testing.T) {
func TestProfileRetryClampedOnSave(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -98,7 +98,7 @@ func TestProfileRetryClampedOnSave(t *testing.T) {
func TestLLMRetryPolicyRoundTrip(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the restore below.
t.Cleanup(func() { d.Close() })
+6 -5
View File
@@ -52,11 +52,11 @@ func scratchDSN(t *testing.T, suffix string) string {
admin.Path = "/postgres"
conn, err := sql.Open("pgx", admin.String())
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
if err := conn.Ping(); err != nil {
conn.Close()
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
drop := func() error {
_, err := conn.Exec(`DROP DATABASE IF EXISTS "` + name + `" WITH (FORCE)`)
@@ -137,9 +137,10 @@ func assertEmbeddedApplied(t *testing.T, d *DB) []appliedMigration {
version int
name string
}{{1, llmRecordsMigration}, {2, llmUsageMigration}, {3, "0003_task_scope_icp_index.sql"}, {4, "0004_llm_usage_ts_index.sql"},
{5, "0005_traffic_rewrite_rules.sql"}, {6, "0006_activity_outbox.sql"}, {7, "0007_finding_report_state.sql"}}
{5, "0005_traffic_rewrite_rules.sql"}, {6, "0006_activity_outbox.sql"}, {7, "0007_finding_report_state.sql"},
{8, "0008_traffic_evidence_rewrite_flag.sql"}}
if len(applied) != len(want) {
t.Fatalf("schema_migrations = %+v, want versions 1 to 7", applied)
t.Fatalf("schema_migrations = %+v, want versions 1 to 8", applied)
}
for i, w := range want {
content, err := fs.ReadFile(migrationsFS(), w.name)
@@ -189,7 +190,7 @@ func TestOpenRecordsEmbeddedMigrations(t *testing.T) {
dsn := testDSN(t)
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
applied := assertEmbeddedApplied(t, d)
d.Close()
@@ -0,0 +1,5 @@
-- Provenance flag on evidence snapshots recording whether the capture proxy
-- applied at least one rewrite rule to the bound exchange, and which rule IDs
-- were applied. Excluded from the content-addressed snapshot ID.
ALTER TABLE traffic_evidence_snapshots ADD COLUMN IF NOT EXISTS rewritten boolean NOT NULL DEFAULT false;
ALTER TABLE traffic_evidence_snapshots ADD COLUMN IF NOT EXISTS rewrite_rule_ids bigint[] NOT NULL DEFAULT '{}'::bigint[];
+1 -1
View File
@@ -18,7 +18,7 @@ func notifyTestDB(t *testing.T) *DB {
t.Helper()
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
return d
+3 -3
View File
@@ -200,7 +200,7 @@ func assertPlainReads(t *testing.T, d *DB, r secretRows) {
func TestSecretsAtRestWriteRead(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() }) // after the fixture cleanup
r := secretFixture(t, d, false)
@@ -214,7 +214,7 @@ func TestSecretsAtRestWriteRead(t *testing.T) {
func TestSecretsBackfillLegacyAndKeyChecks(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() }) // after the fixture cleanup
r := secretFixture(t, d, true)
@@ -287,7 +287,7 @@ func encryptedCount(t *testing.T, d *DB, dir string) int {
func TestSecretSettingsEmptyAndClear(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() {
d.Exec(deleteSecretSettings)
+2 -2
View File
@@ -13,7 +13,7 @@ import (
func TestInsertSettingIfAbsentDoesNotOverwrite(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -50,7 +50,7 @@ func TestInsertSettingIfAbsentDoesNotOverwrite(t *testing.T) {
func TestRotatePasswordRollsBackHashWhenVersionCannotIncrement(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -11,7 +11,7 @@ import (
func TestSkillUsageLedger(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -19,7 +19,7 @@ func openTargetDB(t *testing.T) *DB {
t.Helper()
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
return d
+3 -3
View File
@@ -123,7 +123,7 @@ func TestTaskArchiveFormatCompatibility(t *testing.T) {
func TestTaskArchiveDatabaseRoundTrip(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
if err := d.EnsureLLMRecordsTable(); err != nil {
@@ -335,7 +335,7 @@ VALUES($1,$2,0,'quota_exhausted','balance exhausted',$3,$4,$3)`, task.ID, llmPro
func TestTaskArchiveBlockersIgnoreQueuedDependents(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
source, err := d.CreateTaskWithOptions("archive blocker source", "source", TaskCreateOptions{})
@@ -400,7 +400,7 @@ func assertPerTaskBlockers(t *testing.T, d *DB, global map[int64]int64, want map
func TestRecoverInterruptedArchiveRequiresManualRetry(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
task, err := d.CreateTaskWithOptions("interrupted archive", "must not restart automatically", TaskCreateOptions{})
+3 -3
View File
@@ -10,7 +10,7 @@ import (
func TestRegisterTaskAssetScopesCreatesAssetsAndPersistsTextScope(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -93,7 +93,7 @@ func TestRegisterTaskAssetScopesCreatesAssetsAndPersistsTextScope(t *testing.T)
func TestTaskAssetAttachDetachPreservesGlobalAssetAndAnchors(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -150,7 +150,7 @@ func TestTaskAssetAttachDetachPreservesGlobalAssetAndAnchors(t *testing.T) {
func TestIntentAssetsIncludesDirectSourceProvenance(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
+4 -4
View File
@@ -11,7 +11,7 @@ import (
func TestTaskCategoryCRUDAndTaskAssignment(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -73,7 +73,7 @@ func TestTaskCategoryCRUDAndTaskAssignment(t *testing.T) {
func TestCreateTaskRejectsMissingCategoryAtomically(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -94,7 +94,7 @@ func TestCreateTaskRejectsMissingCategoryAtomically(t *testing.T) {
func TestSetTasksCategoryBatch(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -155,7 +155,7 @@ func TestSetTasksCategoryBatch(t *testing.T) {
func TestSetTasksCategoryRejectsBadInputAtomically(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
+7 -7
View File
@@ -11,7 +11,7 @@ func TestTaskLLMProfileMutationsLockTaskBeforeProfile(t *testing.T) {
dsn := testDSN(t)
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -111,7 +111,7 @@ func TestDeleteProfileLocksNonTaskReferencesBeforeProfile(t *testing.T) {
t.Run("agent", func(t *testing.T) {
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -169,7 +169,7 @@ func TestDeleteProfileLocksNonTaskReferencesBeforeProfile(t *testing.T) {
t.Run("conversation", func(t *testing.T) {
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -226,7 +226,7 @@ func TestNonTaskProfileMutationsLockReferenceBeforeProfile(t *testing.T) {
t.Run("agent", func(t *testing.T) {
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -275,7 +275,7 @@ func TestNonTaskProfileMutationsLockReferenceBeforeProfile(t *testing.T) {
t.Run("conversation", func(t *testing.T) {
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -326,7 +326,7 @@ func TestCreateConversationAndDeleteProfileDoNotDeadlock(t *testing.T) {
dsn := testDSN(t)
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -407,7 +407,7 @@ func TestDeleteProfileRetriesReferenceCommittedAfterInitialScan(t *testing.T) {
dsn := testDSN(t)
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
+1 -1
View File
@@ -13,7 +13,7 @@ func TestDeleteTaskTrafficHostsUseOneLockedTransaction(t *testing.T) {
dsn := testDSN(t)
d, err := Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+3 -3
View File
@@ -9,7 +9,7 @@ import (
func TestListTasksBulkHydratesTaskContext(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -71,7 +71,7 @@ func TestListTasksBulkHydratesTaskContext(t *testing.T) {
func TestTaskListMetricsAll(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -123,7 +123,7 @@ VALUES ($1,'goal','{}','met'), ($1,'goal','{}','open')`, task.ExplorationID); er
func TestTaskListMetricsMatchesAll(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
+1 -1
View File
@@ -11,7 +11,7 @@ import (
func TestTaskLLMSpendCountsJudgeNotChat(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+2 -2
View File
@@ -9,7 +9,7 @@ import (
func TestTaskPinOrderingAndRename(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
@@ -71,7 +71,7 @@ func TestTaskPinOrderingAndRename(t *testing.T) {
func TestDeleteConversationsReturnsExistingIDs(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -8,7 +8,7 @@ import (
func TestTaskQueuePreservesBootstrapAndFIFOPosition(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+1 -1
View File
@@ -9,7 +9,7 @@ import (
func TestTaskRotationPersistenceAndFailover(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable: %v", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
ids := []int64{}
+2 -2
View File
@@ -12,7 +12,7 @@ func TestCoverageEnabledWithoutTask(t *testing.T) {
func TestCoverageEnabledMissingRowStaysOn(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
on, err := d.Assets().CoverageEnabled(9_000_000_000_001)
@@ -24,7 +24,7 @@ func TestCoverageEnabledMissingRowStaysOn(t *testing.T) {
func TestCoverageEnabledQueryErrorStaysOn(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
SkipPGUnavailable(t, err)
}
if err := d.Close(); err != nil {
t.Fatal(err)
+2 -2
View File
@@ -12,7 +12,7 @@ import (
func TestTaskTemplateCRUDAndNormalizedUniqueness(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
@@ -87,7 +87,7 @@ func TestTaskTemplateCRUDAndNormalizedUniqueness(t *testing.T) {
func TestTaskTemplateDisjointPatchesCompose(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanup below.
t.Cleanup(func() { d.Close() })
+6 -6
View File
@@ -11,7 +11,7 @@ import (
func TestTaskLifecycleAndDeleteCascade(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() { d.Close() })
@@ -85,7 +85,7 @@ func TestTaskLifecycleAndDeleteCascade(t *testing.T) {
func TestTaskDeleteCascadeAssets(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -211,7 +211,7 @@ func TestTaskDeleteCascadeAssets(t *testing.T) {
func TestTaskRelationsAndLLMFailoverChain(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -470,7 +470,7 @@ func TestTaskRelationsAndLLMFailoverChain(t *testing.T) {
func TestTaskContextRejectsDuplicatesAndAllowsTerminalLLMEdits(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -525,7 +525,7 @@ func TestTaskContextRejectsDuplicatesAndAllowsTerminalLLMEdits(t *testing.T) {
func TestCreateTaskWithCompanyScopes(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
@@ -680,7 +680,7 @@ RETURNING id`, domainB, companyB).Scan(&companyAssetB); err != nil {
func TestListTasksOrderByIDDesc(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
// Registered FIRST so it runs LAST, after the row cleanups below.
t.Cleanup(func() { d.Close() })
+33
View File
@@ -0,0 +1,33 @@
package db
// skipper is the part of testing.TB used by the PostgreSQL test helpers below.
// A local interface keeps the production package from importing "testing":
// *testing.T, *testing.B and the test-only wrappers all satisfy it.
type skipper interface {
Helper()
Skip(args ...any)
Skipf(format string, args ...any)
}
// SkipPGUnavailable skips the calling test with one uniform reason when err
// reports that the test database could not be configured or opened.
//
// Every PostgreSQL-dependent test uses this instead of spelling the skip text
// inline (the old variants differed by package: "no database config",
// "database unavailable", "postgres unavailable — skipping", ...). Call it
// with the err returned by DSN(), Open() or NewManager(). It does not change
// when a test skips — only how the reason is reported.
func SkipPGUnavailable(t skipper, err error) {
t.Helper()
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
}
}
// SkipNoPG skips with the same uniform reason when a store is unexpectedly nil
// after a successful open (for example Manager.pg when only some backends are
// configured). It carries no error because there is none to report.
func SkipNoPG(t skipper) {
t.Helper()
t.Skip("postgres unavailable")
}
+1 -1
View File
@@ -5,7 +5,7 @@ import "testing"
func TestToolUsageLedger(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
SkipPGUnavailable(t, err)
}
defer d.Close()
+2 -2
View File
@@ -77,7 +77,7 @@ func jsonEqual(t *testing.T, a, b string) bool {
func TestToolExecSealedAtRest(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() {
d.Exec(`DELETE FROM tools WHERE key LIKE 't_secexec%'`)
@@ -129,7 +129,7 @@ func TestToolExecSealedAtRest(t *testing.T) {
func TestToolExecBackfillAndKeyChecks(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
t.Cleanup(func() {
d.Exec(`DELETE FROM tools WHERE key LIKE 't_secexec%'`)
+1 -1
View File
@@ -26,7 +26,7 @@ func openRewriteRules(t *testing.T) *DB {
t.Helper()
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
SkipPGUnavailable(t, err)
}
clean := func() { d.Exec(`DELETE FROM traffic_rewrite_rules WHERE name LIKE 't-rw-%'`) }
clean()
+112
View File
@@ -0,0 +1,112 @@
# ARTEX architecture (code-level companion)
The README's [Architecture](https://github.com/BaaRai/ARTEX#architecture) section has the
component diagram and the 10,000-foot view. This document records the code-level map,
the execution flow and the trust boundaries that are easy to miss when reading the
packages in isolation.
## 1. The two graphs
ARTEX keeps two linked graphs, both in PostgreSQL (`db/schema.sql`):
| Graph | Answers | Main tables | Code |
| --- | --- | --- | --- |
| **Asset graph** | *What exists?* | `companies`, `company_scope`, `assets`, `task_scope` | `db/assets.go`, `db/companies.go`, `db/company_scope.go`, `db/task_assets.go` |
| **Exploration graph** | *What was tried, with what result and why?* | `explorations`, `exploration_nodes`, `activity`, `intent_*`, `coverage_*` | `db/exploration.go`, `db/coverage_ledger.go`, `db/finding_assets.go` |
Anchors tie a finding to the assets that justify it (`db/finding_assets.go`). Traffic
snapshots are archived separately and referenced from findings
(`db/finding_traffic.go`, `db/finding_traffic_archive.go`, `traffic/archive.go`).
`db.Open` executes the idempotent `schema.sql` on every startup, then applies pending
numbered migrations from `db/migrations/*.sql`, recording name + SHA-256 in
`schema_migrations`. A published migration is never edited; fix forward with a new one.
## 2. Package map
| Package | Responsibility | Notes |
| --- | --- | --- |
| `cmd/artex` | Process entry point | Flags (`-addr`, `-data`, `-proxy`), startup config snapshot, signal handling, graceful drain |
| `config` | Resolve config file + env | `Path`, `Load`, `PostgresDSN`, `SkillDir`, `KeyDir`, `Redact`. Env wins over file |
| `db` | All persistence | `DB`, `Manager`-independent stores; AES-256-GCM secret columns in `secrets.go` |
| `agent` | LLM roles and tools | `goals`, `planner`, `worker`, `mainagent`, `retester`; `ToolSet` in `tools.go`; tool catalog in `toolcatalog.go` |
| `server` | HTTP API + orchestration | `net/http` mux (~270 routes), `Manager`, `Engine`, JWT auth, SSE broadcaster |
| `traffic` | Recording proxy | MITM proxy built on `go-mitmproxy`, replay, rewrite rules, body archive |
| `evidence` | Evidence store | Persisted request/response snapshots used by findings |
| `intercept` | Approval gate | Judge, argv grammar, target gate, pins, risk scoring (a `PreToolUse` hook) |
| `guard` | Hook wiring | Binds `intercept` decisions into Norma's hook registry; `NewTargetOnly` for planner/mainagent |
| `egress` | Noise governor | Per-host token bucket, jitter, concurrency caps, 429/503 backoff; default profile is strict passthrough |
| `llmpool` / `llmrec` | Provider pool / recorder | Rotation, health, usage + raw wire capture |
| `mcphttp` | MCP client | HTTP/SSE transport for configured MCP servers |
| `notify` | Outbound notifications | Email, webhook, Telegram, Feishu, DingTalk with masking and SSRF guards |
| `report` | Deliverables | Markdown, CSV and JSON rendering of findings |
| `sidequestion` | Operator Q&A | Asks a human mid-run and suspends intent until answered |
| `skills/` | Skill definitions | One `SKILL.md` per subdirectory, loaded from `SkillDir()` |
The frontend lives in `web/` (Next.js 16, React 19) and is embedded into the Go
binary via `server/webui_embed.go` when built with `-tags embedui`.
## 3. End-to-end flow: goal → finding with evidence
1. **Task creation.** `server.createTask` → `Manager.CreateTaskWithOptions` writes the
task, its scope and seed goals, and creates the exploration graph (`db/tasks.go`).
2. **Loop start.** `Engine.Run` (`server/engine.go`) starts a planner loop and N worker
goroutines under a task-scoped context. Nothing needs an LLM to be configured for the
loops to exist; they idle until one is.
3. **Decomposition.** The `goals` role writes goals and operation constraints through
`set_goals` / `set_constraints` (`agent/toolcatalog.go`, `agent/tools.go`).
4. **Planning.** `Planner.Plan` (`agent/planner.go`) reads the route, queries the asset
graph, judges goals and emits intents (`addOneIntent`). Every intent is an
exploration node, so it is auditable like any other activity.
5. **Execution.** `Worker.Execute` (`agent/worker.go`) claims an intent, runs tools with
the guard hooks attached and writes observed facts back to the exploration graph.
6. **Tools and traffic.** Shell and HTTP/WebFetch calls go through the recording proxy
and the egress governor. Platform-owned HTTP clients use `intercept.PinnedDialer`
and `intercept.RedirectCheck`, so a redirect or a direct retry is re-checked at
dial time. When a proxy is in the path the gate records the pin as name-only: the
proxy makes the outbound connection, so no IP binding can be claimed.
7. **Findings.** `addFinding` (`agent/tools.go`) records severity, class, confidence and
asset references; `db/findings.go` dedups and keeps the status workflow. Evidence is
linked to captured traffic (`traffic/evidence.go`, `db/finding_traffic.go`).
8. **Completion.** `prove_goal` marks a goal met; the last met goal completes the task.
`goal_met` is deliberately unbound by default because it bypasses per-goal proof.
## 4. Execution control
- **Per-task context:** pause/cancel/kill attach a named cause via
`context.WithCancelCause` so the activity trace records who initiated a stop.
- **Per-intent control:** the UI can pause/cancel one intent; `steerBox` queues a
planner course-correction that the worker's `PreToolUse` hook drains before the next
tool call, re-planning without killing the run.
- **Delete barrier:** `Engine.BeginDelete` makes installing the barrier atomic with
registering an in-flight operation; after it returns, no new task-owned writes are
admitted and the task context is cancelled once existing writers drain.
- **Quiet mode:** an unchanged open-goal fingerprint suppresses redundant planning
rounds while work is still in flight.
## 5. Trust boundaries
| Boundary | Implementation |
| --- | --- |
| Tool output → model | Norma's untrusted-output boundary: nonce-tagged markers around tool results. On by default; `ARTEX_UNTRUSTED_BOUNDARY=0` disables it. See `agent/untrusted_boundary.go` |
| Command → host | Allow rules match one Bash simple command with the exact prefix: a pipeline, redirection, wrapper (`env`, `sudo`, `xargs`, `find -exec`, a shell `-c`), substitution or expansion does not inherit an allow. Deny and ask rules also inspect every simple command inside wrappers, lists and pipelines. `ArgvPrefixOf` derives the human-readable prefix |
| URL → network | Target gate + `PinnedDialer` + `RedirectCheck`: scope decisions are re-made against a fresh DNS lookup at dial time, and without a proxy the client connects only to the addresses it compared; with a proxy the pin is name-only |
| Secrets → disk/log | `db/secrets.go` seals API keys, proxy credentials, MCP env, task credentials and captured auth headers with AES-256-GCM, nonce per value, AAD `table.column`. JWT keys live in `KeyDir()`, never in the browsable data dir. The boot snapshot logs only `config.Redact(dsn)` |
| Approval → audit | Every decision records the approver (`jwtSubject`), source and decision; history is queryable through `/api/intercept/history` and `/api/intercept/history/{id}` (with `/execution` for the audit detail). `/api/tasks/{id}/intercept-rules` manages rules, not history |
## 6. Decisions
- **ADR-001 — Nonce-tagged untrusted output.** Tool output is framed with a per-run
nonce rather than trusting the model to distinguish instructions from data.
- **ADR-002 — Pin at dial time.** A URL decision is not enough: DNS can change between
the hook and the connection, so the dialer re-resolves and connects only to compared
addresses.
- **ADR-003 — AES-GCM with per-column AAD.** Sealed values are bound to their table and
column (and settings to their key), so a ciphertext cannot be moved to another field.
## 7. Where to look next
- `docs/OPS.md` for startup, configuration precedence, backup and key rotation.
- `docs/vulnerability-traffic-evidence.md` for how a finding keeps its receipts.
- `docs/traffic-rewrite-rules.md` for the rewrite engine.
- `scripts/verify.sh` to reproduce the CI checks locally.
+93
View File
@@ -0,0 +1,93 @@
# ARTEX operations
## 1. Startup
Docker (recommended):
```bash
cp .env.example .env # set POSTGRES_PASSWORD; optionally an LLM key
docker compose up -d --build
docker compose logs artex | grep "Setup token"
```
Open `http://localhost:8787`. The first visit to `/setup` asks for the one-time setup
token printed at startup (`[auth] no admin password is set. Setup token for the /setup
page: ...` in `server/auth.go`).
From source (no embedded UI):
```bash
go run ./cmd/artex # backend only, 127.0.0.1:8787
cd web && npm run dev # UI on :5173, /api proxied to the backend
# or: ./dev.sh # both, plus the traffic proxy on :8788
```
The process prints one `[config] snapshot {...}` line at startup. It is the authoritative
record of the effective configuration: config path, DSN source and **redacted** DSN,
data/skill/key directories, listen address, proxy, noise profile, untrusted-output
boundary and trusted proxies. No password or token value is ever logged there.
## 2. Configuration precedence
| Setting | Resolution order | Code |
| --- | --- | --- |
| Config file | `ARTEX_CONFIG` → `./config.json` → `config.json` next to the binary | `config.Path` |
| PostgreSQL DSN | `ARTEX_PG_DSN` → `database.dsn` → assembled `host/port/user/password/dbname` | `config.PostgresDSN` |
| Skill directory | `ARTEX_SKILL_DIR` → `skill_dir` → `<binary dir>/skills` | `config.SkillDir` |
| JWT / secret key directory | `ARTEX_JWT_KEY_DIR` → `<binary dir>` (never the browsable data dir) | `config.KeyDir` |
| Secret encryption key | `ARTEX_SECRET_KEY` → `<key dir>/secret.key` → newly generated key file | `db.InitSecrets` |
| Noise profile | `ARTEX_NOISE_PROFILE` (`off` default; `quiet`, `balanced`, `aggressive`), also settable in UI | `egress.ProfileFromEnv` |
| Untrusted-output boundary | on by default; `ARTEX_UNTRUSTED_BOUNDARY=0\|false\|off\|no` disables it | `agent/untrusted_boundary.go` |
| Trusted proxies for login rate limit | `ARTEX_TRUSTED_PROXIES` (comma-separated IPs/CIDRs; empty trusts none) | `server.ParseTrustedProxies` |
| Target user agent | `ARTEX_TARGET_USER_AGENT` (applied by non-`off` noise profiles) | `egress.ProfileByName` |
| Budgets | `ARTEX_TASK_BUDGET_TOKENS` / `_TURNS`, `ARTEX_INTENT_BUDGET_TOKENS` / `_TURNS` | `agent/budget.go` |
| Live model judge test | `ARTEX_REVIEW_LIVE_CONFIG` (opt-in integration test only) | `server/intercept_live_test.go` |
Agent profiles, providers, notifications and other runtime settings live in the database
and are edited in **System → LLM / Settings / Notify**, not in `config.json`.
## 3. Backup and restore
```bash
scripts/backup.sh # database dump only, 14 rotations
scripts/backup.sh --with-data --keep 30 # also archive the data/ bind mount
```
The dump runs inside the `postgres` compose service, so no host `psql` is needed. Restore:
```bash
gunzip -c backups/artex-db-<ts>.sql.gz | docker compose exec -T postgres \
psql -U "$POSTGRES_USER" -d "$POSTGRES_DB"
```
Back up the key directory as well: without it, JWTs cannot be verified and sealed
secrets cannot be decrypted. Keep it outside the database backup but under the same
access controls.
## 4. Key rotation
- **`jwt.key`** — signing key for the 7-day HS256 session tokens. Replacing it
invalidates every existing session; all users must log in again. Existing key files
are never overwritten by startup (`loadOrCreateJWTKey`).
- **`secret.key`** (or `ARTEX_SECRET_KEY`) — AES-256-GCM key for API keys, MCP env,
task credentials and captured auth headers. Startup never overwrites an existing key.
If it changes, values sealed under the old key cannot be decrypted (the code returns
an error rather than leaking ciphertext as plaintext). Re-enter those secrets in the
UI, or keep the previous key available until the migration is complete.
Plan a short maintenance window for either rotation.
## 5. Diagnostics and local verification
- `GET /api/health` — liveness, build version and provider health.
- `GET /api/logs`, `GET /api/logs/stream` — live backend log sink; the UI shows it under
**System → Logs**. The startup snapshot line is here too.
- `scripts/verify.sh` — reproduces the CI checks locally (gofmt, build, vet, tests,
web type-check and unit tests, shellcheck, python checks). `--full` adds `-race` and
the web static build; `--go-only` / `--web-only` narrow the run.
- `./scripts/backup.sh --help` — backup options.
When Postgres is not configured, database-dependent tests skip themselves; that is
expected. Set `ARTEX_PG_DSN` to run them. CI uses `go test -p 1 ./...` because test
packages share a Postgres service and use suffixed per-package databases
(`SuffixTestDatabase`).
+18 -4
View File
@@ -4,8 +4,9 @@ A rewrite rule matches a request or a response that goes through the capture pro
how to change it: the method, the path and query, headers, the body, the status code.
**Status: the capture proxy applies enabled rules to HTTP traffic (request and response phases),
and records what it did.** Not yet in this change: archive format, a per-replay rule toggle, the web UI,
and the proof flag. See [Application in the capture proxy](#application-in-the-capture-proxy) for
records what it did, tags bound evidence snapshots, supports per-replay rule opt-in, and exposes
rewrite counters on `GET /api/traffic`.** Not yet in this change: archive format and the traffic-list /
rule-management web UI. See [Application in the capture proxy](#application-in-the-capture-proxy) for
what is covered and what is not.
Rules are a convenience for the operator, not a security control: a rule that fails leaves the
@@ -207,5 +208,18 @@ the slot may be converted into a tombstone if a new reservation needs the room.
### Replay
Replay goes through the recording proxy like any other traffic, so the enabled rules apply to a
replay and the resulting exchange is traced like a captured one. There is no per-replay toggle yet.
`POST /api/traffic/replay` sends the edited request through the recording proxy so the exchange is
captured in Traffic, but **bypasses capture rewrite rules by default** (`apply_rules: false`). To
run enabled capture rules against a replay and trace them like captured traffic, opt in with
`"apply_rules": true` (or check **Apply capture rewrite rules** in the Replay panel).
When `apply_rules` is false, `traffic.Replay` registers a single-use random 32-byte hex nonce in an
in-memory registry, attaches the internal header `X-ARTEX-Replay-Nonce: <nonce>`, and removes the
nonce when the call returns. The capture proxy strips `X-ARTEX-Replay-Nonce` unconditionally in the
request hook so the target never sees it; only a value matching a registered nonce bypasses both
request and response rule phases, while any forged header is stripped and ignored.
### Stats
`GET /api/traffic` includes a `rewrite` object exposing the proxy's `RewriteStats` counters since
start: `large_responses`, `capacity_refusals`, `tombstone_drops`, `conversions`, and `sweeps`.
+21 -1
View File
@@ -13,6 +13,7 @@ import (
"log"
"os"
"path/filepath"
"slices"
"strings"
"time"
@@ -176,7 +177,26 @@ func (s *Store) prepare(ctx context.Context, refs []db.TrafficRef) ([]db.Prepare
out = append(out, db.PreparedTrafficEvidence{Ref: byID[e.ID], Snapshot: v})
return nil
})
return out, err
if err != nil {
return nil, err
}
// Enrich after ReadEvidence returns so RewriteInfo is never called while
// ReadEvidence holds the traffic writer lock.
for i := range out {
rewritten, ruleIDs, err := s.Traffic.RewriteInfo(out[i].Snapshot.SourceTrafficID)
if err != nil {
return nil, fmt.Errorf("read rewrite provenance for %s: %w", out[i].Snapshot.SourceTrafficID, err)
}
if rewritten {
out[i].Snapshot.Rewritten = true
if len(ruleIDs) > 0 {
ids := slices.Clone(ruleIDs)
slices.Sort(ids)
out[i].Snapshot.RewriteRuleIDs = slices.Compact(ids)
}
}
}
return out, nil
}
func (s *Store) Record(ctx context.Context, in db.RecordFindingInput, refs []db.TrafficRef) (out *db.RecordedFinding, err error) {
+166
View File
@@ -8,14 +8,21 @@ import (
"errors"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
"time"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/traffic"
"github.com/Autumn-27/artex/traffic/rewrite"
)
func evidenceFixture(t *testing.T) (*Store, db.RecordFindingInput, string) {
@@ -331,3 +338,162 @@ func TestEvidenceGCGraceAndActiveRestore(t *testing.T) {
t.Fatal("accepted tampered snapshot")
}
}
func startRewriteProxy(t *testing.T, dir string) (*traffic.Traffic, string) {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := ln.Addr().String()
ln.Close()
tr, err := traffic.Open(dir, addr)
if err != nil {
t.Fatal(err)
}
go func() { _ = tr.Start() }()
t.Cleanup(func() { _ = tr.Close() })
deadline := time.Now().Add(5 * time.Second)
for {
c, err := net.Dial("tcp", addr)
if err == nil {
c.Close()
break
}
if time.Now().After(deadline) {
t.Fatalf("proxy did not listen on %s: %v", addr, err)
}
time.Sleep(10 * time.Millisecond)
}
return tr, "http://" + addr
}
func TestEvidenceBindingRewrittenExchange(t *testing.T) {
dsn := os.Getenv("ARTEX_PG_DSN")
if dsn == "" {
t.Skip("ARTEX_PG_DSN is required for evidence integration tests")
}
pg, err := db.Open(dsn)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { pg.Close() })
task, err := pg.CreateTask("evidence rewrite "+t.Name(), "rewrite provenance", nil, 0, 0)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { pg.DeleteFindingsByTask(task.ID); pg.DeleteTask(task.ID) })
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
_, _ = io.WriteString(w, "upstream response")
}))
t.Cleanup(target.Close)
dir := t.TempDir()
tr, proxyURL := startRewriteProxy(t, filepath.Join(dir, "traffic"))
// Rules 5 (request) and 2 (response) apply to /rewritten; rule 9 only matches
// /never-matches so it is skipped. Neither applies to /plain.
set := rewrite.Compile([]rewrite.Rule{
{
ID: 5, Name: "req-header", Enabled: true, Phase: rewrite.PhaseRequest,
Matchers: rewrite.Matchers{URLRegex: "^/rewritten$"},
Actions: []rewrite.Action{{Type: rewrite.ActionSetHeader, Name: "X-Injected", Value: "yes"}},
},
{
ID: 2, Name: "resp-header", Enabled: true, Phase: rewrite.PhaseResponse,
Matchers: rewrite.Matchers{URLRegex: "^/rewritten$"},
Actions: []rewrite.Action{{Type: rewrite.ActionSetHeader, Name: "X-Resp-Tag", Value: "1"}},
},
{
ID: 9, Name: "skipped-rule", Enabled: true, Phase: rewrite.PhaseRequest,
Matchers: rewrite.Matchers{URLRegex: "^/never-matches$"},
Actions: []rewrite.Action{{Type: rewrite.ActionSetHeader, Name: "X-Skipped", Value: "1"}},
},
}, rewrite.Options{})
if len(set.LoadErrors()) != 0 {
t.Fatalf("rewrite rule load errors: %v", set.LoadErrors())
}
tr.SetRewriteRules(set)
pu, err := url.Parse(proxyURL)
if err != nil {
t.Fatal(err)
}
client := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(pu)}}
for _, path := range []string{"/rewritten", "/plain"} {
req, err := http.NewRequest(http.MethodGet, target.URL+path, nil)
if err != nil {
t.Fatal(err)
}
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
_, _ = io.ReadAll(resp.Body)
resp.Body.Close()
}
exchangeID := func(suffix string) string {
var id string
deadline := time.Now().Add(3 * time.Second)
for {
err := tr.DB().QueryRow(`SELECT id FROM exchanges WHERE url LIKE ?`, "%"+suffix).Scan(&id)
if err == nil {
return id
}
if time.Now().After(deadline) {
t.Fatalf("exchange %s not recorded: %v", suffix, err)
}
time.Sleep(20 * time.Millisecond)
}
}
rewrittenID := exchangeID("/rewritten")
plainID := exchangeID("/plain")
store := New(pg, tr, filepath.Join(dir, "evidence"))
in := db.RecordFindingInput{
TaskID: task.ID, ExplorationID: task.ExplorationID, Worker: "test",
VulnClass: "TEST", Name: "Rewrite provenance", Severity: "low", Summary: "proxy rewrite test",
}
ctx := context.Background()
rec, err := store.Record(ctx, in, []db.TrafficRef{
{TrafficID: rewrittenID, Role: "proof", Note: "rewritten"},
{TrafficID: plainID, Role: "baseline", Note: "plain"},
})
if err != nil {
t.Fatalf("store.Record: %v", err)
}
if len(rec.Traffic.Bindings) != 2 {
t.Fatalf("bindings = %d, want 2", len(rec.Traffic.Bindings))
}
rewSnap := rec.Traffic.Bindings[0].Snapshot
if !rewSnap.Rewritten {
t.Fatalf("rewritten snapshot has Rewritten=false, want true")
}
if !slices.Equal(rewSnap.RewriteRuleIDs, []int64{2, 5}) {
t.Fatalf("rewritten snapshot rule IDs = %v, want [2 5]", rewSnap.RewriteRuleIDs)
}
// Content-addressed ID must exclude the rewrite flag and rule IDs.
stripped := rewSnap
stripped.Rewritten = false
stripped.RewriteRuleIDs = nil
if gotID := db.TrafficSnapshotID(stripped); gotID != rewSnap.ID {
t.Fatalf("snapshot ID depended on rewrite flag: stripped=%s stored=%s", gotID, rewSnap.ID)
}
plainSnap := rec.Traffic.Bindings[1].Snapshot
if plainSnap.Rewritten || len(plainSnap.RewriteRuleIDs) != 0 {
t.Fatalf("plain snapshot = rewritten:%v ids:%v, want false/empty", plainSnap.Rewritten, plainSnap.RewriteRuleIDs)
}
// Verify RewriteInfo errors fail preparation instead of recording a silent false.
if _, err := tr.DB().Exec(`DROP TABLE exchange_rewrites`); err != nil {
t.Fatal(err)
}
if _, err := store.prepare(ctx, []db.TrafficRef{{TrafficID: plainID, Role: "proof"}}); err == nil || !strings.Contains(err.Error(), "rewrite provenance") {
t.Fatalf("prepare with broken exchange_rewrites = %v, want rewrite provenance error", err)
}
}
+1 -1
View File
@@ -325,7 +325,7 @@ func newGateDB(t *testing.T) *gateDB {
t.Helper()
dsn, _, err := db.DSN()
if err != nil {
t.Skipf("no database config (%v) - skipping", err)
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+1 -1
View File
@@ -55,7 +55,7 @@ func targetTestDB(t *testing.T) *db.DB {
t.Helper()
dsn, _, err := db.DSN()
if err != nil {
t.Skipf("no database config (%v) - skipping", err)
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+9 -1
View File
@@ -151,7 +151,15 @@ func (meteredStreamProvider) Stream(context.Context, llm.CompletionRequest) iter
}
func TestSideUsageRecordedOnceOnConsumerCancellation(t *testing.T) {
t.Skip("timing-sensitive: passes locally, flaky on CI runners; tracked in BaaRai/ARTEX#9")
// Timing-sensitive on 2-core CI runners (BaaRai/ARTEX#9). Repro:
// ARTEX_PG_DSN=postgres://artex:testpass@localhost:5432/artextest_race \
// go test ./llmrec -run TestSideUsageRecordedOnceOnConsumerCancellation -count=20 -race -v
// The `early=` t.Fatalf assertion below compares the single llm_usage row's
// count/input/output/worker/status; in the `early` branch the status
// occasionally comes back "ok" instead of "error" because the consumer's
// cancellation and the recorder's deferred finish race on a loaded runner.
// Keep the skip until the metering status is made cancellation-explicit.
t.Skip("timing-sensitive on CI runners; repro and failing assertion documented above; BaaRai/ARTEX#9")
dsn := os.Getenv("ARTEX_PG_DSN")
if dsn == "" {
t.Skip("requires isolated ARTEX_PG_DSN")
+2 -2
View File
@@ -88,11 +88,11 @@ func TestStreamUsageEchoCountedOnce(t *testing.T) {
func TestStreamUsageEchoLedgerRow(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skipf("no database config (%v)", err)
db.SkipPGUnavailable(t, err)
}
pg, err := db.Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer pg.Close()
if err := pg.EnsureLLMUsageTable(); err != nil {
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env bash
# Verify ARTEX locally, reproducing the checks from .github/workflows/ci.yml.
#
# Usage:
# scripts/verify.sh [--full] [--go-only] [--web-only] [--keep-going]
#
# --full also run `go test -race` (the CI package list) and the web static build
# --go-only run only the Go checks
# --web-only run only the web checks
# --keep-going continue after a failed step and report every failure at the end
# -h, --help show this help
#
# The default is the fast set: gofmt, build, vet, tests, web type-check and web
# unit tests, shellcheck and the shell/python checks. It does not need Docker.
set -euo pipefail
cd "$(dirname "$0")/.."
usage() { sed -n '2,14p' "$0" | sed 's/^# \?//'; }
FULL=0
GO_ONLY=0
WEB_ONLY=0
KEEP_GOING=0
while [ $# -gt 0 ]; do
case "$1" in
--full) FULL=1; shift ;;
--go-only) GO_ONLY=1; shift ;;
--web-only) WEB_ONLY=1; shift ;;
--keep-going) KEEP_GOING=1; shift ;;
-h|--help) usage; exit 0 ;;
*) echo "unknown option: $1" >&2; usage >&2; exit 2 ;;
esac
done
if [ "$GO_ONLY" -eq 1 ] && [ "$WEB_ONLY" -eq 1 ]; then
echo "--go-only and --web-only are mutually exclusive" >&2
exit 2
fi
FAILED=0
run_step() {
local label="$1"
shift
printf '\n==> %s\n' "$label"
if "$@"; then
return 0
fi
printf 'FAILED: %s\n' "$label" >&2
FAILED=1
if [ "$KEEP_GOING" -eq 0 ]; then
exit 1
fi
return 0
}
require_cmd() {
if ! command -v "$1" >/dev/null 2>&1; then
printf 'missing prerequisite: %s\n' "$1" >&2
exit 127
fi
}
# --go-only and --web-only narrow the run; otherwise both halves are checked.
want_go() { [ "$WEB_ONLY" -eq 0 ]; }
want_web() { [ "$GO_ONLY" -eq 0 ]; }
if want_go; then
require_cmd go
require_cmd git
want=$(awk '/^go /{print $2; exit}' go.mod)
got=$(go env GOVERSION | sed 's/^go//')
# Numeric compare, portable to macOS (no GNU `sort -V`). A newer toolchain
# is fine: CI deliberately runs a `go-stable` job alongside the pinned floor.
version_ge() {
awk -v a="$1" -v b="$2" 'BEGIN {
split(a, A, "."); split(b, B, ".")
for (i = 1; i <= 3; i++) {
if (A[i] + 0 > B[i] + 0) exit 0
if (A[i] + 0 < B[i] + 0) exit 1
}
exit 0
}'
}
if [ -z "$want" ] || [ -z "$got" ] || ! version_ge "$got" "$want"; then
printf 'Go %s does not meet the go.mod floor %s\n' "$got" "$want" >&2
exit 1
fi
if [ -z "${ARTEX_PG_DSN:-}" ]; then
printf 'note: ARTEX_PG_DSN is unset; PostgreSQL-dependent tests will skip themselves\n'
fi
check_gofmt() {
local -a files=()
local out
mapfile -t files < <(git ls-files '*.go')
if [ "${#files[@]}" -eq 0 ]; then
return 0
fi
out=$(gofmt -l "${files[@]}")
if [ -n "$out" ]; then
printf 'gofmt required:\n%s\n' "$out"
return 1
fi
}
run_step "gofmt" check_gofmt
run_step "go build" go build ./...
run_step "go vet" go vet ./...
run_step "go test -p 1" go test -p 1 ./...
if [ "$FULL" -eq 1 ]; then
RACE_PKGS=(
./agent ./cmd/artex ./config ./db ./evidence ./guard
./intercept ./llmpool ./llmrec ./mcphttp ./notify ./server
./sidequestion ./traffic ./traffic/rewrite
)
run_step "go test -race -p 1" go test -race -p 1 "${RACE_PKGS[@]}"
fi
fi
if want_web; then
require_cmd npm
require_cmd npx
if [ ! -d web/node_modules ] || [ web/package-lock.json -nt web/node_modules ]; then
run_step "npm ci" npm --prefix web ci
fi
run_step "tsc --noEmit" bash -c 'cd web && npx tsc --noEmit'
run_step "web polling tests" bash -c 'cd web && npm run test:polling'
run_step "web replay tests" bash -c 'cd web && npm run test:replay'
run_step "web upload tests" bash -c 'cd web && npm run test:upload'
if [ "$FULL" -eq 1 ]; then
run_step "web static build" bash -c 'cd web && npm run build:static'
fi
fi
# The shell/python checks are part of neither --go-only nor --web-only: they
# are cheap and guard the install/update path used by every deployment.
if [ "$GO_ONLY" -eq 0 ] && [ "$WEB_ONLY" -eq 0 ]; then
require_cmd python3
if command -v shellcheck >/dev/null 2>&1; then
run_step "shellcheck" shellcheck -S warning \
install.sh update.sh start.sh scripts/backup.sh scripts/verify.sh \
reset-password.sh scripts/extract-changelog-section.sh \
scripts/test-extract-changelog-section.sh .github/scripts/bot.sh \
scripts/test-bot-context.sh
else
printf 'note: shellcheck is not installed; skipping the shellcheck step\n'
fi
run_step "posture check" python3 scripts/posture_check_test.py
run_step "changelog section tests" bash scripts/test-extract-changelog-section.sh
run_step "bot context tests" bash scripts/test-bot-context.sh
fi
printf '\n'
if [ "$FAILED" -ne 0 ]; then
printf 'verification FAILED (one or more steps above)\n' >&2
exit 1
fi
printf 'verification passed\n'
+2 -2
View File
@@ -28,11 +28,11 @@ func hasSkillTool(tools []actool.CoreTool) bool {
func TestAssembleVisibleSkill(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skipf("no database config (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
pg, err := db.Open(dsn)
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer pg.Close()
+6 -6
View File
@@ -36,7 +36,7 @@ func TestCompanyScopeInputsAcceptStructuredAndLegacyRules(t *testing.T) {
func TestCreateCompanyRejectsNormalizedDuplicateWithoutChangingScope(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
@@ -76,7 +76,7 @@ func TestCreateCompanyRejectsNormalizedDuplicateWithoutChangingScope(t *testing.
func TestDeleteCompanyRefreshesLiveTaskCompanyIDs(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
@@ -117,7 +117,7 @@ func TestDeleteCompanyRefreshesLiveTaskCompanyIDs(t *testing.T) {
func TestCompanyScopeHTTPErrorClassificationAndBounds(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
s := &Server{m: m}
@@ -191,7 +191,7 @@ func TestCompanyScopeHTTPErrorClassificationAndBounds(t *testing.T) {
func TestListAssetsClassifiesValidationAndDatabaseErrors(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
s := &Server{m: m}
@@ -217,7 +217,7 @@ func TestListAssetsClassifiesValidationAndDatabaseErrors(t *testing.T) {
func TestDeleteCompanyRejectsBadJSONAndReportsMissing(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
s := &Server{m: m}
@@ -256,7 +256,7 @@ func TestDeleteCompanyRejectsBadJSONAndReportsMissing(t *testing.T) {
func TestCompanyScopeSystemFailureIsHTTP500(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
s := &Server{m: m}
+6 -4
View File
@@ -9,6 +9,8 @@ import (
"sync"
"testing"
"time"
"github.com/Autumn-27/artex/db"
)
func TestPasswordVersionClaim(t *testing.T) {
@@ -50,7 +52,7 @@ func TestPasswordVersionClaim(t *testing.T) {
func TestChangePasswordRevokesOldTokenAndKeepsItWhenSaveFails(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
@@ -150,7 +152,7 @@ func TestChangePasswordRevokesOldTokenAndKeepsItWhenSaveFails(t *testing.T) {
func TestUnreadablePasswordVersionRejectsSessions(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
key := []byte("0123456789abcdef0123456789abcdef")
@@ -186,7 +188,7 @@ func TestUnreadablePasswordVersionRejectsSessions(t *testing.T) {
func TestLoginRereadsPasswordVersionWhenUnreadable(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
@@ -250,7 +252,7 @@ func TestLoginRereadsPasswordVersionWhenUnreadable(t *testing.T) {
func TestAbsentPasswordVersionKeepsSessions(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
key := []byte("0123456789abcdef0123456789abcdef")
+5 -3
View File
@@ -8,6 +8,8 @@ import (
"runtime"
"strings"
"testing"
"github.com/Autumn-27/artex/db"
)
// jwtKeyDirs returns fresh key, data and legacy base dirs and points
@@ -194,7 +196,7 @@ func TestLoadOrCreateJWTKeyNeverOverwrites(t *testing.T) {
func TestAuthStatusFailsClosedWhenDataSourceUnavailable(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
// Close the pool so subsequent GetSetting returns an error instead of sql.ErrNoRows.
@@ -220,7 +222,7 @@ func TestAuthStatusFailsClosedWhenDataSourceUnavailable(t *testing.T) {
func TestAuthInitFailsClosedWhenDataSourceUnavailable(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
if err := m.pg.Close(); err != nil {
@@ -245,7 +247,7 @@ func TestAuthInitFailsClosedWhenDataSourceUnavailable(t *testing.T) {
func TestAuthInitRequiresSetupToken(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
clearPassword := func() {
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"testing"
"time"
"github.com/Autumn-27/artex/db"
"golang.org/x/crypto/bcrypt"
)
@@ -61,7 +62,7 @@ func changePasswordServer(t *testing.T, clk *fakeClock) (*Server, string, func(r
t.Helper()
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
t.Cleanup(func() { m.Close() })
+2 -2
View File
@@ -14,7 +14,7 @@ import (
func TestChatUnavailableReasonDistinguishesStates(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
td := t.TempDir()
@@ -66,7 +66,7 @@ func TestChatUnavailableReasonDistinguishesStates(t *testing.T) {
func TestResolveChatAgentHonoursConversationProfile(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
td := t.TempDir()
+1 -1
View File
@@ -119,7 +119,7 @@ func TestChatMentionPagination(t *testing.T) {
func TestChatMentionWorkerReceivesServerDetails(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
ctx, cancel := context.WithCancel(context.Background())
+1 -1
View File
@@ -20,7 +20,7 @@ import (
func TestCoreTaskLifecyclePG(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
td := t.TempDir()
+1 -1
View File
@@ -16,7 +16,7 @@ func TestCostLedgerEurOnUsage(t *testing.T) {
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
db.SkipNoPG(t)
}
pid, err := m.pg.SaveProfile(&db.LLMProfile{
+1 -1
View File
@@ -82,7 +82,7 @@ func TestResetSettledRunReseedsBudgetFromUsage(t *testing.T) {
t.Setenv("ARTEX_TASK_BUDGET_TOKENS", "100000")
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
pgdb.SkipPGUnavailable(t, err)
}
defer m.Close()
+1 -1
View File
@@ -38,7 +38,7 @@ func newRetestServer(t *testing.T) (*Server, int64) {
t.Helper()
dsn, _, err := db.DSN()
if err != nil {
t.Skip("test postgres not configured")
db.SkipPGUnavailable(t, err)
}
pg, err := db.Open(dsn)
if err != nil {
+3 -3
View File
@@ -39,7 +39,7 @@ func TestFindingPaginationParam(t *testing.T) {
func TestFindingGroupsReturnsTaskBucketsAndNormalizesPagination(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
marker := fmt.Sprintf("__server_finding_groups_%d__", time.Now().UnixNano())
@@ -149,7 +149,7 @@ func TestFindingGroupsReturnsTaskBucketsAndNormalizesPagination(t *testing.T) {
func TestDeepenFindingCreatesAuditedIntentAndRevivesTask(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
@@ -278,7 +278,7 @@ func TestDeepenFindingValidatesDescription(t *testing.T) {
func TestDeepenAdmissionFailureDiscardsFollowUpIntent(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
task, err := m.CreateTask("deepen rollback", "leave no orphan intent", nil, 0, 0)
+1 -1
View File
@@ -26,7 +26,7 @@ func TestInheritedActivityDetailAndRelationDeletion(t *testing.T) {
m, err := NewManager(dir, "")
if err != nil {
_ = os.RemoveAll(dir)
t.Skipf("postgres unavailable (%v) - skipping", err)
db.SkipPGUnavailable(t, err)
}
t.Cleanup(func() {
_ = m.Close()
+1 -1
View File
@@ -17,7 +17,7 @@ import (
func TestInterceptAlwaysAllowPG(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+1 -1
View File
@@ -17,7 +17,7 @@ import (
func TestInterceptBatchDecideHTTP(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+1 -1
View File
@@ -34,7 +34,7 @@ func TestInterceptFilterParamsDecidedBy(t *testing.T) {
func TestInterceptDecidedByHTTP(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+1 -1
View File
@@ -15,7 +15,7 @@ import (
func TestInterceptDetailHTTP(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+1 -1
View File
@@ -13,7 +13,7 @@ import (
func TestInterceptFilterHTTP(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
db.SkipPGUnavailable(t, err)
}
d, err := db.Open(dsn)
if err != nil {
+4 -2
View File
@@ -8,6 +8,8 @@ import (
"strconv"
"strings"
"testing"
"github.com/Autumn-27/artex/db"
)
func TestRSILessonsMode(t *testing.T) {
@@ -23,7 +25,7 @@ func TestRSILessonsMode(t *testing.T) {
func TestTaskLessonsExtractionAndIdempotency(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
if _, err := m.pg.Exec(`DELETE FROM rsi_lessons`); err != nil {
@@ -124,7 +126,7 @@ SELECT id,'completed','fixed','{}'::jsonb FROM findings WHERE task_id=$1 AND nam
func TestTerminalStatusExtractsLessons(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
if _, err := m.pg.Exec(`DELETE FROM rsi_lessons`); err != nil {
+1 -1
View File
@@ -124,7 +124,7 @@ func TestLLMTestValidatesUserAgent(t *testing.T) {
func TestStoredProfileKeyOnlyForStoredEndpoint(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
db.SkipPGUnavailable(t, err)
}
t.Cleanup(func() { m.Close() })
ctx, cancel := context.WithCancel(context.Background())
+2 -2
View File
@@ -110,7 +110,7 @@ func TestOllamaKeylessProfileLifecycle(t *testing.T) {
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
db.SkipNoPG(t)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
@@ -197,7 +197,7 @@ func TestOllamaListModels(t *testing.T) {
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
db.SkipNoPG(t)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
+2 -2
View File
@@ -25,7 +25,7 @@ func TestLLMTestStoredKeyFallbackIsTrimmed(t *testing.T) {
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
db.SkipNoPG(t)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
@@ -105,7 +105,7 @@ func TestLLMTestStoredProfileUsesProfileEndpoint(t *testing.T) {
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
db.SkipNoPG(t)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
+2 -1
View File
@@ -10,6 +10,7 @@ import (
"testing"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/llmrec"
"github.com/Autumn-27/norma/llm"
"github.com/Autumn-27/norma/transcript"
@@ -21,7 +22,7 @@ import (
func TestRecorderPersistsRawWireBodies(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
db.SkipPGUnavailable(t, err)
}
defer m.Close()
pg := m.PG()
+2 -2
View File
@@ -15,7 +15,7 @@ import (
func TestSeedAssociatesTargetAssetWithTask(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
pgdb.SkipPGUnavailable(t, err)
}
defer m.Close()
task, err := m.CreateTask("seed ownership", "seed ownership", nil, 0, 0)
@@ -174,7 +174,7 @@ func TestManagerDeleteTaskRestoresFilesAndTrafficWhenPostgresDeleteFails(t *test
dataDir := t.TempDir()
m, err := NewManager(dataDir, "")
if err != nil {
t.Skipf("postgres unavailable (%v) - skipping", err)
pgdb.SkipPGUnavailable(t, err)
}
t.Cleanup(func() { _ = m.Close() })
+4 -2
View File
@@ -7,16 +7,18 @@ import (
"net/http"
"net/http/httptest"
"testing"
"github.com/Autumn-27/artex/db"
)
// TestMgmtAPI exercises the PostgreSQL-backed management API through the real mux.
func TestMgmtAPI(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("database unavailable (%v) — skipping management API test", err)
db.SkipPGUnavailable(t, err)
}
if m.pg == nil {
t.Skip("postgres unavailable — skipping management API test")
db.SkipNoPG(t)
}
// New starts the notifier and task-archive workers on context.Background();
// closing the pool stops them from claiming later tests' digest batches

Some files were not shown because too many files have changed in this diff Show More