Compare commits

..
Author SHA1 Message Date
BaaRai 51e2c603b9 Merge pull request #282 from renaesop/codex/fix-record-fact-input-contract
fix: repair record_fact summary contracts and asset aliases
2026-11-18 03:30:48 +01:00
BaaRai 9355d83eea Merge pull request #314 from yeting1107/docs/post-0-6-0-changelog
docs: record the post-0.6.0 changes
2026-11-18 03:03:44 +01:00
BaaRai 9b850a090c docs: add the post-#305 changelog entries
The rebase onto current main keeps the original entries and adds the three
user-facing changes merged after its base: the finding report_state field,
the ordered LLM health persistence, and the framed intent summaries.
2026-11-18 02:52:25 +01:00
yeting1107 398a08814a docs: record the post-0.6.0 changes 2026-11-18 02:52:06 +01:00
Chubboy Ren c833130c28 fix: enforce exact record_fact fields and validate asset anchors 2026-11-18 01:51:41 +00:00
Chubboy Ren c0c568b31f Merge remote-tracking branch 'upstream/main' into codex/fix-record-fact-input-contract 2026-11-18 01:34:26 +00:00
BaaRai ac5e714eb6 Merge pull request #313 from BaaRai/fix/task-llm-health-badge
fix(llm): record breaker health on every task path
2026-11-18 01:42:49 +01:00
BaaRai 6568a8eb44 fix(llm): record breaker health on every task LLM path
The /system/llm/ badge read llm_profile_health, which only the failover
Pool wrote. A task pinned to an explicit chain, a bound agent with
failover off, and the global provider all call the provider directly, so
a dead backend stayed "Healthy".

One policy now lives in llmpool.Registry.RecordCall, used by the Pool and
by the task runtimes: a success clears the state, a failover-eligible
failure trips it, context cancellation and deterministic 400s do not, and
a stream that already delivered output is left alone. The selection
carries the profile whose breaker the call updates, and a provider
already wrapped in the Pool reports nothing to avoid double counting.

The LLM recording page also explains an empty list while the llm_record
toggle is off, after the settings call resolves.

Originally reported in #257.
2026-11-18 01:31:18 +01:00
BaaRai 9a2a9e61ef Merge pull request #308 from Coff0xc/feat/finding-report-state
feat(findings): report stage separate from triage status
2026-11-18 00:17:48 +01:00
Coff0xc 6a0b79d34f feat(findings): add report_state separate from triage 2026-11-18 00:07:50 +01:00
BaaRai 729f935a21 Merge pull request #311 from BaaRai/fix/agent-frame-intent-summaries
fix(agent): frame intent summaries as data (issue #124)
2026-11-17 23:55:50 +01:00
Chubboy Ren 70d6833f35 fix: repair record_fact input contracts and asset aliases 2026-11-17 06:14:31 +00:00
39 changed files with 1961 additions and 136 deletions
+23
View File
@@ -25,6 +25,13 @@ All notable changes to this project are documented in this file. The format is b
- **Release provenance**: release archives carry GitHub build provenance attestations and a CycloneDX SBOM (`gh attestation verify <zip> --repo BaaRai/ARTEX`); release notes are the matching `CHANGELOG.md` section (#219, #201).
- **Capture-proxy rewrite rules, storage and API**: `GET`, `POST`, `PUT` and `DELETE /api/traffic/rewrite-rules` and `POST /api/traffic/rewrite-rules/{id}/toggle` manage rules that match a request or a response (method, host, URL, headers, body, status) and describe a rewrite (method, path and query, headers, body, status). Rules are validated on save, limited to 100 of which 50 enabled, created disabled, and stored in the new table `traffic_rewrite_rules` (migration `0005_traffic_rewrite_rules.sql`) with `matchers` and `actions` encrypted at rest. The value of an action marked `sensitive` is never returned, and an update sent without it keeps the stored one. The server compiles the enabled rules at startup and after every change and hands the snapshot to the proxy. The proxy applies them from the next entry. See `docs/traffic-rewrite-rules.md`.
- **Capture-proxy rewrite rules applied**: the capture proxy applies enabled rules to each HTTP request before it is sent to the target and to each response before it reaches the client. Each action that ran is recorded in the new table `exchange_rewrites` in the same transaction as the exchange, `traffic_get` shows a `REWRITTEN` line, and `traffic_search` never finds a value a rule wrote. SSE status and headers are rewritten at the start of the stream. **WebSocket handshakes are not rewritten**: the capture library runs its hook after the upgrade, so the status and headers cannot be changed, and frames pass unchanged. **Responses of 5 MiB or more** are forwarded unchanged and not recorded in full, because the library does not call the response hook for them. Snapshots of original requests are capped at 1024 entries and 64 MiB. Replays go through the proxy, so the rules apply to them. Archives, a per-replay rule toggle, the web UI and the proof flag come later.
- **Bulk finding status** (#298): select findings in the findings table and apply one disposition to all of them, up to 500 per call, in one transaction; each row still goes through the single-decision notification semantics and the change is mirrored onto its exploration node (#289).
- **Traffic replay** (#266): replay a recorded request from the traffic detail with its method, URL, headers and stored body; replay is disabled when only the body preview is loaded.
- **Finding proof strength in reports** (#268): the Markdown report and the export show each finding's proof strength next to its verification status.
- **Opt-in Keenable web search backend** (#295): the search tool gains Keenable alongside Brave and Tavily; the key is stored encrypted and can be cleared independently.
- **Durable activity outbox** (#300): activity rows that cannot be stored are written to the new `activity_outbox` table (migration `0006_activity_outbox.sql`) and replayed once on the next start. The server stops admitting new work before the drain, and a shutdown cancel leaves an in-flight intent claim running so the next start can reset it to open instead of recording it as done or blocked.
- **LLM model search** (#284): the System → LLM model picker can search the model list, and the API-key field opts out of password-manager autofill.
- **Finding report state** (#308): findings carry a `report_state` (`none`, `submitted`, `accepted`, `rejected`, `duplicate`) for what was sent to a bug bounty program, separate from the triage status; the bulk update accepts both fields, and restoring an older archive fills `none` (migration `0007_finding_report_state.sql`).
### Changed
@@ -33,15 +40,31 @@ All notable changes to this project are documented in this file. The format is b
- **Untrusted-output boundary is now on by default**: Norma frames tool results for the worker, planner, chat and main agent unless `ARTEX_UNTRUSTED_BOUNDARY` is set to `0`, `false`, `off` or `no`. It was opt-in via `ARTEX_UNTRUSTED_BOUNDARY=1` in 0.6.0 (#166); the default flips here (#203).
- **Versioned migrations**: the `db` package applies numbered `db/migrations/NNNN_slug.sql` files once, in order, under the schema advisory lock, and records a SHA-256 checksum in `schema_migrations`; missing, renamed, changed or out-of-order migrations fail closed (#226).
- **Coverage graph performance**: the coverage graph batches its connector lookups (three queries instead of one per root and per company) and adds the partial index `idx_ts_icp`; on a task of about 3,000 assets the p50 dropped from 126 ms to 27 ms and the queries per call from 221 to 3 (#230).
- **Request body bounds** (#305): JSON request bodies decode at most 32 MiB, every non-multipart body is capped, and an oversized body answers 413 instead of a generic bad request. `ReadTimeout` bounds how long headers and the body may take to arrive without cutting a handler that is already running.
- **Chat attachments up to 512 MiB** (#252): the client, server, compose file and reverse-proxy sample agree on a 512 MiB combined multipart limit.
- **MITM passthrough TTL** (#306): a host that cannot be intercepted is tunnelled for one minute, then interception is tried again; an expired mark counts as intercepted.
- **Norma v0.4.11** (#294).
- **CI**: the suite also runs on the latest Go release as an advisory job, the rewrite-rule engine runs under the race detector, and the gzip/compression fixtures no longer depend on the local Go version (#279, #278).
- **Repository hygiene** (#275): `docs/` is no longer ignored, and contributing documents the 7-open-pull-request cap (#307).
### Fixed
- **LLM health badge on every task path**: `/system/llm/` read a breaker state that only the failover Pool wrote, so a profile used by an explicit task chain, by a bound agent with failover off, or as the global provider stayed "Healthy" through a real outage. Every task LLM attempt now records its outcome through the same policy the Pool uses, keyed by the profile that served it. The LLM recording page also explains an empty list when the `llm_record` toggle is off instead of looking broken.
- **LLM health persistence** (#304): persisted breaker rows are applied in sequence at startup and written before the database closes, and a slow write no longer holds the registry lock while it runs.
- **Skill visibility survives restarts**: turning a skill off now writes an explicit disabled row instead of deleting it, so the startup seed, which only inserts missing rows, can no longer bring the skill back. Skills that were turned off before this version have no row to keep, so they come back once after the upgrade and must be turned off again.
- **Browser extension traffic capture**: CDP delivers `Network.responseReceived` response headers as an object, not an array of `{name, value}` records, so the extension threw on every response and never queued an exchange. It now accepts both shapes and lower-cases names before the body-capture gate, so captures work again and `content-type` correctly controls body capture.
- **bbot on arm64**: the image now builds on arm64 hosts such as Apple Silicon. bbot's `gowitness` downloaded an x86-64 Chrome and `retirejs` an x86-64 Node, which can't run there, so the build failed. On arm64 the image uses Debian's `chromium` for gowitness and an arm64 Node for retirejs, and the build check now covers `retirejs` too. amd64 builds are unchanged.
- **Auto-scope hygiene**: auto `task_scope` rows count for coverage and untested assets only under a declared root, and direct coverage counts anchored assets through fact anchors only; the scope is described as "coverage scope, not enforced" (#196).
- **Enrich HTTP probe**: it now dials through `intercept.PinnedDialer`, so the target gate applies and a probe without a target scope is refused (#198).
- **Agent hook data races**: the eight package-level hooks are synchronised, so a second wire-up no longer races tasks that already read them (#227).
- **Coverage flag read failure** (#303): asset-coverage tools stay available when the coverage flag cannot be read, and a failed read no longer disables coverage for the task.
- **RSI lesson correctness** (#287, #290): lessons are scoped from `task_scope`, UTF-8 is no longer split mid-character, caution lessons are looked up by cue, lesson evidence and every retest are retracted when a finding's disposition reverses, and extraction is skipped on a task with more than one company.
- **Finding disposition mirror** (#289): finding triage is mirrored onto its exploration node, so the graph and the findings table agree.
- **RSI lessons in the data envelope** (#288): lesson text reaches the planner and worker inside the same nonce-framed envelope as other target-derived content.
- **Intent summaries framed** (#311): the intent summary on the worker-finished and finding lines, and the summaries in `graph_overview`, now reach the planner inside the nonce frame; the structured intent fields stay in the plain JSON, and the operator-typed deletion reason stays outside.
- **LLM pool context window** (#286): the pool and side-question paths use the model's real context window instead of a default when computing compression thresholds.
- **Sensitive rewrite values** (#273 follow-ups): a kept sensitive value is matched on action type, header name and regex, and a host change is refused while a stored sensitive value is kept.
- **Triage bot context** (#274): the PR triage bot receives a directory map instead of a truncated file list.
### Security
+8 -1
View File
@@ -233,7 +233,11 @@ worker remains confined to the intent it claimed.
</details>
- **Verified findings**: findings carry a POE level, a verification status and a note; a finding is
`verified` only when a bound proof/response and verbatim evidence quotes support it. Gzip/deflate
bodies are decoded before the quote check.
bodies are decoded before the quote check. The findings table supports bulk status updates, and
the report and export show each finding's proof strength next to its status.
- **Traffic replay**: open a captured exchange and replay the recorded request (method, URL,
headers and stored body) from the traffic detail; replay stays disabled while only the body
preview is loaded.
- **Vulnerability retest loop**: a dedicated `retester` agent re-runs a finding with optional fixed
version/conditions, records `Still reproducible / Fixed / Cannot confirm`, and can move the
finding to `Fixed` automatically. Original evidence and reports are preserved.
@@ -295,6 +299,9 @@ worker remains confined to the intent it claimed.
for every call.
- **MCP servers, skills, memory and a workspace**: extend the agent with remote tools and reusable
prompt/playbook material.
- **Web search**: the agent's search tool uses Brave, Tavily or the opt-in Keenable backend,
configured in **System → Settings** with the key encrypted at rest; search can also egress
through its own proxy.
- **Single binary**: the Next.js UI is embedded with `go:embed`; PostgreSQL is the only external
dependency.
+41
View File
@@ -0,0 +1,41 @@
package agent
import (
"encoding/json"
"fmt"
"slices"
)
// factAssetAnchors accepts the historical asset_ids spelling and its explicit
// tested_asset_ids alias. Never union conflicting lists: that could attribute
// a conclusion to assets the caller did not actually check.
func (it factItem) factAssetAnchors() ([]int64, error) {
assets, err := parseFactAssetIDs("asset_ids", it.AssetIDs)
if err != nil {
return nil, err
}
tested, err := parseFactAssetIDs("tested_asset_ids", it.TestedAssetIDs)
if err != nil {
return nil, err
}
if it.AssetIDs != nil && it.TestedAssetIDs != nil && !slices.Equal(assets, tested) {
return nil, fmt.Errorf("asset_ids and tested_asset_ids must contain the same asset IDs when both are provided; use one field or correct the conflicting lists")
}
if it.TestedAssetIDs != nil {
return tested, nil
}
return assets, nil
}
func parseFactAssetIDs(field string, raw []json.RawMessage) ([]int64, error) {
ids := make([]int64, 0, len(raw))
for i, value := range raw {
id := pid(value)
if id <= 0 {
return nil, fmt.Errorf("%s[%d] must be a positive integer asset ID", field, i)
}
ids = append(ids, id)
}
slices.Sort(ids)
return slices.Compact(ids), nil
}
+182
View File
@@ -0,0 +1,182 @@
package agent
import (
"encoding/json"
"fmt"
"slices"
"strconv"
"strings"
"testing"
"github.com/Autumn-27/artex/db"
)
func TestRecordFactAssetAliases(t *testing.T) {
d := testDB(t)
defer d.Close()
task, err := d.CreateTask("fact asset alias", "test", nil, 0, 0)
if err != nil {
t.Fatal(err)
}
defer d.DeleteTask(task.ID)
assets := d.Assets()
first, err := assets.UpsertRootDomain(db.UpsertRootDomainReq{Domain: "fact-alias-one.example", TaskID: task.ID})
if err != nil {
t.Fatal(err)
}
second, err := assets.UpsertRootDomain(db.UpsertRootDomainReq{Domain: "fact-alias-two.example", TaskID: task.ID})
if err != nil {
t.Fatal(err)
}
defer assets.DeleteByIDs([]int64{first, second})
store := d.Exploration(task.ExplorationID)
tools := NewToolSet(store, "worker")
check := func(t *testing.T, id int64, want []int64) {
t.Helper()
rows, err := d.Query("SELECT asset_id FROM exploration_anchors WHERE node_id=$1 ORDER BY asset_id", id)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
var got []int64
for rows.Next() {
var asset int64
if err := rows.Scan(&asset); err != nil {
t.Fatal(err)
}
got = append(got, asset)
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
slices.Sort(want)
if !slices.Equal(got, want) {
t.Fatalf("anchors = %v, want %v", got, want)
}
}
for _, tc := range []struct{ name, fields string }{
{"canonical", fmt.Sprintf(`"asset_ids":[%d,%d,%d]`, first, second, first)},
{"alias", fmt.Sprintf(`"tested_asset_ids":[%d,%d]`, first, second)},
{"equal sets", fmt.Sprintf(`"asset_ids":[%d,%d],"tested_asset_ids":[%d,%d,%d]`, first, second, second, first, first)},
{"numeric strings", fmt.Sprintf(`"tested_asset_ids":["%d","%d"]`, first, second)},
} {
t.Run(tc.name, func(t *testing.T) {
out := callGuardTool(t, tools.recordFact(), fmt.Sprintf(`{"summary":%q,%s}`, tc.name, tc.fields))
if out.IsError {
t.Fatal(out.Flatten())
}
id, err := strconv.ParseInt(strings.TrimPrefix(out.Flatten(), "fact recorded: "), 10, 64)
if err != nil {
t.Fatal(out.Flatten(), err)
}
check(t, id, []int64{first, second})
})
}
before := tools.Writes().Facts
for _, fields := range []string{
fmt.Sprintf(`"asset_ids":[%d],"tested_asset_ids":[%d]`, first, second),
fmt.Sprintf(`"asset_ids":[],"tested_asset_ids":[%d]`, first),
fmt.Sprintf(`"tested_asset_ids":[%d,0]`, first),
fmt.Sprintf(`"asset_ids":[%d,-1]`, first),
`"tested_asset_ids":[null]`,
`"tested_asset_ids":[1.5]`,
`"tested_asset_ids":["invalid"]`,
} {
out := callGuardTool(t, tools.recordFact(), `{"summary":"invalid assets",`+fields+`}`)
if !out.IsError || !strings.Contains(out.Flatten(), "asset_ids") {
t.Fatalf("accepted %s: %q", fields, out.Flatten())
}
}
if tools.Writes().Facts != before {
t.Fatal("invalid asset lists wrote facts")
}
out := callGuardTool(t, tools.recordFact(), fmt.Sprintf(`{"facts":[{"summary":"batch alias","tested_asset_ids":[%d]},{"summary":"batch conflict","asset_ids":[%d],"tested_asset_ids":[%d]},{"summary":"no inherited assets"}]}`, first, first, second))
var batch struct {
IDs []int64 `json:"ids"`
Errors map[string]string `json:"errors"`
}
if err := json.Unmarshal([]byte(out.Flatten()), &batch); err != nil {
t.Fatal(out.Flatten(), err)
}
if len(batch.IDs) != 3 || batch.IDs[0] <= 0 || batch.IDs[1] != 0 || batch.IDs[2] <= 0 || !strings.Contains(batch.Errors["1"], "same asset IDs") {
t.Fatalf("batch: %s", out.Flatten())
}
check(t, batch.IDs[0], []int64{first})
check(t, batch.IDs[2], nil)
before = tools.Writes().Facts
malformed := callGuardTool(t, tools.recordFact(), fmt.Sprintf(`{"facts":[{"summary":"must not be saved"},{"summary":"bad list","tested_asset_ids":%d}]}`, first))
if !malformed.IsError || tools.Writes().Facts != before {
t.Fatal("malformed list caused a partial write", malformed.Flatten())
}
}
func TestRecordFactAssetAliasSchema(t *testing.T) {
schema := NewToolSet(nil, "").recordFact().InputSchema()
props := schema["properties"].(map[string]any)
for _, name := range []string{"asset_ids", "tested_asset_ids"} {
if props[name] == nil {
t.Fatalf("single schema missing %s", name)
}
}
item := props["facts"].(map[string]any)["items"].(map[string]any)
itemProps, _ := item["properties"].(map[string]any)
for _, name := range []string{"summary", "asset_ids", "tested_asset_ids"} {
if itemProps[name] == nil {
t.Fatalf("batch schema missing %s", name)
}
}
}
func TestRecordFactRejectsMissingAssets(t *testing.T) {
d := testDB(t)
defer d.Close()
task, err := d.CreateTask("missing fact anchors", "test", nil, 0, 0)
if err != nil {
t.Fatal(err)
}
defer d.DeleteTask(task.ID)
assets := d.Assets()
var ids []int64
for _, name := range []string{"missing-fact-one.example", "missing-fact-two.example", "existing-fact.example"} {
id, err := assets.UpsertRootDomain(db.UpsertRootDomainReq{Domain: name, TaskID: task.ID})
if err != nil {
t.Fatal(err)
}
ids = append(ids, id)
}
defer assets.DeleteByIDs(ids)
if _, err := assets.DeleteByIDs(ids[:2]); err != nil {
t.Fatal(err)
}
tools := NewToolSet(d.Exploration(task.ExplorationID), "worker")
var beforeFacts int
if err := d.QueryRow("SELECT count(*) FROM exploration_nodes WHERE exploration_id=$1 AND kind='fact'", task.ExplorationID).Scan(&beforeFacts); err != nil {
t.Fatal(err)
}
for _, field := range []string{"asset_ids", "tested_asset_ids"} {
out := callGuardTool(t, tools.recordFact(), fmt.Sprintf(`{"summary":"missing anchors","%s":[%d,%d,%d]}`, field, ids[0], ids[1], ids[2]))
if !out.IsError || !strings.Contains(out.Flatten(), "do not exist") ||
!strings.Contains(out.Flatten(), strconv.FormatInt(ids[0], 10)) ||
!strings.Contains(out.Flatten(), strconv.FormatInt(ids[1], 10)) ||
strings.Contains(out.Flatten(), "23503") || strings.Contains(out.Flatten(), "foreign key") {
t.Fatalf("missing assets did not produce an actionable error: %s", out.Flatten())
}
}
var count int
if err := d.QueryRow("SELECT count(*) FROM exploration_nodes WHERE exploration_id=$1 AND kind='fact'", task.ExplorationID).Scan(&count); err != nil || count != beforeFacts || tools.Writes().Facts != 0 {
t.Fatalf("failed anchors left facts: count=%d err=%v", count, err)
}
out := callGuardTool(t, tools.recordFact(), fmt.Sprintf(`{"facts":[{"summary":"valid","asset_ids":[%d]},{"summary":"missing","tested_asset_ids":[%d]}]}`, ids[2], ids[0]))
var batch struct {
IDs []int64 `json:"ids"`
Errors map[string]string `json:"errors"`
}
if err := json.Unmarshal([]byte(out.Flatten()), &batch); err != nil {
t.Fatal(out.Flatten(), err)
}
if !out.IsError || len(batch.IDs) != 2 || batch.IDs[0] <= 0 || batch.IDs[1] != 0 || !strings.Contains(batch.Errors["1"], "do not exist") || tools.Writes().Facts != 1 {
t.Fatal("missing asset batch handling:", out.Flatten())
}
}
+160
View File
@@ -0,0 +1,160 @@
package agent
import (
"bytes"
"encoding/json"
"fmt"
)
// Match strings.TrimSpace, including Unicode whitespace, in the published schema.
const factWhitespace = "\t\n\v\f\r \u0085\u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u3000"
func factAssetIDSchema() map[string]any {
return map[string]any{
"type": []string{"integer", "string"}, "minimum": 1,
"pattern": "^[" + factWhitespace + "]*\\+?0*[1-9][0-9]*[" + factWhitespace + "]*$",
}
}
func factItemSchema() map[string]any {
schema := obj(map[string]any{
"summary": str("Required on every fact: a non-empty one-sentence summary of this conclusion. Batch items do not inherit summary from the top level."),
"intent_id": idp("The intent that produced this fact; only this field inherits from the top level in a batch."),
"detail": str("Details of this fact, including all related observations."),
"evidence": str("One line of key evidence: the command and the most proving output lines. Put longer output in detail."),
"confidence": str("observed (seen directly in output) | inferred (inferred from phenomena). Label negative conclusions honestly."),
"source": map[string]any{"type": "string", "enum": factSources, "description": "where the content came from: target_response | tool_output | third_party_intel | operator_input | inference (default when omitted)"},
"asset_ids": map[string]any{"type": "array", "items": factAssetIDSchema(), "description": "Asset IDs explicitly involved in this fact. Use asset_ids as the canonical field; tested_asset_ids is accepted as an alias. If both are provided, they must name the same set. Do not infer IDs from the intent."},
"tested_asset_ids": map[string]any{"type": "array", "items": factAssetIDSchema(), "description": "Compatibility alias for asset_ids, not an additional list. When both are provided their sets must match. Only include assets actually involved in this conclusion."},
}, "summary")
summary := schema["properties"].(map[string]any)["summary"].(map[string]any)
summary["minLength"] = 1
summary["pattern"] = "[^" + factWhitespace + "]"
schema["additionalProperties"] = false
return schema
}
func recordFactSchema() map[string]any {
props := factItemSchema()["properties"].(map[string]any)
props["facts"] = map[string]any{
"type": "array", "minItems": 1, "items": factItemSchema(),
"description": "A non-empty batch of complete facts. Every item needs its own summary. Only intent_id inherits from the top level. Returned ids match item order; retry only failed indices.",
}
schema := obj(props)
// Providers reject root-level anyOf. The handler enforces single/batch mode.
schema["additionalProperties"] = false
return schema
}
// RepairRecordFactSchema restores the single/batch input contract in stored or
// cached schemas, preserving annotations only on fields the handler accepts.
func RepairRecordFactSchema(stored map[string]any) map[string]any {
raw, _ := json.Marshal(stored)
var out map[string]any
_ = json.Unmarshal(raw, &out)
if out == nil {
out = map[string]any{}
}
mergeFactContract(out, recordFactSchema())
// A batch requires summaries on its items, never a redundant top-level one.
delete(out, "required")
delete(out, "anyOf")
canonical := recordFactSchema()["properties"].(map[string]any)
props := out["properties"].(map[string]any)
for name := range props {
if _, known := canonical[name]; !known {
delete(props, name)
}
}
items := props["facts"].(map[string]any)["items"].(map[string]any)
itemProps := items["properties"].(map[string]any)
for name := range itemProps {
if _, known := canonical[name]; !known || name == "facts" {
delete(itemProps, name)
}
}
return out
}
func mergeFactContract(dst, src map[string]any) {
for key, value := range src {
if key == "description" || key == "default" {
if _, ok := dst[key]; ok {
continue
}
}
if child, ok := value.(map[string]any); ok {
target, _ := dst[key].(map[string]any)
if target == nil {
target = map[string]any{}
dst[key] = target
}
mergeFactContract(target, child)
} else {
dst[key] = value
}
}
}
type recordFactInput struct {
Facts []factItem `json:"facts"`
factItem
}
// Validate exact field names before decoding: encoding/json otherwise treats
// Summary, SUMMARY, etc. as aliases even with DisallowUnknownFields enabled.
func decodeRecordFactInput(in json.RawMessage) (recordFactInput, bool, error) {
var out recordFactInput
fields, err := exactFactFields(in, true)
if err != nil {
return out, false, err
}
_, batch := fields["facts"]
if batch {
for name := range fields {
if name != "intent_id" && name != "facts" {
return out, true, fmt.Errorf("field %q is not allowed at the top level in batch mode; only intent_id and facts are accepted", name)
}
}
var items []json.RawMessage
if err := json.Unmarshal(fields["facts"], &items); err != nil {
return out, true, fmt.Errorf("facts must be an array: %w", err)
}
if len(items) == 0 {
return out, true, fmt.Errorf("facts must be a non-empty array of complete facts, each with its own summary")
}
for i, item := range items {
if _, err := exactFactFields(item, false); err != nil {
return out, true, fmt.Errorf("facts[%d]: %w", i, err)
}
}
}
decoder := json.NewDecoder(bytes.NewReader(in))
decoder.DisallowUnknownFields() // Also applies recursively to every factItem.
if err := decoder.Decode(&out); err != nil {
return out, batch, err
}
return out, batch, nil
}
func exactFactFields(in json.RawMessage, topLevel bool) (map[string]json.RawMessage, error) {
var fields map[string]json.RawMessage
if err := json.Unmarshal(in, &fields); err != nil {
return nil, err
}
if fields == nil {
return nil, fmt.Errorf("record_fact input and each fact must be objects")
}
for name := range fields {
switch name {
case "summary", "detail", "evidence", "confidence", "source", "intent_id", "asset_ids", "tested_asset_ids":
case "facts":
if !topLevel {
return nil, fmt.Errorf("unknown field %q", name)
}
default:
return nil, fmt.Errorf("unknown field %q (field names are case-sensitive)", name)
}
}
return fields, nil
}
+165
View File
@@ -0,0 +1,165 @@
package agent
import (
"encoding/json"
"regexp"
"strings"
"testing"
actool "github.com/Autumn-27/norma/tool"
)
func TestRecordFactSummaryContract(t *testing.T) {
tools := NewToolSet(guardTaskStore(t), "worker")
tool := tools.recordFact()
for _, input := range []string{
`{}`, `null`, `[]`,
`{"summary":""}`, `{"summary":" "}`,
`{"summary":"must not substitute","facts":[]}`,
`{"summary":"must not substitute","facts":null}`,
`{"summary":"must not inherit","facts":[{"detail":"missing summary"}]}`,
`{"facts":[{"summary":"must not save"},{"summary":42}]}`,
} {
out := callGuardTool(t, tool, input)
if !out.IsError {
t.Errorf("accepted %s: %s", input, out.Flatten())
}
}
if tools.Writes().Facts != 0 {
t.Fatal("invalid inputs wrote facts")
}
single := callGuardTool(t, tool, `{"summary":"valid single"}`)
if single.IsError {
t.Fatal(single.Flatten())
}
batch := callGuardTool(t, tool, `{"facts":[{"summary":"valid batch"},{"detail":"missing summary"}]}`)
var result struct {
IDs []int64 `json:"ids"`
Errors map[string]string `json:"errors"`
Retry string `json:"retry_instruction"`
}
if err := json.Unmarshal([]byte(batch.Flatten()), &result); err != nil {
t.Fatal(batch.Flatten(), err)
}
if !batch.IsError || len(result.IDs) != 2 || result.IDs[0] <= 0 || result.IDs[1] != 0 ||
!strings.Contains(result.Errors["1"], "summary") || result.Retry == "" || tools.Writes().Facts != 2 {
t.Fatalf("partial failure is not actionable: %s", batch.Flatten())
}
}
func TestRecordFactStoredContractRepair(t *testing.T) {
old := map[string]any{
"type": "object", "required": []any{"summary"},
"anyOf": []any{map[string]any{"required": []string{"summary"}}},
"properties": map[string]any{
"summary": map[string]any{"type": "string", "description": "operator summary"},
"confidence": map[string]any{"type": "string", "default": "inferred"},
"custom_field": map[string]any{"type": "string"},
"facts": map[string]any{"type": "array", "items": map[string]any{"type": "object"}},
},
}
base := NewToolSet(nil, "").recordFact()
repaired := DecorateTool(base, "operator tool", old)
schema := repaired.InputSchema()
if _, required := schema["required"]; required {
t.Fatal("batch still requires a top-level summary")
}
if _, exists := schema["anyOf"]; exists {
t.Fatal("root anyOf reaches the provider")
}
props := schema["properties"].(map[string]any)
item := props["facts"].(map[string]any)["items"].(map[string]any)
required := item["required"].([]string)
if len(required) != 1 || required[0] != "summary" || props["facts"].(map[string]any)["minItems"] != 1 {
t.Fatal("batch must require a nonempty list with a summary per item")
}
if props["summary"].(map[string]any)["minLength"] != 1 ||
item["properties"].(map[string]any)["summary"].(map[string]any)["minLength"] != 1 {
t.Fatal("empty summaries permitted by schema")
}
if repaired.Description() != "operator tool" ||
props["summary"].(map[string]any)["description"] != "operator summary" ||
props["confidence"].(map[string]any)["default"] != "inferred" || props["custom_field"] != nil {
t.Fatal("operator annotations lost")
}
if _, ok := old["properties"].(map[string]any)["tested_asset_ids"]; ok {
t.Fatal("shared old schema mutated")
}
first, _ := json.Marshal(schema)
second, _ := json.Marshal(DecorateTool(base, "", schema).InputSchema())
if string(first) != string(second) {
t.Fatal("repair is not idempotent")
}
var roundTrip map[string]any
if err := json.Unmarshal(first, &roundTrip); err != nil {
t.Fatal(err)
}
for _, input := range []string{`{"summary":"single"}`, `{"facts":[{"summary":"batch"}]}`} {
if err := actool.ValidateInput(roundTrip, json.RawMessage(input)); err != nil {
t.Fatalf("SDK rejects valid %s after catalog round trip: %v", input, err)
}
}
}
func TestRecordFactStrictFields(t *testing.T) {
tools := NewToolSet(guardTaskStore(t), "worker")
for _, tc := range []struct{ input, field string }{
{`{"summary":"valid","typo":1}`, "typo"},
{`{"Summary":"case alias"}`, "Summary"},
{`{"summary":"valid","Facts":[]}`, "Facts"},
{`{"facts":[{"summary":"first"},{"summary":"second","typo":1}]}`, "typo"},
{`{"facts":[{"summary":"first"},{"Summary":"case alias"}]}`, "Summary"},
{`{"facts":[{"summary":"first"},{"summary":"second","ASSET_IDS":[]}]}`, "ASSET_IDS"},
{`{"facts":[{"summary":"nested","facts":[]}]}`, "facts"},
{`{"summary":"ignored","facts":[{"summary":"batch"}]}`, "summary"},
{`{"detail":"ignored","facts":[{"summary":"batch"}]}`, "detail"},
{`{"evidence":"ignored","facts":[{"summary":"batch"}]}`, "evidence"},
{`{"source":"inference","facts":[{"summary":"batch"}]}`, "source"},
{`{"confidence":"inferred","facts":[{"summary":"batch"}]}`, "confidence"},
{`{"asset_ids":[],"facts":[{"summary":"batch"}]}`, "asset_ids"},
{`{"tested_asset_ids":[],"facts":[{"summary":"batch"}]}`, "tested_asset_ids"},
} {
out := callGuardTool(t, tools.recordFact(), tc.input)
if !out.IsError || !strings.Contains(out.Flatten(), tc.field) {
t.Errorf("input %s: want named rejection for %s, got %s", tc.input, tc.field, out.Flatten())
}
}
if tools.Writes().Facts != 0 {
t.Fatal("malformed fields caused partial writes")
}
for _, input := range []string{
`{"facts":[null]}`, `{"facts":[[]]}`, `{"summary":"valid"} {"summary":"extra object"}`,
} {
if _, _, err := decodeRecordFactInput(json.RawMessage(input)); err == nil {
t.Errorf("accepted %s", input)
}
}
if _, batch, err := decodeRecordFactInput(json.RawMessage(`{"intent_id":1,"facts":[{"summary":"valid","intent_id":2}]}`)); err != nil || !batch {
t.Fatalf("valid batch: %v %v", batch, err)
}
}
func TestRecordFactSchemaMatchesStrings(t *testing.T) {
props := recordFactSchema()["properties"].(map[string]any)
summary := regexp.MustCompile(props["summary"].(map[string]any)["pattern"].(string))
for _, value := range []string{"", " ", "\t\r\n", "\u00a0\u0085", "\u2003\u3000", "conclusion", " observed "} {
if got, want := summary.MatchString(value), strings.TrimSpace(value) != ""; got != want {
t.Errorf("summary %q: schema=%v runtime=%v", value, got, want)
}
}
id := regexp.MustCompile(factAssetIDSchema()["pattern"].(string))
for _, value := range []string{"1", "001", "+1", " +001 ", "\u200312\u3000", "", "0", "-1", "1.5", "a"} {
raw, _ := json.Marshal(value)
if got, want := id.MatchString(value), pid(raw) > 0; got != want {
t.Errorf("asset ID %q: schema=%v runtime=%v", value, got, want)
}
}
for _, input := range []string{
`{"summary":"single","asset_ids":["1",2]}`,
`{"facts":[{"summary":"batch","tested_asset_ids":["1",2]}]}`,
} {
if err := actool.ValidateInput(recordFactSchema(), json.RawMessage(input)); err != nil {
t.Errorf("numeric-string compatibility rejected: %v", err)
}
}
}
+3
View File
@@ -162,6 +162,9 @@ func DecorateTool(t actool.CoreTool, desc string, schema map[string]any) actool.
if len(schema) == 0 {
schema = t.InputSchema()
}
if t.Name() == "record_fact" {
schema = RepairRecordFactSchema(schema)
}
return &overriddenTool{CoreTool: t, desc: desc, schema: schema}
}
+22 -25
View File
@@ -1495,13 +1495,14 @@ func (t *ToolSet) addFinding() actool.CoreTool {
// must NOT be stuffed into the asset graph via upsert_asset.
// factItem is one fact of a record_fact batch/single call.
type factItem struct {
Summary string `json:"summary"`
Detail string `json:"detail"`
Evidence string `json:"evidence"` // one line of key evidence (command + key output lines) supporting the conclusion, for later verification
Confidence string `json:"confidence"` // observed (seen directly) | inferred (inferred from phenomena)
Source string `json:"source"` // where the content came from, one of factSources; empty = inference
IntentID json.RawMessage `json:"intent_id"`
AssetIDs []json.RawMessage `json:"asset_ids"`
Summary string `json:"summary"`
Detail string `json:"detail"`
Evidence string `json:"evidence"` // one line of key evidence (command + key output lines) supporting the conclusion, for later verification
Confidence string `json:"confidence"` // observed (seen directly) | inferred (inferred from phenomena)
Source string `json:"source"` // where the content came from, one of factSources; empty = inference
IntentID json.RawMessage `json:"intent_id"`
AssetIDs []json.RawMessage `json:"asset_ids"`
TestedAssetIDs []json.RawMessage `json:"tested_asset_ids"`
}
// Fact sources (#124, F4): where a fact's content came from, as declared by the
@@ -1537,12 +1538,16 @@ func normalizeFactSource(s string) (string, error) {
// intent_id).
func (t *ToolSet) recordOneFact(it factItem, defaultIntent int64) (int64, error) {
if strings.TrimSpace(it.Summary) == "" {
return 0, fmt.Errorf("summary must not be empty")
return 0, fmt.Errorf("summary must not be empty: every fact, including each batch item, requires its own one-sentence summary")
}
source, err := normalizeFactSource(it.Source)
if err != nil {
return 0, err
}
anchors, err := it.factAssetAnchors()
if err != nil {
return 0, err
}
payload := map[string]any{"summary": it.Summary, "source": source}
if it.Detail != "" {
payload["detail"] = it.Detail
@@ -1564,7 +1569,7 @@ func (t *ToolSet) recordOneFact(it factItem, defaultIntent int64) (int64, error)
}
}
// a fact is its OWN node kind (distinct from a vuln finding).
id, err := t.ts.AddNode(db.KindFact, payload, 5, "confirmed", t.worker, pidList(it.AssetIDs))
id, err := t.ts.AddNode(db.KindFact, payload, 5, "confirmed", t.worker, anchors)
if err != nil {
return 0, err
}
@@ -1584,23 +1589,12 @@ func (t *ToolSet) recordFact() actool.CoreTool {
" · confidence = observed (seen directly in output) | inferred (inferred from phenomena).\n"+
" · source = where the content came from: target_response (content the target itself returned: page bodies, headers, error messages, banners, even when a tool fetched it) | tool_output (a tool's own result about the target: port states, scanner verdicts, status codes, timings) | third_party_intel (OSINT, CVE databases, public reports) | operator_input (given by the human operator) | inference (your own reasoning; the default when omitted). Any other value is refused.\n"+
" · **Negative conclusions** (not injectable / port closed / no entry found, etc.) state only \"observation + tentative reading\" — state what you actually saw; whether to abandon the direction is for the planner to decide globally; always provide evidence, mark inferred when the means weren't exhausted or the evidence is weak (including probed only once, looks-like), and mark observed only when truly exhausted and directly seen.",
obj(map[string]any{
"facts": map[string]any{"type": "array", "description": "[use when there are several distinct conclusions] array of facts; element fields same as the top-level fields below (summary/detail/evidence/confidence/source/intent_id/asset_ids); omitting intent_id falls back to the top-level intent_id. The returned ids match this array in length and order.", "items": map[string]any{"type": "object"}},
"summary": str("a [one-sentence summary] of this exploration's conclusion (a condensation of detail)"),
"intent_id": idp("the id of the intent that produced this fact (the intent you claimed; the default for each item in batch mode)"),
"detail": str("details related to this fact: write all of this exploration's observations here"),
"evidence": str("[one line] of key evidence: the command + the one or two most proving output lines. Keep it brief; don't paste large output blocks (details go in detail)."),
"confidence": str("observed (seen directly in output) | inferred (inferred from phenomena). Always label negative conclusions honestly."),
"source": map[string]any{"type": "string", "enum": factSources, "description": "where the content came from: target_response | tool_output | third_party_intel | operator_input | inference (default when omitted)"},
"asset_ids": map[string]any{"type": "array", "items": map[string]any{"type": "integer"}, "description": "related asset ids (optional, 0/1/many): which assets this fact involves"},
}),
recordFactSchema(),
func(_ context.Context, in json.RawMessage) (actool.Result, error) {
var a struct {
Facts []factItem `json:"facts"`
factItem // single mode + batch default intent_id
a, batch, err := decodeRecordFactInput(in)
if err != nil {
return actool.Errorf("invalid record_fact input: " + err.Error() + "; no facts were written"), nil
}
_ = json.Unmarshal(in, &a)
batch := len(a.Facts) > 0
items := a.Facts
if !batch {
items = []factItem{a.factItem}
@@ -1627,8 +1621,11 @@ func (t *ToolSet) recordFact() actool.CoreTool {
out := map[string]any{"ids": ids}
if len(errs) > 0 {
out["errors"] = errs
out["retry_instruction"] = "Retry only failed indices with complete facts, including summary. Successful indices were already saved; do not retry them."
}
return jsonResult(out)
result, err := jsonResult(out)
result.IsError = len(errs) > 0
return result, err
})
}
+41
View File
@@ -230,6 +230,13 @@ func (s *ExplorationStore) AddNode(kind string, payload map[string]any, priority
return 0, err
}
defer tx.Rollback()
// Validate fact anchors before creating a node. Key-share locks retain the
// checked assets through commit, closing the check/insert deletion race.
if kind == KindFact && len(anchors) > 0 {
if err := checkFactAnchorAssets(tx, anchors); err != nil {
return 0, err
}
}
var id int64
if err := tx.QueryRow(`
INSERT INTO exploration_nodes(exploration_id, kind, payload, priority, state, origin)
@@ -245,6 +252,40 @@ VALUES ($1, $2, $3, $4, $5, $6) RETURNING id`,
return id, tx.Commit()
}
func checkFactAnchorAssets(tx *sql.Tx, ids []int64) error {
rows, err := tx.Query("SELECT id FROM assets WHERE id = ANY($1::bigint[]) ORDER BY id FOR KEY SHARE", ids)
if err != nil {
return err
}
found := make(map[int64]bool, len(ids))
for rows.Next() {
var id int64
if err := rows.Scan(&id); err != nil {
rows.Close()
return err
}
found[id] = true
}
err = rows.Err()
closeErr := rows.Close()
if err != nil {
return err
}
if closeErr != nil {
return closeErr
}
var missing []int64
for _, id := range ids {
if !found[id] {
missing = append(missing, id)
}
}
if len(missing) != 0 {
return fmt.Errorf("asset IDs do not exist: %v; use existing IDs or register the assets first", missing)
}
return nil
}
// AddIntent is a convenience: an open intent.
func (s *ExplorationStore) AddIntent(payload map[string]any, priority int, anchors []int64, origin string) (int64, error) {
if origin == "" {
+102 -7
View File
@@ -31,6 +31,7 @@ type DBFinding struct {
Worker string
AssetIDs []int64
Status string
ReportState string // none / submitted / accepted / rejected / duplicate; separate from Status
PoELevel int // proof-of-exploitation level 0-4 (S1); 0 = none declared
VerificationStatus string // verified / unverified / blocked / not_assessed / false_positive
VerificationNote string // why the status was reached; empty when verified
@@ -65,6 +66,27 @@ func ValidFindingStatus(s string) bool {
return false
}
// Finding report stages (findings.report_state). A workflow stage after triage,
// not a verdict: a finding can be confirmed and submitted at the same time.
// RSI lesson extraction still reads findings.status.
const (
FindingReportNone = "none"
FindingReportSubmitted = "submitted"
FindingReportAccepted = "accepted"
FindingReportRejected = "rejected"
FindingReportDuplicate = "duplicate"
)
// ValidFindingReportState reports whether s is a known report stage.
func ValidFindingReportState(s string) bool {
switch s {
case FindingReportNone, FindingReportSubmitted, FindingReportAccepted,
FindingReportRejected, FindingReportDuplicate:
return true
}
return false
}
// Finding severity levels (findings.severity).
const (
SeverityCritical = "critical" // Critical
@@ -109,7 +131,7 @@ func (d *DB) AddFinding(taskID, nodeID int64, vulnclass, name, severity, summary
// findingSelectCols is the column list (with task_description join) every finding
// list query selects, so scanFinding stays in sync across callers.
const findingSelectCols = `f.id, f.task_id, f.node_id, f.vulnclass, COALESCE(f.name, ''), f.severity, f.summary,
f.evidence, f.worker, f.asset_ids, COALESCE(f.status, 'pending'), f.created_at,
f.evidence, f.worker, f.asset_ids, COALESCE(f.status, 'pending'), COALESCE(f.report_state, 'none'), f.created_at,
COALESCE(t.description, '') AS task_description, f.evidence_version, f.report_evidence_version,
f.poe_level, COALESCE(f.verification_status, 'unverified'), COALESCE(f.verification_note, ''),
f.counterevidence, f.confidence_rationale,
@@ -126,7 +148,7 @@ func scanFindings(rows interface {
f := &DBFinding{}
var aidsJSON string
if err := rows.Scan(&f.ID, &f.TaskID, &f.NodeID, &f.VulnClass, &f.Name, &f.Severity,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt, &f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.Counterevidence, &f.ConfidenceRationale, &f.TrafficCount); err != nil {
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.ReportState, &f.CreatedAt, &f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.Counterevidence, &f.ConfidenceRationale, &f.TrafficCount); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(aidsJSON), &f.AssetIDs)
@@ -160,6 +182,7 @@ func (d *DB) ListFindings(limit int) ([]*DBFinding, error) {
type FindingFilter struct {
Severity string // high | medium | low
Status string // pending | false_positive | ignored | resolved
ReportState string // none | submitted | accepted | rejected | duplicate
VerificationStatus string // verified | unverified | blocked | not_assessed | false_positive
VulnClass string
TaskID string // task id (in string form; empty/invalid = do not filter by task)
@@ -197,6 +220,7 @@ func (f FindingFilter) where() (string, []any) {
}
add("severity", f.Severity)
add("status", f.Status)
add("report_state", f.ReportState)
add("verification_status", f.VerificationStatus)
add("vulnclass", f.VulnClass)
// task_id is a bigint column, compared as an integer (it cannot go through
@@ -528,7 +552,7 @@ func (d *DB) ListFindingsForExport(f FindingFilter, ids []int64) ([]*DBFinding,
f := &DBFinding{}
var aidsJSON string
if err := rows.Scan(&f.ID, &f.TaskID, &f.NodeID, &f.VulnClass, &f.Name, &f.Severity,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.ReportState, &f.CreatedAt,
&f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.Counterevidence, &f.ConfidenceRationale, &f.TrafficCount, &f.Report); err != nil {
return nil, err
}
@@ -544,6 +568,8 @@ func (d *DB) ListFindingsForExport(f FindingFilter, ids []int64) ([]*DBFinding,
type FindingStats struct {
Total int `json:"total"`
Pending int `json:"pending"`
Submitted int `json:"submitted"`
Accepted int `json:"accepted"`
Critical int `json:"critical"`
High int `json:"high"`
Medium int `json:"medium"`
@@ -563,18 +589,20 @@ type FindingTaskOption struct {
Count int `json:"count"`
}
// FindingStats returns whole-table counts (by severity + pending) and the sorted
// set of distinct vuln classes.
// FindingStats returns whole-table counts (by severity, pending, and report
// stage) and the sorted set of distinct vuln classes.
func (d *DB) FindingStats() (*FindingStats, error) {
st := &FindingStats{VulnClasses: []string{}, Tasks: []FindingTaskOption{}}
err := d.QueryRow(`SELECT
COUNT(*),
COUNT(*) FILTER (WHERE status = 'pending'),
COUNT(*) FILTER (WHERE report_state = 'submitted'),
COUNT(*) FILTER (WHERE report_state = 'accepted'),
COUNT(*) FILTER (WHERE severity = 'critical'),
COUNT(*) FILTER (WHERE severity = 'high'),
COUNT(*) FILTER (WHERE severity = 'medium'),
COUNT(*) FILTER (WHERE severity = 'low')
FROM findings`).Scan(&st.Total, &st.Pending, &st.Critical, &st.High, &st.Medium, &st.Low)
FROM findings`).Scan(&st.Total, &st.Pending, &st.Submitted, &st.Accepted, &st.Critical, &st.High, &st.Medium, &st.Low)
if err != nil {
return nil, err
}
@@ -629,6 +657,8 @@ func (d *DB) FindingStats() (*FindingStats, error) {
cold := aggregate.FindingStats
st.Total += cold.Total
st.Pending += cold.Pending
st.Submitted += cold.Submitted
st.Accepted += cold.Accepted
st.Critical += cold.Critical
st.High += cold.High
st.Medium += cold.Medium
@@ -658,7 +688,7 @@ func (d *DB) GetFinding(id int64) (*DBFinding, error) {
LEFT JOIN tasks t ON f.task_id = t.id
WHERE f.id = $1`, id).Scan(
&f.ID, &f.TaskID, &f.NodeID, &f.VulnClass, &f.Name, &f.Severity,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.ReportState, &f.CreatedAt,
&f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.Counterevidence, &f.ConfidenceRationale, &f.TrafficCount, &f.Report)
if err == sql.ErrNoRows {
return nil, nil
@@ -723,6 +753,71 @@ func (d *DB) SetFindingStatus(id int64, status string) (int64, error) {
return d.setFindingCol(id, "status", "status", status)
}
// SetFindingReportState updates one finding's report stage and mirrors it onto
// the originating exploration node's payload under "report_state". It does not
// register a status-change notification: report_state is a workflow stage, not
// a triage verdict. Returns rows affected.
func (d *DB) SetFindingReportState(id int64, state string) (int64, error) {
return d.setFindingCol(id, "report_state", "report_state", state)
}
// FindingReportStateBatchResult is one id's outcome in a bulk report-stage update.
type FindingReportStateBatchResult struct {
ID int64 `json:"id"`
From string `json:"from,omitempty"`
Found bool `json:"found"`
Changed bool `json:"changed"`
}
// SetFindingReportStates updates several findings to the same report stage in
// one transaction. Unknown ids come back as found=false instead of hiding the
// others; a database error rolls the whole batch back. No notification event
// is written — status-change pushes stay on findings.status.
func (d *DB) SetFindingReportStates(ctx context.Context, ids []int64, state string) ([]FindingReportStateBatchResult, error) {
out := make([]FindingReportStateBatchResult, 0, len(ids))
err := d.WithEvidenceTx(ctx, func(tx *sql.Tx) error {
for _, id := range ids {
from, found, changed, err := setFindingReportStateTx(ctx, tx, id, state)
if err != nil {
return err
}
out = append(out, FindingReportStateBatchResult{ID: id, From: from, Found: found, Changed: changed})
}
return nil
})
if err != nil {
return nil, err
}
return out, nil
}
func setFindingReportStateTx(ctx context.Context, tx *sql.Tx, id int64, state string) (from string, found bool, changed bool, err error) {
var nodeID sql.NullInt64
scanErr := tx.QueryRowContext(ctx, `SELECT COALESCE(report_state,'none'), node_id
FROM findings WHERE id=$1 FOR UPDATE`, id).Scan(&from, &nodeID)
if scanErr == sql.ErrNoRows {
return "", false, false, nil
}
if scanErr != nil {
return "", false, false, scanErr
}
found = true
if from == state {
return from, true, false, nil
}
if _, err := tx.ExecContext(ctx, `UPDATE findings SET report_state=$2 WHERE id=$1`, id, state); err != nil {
return from, true, false, err
}
if nodeID.Valid {
if _, err := tx.ExecContext(ctx, `UPDATE exploration_nodes
SET payload = jsonb_set(payload, '{report_state}', to_jsonb($1::text))
WHERE id = $2`, state, nodeID.Int64); err != nil {
return from, true, false, err
}
}
return from, true, true, nil
}
// SetFindingReportByNodeID sets the Markdown report on the standalone finding row
// whose node_id matches — report_finding returns that node id, so an agent tool
// can address the finding it just created. Returns rows affected (0 when no row).
+131
View File
@@ -13,6 +13,25 @@ import (
// TestDeleteFinding verifies that deleting a finding removes both the findings
// row and its originating exploration node (kind='finding').
func TestValidFindingReportState(t *testing.T) {
for _, s := range []string{
FindingReportNone, FindingReportSubmitted, FindingReportAccepted,
FindingReportRejected, FindingReportDuplicate,
} {
if !ValidFindingReportState(s) {
t.Fatalf("%q should be valid", s)
}
}
for _, s := range []string{"", "confirmed", "pending", "SUBMITTED", "submitted "} {
if ValidFindingReportState(s) {
t.Fatalf("%q should be invalid", s)
}
}
if ValidFindingStatus(FindingReportSubmitted) {
t.Fatal("submitted must not be a triage status")
}
}
func TestDeleteFinding(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
@@ -122,6 +141,115 @@ func TestSetFindingStatusMirrorsNodePayload(t *testing.T) {
}
}
func TestFindingReportState(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Skipf("postgres unavailable (%v) — skipping", err)
}
defer d.Close()
tk, err := d.CreateTask("finding report state", "goal", nil, 0, 0)
if err != nil {
t.Fatal(err)
}
defer d.DeleteTask(tk.ID)
es := d.Exploration(tk.ExplorationID)
nodeID, err := es.AddNode(KindFinding, map[string]any{"summary": "x", "severity": "high"}, 5, "confirmed", "worker", nil)
if err != nil {
t.Fatal(err)
}
fid, err := d.AddFinding(tk.ID, nodeID, "XSS", "reflected XSS", "high", "summary", "poc", "worker", nil)
if err != nil {
t.Fatal(err)
}
second, err := d.AddFinding(tk.ID, 0, "XSS", "other", "low", "summary", "poc", "worker", nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { d.Exec(`DELETE FROM notification_events WHERE finding_id IN ($1,$2)`, fid, second) })
one, err := d.GetFinding(fid)
if err != nil || one == nil || one.ReportState != FindingReportNone {
t.Fatalf("new finding report_state: %+v err=%v", one, err)
}
n, err := d.SetFindingReportState(fid, FindingReportSubmitted)
if err != nil || n != 1 {
t.Fatalf("SetFindingReportState: n=%d err=%v", n, err)
}
one, _ = d.GetFinding(fid)
if one.ReportState != FindingReportSubmitted {
t.Fatalf("report_state=%q", one.ReportState)
}
if one.Status != FindingPending {
t.Fatalf("triage status changed: %q", one.Status)
}
var payload []byte
if err := d.QueryRow(`SELECT payload FROM exploration_nodes WHERE id=$1`, nodeID).Scan(&payload); err != nil {
t.Fatal(err)
}
var p map[string]any
if err := json.Unmarshal(payload, &p); err != nil {
t.Fatal(err)
}
if p["report_state"] != FindingReportSubmitted {
t.Fatalf("payload report_state=%v", p["report_state"])
}
listed, total, err := d.ListFindingsPage(FindingFilter{TaskID: fmt.Sprintf("%d", tk.ID), ReportState: FindingReportSubmitted}, 1, 20)
if err != nil || total != 1 || len(listed) != 1 || listed[0].ID != fid {
t.Fatalf("filter submitted: total=%d listed=%d err=%v", total, len(listed), err)
}
st, err := d.FindingStats()
if err != nil || st.Submitted < 1 {
t.Fatalf("stats submitted undercount: %+v err=%v", st, err)
}
const missingID = int64(999999999)
results, err := d.SetFindingReportStates(context.Background(), []int64{fid, second, missingID}, FindingReportAccepted)
if err != nil {
t.Fatal(err)
}
if len(results) != 3 {
t.Fatalf("want 3 results, got %d", len(results))
}
if !results[0].Found || !results[0].Changed || results[0].From != FindingReportSubmitted {
t.Fatalf("first: %+v", results[0])
}
if !results[1].Found || !results[1].Changed || results[1].From != FindingReportNone {
t.Fatalf("second: %+v", results[1])
}
if results[2].Found || results[2].Changed {
t.Fatalf("missing: %+v", results[2])
}
again, err := d.SetFindingReportStates(context.Background(), []int64{fid}, FindingReportAccepted)
if err != nil || len(again) != 1 || again[0].Changed {
t.Fatalf("noop batch: %v %v", again, err)
}
var events int
if err := d.QueryRow(`SELECT count(*) FROM notification_events WHERE finding_id IN ($1,$2)`, fid, second).Scan(&events); err != nil || events != 0 {
t.Fatalf("report_state must not write notification_events: %d (%v)", events, err)
}
one, _ = d.GetFinding(fid)
if one.Status != FindingPending {
t.Fatalf("bulk report_state mutated triage: %q", one.Status)
}
if one.ReportState != FindingReportAccepted {
t.Fatalf("bulk report_state=%q", one.ReportState)
}
st, err = d.FindingStats()
if err != nil || st.Accepted < 2 {
t.Fatalf("stats accepted undercount: %+v err=%v", st, err)
}
}
// TestFindingsPageAndStats exercises ListFindingsPage (filter/sort/paging) and
// FindingStats against the live dev PG. It tags its rows with a unique vulnclass
// so assertions are isolated from any pre-existing data, and cleans up after.
@@ -226,6 +354,9 @@ func TestFindingsPageAndStats(t *testing.T) {
if one == nil || one.ID != ids[0] || one.Severity != "critical" || one.Name != "Critical Vulnerability Title" {
t.Fatalf("GetFinding mismatch: %+v", one)
}
if one.ReportState != FindingReportNone {
t.Fatalf("new finding report_state should be none, got %q", one.ReportState)
}
if one.Report != "" {
t.Fatalf("new finding report should be empty, got %q", one.Report)
}
+2 -2
View File
@@ -137,9 +137,9 @@ func assertEmbeddedApplied(t *testing.T, d *DB) []appliedMigration {
version int
name string
}{{1, llmRecordsMigration}, {2, llmUsageMigration}, {3, "0003_task_scope_icp_index.sql"}, {4, "0004_llm_usage_ts_index.sql"},
{5, "0005_traffic_rewrite_rules.sql"}, {6, "0006_activity_outbox.sql"}}
{5, "0005_traffic_rewrite_rules.sql"}, {6, "0006_activity_outbox.sql"}, {7, "0007_finding_report_state.sql"}}
if len(applied) != len(want) {
t.Fatalf("schema_migrations = %+v, want versions 1 to 6", applied)
t.Fatalf("schema_migrations = %+v, want versions 1 to 7", applied)
}
for i, w := range want {
content, err := fs.ReadFile(migrationsFS(), w.name)
@@ -0,0 +1,5 @@
-- Report stage on findings, separate from triage status. Existing rows stay
-- 'none'. Values are validated server-side (no CHECK): none / submitted /
-- accepted / rejected / duplicate. RSI lessons still read findings.status.
ALTER TABLE findings ADD COLUMN IF NOT EXISTS report_state TEXT NOT NULL DEFAULT 'none';
CREATE INDEX IF NOT EXISTS idx_findings_report_state ON findings (report_state, created_at DESC);
+5
View File
@@ -1256,6 +1256,11 @@ ALTER TABLE findings ADD COLUMN IF NOT EXISTS verification_note TEXT NOT NULL DE
-- tested and did not work; confidence_rationale is why the conclusion holds.
ALTER TABLE findings ADD COLUMN IF NOT EXISTS counterevidence TEXT NOT NULL DEFAULT '';
ALTER TABLE findings ADD COLUMN IF NOT EXISTS confidence_rationale TEXT NOT NULL DEFAULT '';
-- Report stage after triage (none / submitted / accepted / rejected / duplicate).
-- Separate from status so a finding can be confirmed and submitted at once; RSI
-- lessons still read status. Server whitelist, no CHECK (same as status).
ALTER TABLE findings ADD COLUMN IF NOT EXISTS report_state TEXT NOT NULL DEFAULT 'none';
CREATE INDEX IF NOT EXISTS idx_findings_report_state ON findings (report_state, created_at DESC);
CREATE TABLE IF NOT EXISTS traffic_evidence_snapshots (
id TEXT PRIMARY KEY,
+2
View File
@@ -770,6 +770,8 @@ FROM skill_usage WHERE (task_id=$1 OR exploration_id=$2) AND NOT found GROUP BY
{"findings", `SELECT COALESCE(jsonb_object_agg(name,n),'{}'::jsonb) FROM (SELECT COALESCE(NULLIF(severity,''),'unknown') name,count(*) n FROM findings WHERE task_id=$1 GROUP BY severity) x`, []any{taskID}},
{"finding_stats", `SELECT jsonb_build_object(
'total',count(*),'pending',count(*) FILTER (WHERE status='pending'),
'submitted',count(*) FILTER (WHERE report_state='submitted'),
'accepted',count(*) FILTER (WHERE report_state='accepted'),
'critical',count(*) FILTER (WHERE severity='critical'),'high',count(*) FILTER (WHERE severity='high'),
'medium',count(*) FILTER (WHERE severity='medium'),'low',count(*) FILTER (WHERE severity='low'),
'vulnclasses',COALESCE(jsonb_agg(DISTINCT vulnclass) FILTER (WHERE vulnclass<>''),'[]'::jsonb))
+32 -20
View File
@@ -361,28 +361,40 @@ func rowExists(tx *sql.Tx, table string, id int64) bool {
return exists
}
func insertArchiveRows(tx *sql.Tx, table string, raw json.RawMessage) error {
// json_populate_recordset inserts NULL for absent columns, bypassing SQL
// defaults. Preserve compatibility with v3 archives predating side memory.
if (table == "side_question_sessions" || table == "side_question_requests") && len(raw) > 0 {
var rows []map[string]json.RawMessage
if err := json.Unmarshal(raw, &rows); err != nil {
return err
}
field := "memory"
if table == "side_question_requests" {
field = "context_info"
}
for _, row := range rows {
// archiveDefaults are the columns that did not exist when older archives were
// written. json_populate_recordset inserts NULL for an absent key, bypassing the
// SQL default, and a NOT NULL column then rejects the whole restore.
var archiveDefaults = map[string]map[string]string{
"side_question_sessions": {"memory": `{}`},
"side_question_requests": {"context_info": `{}`},
"findings": {"report_state": `"none"`},
}
// fillArchiveDefaults restores the default of a column an older archive
// predates, so a restore keeps working across schema additions.
func fillArchiveDefaults(table string, raw json.RawMessage) (json.RawMessage, error) {
defaults, ok := archiveDefaults[table]
if !ok || len(raw) == 0 {
return raw, nil
}
var rows []map[string]json.RawMessage
if err := json.Unmarshal(raw, &rows); err != nil {
return nil, err
}
for _, row := range rows {
for field, def := range defaults {
if len(row[field]) == 0 || string(row[field]) == "null" {
row[field] = json.RawMessage(`{}`)
row[field] = json.RawMessage(def)
}
}
var err error
raw, err = json.Marshal(rows)
if err != nil {
return err
}
}
return json.Marshal(rows)
}
func insertArchiveRows(tx *sql.Tx, table string, raw json.RawMessage) error {
raw, err := fillArchiveDefaults(table, raw)
if err != nil {
return err
}
allowed := map[string]bool{
"exploration_nodes": true, "exploration_edges": true, "exploration_anchors": true,
@@ -396,7 +408,7 @@ func insertArchiveRows(tx *sql.Tx, table string, raw json.RawMessage) error {
if rawRowCount(raw) == 0 {
return nil
}
_, err := tx.Exec(`INSERT INTO `+table+` SELECT * FROM json_populate_recordset(NULL::`+table+`,$1::json)`, string(raw))
_, err = tx.Exec(`INSERT INTO `+table+` SELECT * FROM json_populate_recordset(NULL::`+table+`,$1::json)`, string(raw))
return err
}
+31
View File
@@ -486,3 +486,34 @@ func assertArchiveGlobalStats(t *testing.T, d *DB, profileName, skillName, toolN
t.Fatalf("archive finding class %q missing", vulnclass)
}
}
// An archive written before report_state existed restores with the column's
// default instead of failing on the NOT NULL constraint.
func TestFillArchiveDefaults(t *testing.T) {
raw := json.RawMessage(`[{"id":1,"summary":"old"},{"id":2,"report_state":null},{"id":3,"report_state":"submitted"}]`)
out, err := fillArchiveDefaults("findings", raw)
if err != nil {
t.Fatal(err)
}
var rows []map[string]json.RawMessage
if err := json.Unmarshal(out, &rows); err != nil {
t.Fatal(err)
}
want := []string{`"none"`, `"none"`, `"submitted"`}
for i, row := range rows {
if string(row["report_state"]) != want[i] {
t.Fatalf("row %d report_state=%s, want %s", i, row["report_state"], want[i])
}
}
// Side memory keeps its own default, and other tables pass through.
out, err = fillArchiveDefaults("side_question_sessions", json.RawMessage(`[{"id":1}]`))
if err != nil || !strings.Contains(string(out), `"memory":{}`) {
t.Fatalf("side memory default missing: %s err=%v", out, err)
}
in := json.RawMessage(`[{"id":1}]`)
out, err = fillArchiveDefaults("activity", in)
if err != nil || string(out) != string(in) {
t.Fatalf("activity changed: %s err=%v", out, err)
}
}
+42
View File
@@ -10,6 +10,7 @@
package llmpool
import (
"context"
"sync"
"time"
)
@@ -159,6 +160,47 @@ func (r *Registry) Trip(id int64, errMsg string, hard bool) (tripped bool) {
return tripped
}
// RecordCall applies the failover health policy to one provider call. The Pool
// calls it for every member attempt, and a caller that runs a provider directly
// (a task whose selection bypasses the Pool) calls it too, so a profile's state
// does not depend on which path the request took.
//
// stream tells a streaming call from an atomic one; committed says whether any
// event already reached the caller (streaming only). The rules are the ones the
// Pool applies:
//
// - a completed call (err == nil) clears the state with Pass;
// - a failure that does not trigger failover (context cancellation, a
// deterministic 4xx) is ignored on a stream, and clears the state on an
// atomic call, matching Complete's non-failover path;
// - a failover-eligible stream failure trips only before the first event:
// after that the caller already has output and the state is left alone;
// - a failover-eligible atomic failure trips.
//
// id 0 is a no-op (no profile to key, e.g. an environment-only configuration).
// It returns whether this call opened the breaker, and the shortened error
// stored with it.
func (r *Registry) RecordCall(ctx context.Context, id int64, stream, committed bool, err error) (tripped bool, errMsg string) {
if id == 0 {
return false, ""
}
if err == nil {
r.Pass(id)
return false, ""
}
if !shouldFailover(ctx, err) {
if !stream {
r.Pass(id)
}
return false, ""
}
if stream && committed {
return false, ""
}
msg := trimErr(err)
return r.Trip(id, msg, isHardFailure(err)), msg
}
// Reset clears one profile's state — the UI's "Recover now" action.
func (r *Registry) Reset(id int64) {
r.mu.Lock()
+119
View File
@@ -0,0 +1,119 @@
package llmpool
import (
"context"
"fmt"
"testing"
"github.com/Autumn-27/norma/llm"
)
// RecordCall is the policy a provider call applies when no Pool wraps it. A
// completed call clears the state; a failover-eligible failure trips; a
// non-failover failure neither trips nor, on a stream, clears.
func TestRecordCallPolicy(t *testing.T) {
ctx := context.Background()
soft := fmt.Errorf("openai: status 500: down")
bad := fmt.Errorf("openai: status 400: bad request")
seed := func(t *testing.T) *Registry {
t.Helper()
reg := NewRegistry(nil, nil)
reg.RecordCall(ctx, 7, false, false, soft)
if st := reg.Get(7); st.Fails != 1 || st.Trips != 0 {
t.Fatalf("seed state=%+v, want one soft failure", st)
}
return reg
}
t.Run("success clears", func(t *testing.T) {
reg := seed(t)
reg.RecordCall(ctx, 7, false, false, nil)
if st := reg.Get(7); st.Fails != 0 || st.Trips != 0 || st.Open() {
t.Fatalf("state=%+v after a success, want clean", st)
}
})
t.Run("atomic non-failover clears", func(t *testing.T) {
reg := seed(t)
reg.RecordCall(ctx, 7, false, false, bad)
if st := reg.Get(7); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("state=%+v after an atomic 400, want clean (Pool.Complete policy)", st)
}
})
t.Run("stream non-failover keeps", func(t *testing.T) {
reg := seed(t)
reg.RecordCall(ctx, 7, true, false, bad)
if st := reg.Get(7); st.Fails != 1 || st.Trips != 0 {
t.Fatalf("state=%+v after a stream 400, want the failure kept", st)
}
})
t.Run("stream cancel keeps", func(t *testing.T) {
reg := seed(t)
reg.RecordCall(ctx, 7, true, false, context.Canceled)
if st := reg.Get(7); st.Fails != 1 || st.Trips != 0 {
t.Fatalf("state=%+v after a cancel, want the failure kept", st)
}
})
t.Run("committed stream failure keeps", func(t *testing.T) {
reg := seed(t)
reg.RecordCall(ctx, 7, true, true, soft)
if st := reg.Get(7); st.Fails != 1 || st.Trips != 0 {
t.Fatalf("state=%+v after a committed failure, want it ignored", st)
}
})
t.Run("pre-commit stream failure counts", func(t *testing.T) {
reg := NewRegistry(nil, nil)
reg.RecordCall(ctx, 7, true, false, soft)
if st := reg.Get(7); st.Fails != 1 || st.Trips != 0 || st.Open() {
t.Fatalf("state=%+v, want one soft failure", st)
}
})
t.Run("hard failure trips at once", func(t *testing.T) {
reg := NewRegistry(nil, nil)
reg.RecordCall(ctx, 7, false, false, fmt.Errorf("openai: status 401: bad key"))
if st := reg.Get(7); st.Trips != 1 || !st.Open() {
t.Fatalf("state=%+v, want an open breaker after one hard failure", st)
}
})
t.Run("profile 0 is a no-op", func(t *testing.T) {
reg := NewRegistry(nil, nil)
reg.RecordCall(ctx, 0, false, false, soft)
if n := len(reg.Snapshot()); n != 0 {
t.Fatalf("%d states recorded for profile 0", n)
}
})
}
// The Pool feeds the same policy: a failure before the first event counts on
// the member that failed, a failure after output reached the caller does not,
// because the Pool neither fails over nor blames the profile in that case.
func TestPoolFeedsRecordCallPolicy(t *testing.T) {
t.Run("pre-commit failure counts", func(t *testing.T) {
a := &fakeProv{name: "a", errs: []error{fmt.Errorf("status 500: down")}}
reg := NewRegistry(nil, nil)
p := New([]*Member{member(1, "a", 10, a), member(2, "b", 5, okProv("b", "full"))}, reg)
if _, err := drain(p.Stream(context.Background(), llm.CompletionRequest{})); err != nil {
t.Fatalf("stream error: %v", err)
}
if st := reg.Get(1); st.Fails != 1 || st.Trips != 0 {
t.Fatalf("state=%+v after a pre-commit failure, want one soft failure", st)
}
if st := reg.Get(2); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("state=%+v on the backup member, want clean", st)
}
})
t.Run("committed failure is ignored", func(t *testing.T) {
a := &fakeProv{
name: "a",
events: [][]llm.StreamEvent{{{Type: llm.SETextDelta, Text: "partial"}}},
errs: []error{fmt.Errorf("status 500: mid-stream drop")},
}
reg := NewRegistry(nil, nil)
p := New([]*Member{member(1, "a", 10, a), member(2, "b", 5, okProv("b", "full"))}, reg)
if _, err := drain(p.Stream(context.Background(), llm.CompletionRequest{})); err == nil {
t.Fatal("mid-stream error should surface")
}
if st := reg.Get(1); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("state=%+v after a committed failure, want clean", st)
}
})
}
+9 -10
View File
@@ -100,13 +100,12 @@ func (p *Pool) Stream(ctx context.Context, req llm.CompletionRequest) iter.Seq2[
}
}
if failed == nil {
p.health.Pass(m.ID) // completed (or failed in a non-failover way)
p.health.RecordCall(ctx, m.ID, true, false, nil) // completed: Pass
return
}
lastErr = failed
hard := isHardFailure(failed)
if p.health.Trip(m.ID, trimErr(failed), hard) {
log.Printf("[llmpool] profile %q(%s) breaker tripped: %s", m.Name, m.Model, trimErr(failed))
if tripped, msg := p.health.RecordCall(ctx, m.ID, true, false, failed); tripped {
log.Printf("[llmpool] profile %q(%s) breaker tripped: %s", m.Name, m.Model, msg)
}
if i+1 < len(order) {
n := order[i+1]
@@ -133,19 +132,19 @@ func (p *Pool) Complete(ctx context.Context, req llm.CompletionRequest) (llm.Mes
for i, m := range order {
msg, sr, usage, err := m.Prov.Complete(ctx, req)
if err == nil {
p.health.Pass(m.ID)
p.health.RecordCall(ctx, m.ID, false, false, nil) // Pass
return msg, sr, usage, nil
}
if !shouldFailover(ctx, err) {
// Non-failover error (e.g. ctx cancel, deterministic 4xx): surface as-is
// without tripping health, matching Stream's non-failover path.
p.health.Pass(m.ID)
// without tripping health. RecordCall clears the state on an atomic
// call, matching the historical Pass here.
p.health.RecordCall(ctx, m.ID, false, false, err)
return llm.Message{}, "", llm.Usage{}, err
}
lastErr = err
hard := isHardFailure(err)
if p.health.Trip(m.ID, trimErr(err), hard) {
log.Printf("[llmpool] profile %q(%s) breaker tripped: %s", m.Name, m.Model, trimErr(err))
if tripped, msg := p.health.RecordCall(ctx, m.ID, false, false, err); tripped {
log.Printf("[llmpool] profile %q(%s) breaker tripped: %s", m.Name, m.Model, msg)
}
if i+1 < len(order) {
n := order[i+1]
+4 -1
View File
@@ -70,6 +70,7 @@ func FindingsMarkdown(fs []*db.DBFinding, generatedAt time.Time) string {
fmt.Fprintf(&b, "- **Class**: %s\n", f.VulnClass)
}
fmt.Fprintf(&b, "- **Status**: %s\n", nz(f.Status, "pending"))
fmt.Fprintf(&b, "- **Report stage**: %s\n", nz(f.ReportState, "none"))
if desc := strings.TrimSpace(f.TaskDescription); desc != "" {
fmt.Fprintf(&b, "- **Task**: %s\n", desc)
}
@@ -99,6 +100,7 @@ func SingleFindingMarkdown(f *db.DBFinding, generatedAt time.Time) string {
}
fmt.Fprintf(&b, "- **Severity**: %s\n", nz(f.Severity, "info"))
fmt.Fprintf(&b, "- **Status**: %s\n", nz(f.Status, "pending"))
fmt.Fprintf(&b, "- **Report stage**: %s\n", nz(f.ReportState, "none"))
if desc := strings.TrimSpace(f.TaskDescription); desc != "" {
fmt.Fprintf(&b, "- **Task**: %s\n", desc)
}
@@ -148,7 +150,7 @@ func FindingsCSV(fs []*db.DBFinding) []byte {
var buf bytes.Buffer
buf.WriteString("\xEF\xBB\xBF") // UTF-8 BOM
w := csv.NewWriter(&buf)
_ = w.Write([]string{"ID", "Name", "Class", "Severity", "Status", "Task", "Found At", "Summary", "Traffic Evidence Count", "Traffic Evidence IDs"})
_ = w.Write([]string{"ID", "Name", "Class", "Severity", "Status", "Report Stage", "Task", "Found At", "Summary", "Traffic Evidence Count", "Traffic Evidence IDs"})
for _, f := range items {
_ = w.Write([]string{
fmt.Sprintf("%d", f.ID),
@@ -156,6 +158,7 @@ func FindingsCSV(fs []*db.DBFinding) []byte {
f.VulnClass,
nz(f.Severity, "info"),
nz(f.Status, "pending"),
nz(f.ReportState, "none"),
f.TaskDescription,
f.CreatedAt.Format("2006-01-02 15:04:05"),
strings.TrimSpace(f.Summary),
+34
View File
@@ -32,3 +32,37 @@ func TestBulkFindingStatusValidation(t *testing.T) {
})
}
}
func TestNormalizeFindingIDs(t *testing.T) {
got := normalizeFindingIDs([]int64{0, -1, 2, 2, 3, 4}, 2)
if len(got) != 2 || got[0] != 2 || got[1] != 3 {
t.Fatalf("got %v, want [2 3]", got)
}
if got := normalizeFindingIDs(nil, 500); len(got) != 0 {
t.Fatalf("empty input: %v", got)
}
}
func TestBulkFindingReportStateValidation(t *testing.T) {
s := &Server{m: &Manager{}}
cases := []struct {
name string
body string
want int
}{
{"bad json", `{`, http.StatusBadRequest},
{"unknown state", `{"ids":[1],"report_state":"confirmed"}`, http.StatusBadRequest},
{"empty ids", `{"ids":[],"report_state":"submitted"}`, http.StatusBadRequest},
{"non-positive ids", `{"ids":[0,-3],"report_state":"submitted"}`, http.StatusBadRequest},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/api/exploration/findings/bulk-report-state", strings.NewReader(tc.body))
s.bulkPatchFindingReportState(rec, req)
if rec.Code != tc.want {
t.Fatalf("status %d, want %d: %s", rec.Code, tc.want, rec.Body.String())
}
})
}
}
+23 -17
View File
@@ -327,15 +327,16 @@ type FindingDTO struct {
Counterevidence string `json:"counterevidence,omitempty"` // what was tested and did not work; optional
ConfidenceRationale string `json:"confidence_rationale,omitempty"` // why the conclusion holds; optional
ID string `json:"id"`
FindingID string `json:"finding_id,omitempty"` // standalone findings-table id — the handle for status updates
VulnClass string `json:"vulnclass"`
Name string `json:"name,omitempty"` // vulnerability name; the frontend falls back to displaying vulnclass when empty
Severity string `json:"severity"` // critical | high | medium | low
Status string `json:"status"` // pending | in_progress | confirmed | resolved | fixed | false_positive | ignored | duplicate | risk_accepted
Summary string `json:"summary"`
Evidence string `json:"evidence"`
Report string `json:"report,omitempty"` // detailed report (Markdown); returned only by the detail endpoint, empty in lists
ID string `json:"id"`
FindingID string `json:"finding_id,omitempty"` // standalone findings-table id — the handle for status updates
VulnClass string `json:"vulnclass"`
Name string `json:"name,omitempty"` // vulnerability name; the frontend falls back to displaying vulnclass when empty
Severity string `json:"severity"` // critical | high | medium | low
Status string `json:"status"` // pending | in_progress | confirmed | resolved | fixed | false_positive | ignored | duplicate | risk_accepted
ReportState string `json:"report_state"` // none | submitted | accepted | rejected | duplicate; workflow stage, not a triage verdict
Summary string `json:"summary"`
Evidence string `json:"evidence"`
Report string `json:"report,omitempty"` // detailed report (Markdown); returned only by the detail endpoint, empty in lists
IntentID string `json:"intent_id,omitempty"`
ParamID string `json:"param_id,omitempty"`
@@ -461,6 +462,10 @@ func findingFromDB(f *db.DBFinding, assets map[int64]*db.Asset) FindingDTO {
if status == "" {
status = db.FindingPending
}
reportState := f.ReportState
if reportState == "" {
reportState = db.FindingReportNone
}
d := FindingDTO{
ID: i64s(f.ID),
FindingID: i64s(f.ID),
@@ -468,14 +473,15 @@ func findingFromDB(f *db.DBFinding, assets map[int64]*db.Asset) FindingDTO {
ReportStale: f.Report != "" && f.EvidenceVersion != f.ReportEvidenceVersion, TrafficBindings: f.TrafficBindings,
PoELevel: f.PoELevel, VerificationStatus: f.VerificationStatus, VerificationNote: f.VerificationNote,
Counterevidence: f.Counterevidence, ConfidenceRationale: f.ConfidenceRationale,
VulnClass: f.VulnClass,
Name: f.Name,
Severity: f.Severity,
Status: status,
Summary: f.Summary,
Evidence: f.Evidence,
Report: f.Report,
TS: rfc3339(f.CreatedAt),
VulnClass: f.VulnClass,
Name: f.Name,
Severity: f.Severity,
Status: status,
ReportState: reportState,
Summary: f.Summary,
Evidence: f.Evidence,
Report: f.Report,
TS: rfc3339(f.CreatedAt),
}
d.Assets = findingAssetDTOs(f.AssetIDs, assets)
if f.TaskID != nil {
+38
View File
@@ -0,0 +1,38 @@
package server
import (
"encoding/json"
"log"
"github.com/Autumn-27/artex/agent"
)
// SeedTool only inserts new rows. Repair existing built-ins too, while keeping
// descriptions, bindings, enabled/deferred flags and parameter annotations.
func (s *Server) seedFactInputSchema() {
row, err := s.m.pg.GetTool("record_fact")
if err != nil {
log.Printf("[facts] read record_fact schema: %v", err)
return
}
if row == nil || !row.System || row.Kind != "builtin" {
return
}
var schema map[string]any
if err := json.Unmarshal(row.Schema, &schema); err != nil {
log.Printf("[facts] decode record_fact schema: %v", err)
return
}
raw, err := json.Marshal(agent.RepairRecordFactSchema(schema))
if err != nil {
log.Printf("[facts] encode record_fact schema: %v", err)
return
}
// Compare-and-swap preserves concurrent operator edits; JSONB equality
// avoids changing updated_at again on every startup.
if _, err := s.m.pg.Exec(`UPDATE tools SET schema=$2::jsonb, updated_at=now()
WHERE key='record_fact' AND system AND kind='builtin'
AND schema=$1::jsonb AND schema IS DISTINCT FROM $2::jsonb`, string(row.Schema), string(raw)); err != nil {
log.Printf("[facts] repair record_fact schema: %v", err)
}
}
+106
View File
@@ -0,0 +1,106 @@
package server
import (
"context"
"encoding/json"
"testing"
"time"
"github.com/Autumn-27/artex/agent"
actool "github.com/Autumn-27/norma/tool"
)
func TestRecordFactInputSchemaUpgrade(t *testing.T) {
skipWithoutDSN(t)
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { m.Close() })
ctx, cancel := context.WithCancel(context.Background())
cancel()
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
t.Cleanup(func() { agent.SetToolResolve(nil) })
pg := m.pg
old, err := pg.GetTool("record_fact")
if err != nil || old == nil {
t.Fatal("missing record_fact", err)
}
bindings, _ := json.Marshal(old.Agents)
t.Cleanup(func() { pg.UpdateTool(old.Key, old.Description, old.Schema, bindings, old.Enabled) })
legacy := map[string]any{
"type": "object", "required": []string{"summary"},
"anyOf": []any{map[string]any{"required": []string{"summary"}}},
"properties": map[string]any{
"summary": map[string]any{"type": "string", "description": "operator summary"},
"confidence": map[string]any{"type": "string", "default": "inferred"},
"custom": map[string]any{"type": "string"},
"facts": map[string]any{"type": "array", "items": map[string]any{"type": "object"}},
},
}
raw, _ := json.Marshal(legacy)
if err := pg.UpdateTool("record_fact", "operator tool", raw, bindings, old.Enabled); err != nil {
t.Fatal(err)
}
check := func(schema map[string]any) {
t.Helper()
if _, ok := schema["required"]; ok {
t.Fatal("top-level summary still required for batches")
}
if _, ok := schema["anyOf"]; ok {
t.Fatal("root anyOf still present")
}
if schema["additionalProperties"] != false {
t.Fatal("unknown fields allowed")
}
props := schema["properties"].(map[string]any)
items := props["facts"].(map[string]any)["items"].(map[string]any)
itemProps := items["properties"].(map[string]any)
if props["tested_asset_ids"] == nil || itemProps["tested_asset_ids"] == nil || itemProps["summary"] == nil {
t.Fatal("repaired schema missing alias or batch summary")
}
if props["summary"].(map[string]any)["description"] != "operator summary" ||
props["confidence"].(map[string]any)["default"] != "inferred" || props["custom"] != nil {
t.Fatal("operator annotations lost")
}
if err := actool.ValidateInput(schema, json.RawMessage(`{"facts":[{"summary":"complete batch item"}]}`)); err != nil {
t.Fatal("valid batch rejected", err)
}
}
// Runtime repair protects cached or edited rows even before startup migration.
tools := agent.ToolResolve()(context.Background(), "worker", agent.NewToolSet(nil, "").WorkerTools())
found := false
for _, tool := range tools {
if tool.Name() == "record_fact" {
check(tool.InputSchema())
found = true
}
}
if !found {
t.Fatal("record_fact not resolved")
}
s.seedFactInputSchema()
repaired, err := pg.GetTool("record_fact")
if err != nil || repaired == nil {
t.Fatal(err)
}
var schema map[string]any
if err := json.Unmarshal(repaired.Schema, &schema); err != nil {
t.Fatal(err)
}
check(schema)
if repaired.Description != "operator tool" || repaired.Enabled != old.Enabled {
t.Fatal("operator tool settings changed")
}
var before, after time.Time
if err := pg.QueryRow("SELECT updated_at FROM tools WHERE key='record_fact'").Scan(&before); err != nil {
t.Fatal(err)
}
s.seedFactInputSchema()
if err := pg.QueryRow("SELECT updated_at FROM tools WHERE key='record_fact'").Scan(&after); err != nil {
t.Fatal(err)
}
if !before.Equal(after) {
t.Fatal("second migration changed the row")
}
}
+2 -2
View File
@@ -47,8 +47,8 @@ func TestRecordFactSourceSchemaUpgrade(t *testing.T) {
if err := pg.UpdateTool("record_fact", "operator description", custom, bindings, true); err != nil {
t.Fatal(err)
}
if src := resolvedRecordFactProps(t)["source"]; src != nil {
t.Fatalf("precondition: stored schema should not expose source yet: %v", src)
if src := resolvedRecordFactProps(t)["source"]; src == nil {
t.Fatal("runtime contract repair must expose source even before startup migration")
}
snapshot := func() (string, string, time.Time) {
+11
View File
@@ -26,6 +26,17 @@ func TestFindingDTOReasoningFields(t *testing.T) {
}
}
func TestFindingDTOReportState(t *testing.T) {
raw, _ := json.Marshal(findingFromDB(&db.DBFinding{ID: 1}, nil))
if !strings.Contains(string(raw), `"report_state":"none"`) {
t.Fatalf("empty report_state should serialize as none: %s", raw)
}
d := findingFromDB(&db.DBFinding{ID: 2, ReportState: db.FindingReportSubmitted}, nil)
if d.ReportState != db.FindingReportSubmitted {
t.Fatalf("report_state lost: %q", d.ReportState)
}
}
// PG: the detail endpoint returns the stored fields, and the report_finding
// schema upgrade adds both properties to an existing (customized) row without
// overwriting it.
+1
View File
@@ -31,6 +31,7 @@ func findingFilterFromQuery(q url.Values) db.FindingFilter {
return db.FindingFilter{
Severity: normFilter(q.Get("severity")),
Status: normFilter(q.Get("status")),
ReportState: normFilter(q.Get("report_state")),
VerificationStatus: normFilter(q.Get("verification_status")),
VulnClass: normFilter(q.Get("vulnclass")),
// task_id (independent of the task param, which switches to the per-task-node branch): filters the global table by task.
+1
View File
@@ -571,6 +571,7 @@ func (s *Server) seedOrchestrationTools() {
s.seedFindingWorkflowTools()
s.seedFindingVerificationSchema() // S1: poe_level / evidence_quotes on update_finding_report, added once per DB
s.seedFindingEvidenceSchema() // optional counterevidence / confidence_rationale on report_finding, added once per DB
s.seedFactInputSchema() // restore record_fact single/batch contracts
s.seedFactSourceSchema() // source enum on record_fact, added once per DB
s.seedTaskScopeDescription() // add_task_scope: "coverage scope, not enforced" replaces the authorization-boundary text
// Note: pentest's default tool bindings need no migration — BuiltinToolSeeds already seeds
+89 -21
View File
@@ -80,8 +80,12 @@ type Server struct {
llmProv llm.Provider
llmDirect llm.Provider // concrete global provider, before any failover pool
llmCfg agent.Config // current LLM config (key not exposed)
llmOn bool
llmProf string // active LLM profile name (for llmrec tagging)
// llmHealthProfileID is the llm_profiles row llmProv was built from, or 0
// when it came from environment settings. Task selections use it to key
// breaker health for the global fallback path.
llmHealthProfileID int64
llmOn bool
llmProf string // active LLM profile name (for llmrec tagging)
// chatBusy guards the per-task main-agent run: the chat handler launches the
// agent on the server's background ctx (not the request ctx) and returns
@@ -504,11 +508,13 @@ func maxTokensResolver(cfg agent.Config) func() int {
// the running engine as the GLOBAL active pair. Safe to call at runtime (UI configures LLM).
func (s *Server) applyLLM(cfg agent.Config) error {
var prov llm.Provider
var healthID int64
// A persisted active profile must use the same cached provider as task chains
// and Agent bindings, otherwise each path owns a separate rate limiter.
if active, _ := s.m.pg.ActiveProfile(); active != nil {
if activeCfg, ok := s.loadProfileConfig(active.ID); ok && activeCfg == cfg {
prov, _, _ = s.providerForProfile(active.ID)
healthID = active.ID
}
}
if prov == nil {
@@ -555,6 +561,7 @@ func (s *Server) applyLLM(cfg agent.Config) error {
s.chatAgent.SetNoaEnabled(s.m.NoaCompactionEnabled) // experimental: noa context compaction (read per run)
s.llmProv = prov
s.llmCfg = cfg
s.llmHealthProfileID = healthID
s.llmOn = true
s.cfgMu.Unlock()
s.invalidateTaskAgents()
@@ -848,6 +855,7 @@ func (s *Server) Handler() http.Handler {
mux.HandleFunc("POST /api/exploration/findings/{id}/retests", s.startFindingRetest)
mux.HandleFunc("PATCH /api/exploration/findings/{id}", s.patchFinding)
mux.HandleFunc("POST /api/exploration/findings/bulk-status", s.bulkPatchFindingStatus)
mux.HandleFunc("POST /api/exploration/findings/bulk-report-state", s.bulkPatchFindingReportState)
mux.HandleFunc("DELETE /api/exploration/findings/{id}", s.deleteFinding)
mux.HandleFunc("GET /api/exploration/intents", s.intents)
mux.HandleFunc("GET /api/exploration/graph", s.explorationGraph)
@@ -2364,7 +2372,7 @@ func (s *Server) getFinding(w http.ResponseWriter, r *http.Request) {
// format = md-single (one combined .md) | md-zip (one .md per vulnerability, zipped)
// | csv | json
// ids = comma-separated finding ids (required when scope=selected)
// Filter params severity/status/vulnclass/task_id/q/sort match the list endpoint (used when scope=filtered).
// Filter params severity/status/report_state/vulnclass/task_id/q/sort match the list endpoint (used when scope=filtered).
func (s *Server) findingsExport(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
scope := q.Get("scope")
@@ -2529,17 +2537,18 @@ func (s *Server) patchFinding(w http.ResponseWriter, r *http.Request) {
return
}
var body struct {
Status *string `json:"status"`
Severity *string `json:"severity"`
Name *string `json:"name"`
VulnClass *string `json:"vulnclass"`
Status *string `json:"status"`
ReportState *string `json:"report_state"`
Severity *string `json:"severity"`
Name *string `json:"name"`
VulnClass *string `json:"vulnclass"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, 400, "bad json: "+err.Error())
return
}
if body.Status == nil && body.Severity == nil && body.Name == nil && body.VulnClass == nil {
writeErr(w, 400, "nothing to update: provide status/severity/name/vulnclass")
if body.Status == nil && body.ReportState == nil && body.Severity == nil && body.Name == nil && body.VulnClass == nil {
writeErr(w, 400, "nothing to update: provide status/severity/name/vulnclass/report_state")
return
}
if body.Status != nil {
@@ -2563,6 +2572,21 @@ func (s *Server) patchFinding(w http.ResponseWriter, r *http.Request) {
log.Printf("[notify] status-change event not registered finding=%d %s→%s (status was updated)", id, from, *body.Status)
}
}
if body.ReportState != nil {
if !db.ValidFindingReportState(*body.ReportState) {
writeErr(w, 400, "bad report_state: "+*body.ReportState)
return
}
n, err := s.m.pg.SetFindingReportState(id, *body.ReportState)
if err != nil {
writeErr(w, 500, err.Error())
return
}
if n == 0 {
writeErr(w, 404, "finding not found")
return
}
}
if body.Severity != nil {
if !db.ValidSeverity(*body.Severity) {
writeErr(w, 400, "bad severity: "+*body.Severity)
@@ -2634,18 +2658,7 @@ func (s *Server) bulkPatchFindingStatus(w http.ResponseWriter, r *http.Request)
writeErr(w, 400, "bad status: "+body.Status)
return
}
seen := make(map[int64]bool, len(body.IDs))
ids := make([]int64, 0, len(body.IDs))
for _, id := range body.IDs {
if id <= 0 || seen[id] {
continue
}
seen[id] = true
ids = append(ids, id)
if len(ids) == maxBulkFindingStatus {
break
}
}
ids := normalizeFindingIDs(body.IDs, maxBulkFindingStatus)
if len(ids) == 0 {
writeErr(w, 400, "no finding ids")
return
@@ -2667,6 +2680,61 @@ func (s *Server) bulkPatchFindingStatus(w http.ResponseWriter, r *http.Request)
writeJSON(w, 200, map[string]any{"updated": updated, "missing": missing, "results": results})
}
// normalizeFindingIDs drops non-positive and duplicate ids, then caps the list.
func normalizeFindingIDs(ids []int64, capN int) []int64 {
seen := make(map[int64]bool, len(ids))
out := make([]int64, 0, len(ids))
for _, id := range ids {
if id <= 0 || seen[id] {
continue
}
seen[id] = true
out = append(out, id)
if capN > 0 && len(out) == capN {
break
}
}
return out
}
// bulkPatchFindingReportState sets the same report stage on a batch of findings
// in one transaction. Same cap/dedupe/per-id result as bulk status, without
// status-change notifications: report_state is a workflow stage, not a verdict.
func (s *Server) bulkPatchFindingReportState(w http.ResponseWriter, r *http.Request) {
var body struct {
IDs []int64 `json:"ids"`
ReportState string `json:"report_state"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, 400, "bad json: "+err.Error())
return
}
if !db.ValidFindingReportState(body.ReportState) {
writeErr(w, 400, "bad report_state: "+body.ReportState)
return
}
ids := normalizeFindingIDs(body.IDs, maxBulkFindingStatus)
if len(ids) == 0 {
writeErr(w, 400, "no finding ids")
return
}
results, err := s.m.pg.SetFindingReportStates(r.Context(), ids, body.ReportState)
if err != nil {
writeErr(w, 500, err.Error())
return
}
updated, missing := 0, 0
for _, res := range results {
if res.Changed {
updated++
}
if !res.Found {
missing++
}
}
writeJSON(w, 200, map[string]any{"updated": updated, "missing": missing, "results": results})
}
// deleteFinding removes a finding (findings row + originating exploration node).
// id is the standalone findings-table id (DTO finding_id).
func (s *Server) deleteFinding(w http.ResponseWriter, r *http.Request) {
+72 -13
View File
@@ -15,6 +15,7 @@ import (
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/llmpool"
"github.com/Autumn-27/artex/llmrec"
"github.com/Autumn-27/norma/llm"
"github.com/Autumn-27/norma/transcript"
@@ -137,8 +138,15 @@ func resetLLMFailStreak(taskID string, profileID int64) {
type taskLLMSelection struct {
task *Task
profileID int64
revision int64
provider llm.Provider
// healthProfileID is the llm_profiles row whose breaker state the calls on
// this selection update. It equals profileID on the task-chain path; a bound
// agent or the global provider carries its own id here while profileID stays
// 0 (no chain semantics). 0 means "do not report": either there is no
// profile (environment-only configuration) or the provider is already the
// failover Pool, which updates the registry itself.
healthProfileID int64
revision int64
provider llm.Provider
// retry holds the retry parameters resolved for the profile selected this time (profile override → global policy → built-in default).
// Same-provider safe-window retries follow it, so switching profiles switches the retry cadence.
retry agent.RetryConfig
@@ -165,6 +173,24 @@ type taskLLMStreamHooks struct {
current func(context.Context) (taskLLMSelection, error)
exhaust func(taskLLMSelection, error) (db.TaskLLMTransition, error)
transition func(taskLLMSelection, db.TaskLLMTransition, error)
// recordHealth reports one physical provider attempt to the process-wide
// breaker. It keeps /system/llm/ truthful for every selection whose provider
// is not already the failover Pool (the chain, a bound agent, the global
// profile). Optional: test literals leave it nil and the call sites skip it.
recordHealth func(ctx context.Context, profileID int64, stream, committed bool, err error)
}
// healthIDFor returns the profile whose breaker state a call on this provider
// should update. It returns 0 when there is no profile, or when the provider is
// the failover Pool, which records every member attempt itself.
func healthIDFor(prov llm.Provider, profileID int64) int64 {
if profileID == 0 {
return 0
}
if _, pooled := prov.(*llmpool.Pool); pooled {
return 0
}
return profileID
}
// current resolves the provider this role runs on, by precedence:
@@ -196,8 +222,9 @@ func (r *taskLLMRuntime) currentFor(ctx context.Context) (taskLLMSelection, erro
r.s.syncTaskLLMState(pt)
t, _ := r.s.m.Task(r.taskID)
sel := taskLLMSelection{task: t, revision: pt.LLMChainRevision}
if prov, cfg, ok := r.s.agentBindingProvider(r.agentKey); ok {
if prov, cfg, id, ok := r.s.agentBindingProvider(r.agentKey); ok {
sel.provider, sel.retry = prov, cfg.Retry
sel.healthProfileID = healthIDFor(prov, id)
return sel, nil
}
if len(pt.LLMProfileIDs) > 0 {
@@ -218,13 +245,15 @@ func (r *taskLLMRuntime) currentFor(ctx context.Context) (taskLLMSelection, erro
return sel, fmt.Errorf("LLM profile #%d is missing or invalid", sel.profileID)
}
sel.provider, sel.retry = prov, cfg.Retry
sel.healthProfileID = sel.profileID // chain selections never go through the Pool
return sel, nil
}
prov, cfg, ok := r.s.globalProvider()
prov, cfg, id, ok := r.s.globalProviderWithProfile()
if !ok {
return sel, fmt.Errorf("task %s has no available fallback LLM provider", r.taskID)
}
sel.provider, sel.retry = prov, cfg.Retry
sel.healthProfileID = healthIDFor(prov, id)
return sel, nil
}
@@ -242,7 +271,7 @@ func (r *taskLLMRuntime) activeCfg() (agent.Config, bool) {
if err != nil || pt == nil {
return agent.Config{}, false
}
if _, cfg, ok := r.s.agentBindingProvider(r.agentKey); ok {
if _, cfg, _, ok := r.s.agentBindingProvider(r.agentKey); ok {
return cfg, true
}
if len(pt.LLMProfileIDs) > 0 && pt.ActiveLLMProfileID != nil {
@@ -280,10 +309,19 @@ func parseTaskID(id string) (int64, error) {
// streamHooks builds the failover/exhaustion callbacks shared by Stream and
// Complete: how to read the current profile selection, how to mark it quota
// exhausted, and how to emit a failover transition.
// exhausted, how to emit a failover transition, and how to report breaker
// health for providers that are not the failover Pool.
func (r *taskLLMRuntime) streamHooks() taskLLMStreamHooks {
return taskLLMStreamHooks{
current: func(ctx context.Context) (taskLLMSelection, error) { return r.currentFor(ctx) },
recordHealth: func(ctx context.Context, profileID int64, stream, committed bool, err error) {
if profileID == 0 || r.s == nil || r.s.llmHealth == nil {
return
}
if tripped, msg := r.s.llmHealth.RecordCall(ctx, profileID, stream, committed, err); tripped {
log.Printf("[task-llm] profile #%d breaker tripped: %s", profileID, msg)
}
},
exhaust: func(selection taskLLMSelection, cause error) (db.TaskLLMTransition, error) {
taskNum, _ := parseTaskID(r.taskID)
transition, err := r.s.m.pg.MarkTaskLLMProfileQuotaExhaustedAtRevision(taskNum, selection.profileID, selection.revision, cause.Error())
@@ -338,6 +376,9 @@ func completeTaskLLM(ctx context.Context, taskID string, req llm.CompletionReque
for attempt := 0; ; attempt++ {
msg, sr, usage, callErr = selection.provider.Complete(ctx, req)
total.Add(usage)
if hooks.recordHealth != nil {
hooks.recordHealth(ctx, selection.healthProfileID, false, false, callErr)
}
if callErr != nil && ctx.Err() == nil &&
attempt < retries && isRetryableStreamError(callErr) {
backoff := backoffOf(attempt)
@@ -462,6 +503,9 @@ func streamTaskLLM(ctx context.Context, taskID string, req llm.CompletionRequest
return
}
}
if hooks.recordHealth != nil {
hooks.recordHealth(ctx, selection.healthProfileID, true, committed, streamErr)
}
if streamErr != nil && !committed {
// A failed uncommitted attempt delivers none of its events (they are
// never replayed), but the usage it reported survives as one
@@ -635,7 +679,7 @@ func isRetryableStreamError(err error) bool {
// CompactionWindow mirrors current()'s precedence so the context window always
// matches the provider the role will actually stream on.
func (r *taskLLMRuntime) CompactionWindow() int {
if _, cfg, ok := r.s.agentBindingProvider(r.agentKey); ok {
if _, cfg, _, ok := r.s.agentBindingProvider(r.agentKey); ok {
return cfg.CompactionWindow()
}
taskNum, err := parseTaskID(r.taskID)
@@ -671,17 +715,19 @@ func (r *taskLLMRuntime) CompactionWindow() int {
// (agents.llm_profile_id) — the highest-precedence level for task agents. ok=false
// when the role has no binding or the bound profile no longer builds, so callers
// fall through to the task chain. A bound profile stays exclusive unless
// llm_pool_bind_fallback is on, which is what poolForBinding encodes.
func (s *Server) agentBindingProvider(agentKey string) (llm.Provider, agent.Config, bool) {
// llm_pool_bind_fallback is on, which is what poolForBinding encodes. The
// returned id is the binding's profile, used to key breaker health when the
// returned provider is not the Pool.
func (s *Server) agentBindingProvider(agentKey string) (llm.Provider, agent.Config, int64, bool) {
id := s.effectiveProfileForAgent(agentKey, nil)
if id == nil {
return nil, agent.Config{}, false
return nil, agent.Config{}, 0, false
}
prov, cfg, ok := s.providerForProfile(*id)
if !ok {
return nil, agent.Config{}, false
return nil, agent.Config{}, 0, false
}
return s.poolForBinding(*id, prov, cfg), cfg, true
return s.poolForBinding(*id, prov, cfg), cfg, *id, true
}
// globalProvider returns the process-wide provider (persisted active profile or
@@ -696,6 +742,19 @@ func (s *Server) globalProvider() (llm.Provider, agent.Config, bool) {
return s.llmProv, s.llmCfg, true
}
// globalProviderWithProfile is globalProvider plus the llm_profiles row the
// provider was built from (0 for an environment-only configuration). The id is
// read under the same lock as the provider so the pair cannot be mixed across
// an applyLLM rebuild.
func (s *Server) globalProviderWithProfile() (llm.Provider, agent.Config, int64, bool) {
s.cfgMu.Lock()
defer s.cfgMu.Unlock()
if !s.llmOn || s.llmProv == nil {
return nil, agent.Config{}, 0, false
}
return s.llmProv, s.llmCfg, s.llmHealthProfileID, true
}
// taskRuntimeAvailable reports whether every listed role can resolve a provider
// under the runtime precedence in current(): the role's own binding first, then
// the task chain, then global. An exhausted chain is a hard stop for unbound
@@ -714,7 +773,7 @@ func (s *Server) taskRuntimeAvailable(t *Task, agentKeys ...string) bool {
_, _, unboundReady = s.globalProvider()
}
for _, key := range agentKeys {
if _, _, ok := s.agentBindingProvider(key); ok {
if _, _, _, ok := s.agentBindingProvider(key); ok {
continue
}
if !unboundReady {
+150
View File
@@ -0,0 +1,150 @@
package server
import (
"context"
"fmt"
"testing"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/llmpool"
"github.com/Autumn-27/norma/llm"
)
// healthHooks runs one logical call on p and reports every physical attempt to
// reg under healthID, the way streamHooks wires it in production.
func healthHooks(reg *llmpool.Registry, p llm.Provider, healthID int64) taskLLMStreamHooks {
return taskLLMStreamHooks{
current: func(context.Context) (taskLLMSelection, error) {
return taskLLMSelection{provider: p, healthProfileID: healthID, retry: agent.RetryConfig{StreamAttempts: 1}}, nil
},
exhaust: func(taskLLMSelection, error) (db.TaskLLMTransition, error) { return db.TaskLLMTransition{}, nil },
recordHealth: func(ctx context.Context, profileID int64, stream, committed bool, err error) {
reg.RecordCall(ctx, profileID, stream, committed, err)
},
}
}
// A successful call clears a degraded profile, on both call shapes.
func TestTaskLLMRecordsHealthOnSuccess(t *testing.T) {
resetLLMFailStreaksForTest()
ctx := context.Background()
for _, stream := range []bool{false, true} {
t.Run(fmt.Sprintf("stream=%v", stream), func(t *testing.T) {
reg := llmpool.NewRegistry(nil, nil)
reg.RecordCall(ctx, 7, false, false, fmt.Errorf("openai: status 500: down"))
if st := reg.Get(7); st.Fails != 1 {
t.Fatalf("seed state=%+v, want one failure", st)
}
runLogicalCall(ctx, stream, "health-success", healthHooks(reg, &scriptedLLMProvider{}, 7))
if st := reg.Get(7); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("state=%+v after success, want clean", st)
}
})
}
}
// A deterministic 400 on an atomic call clears the streak, matching
// Pool.Complete; the same error on a stream leaves it alone, matching
// Pool.Stream.
func TestTaskLLMRecordsNonFailoverErrorLikeThePool(t *testing.T) {
resetLLMFailStreaksForTest()
ctx := context.Background()
bad := fmt.Errorf("openai: status 400: bad request")
reg := llmpool.NewRegistry(nil, nil)
reg.RecordCall(ctx, 7, false, false, fmt.Errorf("openai: status 500: down"))
runLogicalCall(ctx, false, "health-atomic-400", healthHooks(reg, &scriptedLLMProvider{err: bad}, 7))
if st := reg.Get(7); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("atomic state=%+v after 400, want clean", st)
}
reg2 := llmpool.NewRegistry(nil, nil)
reg2.RecordCall(ctx, 8, false, false, fmt.Errorf("openai: status 500: down"))
runLogicalCall(ctx, true, "health-stream-400", healthHooks(reg2, &scriptedLLMProvider{err: bad}, 8))
if st := reg2.Get(8); st.Fails != 1 || st.Trips != 0 {
t.Fatalf("stream state=%+v after 400, want the failure kept", st)
}
}
// Every physical stream attempt is reported: a retried pre-commit 500 counts
// twice and does not trip a three-strike breaker.
func TestTaskLLMRecordsEveryStreamAttempt(t *testing.T) {
resetLLMFailStreaksForTest()
defer withZeroRetryBackoff()()
ctx := context.Background()
reg := llmpool.NewRegistry(nil, nil)
p := &scriptedLLMProvider{err: fmt.Errorf("openai: status 500: down")}
runLogicalCall(ctx, true, "health-retry", healthHooks(reg, p, 7))
if p.calls != 2 {
t.Fatalf("provider calls=%d, want one attempt plus one retry", p.calls)
}
if st := reg.Get(7); st.Fails != 2 || st.Trips != 0 {
t.Fatalf("state=%+v, want two soft failures and no trip", st)
}
}
// A failure after output reached the caller is committed: the Pool does not
// fail over there and the breaker is left alone.
func TestTaskLLMRecordsCommittedFailureAsNoHealthEvent(t *testing.T) {
resetLLMFailStreaksForTest()
reg := llmpool.NewRegistry(nil, nil)
p := &scriptedLLMProvider{
events: []llm.StreamEvent{{Type: llm.SETextDelta, Text: "partial"}},
err: fmt.Errorf("openai: status 500: mid-stream drop"),
}
runLogicalCall(context.Background(), true, "health-committed", healthHooks(reg, p, 7))
if st := reg.Get(7); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("state=%+v after a committed failure, want clean", st)
}
}
// Without a health profile (environment-only configuration) nothing is
// recorded.
func TestTaskLLMRecordsNoHealthWithoutProfile(t *testing.T) {
resetLLMFailStreaksForTest()
reg := llmpool.NewRegistry(nil, nil)
p := &scriptedLLMProvider{err: fmt.Errorf("openai: status 500: down")}
runLogicalCall(context.Background(), true, "health-none", healthHooks(reg, p, 0))
if n := len(reg.Snapshot()); n != 0 {
t.Fatalf("%d states recorded with healthProfileID 0", n)
}
}
// healthIDFor keys the registry only for a provider that does not record by
// itself: a Pool already updates every member attempt.
func TestHealthIDForSkipsPooledProviders(t *testing.T) {
bare := &scriptedLLMProvider{}
pool := llmpool.New([]*llmpool.Member{{ID: 9, Prov: bare}}, nil)
if pool == nil {
t.Fatal("pool should be non-nil")
}
if got := healthIDFor(bare, 7); got != 7 {
t.Fatalf("bare provider id=%d, want 7", got)
}
if got := healthIDFor(pool, 7); got != 0 {
t.Fatalf("pooled provider id=%d, want 0", got)
}
if got := healthIDFor(bare, 0); got != 0 {
t.Fatalf("profile 0 id=%d, want 0", got)
}
}
// The production hook closure records under the given profile and no-ops on 0.
func TestStreamHooksRecordBreakerHealth(t *testing.T) {
reg := llmpool.NewRegistry(nil, nil)
r := &taskLLMRuntime{s: &Server{llmHealth: reg}}
h := r.streamHooks()
ctx := context.Background()
h.recordHealth(ctx, 0, false, false, fmt.Errorf("openai: status 500: down"))
if n := len(reg.Snapshot()); n != 0 {
t.Fatalf("%d states recorded for profile 0", n)
}
h.recordHealth(ctx, 7, false, false, fmt.Errorf("openai: status 500: down"))
if st := reg.Get(7); st.Fails != 1 {
t.Fatalf("state=%+v, want one failure", st)
}
h.recordHealth(ctx, 7, false, false, nil)
if st := reg.Get(7); st.Fails != 0 || st.Trips != 0 {
t.Fatalf("state=%+v after success, want clean", st)
}
}
@@ -37,7 +37,14 @@ import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@
import { Spinner } from "@/components/ui/spinner";
import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table";
import { statusMeta, type Tone, toneClasses } from "@/lib/status";
import type { ActiveFindingRetest, Finding, FindingStatus, Severity, VerificationStatus } from "@/lib/types";
import type {
ActiveFindingRetest,
Finding,
FindingReportState,
FindingStatus,
Severity,
VerificationStatus,
} from "@/lib/types";
import { cn } from "@/lib/utils";
export const SEVERITIES: Severity[] = ["critical", "high", "medium", "low"];
@@ -62,6 +69,37 @@ export const VERIFICATION_STATUSES: VerificationStatus[] = [
"false_positive",
];
export const FINDING_REPORT_STATES: FindingReportState[] = [
"none",
"submitted",
"accepted",
"rejected",
"duplicate",
];
export const REPORT_STATE_META: Record<FindingReportState, { label: string; tone: Tone }> = {
none: { label: "Not submitted", tone: "slate" },
submitted: { label: "Submitted", tone: "amber" },
accepted: { label: "Accepted", tone: "green" },
rejected: { label: "Rejected", tone: "red" },
duplicate: { label: "Duplicate report", tone: "neutral" },
};
export function ReportStateBadge({ state }: { state?: FindingReportState }) {
const value = state ?? "none";
const meta = REPORT_STATE_META[value] ?? REPORT_STATE_META.none;
return (
<span
className={cn(
"inline-flex w-fit items-center whitespace-nowrap rounded-md border px-2 py-0.5 font-medium text-xs",
toneClasses[meta.tone],
)}
>
{meta.label}
</span>
);
}
// Verification outcome → label/tone, on the same tone tokens StatusBadge uses.
export const VERIFICATION_META: Record<VerificationStatus, { label: string; tone: Tone }> = {
verified: { label: "Verified", tone: "green" },
@@ -125,7 +163,7 @@ export function fmtTime(ts: string) {
});
}
const COLUMN_COUNT = 9;
const COLUMN_COUNT = 10;
interface FindingsTableProps {
items: Finding[];
@@ -141,6 +179,7 @@ interface FindingsTableProps {
saving: boolean;
onSave: (finding: Finding) => void;
onStatusChange: (finding: Finding, next: FindingStatus) => void;
onReportStateChange: (finding: Finding, next: FindingReportState) => void;
onRetest: (finding: Finding) => void;
activeRetests: Record<string, ActiveFindingRetest>;
onDeepen: (finding: Finding) => void;
@@ -164,6 +203,7 @@ export function FindingsTable({
saving,
onSave,
onStatusChange,
onReportStateChange,
onRetest,
activeRetests,
onDeepen,
@@ -181,7 +221,7 @@ export function FindingsTable({
return (
/* Fixed column widths keep expanded content from stretching the table; narrow screens scroll horizontally inside the table only. */
<Table className="min-w-[60rem] table-fixed">
<Table className="min-w-[68rem] table-fixed">
<TableHeader>
<TableRow>
<TableHead className="w-8">
@@ -196,6 +236,7 @@ export function FindingsTable({
<TableHead>Vulnerability</TableHead>
<TableHead className="w-44">Asset</TableHead>
<TableHead className="w-28">Status</TableHead>
<TableHead className="w-36">Report</TableHead>
<TableHead className="w-32">Task</TableHead>
<TableHead className="w-24">Time</TableHead>
<TableHead className="w-48">Actions</TableHead>
@@ -296,6 +337,27 @@ export function FindingsTable({
<StatusBadge domain="finding" value={f.status} dot />
)}
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
{f.finding_id ? (
<Select
value={f.report_state ?? "none"}
onValueChange={(v) => onReportStateChange(f, v as FindingReportState)}
>
<SelectTrigger size="sm" className="h-7 w-full border-none px-1 shadow-none focus-visible:ring-0">
<ReportStateBadge state={f.report_state} />
</SelectTrigger>
<SelectContent position="popper" align="end">
{FINDING_REPORT_STATES.map((st) => (
<SelectItem key={st} value={st}>
{REPORT_STATE_META[st].label}
</SelectItem>
))}
</SelectContent>
</Select>
) : (
<ReportStateBadge state={f.report_state} />
)}
</TableCell>
<TableCell>
{f.task_id ? (
<Link
@@ -23,9 +23,14 @@ import { SidebarTrigger } from "@/components/ui/sidebar";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { api } from "@/lib/api";
import { statusMeta } from "@/lib/status";
import type { Finding, FindingStatus, Severity } from "@/lib/types";
import type { Finding, FindingReportState, FindingStatus, Severity } from "@/lib/types";
import { VerificationBadge } from "../_components/findings-table";
import {
FINDING_REPORT_STATES,
REPORT_STATE_META,
ReportStateBadge,
VerificationBadge,
} from "../_components/findings-table";
import { FindingLineageView } from "./lineage";
const SEVERITIES: Severity[] = ["critical", "high", "medium", "low"];
@@ -112,6 +117,23 @@ function FindingDetailInner() {
[finding, id],
);
const changeReportState = React.useCallback(
async (next: FindingReportState) => {
if (!finding || finding.inherited || next === (finding.report_state ?? "none")) return;
const prev = finding.report_state ?? "none";
setFinding({ ...finding, report_state: next });
try {
const updated = await api.setFindingReportState(id, next);
setFinding(updated);
toast.success(`Report stage changed to "${REPORT_STATE_META[next].label}"`);
} catch (e) {
setFinding((cur) => (cur ? { ...cur, report_state: prev } : cur));
toast.error("Update failed: " + (e as Error).message);
}
},
[finding, id],
);
if (!finding) {
return (
<div className="flex flex-1 flex-col items-center justify-center gap-3 p-10 text-center">
@@ -148,6 +170,7 @@ function FindingDetailInner() {
<Separator orientation="vertical" className="mx-1 h-4" />
<StatusBadge domain="severity" value={finding.severity} dot />
<StatusBadge domain="finding" value={finding.status} dot />
<ReportStateBadge state={finding.report_state} />
<VerificationBadge finding={finding} />
{finding.inherited && finding.source_task_id && (
<Badge variant="outline">Source task #{finding.source_task_id} · read-only</Badge>
@@ -294,6 +317,30 @@ function FindingDetailInner() {
)}
</FieldRow>
<FieldRow label="Report stage">
{finding.inherited ? (
<ReportStateBadge state={finding.report_state} />
) : (
<Select
value={finding.report_state ?? "none"}
onValueChange={(v) => changeReportState(v as FindingReportState)}
>
<SelectTrigger size="sm" className="h-7 w-auto border-none px-1 shadow-none focus-visible:ring-0">
<ReportStateBadge state={finding.report_state} />
</SelectTrigger>
<SelectContent position="popper" align="end">
<SelectGroup>
{FINDING_REPORT_STATES.map((st) => (
<SelectItem key={st} value={st}>
{REPORT_STATE_META[st].label}
</SelectItem>
))}
</SelectGroup>
</SelectContent>
</Select>
)}
</FieldRow>
{/* Verification: automated outcome; the note explains any state other than verified */}
{finding.verification_status && (
<FieldRow label="Verification">
+136 -9
View File
@@ -8,8 +8,10 @@ import {
ArrowUpRightIcon,
BugIcon,
ChevronRightIcon,
CircleCheckIcon,
ClockIcon,
DownloadIcon,
FileUpIcon,
InfoIcon,
SearchIcon,
ShieldAlertIcon,
@@ -46,6 +48,7 @@ import type {
Finding,
FindingAssetNode,
FindingGroup,
FindingReportState,
FindingStats,
FindingStatus,
Severity,
@@ -55,6 +58,7 @@ import { cn } from "@/lib/utils";
import { AssetTree, assetPathOf } from "./_components/asset-tree";
import {
FINDING_REPORT_STATES,
FINDING_STATUSES,
type FindingEdit,
type FindingReport,
@@ -62,6 +66,7 @@ import {
findingRowKey,
fmtTime,
isSameFinding,
REPORT_STATE_META,
SEVERITIES,
UNASSIGNED_TASK,
VERIFICATION_META,
@@ -124,6 +129,8 @@ function findingGroupKey(group: FindingGroup) {
const EMPTY_STATS: FindingStats = {
total: 0,
pending: 0,
submitted: 0,
accepted: 0,
critical: 0,
high: 0,
medium: 0,
@@ -136,6 +143,7 @@ export default function FindingsPage() {
const [view, setView] = React.useState<FindingView>("flat");
const [severity, setSeverity] = React.useState<"all" | Severity>("all");
const [status, setStatus] = React.useState<"all" | FindingStatus>("all");
const [reportState, setReportState] = React.useState<"all" | FindingReportState>("all");
const [verificationStatus, setVerificationStatus] = React.useState<"all" | VerificationStatus>("all");
const [vulnclass, setVulnclass] = React.useState<string>("all");
const [task, setTask] = React.useState<string>("all");
@@ -167,7 +175,16 @@ export default function FindingsPage() {
const retestRefreshVersion = React.useRef(0);
const [deepenDescription, setDeepenDescription] = React.useState("");
const [deepening, setDeepening] = React.useState(false);
const filterFingerprint = JSON.stringify([severity, status, verificationStatus, vulnclass, task, sort, query]);
const filterFingerprint = JSON.stringify([
severity,
status,
reportState,
verificationStatus,
vulnclass,
task,
sort,
query,
]);
const activeFilterFingerprint = React.useRef(filterFingerprint);
activeFilterFingerprint.current = filterFingerprint;
@@ -205,6 +222,7 @@ export default function FindingsPage() {
view?: unknown;
severity?: unknown;
status?: unknown;
reportState?: unknown;
verificationStatus?: unknown;
vulnclass?: unknown;
task?: unknown;
@@ -217,6 +235,9 @@ export default function FindingsPage() {
if (parsed.status === "all" || FINDING_STATUSES.includes(parsed.status as FindingStatus)) {
setStatus(parsed.status as "all" | FindingStatus);
}
if (parsed.reportState === "all" || FINDING_REPORT_STATES.includes(parsed.reportState as FindingReportState)) {
setReportState(parsed.reportState as "all" | FindingReportState);
}
if (
parsed.verificationStatus === "all" ||
VERIFICATION_STATUSES.includes(parsed.verificationStatus as VerificationStatus)
@@ -237,9 +258,9 @@ export default function FindingsPage() {
if (!preferencesHydrated) return;
setLocalStorageValue(
FINDING_LIST_PREFERENCE_KEY,
JSON.stringify({ view, severity, status, verificationStatus, vulnclass, task, sort }),
JSON.stringify({ view, severity, status, reportState, verificationStatus, vulnclass, task, sort }),
);
}, [preferencesHydrated, severity, sort, status, task, verificationStatus, view, vulnclass]);
}, [preferencesHydrated, reportState, severity, sort, status, task, verificationStatus, view, vulnclass]);
React.useEffect(() => {
const timer = window.setTimeout(() => setQuery(search.trim()), 300);
@@ -299,7 +320,7 @@ export default function FindingsPage() {
await api.exportFindings({
format: exportFormat,
scope: exportScope,
filters: { severity, status, verificationStatus, vulnclass, task, query, sort },
filters: { severity, status, reportState, verificationStatus, vulnclass, task, query, sort },
ids: [...selectedIds],
});
setExportOpen(false);
@@ -339,6 +360,7 @@ export default function FindingsPage() {
pageSize: flatPageSize,
severity,
status,
reportState,
verificationStatus,
vulnclass,
task,
@@ -360,6 +382,7 @@ export default function FindingsPage() {
flatPageSize,
severity,
status,
reportState,
verificationStatus,
vulnclass,
task,
@@ -375,7 +398,16 @@ export default function FindingsPage() {
const request = ++assetTreeRequest.current;
setAssetTree((current) => ({ ...current, loading: true }));
try {
const result = await api.findingAssetTree({ severity, status, verificationStatus, vulnclass, task, query, sort });
const result = await api.findingAssetTree({
severity,
status,
reportState,
verificationStatus,
vulnclass,
task,
query,
sort,
});
if (request !== assetTreeRequest.current || activeFilterFingerprint.current !== requestFilter) return;
setAssetTree({
nodes: result.nodes ?? [],
@@ -390,7 +422,7 @@ export default function FindingsPage() {
setAssetTree((current) => ({ ...current, loading: false }));
toast.error(`Failed to load the asset tree: ${(e as Error).message}`);
}
}, [filterFingerprint, severity, status, verificationStatus, vulnclass, task, query, sort]);
}, [filterFingerprint, reportState, severity, status, verificationStatus, vulnclass, task, query, sort]);
const refreshGroups = React.useCallback(async () => {
const requestFilter = filterFingerprint;
@@ -402,6 +434,7 @@ export default function FindingsPage() {
pageSize,
severity,
status,
reportState,
verificationStatus,
vulnclass,
task,
@@ -415,7 +448,7 @@ export default function FindingsPage() {
} catch {
// Polling keeps the last successful snapshot visible.
}
}, [filterFingerprint, page, pageSize, severity, status, verificationStatus, vulnclass, task, query, sort]);
}, [filterFingerprint, page, pageSize, reportState, severity, status, verificationStatus, vulnclass, task, query, sort]);
const loadGroup = React.useCallback(
async (key: string, groupPage: number, groupPageSize: number) => {
@@ -440,6 +473,7 @@ export default function FindingsPage() {
pageSize: groupPageSize,
severity,
status,
reportState,
verificationStatus,
vulnclass,
task: key,
@@ -475,7 +509,7 @@ export default function FindingsPage() {
}));
}
},
[filterFingerprint, severity, status, verificationStatus, vulnclass, query, sort],
[filterFingerprint, reportState, severity, status, verificationStatus, vulnclass, query, sort],
);
React.useEffect(() => {
@@ -657,6 +691,34 @@ export default function FindingsPage() {
[refreshAfterMutation, setFindings, status],
);
const updateReportState = React.useCallback(
async (f: Finding, next: FindingReportState) => {
if (!f.finding_id || next === (f.report_state ?? "none")) return;
const prev = f.report_state ?? "none";
setFindings((cur) => cur.map((x) => (isSameFinding(x, f) ? { ...x, report_state: next } : x)));
try {
await api.setFindingReportState(f.finding_id, next);
toast.success(`Report stage set to "${REPORT_STATE_META[next].label}"`);
api
.findingStats()
.then(setStats)
.catch(() => {
// The row update remains valid even if the aggregate refresh fails.
});
if (reportState !== "all" && next !== reportState) {
setFindings((cur) => cur.filter((x) => !isSameFinding(x, f)));
setTotal((t) => Math.max(0, t - 1));
setFlat((cur) => ({ ...cur, total: Math.max(0, cur.total - 1) }));
}
refreshAfterMutation(f);
} catch (e) {
setFindings((cur) => cur.map((x) => (isSameFinding(x, f) ? { ...x, report_state: prev } : x)));
toast.error(`Update failed: ${(e as Error).message}`);
}
},
[refreshAfterMutation, reportState, setFindings],
);
// applyBulkStatus sets the same triage status on every selected finding, then
// reloads the current view so rows hidden by the status filter disappear.
const applyBulkStatus = React.useCallback(
@@ -687,6 +749,36 @@ export default function FindingsPage() {
[bulkBusy, loadAssetTree, loadFlat, selectedIds, view],
);
const applyBulkReportState = React.useCallback(
async (next: FindingReportState) => {
if (selectedIds.size === 0 || bulkBusy) return;
setBulkBusy(true);
try {
const r = await api.bulkSetFindingReportState([...selectedIds], next);
setSelectedIds(new Set());
const label = REPORT_STATE_META[next].label;
toast.success(
r.updated === 1
? `Set report stage of 1 finding to "${label}"`
: `Set report stage of ${r.updated} findings to "${label}"`,
);
void loadFlat();
if (view === "asset") void loadAssetTree();
api
.findingStats()
.then(setStats)
.catch(() => {
// The list reload remains valid even if the aggregate refresh fails.
});
} catch (e) {
toast.error(`Bulk update failed: ${(e as Error).message}`);
} finally {
setBulkBusy(false);
}
},
[bulkBusy, loadAssetTree, loadFlat, selectedIds, view],
);
// The inline expanded detailed-report cache is keyed by the globally stable row key. report is long-form
// Markdown the list query doesn't include, so it is fetched once by finding_id on expand; done with empty text = no report yet.
const [reports, setReports] = React.useState<Record<string, FindingReport>>({});
@@ -812,6 +904,8 @@ export default function FindingsPage() {
const statCards = [
{ label: "Total findings", value: stats.total, icon: BugIcon },
{ label: "Pending", value: stats.pending, tone: "text-amber-500", icon: ClockIcon },
{ label: "Submitted", value: stats.submitted, tone: "text-amber-500", icon: FileUpIcon },
{ label: "Accepted", value: stats.accepted, tone: "text-emerald-500", icon: CircleCheckIcon },
{ label: "Critical", value: stats.critical, tone: "text-rose-600", icon: ShieldAlertIcon },
{ label: "High", value: stats.high, tone: "text-red-500", icon: TriangleAlertIcon },
{ label: "Medium", value: stats.medium, tone: "text-amber-500", icon: TriangleAlertIcon },
@@ -838,6 +932,7 @@ export default function FindingsPage() {
saving,
onSave: saveEdit,
onStatusChange: updateStatus,
onReportStateChange: updateReportState,
onRetest: setRetestFinding,
activeRetests,
onDeepen: openDeepen,
@@ -888,7 +983,7 @@ export default function FindingsPage() {
</Tabs>
</div>
<div className="flex flex-1 flex-col gap-4 md:gap-6">
<div className="grid grid-cols-2 gap-4 sm:grid-cols-3 lg:grid-cols-6">
<div className="grid grid-cols-2 gap-4 sm:grid-cols-4 lg:grid-cols-8">
{statCards.map((stat) => {
const StatIcon = stat.icon;
return (
@@ -956,6 +1051,20 @@ export default function FindingsPage() {
</SelectContent>
</Select>
<Select value={reportState} onValueChange={(v) => setReportState(v as "all" | FindingReportState)}>
<SelectTrigger size="sm" className="w-40">
<SelectValue placeholder="Report stage" />
</SelectTrigger>
<SelectContent>
<SelectItem value="all">All report stages</SelectItem>
{FINDING_REPORT_STATES.map((st) => (
<SelectItem key={st} value={st}>
{REPORT_STATE_META[st].label}
</SelectItem>
))}
</SelectContent>
</Select>
<Select
value={verificationStatus}
onValueChange={(v) => setVerificationStatus(v as "all" | VerificationStatus)}
@@ -1046,6 +1155,24 @@ export default function FindingsPage() {
))}
</SelectContent>
</Select>
<Select
value=""
onValueChange={(v) => {
if (v) void applyBulkReportState(v as FindingReportState);
}}
disabled={bulkBusy}
>
<SelectTrigger size="sm" className="w-44" aria-label="Set the report stage of the selected findings">
<SelectValue placeholder={bulkBusy ? "Updating…" : "Set report stage"} />
</SelectTrigger>
<SelectContent>
{FINDING_REPORT_STATES.map((st) => (
<SelectItem key={st} value={st}>
{REPORT_STATE_META[st].label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
)}
<Button size="sm" variant="outline" onClick={openExport}>
@@ -133,6 +133,9 @@ export default function LLMRecordsPage() {
// Recording on/off toggle (settings.llm_record; default off). When off the
// backend records nothing.
const [recEnabled, setRecEnabled] = React.useState(false);
// recKnown flips once the settings call resolves, so the empty state does not
// claim "recording is off" while the answer is still in flight.
const [recKnown, setRecKnown] = React.useState(false);
const [recBusy, setRecBusy] = React.useState(false);
// Inline detail panel (Burp-style split, not a dialog)
@@ -167,7 +170,11 @@ export default function LLMRecordsPage() {
let alive = true;
api
.settings()
.then((s) => { if (alive) setRecEnabled(!!s.llm_record); })
.then((s) => {
if (!alive) return;
setRecEnabled(!!s.llm_record);
setRecKnown(true);
})
.catch(() => {});
return () => { alive = false; };
}, []);
@@ -400,7 +407,9 @@ export default function LLMRecordsPage() {
) : records.length === 0 ? (
<TableRow>
<TableCell colSpan={8} className="py-12 text-center text-sm text-muted-foreground">
No LLM call records yet
{!recKnown || recEnabled
? "No LLM call records yet"
: "LLM call recording is off. Turn on the Recording toggle above to start capturing calls."}
</TableCell>
</TableRow>
) : (
+10 -1
View File
@@ -43,6 +43,7 @@ import type {
FindingGroupsPage,
FindingQuery,
FindingRetest,
FindingReportState,
FindingStats,
FindingStatus,
FindingsPage,
@@ -241,6 +242,7 @@ function findingFilterParams(q: Omit<FindingQuery, "page" | "pageSize">): URLSea
const p = new URLSearchParams();
if (q.severity && q.severity !== "all") p.set("severity", q.severity);
if (q.status && q.status !== "all") p.set("status", q.status);
if (q.reportState && q.reportState !== "all") p.set("report_state", q.reportState);
if (q.verificationStatus && q.verificationStatus !== "all") p.set("verification_status", q.verificationStatus);
if (q.vulnclass && q.vulnclass !== "all") p.set("vulnclass", q.vulnclass);
if (q.task && q.task !== "all") p.set("task_id", q.task);
@@ -703,6 +705,8 @@ export const api = {
// Vulnerability lineage: the subgraph (nodes + edges) from the vulnerability node back to the task's initial node.
findingLineage: (id: string) => get<{ nodes: TaskNode[]; edges: Edge[] }>(`/exploration/findings/${id}/lineage`),
setFindingStatus: (id: string, status: FindingStatus) => patch<Finding>(`/exploration/findings/${id}`, { status }),
setFindingReportState: (id: string, report_state: FindingReportState) =>
patch<Finding>(`/exploration/findings/${id}`, { report_state }),
// Bulk triage: one transaction server-side, with a per-id result. Unknown ids
// come back as found=false instead of failing the whole batch.
bulkSetFindingStatus: (ids: string[], status: FindingStatus) =>
@@ -710,11 +714,16 @@ export const api = {
"/exploration/findings/bulk-status",
{ ids: ids.map(Number), status },
),
bulkSetFindingReportState: (ids: string[], report_state: FindingReportState) =>
post<{ updated: number; missing: number; results: { id: number; found: boolean; changed: boolean }[] }>(
"/exploration/findings/bulk-report-state",
{ ids: ids.map(Number), report_state },
),
setFindingSeverity: (id: string, severity: Severity) => patch<Finding>(`/exploration/findings/${id}`, { severity }),
// Save a vulnerability's name/class/severity in one call (inline edit in the findings list); only present fields are sent.
updateFinding: (
id: string,
fields: { name?: string; vulnclass?: string; severity?: Severity; status?: FindingStatus },
fields: { name?: string; vulnclass?: string; severity?: Severity; status?: FindingStatus; report_state?: FindingReportState },
) => patch<Finding>(`/exploration/findings/${id}`, fields),
// Delete a vulnerability: removes the findings record + the source exploration node (gone from the findings list, the task findings tab, and the exploration graph together).
deleteFinding: (id: string) => del<{ deleted: boolean; id: number }>(`/exploration/findings/${id}`),
+27
View File
@@ -471,6 +471,8 @@ function mockFilterFindings(q: URLSearchParams): (typeof mockFindings)[number][]
const taskID = q.get("task_id");
if (severity) list = list.filter((finding) => finding.severity === severity);
if (status) list = list.filter((finding) => finding.status === status);
const reportState = q.get("report_state");
if (reportState) list = list.filter((finding) => (finding.report_state ?? "none") === reportState);
if (vulnclass) list = list.filter((finding) => finding.vulnclass === vulnclass);
if (taskID === "__unassigned__") list = list.filter((finding) => !finding.task_id);
else if (taskID) list = list.filter((finding) => finding.task_id === taskID);
@@ -1866,6 +1868,8 @@ function route(m: string, path: string, seg: string[], q: URLSearchParams, b: Re
return {
total: mockFindings.length,
pending: mockFindings.filter((f) => f.status === "pending").length,
submitted: mockFindings.filter((f) => (f.report_state ?? "none") === "submitted").length,
accepted: mockFindings.filter((f) => f.report_state === "accepted").length,
critical: mockFindings.filter((f) => f.severity === "critical").length,
high: mockFindings.filter((f) => f.severity === "high").length,
medium: mockFindings.filter((f) => f.severity === "medium").length,
@@ -1874,6 +1878,28 @@ function route(m: string, path: string, seg: string[], q: URLSearchParams, b: Re
tasks,
};
}
if (path === "/exploration/findings/bulk-report-state" && m === "POST") {
const ids = Array.isArray(b.ids) ? b.ids : [];
const state = typeof b.report_state === "string" ? b.report_state : "";
let updated = 0;
let missing = 0;
const results = ids.map((id) => {
const key = String(id);
const f = mockFindings.find((item) => item.id === key);
if (!f) {
missing++;
return { id, found: false, changed: false };
}
const from = f.report_state ?? "none";
const changed = from !== state;
if (changed) {
f.report_state = state as typeof f.report_state;
updated++;
}
return { id, found: true, changed, from };
});
return { updated, missing, results };
}
if (path === "/exploration/findings/asset-tree") {
const list = mockApplyAssetScope(mockFilterFindings(q), null);
return { nodes: mockBuildAssetTree(list), finding_total: list.length, truncated: false };
@@ -2081,6 +2107,7 @@ function route(m: string, path: string, seg: string[], q: URLSearchParams, b: Re
if (!f) return {};
if (m === "PATCH") {
if (typeof b.status === "string") f.status = b.status as typeof f.status;
if (typeof b.report_state === "string") f.report_state = b.report_state as typeof f.report_state;
if (typeof b.severity === "string") f.severity = b.severity as typeof f.severity;
if (typeof b.name === "string") f.name = b.name;
if (typeof b.vulnclass === "string") f.vulnclass = b.vulnclass;
+7
View File
@@ -486,6 +486,9 @@ export type FindingStatus =
// human triage status above. "blocked" is produced by no path yet but is displayed when present.
export type VerificationStatus = "verified" | "unverified" | "blocked" | "not_assessed" | "false_positive";
// Report stage after triage: a finding can be confirmed and submitted at the same time.
export type FindingReportState = "none" | "submitted" | "accepted" | "rejected" | "duplicate";
// FindingAsset is an asset bound to a vulnerability (label pre-rendered by the backend).
export interface FindingAsset {
id: string;
@@ -509,6 +512,7 @@ export interface Finding {
name?: string; // vulnerability name; display falls back to vulnclass when empty
severity: Severity;
status: FindingStatus;
report_state?: FindingReportState; // workflow stage after triage; default none
summary: string;
evidence: string;
report?: string; // detailed report (Markdown); returned only by the detail endpoint, empty in lists
@@ -562,6 +566,8 @@ export interface FindingDeepenResponse {
export interface FindingStats {
total: number;
pending: number;
submitted: number;
accepted: number;
critical: number;
high: number;
medium: number;
@@ -585,6 +591,7 @@ export interface FindingQuery {
pageSize: number;
severity?: "all" | Severity;
status?: "all" | FindingStatus;
reportState?: "all" | FindingReportState;
verificationStatus?: "all" | VerificationStatus;
vulnclass?: string;
task?: string; // task id; "all"/empty = no task filter