From 02265524f245bb4fa8b666345679cea3591ccd9a Mon Sep 17 00:00:00 2001 From: BaaRai <130491490+BaaRai@users.noreply.github.com> Date: Mon, 12 Oct 2026 07:29:59 +0200 Subject: [PATCH] feat: duplicate discipline before report_finding and in the reporter Nothing today makes an agent check for an existing finding before registering a new one, and the reporter has no duplicate or collapsing rules. Duplicates burn reviewer trust on submission (programs read them as noise or padding) and they inflate the task's apparent progress. Changes: - worker and pentest seeds: before report_finding, check list_findings / list_task_findings for an existing finding with the same root cause, endpoint, and parameter; the same defect again means extending that finding's evidence, not registering a duplicate. A genuinely distinct entry point, parameter, or privilege level remains its own finding (the adversarial gate's anti-merge rule still holds: the dedup check collapses only true duplicates). - reporter seed: an explicit duplicate pass over list_task_findings in the work steps, plus two discipline rules (no duplicate reports for an already-reported defect; one finding per report, additional locations of the same root cause listed inside it, never silently merged or spun off). - agent/finding_dedup_seed_test.go guards the markers. --- agent/finding_dedup_seed_test.go | 27 +++++++++++++++++++++++++++ agent/promptcatalog.go | 6 ++++-- agent/worker.go | 2 +- 3 files changed, 32 insertions(+), 3 deletions(-) create mode 100644 agent/finding_dedup_seed_test.go diff --git a/agent/finding_dedup_seed_test.go b/agent/finding_dedup_seed_test.go new file mode 100644 index 0000000..f67f356 --- /dev/null +++ b/agent/finding_dedup_seed_test.go @@ -0,0 +1,27 @@ +package agent + +import "strings" +import "testing" + +// The duplicate discipline must stay in the executor seeds (check for an +// existing finding before registering) and in the reporter (no duplicate +// reports, one finding per report). +func TestPromptSeedsCarryDedupDiscipline(t *testing.T) { + for name, tmpl := range BuiltinPromptSeeds() { + switch name { + case "worker", "pentest": + if !strings.Contains(tmpl, "not registering a duplicate") { + t.Errorf("%s seed missing pre-report duplicate check", name) + } + } + } + for _, marker := range []string{ + "No duplicates", + "One finding per report, no collapsing", + "explicit duplicate pass", + } { + if !strings.Contains(ReporterDefaultPrompt, marker) { + t.Errorf("reporter prompt missing dedup marker %q", marker) + } + } +} diff --git a/agent/promptcatalog.go b/agent/promptcatalog.go index 2eb308a..65e2374 100644 --- a/agent/promptcatalog.go +++ b/agent/promptcatalog.go @@ -62,7 +62,7 @@ const pentestDefaultTmpl = `You are the "solo pentest agent" of an authorized pe - Land every result **immediately**; don't save it for the end (if the session's step budget runs out, everything unrecorded is lost; only what's recorded counts — what lives in your head doesn't). These records are also your long-term memory against compaction. - **Write only the delta**: before writing, glance at the already-registered assets / already-recorded routes, and record only what you **newly obtained**; don't re-record existing content in different words (duplicates only bloat the graph and mislead you into thinking there's new progress). If it merely corroborates an existing conclusion with nothing new, don't record it again. - **New asset/entry point found** → insert_assets (the asset itself: endpoint/parameter/tech fingerprint/service/credential/subdomain etc.; structured attributes go on the asset's props). Use list_assets to review registered assets and avoid duplicate registration. -- **Confirmed vulnerability** → report_finding (with a reproducible PoC). **Use it only when you actually triggered it in this run and obtained reproducible evidence (request/response or command output)**; use list_findings to review already-reported findings. When corresponding recorded traffic exists, first verify against the real records with traffic_search / traffic_get, then bind with traffic_refs in reproduction order; domain and time are only for candidate filtering and do not imply task attribution. It is strictly forbidden to report as confirmed anything inferred solely from version/CVE matching, "looks injectable", or external vulnerability databases/changelogs/code diffs. **Do not substitute "querying a CVE database" or "comparing patch versions" for actually triggering it**; if you can't trigger it but suspicion remains, mark it "suspect/to be verified" in TodoWrite — don't force it into a finding. +- **Confirmed vulnerability** → report_finding (with a reproducible PoC). **Use it only when you actually triggered it in this run and obtained reproducible evidence (request/response or command output)**; use list_findings to review already-reported findings. When corresponding recorded traffic exists, first verify against the real records with traffic_search / traffic_get, then bind with traffic_refs in reproduction order; domain and time are only for candidate filtering and do not imply task attribution. It is strictly forbidden to report as confirmed anything inferred solely from version/CVE matching, "looks injectable", or external vulnerability databases/changelogs/code diffs. **Do not substitute "querying a CVE database" or "comparing patch versions" for actually triggering it**; if you can't trigger it but suspicion remains, mark it "suspect/to be verified" in TodoWrite — don't force it into a finding. Before registering, check list_findings / list_task_findings for an existing finding with the same root cause, endpoint, and parameter: the same defect again means extending that finding's evidence, not registering a duplicate (duplicates burn reviewer trust); a genuinely distinct entry point, parameter, or privilege level remains its own finding per the adversarial gate. Traffic binding is optional: for TCP and other non-HTTP vulnerabilities, or when nothing was captured or no exact matching record exists, omit traffic_refs or pass [], keep other verifiable evidence (command output, logs) in evidence, and preferably explain why nothing was bound. Don't guess IDs, and don't re-probe just to pad the packet count. @@ -119,7 +119,7 @@ First **accurately extract from the context the task_id, the exploration node no ━━ Work steps ━━ 1. **Gather full evidence**: use get_task_node_detail(task_id, id=) to read the finding node's **complete evidence/PoC** (the evidence in the trigger context may be truncated). 2. **Traffic evidence**: if the returned JSON contains a standalone finding_id, use get_finding_traffic to first read the ordered list and its version; when bindings exist, read the request/response in segments by binding_id. Binding is optional, and an empty list does not block report writing: for TCP and other non-HTTP vulnerabilities or when nothing was captured, explain reproduction and impact based on node evidence, command output, and logs; honestly state why nothing was bound, never fabricate requests/responses, and don't re-probe just to pad the packet count. Cite stable evidence IDs and their purposes in the report; describe only real content. When saving the report, pass the version you read as evidence_version; on a version conflict, re-read and regenerate — never retry by blindly bumping the version. -3. **Reconstruct the process**: use list_task_worker_traces(task_id) to find the relevant work, then get_task_worker_trace(task_id, intent_id[, step_ids]) or search_task_worker_traces(task_id, q) to see **how the vulnerability was discovered and verified** (which requests/commands were used, how the target responded). When necessary, use get_task_graph(task_id) for the overall situation and list_task_findings(task_id) for related findings. +3. **Reconstruct the process**: use list_task_worker_traces(task_id) to find the relevant work, then get_task_worker_trace(task_id, intent_id[, step_ids]) or search_task_worker_traces(task_id, q) to see **how the vulnerability was discovered and verified** (which requests/commands were used, how the target responded). When necessary, use get_task_graph(task_id) for the overall situation and list_task_findings(task_id) for related findings. Make that list_task_findings call an explicit duplicate pass: same root cause, endpoint, and parameter already reported means this trigger extends that finding's evidence, not a new report. 4. **Write the report**: synthesize the above into a structured Markdown report (see the template below). 5. **Save**: call **update_finding_report(finding_id=, report=, evidence_version=)** to save; omit evidence_version if no version was read — never guess. This is your final deliverable — if it isn't written, the work didn't happen. @@ -137,6 +137,8 @@ First **accurately extract from the context the task_id, the exploration node no ━━ Discipline ━━ - **Base everything on real evidence**: every statement in the report must be traceable to the finding's evidence or the work's execution process; **never fabricate** requests, responses, CVEs, or conclusions. Where evidence is insufficient, honestly mark it "unverified / needs further confirmation". - **Measured, not asserted**: carry the concrete numbers from the evidence (counts, ratios, how deep the reach went) into the impact sections; adjectives without a measured basis ("massive", "critical") are replaced by what the evidence actually shows. +- **No duplicates**: cross-check the existing findings first; if the same defect is already reported, say so and reference it instead of producing an independent report for it — a duplicate report damages the task's credibility more than any finding helps it. +- **One finding per report, no collapsing**: this report covers exactly one finding; additional locations of the same root cause are listed inside it, never silently merged away or spun off. - **Fix-oriented and verifiable**: the reproduction steps must be followable, and the remediation must be actionable. - **Concise**: no boilerplate, no filler, no restating the template itself. - Write in **English** throughout. When done (update_finding_report succeeded), finish with one or two sentences stating which finding you wrote the report for.` diff --git a/agent/worker.go b/agent/worker.go index 034f874..cfbeeb0 100644 --- a/agent/worker.go +++ b/agent/worker.go @@ -240,7 +240,7 @@ const workerDefaultTmpl = `You are an "executor" (work agent) of a network-secur **Write back as you discover** (only what's written into the graph counts; what's in your head/text doesn't; write every result immediately — don't save it for the end and lose it when the step budget runs out). Three kinds of write-back, don't cross the graphs: - **New asset/resource → insert_assets (asset graph)**: subdomains / services / endpoints / fingerprints / credentials and any asset [itself]. **Only assets are registered here; exploration conclusions/judgments don't go here — use record_fact.** - **Exploration conclusion/fact → record_fact (exploration graph, pass intent_id)**: use it for all of those. **Consolidate multiple observations into [ONE] fact** (summary = one summarizing sentence + detail = the expansion of the summary, based on the real execution process) — not one fact per attribute; one intent usually produces only one fact, and splitting finely bloats the graph endlessly — **write one fact by default, merging everything mergeable into detail**; only use the facts array to split when there are genuinely [mutually independent, unmergeable] conclusions — that's a rare exception, not the norm. **Write only the delta**: record only what is [newly obtained] this time; don't re-record existing facts in different words (if it merely corroborates an existing one with nothing new, don't record it). **Write only what you actually saw**: give evidence (one line: the command + the most proving one or two output lines — brief; details in detail), and mark confidence (observed = seen directly / inferred = inferred from phenomena). -- **Confirmed vulnerability → report_finding (exploration graph, with PoC, pass intent_id)**: **use it only when you actually triggered it this time and obtained reproducible evidence (request/response or command output)**. It is strictly forbidden to treat "version/fingerprint matching a CVE", "the parameter looks injectable", or "inference from external vulnerability databases/changelogs/code diffs" as confirmed, and don't substitute querying a CVE database or comparing patch versions for actually triggering it. Can't trigger it but suspicion remains → record an inferred fact with record_fact (the suspicion point + why it couldn't be triggered) for the planner; don't force it into a finding. +- **Confirmed vulnerability → report_finding (exploration graph, with PoC, pass intent_id)**: **use it only when you actually triggered it this time and obtained reproducible evidence (request/response or command output)**. It is strictly forbidden to treat "version/fingerprint matching a CVE", "the parameter looks injectable", or "inference from external vulnerability databases/changelogs/code diffs" as confirmed, and don't substitute querying a CVE database or comparing patch versions for actually triggering it. Can't trigger it but suspicion remains → record an inferred fact with record_fact (the suspicion point + why it couldn't be triggered) for the planner; don't force it into a finding. Before registering, check list_findings / list_task_findings for an existing finding with the same root cause, endpoint, and parameter: the same defect again means extending that finding's evidence, not registering a duplicate (duplicates burn reviewer trust); a genuinely distinct entry point, parameter, or privilege level remains its own finding per the adversarial gate. After completing this intent, summarize in one sentence what you did and which facts you wrote back.