mirror of
https://github.com/BaaRai/ARTEX.git
synced 2026-11-13 19:50:54 +08:00
Compare commits
20
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e6f4de5ddc | ||
|
|
ac601b7e59 | ||
|
|
9ebc8b6703 | ||
|
|
1c35c9cf68 | ||
|
|
1ed154e86c | ||
|
|
4034ce4562 | ||
|
|
7f8c7cbcae | ||
|
|
9e63a8031b | ||
|
|
978f308a9e | ||
|
|
80a1ee96f6 | ||
|
|
b3717f7a5f | ||
|
|
5d3b1d142d | ||
|
|
2d5f9677f4 | ||
|
|
faf76d5f10 | ||
|
|
27d2241cda | ||
|
|
70f9ea3f01 | ||
|
|
271deabeb0 | ||
|
|
e6b4f6c5f4 | ||
|
|
c7bef98689 | ||
|
|
b5bee2b8d2 |
+1
-1
@@ -1,5 +1,5 @@
|
||||
# Docker deployment configuration (copy to .env)
|
||||
ARTEX_TAG=latest
|
||||
# The image is built locally by `docker compose build`; there is no registry tag.
|
||||
POSTGRES_USER=artex
|
||||
POSTGRES_PASSWORD=change-me-please
|
||||
POSTGRES_DB=artex
|
||||
|
||||
@@ -95,13 +95,18 @@ jobs:
|
||||
files: |
|
||||
dist/*.zip
|
||||
dist/SHA256SUMS
|
||||
generate_release_notes: true
|
||||
generate_release_notes: false
|
||||
body: "See CHANGELOG.md for the list of changes in this release."
|
||||
|
||||
# ④ Multi-arch image → push to Docker Hub (reuses the Linux binaries built by the
|
||||
# binaries job; the image only installs tools + ships the binary; arm64 only needs
|
||||
# to emulate the apt layer, so builds are much faster)
|
||||
# Publishing the image needs Docker Hub credentials and a namespace we own, so the
|
||||
# job stays skipped unless the repository defines DOCKERHUB_ENABLED (set it to true
|
||||
# and provide DOCKERHUB_USERNAME / DOCKERHUB_TOKEN secrets plus DOCKERHUB_IMAGE).
|
||||
docker:
|
||||
needs: binaries
|
||||
if: ${{ vars.DOCKERHUB_ENABLED == 'true' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4 # fetch the Dockerfile and skills/
|
||||
@@ -122,7 +127,7 @@ jobs:
|
||||
- uses: docker/metadata-action@v5
|
||||
id: meta
|
||||
with:
|
||||
images: autumn27/artex
|
||||
images: ${{ vars.DOCKERHUB_IMAGE }}
|
||||
tags: |
|
||||
type=ref,event=tag
|
||||
type=raw,value=latest
|
||||
|
||||
+50
-1
@@ -4,6 +4,54 @@ All notable changes to this project are documented in this file. The format is b
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
|
||||
- **The image is built from the checkout instead of pulled**: `docker-compose.yml` no longer references the `autumn27/artex` namespace, whose repository is gone and whose name its holder could republish at any time. `install.sh` and `update.sh` build the image locally.
|
||||
- **`update.sh` refuses to rebuild stale code**: when `git pull --ff-only` fails on a diverged clone (including every clone made before the 2026-10-23 history rewrite), it now stops with recovery commands instead of warning, rebuilding the old tree and printing "Build complete".
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Ollama base URL on the environment path**: `ARTEX_LLM_BASE_URL=http://host:11434` is normalized to the `/v1` root the same way the saved-profile path already was, so a LAN Ollama server configured through the environment no longer answers 404 on chat.
|
||||
|
||||
### Added
|
||||
|
||||
- **Ollama support**: an `ollama` LLM profile format for a local or LAN Ollama server. Keyless by design: no API key is needed, `OPENAI_API_KEY` is never inherited, the base URL defaults to `http://127.0.0.1:11434/v1` and is normalized to the `/v1` root, and the profile works in the failover pool, task LLM chains, model listing and connection tests. Contributed by @glauberportella (#128).
|
||||
|
||||
## [0.4.0] - 2026-10-23
|
||||
|
||||
First release of this fork. The engine underneath is still upstream's, but this tree carries a long stretch of operational work on top of it: a recon toolbox and skills, findings discipline, cost accounting, and container packaging.
|
||||
|
||||
### Added
|
||||
|
||||
- **Recon toolbox in the image**: nuclei, httpx, subfinder, katana, ffuf and dalfox installed at release-pinned versions, plus a sqlmap checkout that tracks the default branch, each smoke-tested at build time (run once, not checksum-verified), so hunt workers can call them directly (#112). Checksum verification arrives with #127.
|
||||
- **Recursive recon skill** (`bbot-recon`): bbot v3.0.2 in its own virtualenv with module dependencies preinstalled. Policy-gated modes (passive seeds-only when scanners are banned, active-safe, loud only on explicit permission) and JSONL ingestion into the facts graph (#118).
|
||||
- **php2shell skill**: turns a file-inclusion finding into verified RCE through the session.upload_progress chain, lab first and gated on program policy (#113).
|
||||
- **Code audit and reverse-engineering skills** (#55, #77).
|
||||
- **Goal ambiguity gate**: task creation rejects goals that name no target and goals under 80 characters; `[targetless]` opts out for tasks that legitimately have none (#111).
|
||||
- **LLM chain hardening**: consecutive-failure escalation across the ordered profile chain, plus per-intent profile rotation with a strict round-robin (`llm_rotate`) (#111, #114).
|
||||
- **Stagnation warning**: the graph overview tells a worker when it has repeated the same direction without producing findings (#111).
|
||||
- **Per-task credentials vault**: accounts stored per task, agent-generated passwords allowed, injected into the worker context and managed from the task page.
|
||||
- **Cost ledger**: token spend metered per call, with usage surfaced in the UI and judge/fallback calls counted separately (#71, #74).
|
||||
- **Backup script and MCP health reporting** (#43, #44).
|
||||
- **Ask-operator tool with environment handoff** (#45, #57).
|
||||
- **DrostBench corpus and benchmark harness** (#59, #95, #96, #97, #98).
|
||||
- **One database per test package**, removing cross-package test races (#82).
|
||||
|
||||
### Changed
|
||||
|
||||
- **Norma upgraded to v0.4.4**: per-task process-tree memory watchdog, UTF-8 aligned output truncation, non-interactive subprocess environments, message_delta usage recovery, and streaming fixes (#121).
|
||||
- **Planner quiet mode** for less narration during long runs (#81, #88).
|
||||
- **Findings discipline**: proof-basis gates, measured impact, adversarial verification and dedup rules decide what a finding must contain before it is reported (#51, #53, #63, #69).
|
||||
- **Prompt-injection hardening** on seed material and MCP tool descriptions (#49).
|
||||
- **Pause preserves the task timeout budget**, and paused time is frozen in the task clock (#2, #110).
|
||||
- **Deadline contract** in the executor seeds (#65).
|
||||
- **Task detail metrics and outcome cards** summarise a finished run (#92, #102).
|
||||
- **Skill-gap and loop-contract reporting** for the planner (#104, #106); the RSI notes land as documentation in #108.
|
||||
|
||||
### Removed
|
||||
|
||||
- **In-app self-update and the automatic version check**: the update source was hard-coded to a repository that no longer exists, and its checksums came from the same release. The `selfupdate` package, the `/api/update/*` endpoints, the top-bar update badge and the "Version & Updates" settings card are gone; `start.sh` / `start.bat` now only restart the process after a crash. Upgrade with `update.sh` or a release archive. (#125, contributed by @stevenAthompson)
|
||||
|
||||
## [0.3.15] - 2026-10-07
|
||||
|
||||
### Interception
|
||||
@@ -886,7 +934,8 @@ All notable changes to this project are documented in this file. The format is b
|
||||
- [@Autumn-27](https://github.com/Autumn-27)
|
||||
- [@neouks](https://github.com/neouks)
|
||||
|
||||
[Unreleased]: https://github.com/Autumn-27/ARTEX/compare/v0.3.10...HEAD
|
||||
[Unreleased]: https://github.com/BaaRai/ARTEX/compare/v0.4.0...HEAD
|
||||
[0.4.0]: https://github.com/BaaRai/ARTEX/compare/v0.3.15...v0.4.0
|
||||
[0.3.10]: https://github.com/Autumn-27/ARTEX/compare/v0.3.9...v0.3.10
|
||||
[0.3.9]: https://github.com/Autumn-27/ARTEX/compare/v0.3.8...v0.3.9
|
||||
[0.3.8]: https://github.com/Autumn-27/ARTEX/compare/v0.3.7...v0.3.8
|
||||
|
||||
+1
-2
@@ -85,8 +85,7 @@ RUN npm install -g @playwright/mcp@latest @playwright/cli@latest playwright@late
|
||||
WORKDIR /app
|
||||
# pre-built binary for the matching architecture (dist/amd64/artex or dist/arm64/artex)
|
||||
COPY dist/${TARGETARCH}/artex /app/artex
|
||||
# Daemon startup script: decides whether to relaunch the process based on its exit code;
|
||||
# the in-page one-click update relies on it to swap the binary. It also forwards SIGTERM
|
||||
# Daemon startup script: relaunches the process after a crash and forwards SIGTERM
|
||||
# to artex — docker stop only signals PID 1; without forwarding, artex never receives it,
|
||||
# cannot shut down gracefully, and is hard-killed by SIGKILL after 10 seconds.
|
||||
COPY start.sh /app/start.sh
|
||||
|
||||
@@ -84,7 +84,7 @@ cd ARTEX
|
||||
|
||||
The script will: detect / auto-install Docker → then let you choose **① all-Docker** or **② local build and run**:
|
||||
|
||||
- **① All-Docker**: enter a Postgres password (press Enter for a random one) → auto-writes `.env` → `docker compose up -d`.
|
||||
- **① All-Docker**: enter a Postgres password (press Enter for a random one) → auto-writes `.env` → `docker compose up -d --build` (the image is built from this checkout).
|
||||
- **② Local run**: choose a database (connect to an existing one / start one with Docker) → generates `config.json` → `go` builds the embedded single binary → starts.
|
||||
|
||||
Once installed, open **http://localhost:8787** (on first entry, `/setup` sets the admin password).
|
||||
@@ -95,10 +95,14 @@ Once installed, open **http://localhost:8787** (on first entry, `/setup` sets th
|
||||
git clone https://github.com/BaaRai/ARTEX.git
|
||||
cd ARTEX
|
||||
cp .env.example .env # set POSTGRES_PASSWORD, optionally ANTHROPIC_API_KEY
|
||||
docker compose up -d # pulls the autumn27/artex image + postgres
|
||||
# build the single binary first (frontend, then go build -tags embedui), see the Dockerfile header
|
||||
docker compose up -d --build # builds artex from this checkout + starts postgres
|
||||
# → http://localhost:8787
|
||||
```
|
||||
|
||||
> The image is built locally. There is no published image to pull: the previous
|
||||
> default pointed at a third-party namespace that no longer hosts this project.
|
||||
|
||||
The image already includes common tools (ripgrep/curl/vim/npm/nmap…); `./skills` and `./data` are persisted via bind mounts.
|
||||
|
||||
For remote MCP, you can choose `http` (Streamable HTTP) or `sse` (legacy SSE) in system settings.
|
||||
@@ -115,7 +119,7 @@ cp config.example.json config.json # fill in the database connection
|
||||
./start.sh # → http://localhost:8787
|
||||
```
|
||||
|
||||
> Please start with `start.sh` / `start.bat` instead of running `./artex` directly. It is a daemon script: after the program exits it decides whether to relaunch based on the exit code, and **the in-page [one-click update](#option-1-one-click-update-from-the-page-recommended) relies on it to swap the binary**. If you run `./artex` directly, it will not be relaunched after an update.
|
||||
> Please start with `start.sh` / `start.bat` instead of running `./artex` directly. It is a daemon script: if the process crashes, it restarts it after a backoff.
|
||||
> To keep it running in the background: `nohup ./start.sh >artex.log 2>&1 &`.
|
||||
|
||||
### Option 4: build the single binary from source
|
||||
@@ -175,20 +179,20 @@ Daily cron example (03:20, operator crontab):
|
||||
|
||||
> Upgrading only swaps the program and never touches data: the Postgres volume `pgdata`, `./data` (jwt.key / SQLite, etc.), and `./skills` are all preserved. **No manual database migration is needed**: `artex` idempotently re-runs `schema.sql` on every start (with `ADD COLUMN` / `CREATE INDEX IF NOT EXISTS`), i.e. "restart equals migrate". It is still recommended to back up `./data` and the database before upgrading.
|
||||
|
||||
### Option 1: one-click update from the page (recommended)
|
||||
### Upgrading from a clone made before 2026-10-23
|
||||
|
||||
On the **System Settings** page (sidebar "System Settings" → `/system/settings`), the **Version & Updates** card lets you check for and install new versions directly, without logging into the server.
|
||||
The repository history was rewritten that day to drop a personal email address from the commit metadata. A clone made before then cannot fast-forward, and `./update.sh` now stops instead of silently rebuilding stale code. Recover once, then upgrade normally:
|
||||
|
||||
After clicking "Update": it downloads the release package for the current platform → verifies it against the release's `SHA256SUMS` → smoke-tests the new binary with `-h` → stages it as `artex.new` → the program exits and is relaunched by `start.sh` / `start.bat`, which completes the swap. The page automatically waits for the new version to come online, then refreshes.
|
||||
```bash
|
||||
git stash # only if you have local changes
|
||||
git fetch origin
|
||||
git reset --hard origin/main
|
||||
git stash pop # only if you stashed
|
||||
```
|
||||
|
||||
- **A failure never leaves a broken program**: if verification or the smoke test fails, the staged file is discarded and the current version keeps running; if the swapped-in new version fails to start 3 times in a row, it automatically rolls back to `artex.old` (the failed one is kept as `artex.failed` for troubleshooting).
|
||||
- **Rollback anytime**: the previous version is kept as `artex.old`, and the card has "Roll back to previous version". Note that the database schema is not rolled back.
|
||||
- **Updating interrupts running tasks**: updating means restarting, so do it when the system is idle.
|
||||
- **Dev builds can't update**: updating is disabled when the version is `dev` or a `git describe` output with a suffix, preventing a release from overwriting your local debug binary.
|
||||
- **Under Docker, only the program is swapped, not the image**: toolchains inside the image like playwright / nmap are not upgraded along with it, and recreating the container with `docker compose up -d` reverts to the version bundled in the image. To upgrade the image as well, still use `docker compose pull artex && docker compose up -d artex`.
|
||||
- If accessing GitHub requires a proxy, configure the **global proxy** on the same page and the update chain will use it. Updates only download from GitHub domains and enforce HTTPS.
|
||||
Your data is not affected: `./data`, the `pgdata` volume and `./skills` are not part of the git history.
|
||||
|
||||
### Option 2: one-click update script
|
||||
### Option 1: one-click update script
|
||||
|
||||
```bash
|
||||
cd ARTEX
|
||||
@@ -197,21 +201,20 @@ cd ARTEX
|
||||
|
||||
The script first optionally `git pull`s the latest code, then lets you choose **① Docker update** or **② local build update** (matching `install.sh`):
|
||||
|
||||
- **① Docker**: you can specify a target image tag (press Enter to keep `.env`'s `ARTEX_TAG`, default `latest`) → `docker compose pull` → `docker compose up -d` (restarting with the new image migrates automatically).
|
||||
- **① Docker**: rebuilds the image from this checkout → `docker compose build artex` → `docker compose up -d artex` (restarting migrates the schema automatically).
|
||||
- **② Local**: rebuild the frontend static assets → recompile `./artex` (restart the process for it to take effect).
|
||||
|
||||
### Option 3: Docker Compose (manual)
|
||||
### Option 2: Docker Compose (manual)
|
||||
|
||||
```bash
|
||||
cd ARTEX
|
||||
git pull # update compose / scripts (optional)
|
||||
# pin a version: set ARTEX_TAG=v0.2.0 in .env; otherwise latest is used
|
||||
docker compose pull artex
|
||||
docker compose up -d artex # restart with the new image → schema migrates automatically
|
||||
git pull # update compose / scripts
|
||||
docker compose build artex # rebuild the image from this checkout
|
||||
docker compose up -d artex # recreate the container → schema migrates automatically
|
||||
docker image prune -f # clean up old images (optional)
|
||||
```
|
||||
|
||||
### Option 4: prebuilt binaries (Releases)
|
||||
### Option 3: prebuilt binaries (Releases)
|
||||
|
||||
Download the new version's zip from [Releases](https://github.com/BaaRai/ARTEX/releases), stop the old process, then overwrite `artex` and `skills/` (keep your `config.json` and `data/`), and restart:
|
||||
|
||||
@@ -220,7 +223,7 @@ cp -r <extracted dir>/skills ./ && cp <extracted dir>/artex ./
|
||||
./start.sh
|
||||
```
|
||||
|
||||
### Option 5: build from source
|
||||
### Option 4: build from source
|
||||
|
||||
```bash
|
||||
git pull
|
||||
@@ -246,8 +249,8 @@ CGO_ENABLED=0 go build -tags embedui -o artex ./cmd/artex
|
||||
}
|
||||
```
|
||||
|
||||
**LLM**: `export ANTHROPIC_API_KEY=sk-...` (or `OPENAI_API_KEY`); you can also fill it in on the UI's "LLM Configuration" page.
|
||||
Optional: `ARTEX_LLM_PROVIDER` / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ARTEX_LLM_PROXY`.
|
||||
**LLM**: `export ANTHROPIC_API_KEY=sk-...` (or `OPENAI_API_KEY`); you can also fill it in on the UI's "LLM Configuration" page — which can add a local Ollama server as a keyless provider (format "Ollama", base URL default `http://127.0.0.1:11434/v1`, models loaded from the server).
|
||||
Optional: `ARTEX_LLM_PROVIDER` (anthropic | openai | openai-responses | ollama) / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ARTEX_LLM_PROXY`.
|
||||
|
||||
**Concurrency**: the number of work agents per task is configured in "System Settings" (default 3).
|
||||
|
||||
|
||||
+94
-10
@@ -3,8 +3,10 @@
|
||||
// §4.3 (planner) and §4.4 (work agent).
|
||||
//
|
||||
// Provider configuration is read from the environment so the system runs with
|
||||
// any Anthropic- or OpenAI-format endpoint. If no key is configured, FromEnv
|
||||
// returns ok=false and the exploration engine stays idle (an LLM is required).
|
||||
// any Anthropic-, OpenAI- or Ollama-format endpoint (the last being keyless).
|
||||
// If no usable configuration is found — a key for the cloud formats, a base
|
||||
// URL for keyless Ollama — FromEnv returns ok=false and the exploration engine
|
||||
// stays idle (an LLM is required).
|
||||
package agent
|
||||
|
||||
import (
|
||||
@@ -125,6 +127,47 @@ const (
|
||||
maxWindowK = 1000 // cap at 1M tokens (user request)
|
||||
)
|
||||
|
||||
// FormatOllama marks a LOCAL Ollama server profile. Ollama exposes an
|
||||
// OpenAI-compatible Chat Completions endpoint (/v1), so on the wire it speaks
|
||||
// llm.FormatOpenAI; the distinct sentinel matters everywhere a profile must be
|
||||
// treated as usable WITHOUT an API key — Ollama runs keyless on localhost/LAN.
|
||||
// See NeedsAPIKey and NewProvider (which maps it onto FormatOpenAI while
|
||||
// forcing our base-URL/key defaults so an empty key never falls back to
|
||||
// OpenAI's public endpoint).
|
||||
const FormatOllama llm.Format = "ollama"
|
||||
|
||||
// DefaultOllamaBaseURL is the base of Ollama's OpenAI-compatible endpoint the
|
||||
// provider appends /chat/completions to. Must never end up empty: with an
|
||||
// empty BaseURL, the SDK would fall back to OpenAI's public endpoint.
|
||||
const DefaultOllamaBaseURL = "http://127.0.0.1:11434/v1"
|
||||
|
||||
// ollamaPlaceholderKey is sent as the Bearer token when no key is configured.
|
||||
// Ollama ignores authorization, but a concrete value keeps the SDK from
|
||||
// falling back to OPENAI_API_KEY (which would silently ship a real cloud key
|
||||
// to the local server) and keeps the header stable.
|
||||
const ollamaPlaceholderKey = "ollama"
|
||||
|
||||
// normalizeOllamaBaseURL maps whatever the operator typed onto the /v1 root the
|
||||
// OpenAI-compatible provider expects: it strips a pasted "/chat/completions"
|
||||
// suffix, an empty value becomes the local default, and a missing "/v1" is
|
||||
// appended so the natural "http://host:11434" also works. Shared by the saved
|
||||
// profile path (ConfigFrom) and the environment path (FromEnv): both used to
|
||||
// differ, and only the profile path normalized.
|
||||
func normalizeOllamaBaseURL(raw string) string {
|
||||
b := strings.TrimRight(strings.TrimSuffix(strings.TrimSpace(raw), "/chat/completions"), "/")
|
||||
if b == "" {
|
||||
return DefaultOllamaBaseURL
|
||||
}
|
||||
if !strings.HasSuffix(b, "/v1") {
|
||||
b += "/v1"
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// NeedsAPIKey reports whether this provider format requires an API key to be
|
||||
// usable. Ollama (local server) is keyless; every cloud format needs one.
|
||||
func (c Config) NeedsAPIKey() bool { return c.Format != FormatOllama }
|
||||
|
||||
// CompactionWindow returns the model context window in TOKENS for compaction
|
||||
// thresholds, resolved from the user-configured size (ContextWindowK). 0/unset →
|
||||
// a 200K default; otherwise clamped to [32K, 1M] so compaction stays effective.
|
||||
@@ -154,11 +197,11 @@ func compactionConfig(windowTokens int) *compaction.Config {
|
||||
|
||||
// FromEnv resolves the LLM provider config:
|
||||
//
|
||||
// ARTEX_LLM_PROVIDER = anthropic|openai (default: inferred from keys)
|
||||
// ARTEX_LLM_PROVIDER = anthropic|openai|openai-responses|ollama (default: inferred from keys)
|
||||
// ARTEX_LLM_MODEL = model id (default: per provider)
|
||||
// ARTEX_LLM_BASE_URL = endpoint (optional)
|
||||
// ARTEX_LLM_BASE_URL = endpoint (optional; ollama defaults to localhost:11434/v1)
|
||||
// ARTEX_LLM_PROXY = proxy URL (optional; http/https/socks5)
|
||||
// ANTHROPIC_API_KEY / OPENAI_API_KEY = credentials
|
||||
// ANTHROPIC_API_KEY / OPENAI_API_KEY = credentials (unused for ollama)
|
||||
func FromEnv() (Config, bool) {
|
||||
prov := os.Getenv("ARTEX_LLM_PROVIDER")
|
||||
anthKey := os.Getenv("ANTHROPIC_API_KEY")
|
||||
@@ -196,6 +239,16 @@ func FromEnv() (Config, bool) {
|
||||
if c.Model == "" {
|
||||
c.Model = "gpt-5"
|
||||
}
|
||||
case "ollama":
|
||||
c.Format = FormatOllama
|
||||
// Deliberately NOT inheriting OPENAI_API_KEY: a local Ollama ignores the
|
||||
// bearer token, so the only thing the inheritance could do is ship a real
|
||||
// cloud key to the endpoint. A gateway that does require a key can still
|
||||
// be configured as a saved profile with an explicit one.
|
||||
c.BaseURL = normalizeOllamaBaseURL(c.BaseURL)
|
||||
if c.Model == "" {
|
||||
c.Model = "llama3.2"
|
||||
}
|
||||
default:
|
||||
c.Format = llm.FormatAnthropic
|
||||
c.APIKey = anthKey
|
||||
@@ -203,7 +256,8 @@ func FromEnv() (Config, bool) {
|
||||
c.Model = "claude-opus-4-8"
|
||||
}
|
||||
}
|
||||
if c.APIKey == "" {
|
||||
// Ollama runs without a key: only the cloud formats gate on one.
|
||||
if c.APIKey == "" && c.NeedsAPIKey() {
|
||||
return Config{}, false
|
||||
}
|
||||
return c, true
|
||||
@@ -236,6 +290,16 @@ func ConfigFrom(provider, model, baseURL, apiKey, proxy string) Config {
|
||||
if c.Model == "" {
|
||||
c.Model = "gpt-5"
|
||||
}
|
||||
case "ollama":
|
||||
c.Format = FormatOllama
|
||||
// Ollama's OpenAI-compatible endpoint lives at /v1 and the provider
|
||||
// appends "/chat/completions". Never leave it empty: an empty BaseURL
|
||||
// would let the SDK fall back to OpenAI's public endpoint with this
|
||||
// profile's request.
|
||||
c.BaseURL = normalizeOllamaBaseURL(c.BaseURL)
|
||||
if c.Model == "" {
|
||||
c.Model = "llama3.2"
|
||||
}
|
||||
default:
|
||||
c.Format = llm.FormatAnthropic
|
||||
// provider appends "/v1/messages".
|
||||
@@ -257,13 +321,16 @@ func isFalsy(s string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// Provider returns the short provider name ("anthropic"/"openai").
|
||||
// Provider returns the short provider name
|
||||
// ("anthropic"/"openai"/"openai-responses"/"ollama").
|
||||
func (c Config) Provider() string {
|
||||
switch c.Format {
|
||||
case llm.FormatOpenAI:
|
||||
return "openai"
|
||||
case llm.FormatOpenAIResponses:
|
||||
return "openai-responses"
|
||||
case FormatOllama:
|
||||
return "ollama"
|
||||
}
|
||||
return "anthropic"
|
||||
}
|
||||
@@ -276,10 +343,27 @@ func (c Config) NewProvider() (llm.Provider, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
wireFormat := c.Format
|
||||
baseURL := c.BaseURL
|
||||
apiKey := c.APIKey
|
||||
// Ollama speaks OpenAI's Chat Completions on the wire. Bundle the keyless
|
||||
// defaults here so an Ollama config can never leak toward the SDK's env-
|
||||
// driven fallbacks: BaseURL to OpenAI's public endpoint, APIKey to
|
||||
// OPENAI_API_KEY. A local Ollama ignores the placeholder token; a gateway
|
||||
// that requires one carries a real key in the profile.
|
||||
if wireFormat == FormatOllama {
|
||||
wireFormat = llm.FormatOpenAI
|
||||
if strings.TrimSpace(baseURL) == "" {
|
||||
baseURL = DefaultOllamaBaseURL
|
||||
}
|
||||
if strings.TrimSpace(apiKey) == "" {
|
||||
apiKey = ollamaPlaceholderKey
|
||||
}
|
||||
}
|
||||
lc := llm.Config{
|
||||
Format: c.Format,
|
||||
BaseURL: c.BaseURL,
|
||||
APIKey: c.APIKey,
|
||||
Format: wireFormat,
|
||||
BaseURL: baseURL,
|
||||
APIKey: apiKey,
|
||||
Model: c.Model,
|
||||
HTTPClient: client,
|
||||
}
|
||||
|
||||
+6
-33
@@ -18,7 +18,6 @@ import (
|
||||
|
||||
"github.com/Autumn-27/artex/agent"
|
||||
"github.com/Autumn-27/artex/config"
|
||||
"github.com/Autumn-27/artex/selfupdate"
|
||||
"github.com/Autumn-27/artex/server"
|
||||
)
|
||||
|
||||
@@ -42,10 +41,8 @@ func printBanner(addr string) {
|
||||
version, runtime.GOOS, runtime.GOARCH, runtime.Version(), addr)
|
||||
}
|
||||
|
||||
// main only maps run's result onto the process exit code. The exit code is part
|
||||
// of the update protocol — the supervising start script reads it to decide
|
||||
// whether to relaunch us (see selfupdate.ExitRestart) — so the body has to live
|
||||
// in a function that can *return* rather than os.Exit past its own defers.
|
||||
// main maps run's result onto the process exit code. The body lives in a
|
||||
// function that can *return* rather than os.Exit, so its own defers still run.
|
||||
func main() {
|
||||
os.Exit(run())
|
||||
}
|
||||
@@ -68,16 +65,6 @@ func run() int {
|
||||
// page can show a live log stream. Do this first, to catch startup logs too.
|
||||
server.StartLogCapture()
|
||||
|
||||
// Self-update bootstrap: swap in a staged binary, or count a post-swap boot
|
||||
// attempt and roll back if the new build keeps dying. Must run before we open
|
||||
// the stores or bind a port — this may end with "exit and let the start script
|
||||
// relaunch me", and there is no point paying for either first.
|
||||
action, upState := selfupdate.Bootstrap()
|
||||
server.SetBootUpdateState(upState)
|
||||
if action == selfupdate.Restart {
|
||||
return selfupdate.ExitRestart
|
||||
}
|
||||
|
||||
// surface which config file the binary reads (absolute, so `go run`'s relative
|
||||
// "config.json" — resolved against the CWD — is unambiguous).
|
||||
cfgPath := config.Path()
|
||||
@@ -101,18 +88,12 @@ func run() int {
|
||||
}
|
||||
defer mgr.Close()
|
||||
|
||||
// Surviving this long means a freshly swapped-in build actually works, so drop
|
||||
// the upgrade marker and stop counting attempts. Until it fires, every boot
|
||||
// increments the count and a build that keeps dying gets rolled back.
|
||||
settle := time.AfterFunc(selfupdate.SettleDelay, selfupdate.Settle)
|
||||
defer settle.Stop()
|
||||
|
||||
skillDir := config.SkillDir()
|
||||
if abs, err := filepath.Abs(skillDir); err == nil {
|
||||
skillDir = abs
|
||||
}
|
||||
log.Printf("[config] skill directory: %s", skillDir)
|
||||
srv := server.New(ctx, mgr, skillDir, *dataDir, config.BaseDir())
|
||||
srv := server.New(ctx, mgr, skillDir, *dataDir, config.BaseDir(), server.WithSecretScrub())
|
||||
httpSrv := &http.Server{
|
||||
Addr: *addr,
|
||||
Handler: srv.Handler(),
|
||||
@@ -126,22 +107,14 @@ func run() int {
|
||||
}
|
||||
}()
|
||||
|
||||
// Two ways out: a signal (normal stop → exit 0, the start script stops looping)
|
||||
// or a staged update / rollback (→ exit 75, the script relaunches us and the
|
||||
// bootstrap above installs the new build).
|
||||
code := 0
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
case <-server.RestartRequested():
|
||||
code = selfupdate.ExitRestart
|
||||
shutdown(agent.AbortShutdown)
|
||||
}
|
||||
// Wait for a stop signal.
|
||||
<-ctx.Done()
|
||||
|
||||
log.Println("shutting down...")
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
_ = httpSrv.Shutdown(shutdownCtx)
|
||||
return code
|
||||
return 0
|
||||
}
|
||||
|
||||
// shutdownContext deliberately does not derive from signalCtx. If it did, the
|
||||
|
||||
@@ -7,6 +7,8 @@ import (
|
||||
"github.com/Autumn-27/artex/agent"
|
||||
)
|
||||
|
||||
// The shutdown cause must survive the parent context being cancelled first;
|
||||
// losing it would drop the diagnostic on every graceful stop.
|
||||
func TestShutdownContextPreservesNamedCause(t *testing.T) {
|
||||
signalCtx, signalCancel := context.WithCancel(context.Background())
|
||||
ctx, shutdown := shutdownContext(signalCtx)
|
||||
|
||||
+5
-4
@@ -143,13 +143,14 @@ func (d *DB) ProfileByID(id int64) (*LLMProfile, error) {
|
||||
}
|
||||
|
||||
// PoolProfiles returns the failover chain in run order, api keys included: the
|
||||
// active profile first, then every other keyed profile that isn't excluded, by
|
||||
// active profile first, then every other usable profile that isn't excluded, by
|
||||
// priority DESC (id ASC to stay stable). Profiles without an api key can't serve
|
||||
// a request, so they never enter the chain. The ordering IS the policy — callers
|
||||
// walk the slice front to back.
|
||||
// a request — except Ollama ('ollama'), which is a keyless local server — so
|
||||
// they never enter the chain. The ordering IS the policy — callers walk the
|
||||
// slice front to back.
|
||||
func (d *DB) PoolProfiles() ([]*LLMProfile, error) {
|
||||
rows, err := d.Query(`SELECT ` + profileColsKey + ` FROM llm_profiles
|
||||
WHERE COALESCE(api_key,'') <> '' AND (is_default OR NOT pool_exclude)
|
||||
WHERE (COALESCE(api_key,'') <> '' OR format='ollama') AND (is_default OR NOT pool_exclude)
|
||||
ORDER BY is_default DESC, priority DESC, id ASC`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
+5
-4
@@ -363,7 +363,7 @@ CREATE TABLE IF NOT EXISTS settings (
|
||||
CREATE TABLE IF NOT EXISTS llm_profiles (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
format TEXT NOT NULL CHECK (format IN ('openai','anthropic','openai-responses')),
|
||||
format TEXT NOT NULL CHECK (format IN ('openai','anthropic','openai-responses','ollama')),
|
||||
base_url TEXT,
|
||||
proxy TEXT,
|
||||
model TEXT NOT NULL,
|
||||
@@ -416,11 +416,12 @@ ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS priority INTEGER NOT NULL
|
||||
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS pool_exclude BOOLEAN NOT NULL DEFAULT false;
|
||||
-- Streaming toggle; backfills old databases. Default true = keep existing streaming behavior; old configs upgrade transparently.
|
||||
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS streaming BOOLEAN NOT NULL DEFAULT true;
|
||||
-- Relax the format constraint to allow openai-responses (OpenAI Responses API); backfills old databases.
|
||||
-- Runs at every startup, idempotent: drop the old CHECK first, then create the new three-value CHECK.
|
||||
-- Relax the format constraint to allow openai-responses (OpenAI Responses API) and
|
||||
-- ollama (local Ollama server, keyless OpenAI-compatible endpoint); backfills old databases.
|
||||
-- Runs at every startup, idempotent: drop the old CHECK first, then create the new four-value CHECK.
|
||||
ALTER TABLE llm_profiles DROP CONSTRAINT IF EXISTS llm_profiles_format_check;
|
||||
ALTER TABLE llm_profiles ADD CONSTRAINT llm_profiles_format_check
|
||||
CHECK (format IN ('openai','anthropic','openai-responses'));
|
||||
CHECK (format IN ('openai','anthropic','openai-responses','ollama'));
|
||||
|
||||
-- Output cap and its field name; backfills old databases. Defaults 0 / '' = send no cap and keep the max_tokens field name;
|
||||
-- old configs behave exactly as before.
|
||||
|
||||
+8
-1
@@ -15,7 +15,14 @@ services:
|
||||
restart: unless-stopped
|
||||
|
||||
artex:
|
||||
image: autumn27/artex:${ARTEX_TAG:-latest}
|
||||
# Built from this checkout. The previous reference was a third-party
|
||||
# namespace whose repository is gone: whoever controls that name could
|
||||
# publish the tag this file would pull.
|
||||
build:
|
||||
context: .
|
||||
args:
|
||||
ARTEX_VERSION: ${ARTEX_VERSION:-dev}
|
||||
image: artex:local
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
|
||||
@@ -0,0 +1,402 @@
|
||||
# AI PR Review: REA Analysis and ARTEX Integration Plan
|
||||
|
||||
Status: research + design. No source code, prompts, or proprietary assets from the reference product are reproduced here. The analysis is based on a public client bundle observed with REA, the authenticated UI of a trial organization, and the vendor's public documentation and marketing pages. Field and enum names below come from generated client schemas and are evidence of the wire/data model, not a copy of server code.
|
||||
|
||||
Date: 2026-10-22
|
||||
Target: the reference web application
|
||||
Tooling: REA 6.7.0 (`analyze-web-bundle`, `analyze-javascript-application`, `capture-web-screenshot`, `list-browser-targets`), Chrome 155 over CDP, the vendor's public documentation.
|
||||
|
||||
## 1. TL;DR
|
||||
|
||||
The reference PR check is not a single "LLM reviews the diff" prompt. The client data model exposes a pipeline with these distinct stages:
|
||||
|
||||
1. **Webhook-triggered, policy-gated PR scan** — runs on PR/MR open or update, skips drafts/forks/outside contributors, caps changed files at 500, and honors repo config and exclusions.
|
||||
2. **Repository context assembly** — repo indexing (symbols, call graph, token counts), an auto-generated threat model, uploaded context documents, and repository rules.
|
||||
3. **Candidate discovery** — static data-flow/taint analysis plus LLM review agents; findings carry `taint_path`, `affected_code`, `affected_file`, `cwe_id`, and `owasp_category`.
|
||||
4. **Validation** — a validation agent produces `verified_claims` / `unverified_claims` and may perform optional dynamic verification in a sandbox, yielding a `dynamic_verdict` and `dynamic_evidence`.
|
||||
5. **Dedup + triage** — fingerprint/embedding-based duplicate handling, `detection_count`, and states such as `true_positive`, `false_positive`, `accepted_risk`, `resolved`.
|
||||
6. **GitHub output** — inline PR comments plus a check run with either annotations or status-only output; the check fails only when `fail_on.severity` is configured and an open/valid finding meets it.
|
||||
7. **Remediation** — an agent validates exploitability, writes a minimal fix, runs tests, adds a regression test, and opens a PR. A retest later marks fixes `verified_fixed` or `not_fixed`.
|
||||
|
||||
For ARTEX, the highest-value additions are the **PR eligibility/diff layer**, **code-context index + taint candidates**, **validator pass**, **dedup/triage lifecycle**, and **check-run publisher**. The autonomous pentest graph and existing finding/retest machinery can be reused rather than duplicated.
|
||||
|
||||
### 1.1 Platform context
|
||||
|
||||
The vendor's public homepage markets three offerings. The PR check is only the first; the other two matter because they share the finding, triage, and verification model.
|
||||
|
||||
| Area | Offering | What is public |
|
||||
| --- | --- | --- |
|
||||
| Code security | PR review | Reviews every PR/MR, validates vulnerabilities before merge, inline comments and checks. This document reconstructs it. |
|
||||
| Product security | Whitebox pentest | Combines repository context with active testing against target URLs and auth instructions; one free retest; heavier use of dynamic verification. |
|
||||
| Organization security | Continuous red teaming | "Simulate a persistent adversary." Advertised as a product area, but no public technical documentation describes scheduling, scope, agents, or output. |
|
||||
|
||||
The homepage also markets two cross-product loops: auto-triage/autofix to verify findings and prepare tested fixes, and self-improving threat models based on triage feedback. Both are consistent with the client schemas and the remediation workflow below, but the red-teaming internals are not observable from public evidence.
|
||||
|
||||
## 2. Evidence and confidence
|
||||
|
||||
| Evidence | What it establishes |
|
||||
| --- | --- |
|
||||
| REA `analyze_web_bundle` on the authenticated app (135 scripts, 64 routes), Evidence `ev_455acd3a8b10796167f2499ac63b1e131a63e48e9696065a8b6633c6f9a47b3b` | Route map, script inventory, authenticated app shell, settings and copy |
|
||||
| REA `analyze_javascript_application` (127 files, 6,877 findings), Evidence `ev_f0d14c017237436a907624c1d8f4db8b7a81ee9309cfa2eb9cead52f6e9dcf9b` | Static module graph and endpoint/storage observations |
|
||||
| Generated protobuf-es parsers/serializers in `_nuxt/ba334sRx.js` | Full client-side wire model for findings, scans, PR checks, fixes, validation, and repo indexing |
|
||||
| Public docs: `code-review/overview`, `code-review/config`, `code-review/rules`, `code-review/findings-feedback`, `platform/automations/remediation`, `white-box-pentest/overview`, `api-reference/scans/create-scan` | Trigger behavior, gating semantics, review limits, remediation phases, scan request fields |
|
||||
| Authenticated UI copy | Empty-state wording, settings, triage/fix/retest actions, auth-instruction handling |
|
||||
| Vendor marketing homepage (EN/FR) | Three product areas, including continuous red teaming; cross-product auto-triage/autofix and self-improving threat-model claims; no red-team internals |
|
||||
|
||||
**Proven:** route and field existence; enums and state names; PR check/comment/gating behavior as documented; remediation phases; 500-file limit; config/rules behavior; validation/dynamic-verification fields.
|
||||
|
||||
**Inferred:** the exact ordering and implementation of static taint analysis, LLM prompting, and validation; whether every PR scan performs dynamic testing; which parsing/static-analysis engines are used server-side.
|
||||
|
||||
**Unknown:** server implementation, model providers, prompt text, rule engine internals, the internal scoring/ranking formula, and the continuous red-teaming pipeline (scheduling, scope, agent topology, output). The client exposes only `Default` / `Legacy` model sets and cost/token accounting, not provider names.
|
||||
|
||||
## 3. What the PR check does
|
||||
|
||||
### 3.1 Trigger and eligibility
|
||||
|
||||
- On GitHub, the integration is a GitHub App; GitLab/Bitbucket/Azure DevOps have equivalent flows. Webhooks deliver on PR/MR open or update. A delivery-status model (`Webhooks delivering` / `failing` / `Webhook status unavailable`) is surfaced in the UI.
|
||||
- PR Review is enabled per repository. Drafts and PRs from forks or outside contributors can be skipped without a comment; other skips post the reason on the PR while comments are enabled.
|
||||
- Each PR/MR scan supports **up to 500 changed files**; larger PRs are skipped (public docs).
|
||||
- A force-review mention command (for example, a bot mention in a PR comment) forces a review when config would otherwise exclude the PR.
|
||||
- The check record carries `checkRunId`, `headSha`, `installationId`, `prNumber`, repository owner/name, PR author identity, a `prReviewReactionId`, and a status enum of `pending / in_progress / completed / failed / cancelled / skipped`.
|
||||
|
||||
### 3.2 Configuration and gating
|
||||
|
||||
The repository config file is read from the **default branch** so a PR cannot change its own rules. It supports:
|
||||
|
||||
- `exclude.{branches,labels,keywords,paths,authors}` and `include.{branches,labels,...}`. Path exclusion applies only when **every** changed file matches.
|
||||
- `fail_on.severity`: `critical | high | medium | low | info`. Without it, the check stays green when the scan completes; findings are comments only. With it, the check fails when an **open or confirmed-valid** finding is at or above the threshold.
|
||||
- Repo config wins per category; unset categories inherit organization defaults; `[]` clears an org default.
|
||||
- Invalid config fails open (missing/malformed ignored; unknown keys skipped; a type mismatch discards the file).
|
||||
|
||||
The repository rules file is separate from scope: it supplies app-specific context (trusted services, authz patterns, risk areas, scoped exclusions) to reduce false positives.
|
||||
|
||||
Triage updates the check immediately: marking a gated finding `false_positive` or `accepted_risk` can make the check pass; reopening it makes it fail again. Feedback commands on GitHub/GitLab include `!fp`, `!accepted_risk`, `!valid`, `!fixed`.
|
||||
|
||||
### 3.3 Review output
|
||||
|
||||
- Inline comments are posted on affected code by default.
|
||||
- GitHub check output mode is `annotations` or `status-only`.
|
||||
- Findings posted as PR comments; when source-control comments are off, findings stay in the platform while scans and check statuses still update.
|
||||
- For public repositories, outside-diff findings hide sensitive details (title, description, PoC, file locations); inline findings on changed code remain visible.
|
||||
- The scan record tracks `pr_number`, `pr_url`, `pr_lines_added`, `pr_lines_removed`, `scan_type` (`pr` or `full`).
|
||||
|
||||
### 3.4 Relation to the other product areas
|
||||
|
||||
- **Whitebox pentest** uses the same finding and verification vocabulary (`scan_type: full`, target URLs, auth instructions, `dynamic_verdict`, `dynamic_evidence`, retest). The PR check can be treated as the continuous, diff-scoped path through the same platform.
|
||||
- **Continuous red teaming** is marketed as "simulate a persistent adversary" but has no public technical documentation. It may share the finding, triage, and threat-model model, but that is unverified; do not assume its scanner or scheduling model in an implementation.
|
||||
- The shared loop is finding -> triage feedback -> threat-model update -> better review/pentest signal. The client exposes threat-model timestamps and triage activity, not the update algorithm.
|
||||
|
||||
## 4. Reconstructed vulnerability-detection pipeline
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
W["GitHub/GitLab webhook<br/>PR opened / updated"] --> E{"Eligibility<br/>draft? fork? excluded? >500 files?"}
|
||||
E -->|skip| SK["Check: skipped<br/>(reason in status)"]
|
||||
E -->|run| CTX["Assemble context<br/>diff @ head SHA + repo index<br/>+ threat model + rules.md + prior triage"]
|
||||
CTX --> SA["Static analysis<br/>symbols, call graph,<br/>taint/data-flow rules"]
|
||||
CTX --> LLM["LLM review agents<br/>changed hunks + repository context"]
|
||||
CTX --> DEP["Dependency verdicts<br/>known-malicious / OSV feeds"]
|
||||
SA --> CAND["Candidate findings<br/>file, code, taint path, CWE/OWASP"]
|
||||
LLM --> CAND
|
||||
DEP --> CAND
|
||||
CAND --> VAL["Validator agent<br/>verified/unverified claims"]
|
||||
VAL --> DYN{"requires verification<br/>and auth/target available?"}
|
||||
DYN -->|yes| DVER["Sandbox/browser verification<br/>confirmed / partial / not reproduced / blocked"]
|
||||
DYN -->|no| DEDUP
|
||||
DVER --> DEDUP["Dedup + fingerprint<br/>detection_count, duplicate_of, similarity"]
|
||||
DEDUP --> PUB["Publish<br/>inline comments + check run<br/>(annotations or status-only)"]
|
||||
PUB --> GATE{"fail_on severity<br/>and open/valid finding?"}
|
||||
GATE -->|yes| RED["Check: failure"]
|
||||
GATE -->|no| GREEN["Check: success"]
|
||||
PUB --> TRIAGE["Triage / feedback<br/>!valid !fp !accepted_risk !fixed"]
|
||||
TRIAGE --> PUB
|
||||
TRIAGE --> FIX["Remediation agent<br/>validate → minimal fix → tests → PR"]
|
||||
FIX --> RETEST["Retest<br/>verified_fixed / not_fixed"]
|
||||
```
|
||||
|
||||
### 4.1 Context assembly
|
||||
|
||||
The scan record and client model show repository indexing with `symbol_count`, `total_repo_tokens`, `unsupported_languages`, `commit_sha`, `from_cache`, predicted cost, and a provider/status field. A separate call-graph download exists for scans. A threat model is generated automatically the first time a PR is reviewed in a repo (or when context documents are uploaded), and can be synthesized at the application level across repos. Reviews also consume the repository rules file and prior triage feedback.
|
||||
|
||||
Likely purpose: give the reviewer a compact, semantically indexed view of the changed code plus reachability context (call graph, threat model) instead of raw repository text.
|
||||
|
||||
### 4.2 Static candidate discovery
|
||||
|
||||
The finding schema is the strongest evidence of data-flow analysis:
|
||||
|
||||
- `affected_file`, `affected_code`
|
||||
- `taint_path` (ordered collection)
|
||||
- `cwe_id`, `owasp_category`
|
||||
- `category`, `title`, `description`, `recommendation`
|
||||
- `proof_of_concept`
|
||||
- `scan_pr_number`, `scan_pr_url`, `scan_type`
|
||||
|
||||
`taint_path` plus the downloadable call graph implies source-to-sink data-flow tracking, not just pattern matching. The exact rules engine is not exposed; candidates may come from built-in rules, repository rules, and LLM review.
|
||||
|
||||
### 4.3 LLM review and validation
|
||||
|
||||
Client strings and schemas expose:
|
||||
|
||||
- "Review agent", "Agents", "Delegating to subagent", "Delegated to subagent".
|
||||
- "Starting validation agent...", "View validation activity", "Validated during triage".
|
||||
- A validation object: `verified_claims`, `unverified_claims`, `evidence_summary`, `validation_output`, `blocker_detail`.
|
||||
- Review summaries with `conclusion`, `riskRating`, `riskRationale`, `narrativeMarkdown`, `nextSteps`, `severityCounts`, and `model`.
|
||||
|
||||
The safest reconstruction is a two-pass design: a **reviewer** proposes findings from the diff plus repository context, and a **validator** attempts to falsify or confirm each claim against code and evidence. Only validated claims are eligible for gating.
|
||||
|
||||
### 4.4 Optional dynamic verification
|
||||
|
||||
The scan model contains `requires_verification`, `has_auth_instructions`, `auth_verification_status`, and `auth_verification_failure_reason`. Findings carry `dynamic_verdict` (`confirmed`, `partially_confirmed`, `not_reproduced`, `blocked`) and `dynamic_evidence`. UI copy warns that auth instructions are shared with AI agents for testing and recommends a test environment.
|
||||
|
||||
This is a sandbox/browser verification stage. It is optional and more prominent in whitebox pentests, but the PR-scan schema supports it.
|
||||
|
||||
### 4.5 Dedup and triage
|
||||
|
||||
Finding records include `detection_count`, `duplicate_of`, `first_seen_at`, `last_seen_at`, `first_seen_scan_id`, `state`, `verification_status`, and `latest_state_change`. One variant includes an `embedding` and `similarity`, indicating semantic duplicate matching. States:
|
||||
|
||||
- Finding state: `open`, `true_positive`, `false_positive`, `accepted_risk`, `resolved`, `closed`.
|
||||
- Triage verification: `pending`, `approved`, `rejected`.
|
||||
- Dynamic verdict: `confirmed`, `partially_confirmed`, `not_reproduced`, `blocked`.
|
||||
|
||||
### 4.6 Remediation
|
||||
|
||||
The remediation docs and fix schema agree:
|
||||
|
||||
1. Validate the finding: test exploitability; if unreachable/sound, mark `false_positive` instead of patching.
|
||||
2. Implement the minimal correct fix.
|
||||
3. Run the repo's test suite, add relevant regression tests, and verify the fix.
|
||||
4. Commit to a fix branch and open a PR/MR.
|
||||
|
||||
Fix record: `base_commit_sha`, `unified_diff`, `commit_message`, `pr_title`, `pr_body`, `reason`, `summary`, `cost_usd`, `task_id_cli`, and verification `{notes, reproduced, regression_test_added, test_command, test_output}`. Fix status: `dispatched`, `pr_created`, `no_fix`, `false_positive`, `failed`, `stopped_cap_exceeded`, `pr_closed`. Verification: `not_requested`, `pending`, `verified_fixed`, `not_fixed`.
|
||||
|
||||
## 5. Data model worth mirroring in ARTEX
|
||||
|
||||
These are client field names, not a server schema. They are still a useful target model.
|
||||
|
||||
| Concept | Important fields |
|
||||
| --- | --- |
|
||||
| PR check task | `checkRunId`, `headSha`, `installationId`, `prNumber`, `repositoryOwner`, `repositoryName`, `pr_author_login`, `prReviewReactionId`, `status` |
|
||||
| Scan | `scan_type` (`pr`/`full`), `pr_number`, `pr_url`, `branch`, `commit`, `current_step`, `display_status`, `model` (`default`/`legacy`), `requires_verification`, `has_auth_instructions`, `triage_enabled`, `findings_summary`, `credits_used`, `pr_lines_added/removed` |
|
||||
| Finding | `affected_file`, `affected_code`, `taint_path[]`, `cwe_id`, `owasp_category`, `severity`, `category`, `title`, `description`, `recommendation`, `proof_of_concept`, `detection_count`, `duplicate_of`, `state`, `verification_status`, `dynamic_verdict`, `dynamic_evidence`, `scan_pr_number/url`, `first_seen_at`, `last_seen_at`, `resolved_at` |
|
||||
| Validation | `verified_claims[]`, `unverified_claims[]`, `evidence_summary`, `validation_output`, `blocker_detail` |
|
||||
| Review summary | `conclusion`, `riskRating`, `riskRationale`, `narrativeMarkdown`, `nextSteps`, `severityCounts`, `model` |
|
||||
| Repo indexing | `symbol_count`, `total_repo_tokens`, `unsupported_languages[]`, `provider`, `from_cache`, `commit_sha`, predicted cost bounds |
|
||||
| Fix | `fix{base_commit_sha, unified_diff}`, `pr_title`, `pr_body`, `commit_message`, `status`, `verification{reproduced, regression_test_added, test_command, test_output, notes}` |
|
||||
|
||||
Scan status values observed: `pending`, `running`, `completed`, `failed`, `stopped`, `cancelled`, `skipped`, `pending_verification`, `pending_triage`, `draft`, `estimating`, `estimated`.
|
||||
|
||||
## 6. How the check decides pass/fail
|
||||
|
||||
The check is deliberately **advisory by default**. A completed scan is green; findings do not block. `fail_on.severity` opts into gating. Only open or confirmed-valid findings count. Triage immediately recomputes the check, and the config is read from the default branch.
|
||||
|
||||
For ARTEX this means the check runner must separate:
|
||||
|
||||
- **Scan outcome** (completed/failed/cancelled/skipped) from
|
||||
- **Gate outcome** (success/failure based on policy + current finding disposition).
|
||||
|
||||
Keeping those independent avoids the common mistake of failing a check because the scanner crashed, or passing it because a finding was dismissed in the platform but not recomputed.
|
||||
|
||||
## 7. ARTEX integration blueprint
|
||||
|
||||
ARTEX today is an autonomous pentest system: Go backend, Next.js UI, PostgreSQL, dual graph (assets + exploration), planner/worker agents, traffic proxy, findings, and retests. It does **not** have a GitHub PR review subsystem. The following design adds one without turning ARTEX into a full static-analysis platform on day one.
|
||||
|
||||
### 7.1 Design principles
|
||||
|
||||
- Reuse findings/retests/notifications; don't create a parallel vulnerability model.
|
||||
- Treat repository content and PR text as untrusted input (prompt injection, secrets).
|
||||
- Separate deterministic gates (eligibility, config, fail_on) from probabilistic detection (LLM/static candidates).
|
||||
- Make every scan idempotent by `(repository, head_sha)`.
|
||||
- Keep the check advisory by default; gating is explicit policy.
|
||||
- Never execute untrusted repository code on the host. Dynamic verification runs in the existing sandbox/proxy boundary.
|
||||
|
||||
### 7.2 Proposed package layout
|
||||
|
||||
```
|
||||
codereview/
|
||||
webhook.go GitHub/GitLab webhook verification and event normalization
|
||||
model.go Scan, CheckRun, PullRequest, Finding, FixRun, enums
|
||||
diff.go unified-diff parsing, hunk→file mapping, 500-file cap
|
||||
config.go .artex/config.yaml (repository policy)
|
||||
index.go repository snapshot + symbol/call-graph index
|
||||
rules.go .artex/rules.md, org/repo rules, path/author/label filters
|
||||
analyze.go deterministic rules + taint candidate extraction
|
||||
review.go LLM reviewer agent (diff + context)
|
||||
validate.go validator agent (claims, reachability, false-positive checks)
|
||||
dynamic.go optional verification through ARTEX target/traffic tooling
|
||||
dedup.go fingerprints + embeddings + duplicate_of
|
||||
publish.go GitHub check run + inline annotations/comments
|
||||
fix.go remediation run and fix-PR creation
|
||||
retest.go post-fix verification and state transitions
|
||||
```
|
||||
|
||||
Each package should expose interfaces, not global state, so the pipeline can run in the existing engine or as a separate worker.
|
||||
|
||||
### 7.3 Storage
|
||||
|
||||
Add a small code-review schema rather than overloading `findings`:
|
||||
|
||||
- `cr_repositories`: provider, installation id, owner/name, enabled, default branch, config hash, index state.
|
||||
- `cr_pull_requests`: repo id, number, head SHA, base SHA, author, draft/fork flags, state, `check_run_id`, `check_conclusion`, `check_output_mode`, `last_event_at`.
|
||||
- `cr_scans`: PR id, head SHA, status, current step, model set, token/cost counters, started/finished, error.
|
||||
- `cr_findings`: scan id, fingerprint, file, line range, taint path JSONB, CWE, OWASP, severity, title, description, recommendation, PoC, detection count, duplicate_of, state, verification status, dynamic verdict/evidence, resolved_at.
|
||||
- `cr_fix_runs`: finding id, status, base commit, unified diff, test command/output, reproduced/regression flags, PR URL, cost.
|
||||
- `cr_feedback`: finding id, actor, command/state, reason, created_at, check recomputed_at.
|
||||
|
||||
Alternatively, extend `findings` with nullable `source`, `file`, `line`, `fingerprint`, `cwe`, `owasp`, `taint_path`; but a separate table keeps pentest findings and code findings cleanly separable and avoids changing the existing API contract.
|
||||
|
||||
### 7.4 API surface
|
||||
|
||||
- `POST /api/webhooks/github` — verify `X-Hub-Signature-256`, normalize `pull_request` events, enqueue.
|
||||
- `POST /api/codereview/pull-requests/{id}/review` — force review (equivalent to a bot mention in a PR comment).
|
||||
- `GET /api/codereview/pull-requests/{id}` — status, check, findings.
|
||||
- `GET /api/codereview/findings` — filters: repo, PR, severity, state, CWE/OWASP.
|
||||
- `PATCH /api/codereview/findings/{id}` — triage state, severity, reason; recompute check.
|
||||
- `POST /api/codereview/findings/{id}/fix` — start remediation (idempotent).
|
||||
- `POST /api/codereview/findings/{id}/retest` — after a fix.
|
||||
- `GET /api/codereview/repos/{id}/index` — index stats and unsupported languages.
|
||||
|
||||
Server routes in ARTEX use Go 1.22+ `mux.HandleFunc("METHOD /path/{id}", ...)`; follow that style.
|
||||
|
||||
### 7.5 PR lifecycle
|
||||
|
||||
1. Verify webhook signature; reject replayed deliveries using the delivery id.
|
||||
2. Upsert `cr_pull_requests` by `(repo, number)`; ignore events for an already-scanned `head_sha`.
|
||||
3. Load config from the **base/default branch**; evaluate include/exclude; skip drafts/forks/outside contributors; enforce the changed-file cap.
|
||||
4. Create `cr_scan` with status `pending`; set check to `in_progress` with a stable `details_url`.
|
||||
5. Fetch diff and base/head metadata; parse hunks; map findings to changed line ranges.
|
||||
6. Build/refresh the repo index (incremental by commit; cache by `(repo, commit)`).
|
||||
7. Run deterministic rules and taint extraction; run the LLM reviewer; merge candidates by fingerprint.
|
||||
8. Validate each candidate; drop unreachable/false candidates; optionally run dynamic verification.
|
||||
9. Dedup, assign display IDs, persist findings.
|
||||
10. Publish inline comments (respecting public-repo redaction) and a check run (`annotations` or `status-only`).
|
||||
11. Compute the gate from policy + open/valid findings; transition check to `completed`.
|
||||
12. On triage, recompute the gate and optionally delete/update the stale comment.
|
||||
13. On `!fixed` or a new commit, schedule retest; on `verified_fixed`, mark `resolved`.
|
||||
|
||||
### 7.6 Agent roles and prompts
|
||||
|
||||
- **Reviewer** (`codereview/review.go`): input = changed hunks + file summaries + call-graph neighborhood + threat model + rules. Output = candidate findings with file/line, category, severity, CWE/OWASP guess, evidence, and a proposed taint path.
|
||||
- **Validator** (`codereview/validate.go`): input = candidate + surrounding code + reachability facts + prior feedback. Output = `verified_claims`, `unverified_claims`, `evidence_summary`, `validation_output`, `blocker_detail`, and a final disposition suggestion. The validator must be able to say "not exploitable" and suppress the finding.
|
||||
- **Fixer** (`codereview/fix.go`): input = validated finding + repo snapshot. Output = minimal unified diff, commit message, PR title/body, test command, and verification notes.
|
||||
- **Retester** (`codereview/retest.go`): re-run the reproduction or the relevant tests against the fixed commit; produce `verified_fixed` or `not_fixed`.
|
||||
|
||||
Use ARTEX's existing `llmpool` for provider selection, per-run token budgets, retries, and usage recording. Store model/provider metadata and cost per scan, matching the reference product's `LLM calls` / `Tokens (in/out)` visibility.
|
||||
|
||||
### 7.7 Static analysis for a Go codebase
|
||||
|
||||
A pragmatic first pass:
|
||||
|
||||
- **Tree-sitter** parsers for the languages you support; persist symbols, imports, and call edges.
|
||||
- **Call graph** from definitions + references (start intra-repo; mark unresolved dynamic dispatch as unknown).
|
||||
- **Taint rules** for common sources/sinks (HTTP handlers, SQL, template rendering, command execution, file paths, deserialization) with explicit sanitizers.
|
||||
- **Custom rules** from `.artex/rules.md` and repo config; allow path/label/author exclusions.
|
||||
- **Dependency verdicts**: optional OSV/advisory lookup plus a known-malicious package feed, surfaced as a separate finding category.
|
||||
|
||||
Do not claim full-program reachability when the call graph is partial. Record `unknown` for unresolved edges; this mirrors the reference product's `unsupported_languages` and call-graph limitations.
|
||||
|
||||
### 7.8 Check publisher
|
||||
|
||||
- Use GitHub Checks API with `name`, `head_sha`, `status`, `conclusion`, `output.title`, `output.summary`, and annotations (file, start_line, end_line, annotation_level, message).
|
||||
- `status-only` mode omits annotations and only reports the summary/check state.
|
||||
- `fail_on.severity` maps to the gate; severity order: `critical > high > medium > low > info`.
|
||||
- On skip, post `conclusion: skipped` with a reason when comments are enabled.
|
||||
- Public repositories: redact title/description/PoC/file locations for findings outside the diff; keep inline findings on changed lines visible.
|
||||
- Recompute the check on triage, not only on scan completion.
|
||||
|
||||
### 7.9 Remediation and retest
|
||||
|
||||
- Only validated findings are eligible for remediation.
|
||||
- Create a dedicated branch; never push to the PR head by default.
|
||||
- Run the repository's existing test command in the sandbox; add a regression test only when the repository has a clear test convention.
|
||||
- Store the unified diff, test command/output, and reproduction result. Open the PR/MR with a body that links the finding and the evidence.
|
||||
- If the source PR merges mid-run, retarget the fix to the default branch (the vendor's documented behavior).
|
||||
- A retest re-runs the reproduction/tests; `verified_fixed` marks the finding resolved, `not_fixed` keeps it open with evidence.
|
||||
|
||||
### 7.10 Security, cost, and limits
|
||||
|
||||
- **Prompt injection**: repository files, PR descriptions, and rules are untrusted; never let them override system instructions. Strip secrets before sending to a model.
|
||||
- **Code execution**: verification runs in ARTEX's existing sandbox/proxy boundary; do not execute repo code on the server host.
|
||||
- **Public repos**: apply redaction before publishing comments; a public comment can leak a vulnerability.
|
||||
- **Idempotency**: `(repo, head_sha)` is the scan key; triage commands are idempotent by finding/state.
|
||||
- **Cost control**: per-PR token and credit budget; fail closed to `skipped` when the budget is exhausted rather than scanning partially and publishing a misleading check.
|
||||
- **Config trust**: read `.artex/config.yaml` from the default branch, not the PR head; invalid config fails open.
|
||||
- **Audit**: store the check conclusion, policy threshold, finding dispositions, and recomputation timestamps.
|
||||
|
||||
### 7.11 Reuse map for ARTEX
|
||||
|
||||
| ARTEX component | Reuse |
|
||||
| --- | --- |
|
||||
| `db/findings.go`, `findings` table | Status vocabulary, lists, grouping, notifications; either extend or mirror in `cr_findings` |
|
||||
| `db/finding_retests.go`, `agent/retester.go` | Retest lifecycle and `reproduced`/`fixed`/`inconclusive` verdicts |
|
||||
| `server/finding_workflow.go` | Tool wiring for agent-driven finding updates |
|
||||
| `llmpool`, `agent/provider.go` | Model selection, retries, quota, token accounting |
|
||||
| `traffic/`, `intercept/` | Dynamic verification evidence and HTTP replay |
|
||||
| `notify/` | Slack/email alerts for new gated findings |
|
||||
| `guard/` | Tool approval and execution boundary |
|
||||
| `web/` | New Code Review pages and PR detail views |
|
||||
|
||||
### 7.12 Phased delivery
|
||||
|
||||
**Phase 0 — Ingest and publish (no detection).**
|
||||
GitHub App/webhook, repository config, eligibility, diff parse, check run with `skipped`/`in_progress`/`completed`, idempotency by head SHA. Acceptance: a PR with no findings gets a green check; an excluded PR gets `skipped` with the reason; a duplicate webhook does not create a second scan.
|
||||
|
||||
**Phase 1 — Deterministic rules and comments.**
|
||||
Tree-sitter index, a small taint ruleset, inline annotations, `fail_on` gate, triage recompute. Acceptance: a seeded vulnerable handler is flagged at the right file/line; a `false_positive` triage turns the check green; a `fail_on: critical` policy fails only on critical/high.
|
||||
|
||||
**Phase 2 — LLM reviewer + validator.**
|
||||
Reviewer proposes candidates; validator produces verified/unverified claims; merge by fingerprint; cost/token accounting. Acceptance: a false candidate is suppressed by the validator; the scan stays green; token usage is recorded per scan.
|
||||
|
||||
**Phase 3 — Dedup, feedback, and operations.**
|
||||
Embedding/fingerprint dedup, `detection_count`, duplicate links, feedback commands (`!fp`, `!valid`, `!accepted_risk`, `!fixed`), Slack notifications, dashboard. Acceptance: the same finding across commits is deduped; feedback survives rescans; the dashboard shows fix rate and suppression rate.
|
||||
|
||||
**Phase 4 — Remediation and retest.**
|
||||
Fix agent, unified diff, test execution, fix PR, retest. Acceptance: a validated finding produces a PR with a regression test; retest marks `verified_fixed`; a failed fix stays open with test output.
|
||||
|
||||
**Phase 5 — Dynamic verification.**
|
||||
Target URLs + auth instructions, browser verification, dynamic verdict/evidence, public-repo redaction. Acceptance: a flagged SSRF/SQLi candidate is confirmed or refuted against a sandbox target; evidence is attached; no secrets leak into comments.
|
||||
|
||||
### 7.13 Acceptance criteria for "PR check works"
|
||||
|
||||
- Webhook → check in under a bounded time; idempotent retries.
|
||||
- Check is advisory by default; `fail_on` is the only blocker.
|
||||
- Findings map to real changed lines and include file, severity, category, evidence.
|
||||
- False positives can be dismissed from GitHub without visiting the platform, and the check updates.
|
||||
- Public comments never disclose PoC/secrets for outside-diff findings.
|
||||
- Every scan records model, token/cost, commit, and policy used.
|
||||
- No repository code executes outside the sandbox.
|
||||
- All state transitions are visible and auditable.
|
||||
|
||||
## 8. Mapping: Reference behavior → ARTEX implementation
|
||||
|
||||
| Reference behavior | ARTEX implementation |
|
||||
| --- | --- |
|
||||
| GitHub App webhook + check run | New `codereview/webhook.go` + `publish.go`; GitHub App installation stored in `cr_repositories` |
|
||||
| Repo indexing + call graph | `codereview/index.go`; tree-sitter symbols + call edges; persist in `cr_symbols`/`cr_call_edges` |
|
||||
| Threat model + rules | Reuse ARTEX context documents/task context; add `.artex/rules.md` loader |
|
||||
| Taint findings | `codereview/analyze.go` taint rules; store `taint_path` JSONB |
|
||||
| Reviewer/validator agents | `codereview/review.go` + `validate.go`; reuse `llmpool` |
|
||||
| Dynamic verification | `codereview/dynamic.go`; reuse `traffic`/intercept + sandbox |
|
||||
| Triage state machine | Reuse `findings` status vocabulary or `cr_findings` with the same states |
|
||||
| Remediation | `codereview/fix.go`; branch/PR creation; run tests in sandbox |
|
||||
| Retest | Reuse `finding_retests` pattern; add `verified_fixed`/`not_fixed` |
|
||||
| Slack notifications | Reuse `notify` |
|
||||
|
||||
## 9. Open questions
|
||||
|
||||
- Which static-analysis engine does the reference product use server-side? The client proves taint/call-graph data but not the engine.
|
||||
- Are PR scans always LLM-reviewed, or only when static candidates exist? The model and agent fields suggest LLM review is normal.
|
||||
- How are repository embeddings stored and refreshed? `embedding`/`similarity` exist in one finding variant, but the index internals are unknown.
|
||||
- What is the exact ranking/scoring formula for PR risk? Only `riskRating`, `severityCounts`, and review narrative are exposed.
|
||||
- How does dynamic verification authenticate to customer apps? `auth_instructions` (max 2,000 chars) and `auth_verification_status` are exposed; the browser automation is not.
|
||||
- What are the model providers behind `Default`/`Legacy`? Not present in the client bundle.
|
||||
- How does continuous red teaming schedule runs, choose scope, and hand findings into triage? The homepage advertises it, but no public documentation or client surface explains it.
|
||||
|
||||
## 10. Sources
|
||||
|
||||
- REA Evidence: `ev_455acd3a8b10796167f2499ac63b1e131a63e48e9696065a8b6633c6f9a47b3b` (authenticated bundle), `ev_f0d14c017237436a907624c1d8f4db8b7a81ee9309cfa2eb9cead52f6e9dcf9b` (static app graph), `ev_bb2c67fec41a43f211b557d91c9aff64e11b349cf641ca34ebbcc624a81d2d99` (dashboard screenshot).
|
||||
- Vendor docs: `code-review/overview`, `/code-review/config`, `/code-review/rules`, `/code-review/findings-feedback`, `/platform/automations/remediation`, `/white-box-pentest/overview`, `/api-reference/scans/create-scan`, `/api-reference/hfw/get-package-verdict`.
|
||||
- Client bundle: the app's `_nuxt/ba334sRx.js` bundle and related chunks; generated message fields/enums extracted statically. Minified function names are not original type names.
|
||||
|
||||
## 11. License and use
|
||||
|
||||
This document is original analysis for interoperability and research. The reference product is proprietary; do not copy its code, prompts, or assets. The reconstructed field names are factual observations of a public client schema. Any ARTEX implementation should be written independently against this behavioral specification and reviewed for license/legal compatibility before redistribution. Continuous red teaming is advertised as a third offering but has no public technical documentation; this document treats its internals as unknown and out of scope.
|
||||
@@ -61,4 +61,4 @@ require (
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
)
|
||||
|
||||
replace github.com/Autumn-27/norma => github.com/BaaRai/Norma v0.4.4-0.20261011224847-63b38dabc837
|
||||
replace github.com/Autumn-27/norma => github.com/BaaRai/Norma v0.4.4
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
github.com/BaaRai/Norma v0.4.4-0.20261011224847-63b38dabc837 h1:slZeNksaiCKS4XZSNnNIlmyqnfgDT/PGoPjOPrijnoE=
|
||||
github.com/BaaRai/Norma v0.4.4-0.20261011224847-63b38dabc837/go.mod h1:BudWrf7iHl96FgHqFr/HwN3mKYmGjJDt+xL2ZHXfFZw=
|
||||
github.com/BaaRai/Norma v0.4.4 h1:x5UAeaTuhREI1GiIYuwIEybRxr/pLCeCUvXmUbwv5Is=
|
||||
github.com/BaaRai/Norma v0.4.4/go.mod h1:BudWrf7iHl96FgHqFr/HwN3mKYmGjJDt+xL2ZHXfFZw=
|
||||
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
|
||||
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
|
||||
|
||||
+5
-3
@@ -45,9 +45,11 @@ install_docker(){
|
||||
else
|
||||
info "reusing existing .env"
|
||||
fi
|
||||
info "pulling images and starting..."
|
||||
docker compose pull || true
|
||||
docker compose up -d
|
||||
info "building the image from this checkout and starting..."
|
||||
if [ ! -f "dist/amd64/artex" ] && [ ! -f "dist/arm64/artex" ]; then
|
||||
warn "no prebuilt binary under dist/ - build it first (frontend, then go build -tags embedui); see the Dockerfile header"
|
||||
fi
|
||||
docker compose up -d --build
|
||||
ok "started → http://localhost:8787"
|
||||
info "view logs: docker compose logs -f artex"
|
||||
}
|
||||
|
||||
@@ -1,313 +0,0 @@
|
||||
package selfupdate
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// smokeEnv lets the child process launched by the smoke test skip Bootstrap entirely.
|
||||
//
|
||||
// Strictly speaking it would be fine without it: the child's os.Executable() is
|
||||
// artex.new, so every derived path carries the .new infix and can never touch the
|
||||
// real upgrade files. But relying on that coincidence is too fragile — an explicit
|
||||
// short-circuit is obvious at a glance and saves the child a pointless disk probe.
|
||||
const smokeEnv = "ARTEX_SELFUPDATE_SMOKE"
|
||||
|
||||
// Action is Bootstrap's instruction to main.
|
||||
type Action int
|
||||
|
||||
const (
|
||||
// Continue: start the server as usual.
|
||||
Continue Action = iota
|
||||
// Restart: exit immediately with ExitRestart so the supervisor script relaunches us.
|
||||
Restart
|
||||
)
|
||||
|
||||
// State describes the upgrade state at this startup, so /api/update/check can
|
||||
// faithfully tell the frontend "did the last upgrade succeed or was it rolled back".
|
||||
type State struct {
|
||||
Pending bool // swapped but not yet confirmed stable
|
||||
RolledBack bool // this startup just performed an automatic rollback
|
||||
FailedStage bool // staged file failed verification/smoke test and was discarded
|
||||
Detail string // one-sentence user-facing explanation
|
||||
}
|
||||
|
||||
// Bootstrap runs at the very beginning of main, and must be called before any port
|
||||
// is listened on or any database is opened.
|
||||
//
|
||||
// Three situations:
|
||||
//
|
||||
// ① staged file artex.new exists → verify + smoke test; on success swap and ask
|
||||
// for a restart; on failure discard it and keep
|
||||
// running the old version
|
||||
// ② only the marker file remains → the swap just happened; count one attempt;
|
||||
// roll back after enough consecutive failures
|
||||
// ③ nothing → normal startup
|
||||
func Bootstrap() (Action, State) {
|
||||
if os.Getenv(smokeEnv) != "" {
|
||||
return Continue, State{}
|
||||
}
|
||||
p, err := ResolvePaths()
|
||||
if err != nil {
|
||||
log.Printf("[update] skipping bootstrap: %v", err)
|
||||
return Continue, State{}
|
||||
}
|
||||
|
||||
if _, err := os.Stat(p.New); err == nil {
|
||||
return applyStaged(p)
|
||||
}
|
||||
|
||||
m, ok := readMarker(p.Marker)
|
||||
if !ok {
|
||||
return Continue, State{}
|
||||
}
|
||||
return confirmOrRollback(p, m)
|
||||
}
|
||||
|
||||
// applyStaged handles the "staged file exists" situation: swap if it verifies,
|
||||
// discard it if not.
|
||||
//
|
||||
// This is the only place in the whole upgrade chain that overwrites the executable,
|
||||
// and the final gate — the smoke test stops problems like a corrupted download,
|
||||
// a wrong architecture choice, or missing dynamic libraries. Once a binary that
|
||||
// cannot run is let through, the supervisor script will tirelessly keep relaunching
|
||||
// it while the Go code never gets a chance to run, making automatic rollback impossible.
|
||||
func applyStaged(p Paths) (Action, State) {
|
||||
m, _ := readMarker(p.Marker)
|
||||
|
||||
if err := verifyStaged(p); err != nil {
|
||||
log.Printf("[update] staged new version failed verification, discarded, continuing with the current version: %v", err)
|
||||
cleanStaged(p)
|
||||
_ = os.Remove(p.Marker)
|
||||
return Continue, State{FailedStage: true, Detail: "new version failed verification and was discarded: " + err.Error()}
|
||||
}
|
||||
|
||||
if err := swap(p); err != nil {
|
||||
log.Printf("[update] swap failed, continuing with the current version: %v", err)
|
||||
cleanStaged(p)
|
||||
_ = os.Remove(p.Marker)
|
||||
return Continue, State{FailedStage: true, Detail: "swap failed: " + err.Error()}
|
||||
}
|
||||
|
||||
// Swap succeeded. Keep the marker; the next startup (running the new version) confirms whether it is stable.
|
||||
m.Attempts = 0
|
||||
if m.StagedAt == 0 {
|
||||
m.StagedAt = time.Now().Unix()
|
||||
}
|
||||
if err := writeMarker(p.Marker, m); err != nil {
|
||||
log.Printf("[update] failed to write upgrade marker (automatic rollback capability lost): %v", err)
|
||||
}
|
||||
log.Printf("[update] swapped to %s, exiting to restart (exit %d)", orUnknown(m.To), ExitRestart)
|
||||
return Restart, State{Pending: true}
|
||||
}
|
||||
|
||||
// confirmOrRollback handles "the startup after a swap": counts attempts, and swaps
|
||||
// the old version back once the limit is exceeded.
|
||||
//
|
||||
// The counter only increments after the Go code is running, so it covers failures
|
||||
// of the "executable but crashes during initialization" kind (incompatible config,
|
||||
// port occupied, DB migration blew up); "cannot exec at all" is stopped by the
|
||||
// smoke test before the swap — together they are complete.
|
||||
func confirmOrRollback(p Paths, m marker) (Action, State) {
|
||||
m.Attempts++
|
||||
if m.Attempts > maxAttempts {
|
||||
if err := rollback(p); err != nil {
|
||||
// If even the rollback fails, do not restart again, or we fall into an
|
||||
// infinite restart loop. Remove the marker and let the process start
|
||||
// as-is — if it cannot come up, the user can at least see why in the logs.
|
||||
log.Printf("[update] new version failed to start %d times in a row, and rollback failed too: %v", maxAttempts, err)
|
||||
_ = os.Remove(p.Marker)
|
||||
return Continue, State{Detail: "new version failed to start and rollback failed: " + err.Error()}
|
||||
}
|
||||
log.Printf("[update] new version failed to start %d times in a row, rolled back to %s, exiting to restart (exit %d)",
|
||||
maxAttempts, orUnknown(m.From), ExitRestart)
|
||||
_ = os.Remove(p.Marker)
|
||||
return Restart, State{RolledBack: true, Detail: fmt.Sprintf("new version failed to start, rolled back to %s", orUnknown(m.From))}
|
||||
}
|
||||
if err := writeMarker(p.Marker, m); err != nil {
|
||||
log.Printf("[update] failed to update upgrade marker: %v", err)
|
||||
}
|
||||
log.Printf("[update] new version starting (attempt %d/%d); the upgrade will be confirmed once it runs stably",
|
||||
m.Attempts, maxAttempts)
|
||||
return Continue, State{Pending: true}
|
||||
}
|
||||
|
||||
// Settle confirms the new version has been running stably and clears the upgrade marker.
|
||||
//
|
||||
// Called by main with a delay after the HTTP listener is up: only surviving that
|
||||
// period counts; otherwise the marker stays in place and the next startup keeps
|
||||
// counting attempts until a rollback is triggered.
|
||||
func Settle() {
|
||||
p, err := ResolvePaths()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
settle(p)
|
||||
}
|
||||
|
||||
func settle(p Paths) {
|
||||
if _, ok := readMarker(p.Marker); !ok {
|
||||
return // not a post-upgrade startup, nothing to do
|
||||
}
|
||||
if err := os.Remove(p.Marker); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
log.Printf("[update] failed to clear upgrade marker: %v", err)
|
||||
return
|
||||
}
|
||||
log.Printf("[update] new version is running stably, upgrade complete (previous version kept as %s)", p.Old)
|
||||
}
|
||||
|
||||
// SettleDelay is the runtime duration required to decide "the new version survived".
|
||||
const SettleDelay = 30 * time.Second
|
||||
|
||||
// verifyStaged verifies the staged file: compare SHA256 first, then actually launch it once.
|
||||
func verifyStaged(p Paths) error {
|
||||
want, err := os.ReadFile(p.Sum)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to read checksum: %w", err)
|
||||
}
|
||||
got, err := fileSHA256(p.New)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to compute checksum: %w", err)
|
||||
}
|
||||
if !strings.EqualFold(strings.TrimSpace(string(want)), got) {
|
||||
return errors.New("SHA256 mismatch (download corrupted or tampered with)")
|
||||
}
|
||||
return smokeTest(p.New)
|
||||
}
|
||||
|
||||
// smokeTest launches the new binary with -h to confirm it can really execute on
|
||||
// the current system. This stops a whole class of problems: truncated downloads,
|
||||
// wrong architecture (exec format error), missing dependencies, and so on.
|
||||
func smokeTest(bin string) error {
|
||||
if err := os.Chmod(bin, 0o755); err != nil {
|
||||
return fmt.Errorf("failed to grant execute permission: %w", err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
cmd := exec.CommandContext(ctx, bin, "-h")
|
||||
cmd.Env = append(os.Environ(), smokeEnv+"=1")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if ctx.Err() != nil {
|
||||
return errors.New("smoke test timed out (new binary unresponsive)")
|
||||
}
|
||||
if err != nil {
|
||||
snippet := strings.TrimSpace(string(out))
|
||||
if len(snippet) > 300 {
|
||||
snippet = snippet[:300] + "…"
|
||||
}
|
||||
return fmt.Errorf("smoke test failed: %v: %s", err, snippet)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// swap replaces the current binary with the staged new version.
|
||||
//
|
||||
// Both Unix and Windows allow renaming a running executable (what Windows forbids
|
||||
// is deleting and overwriting; rename is not on that list), so there is no need to
|
||||
// split by platform here, nor to stop ourselves first.
|
||||
func swap(p Paths) error {
|
||||
// Windows rename does not overwrite an existing target, so the .old left by the previous upgrade must be removed first.
|
||||
if err := os.Remove(p.Old); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("failed to clean up old backup %s: %w", p.Old, err)
|
||||
}
|
||||
if err := os.Rename(p.Current, p.Old); err != nil {
|
||||
return fmt.Errorf("failed to back up the current version: %w", err)
|
||||
}
|
||||
if err := os.Rename(p.New, p.Current); err != nil {
|
||||
// The swap failed but the current version has already been moved away; it must be put back as-is, or the next startup has no executable.
|
||||
if rerr := os.Rename(p.Old, p.Current); rerr != nil {
|
||||
return fmt.Errorf("failed to install the new version (%v), and restoring the current version failed too: %w", err, rerr)
|
||||
}
|
||||
return fmt.Errorf("failed to install the new version: %w", err)
|
||||
}
|
||||
_ = os.Remove(p.Sum)
|
||||
return nil
|
||||
}
|
||||
|
||||
// rollback swaps back the old version that swap backed up.
|
||||
func rollback(p Paths) error {
|
||||
if _, err := os.Stat(p.Old); err != nil {
|
||||
return fmt.Errorf("no backup to roll back to %s: %w", p.Old, err)
|
||||
}
|
||||
// Move the unstartable new version to .failed for troubleshooting instead of deleting it outright.
|
||||
failed := p.Current + ".failed"
|
||||
_ = os.Remove(failed)
|
||||
if err := os.Rename(p.Current, failed); err != nil {
|
||||
return fmt.Errorf("failed to move the failing version away: %w", err)
|
||||
}
|
||||
if err := os.Rename(p.Old, p.Current); err != nil {
|
||||
return fmt.Errorf("failed to restore the old version: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Rollback implements /api/update/rollback: actively return to the previous version.
|
||||
// It only does the swap; the restart is likewise left to the supervisor script (the caller then exits with ExitRestart).
|
||||
func Rollback() error {
|
||||
p, err := ResolvePaths()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := os.Stat(p.Old); err != nil {
|
||||
return errors.New("no previous version to roll back to (" + p.Old + " does not exist)")
|
||||
}
|
||||
cleanStaged(p)
|
||||
if err := smokeTest(p.Old); err != nil {
|
||||
return fmt.Errorf("the previous version cannot execute, refusing to roll back: %w", err)
|
||||
}
|
||||
// Swap current and backup: after the rollback you can still roll forward again.
|
||||
tmp := p.Current + ".swap"
|
||||
_ = os.Remove(tmp)
|
||||
if err := os.Rename(p.Current, tmp); err != nil {
|
||||
return fmt.Errorf("failed to move the current version away: %w", err)
|
||||
}
|
||||
if err := os.Rename(p.Old, p.Current); err != nil {
|
||||
_ = os.Rename(tmp, p.Current)
|
||||
return fmt.Errorf("failed to install the previous version: %w", err)
|
||||
}
|
||||
if err := os.Rename(tmp, p.Old); err != nil {
|
||||
log.Printf("[update] failed to tidy up the backup after rollback (does not affect operation): %v", err)
|
||||
}
|
||||
_ = os.Remove(p.Marker)
|
||||
return nil
|
||||
}
|
||||
|
||||
// HasBackup reports whether a previous version exists to roll back to, so the frontend can decide whether to show the rollback button.
|
||||
func HasBackup() bool {
|
||||
p, err := ResolvePaths()
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
_, err = os.Stat(p.Old)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func fileSHA256(path string) (string, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer f.Close()
|
||||
h := sha256.New()
|
||||
if _, err := io.Copy(h, f); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil)), nil
|
||||
}
|
||||
|
||||
func orUnknown(s string) string {
|
||||
if strings.TrimSpace(s) == "" {
|
||||
return "unknown version"
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -1,143 +0,0 @@
|
||||
package selfupdate
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Repo is the release source. Hardcoded rather than configurable: a configurable
|
||||
// update source hands anyone who can change the config a remote code execution
|
||||
// channel — a door a penetration testing platform must never open.
|
||||
const Repo = "Autumn-27/artex"
|
||||
|
||||
// latestURL is GitHub's "latest stable release" endpoint. It automatically skips prereleases and drafts.
|
||||
const latestURL = "https://api.github.com/repos/" + Repo + "/releases/latest"
|
||||
|
||||
// allowedHosts restricts the domains the upgrade chain may visit. Together with
|
||||
// checkRedirect below, any hop redirected to a host outside this list fails
|
||||
// immediately — this is the first gate against DNS poisoning / MITM swapping out
|
||||
// the binary; the second is the SHA256SUMS comparison.
|
||||
var allowedHosts = map[string]bool{
|
||||
"api.github.com": true,
|
||||
"github.com": true,
|
||||
"objects.githubusercontent.com": true, // the object storage where release assets actually land
|
||||
"release-assets.githubusercontent.com": true,
|
||||
"raw.githubusercontent.com": true,
|
||||
}
|
||||
|
||||
// Release holds the fields we care about in a GitHub Release.
|
||||
type Release struct {
|
||||
TagName string `json:"tag_name"`
|
||||
Name string `json:"name"`
|
||||
Body string `json:"body"`
|
||||
Draft bool `json:"draft"`
|
||||
Prerelease bool `json:"prerelease"`
|
||||
PublishedAt time.Time `json:"published_at"`
|
||||
HTMLURL string `json:"html_url"`
|
||||
Assets []Asset `json:"assets"`
|
||||
}
|
||||
|
||||
// Asset is a file attached to a Release.
|
||||
type Asset struct {
|
||||
Name string `json:"name"`
|
||||
URL string `json:"browser_download_url"`
|
||||
Size int64 `json:"size"`
|
||||
}
|
||||
|
||||
// NewClient builds an HTTP client that only trusts GitHub domains. Direct
|
||||
// connection when proxy is empty.
|
||||
//
|
||||
// Deliberately does not reuse the default Transport: the upgrade chain must force
|
||||
// TLS with certificate verification, and must not be affected by things like
|
||||
// InsecureSkipVerify set elsewhere.
|
||||
func NewClient(proxy string) *http.Client {
|
||||
tr := &http.Transport{
|
||||
ForceAttemptHTTP2: true,
|
||||
TLSHandshakeTimeout: 15 * time.Second,
|
||||
}
|
||||
if p := strings.TrimSpace(proxy); p != "" {
|
||||
if pu, err := url.Parse(p); err == nil {
|
||||
tr.Proxy = http.ProxyURL(pu)
|
||||
}
|
||||
}
|
||||
return &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: 30 * time.Minute, // downloading a full package must not be killed by a request-level timeout
|
||||
CheckRedirect: func(req *http.Request, via []*http.Request) error {
|
||||
if len(via) >= 10 {
|
||||
return fmt.Errorf("too many redirects")
|
||||
}
|
||||
return checkURL(req.URL)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// checkURL enforces https + a domain whitelist.
|
||||
func checkURL(u *url.URL) error {
|
||||
if u.Scheme != "https" {
|
||||
return fmt.Errorf("refusing non-HTTPS address: %s", u.Scheme+"://"+u.Host)
|
||||
}
|
||||
if !allowedHosts[strings.ToLower(u.Hostname())] {
|
||||
return fmt.Errorf("refusing non-GitHub domain: %s", u.Hostname())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// FetchLatest queries the latest stable release.
|
||||
func FetchLatest(ctx context.Context, c *http.Client) (*Release, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, latestURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := checkURL(req.URL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Accept", "application/vnd.github+json")
|
||||
req.Header.Set("User-Agent", "artex-selfupdate")
|
||||
|
||||
resp, err := c.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to reach GitHub (a global proxy can be configured in system settings): %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
switch {
|
||||
case resp.StatusCode == http.StatusForbidden, resp.StatusCode == http.StatusTooManyRequests:
|
||||
// Unauthenticated GitHub API allows 60 requests per IP per hour; easy to hit when sharing an egress IP.
|
||||
return nil, fmt.Errorf("GitHub API rate limit (60 per hour), please try again later")
|
||||
case resp.StatusCode == http.StatusNotFound:
|
||||
return nil, fmt.Errorf("repository %s has no stable release yet", Repo)
|
||||
case resp.StatusCode != http.StatusOK:
|
||||
return nil, fmt.Errorf("GitHub returned %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
var rel Release
|
||||
if err := json.NewDecoder(resp.Body).Decode(&rel); err != nil {
|
||||
return nil, fmt.Errorf("failed to parse Release: %w", err)
|
||||
}
|
||||
if strings.TrimSpace(rel.TagName) == "" {
|
||||
return nil, fmt.Errorf("Release is missing a tag")
|
||||
}
|
||||
return &rel, nil
|
||||
}
|
||||
|
||||
// AssetName returns the release package name for the current platform, matching
|
||||
// build.sh's package_binary: artex-<version>-<os>-<arch>.zip (version without the v prefix).
|
||||
func AssetName(tag, goos, goarch string) string {
|
||||
return fmt.Sprintf("artex-%s-%s-%s.zip", strings.TrimPrefix(tag, "v"), goos, goarch)
|
||||
}
|
||||
|
||||
// FindAsset finds an asset by name in the Release.
|
||||
func (r *Release) FindAsset(name string) (Asset, bool) {
|
||||
for _, a := range r.Assets {
|
||||
if strings.EqualFold(a.Name, name) {
|
||||
return a, true
|
||||
}
|
||||
}
|
||||
return Asset{}, false
|
||||
}
|
||||
@@ -1,175 +0,0 @@
|
||||
// Package selfupdate implements ARTEX's one-click update from the web UI: it pulls
|
||||
// the new binary from a GitHub Release, verifies and stages it, and swaps it in atomically on the next start.
|
||||
//
|
||||
// Overall division of labor (see start.sh / start.bat):
|
||||
//
|
||||
// Start script = a dumb supervision loop, responsible only for "deciding whether to relaunch the process after it exits, based on its exit code"
|
||||
// This package = all the error-prone logic (download / SHA256 verify / smoke test / swap / rollback on failure)
|
||||
//
|
||||
// The swap lives in Go rather than the scripts because sha256 verification and the smoke test would need two implementations in sh and bat,
|
||||
// (sha256sum / shasum / certutil), and this is precisely the step that must never fail — swap in a binary that cannot run,
|
||||
// and the supervisor will faithfully relaunch it forever, leaving the user to rescue things on the machine by hand.
|
||||
//
|
||||
// A full upgrade goes through three process starts:
|
||||
//
|
||||
// (1) the old server receives /api/update/apply → download and verify → stage artex.new → exit 75
|
||||
// (2) the script relaunches the old version → Bootstrap spots artex.new → verify + smoke test → swap → exit 75
|
||||
// (3) the script relaunches again, now the new version → Bootstrap records one attempt → clears the marker after a successful start
|
||||
//
|
||||
// Any failed step falls back to the old version: in (2) a failed verification deletes the staged file and keeps the old version running; in (3) three
|
||||
// consecutive crashes before the marker is cleared mean the new version cannot start, so artex.old is swapped back automatically.
|
||||
package selfupdate
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ExitRestart is the "please relaunch me" exit code (EX_TEMPFAIL). When the start script sees it,
|
||||
// it reruns immediately without counting toward crash backoff. 0 means the user stopped it normally (the script exits the loop); anything else is treated as a crash.
|
||||
const ExitRestart = 75
|
||||
|
||||
// maxAttempts is the number of start attempts allowed after the swap. Every start of the new version increments the counter; surviving
|
||||
// settleDelay clears the marker; maxAttempts consecutive crashes mean the new version simply cannot start, so it rolls back automatically.
|
||||
const maxAttempts = 3
|
||||
|
||||
// Paths lists every file an upgrade touches, all anchored under **the executable's directory**.
|
||||
// CWD is deliberately avoided: when running as a service the working directory may be / or any path, and using CWD would drop the staged files
|
||||
// somewhere else, breaking the swap logic outright.
|
||||
type Paths struct {
|
||||
Dir string // directory of the executable
|
||||
Current string // currently running binary artex / artex.exe
|
||||
New string // staged new version artex.new / artex.new.exe
|
||||
Sum string // new version sha256 (hex) artex.new.sha256 / artex.new.exe.sha256
|
||||
Old string // pre-swap backup of old version artex.old / artex.old.exe
|
||||
Marker string // upgrade state marker artex.upgrade.json
|
||||
}
|
||||
|
||||
// ResolvePaths derives all upgrade paths from the current executable.
|
||||
//
|
||||
// On Windows .new/.old must also carry the .exe suffix, otherwise both the smoke test and post-swap execution fail,
|
||||
// so the suffix is stripped off first and re-appended, keeping the naming symmetric across the two platforms.
|
||||
func ResolvePaths() (Paths, error) {
|
||||
exe, err := os.Executable()
|
||||
if err != nil {
|
||||
return Paths{}, fmt.Errorf("locate executable: %w", err)
|
||||
}
|
||||
if resolved, err := filepath.EvalSymlinks(exe); err == nil {
|
||||
exe = resolved
|
||||
}
|
||||
dir := filepath.Dir(exe)
|
||||
name := filepath.Base(exe)
|
||||
ext := filepath.Ext(name) // ".exe" on Windows, usually empty on Unix
|
||||
stem := strings.TrimSuffix(name, ext)
|
||||
|
||||
join := func(suffix string) string { return filepath.Join(dir, stem+suffix+ext) }
|
||||
return Paths{
|
||||
Dir: dir,
|
||||
Current: exe,
|
||||
New: join(".new"),
|
||||
Sum: join(".new") + ".sha256",
|
||||
Old: join(".old"),
|
||||
Marker: filepath.Join(dir, stem+".upgrade.json"),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// marker records the progress of one swap and triggers automatic rollback when the new version cannot start.
|
||||
type marker struct {
|
||||
From string `json:"from"` // version before the upgrade
|
||||
To string `json:"to"` // target version
|
||||
Attempts int `json:"attempts"` // start attempts made after the swap
|
||||
StagedAt int64 `json:"staged_at"`
|
||||
}
|
||||
|
||||
func readMarker(path string) (marker, bool) {
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return marker{}, false
|
||||
}
|
||||
var m marker
|
||||
if json.Unmarshal(b, &m) != nil {
|
||||
return marker{}, false
|
||||
}
|
||||
return m, true
|
||||
}
|
||||
|
||||
func writeMarker(path string, m marker) error {
|
||||
b, err := json.MarshalIndent(m, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.WriteFile(path, b, 0o644)
|
||||
}
|
||||
|
||||
// cleanStaged removes the staged files. Successful swap, failed verification, and user cancellation all go through it, so a leftover
|
||||
// artex.new is not retried on the next start.
|
||||
func cleanStaged(p Paths) {
|
||||
_ = os.Remove(p.New)
|
||||
_ = os.Remove(p.Sum)
|
||||
}
|
||||
|
||||
// CompareVersions compares two version numbers and returns -1/0/1 (a<b / a==b / a>b).
|
||||
// ok=false means at least one side is not a comparable version number (such as a local dev build's "dev" or
|
||||
// a git describe output like "0.3.7-2-gabc1234-dirty"); the caller must disable one-click update then,
|
||||
// otherwise a dev build would be "upgraded" to a release, overwriting uncommitted changes.
|
||||
func CompareVersions(a, b string) (int, bool) {
|
||||
av, aok := parseVersion(a)
|
||||
bv, bok := parseVersion(b)
|
||||
if !aok || !bok {
|
||||
return 0, false
|
||||
}
|
||||
for i := range 3 {
|
||||
if av[i] != bv[i] {
|
||||
if av[i] < bv[i] {
|
||||
return -1, true
|
||||
}
|
||||
return 1, true
|
||||
}
|
||||
}
|
||||
return 0, true
|
||||
}
|
||||
|
||||
// parseVersion parses a "v0.3.7" / "0.3.7" style version number into [3]int.
|
||||
//
|
||||
// Only a clean three-part form is accepted: for non-tag builds build.sh produces suffixed
|
||||
// versions via git describe like "0.3.7-2-gabc1234"; those must be judged non-comparable rather than treated as
|
||||
// 0.3.7 — otherwise a dev build gets misjudged as "already latest" or overwritten by a release.
|
||||
func parseVersion(s string) ([3]int, bool) {
|
||||
s = strings.TrimSpace(s)
|
||||
s = strings.TrimPrefix(s, "v")
|
||||
if s == "" {
|
||||
return [3]int{}, false
|
||||
}
|
||||
parts := strings.Split(s, ".")
|
||||
if len(parts) != 3 {
|
||||
return [3]int{}, false
|
||||
}
|
||||
var out [3]int
|
||||
for i, p := range parts {
|
||||
n, err := strconv.Atoi(p)
|
||||
if err != nil || n < 0 {
|
||||
return [3]int{}, false
|
||||
}
|
||||
out[i] = n
|
||||
}
|
||||
return out, true
|
||||
}
|
||||
|
||||
// InDocker reports whether the process runs inside a container. Under Docker the swap writes to the container's writable layer,
|
||||
// and `docker compose up -d` recreating the container reverts to the version bundled in the image — that is expected behavior
|
||||
// (the user is pulling a new image at that point anyway), but the frontend needs to be able to explain it clearly.
|
||||
func InDocker() bool {
|
||||
if _, err := os.Stat("/.dockerenv"); err == nil {
|
||||
return true
|
||||
}
|
||||
b, err := os.ReadFile("/proc/1/cgroup")
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
s := string(b)
|
||||
return strings.Contains(s, "docker") || strings.Contains(s, "containerd")
|
||||
}
|
||||
@@ -1,459 +0,0 @@
|
||||
package selfupdate
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// testPaths creates an isolated upgrade directory. ResolvePaths() cannot be used directly — it would point at the test binary itself,
|
||||
// and running it once would rename the go test executable.
|
||||
func testPaths(t *testing.T) Paths {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
return Paths{
|
||||
Dir: dir,
|
||||
Current: filepath.Join(dir, "artex"),
|
||||
New: filepath.Join(dir, "artex.new"),
|
||||
Sum: filepath.Join(dir, "artex.new.sha256"),
|
||||
Old: filepath.Join(dir, "artex.old"),
|
||||
Marker: filepath.Join(dir, "artex.upgrade.json"),
|
||||
}
|
||||
}
|
||||
|
||||
// fakeBin writes an executable shell script impersonating artex. smokeTest merely launches it with -h and checks the exit code,
|
||||
// so a script is entirely sufficient and much faster than compiling a real binary.
|
||||
func fakeBin(t *testing.T, path, marker string, exitCode int) {
|
||||
t.Helper()
|
||||
script := "#!/bin/sh\necho " + marker + "\nexit " + itoa(exitCode) + "\n"
|
||||
if err := os.WriteFile(path, []byte(script), 0o755); err != nil {
|
||||
t.Fatalf("write fake binary %s: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func itoa(n int) string {
|
||||
if n == 0 {
|
||||
return "0"
|
||||
}
|
||||
return string(rune('0' + n))
|
||||
}
|
||||
|
||||
// stage arranges bin to look "staged and ready to swap": it writes artex.new and its checksum.
|
||||
func stage(t *testing.T, p Paths, marker string, exitCode int) {
|
||||
t.Helper()
|
||||
fakeBin(t, p.New, marker, exitCode)
|
||||
sum, err := fileSHA256(p.New)
|
||||
if err != nil {
|
||||
t.Fatalf("compute checksum: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(p.Sum, []byte(sum), 0o644); err != nil {
|
||||
t.Fatalf("write checksum: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func readAll(t *testing.T, path string) string {
|
||||
t.Helper()
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", path, err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func requireUnix(t *testing.T) {
|
||||
t.Helper()
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("the fake binary is an sh script and cannot run on Windows")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCompareVersions(t *testing.T) {
|
||||
cases := []struct {
|
||||
a, b string
|
||||
want int
|
||||
comparable bool
|
||||
}{
|
||||
{"0.3.7", "0.3.8", -1, true},
|
||||
{"0.3.8", "0.3.7", 1, true},
|
||||
{"0.3.7", "0.3.7", 0, true},
|
||||
{"v0.3.7", "0.3.8", -1, true}, // build.sh strips the v while tags carry it; both sides must be accepted
|
||||
{"0.3.7", "v0.3.7", 0, true},
|
||||
{"0.9.0", "0.10.0", -1, true}, // compare numerically, not lexicographically
|
||||
{"1.0.0", "0.99.99", 1, true},
|
||||
// Dev builds must be judged non-comparable, otherwise a release overwrites uncommitted changes.
|
||||
{"dev", "0.3.8", 0, false},
|
||||
{"0.3.7-2-gabc1234", "0.3.8", 0, false},
|
||||
{"0.3.7-dirty", "0.3.8", 0, false},
|
||||
{"0.3", "0.3.8", 0, false},
|
||||
{"", "0.3.8", 0, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got, ok := CompareVersions(c.a, c.b)
|
||||
if ok != c.comparable {
|
||||
t.Errorf("CompareVersions(%q,%q) comparable=%v, want %v", c.a, c.b, ok, c.comparable)
|
||||
continue
|
||||
}
|
||||
if ok && got != c.want {
|
||||
t.Errorf("CompareVersions(%q,%q)=%d, want %d", c.a, c.b, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePathsNaming(t *testing.T) {
|
||||
p, err := ResolvePaths()
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePaths: %v", err)
|
||||
}
|
||||
// Key invariant: every upgrade file sits next to the executable. Landing in CWD would break the swap completely when running as a service
|
||||
// (the working directory may be /).
|
||||
for name, path := range map[string]string{"New": p.New, "Sum": p.Sum, "Old": p.Old, "Marker": p.Marker} {
|
||||
if filepath.Dir(path) != p.Dir {
|
||||
t.Errorf("%s is not in the executable's directory: %s (want %s)", name, path, p.Dir)
|
||||
}
|
||||
}
|
||||
// On Windows .new/.old must keep .exe, otherwise the smoke test and post-swap execution both fail.
|
||||
if runtime.GOOS == "windows" {
|
||||
if !strings.HasSuffix(p.New, ".exe") || !strings.HasSuffix(p.Old, ".exe") {
|
||||
t.Errorf("on Windows .new/.old must end with .exe: new=%s old=%s", p.New, p.Old)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyStagedRejectsTamperedBinary(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
stage(t, p, "new", 0)
|
||||
|
||||
// Modify the file after its checksum is written, simulating a corrupted / tampered download.
|
||||
fakeBin(t, p.New, "tampered", 0)
|
||||
if err := verifyStaged(p); err == nil {
|
||||
t.Fatal("expected the SHA256 mismatch to be rejected, but it passed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyStagedRejectsUnrunnableBinary(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
stage(t, p, "broken", 1) // runs but exits non-zero
|
||||
|
||||
if err := verifyStaged(p); err == nil {
|
||||
t.Fatal("expected the smoke-test failure to be rejected, but it passed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyStagedHappyPath(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "old", 0)
|
||||
stage(t, p, "new", 0)
|
||||
if err := writeMarker(p.Marker, marker{From: "0.3.7", To: "0.3.8"}); err != nil {
|
||||
t.Fatalf("write marker: %v", err)
|
||||
}
|
||||
|
||||
action, st := applyStaged(p)
|
||||
if action != Restart {
|
||||
t.Fatalf("want Restart, got %v", action)
|
||||
}
|
||||
if !st.Pending {
|
||||
t.Error("status should be Pending after the swap")
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Current), "new") {
|
||||
t.Error("artex should have been replaced with the new version")
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Old), "old") {
|
||||
t.Error("the old version should be backed up to artex.old")
|
||||
}
|
||||
if _, err := os.Stat(p.New); !os.IsNotExist(err) {
|
||||
t.Error("artex.new should be gone after the swap")
|
||||
}
|
||||
if _, err := os.Stat(p.Sum); !os.IsNotExist(err) {
|
||||
t.Error("the checksum file should be cleaned up after the swap")
|
||||
}
|
||||
// The marker must stay: the next start (running the new version) counts on it and rolls back if necessary.
|
||||
if _, ok := readMarker(p.Marker); !ok {
|
||||
t.Error("the upgrade marker should be kept after the swap")
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyStagedKeepsCurrentWhenVerifyFails(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "old", 0)
|
||||
stage(t, p, "new", 0)
|
||||
fakeBin(t, p.New, "tampered", 0) // break the checksum
|
||||
|
||||
action, st := applyStaged(p)
|
||||
if action != Continue {
|
||||
t.Fatalf("want Continue on verification failure, got %v", action)
|
||||
}
|
||||
if !st.FailedStage {
|
||||
t.Error("status should be marked FailedStage")
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Current), "old") {
|
||||
t.Fatal("the current version must never be touched on verification failure")
|
||||
}
|
||||
if _, err := os.Stat(p.New); !os.IsNotExist(err) {
|
||||
t.Error("the staged file of a failed verification should be cleaned up, otherwise the next start retries it")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSwapOverwritesPreviousBackup(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "v2", 0)
|
||||
fakeBin(t, p.Old, "v1", 0) // backup left by a previous upgrade
|
||||
stage(t, p, "v3", 0)
|
||||
|
||||
if err := swap(p); err != nil {
|
||||
t.Fatalf("swap: %v", err)
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Current), "v3") {
|
||||
t.Error("should have swapped to v3")
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Old), "v2") {
|
||||
t.Error("the backup should be updated to the just-swapped-out v2")
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfirmCountsAttemptsThenRollsBack(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "broken-new", 0)
|
||||
fakeBin(t, p.Old, "good-old", 0)
|
||||
m := marker{From: "0.3.7", To: "0.3.8"}
|
||||
|
||||
// The first maxAttempts starts only increment the counter, giving the new version a chance to settle.
|
||||
for i := 1; i <= maxAttempts; i++ {
|
||||
action, st := confirmOrRollback(p, m)
|
||||
if action != Continue {
|
||||
t.Fatalf("attempt %d: want Continue, got %v", i, action)
|
||||
}
|
||||
if !st.Pending {
|
||||
t.Errorf("attempt %d: status should be Pending", i)
|
||||
}
|
||||
got, ok := readMarker(p.Marker)
|
||||
if !ok || got.Attempts != i {
|
||||
t.Fatalf("after attempt %d, attempts=%d (ok=%v), want %d", i, got.Attempts, ok, i)
|
||||
}
|
||||
m = got
|
||||
}
|
||||
|
||||
// One more crash exceeds the limit; the old version is swapped back automatically.
|
||||
action, st := confirmOrRollback(p, m)
|
||||
if action != Restart {
|
||||
t.Fatalf("want Restart once the attempt limit is exceeded, got %v", action)
|
||||
}
|
||||
if !st.RolledBack {
|
||||
t.Error("status should be marked RolledBack")
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Current), "good-old") {
|
||||
t.Fatal("should have rolled back to the old version")
|
||||
}
|
||||
if _, err := os.Stat(p.Marker); !os.IsNotExist(err) {
|
||||
t.Error("the marker should be cleared after rollback, otherwise rollback loops forever")
|
||||
}
|
||||
// The version that cannot start is kept for diagnosis, not deleted outright.
|
||||
if _, err := os.Stat(p.Current + ".failed"); err != nil {
|
||||
t.Error("the failed version should be kept as .failed for diagnosis")
|
||||
}
|
||||
}
|
||||
|
||||
func TestManualRollbackIsReversible(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "v2", 0)
|
||||
fakeBin(t, p.Old, "v1", 0)
|
||||
|
||||
// Rollback() goes through ResolvePaths(); this tests the underlying swap semantics directly.
|
||||
tmp := p.Current + ".swap"
|
||||
if err := os.Rename(p.Current, tmp); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Rename(p.Old, p.Current); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Rename(tmp, p.Old); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Current), "v1") {
|
||||
t.Error("after rollback the current version should be v1")
|
||||
}
|
||||
if !strings.Contains(readAll(t, p.Old), "v2") {
|
||||
t.Error("after rollback the backup should become v2, so it can roll back again")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseSums(t *testing.T) {
|
||||
const (
|
||||
linuxSum = "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
winSum = "ABCDEF0000000000000000000000000000000000000000000000000000000000"
|
||||
)
|
||||
// sha256sum output is double-space separated; shasum -a 256 in binary mode prefixes the filename with *.
|
||||
raw := linuxSum + " artex-0.3.8-linux-amd64.zip\n" +
|
||||
winSum + " *artex-0.3.8-windows-amd64.zip\n" +
|
||||
"\n" +
|
||||
"garbage line\n" + // exactly two fields, but the first is not a digest
|
||||
"deadbeef artex-0.3.8-darwin-arm64.zip\n" // wrong digest length
|
||||
|
||||
out := parseSums(raw)
|
||||
if out["artex-0.3.8-linux-amd64.zip"] != linuxSum {
|
||||
t.Errorf("linux entry parsed wrong: %v", out)
|
||||
}
|
||||
// Digests are normalized to lowercase so case differences cannot be misjudged as mismatches.
|
||||
if got := out["artex-0.3.8-windows-amd64.zip"]; got != strings.ToLower(winSum) {
|
||||
t.Errorf("windows entry wrong (* prefix should be stripped, digest lowercased): %q", got)
|
||||
}
|
||||
if len(out) != 2 {
|
||||
t.Errorf("blank lines, non-digest lines, and wrong-length lines should be ignored, got %v", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractBinaryFindsNestedEntry(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("the in-archive basename is artex.exe on Windows; this case is built with Unix naming")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
zipPath := filepath.Join(dir, "release.zip")
|
||||
|
||||
f, err := os.Create(zipPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
zw := zip.NewWriter(f)
|
||||
// Real release archive layout: artex-<version>-<os>-<arch>/artex, plus a few distractor files.
|
||||
for name, body := range map[string]string{
|
||||
"artex-0.3.8-linux-amd64/README.md": "readme",
|
||||
"artex-0.3.8-linux-amd64/skills/a.md": "skill",
|
||||
"artex-0.3.8-linux-amd64/artex": "#!/bin/sh\nexit 0\n",
|
||||
"artex-0.3.8-linux-amd64/config.example.json": "{}",
|
||||
} {
|
||||
w, err := zw.Create(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := w.Write([]byte(body)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := zw.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.Close()
|
||||
|
||||
dst := filepath.Join(dir, "out")
|
||||
if err := extractBinary(zipPath, dst); err != nil {
|
||||
t.Fatalf("extractBinary: %v", err)
|
||||
}
|
||||
if got := readAll(t, dst); !strings.Contains(got, "exit 0") {
|
||||
t.Errorf("what was extracted is not the artex executable: %q", got)
|
||||
}
|
||||
info, err := os.Stat(dst)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.Mode().Perm()&0o111 == 0 {
|
||||
t.Error("the extracted binary must carry the executable bit")
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractBinaryMissingEntry(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
zipPath := filepath.Join(dir, "release.zip")
|
||||
f, err := os.Create(zipPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
zw := zip.NewWriter(f)
|
||||
w, _ := zw.Create("artex-0.3.8-linux-amd64/README.md")
|
||||
_, _ = w.Write([]byte("readme"))
|
||||
_ = zw.Close()
|
||||
f.Close()
|
||||
|
||||
if err := extractBinary(zipPath, filepath.Join(dir, "out")); err == nil {
|
||||
t.Fatal("should error when the archive has no executable")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckURLRejectsNonGitHub(t *testing.T) {
|
||||
bad := []string{
|
||||
"http://github.com/x", // not HTTPS
|
||||
"https://evil.com/artex.zip", // domain not on the allowlist
|
||||
"https://github.com.evil.com/x", // suffix spoofing
|
||||
"https://raw.githubusercontent.com.evil.com/x",
|
||||
}
|
||||
for _, raw := range bad {
|
||||
u := mustParse(t, raw)
|
||||
if err := checkURL(u); err == nil {
|
||||
t.Errorf("checkURL(%q) should reject", raw)
|
||||
}
|
||||
}
|
||||
good := []string{
|
||||
"https://api.github.com/repos/x/releases/latest",
|
||||
"https://objects.githubusercontent.com/blah",
|
||||
"https://GitHub.com/x", // domain matching is case-insensitive
|
||||
}
|
||||
for _, raw := range good {
|
||||
u := mustParse(t, raw)
|
||||
if err := checkURL(u); err != nil {
|
||||
t.Errorf("checkURL(%q) should allow, but errored: %v", raw, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssetNameMatchesBuildScript(t *testing.T) {
|
||||
// build.sh's package_binary uses artex-<version>-<os>-<arch>.zip with the v prefix
|
||||
// stripped from the version. One wrong character here and one-click update finds no asset on any platform.
|
||||
if got := AssetName("v0.3.8", "linux", "amd64"); got != "artex-0.3.8-linux-amd64.zip" {
|
||||
t.Errorf("AssetName = %q", got)
|
||||
}
|
||||
if got := AssetName("0.3.8", "windows", "amd64"); got != "artex-0.3.8-windows-amd64.zip" {
|
||||
t.Errorf("AssetName = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func mustParse(t *testing.T, raw string) *url.URL {
|
||||
t.Helper()
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("parse %q: %v", raw, err)
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
func TestSettleClearsMarkerAndStopsRollback(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "new", 0)
|
||||
fakeBin(t, p.Old, "old", 0)
|
||||
if err := writeMarker(p.Marker, marker{From: "0.3.7", To: "0.3.8", Attempts: 2}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
settle(p)
|
||||
|
||||
if _, err := os.Stat(p.Marker); !os.IsNotExist(err) {
|
||||
t.Fatal("the upgrade marker must be cleared once stability is confirmed")
|
||||
}
|
||||
// With the marker gone, later normal restarts neither count attempts nor trigger a spurious rollback.
|
||||
if _, ok := readMarker(p.Marker); ok {
|
||||
t.Error("reading the marker should fail")
|
||||
}
|
||||
// The backup stays so the user can still roll back manually.
|
||||
if _, err := os.Stat(p.Old); err != nil {
|
||||
t.Error("the previous version's backup should still be kept once stability is confirmed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSettleIsNoopWithoutMarker(t *testing.T) {
|
||||
requireUnix(t)
|
||||
p := testPaths(t)
|
||||
fakeBin(t, p.Current, "cur", 0)
|
||||
settle(p) // the plain start path: must not panic or touch any file
|
||||
if _, err := os.Stat(p.Current); err != nil {
|
||||
t.Error("settle must not affect any file when there is no marker")
|
||||
}
|
||||
}
|
||||
@@ -1,339 +0,0 @@
|
||||
package selfupdate
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// sumsAsset is the checksum manifest generated by release.yml, covering every zip in the Release.
|
||||
const sumsAsset = "SHA256SUMS"
|
||||
|
||||
// maxBinarySize caps the size of the extracted binary so a malformed zip cannot fill the disk.
|
||||
const maxBinarySize = 512 << 20 // 512 MiB
|
||||
|
||||
// Phase is a stage of the upgrade process, used directly as the phase field of SSE events.
|
||||
type Phase string
|
||||
|
||||
const (
|
||||
PhaseIdle Phase = "idle"
|
||||
PhaseDownload Phase = "downloading"
|
||||
PhaseVerify Phase = "verifying"
|
||||
PhaseExtract Phase = "extracting"
|
||||
PhaseStaged Phase = "staged"
|
||||
PhaseFailed Phase = "failed"
|
||||
)
|
||||
|
||||
// Progress is supplied by the caller to push progress to the frontend. pct is only meaningful during the download phase (0-100);
|
||||
// other phases pass -1.
|
||||
type Progress func(ph Phase, pct int, msg string)
|
||||
|
||||
// Stage downloads the current platform's release archive from the given Release, verifies it, and stages the new binary as artex.new.
|
||||
//
|
||||
// It uses the full zip rather than a bare binary for two reasons: existing Releases' SHA256SUMS only cover zips anyway,
|
||||
// so going through the zip needs no CI change and stays compatible with already-published versions; and the zip also carries
|
||||
// skills/, leaving the door open for syncing built-in skills later. The only cost is those few hundred KB of skills.
|
||||
//
|
||||
// Returning means staging is complete; the caller then shuts down gracefully and exits with ExitRestart.
|
||||
func Stage(ctx context.Context, c *http.Client, rel *Release, currentVersion string, prog Progress) error {
|
||||
if prog == nil {
|
||||
prog = func(Phase, int, string) {}
|
||||
}
|
||||
p, err := ResolvePaths()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := checkWritable(p.Dir); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
name := AssetName(rel.TagName, runtime.GOOS, runtime.GOARCH)
|
||||
asset, ok := rel.FindAsset(name)
|
||||
if !ok {
|
||||
return fmt.Errorf("this release ships no package for %s/%s (missing %s)", runtime.GOOS, runtime.GOARCH, name)
|
||||
}
|
||||
|
||||
prog(PhaseDownload, 0, "Fetching checksum manifest…")
|
||||
sums, err := fetchSums(ctx, c, rel)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
want, ok := sums[name]
|
||||
if !ok {
|
||||
return fmt.Errorf("%s does not list %s; refusing to install an unverified binary", sumsAsset, name)
|
||||
}
|
||||
|
||||
// All temp files land in the target directory so the final rename stays an atomic operation within one filesystem
|
||||
// (a cross-device rename fails, and /tmp is often a separate mount point).
|
||||
zipPath := p.New + ".zip.part"
|
||||
binPath := p.New + ".part"
|
||||
defer func() {
|
||||
_ = os.Remove(zipPath)
|
||||
_ = os.Remove(binPath)
|
||||
}()
|
||||
|
||||
prog(PhaseDownload, 0, fmt.Sprintf("Downloading %s (%s)…", name, humanSize(asset.Size)))
|
||||
got, err := download(ctx, c, asset, zipPath, prog)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
prog(PhaseVerify, -1, "Verifying SHA256…")
|
||||
if !strings.EqualFold(got, want) {
|
||||
return fmt.Errorf("SHA256 mismatch: want %s, got %s (corrupted download or tampering)", short(want), short(got))
|
||||
}
|
||||
|
||||
prog(PhaseExtract, -1, "Extracting and smoke testing…")
|
||||
if err := extractBinary(zipPath, binPath); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := smokeTest(binPath); err != nil {
|
||||
return fmt.Errorf("the new version cannot run on this system: %w", err)
|
||||
}
|
||||
|
||||
// The staged file's own sha256 is stored separately: it gets verified once more before the swap on the next start,
|
||||
// guarding against the file being altered or corrupted between staging and restart.
|
||||
binSum, err := fileSHA256(binPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("compute new binary checksum: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(p.Sum, []byte(binSum), 0o644); err != nil {
|
||||
return fmt.Errorf("write checksum: %w", err)
|
||||
}
|
||||
if err := os.Rename(binPath, p.New); err != nil {
|
||||
_ = os.Remove(p.Sum)
|
||||
return fmt.Errorf("stage new version: %w", err)
|
||||
}
|
||||
|
||||
if err := writeMarker(p.Marker, marker{
|
||||
From: currentVersion,
|
||||
To: strings.TrimPrefix(rel.TagName, "v"),
|
||||
StagedAt: time.Now().Unix(),
|
||||
}); err != nil {
|
||||
// The marker only affects automatic rollback; the staged file itself is already in place, so the upgrade is not interrupted over it.
|
||||
prog(PhaseStaged, -1, "Warning: writing the upgrade marker failed; this upgrade will have no automatic rollback protection")
|
||||
}
|
||||
|
||||
prog(PhaseStaged, 100, "New version ready, restarting…")
|
||||
return nil
|
||||
}
|
||||
|
||||
// fetchSums downloads and parses SHA256SUMS, returning filename → hex digest.
|
||||
func fetchSums(ctx context.Context, c *http.Client, rel *Release) (map[string]string, error) {
|
||||
asset, ok := rel.FindAsset(sumsAsset)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("this Release has no %s; integrity cannot be verified, refusing to upgrade", sumsAsset)
|
||||
}
|
||||
body, err := get(ctx, c, asset.URL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download %s: %w", sumsAsset, err)
|
||||
}
|
||||
defer body.Close()
|
||||
|
||||
raw, err := io.ReadAll(io.LimitReader(body, 1<<20))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read %s: %w", sumsAsset, err)
|
||||
}
|
||||
out := parseSums(string(raw))
|
||||
if len(out) == 0 {
|
||||
return nil, fmt.Errorf("%s is empty or in an unrecognized format", sumsAsset)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// parseSums parses a sha256sum-style manifest, returning filename → hex digest.
|
||||
//
|
||||
// The first field must be 64 hex characters to be included. Checking only for "exactly two fields" is not enough —
|
||||
// any line of two-word prose would pass as a valid entry and stuff garbage into the digest table,
|
||||
// letting a real asset match a wrong digest.
|
||||
func parseSums(raw string) map[string]string {
|
||||
out := map[string]string{}
|
||||
for line := range strings.Lines(raw) {
|
||||
// The format is "<sha256> <filename>" (sha256sum uses a double space; shasum's binary
|
||||
// mode prefixes the filename with *).
|
||||
fields := strings.Fields(strings.TrimSpace(line))
|
||||
if len(fields) != 2 || !isHexSHA256(fields[0]) {
|
||||
continue
|
||||
}
|
||||
name := strings.TrimPrefix(fields[1], "*")
|
||||
if name == "" {
|
||||
continue
|
||||
}
|
||||
out[name] = strings.ToLower(fields[0])
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func isHexSHA256(s string) bool {
|
||||
if len(s) != 64 {
|
||||
return false
|
||||
}
|
||||
for _, c := range s {
|
||||
switch {
|
||||
case c >= '0' && c <= '9', c >= 'a' && c <= 'f', c >= 'A' && c <= 'F':
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// download writes the asset to dst, computing SHA256 along the way and reporting progress against Content-Length.
|
||||
func download(ctx context.Context, c *http.Client, a Asset, dst string, prog Progress) (string, error) {
|
||||
body, err := get(ctx, c, a.URL)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("download %s: %w", a.Name, err)
|
||||
}
|
||||
defer body.Close()
|
||||
|
||||
f, err := os.Create(dst)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create temp file: %w", err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
h := sha256.New()
|
||||
pw := &progressWriter{total: a.Size, prog: prog, name: a.Name, last: time.Now()}
|
||||
if _, err := io.Copy(io.MultiWriter(f, h, pw), body); err != nil {
|
||||
return "", fmt.Errorf("download interrupted: %w", err)
|
||||
}
|
||||
if err := f.Sync(); err != nil {
|
||||
return "", fmt.Errorf("flush to disk failed: %w", err)
|
||||
}
|
||||
if a.Size > 0 && pw.written != a.Size {
|
||||
return "", fmt.Errorf("incomplete download: want %d bytes, got %d bytes", a.Size, pw.written)
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil)), nil
|
||||
}
|
||||
|
||||
// get issues an allowlist-constrained GET and returns the response body.
|
||||
func get(ctx context.Context, c *http.Client, rawURL string) (io.ReadCloser, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := checkURL(req.URL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "artex-selfupdate")
|
||||
resp, err := c.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
resp.Body.Close()
|
||||
return nil, fmt.Errorf("HTTP %d", resp.StatusCode)
|
||||
}
|
||||
return resp.Body, nil
|
||||
}
|
||||
|
||||
// extractBinary pulls the artex executable out of the release archive.
|
||||
//
|
||||
// The archive layout is artex-<version>-<os>-<arch>/artex, but this matches by **basename** instead of building the full path:
|
||||
// the version already appears once in the archive name, and one wrong character would fail the whole upgrade; matching by basename is more robust to changes.
|
||||
func extractBinary(zipPath, dst string) error {
|
||||
want := "artex"
|
||||
if runtime.GOOS == "windows" {
|
||||
want = "artex.exe"
|
||||
}
|
||||
zr, err := zip.OpenReader(zipPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open release archive: %w", err)
|
||||
}
|
||||
defer zr.Close()
|
||||
|
||||
for _, entry := range zr.File {
|
||||
if entry.FileInfo().IsDir() || !strings.EqualFold(path.Base(entry.Name), want) {
|
||||
continue
|
||||
}
|
||||
rc, err := entry.Open()
|
||||
if err != nil {
|
||||
return fmt.Errorf("read %s: %w", entry.Name, err)
|
||||
}
|
||||
defer rc.Close()
|
||||
|
||||
f, err := os.OpenFile(dst, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o755)
|
||||
if err != nil {
|
||||
return fmt.Errorf("write out new binary: %w", err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
n, err := io.Copy(f, io.LimitReader(rc, maxBinarySize+1))
|
||||
if err != nil {
|
||||
return fmt.Errorf("extract %s: %w", entry.Name, err)
|
||||
}
|
||||
if n > maxBinarySize {
|
||||
return fmt.Errorf("executable in the release archive exceeds %s; refusing to extract", humanSize(maxBinarySize))
|
||||
}
|
||||
if n == 0 {
|
||||
return fmt.Errorf("%s in the release archive is an empty file", want)
|
||||
}
|
||||
return f.Sync()
|
||||
}
|
||||
return fmt.Errorf("%s not found in the release archive", want)
|
||||
}
|
||||
|
||||
// checkWritable confirms up front that the directory is writable. Without this step, running as non-root or having the binary in a system
|
||||
// directory would only fail at the very moment of the swap, after tens of MB have already been downloaded.
|
||||
func checkWritable(dir string) error {
|
||||
probe, err := os.CreateTemp(dir, ".artex-update-probe-*")
|
||||
if err != nil {
|
||||
return fmt.Errorf("program directory %s is not writable; automatic update is impossible (check permissions or upgrade manually): %w", dir, err)
|
||||
}
|
||||
name := probe.Name()
|
||||
_ = probe.Close()
|
||||
_ = os.Remove(name)
|
||||
return nil
|
||||
}
|
||||
|
||||
// progressWriter counts written bytes and reports at a throttled rate, avoiding one SSE event per 32KiB chunk.
|
||||
type progressWriter struct {
|
||||
total int64
|
||||
written int64
|
||||
name string
|
||||
prog Progress
|
||||
last time.Time
|
||||
}
|
||||
|
||||
func (w *progressWriter) Write(b []byte) (int, error) {
|
||||
w.written += int64(len(b))
|
||||
if time.Since(w.last) < 300*time.Millisecond {
|
||||
return len(b), nil
|
||||
}
|
||||
w.last = time.Now()
|
||||
pct := -1
|
||||
if w.total > 0 {
|
||||
pct = int(w.written * 100 / w.total)
|
||||
}
|
||||
w.prog(PhaseDownload, pct, fmt.Sprintf("Downloading %s / %s", humanSize(w.written), humanSize(w.total)))
|
||||
return len(b), nil
|
||||
}
|
||||
|
||||
func humanSize(n int64) string {
|
||||
const unit = 1024
|
||||
if n < unit {
|
||||
return fmt.Sprintf("%d B", n)
|
||||
}
|
||||
div, exp := int64(unit), 0
|
||||
for v := n / unit; v >= unit; v /= unit {
|
||||
div *= unit
|
||||
exp++
|
||||
}
|
||||
return fmt.Sprintf("%.1f %cB", float64(n)/float64(div), "KMGT"[exp])
|
||||
}
|
||||
|
||||
func short(sum string) string {
|
||||
if len(sum) > 12 {
|
||||
return sum[:12] + "…"
|
||||
}
|
||||
return sum
|
||||
}
|
||||
@@ -103,7 +103,7 @@ func (s *Server) pgCreateConversation(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
if req.LLMProfileID != nil {
|
||||
if _, ok := s.loadProfileConfig(*req.LLMProfileID); !ok {
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or has no API Key set")
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or is not usable (no API key for the key-requiring formats)")
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -136,7 +136,7 @@ func (s *Server) pgUpdateConversation(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
if req.LLMProfileID != nil {
|
||||
if _, ok := s.loadProfileConfig(*req.LLMProfileID); !ok {
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or has no API Key set")
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or is not usable (no API key for the key-requiring formats)")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
package server
|
||||
|
||||
import "os"
|
||||
|
||||
// secretEnv lists the credentials read from the environment only at startup:
|
||||
// ARTEX_PG_DSN by NewManager, the API keys by agent.FromEnv inside New.
|
||||
var secretEnv = []string{"ARTEX_PG_DSN", "ANTHROPIC_API_KEY", "OPENAI_API_KEY"}
|
||||
|
||||
// ScrubSecretEnv removes secretEnv from the process environment once startup has
|
||||
// read it. Child processes (agent shells, custom tools, MCP stdio servers) copy
|
||||
// os.Environ() when they are spawned, so they no longer inherit these values.
|
||||
//
|
||||
// It runs inside New, after both readers and before the MCP discovery goroutine
|
||||
// and the task resume loop: both can spawn children, and resuming a task starts
|
||||
// workers that run shells. Scrubbing in main after New returned left that window
|
||||
// open.
|
||||
func ScrubSecretEnv() {
|
||||
for _, k := range secretEnv {
|
||||
_ = os.Unsetenv(k)
|
||||
}
|
||||
}
|
||||
|
||||
// Option configures New. Options keep the test call sites unchanged: only the
|
||||
// binary opts into scrubbing, because tests create several servers in one
|
||||
// process and still need ARTEX_PG_DSN on later calls.
|
||||
type Option func(*Server)
|
||||
|
||||
// WithSecretScrub makes New drop the startup secrets after reading them, before
|
||||
// the MCP discovery goroutine and the task resume loop can spawn children.
|
||||
func WithSecretScrub() Option { return func(s *Server) { s.scrubSecrets = true } }
|
||||
@@ -0,0 +1,36 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A child process spawned after startup must not see the startup secrets, while
|
||||
// other variables still pass through.
|
||||
func TestScrubSecretEnvHidesSecretsFromChildren(t *testing.T) {
|
||||
sh, err := exec.LookPath("sh")
|
||||
if err != nil {
|
||||
t.Skip("no sh in PATH")
|
||||
}
|
||||
for _, k := range secretEnv {
|
||||
t.Setenv(k, "leak-"+k)
|
||||
}
|
||||
t.Setenv("ARTEX_SCRUB_KEEP", "kept")
|
||||
|
||||
ScrubSecretEnv()
|
||||
|
||||
out, err := exec.Command(sh, "-c", "env").Output()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env := "\n" + string(out)
|
||||
for _, k := range secretEnv {
|
||||
if strings.Contains(env, "\n"+k+"=") {
|
||||
t.Errorf("%s still reaches child processes", k)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(env, "\nARTEX_SCRUB_KEEP=kept\n") {
|
||||
t.Errorf("non-secret variable did not reach the child process")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,272 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/artex/agent"
|
||||
"github.com/Autumn-27/artex/db"
|
||||
"github.com/Autumn-27/norma/llm"
|
||||
)
|
||||
|
||||
// ConfigFrom("ollama", ...) must normalize to the keyless sentinel with our own
|
||||
// defaults, never the SDK's OpenAI-public-endpoint fallback.
|
||||
func TestOllamaConfigFromKeylessDefaults(t *testing.T) {
|
||||
c := agent.ConfigFrom("ollama", "", "", "", "")
|
||||
if c.Format != agent.FormatOllama {
|
||||
t.Fatalf("format=%v, want FormatOllama", c.Format)
|
||||
}
|
||||
if c.Provider() != "ollama" {
|
||||
t.Fatalf("Provider()=%q", c.Provider())
|
||||
}
|
||||
if c.BaseURL != agent.DefaultOllamaBaseURL {
|
||||
t.Fatalf("BaseURL=%q, want default %q", c.BaseURL, agent.DefaultOllamaBaseURL)
|
||||
}
|
||||
if c.Model == "" {
|
||||
t.Fatal("model must default to a non-empty value")
|
||||
}
|
||||
if c.NeedsAPIKey() {
|
||||
t.Fatal("ollama must not require an API key")
|
||||
}
|
||||
// A full endpoint URL is tolerated (the provider appends /chat/completions).
|
||||
e := agent.ConfigFrom("ollama", "llama3.2", "http://box:11434/v1/chat/completions", "", "")
|
||||
if e.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("full endpoint URL not normalized: %q", e.BaseURL)
|
||||
}
|
||||
// A custom key is preserved (e.g. an authenticated Ollama gateway).
|
||||
k := agent.ConfigFrom("ollama", "m", "http://box:11434", "ollama-secret", "")
|
||||
if k.APIKey != "ollama-secret" {
|
||||
t.Fatalf("custom key lost: %q", k.APIKey)
|
||||
}
|
||||
// The natural Ollama URL has no /v1, but the provider appends
|
||||
// /chat/completions, so the base must be normalized to the /v1 root.
|
||||
n := agent.ConfigFrom("ollama", "m", "http://box:11434", "", "")
|
||||
if n.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("bare host URL not normalized: %q", n.BaseURL)
|
||||
}
|
||||
}
|
||||
|
||||
// The provider built from an Ollama config speaks OpenAI Chat Completions on
|
||||
// the wire (the SDK appends /chat/completions) with the placeholder Bearer (no
|
||||
// real key was configured, no OPENAI_API_KEY env leak).
|
||||
func TestOllamaProviderWiresOpenAIProtocol(t *testing.T) {
|
||||
var gotPath, gotAuth, gotModel atomic.Value
|
||||
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotPath.Store(r.URL.Path)
|
||||
gotAuth.Store(r.Header.Get("Authorization"))
|
||||
var body struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
gotModel.Store(body.Model)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"id":"1","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`))
|
||||
}))
|
||||
t.Cleanup(fake.Close)
|
||||
|
||||
cfg := agent.ConfigFrom("ollama", "llama3.2", fake.URL, "", "")
|
||||
cfg.Stream = false // non-streaming: the fake answers one plain JSON body
|
||||
prov, err := cfg.NewProvider()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
req := llm.CompletionRequest{
|
||||
System: []string{"You are a connection test. Output OK."},
|
||||
Messages: []llm.Message{llm.UserText("ping")},
|
||||
}
|
||||
if _, _, _, err = prov.Complete(ctx, req); err != nil {
|
||||
t.Fatalf("Complete: %v", err)
|
||||
}
|
||||
// ConfigFrom normalizes the fake server URL (no /v1) onto the OpenAI root,
|
||||
// so the request must land on /v1/chat/completions. A path of /chat/completions
|
||||
// would mean the base URL leaked through unnormalized (or fell back to OpenAI).
|
||||
if p, _ := gotPath.Load().(string); p != "/v1/chat/completions" {
|
||||
t.Fatalf("path=%q, want /v1/chat/completions", p)
|
||||
}
|
||||
if a, _ := gotAuth.Load().(string); a != "Bearer ollama" {
|
||||
t.Fatalf("Authorization=%q, want placeholder Bearer ollama", a)
|
||||
}
|
||||
if m, _ := gotModel.Load().(string); m != "llama3.2" {
|
||||
t.Fatalf("model on wire=%q", m)
|
||||
}
|
||||
}
|
||||
|
||||
// A keyless Ollama profile must be usable end to end: stored round-trips,
|
||||
// loadProfileConfig accepts it, the task chain validates it, the failover pool
|
||||
// includes it, and the connection test completes against a fake Ollama.
|
||||
func TestOllamaKeylessProfileLifecycle(t *testing.T) {
|
||||
m, err := NewManager(t.TempDir(), "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { m.Close() })
|
||||
if m.pg == nil {
|
||||
t.Skip("no postgres for this run")
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
|
||||
|
||||
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"id":"1","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`))
|
||||
}))
|
||||
t.Cleanup(fake.Close)
|
||||
|
||||
pid, err := m.pg.SaveProfile(&db.LLMProfile{
|
||||
Name: "ollama-local", Format: "ollama", Model: "llama3.2",
|
||||
BaseURL: fake.URL, APIKey: "", IsDefault: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _, _ = m.pg.Exec("DELETE FROM llm_profiles WHERE id=$1", pid) })
|
||||
|
||||
stored, err := m.pg.ProfileByID(pid)
|
||||
if err != nil || stored == nil {
|
||||
t.Fatalf("ProfileByID: %v %v", stored, err)
|
||||
}
|
||||
|
||||
if _, ok := s.loadProfileConfig(pid); !ok {
|
||||
t.Fatal("keyless ollama profile must load")
|
||||
}
|
||||
if err := s.validateTaskProfileIDs([]int64{pid}); err != nil {
|
||||
t.Fatalf("task chain must accept keyless ollama: %v", err)
|
||||
}
|
||||
pool, err := m.pg.PoolProfiles()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found := false
|
||||
for _, p := range pool {
|
||||
if p.ID == pid {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("keyless ollama profile must join the failover pool")
|
||||
}
|
||||
|
||||
// Connection test: no api_key in the form either.
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"profile_id": pid, "provider": "ollama", "model": "llama3.2",
|
||||
"base_url": fake.URL, "api_key": "", "streaming": false,
|
||||
})
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/llm/test", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
s.testLLM(rec, req)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("handler status %d", rec.Code)
|
||||
}
|
||||
var resp struct {
|
||||
OK bool `json:"ok"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
|
||||
if !resp.OK {
|
||||
t.Fatalf("connection test on keyless ollama failed: %s", resp.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// Model listing for an Ollama server works without an API key and tolerates a
|
||||
// base URL with or without the /v1 suffix.
|
||||
func TestOllamaListModels(t *testing.T) {
|
||||
var calls atomic.Value // []string paths
|
||||
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
paths, _ := calls.Load().([]string)
|
||||
calls.Store(append(paths, r.URL.Path))
|
||||
_, _ = w.Write([]byte(`{"object":"list","data":[{"id":"llama3.2"},{"id":"qwen2.5:14b"}]}`))
|
||||
}))
|
||||
t.Cleanup(fake.Close)
|
||||
|
||||
m, err := NewManager(t.TempDir(), "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { m.Close() })
|
||||
if m.pg == nil {
|
||||
t.Skip("no postgres for this run")
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
|
||||
|
||||
call := func(baseURL string) (bool, []string) {
|
||||
body, _ := json.Marshal(map[string]any{"provider": "ollama", "base_url": baseURL, "api_key": ""})
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/llm/models", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
s.pgListModels(rec, req)
|
||||
var resp struct {
|
||||
OK bool `json:"ok"`
|
||||
Models []string `json:"models"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
|
||||
return resp.OK, resp.Models
|
||||
}
|
||||
// Base WITH /v1.
|
||||
ok, models := call(fake.URL + "/v1")
|
||||
if !ok || len(models) != 2 {
|
||||
t.Fatalf("listing with /v1 base failed: ok=%v models=%v", ok, models)
|
||||
}
|
||||
// Base WITHOUT /v1 must still be normalized onto the OpenAI-compatible path
|
||||
// (/v1/models); both calls hit the same endpoint.
|
||||
ok2, models2 := call(fake.URL)
|
||||
if !ok2 || len(models2) != 2 {
|
||||
t.Fatalf("listing without /v1 base failed: ok=%v models=%v", ok2, models2)
|
||||
}
|
||||
if paths, _ := calls.Load().([]string); len(paths) != 2 || paths[0] != "/v1/models" || paths[1] != "/v1/models" {
|
||||
t.Fatalf("both calls must hit /v1/models, got %v", paths)
|
||||
}
|
||||
}
|
||||
|
||||
// The environment path must normalize the base URL too: ARTEX_LLM_BASE_URL is
|
||||
// the natural way to configure a LAN Ollama server, and it used to send chat to
|
||||
// /chat/completions instead of /v1/chat/completions.
|
||||
func TestOllamaFromEnvNormalizesBaseURL(t *testing.T) {
|
||||
t.Setenv("ARTEX_LLM_PROVIDER", "ollama")
|
||||
t.Setenv("OPENAI_API_KEY", "")
|
||||
t.Setenv("ARTEX_LLM_BASE_URL", "http://box:11434")
|
||||
c, ok := agent.FromEnv()
|
||||
if !ok {
|
||||
t.Fatal("ollama env config must be usable")
|
||||
}
|
||||
if c.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("env base URL not normalized: %q", c.BaseURL)
|
||||
}
|
||||
t.Setenv("ARTEX_LLM_BASE_URL", "http://box:11434/v1/chat/completions")
|
||||
c, ok = agent.FromEnv()
|
||||
if !ok {
|
||||
t.Fatal("ollama env config must be usable")
|
||||
}
|
||||
if c.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("full endpoint URL not normalized: %q", c.BaseURL)
|
||||
}
|
||||
}
|
||||
|
||||
// An Ollama config resolved from the environment must not inherit OPENAI_API_KEY:
|
||||
// Ollama ignores the bearer token, so inheritance would only ship the cloud key
|
||||
// to the local or LAN endpoint.
|
||||
func TestOllamaFromEnvDoesNotInheritOpenAIKey(t *testing.T) {
|
||||
t.Setenv("ARTEX_LLM_PROVIDER", "ollama")
|
||||
t.Setenv("OPENAI_API_KEY", "sk-cloud-secret")
|
||||
t.Setenv("ARTEX_LLM_BASE_URL", "")
|
||||
c, ok := agent.FromEnv()
|
||||
if !ok {
|
||||
t.Fatal("ollama env config must be usable without a key")
|
||||
}
|
||||
if c.APIKey != "" {
|
||||
t.Fatalf("ollama inherited a cloud key: %q", c.APIKey)
|
||||
}
|
||||
}
|
||||
@@ -293,7 +293,7 @@ func (s *Server) toolSpawnTask() actool.CoreTool {
|
||||
var pin *int64
|
||||
if id := parseProfileID(a.LLMProfileID); id > 0 {
|
||||
if _, ok := s.loadProfileConfig(id); !ok {
|
||||
return actool.Errorf(fmt.Sprintf("LLM profile #%d not found or has no API key set", id)), nil
|
||||
return actool.Errorf(fmt.Sprintf("LLM profile #%d not found or not usable (no API key for the key-requiring formats)", id)), nil
|
||||
}
|
||||
pin = &id
|
||||
} else if a.ParentRef != "" {
|
||||
|
||||
+43
-36
@@ -46,6 +46,8 @@ type Server struct {
|
||||
skillDir string // root directory for skill subdirectories
|
||||
jwtKey []byte // HS256 signing key loaded from / generated into dataDir/jwt.key
|
||||
|
||||
scrubSecrets bool // drop ARTEX_PG_DSN and the API keys from the environment once read (binary only)
|
||||
|
||||
// concMu serializes concurrency-cap decisions (admission + reconcile) so a
|
||||
// scheduler tick and an HTTP settings change / task creation can't both count
|
||||
// free slots off the same snapshot and over-promote past the limit.
|
||||
@@ -140,7 +142,7 @@ type triggeredRun struct {
|
||||
mergeable bool // true for finding/goal event triggers (merge by taskID)
|
||||
}
|
||||
|
||||
func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDir string) *Server {
|
||||
func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDir string, opts ...Option) *Server {
|
||||
key, err := loadOrCreateJWTKey(keyDir, dataDir)
|
||||
if err != nil {
|
||||
log.Fatalf("[auth] JWT key: %v", err)
|
||||
@@ -232,9 +234,6 @@ func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDi
|
||||
// Vulnerability IM push delivery engine. Sits beside the Scheduler but independent: push latency demands (3s)
|
||||
// differ from the triggers' business cadence, and neither's failures drag the other down — a stuck push must not affect agent triggers.
|
||||
go newNotifier(s).Run(s.ctx)
|
||||
// Fill the tool cache for any enabled MCP that has none yet (notably the
|
||||
// seeded browser MCP on first run). Async so it never blocks startup.
|
||||
go s.discoverEmptyMCPsOnStartup()
|
||||
logSink.SetDB(ctx, m.pg) // restore last 100 log rows and enable async persistence
|
||||
}
|
||||
// precedence: persisted DB config > env.
|
||||
@@ -253,6 +252,16 @@ func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDi
|
||||
} else {
|
||||
log.Printf("[engine] no LLM provider configured — engine idle until set via /api/llm or env")
|
||||
}
|
||||
// Both env readers have now run (NewManager for the DSN, FromEnv above for the
|
||||
// keys). Drop the secrets before anything below can spawn a child process:
|
||||
// restoring runtimes resumes workers that run shells, and MCP discovery starts
|
||||
// stdio servers.
|
||||
if s.scrubSecrets {
|
||||
ScrubSecretEnv()
|
||||
}
|
||||
// Fill the tool cache for any enabled MCP that has none yet (notably the
|
||||
// seeded browser MCP on first run). Async so it never blocks startup.
|
||||
go s.discoverEmptyMCPsOnStartup()
|
||||
s.restoreTaskRuntimes()
|
||||
go s.reconcileConcurrency()
|
||||
s.startTaskArchiveWorker()
|
||||
@@ -335,7 +344,8 @@ func (s *Server) loadLLMConfig() (agent.Config, bool) {
|
||||
cfg.MaxTokens, cfg.MaxTokensField = p.MaxTokens, p.MaxTokensField
|
||||
cfg.SessionHeaderKey = p.SessionHeaderKey
|
||||
s.applyProfileRetry(&cfg, p)
|
||||
if cfg.APIKey == "" {
|
||||
// Ollama profiles run keyless (local server); the other formats need a key.
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
return cfg, false
|
||||
}
|
||||
s.cfgMu.Lock()
|
||||
@@ -491,7 +501,8 @@ func (s *Server) applyLLM(cfg agent.Config) error {
|
||||
}
|
||||
|
||||
// loadProfileConfig builds an agent.Config from a specific profile id (with its key).
|
||||
// ok=false when the profile is missing or has no api key.
|
||||
// ok=false when the profile is missing or unusable (no api key for the key-requiring
|
||||
// formats).
|
||||
func (s *Server) loadProfileConfig(id int64) (agent.Config, bool) {
|
||||
p, err := s.m.pg.ProfileByID(id)
|
||||
if err != nil || p == nil {
|
||||
@@ -506,7 +517,7 @@ func (s *Server) loadProfileConfig(id int64) (agent.Config, bool) {
|
||||
cfg.MaxTokens, cfg.MaxTokensField = p.MaxTokens, p.MaxTokensField
|
||||
cfg.SessionHeaderKey = p.SessionHeaderKey
|
||||
s.applyProfileRetry(&cfg, p)
|
||||
if cfg.APIKey == "" {
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
return cfg, false
|
||||
}
|
||||
return cfg, true
|
||||
@@ -664,13 +675,6 @@ func (s *Server) Handler() http.Handler {
|
||||
mux.HandleFunc("GET /api/logs/history", s.getLogsHistory)
|
||||
mux.HandleFunc("GET /api/logs/stream", s.streamLogs)
|
||||
|
||||
// One-click page update. These go through the default JWT auth (auth.go only exempts /api/auth/* and
|
||||
// /api/health), so the endpoints that modify the program itself naturally require a login.
|
||||
mux.HandleFunc("GET /api/update/check", s.updateCheck)
|
||||
mux.HandleFunc("POST /api/update/apply", s.updateApply)
|
||||
mux.HandleFunc("POST /api/update/rollback", s.updateRollback)
|
||||
mux.HandleFunc("GET /api/update/stream", s.updateStream)
|
||||
|
||||
mux.HandleFunc("GET /api/tasks", s.listTasks)
|
||||
mux.HandleFunc("POST /api/tasks", s.createTask)
|
||||
mux.HandleFunc("GET /api/skill-gaps", s.skillGaps)
|
||||
@@ -1350,12 +1354,13 @@ func (s *Server) setLLM(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
cfg.ContextWindowK = k
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
// Keyless formats (Ollama) need no key; the others keep/require one.
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
s.cfgMu.Lock()
|
||||
cfg.APIKey = s.llmCfg.APIKey // keep existing key if not re-entered
|
||||
s.cfgMu.Unlock()
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
writeErr(w, 400, "api_key required")
|
||||
return
|
||||
}
|
||||
@@ -1448,28 +1453,30 @@ func (s *Server) testLLM(w http.ResponseWriter, r *http.Request) {
|
||||
// see TestConnection). Endpoints like opencode zen that mandate x-opencode-session return
|
||||
// 400 without it, so the test path must send it too — otherwise "chat works, test 400s".
|
||||
cfg.SessionHeaderKey = req.SessionHeaderKey
|
||||
// API key resolution precedence: form input > the named profile's stored key > the global config's key.
|
||||
// A stored profile's key is never sent back to the browser, so the form is empty when testing a stored profile and the key comes from the DB.
|
||||
// Same for the session header name: when the form omits it, the stored profile's value is the fallback.
|
||||
if stored != nil && (cfg.APIKey == "" || cfg.SessionHeaderKey == "") {
|
||||
// Trim like ConfigFrom does for form values: a key stored with
|
||||
// surrounding whitespace runs fine in conversations (the runtime
|
||||
// path re-trims) but would be sent raw here and fail auth.
|
||||
if cfg.APIKey == "" {
|
||||
// The custom session header name follows that profile regardless of format:
|
||||
// when the form omits it, the stored profile's value is the fallback.
|
||||
if stored != nil && cfg.SessionHeaderKey == "" {
|
||||
cfg.SessionHeaderKey = strings.TrimSpace(stored.SessionHeaderKey)
|
||||
}
|
||||
// Keyless formats (Ollama) skip the key resolution entirely.
|
||||
if cfg.NeedsAPIKey() {
|
||||
// API key resolution precedence: form input > the named profile's stored key > the global config's key.
|
||||
// A stored profile's key is never sent back to the browser, so the form is empty when testing a stored profile and the key comes from the DB.
|
||||
if stored != nil && cfg.APIKey == "" {
|
||||
// Trim like ConfigFrom does for form values: a key stored with
|
||||
// surrounding whitespace runs fine in conversations (the runtime
|
||||
// path re-trims) but would be sent raw here and fail auth.
|
||||
cfg.APIKey = strings.TrimSpace(stored.APIKey)
|
||||
}
|
||||
if cfg.SessionHeaderKey == "" {
|
||||
cfg.SessionHeaderKey = strings.TrimSpace(stored.SessionHeaderKey)
|
||||
if cfg.APIKey == "" {
|
||||
s.cfgMu.Lock()
|
||||
cfg.APIKey = s.llmCfg.APIKey
|
||||
s.cfgMu.Unlock()
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
writeJSON(w, 200, map[string]any{"ok": false, "error": "no API key provided"})
|
||||
return
|
||||
}
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
s.cfgMu.Lock()
|
||||
cfg.APIKey = s.llmCfg.APIKey
|
||||
s.cfgMu.Unlock()
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
writeJSON(w, 200, map[string]any{"ok": false, "error": "no API key provided"})
|
||||
return
|
||||
}
|
||||
// Retry params do [not] go into the connection test: the test has a 30s hard timeout, and stacking the configured retry counts/long intervals on top
|
||||
// would only turn a perfectly usable endpoint into a "timeout failure". The test asks "does this endpoint work at all"; retry pacing
|
||||
@@ -1628,7 +1635,7 @@ func (s *Server) validateTaskProfileIDs(ids []int64) error {
|
||||
}
|
||||
seen[id] = true
|
||||
if _, ok := s.loadProfileConfig(id); !ok {
|
||||
return fmt.Errorf("LLM profile #%d not found or has no API key set", id)
|
||||
return fmt.Errorf("LLM profile #%d not found or not usable (no API key for the key-requiring formats)", id)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
|
||||
+22
-4
@@ -2037,12 +2037,13 @@ func (s *Server) pgLLMPoolReset(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
// pgListModels fetches available models from the provider's API endpoint.
|
||||
// Supports OpenAI-format (GET /models) and Anthropic-format (GET /v1/models); for
|
||||
// Supports OpenAI-format (GET /models), Anthropic-format (GET /v1/models) and
|
||||
// Ollama (keyless OpenAI-compatible GET /v1/models on the local server); for
|
||||
// Anthropic-compatible third parties (e.g. DeepSeek) whose model list lives only on
|
||||
// the OpenAI path, it falls back to the OpenAI endpoint at the stripped root.
|
||||
func (s *Server) pgListModels(w http.ResponseWriter, r *http.Request) {
|
||||
var req struct {
|
||||
Provider string `json:"provider"` // "openai" | "anthropic"
|
||||
Provider string `json:"provider"` // "openai" | "anthropic" | "ollama"
|
||||
BaseURL string `json:"base_url"`
|
||||
APIKey string `json:"api_key"`
|
||||
Proxy string `json:"proxy"`
|
||||
@@ -2059,13 +2060,14 @@ func (s *Server) pgListModels(w http.ResponseWriter, r *http.Request) {
|
||||
apiKey = p.APIKey
|
||||
}
|
||||
}
|
||||
if apiKey == "" {
|
||||
provider := strings.TrimSpace(req.Provider)
|
||||
// Ollama serves its model list keyless on the local server: no key gate.
|
||||
if apiKey == "" && provider != "ollama" {
|
||||
writeJSON(w, 200, map[string]any{"ok": false, "error": "no API Key provided"})
|
||||
return
|
||||
}
|
||||
|
||||
baseURL := strings.TrimRight(strings.TrimSpace(req.BaseURL), "/")
|
||||
provider := strings.TrimSpace(req.Provider)
|
||||
|
||||
// Candidate endpoints to try in order. Some Anthropic-compatible providers
|
||||
// (e.g. DeepSeek) implement /v1/messages under an /anthropic path but expose
|
||||
@@ -2097,6 +2099,22 @@ func (s *Server) pgListModels(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
b := strings.TrimRight(strings.TrimSuffix(strings.TrimSuffix(baseURL, "/chat/completions"), "/responses"), "/")
|
||||
candidates = append(candidates, candidate{b + "/models", bearerHdr()})
|
||||
case "ollama":
|
||||
// OpenAI-compatible list at {base}/v1/models. Tolerate base URLs with or
|
||||
// without the /v1 suffix (and a full endpoint URL). No auth needed; when a
|
||||
// key was provided anyway, send it as Bearer (Ollama ignores it).
|
||||
if baseURL == "" {
|
||||
baseURL = agent.DefaultOllamaBaseURL
|
||||
}
|
||||
b := strings.TrimSuffix(baseURL, "/chat/completions")
|
||||
if !strings.HasSuffix(b, "/v1") {
|
||||
b += "/v1"
|
||||
}
|
||||
hdr := http.Header{}
|
||||
if apiKey != "" {
|
||||
hdr.Set("Authorization", "Bearer "+apiKey)
|
||||
}
|
||||
candidates = append(candidates, candidate{b + "/models", hdr})
|
||||
default: // anthropic
|
||||
if baseURL == "" {
|
||||
baseURL = "https://api.anthropic.com"
|
||||
|
||||
@@ -29,7 +29,8 @@ func (s *Server) resolutionFromProfile(p *db.LLMProfile, source string) taskLLMR
|
||||
Model: p.Model,
|
||||
Source: source,
|
||||
}
|
||||
if p.APIKey == "" {
|
||||
// Ollama profiles are keyless by design; the other formats need a key.
|
||||
if p.APIKey == "" && p.Format != "ollama" {
|
||||
result.Reason = "LLM profile has no API Key set"
|
||||
return result
|
||||
}
|
||||
|
||||
@@ -1,403 +0,0 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/http"
|
||||
"runtime"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/artex/selfupdate"
|
||||
)
|
||||
|
||||
// HTTP surface for one-click updates from the UI. The actual
|
||||
// download/verify/swap logic lives in the selfupdate package; this file only
|
||||
// handles the auth boundary, concurrency exclusion, progress broadcast, and
|
||||
// telling main "time to exit".
|
||||
//
|
||||
// The restart is not performed by this process: after staging the new version
|
||||
// the process exits with selfupdate.ExitRestart and the supervisor script
|
||||
// (start.sh / start.bat, or ENTRYPOINT under Docker) brings it back up.
|
||||
|
||||
// restartCh is closed once an upgrade is staged or a rollback completes; main
|
||||
// then exits with ExitRestart.
|
||||
var (
|
||||
restartOnce sync.Once
|
||||
restartCh = make(chan struct{})
|
||||
)
|
||||
|
||||
// RestartRequested returns a channel that is closed when it is time to exit and
|
||||
// let the supervisor relaunch us.
|
||||
func RestartRequested() <-chan struct{} { return restartCh }
|
||||
|
||||
func requestRestart() { restartOnce.Do(func() { close(restartCh) }) }
|
||||
|
||||
// bootState is selfupdate.Bootstrap's verdict for this boot (upgrade succeeded /
|
||||
// just rolled back / staged file discarded), injected by main so that
|
||||
// /api/update/check can faithfully tell the frontend how the last upgrade ended.
|
||||
var (
|
||||
bootStateMu sync.Mutex
|
||||
bootState selfupdate.State
|
||||
)
|
||||
|
||||
// SetBootUpdateState is called once by main at startup.
|
||||
func SetBootUpdateState(st selfupdate.State) {
|
||||
bootStateMu.Lock()
|
||||
defer bootStateMu.Unlock()
|
||||
bootState = st
|
||||
}
|
||||
|
||||
func bootUpdateState() selfupdate.State {
|
||||
bootStateMu.Lock()
|
||||
defer bootStateMu.Unlock()
|
||||
return bootState
|
||||
}
|
||||
|
||||
// releaseCache caches the latest-release lookup against GitHub.
|
||||
//
|
||||
// The top-bar "new version available" indicator queries once per full page load,
|
||||
// and the unauthenticated GitHub API allows 60 requests per IP per hour —
|
||||
// without a cache, opening a few tabs or refreshing a few times burns through
|
||||
// the quota, and a real update check would then fail to get through. When the
|
||||
// user explicitly clicks "check for updates", force bypasses the cache.
|
||||
type releaseCache struct {
|
||||
mu sync.Mutex
|
||||
rel *selfupdate.Release
|
||||
err error
|
||||
at time.Time
|
||||
// fetch is the retrieval function, an injection point kept only for tests;
|
||||
// nil means the real GitHub lookup.
|
||||
fetch func(context.Context, *http.Client) (*selfupdate.Release, error)
|
||||
}
|
||||
|
||||
const (
|
||||
releaseTTL = 30 * time.Minute
|
||||
// Failed results are also cached briefly, otherwise every page load would sit
|
||||
// through a timeout while GitHub is unreachable; but the TTL is short so it
|
||||
// heals itself soon after the network recovers.
|
||||
releaseErrTTL = 2 * time.Minute
|
||||
// Timeout for lookups. NewClient's 30-minute timeout is for downloading the
|
||||
// full package; a version check cannot wait that long.
|
||||
releaseTimeout = 20 * time.Second
|
||||
)
|
||||
|
||||
var relCache = &releaseCache{}
|
||||
|
||||
// get returns the latest Release, serving from cache when possible.
|
||||
//
|
||||
// The lock is held for the whole fetch: concurrent requests queue up and share
|
||||
// one lookup instead of each hitting GitHub (right after page load, when many
|
||||
// tabs query at once, is exactly when rate limiting bites).
|
||||
func (c *releaseCache) get(ctx context.Context, client *http.Client, force bool) (*selfupdate.Release, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
if !force {
|
||||
ttl := releaseTTL
|
||||
if c.err != nil {
|
||||
ttl = releaseErrTTL
|
||||
}
|
||||
if !c.at.IsZero() && time.Since(c.at) < ttl {
|
||||
return c.rel, c.err
|
||||
}
|
||||
}
|
||||
|
||||
fetch := c.fetch
|
||||
if fetch == nil {
|
||||
fetch = selfupdate.FetchLatest
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, releaseTimeout)
|
||||
defer cancel()
|
||||
rel, err := fetch(ctx, client)
|
||||
// A cancelled request (user closed the tab) does not mean GitHub is broken;
|
||||
// do not write it into the cache, or the next visitor gets a baffling
|
||||
// "cancelled" error.
|
||||
if err != nil && ctx.Err() != nil && errors.Is(ctx.Err(), context.Canceled) {
|
||||
return c.rel, err
|
||||
}
|
||||
c.rel, c.err, c.at = rel, err, time.Now()
|
||||
return rel, err
|
||||
}
|
||||
|
||||
// updateProgress is one progress update pushed to the frontend.
|
||||
type updateProgress struct {
|
||||
Phase selfupdate.Phase `json:"phase"`
|
||||
Percent int `json:"percent"` // only meaningful during the download phase; -1 otherwise
|
||||
Message string `json:"message"`
|
||||
Version string `json:"version,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// updateHub holds the progress of one upgrade and broadcasts it to SSE
|
||||
// subscribers.
|
||||
//
|
||||
// running doubles as a mutex: a second POST /api/update/apply during an upgrade
|
||||
// gets a 409, keeping two goroutines from writing the same artex.new at once.
|
||||
type updateHub struct {
|
||||
mu sync.Mutex
|
||||
running bool
|
||||
cur updateProgress
|
||||
subs map[chan updateProgress]struct{}
|
||||
}
|
||||
|
||||
var updHub = &updateHub{
|
||||
cur: updateProgress{Phase: selfupdate.PhaseIdle, Percent: -1},
|
||||
subs: map[chan updateProgress]struct{}{},
|
||||
}
|
||||
|
||||
// begin claims the upgrade slot, returning false if one is already running.
|
||||
func (h *updateHub) begin(version string) bool {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
if h.running {
|
||||
return false
|
||||
}
|
||||
h.running = true
|
||||
h.cur = updateProgress{Phase: selfupdate.PhaseDownload, Percent: 0, Message: "Preparing...", Version: version}
|
||||
h.fanout(h.cur)
|
||||
return true
|
||||
}
|
||||
|
||||
// finish ends an upgrade. A nil err means staging succeeded and a restart is
|
||||
// pending.
|
||||
func (h *updateHub) finish(err error) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.running = false
|
||||
if err != nil {
|
||||
h.cur = updateProgress{Phase: selfupdate.PhaseFailed, Percent: -1, Message: "Update failed", Error: err.Error(), Version: h.cur.Version}
|
||||
} else {
|
||||
h.cur = updateProgress{Phase: selfupdate.PhaseStaged, Percent: 100, Message: "New version ready, restarting...", Version: h.cur.Version}
|
||||
}
|
||||
h.fanout(h.cur)
|
||||
}
|
||||
|
||||
func (h *updateHub) publish(ph selfupdate.Phase, pct int, msg string) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.cur = updateProgress{Phase: ph, Percent: pct, Message: msg, Version: h.cur.Version}
|
||||
h.fanout(h.cur)
|
||||
}
|
||||
|
||||
// fanout must be called with h.mu held. Subscriber channels are buffered and
|
||||
// drops are fine — progress is disposable, transient information, and a stuck
|
||||
// SSE connection must never block the upgrade itself.
|
||||
func (h *updateHub) fanout(p updateProgress) {
|
||||
for ch := range h.subs {
|
||||
select {
|
||||
case ch <- p:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (h *updateHub) snapshot() (updateProgress, bool) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
return h.cur, h.running
|
||||
}
|
||||
|
||||
func (h *updateHub) subscribe() (<-chan updateProgress, func()) {
|
||||
ch := make(chan updateProgress, 64)
|
||||
h.mu.Lock()
|
||||
h.subs[ch] = struct{}{}
|
||||
h.mu.Unlock()
|
||||
var once sync.Once
|
||||
return ch, func() {
|
||||
once.Do(func() {
|
||||
h.mu.Lock()
|
||||
delete(h.subs, ch)
|
||||
h.mu.Unlock()
|
||||
close(ch)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// updateCheck queries the latest stable release on GitHub and compares it with
|
||||
// the current version.
|
||||
//
|
||||
// The frontend also talks to api.github.com directly (GitHub's CORS is *), but
|
||||
// **this endpoint is authoritative**: the download is done by the backend, so an
|
||||
// update is only possible if the backend can reach GitHub. It is common for the
|
||||
// browser to have connectivity while the server does not (server on an internal
|
||||
// network, or the proxy configured only in the browser); in that case clicking
|
||||
// update would inevitably fail, so it is better to report the error honestly at
|
||||
// the check step.
|
||||
func (s *Server) updateCheck(w http.ResponseWriter, r *http.Request) {
|
||||
current := BuildVersion
|
||||
mode := "binary"
|
||||
if selfupdate.InDocker() {
|
||||
mode = "docker"
|
||||
}
|
||||
boot := bootUpdateState()
|
||||
out := map[string]any{
|
||||
"current": current,
|
||||
"mode": mode,
|
||||
"os": runtime.GOOS,
|
||||
"arch": runtime.GOARCH,
|
||||
"has_backup": selfupdate.HasBackup(),
|
||||
"repo": selfupdate.Repo,
|
||||
"boot_notice": boot.Detail,
|
||||
"rolled_back": boot.RolledBack,
|
||||
}
|
||||
|
||||
// The top-bar indicator uses the cache (default); clicking "check for
|
||||
// updates" passes force=1 to refetch from the origin.
|
||||
force := r.URL.Query().Get("force") != ""
|
||||
client := selfupdate.NewClient(s.m.GlobalProxy())
|
||||
rel, err := relCache.get(r.Context(), client, force)
|
||||
if err != nil {
|
||||
out["error"] = err.Error()
|
||||
writeJSON(w, 200, out)
|
||||
return
|
||||
}
|
||||
|
||||
latest := rel.TagName
|
||||
out["latest"] = latest
|
||||
out["notes"] = rel.Body
|
||||
out["html_url"] = rel.HTMLURL
|
||||
if !rel.PublishedAt.IsZero() {
|
||||
out["published_at"] = rel.PublishedAt.Format(time.RFC3339)
|
||||
}
|
||||
|
||||
asset := selfupdate.AssetName(latest, runtime.GOOS, runtime.GOARCH)
|
||||
out["asset"] = asset
|
||||
if a, ok := rel.FindAsset(asset); ok {
|
||||
out["asset_available"] = true
|
||||
out["size"] = a.Size
|
||||
} else {
|
||||
out["asset_available"] = false
|
||||
}
|
||||
|
||||
cmp, comparable := selfupdate.CompareVersions(current, latest)
|
||||
out["comparable"] = comparable
|
||||
out["has_update"] = comparable && cmp < 0
|
||||
if !comparable {
|
||||
// Development builds (dev / git describe with a suffix) have no
|
||||
// comparable version number. Allowing the update would only overwrite
|
||||
// the binary being debugged locally with a release build, so updates
|
||||
// are refused outright.
|
||||
out["reason"] = fmt.Sprintf("current version %q is not a stable release; one-click update is disabled", current)
|
||||
}
|
||||
writeJSON(w, 200, out)
|
||||
}
|
||||
|
||||
// updateApply downloads and stages the new version, then lets the process exit
|
||||
// so the supervisor script restarts it.
|
||||
//
|
||||
// It returns 202 immediately and does the work on a background goroutine: a
|
||||
// full package download can take minutes and would be cut off by reverse-proxy
|
||||
// timeouts if tied to the request. Progress goes over /api/update/stream.
|
||||
func (s *Server) updateApply(w http.ResponseWriter, r *http.Request) {
|
||||
current := BuildVersion
|
||||
|
||||
// Use the cache: ensure the installed version is exactly the one the user
|
||||
// saw and confirmed in the UI.
|
||||
client := selfupdate.NewClient(s.m.GlobalProxy())
|
||||
rel, err := relCache.get(r.Context(), client, false)
|
||||
if err != nil {
|
||||
writeErr(w, 502, err.Error())
|
||||
return
|
||||
}
|
||||
cmp, comparable := selfupdate.CompareVersions(current, rel.TagName)
|
||||
if !comparable {
|
||||
writeErr(w, 400, fmt.Sprintf("current version %q is not a stable release; one-click update is disabled", current))
|
||||
return
|
||||
}
|
||||
if cmp >= 0 {
|
||||
writeErr(w, 400, fmt.Sprintf("already on the latest version %s", current))
|
||||
return
|
||||
}
|
||||
if !updHub.begin(rel.TagName) {
|
||||
writeErr(w, 409, "an update is already in progress")
|
||||
return
|
||||
}
|
||||
|
||||
go func() {
|
||||
// Deliberately use s.ctx rather than the request ctx: the request ends
|
||||
// as soon as the HTTP response returns, and a download tied to it would
|
||||
// be cancelled immediately.
|
||||
err := selfupdate.Stage(s.ctx, client, rel, current, func(ph selfupdate.Phase, pct int, msg string) {
|
||||
updHub.publish(ph, pct, msg)
|
||||
})
|
||||
updHub.finish(err)
|
||||
if err != nil {
|
||||
log.Printf("[update] update failed: %v", err)
|
||||
return
|
||||
}
|
||||
log.Printf("[update] %s → %s staged, exiting to complete the swap", current, rel.TagName)
|
||||
// Leave a moment to push the final progress to the frontend before
|
||||
// triggering the exit.
|
||||
time.Sleep(1500 * time.Millisecond)
|
||||
requestRestart()
|
||||
}()
|
||||
|
||||
writeJSON(w, 202, map[string]any{"ok": true, "target": rel.TagName})
|
||||
}
|
||||
|
||||
// updateRollback actively reverts to the previous version (the artex.old backed
|
||||
// up before the swap).
|
||||
func (s *Server) updateRollback(w http.ResponseWriter, r *http.Request) {
|
||||
if _, running := updHub.snapshot(); running {
|
||||
writeErr(w, 409, "an update is in progress; cannot roll back")
|
||||
return
|
||||
}
|
||||
if err := selfupdate.Rollback(); err != nil {
|
||||
writeErr(w, 400, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("[update] manually rolled back to the previous version, exiting to complete the switch")
|
||||
writeJSON(w, 202, map[string]any{"ok": true})
|
||||
go func() {
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
requestRestart()
|
||||
}()
|
||||
}
|
||||
|
||||
// updateStream pushes update progress over SSE.
|
||||
func (s *Server) updateStream(w http.ResponseWriter, r *http.Request) {
|
||||
flusher, ok := w.(http.Flusher)
|
||||
if !ok {
|
||||
writeErr(w, 500, "streaming unsupported")
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.Header().Set("X-Accel-Buffering", "no")
|
||||
|
||||
ch, unsub := updHub.subscribe()
|
||||
defer unsub()
|
||||
|
||||
send := func(p updateProgress) {
|
||||
b, _ := json.Marshal(p)
|
||||
fmt.Fprintf(w, "data: %s\n\n", b)
|
||||
flusher.Flush()
|
||||
}
|
||||
// Send the current state first, so a page refresh immediately shows an
|
||||
// in-flight upgrade.
|
||||
cur, _ := updHub.snapshot()
|
||||
send(cur)
|
||||
|
||||
ctx := r.Context()
|
||||
ping := time.NewTicker(20 * time.Second)
|
||||
defer ping.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case p, ok := <-ch:
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
send(p)
|
||||
case <-ping.C:
|
||||
fmt.Fprint(w, ": ping\n\n")
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,161 +0,0 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/artex/selfupdate"
|
||||
)
|
||||
|
||||
// releaseCache is the layer that protects the GitHub quota: the unauthenticated
|
||||
// API allows only 60 requests/hour/IP, and the top-bar "new version available"
|
||||
// indicator queries once per full page load. Once the cache stops working, a
|
||||
// user opening a few tabs would burn through the quota, and real update checks
|
||||
// would then fail to get through.
|
||||
|
||||
func newTestCache(fetch func(context.Context, *http.Client) (*selfupdate.Release, error)) *releaseCache {
|
||||
return &releaseCache{fetch: fetch}
|
||||
}
|
||||
|
||||
func TestReleaseCacheServesFromCache(t *testing.T) {
|
||||
calls := 0
|
||||
c := newTestCache(func(context.Context, *http.Client) (*selfupdate.Release, error) {
|
||||
calls++
|
||||
return &selfupdate.Release{TagName: "v0.3.8"}, nil
|
||||
})
|
||||
|
||||
for range 5 {
|
||||
rel, err := c.get(t.Context(), nil, false)
|
||||
if err != nil {
|
||||
t.Fatalf("get: %v", err)
|
||||
}
|
||||
if rel.TagName != "v0.3.8" {
|
||||
t.Fatalf("TagName = %q", rel.TagName)
|
||||
}
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Errorf("5 queries should hit the origin only once, got %d", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseCacheForceBypasses(t *testing.T) {
|
||||
calls := 0
|
||||
c := newTestCache(func(context.Context, *http.Client) (*selfupdate.Release, error) {
|
||||
calls++
|
||||
return &selfupdate.Release{TagName: "v0.3.8"}, nil
|
||||
})
|
||||
|
||||
if _, err := c.get(t.Context(), nil, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Clicking "check for updates" must return a live result, otherwise a freshly
|
||||
// published release stays invisible until the cache expires.
|
||||
if _, err := c.get(t.Context(), nil, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls != 2 {
|
||||
t.Errorf("force should bypass the cache, want 2 origin fetches, got %d", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseCacheExpiresAfterTTL(t *testing.T) {
|
||||
calls := 0
|
||||
c := newTestCache(func(context.Context, *http.Client) (*selfupdate.Release, error) {
|
||||
calls++
|
||||
return &selfupdate.Release{TagName: "v0.3.8"}, nil
|
||||
})
|
||||
|
||||
if _, err := c.get(t.Context(), nil, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Move the stored timestamp back to just past expiry, simulating TTL timeout.
|
||||
c.at = time.Now().Add(-releaseTTL - time.Second)
|
||||
if _, err := c.get(t.Context(), nil, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls != 2 {
|
||||
t.Errorf("after TTL expiry it should fetch from the origin again, want 2, got %d", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseCacheUsesShorterTTLForErrors(t *testing.T) {
|
||||
calls := 0
|
||||
c := newTestCache(func(context.Context, *http.Client) (*selfupdate.Release, error) {
|
||||
calls++
|
||||
return nil, errors.New("github unreachable")
|
||||
})
|
||||
|
||||
if _, err := c.get(t.Context(), nil, false); err == nil {
|
||||
t.Fatal("expected an error")
|
||||
}
|
||||
// Failed results must also be cached briefly, otherwise every page load waits
|
||||
// through a timeout for nothing while GitHub is unreachable.
|
||||
if _, err := c.get(t.Context(), nil, false); err == nil {
|
||||
t.Fatal("expected an error")
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Errorf("errors should be cached briefly, want 1 origin fetch, got %d", calls)
|
||||
}
|
||||
|
||||
// But the error TTL must be noticeably shorter than the success one, so the
|
||||
// cache can heal quickly once the network recovers.
|
||||
if releaseErrTTL >= releaseTTL {
|
||||
t.Fatalf("error TTL (%v) must be shorter than success TTL (%v)", releaseErrTTL, releaseTTL)
|
||||
}
|
||||
c.at = time.Now().Add(-releaseErrTTL - time.Second)
|
||||
if _, err := c.get(t.Context(), nil, false); err == nil {
|
||||
t.Fatal("expected an error")
|
||||
}
|
||||
if calls != 2 {
|
||||
t.Errorf("after error TTL expiry it should retry, want 2, got %d", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseCacheDoesNotPoisonOnCallerCancel(t *testing.T) {
|
||||
good := &selfupdate.Release{TagName: "v0.3.8"}
|
||||
c := newTestCache(func(ctx context.Context, _ *http.Client) (*selfupdate.Release, error) {
|
||||
return good, nil
|
||||
})
|
||||
if _, err := c.get(t.Context(), nil, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// A visitor closing the tab cancels the request. That does not mean GitHub is
|
||||
// broken, and the "cancelled" error must never be written into the cache —
|
||||
// otherwise every visitor would get a baffling error for the next 30 minutes.
|
||||
c.fetch = func(ctx context.Context, _ *http.Client) (*selfupdate.Release, error) {
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
c.at = time.Now().Add(-releaseTTL - time.Second) // expire the cache to force a refetch
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if _, err := c.get(ctx, nil, false); err == nil {
|
||||
t.Fatal("when the caller has cancelled, the error should be passed through to it")
|
||||
}
|
||||
|
||||
// Key invariant: the cancelled call leaves no trace — the cache holds neither
|
||||
// the "cancelled" error nor loses the previous good result.
|
||||
if c.err != nil {
|
||||
t.Fatalf("cancel error must not be written into the cache, got %v", c.err)
|
||||
}
|
||||
if c.rel == nil || c.rel.TagName != "v0.3.8" {
|
||||
t.Fatalf("cache should keep the previous good result, got %+v", c.rel)
|
||||
}
|
||||
|
||||
// The cancellation fetched no new data, so the next visitor should refetch
|
||||
// from the origin — and get a normal result, unaffected by the earlier cancel.
|
||||
c.fetch = func(context.Context, *http.Client) (*selfupdate.Release, error) {
|
||||
return good, nil
|
||||
}
|
||||
rel, err := c.get(t.Context(), nil, false)
|
||||
if err != nil {
|
||||
t.Fatalf("normal requests after a cancellation must not fail: %v", err)
|
||||
}
|
||||
if rel == nil || rel.TagName != "v0.3.8" {
|
||||
t.Fatalf("should get a normal result, got %+v", rel)
|
||||
}
|
||||
}
|
||||
@@ -10,11 +10,7 @@ rem
|
||||
rem It does exactly one thing: run artex.exe and, once the process exits, decide by exit code whether to relaunch it.
|
||||
rem
|
||||
rem 0 user stopped normally -> exit the loop
|
||||
rem 75 program requested a restart -> rerun immediately ("one-click update" or "rollback" was clicked in the web UI)
|
||||
rem other crash -> rerun after backoff (1->2->4... up to 60 seconds)
|
||||
rem
|
||||
rem Download, SHA256 verification, and the swap are NOT done here; artex does all of them itself at startup
|
||||
rem (the selfupdate package). The script stays dumb; see the notes at the top of start.sh.
|
||||
|
||||
setlocal enabledelayedexpansion
|
||||
cd /d "%~dp0"
|
||||
@@ -25,7 +21,6 @@ if not exist "%BIN%" (
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
set "RESTART_CODE=75"
|
||||
set "MAX_DELAY=60"
|
||||
set /a delay=1
|
||||
|
||||
@@ -38,13 +33,6 @@ if "!code!"=="0" (
|
||||
exit /b 0
|
||||
)
|
||||
|
||||
if "!code!"=="%RESTART_CODE%" (
|
||||
rem Update/rollback is ready: after the rerun, artex completes the swap at startup.
|
||||
echo [artex] restart requested (applying new version)...
|
||||
set /a delay=1
|
||||
goto loop
|
||||
)
|
||||
|
||||
echo [artex] abnormal exit ^(code=!code!^), restarting in !delay!s 1>&2
|
||||
rem timeout fails in a redirected console, so fall back to ping (delaying N seconds needs N+1 pings).
|
||||
set /a pings=!delay!+1
|
||||
|
||||
@@ -9,13 +9,7 @@
|
||||
# It does exactly one thing: run artex and, once the process exits, decide by exit code whether to relaunch it.
|
||||
#
|
||||
# 0 user stopped normally -> exit the loop
|
||||
# 75 program requested a restart -> rerun immediately ("one-click update" or "rollback" was clicked in the web UI)
|
||||
# other crash -> rerun after backoff (1→2→4... up to 60 seconds)
|
||||
#
|
||||
# Download, SHA256 verification, and the swap are deliberately not done here: that logic would need two implementations in sh and bat,
|
||||
# and it is precisely the step that must never fail — once an unrunnable binary is swapped in, this script would faithfully relaunch it
|
||||
# forever, and the user could only fix it on the machine by hand. So all verification/swap logic stays in Go (the selfupdate package),
|
||||
# done by artex itself at startup, keeping the script dumb.
|
||||
set -u
|
||||
|
||||
cd "$(dirname "$0")" || exit 1
|
||||
@@ -23,7 +17,6 @@ cd "$(dirname "$0")" || exit 1
|
||||
BIN=./artex
|
||||
[ -x "$BIN" ] || { echo "[artex] executable $BIN not found" >&2; exit 1; }
|
||||
|
||||
RESTART_CODE=75
|
||||
MAX_DELAY=60
|
||||
|
||||
child=0
|
||||
@@ -67,11 +60,6 @@ while :; do
|
||||
echo "[artex] exited normally"
|
||||
exit 0
|
||||
;;
|
||||
"$RESTART_CODE")
|
||||
# Update/rollback is ready: after the rerun, artex completes the swap at startup (see selfupdate.Bootstrap).
|
||||
echo "[artex] restart requested (applying new version)…"
|
||||
delay=1
|
||||
;;
|
||||
*)
|
||||
echo "[artex] abnormal exit (code=$code), restarting in ${delay}s" >&2
|
||||
sleep "$delay"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# ARTEX update script: (1) Docker update (pull new image and rebuild) (2) local build update (rebuild the binary)
|
||||
# ARTEX update script: (1) Docker update (rebuild the image from this checkout) (2) local build update (rebuild the binary)
|
||||
# Counterpart of install.sh: install handles the first-time setup, update upgrades to a new version.
|
||||
# No manual DB migration needed — artex idempotently re-runs schema.sql on every start (with ADD COLUMN/CREATE
|
||||
# INDEX IF NOT EXISTS), so "restart equals migrate". Data (pgdata volume, ./data, ./skills) is not affected.
|
||||
@@ -17,7 +17,16 @@ sync_repo(){
|
||||
[ -d .git ] && command -v git >/dev/null 2>&1 || { warn "not a git working copy, skipping git pull"; return; }
|
||||
[ "$(ask 'Pull the latest code (git pull --ff-only)? (y/n)' y)" = y ] || return
|
||||
if ! git pull --ff-only; then
|
||||
warn "git pull could not fast-forward (local changes or diverged branches) — resolve manually and retry; continuing with the current code"
|
||||
# Continuing would rebuild the OLD tree and still print "Build complete":
|
||||
# exactly the silent stale upgrade this script exists to prevent.
|
||||
cat >&2 <<'RECOVER'
|
||||
[!] git pull could not fast-forward. This happens when the checkout has local
|
||||
changes, or when it was cloned before the history rewrite of 2026-10-23.
|
||||
Refusing to rebuild stale code. Recover, then rerun this script:
|
||||
keep your changes: git stash && git fetch origin && git reset --hard origin/main && git stash pop
|
||||
discard them: git fetch origin && git reset --hard origin/main
|
||||
RECOVER
|
||||
die "aborting: nothing was rebuilt"
|
||||
fi
|
||||
}
|
||||
|
||||
@@ -27,23 +36,12 @@ update_docker(){
|
||||
|| die "docker / docker compose not detected; install and deploy with ./install.sh first"
|
||||
[ -f .env ] || die ".env not found; run ./install.sh first to complete the initial deployment"
|
||||
|
||||
# Optional: upgrade to a specific version tag (empty = keep ARTEX_TAG from .env, default latest)
|
||||
local tag; tag="$(ask 'Target image tag (press Enter to keep .env / latest)' '')"
|
||||
if [ -n "$tag" ]; then
|
||||
if grep -q '^ARTEX_TAG=' .env; then
|
||||
sed -i.bak "s|^ARTEX_TAG=.*|ARTEX_TAG=${tag}|" .env && rm -f .env.bak
|
||||
else
|
||||
printf '\nARTEX_TAG=%s\n' "$tag" >> .env
|
||||
fi
|
||||
ok "ARTEX_TAG set to ${tag}"
|
||||
fi
|
||||
|
||||
# Touch only artex: postgres is pinned to 16-alpine and does not need upgrading along
|
||||
# (pulling it wastes bandwidth, and a major version bump risks compatibility). artex
|
||||
# declares depends_on postgres, so an up with the service name starts pg if needed and leaves a running one untouched.
|
||||
info "Pulling new image (artex only)…"
|
||||
docker compose pull artex
|
||||
info "Rebuilding and starting (schema migrates automatically on artex restart)…"
|
||||
info "Building the image from this checkout (artex only)…"
|
||||
docker compose build artex
|
||||
info "Recreating artex (schema migrates automatically on restart)…"
|
||||
docker compose up -d artex
|
||||
ok "Update complete → http://localhost:8787"
|
||||
info "View logs: docker compose logs -f artex"
|
||||
|
||||
@@ -14,7 +14,6 @@ import { AccountSwitcher } from "./sidebar/account-switcher";
|
||||
import { LayoutControls } from "./sidebar/layout-controls";
|
||||
import { SearchDialog } from "./sidebar/search-dialog";
|
||||
import { ThemeSwitcher } from "./sidebar/theme-switcher";
|
||||
import { UpdateBadge } from "./update-badge";
|
||||
|
||||
// Task detail pages stay as-is: they bring their own header/Tabs and padding, so no extra global header or padding is stacked here.
|
||||
function isFullBleed(pathname: string) {
|
||||
@@ -68,7 +67,6 @@ export function MainContent({ children }: { children: ReactNode }) {
|
||||
{version && (
|
||||
<span className="font-medium text-muted-foreground text-xs tabular-nums">Version · {version}</span>
|
||||
)}
|
||||
<UpdateBadge />
|
||||
<LayoutControls />
|
||||
<ThemeSwitcher />
|
||||
<AccountSwitcher users={[currentUser]} />
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
|
||||
import Link from "next/link";
|
||||
|
||||
import { ArrowUpCircleIcon } from "lucide-react";
|
||||
|
||||
import { api } from "@/lib/api";
|
||||
|
||||
/**
|
||||
* Top-bar "new version" badge: checked once per full page load; lights up next to the version
|
||||
* number when an update exists, and clicking jumps to the "Version and updates" card on the system settings page.
|
||||
*
|
||||
* The backend caches GitHub query results for 30 minutes, so checking once per mount here is safe
|
||||
* — the unauthenticated GitHub API allows only 60 requests/hour/IP; without that cache layer, a few extra tabs
|
||||
* would drain the quota, and a real update check would then stall.
|
||||
*
|
||||
* Failed checks stay silent: the top bar is not the place for errors — the reason shows on the settings page's "Check for updates".
|
||||
*/
|
||||
export function UpdateBadge() {
|
||||
const [latest, setLatest] = React.useState("");
|
||||
|
||||
React.useEffect(() => {
|
||||
let alive = true;
|
||||
api
|
||||
.checkUpdate()
|
||||
.then((r) => {
|
||||
// has_update already includes the "version is comparable" check, so development builds never light this up.
|
||||
if (alive && r.has_update && r.latest) setLatest(r.latest.replace(/^v(?=\d)/, ""));
|
||||
})
|
||||
.catch(() => {
|
||||
// Silent: no network / GitHub rate limiting must not raise an error in the top bar.
|
||||
});
|
||||
return () => {
|
||||
alive = false;
|
||||
};
|
||||
}, []);
|
||||
|
||||
if (!latest) return null;
|
||||
|
||||
return (
|
||||
<Link
|
||||
href="/system/settings"
|
||||
title={`New version ${latest} available — click to update`}
|
||||
className="inline-flex items-center gap-1.5 rounded-full bg-primary px-2.5 py-1 font-medium text-primary-foreground text-xs transition-opacity hover:opacity-90"
|
||||
>
|
||||
{/* Breathing dot: the top bar is crowded and plain text is easy to miss; the animation makes it instantly visible. */}
|
||||
<span className="relative flex size-1.5">
|
||||
<span className="absolute inline-flex size-full animate-ping rounded-full bg-primary-foreground opacity-75" />
|
||||
<span className="relative inline-flex size-1.5 rounded-full bg-primary-foreground" />
|
||||
</span>
|
||||
<ArrowUpCircleIcon className="size-3.5" />
|
||||
<span className="hidden sm:inline">New version {latest}</span>
|
||||
<span className="sm:hidden">New version</span>
|
||||
</Link>
|
||||
);
|
||||
}
|
||||
@@ -49,13 +49,18 @@ const MAX_TOKENS_FIELDS: { value: string; label: string }[] = [
|
||||
{ value: NONE, label: "max_tokens (default)" },
|
||||
{ value: "max_completion_tokens", label: "max_completion_tokens" },
|
||||
];
|
||||
// The other two formats hard-code their field names, so the option is meaningless for them; the hint texts spell this out.
|
||||
// The other formats hard-code their field names, so the option is meaningless for them; the hint texts spell this out.
|
||||
const MAX_TOKENS_FIELD_HINTS: Record<string, string> = {
|
||||
openai:
|
||||
"Which key carries the cap. max_tokens is the default and the vast majority of compatible gateways only accept it; OpenAI's official reasoning models (the o series / GPT-5) conversely only accept max_completion_tokens and fail outright with unsupported_parameter when they receive max_tokens.",
|
||||
anthropic: "Only selectable for the openai format. Anthropic's field name is fixed as max_tokens.",
|
||||
"openai-responses": "Only selectable for the openai format. The Responses API's field name is fixed as max_output_tokens.",
|
||||
ollama: "Only selectable for the openai format. Ollama's field name is fixed as max_tokens.",
|
||||
};
|
||||
// LLM profile formats. The default in the create form is "anthropic"; "ollama" is the
|
||||
// keyless local-server format (base URL defaults to http://127.0.0.1:11434/v1).
|
||||
type ProfileFormat = "anthropic" | "openai" | "openai-responses" | "ollama";
|
||||
const OLLAMA_DEFAULT_BASE_URL = "http://127.0.0.1:11434/v1";
|
||||
const EFFORT_LEVELS: { value: string; label: string }[] = [
|
||||
{ value: NONE, label: "Not sent (default)" },
|
||||
{ value: "low", label: "low" },
|
||||
@@ -72,10 +77,11 @@ function cooldownText(secs: number) {
|
||||
}
|
||||
|
||||
// The "healthy or not" badge a profile shows on its card. A profile without a Key cannot send any request at all — say that before the breaker;
|
||||
// the other states come from the pool's breaker records (with polling off no new records appear, so "healthy" = no known failure).
|
||||
// the exception is Ollama, a keyless local server that IS callable without a key. The other states come from the pool's breaker
|
||||
// records (with polling off no new records appear, so "healthy" = no known failure).
|
||||
type Health = { label: string; cls: string; hint?: string };
|
||||
function healthOf(p: LLMProfile, m?: LLMPoolMember): Health {
|
||||
if (!p.api_key_hint) {
|
||||
if (!p.api_key_hint && p.format !== "ollama") {
|
||||
return {
|
||||
label: "No Key configured",
|
||||
cls: "border-muted-foreground/40 text-muted-foreground",
|
||||
@@ -215,7 +221,8 @@ function PoolSheet({
|
||||
</div>
|
||||
{inChain.length < 2 && (
|
||||
<p className="text-muted-foreground text-xs">
|
||||
Only {inChain.length} usable profile(s) right now, so the pool has no effect — at least 2 profiles with an API Key set and pool participation are required.
|
||||
Only {inChain.length} usable profile(s) right now, so the pool has no effect — at least 2 usable profiles (API Key set, or Ollama) with pool participation are
|
||||
required.
|
||||
</p>
|
||||
)}
|
||||
{chain.map((m) => {
|
||||
@@ -311,7 +318,7 @@ function ProfileSheet({
|
||||
}) {
|
||||
const isNew = !profile;
|
||||
const [name, setName] = React.useState("");
|
||||
const [format, setFormat] = React.useState<"anthropic" | "openai" | "openai-responses">("anthropic");
|
||||
const [format, setFormat] = React.useState<ProfileFormat>("anthropic");
|
||||
const [model, setModel] = React.useState("");
|
||||
const [baseUrl, setBaseUrl] = React.useState("");
|
||||
const [proxy, setProxy] = React.useState("");
|
||||
@@ -340,7 +347,11 @@ function ProfileSheet({
|
||||
React.useEffect(() => {
|
||||
if (!open) return;
|
||||
setName(profile?.name ?? "");
|
||||
setFormat(profile?.format === "openai" || profile?.format === "openai-responses" ? profile.format : "anthropic");
|
||||
setFormat(
|
||||
profile?.format === "openai" || profile?.format === "openai-responses" || profile?.format === "ollama"
|
||||
? profile.format
|
||||
: "anthropic",
|
||||
);
|
||||
setModel(profile?.model ?? "");
|
||||
setBaseUrl(profile?.base_url ?? "");
|
||||
setProxy(profile?.proxy ?? "");
|
||||
@@ -457,6 +468,13 @@ function ProfileSheet({
|
||||
}
|
||||
}
|
||||
|
||||
let keyPlaceholder = "sk-…";
|
||||
if (format === "ollama") {
|
||||
keyPlaceholder = "optional — a local Ollama server runs keyless";
|
||||
} else if (keyHint) {
|
||||
keyPlaceholder = `Set (${keyHint}); leave empty to keep it`;
|
||||
}
|
||||
|
||||
return (
|
||||
<Sheet open={open} onOpenChange={onOpenChange}>
|
||||
<SheetContent
|
||||
@@ -492,7 +510,7 @@ function ProfileSheet({
|
||||
</div>
|
||||
<div className="grid gap-2">
|
||||
<Label>Format</Label>
|
||||
<Select value={format} onValueChange={(v) => setFormat(v as "anthropic" | "openai" | "openai-responses")}>
|
||||
<Select value={format} onValueChange={(v) => setFormat(v as ProfileFormat)}>
|
||||
<SelectTrigger>
|
||||
<SelectValue placeholder="Select a format" />
|
||||
</SelectTrigger>
|
||||
@@ -500,6 +518,7 @@ function ProfileSheet({
|
||||
<SelectItem value="anthropic">Anthropic</SelectItem>
|
||||
<SelectItem value="openai">OpenAI (Chat Completions)</SelectItem>
|
||||
<SelectItem value="openai-responses">OpenAI (Responses API)</SelectItem>
|
||||
<SelectItem value="ollama">Ollama (local, no key needed)</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
@@ -511,7 +530,7 @@ function ProfileSheet({
|
||||
<Input
|
||||
id="p-model"
|
||||
className="font-mono"
|
||||
placeholder="claude-opus-4-8"
|
||||
placeholder={format === "ollama" ? "llama3.2" : "claude-opus-4-8"}
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
/>
|
||||
@@ -557,7 +576,7 @@ function ProfileSheet({
|
||||
<Input
|
||||
id="p-base-url"
|
||||
className="font-mono"
|
||||
placeholder="https://api.openai.com/v1"
|
||||
placeholder={format === "ollama" ? OLLAMA_DEFAULT_BASE_URL : "https://api.openai.com/v1"}
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
/>
|
||||
@@ -595,11 +614,13 @@ function ProfileSheet({
|
||||
</div>
|
||||
|
||||
<div className="grid gap-2">
|
||||
<Label htmlFor="p-api-key">API Key</Label>
|
||||
<Label htmlFor="p-api-key">
|
||||
{format === "ollama" ? "API Key (optional for Ollama)" : "API Key"}
|
||||
</Label>
|
||||
<Input
|
||||
id="p-api-key"
|
||||
type="password"
|
||||
placeholder={keyHint ? `Set (${keyHint}); leave empty to keep it` : "sk-…"}
|
||||
placeholder={keyPlaceholder}
|
||||
value={apiKey}
|
||||
onChange={(e) => setApiKey(e.target.value)}
|
||||
/>
|
||||
|
||||
@@ -1,314 +0,0 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
|
||||
import {
|
||||
CheckCircle2Icon,
|
||||
DownloadIcon,
|
||||
ExternalLinkIcon,
|
||||
RefreshCwIcon,
|
||||
RotateCcwIcon,
|
||||
TriangleAlertIcon,
|
||||
} from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card";
|
||||
import { Progress } from "@/components/ui/progress";
|
||||
import { api, sseUrl } from "@/lib/api";
|
||||
import type { UpdateCheck, UpdateProgress } from "@/lib/types";
|
||||
|
||||
/** The longest wait for a new version to come online. One upgrade goes through three process starts
|
||||
* (stage → swap → new version), each taking seconds; three minutes covers slow disks and Docker container rebuilds. */
|
||||
const RESTART_TIMEOUT_MS = 180_000;
|
||||
|
||||
function humanSize(n?: number): string {
|
||||
if (!n || n <= 0) return "";
|
||||
const units = ["B", "KB", "MB", "GB"];
|
||||
let v = n;
|
||||
let i = 0;
|
||||
while (v >= 1024 && i < units.length - 1) {
|
||||
v /= 1024;
|
||||
i++;
|
||||
}
|
||||
return `${v.toFixed(i === 0 ? 0 : 1)} ${units[i]}`;
|
||||
}
|
||||
|
||||
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
export function UpdateCard() {
|
||||
const [info, setInfo] = React.useState<UpdateCheck | null>(null);
|
||||
const [checking, setChecking] = React.useState(true);
|
||||
const [progress, setProgress] = React.useState<UpdateProgress | null>(null);
|
||||
// Separate from progress: once staging completes the process is gone, SSE drops, and we must switch to polling /api/health.
|
||||
const [restarting, setRestarting] = React.useState(false);
|
||||
const [busy, setBusy] = React.useState(false);
|
||||
|
||||
// quiet also decides whether to bypass the backend cache: the automatic check on page load uses the cache (the top bar just checked),
|
||||
// while a manual "Check for updates" click forces a refresh — otherwise a freshly released version only appears after the cache expires.
|
||||
const check = React.useCallback((quiet = false) => {
|
||||
setChecking(true);
|
||||
api
|
||||
.checkUpdate(!quiet)
|
||||
.then((r) => {
|
||||
setInfo(r);
|
||||
if (!quiet) {
|
||||
if (r.error) toast.error("Update check failed: " + r.error);
|
||||
else if (r.has_update) toast.success(`New version ${r.latest} found`);
|
||||
else if (r.comparable) toast.success("Already on the latest version");
|
||||
}
|
||||
})
|
||||
.catch((e) => {
|
||||
if (!quiet) toast.error("Update check failed: " + (e as Error).message);
|
||||
})
|
||||
.finally(() => setChecking(false));
|
||||
}, []);
|
||||
|
||||
React.useEffect(() => {
|
||||
check(true);
|
||||
}, [check]);
|
||||
|
||||
// Poll /api/health until the version changes.
|
||||
//
|
||||
// The criterion must be "the version changed", not "it connects again": during the swap the old version briefly comes back up once
|
||||
// (that run only swaps in artex.new and exits immediately), so checking connectivity alone would misjudge success.
|
||||
const waitForNewVersion = React.useCallback(async (fromVersion: string) => {
|
||||
setRestarting(true);
|
||||
const deadline = Date.now() + RESTART_TIMEOUT_MS;
|
||||
while (Date.now() < deadline) {
|
||||
await sleep(2000);
|
||||
try {
|
||||
const r = await fetch("/api/health", { cache: "no-store" });
|
||||
if (r.ok) {
|
||||
const j = (await r.json()) as { version?: string };
|
||||
if (j.version && j.version !== fromVersion) {
|
||||
toast.success(`Updated to ${j.version}, reloading the page`);
|
||||
await sleep(800);
|
||||
window.location.reload();
|
||||
return;
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Failing to connect during the restart window is expected; keep polling.
|
||||
}
|
||||
}
|
||||
setRestarting(false);
|
||||
toast.error("Timed out waiting for the service to restart. Check the backend logs, or confirm artex was started via start.sh / start.bat.");
|
||||
}, []);
|
||||
|
||||
// Subscribe to update progress. SSE bypasses Next's /api rewrite (that layer buffers and events never come through).
|
||||
const openStream = React.useCallback(
|
||||
(fromVersion: string) => {
|
||||
const es = new EventSource(sseUrl("/api/update/stream"));
|
||||
es.onmessage = (ev) => {
|
||||
let p: UpdateProgress;
|
||||
try {
|
||||
p = JSON.parse(ev.data) as UpdateProgress;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
setProgress(p);
|
||||
if (p.phase === "failed") {
|
||||
es.close();
|
||||
setBusy(false);
|
||||
toast.error("Update failed: " + (p.error || p.message));
|
||||
return;
|
||||
}
|
||||
if (p.phase === "staged") {
|
||||
es.close();
|
||||
void waitForNewVersion(fromVersion);
|
||||
}
|
||||
};
|
||||
es.onerror = () => {
|
||||
// SSE always drops when the process exits. If we are already waiting for the restart, that is normal —
|
||||
// leave the verdict to the /api/health polling.
|
||||
es.close();
|
||||
};
|
||||
return es;
|
||||
},
|
||||
[waitForNewVersion],
|
||||
);
|
||||
|
||||
const doUpdate = () => {
|
||||
if (!info) return;
|
||||
const from = info.current;
|
||||
const ok = window.confirm(
|
||||
`Update to ${info.latest}?\n\n` +
|
||||
"Updating restarts the program; running tasks will be interrupted.\n" +
|
||||
(info.mode === "docker"
|
||||
? "\nNote: an in-container update only replaces the program itself and does not update the toolchain in the image (playwright / nmap etc.);" +
|
||||
"if the new version needs new tools, use docker compose pull instead."
|
||||
: ""),
|
||||
);
|
||||
if (!ok) return;
|
||||
|
||||
setBusy(true);
|
||||
setProgress({ phase: "downloading", percent: 0, message: "Preparing…" });
|
||||
const es = openStream(from);
|
||||
api.applyUpdate().catch((e) => {
|
||||
es.close();
|
||||
setBusy(false);
|
||||
setProgress(null);
|
||||
toast.error("Failed to start the update: " + (e as Error).message);
|
||||
});
|
||||
};
|
||||
|
||||
const doRollback = () => {
|
||||
if (!info) return;
|
||||
if (
|
||||
!window.confirm(
|
||||
"Roll back to the previous version?\n\nThe program will restart and running tasks will be interrupted.\nNote: the database schema is not rolled back; the old version may not recognize data written by the new one.",
|
||||
)
|
||||
)
|
||||
return;
|
||||
const from = info.current;
|
||||
setBusy(true);
|
||||
api
|
||||
.rollbackUpdate()
|
||||
.then(() => {
|
||||
toast.success("Switched to the previous version, restarting…");
|
||||
void waitForNewVersion(from);
|
||||
})
|
||||
.catch((e) => {
|
||||
setBusy(false);
|
||||
toast.error("Rollback failed: " + (e as Error).message);
|
||||
});
|
||||
};
|
||||
|
||||
const phase = progress?.phase;
|
||||
const showProgress = busy || restarting;
|
||||
// Only the download phase has a real percentage (computed from Content-Length). Verify/unpack/wait-for-restart are
|
||||
// phases of unknown duration, so the bar fills up with a pulse animation meaning "busy, unsure how long".
|
||||
const downloading = !restarting && phase === "downloading";
|
||||
const pct = downloading ? Math.max(progress?.percent ?? 0, 0) : 100;
|
||||
|
||||
return (
|
||||
// The settings page uses a multi-column masonry layout; each card handles its own row spacing and avoids breaking across columns (see the comment in page.tsx).
|
||||
<Card className="mb-4 break-inside-avoid md:mb-6">
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2 text-base">
|
||||
<DownloadIcon className="size-4" />
|
||||
Version & updates
|
||||
</CardTitle>
|
||||
<CardDescription>Check for and install new versions from GitHub. Updating restarts the program; running tasks will be interrupted.</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div className="flex flex-wrap items-center gap-2 text-sm">
|
||||
<span className="text-muted-foreground">Current version</span>
|
||||
<Badge variant="secondary" className="font-mono">
|
||||
{info?.current ?? "…"}
|
||||
</Badge>
|
||||
{info && (
|
||||
<>
|
||||
<Badge variant="outline" className="font-mono">
|
||||
{info.os}/{info.arch}
|
||||
</Badge>
|
||||
<Badge variant="outline">{info.mode === "docker" ? "Docker" : "Standalone"}</Badge>
|
||||
</>
|
||||
)}
|
||||
{info?.latest && (
|
||||
<>
|
||||
<span className="text-muted-foreground">Latest version</span>
|
||||
<Badge variant={info.has_update ? "default" : "secondary"} className="font-mono">
|
||||
{info.latest}
|
||||
</Badge>
|
||||
</>
|
||||
)}
|
||||
{info?.html_url && (
|
||||
<a
|
||||
href={info.html_url}
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs text-muted-foreground underline-offset-4 hover:underline"
|
||||
>
|
||||
Release notes <ExternalLinkIcon className="size-3" />
|
||||
</a>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{info?.boot_notice && (
|
||||
<p className="flex items-start gap-2 rounded-md border border-amber-500/40 bg-amber-500/10 p-2 text-xs text-amber-700 dark:text-amber-400">
|
||||
<TriangleAlertIcon className="mt-0.5 size-3.5 shrink-0" />
|
||||
{info.boot_notice}
|
||||
</p>
|
||||
)}
|
||||
|
||||
{info?.error && (
|
||||
<p className="flex items-start gap-2 rounded-md border border-destructive/40 bg-destructive/10 p-2 text-xs text-destructive">
|
||||
<TriangleAlertIcon className="mt-0.5 size-3.5 shrink-0" />
|
||||
Cannot reach GitHub: {info.error}
|
||||
{" "}Configure the global proxy above and retry.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{info && !info.comparable && info.reason && <p className="text-xs text-muted-foreground">{info.reason}</p>}
|
||||
|
||||
{info?.has_update && info.asset_available === false && (
|
||||
<p className="flex items-start gap-2 rounded-md border border-destructive/40 bg-destructive/10 p-2 text-xs text-destructive">
|
||||
<TriangleAlertIcon className="mt-0.5 size-3.5 shrink-0" />
|
||||
{info.latest} has no release package for {info.os}/{info.arch} (missing {info.asset}); automatic update is unavailable.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{info?.has_update && info.asset_available !== false && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Will download <span className="font-mono">{info.asset}</span>
|
||||
{info.size ? ` (${humanSize(info.size)})` : ""}; it is replaced only after the SHA256 check and a smoke test pass — on failure the current version is kept automatically.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{info && !info.has_update && info.comparable && !info.error && (
|
||||
<p className="flex items-center gap-2 text-xs text-muted-foreground">
|
||||
<CheckCircle2Icon className="size-3.5 text-emerald-600" />
|
||||
Already on the latest version.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{info?.mode === "docker" && info.has_update && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Under Docker an update only replaces the program itself, not the toolchain in the image (playwright / nmap etc.), and
|
||||
<span className="font-mono"> docker compose up -d </span>
|
||||
rebuilding the container reverts to the image's bundled version. To upgrade the image too, run
|
||||
<span className="font-mono"> docker compose pull artex && docker compose up -d artex</span>.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{showProgress && (
|
||||
<div className="space-y-1.5">
|
||||
<Progress value={pct} className={downloading ? undefined : "animate-pulse"} />
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{restarting ? "Restarting and applying the new version, please wait (the page will refresh automatically)…" : progress?.message}
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button variant="outline" size="sm" onClick={() => check(false)} disabled={checking || busy || restarting}>
|
||||
<RefreshCwIcon className={checking ? "size-4 animate-spin" : "size-4"} />
|
||||
Check for updates
|
||||
</Button>
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={doUpdate}
|
||||
disabled={busy || restarting || !info?.has_update || info?.asset_available === false}
|
||||
>
|
||||
<DownloadIcon className="size-4" />
|
||||
{info?.has_update ? `Update to ${info.latest}` : "Update now"}
|
||||
</Button>
|
||||
{info?.has_backup && (
|
||||
<Button variant="ghost" size="sm" onClick={doRollback} disabled={busy || restarting}>
|
||||
<RotateCcwIcon className="size-4" />
|
||||
Roll back to the previous version
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<p className="text-xs text-muted-foreground">
|
||||
One-click updates rely on the supervisor script to restart the program. Please start ARTEX via <span className="font-mono">start.sh</span> (Windows:
|
||||
<span className="font-mono"> start.bat</span>); when running the artex binary directly, the program is not relaunched after exiting.
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
);
|
||||
}
|
||||
@@ -15,8 +15,6 @@ import { api } from "@/lib/api";
|
||||
import { CHAT_SEND_MODE_OPTIONS, type ChatSendMode, setChatSendMode, useChatSendMode } from "@/lib/chat-send-mode";
|
||||
import type { Settings } from "@/lib/types";
|
||||
|
||||
import { UpdateCard } from "./_components/update-card";
|
||||
|
||||
export default function SystemSettingsPage() {
|
||||
const [trafficCapture, setTrafficCapture] = React.useState(false);
|
||||
const [agentTrafficBinding, setAgentTrafficBinding] = React.useState(false);
|
||||
@@ -267,8 +265,6 @@ export default function SystemSettingsPage() {
|
||||
balance-fill by content height. Card spacing uses mb rather than gap — in a multi-column layout
|
||||
column-gap only covers the gaps between columns; row spacing must come from the children. */}
|
||||
<div className="columns-1 gap-4 md:gap-6 lg:columns-2">
|
||||
<UpdateCard />
|
||||
|
||||
<Card className="mb-4 break-inside-avoid md:mb-6">
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2 text-base">
|
||||
|
||||
@@ -102,7 +102,6 @@ import type {
|
||||
TrafficEvidenceRole,
|
||||
TrafficHost,
|
||||
TrafficResp,
|
||||
UpdateCheck,
|
||||
UsageStats,
|
||||
WorkspaceFile,
|
||||
WorkspaceListing,
|
||||
@@ -1310,15 +1309,4 @@ export const api = {
|
||||
// ledger, exact per call, covering per-agent bindings / pooling / interrupted spend).
|
||||
tokensByModel: (task: string) =>
|
||||
get<{ models: ModelTokenStat[] }>(`/llm/records/by-model?task=${encodeURIComponent(task)}`),
|
||||
|
||||
// ---- One-click update ----
|
||||
// The backend arbitrates update checks: it does the downloading, and "the browser reaches
|
||||
// GitHub but the server cannot" is common (server on an intranet, proxy configured only in
|
||||
// the browser) — clicking update would inevitably fail then. The backend caches GitHub
|
||||
// query results for 30 minutes (the unauthenticated GitHub API allows 60 requests/hour/IP;
|
||||
// the top bar queries on every full page load and would burn the quota fast without caching). force=true bypasses the cache, reserved for the user's explicit "Check for updates" click.
|
||||
checkUpdate: (force = false) => get<UpdateCheck>(`/update/check${force ? "?force=1" : ""}`),
|
||||
// Returns 202 immediately; the download runs in the background, progress via /api/update/stream.
|
||||
applyUpdate: () => post<{ ok: boolean; target: string }>(`/update/apply`),
|
||||
rollbackUpdate: () => post<{ ok: boolean }>(`/update/rollback`),
|
||||
};
|
||||
|
||||
+1
-42
@@ -1028,7 +1028,7 @@ export interface NotificationDelivery {
|
||||
export interface LLMProfile {
|
||||
id: string;
|
||||
name: string;
|
||||
format: "openai" | "anthropic" | "openai-responses";
|
||||
format: "openai" | "anthropic" | "openai-responses" | "ollama";
|
||||
base_url?: string;
|
||||
proxy?: string;
|
||||
model: string;
|
||||
@@ -1591,47 +1591,6 @@ export interface FindingTrafficDetail {
|
||||
response: EvidenceBodyPreview;
|
||||
}
|
||||
|
||||
/** GET /api/update/check — comparison of the current version against the latest GitHub stable release. */
|
||||
export interface UpdateCheck {
|
||||
/** The currently running version; development builds are "dev" or a suffixed git describe form. */
|
||||
current: string;
|
||||
/** Runtime form. Under docker the swap only touches the container writable layer; rebuilding the container reverts to the image version. */
|
||||
mode: "docker" | "binary";
|
||||
os: string;
|
||||
arch: string;
|
||||
repo: string;
|
||||
/** Whether a previous version exists to roll back to (artex.old). */
|
||||
has_backup: boolean;
|
||||
/** Conclusion of the self-update bootstrap at this startup (swap failed / rolled back, etc.); empty when nothing happened. */
|
||||
boot_notice?: string;
|
||||
rolled_back?: boolean;
|
||||
/** Reason given when the GitHub query fails; the fields below are absent then. */
|
||||
error?: string;
|
||||
latest?: string;
|
||||
notes?: string;
|
||||
html_url?: string;
|
||||
published_at?: string;
|
||||
/** The release asset name for the current platform, and whether that Release actually carries it. */
|
||||
asset?: string;
|
||||
asset_available?: boolean;
|
||||
size?: number;
|
||||
has_update?: boolean;
|
||||
/** Whether the two version numbers are comparable; false for development builds, which disables one-click update. */
|
||||
comparable?: boolean;
|
||||
/** Explanation shown when comparable is false. */
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
/** One update progress event pushed by /api/update/stream. */
|
||||
export interface UpdateProgress {
|
||||
phase: "idle" | "downloading" | "verifying" | "extracting" | "staged" | "failed";
|
||||
/** Meaningful only during the downloading phase (0-100); -1 otherwise. */
|
||||
percent: number;
|
||||
message: string;
|
||||
version?: string;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
// Original execution selected from an approval, never submitted to the reviewer.
|
||||
export interface InterceptExecution {
|
||||
conversation_id: number | null;
|
||||
|
||||
Reference in New Issue
Block a user