mirror of
https://github.com/BaaRai/ARTEX.git
synced 2026-11-13 11:40:53 +08:00
Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a20628d334 | ||
|
|
23cf14cf0a | ||
|
|
5bad722593 | ||
|
|
2aab3e8210 | ||
|
|
d45f0c7811 | ||
|
|
e6f4de5ddc | ||
|
|
ac601b7e59 | ||
|
|
9ebc8b6703 | ||
|
|
1c35c9cf68 | ||
|
|
1ed154e86c | ||
|
|
4034ce4562 | ||
|
|
7f8c7cbcae | ||
|
|
9e63a8031b | ||
|
|
978f308a9e | ||
|
|
80a1ee96f6 |
+4
-1
@@ -1,5 +1,8 @@
|
||||
# Docker deployment configuration (copy to .env)
|
||||
ARTEX_TAG=latest
|
||||
# The image is built locally by `docker compose build`; there is no registry tag.
|
||||
# Address the web UI is published on. 127.0.0.1 keeps it host-local; set 0.0.0.0
|
||||
# to reach it from another machine or from Windows when ARTEX runs inside WSL.
|
||||
ARTEX_BIND_ADDR=127.0.0.1
|
||||
POSTGRES_USER=artex
|
||||
POSTGRES_PASSWORD=change-me-please
|
||||
POSTGRES_DB=artex
|
||||
|
||||
+17
-3
@@ -4,13 +4,26 @@ All notable changes to this project are documented in this file. The format is b
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
|
||||
- **The image is built from the checkout instead of pulled**: `docker-compose.yml` no longer references the `autumn27/artex` namespace, whose repository is gone and whose name its holder could republish at any time. `install.sh` and `update.sh` build the image locally.
|
||||
- **`update.sh` refuses to rebuild stale code**: when `git pull --ff-only` fails on a diverged clone (including every clone made before the 2026-10-23 history rewrite), it now stops with recovery commands instead of warning, rebuilding the old tree and printing "Build complete".
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Ollama base URL on the environment path**: `ARTEX_LLM_BASE_URL=http://host:11434` is normalized to the `/v1` root the same way the saved-profile path already was, so a LAN Ollama server configured through the environment no longer answers 404 on chat.
|
||||
|
||||
### Added
|
||||
|
||||
- **Ollama support**: an `ollama` LLM profile format for a local or LAN Ollama server. Keyless by design: no API key is needed, `OPENAI_API_KEY` is never inherited, the base URL defaults to `http://127.0.0.1:11434/v1` and is normalized to the `/v1` root, and the profile works in the failover pool, task LLM chains, model listing and connection tests. Contributed by @glauberportella (#128).
|
||||
|
||||
## [0.4.0] - 2026-10-23
|
||||
|
||||
First release of this fork. The engine underneath is still upstream's, but this tree carries a long stretch of operational work on top of it: a recon toolbox and skills, findings discipline, cost accounting, and container packaging.
|
||||
|
||||
### Added
|
||||
|
||||
- **Recon toolbox in the image**: pinned nuclei, httpx, subfinder, katana, ffuf, dalfox and a sqlmap checkout, verified at build time, so hunt workers can call them directly (#112).
|
||||
- **Recon toolbox in the image**: nuclei, httpx, subfinder, katana, ffuf and dalfox installed at release-pinned versions, plus a sqlmap checkout that tracks the default branch, each smoke-tested at build time (run once, not checksum-verified), so hunt workers can call them directly (#112). Checksum verification arrives with #127.
|
||||
- **Recursive recon skill** (`bbot-recon`): bbot v3.0.2 in its own virtualenv with module dependencies preinstalled. Policy-gated modes (passive seeds-only when scanners are banned, active-safe, loud only on explicit permission) and JSONL ingestion into the facts graph (#118).
|
||||
- **php2shell skill**: turns a file-inclusion finding into verified RCE through the session.upload_progress chain, lab first and gated on program policy (#113).
|
||||
- **Code audit and reverse-engineering skills** (#55, #77).
|
||||
@@ -30,9 +43,10 @@ First release of this fork. The engine underneath is still upstream's, but this
|
||||
- **Planner quiet mode** for less narration during long runs (#81, #88).
|
||||
- **Findings discipline**: proof-basis gates, measured impact, adversarial verification and dedup rules decide what a finding must contain before it is reported (#51, #53, #63, #69).
|
||||
- **Prompt-injection hardening** on seed material and MCP tool descriptions (#49).
|
||||
- **Pause preserves the task timeout budget**, and paused time is frozen in the task clock (#2, #65, #110).
|
||||
- **Pause preserves the task timeout budget**, and paused time is frozen in the task clock (#2, #110).
|
||||
- **Deadline contract** in the executor seeds (#65).
|
||||
- **Task detail metrics and outcome cards** summarise a finished run (#92, #102).
|
||||
- **Skill-gap, loop-contract and shadow-lesson reporting** for the planner (#104, #106, #108).
|
||||
- **Skill-gap and loop-contract reporting** for the planner (#104, #106); the RSI notes land as documentation in #108.
|
||||
|
||||
### Removed
|
||||
|
||||
|
||||
@@ -84,10 +84,13 @@ cd ARTEX
|
||||
|
||||
The script will: detect / auto-install Docker → then let you choose **① all-Docker** or **② local build and run**:
|
||||
|
||||
- **① All-Docker**: enter a Postgres password (press Enter for a random one) → auto-writes `.env` → `docker compose up -d`.
|
||||
- **① All-Docker**: enter a Postgres password (press Enter for a random one) → auto-writes `.env` → `docker compose up -d --build` (the image is built from this checkout).
|
||||
- **② Local run**: choose a database (connect to an existing one / start one with Docker) → generates `config.json` → `go` builds the embedded single binary → starts.
|
||||
|
||||
Once installed, open **http://localhost:8787** (on first entry, `/setup` sets the admin password).
|
||||
`/setup` asks for a one-time setup token: while no password is set, the backend prints it at startup in its log (`docker compose logs artex | grep "Setup token"`, or the terminal running `./start.sh`).
|
||||
|
||||
The web UI is published on `127.0.0.1` by default. To reach it from another machine, or from Windows when ARTEX runs inside WSL, set `ARTEX_BIND_ADDR=0.0.0.0` in `.env` (the setup token still protects the first-run claim), or keep loopback and use a reverse proxy or an SSH tunnel.
|
||||
|
||||
### Option 2: Docker Compose (manual)
|
||||
|
||||
@@ -95,10 +98,14 @@ Once installed, open **http://localhost:8787** (on first entry, `/setup` sets th
|
||||
git clone https://github.com/BaaRai/ARTEX.git
|
||||
cd ARTEX
|
||||
cp .env.example .env # set POSTGRES_PASSWORD, optionally ANTHROPIC_API_KEY
|
||||
docker compose up -d # pulls the autumn27/artex image + postgres
|
||||
# build the single binary first (frontend, then go build -tags embedui), see the Dockerfile header
|
||||
docker compose up -d --build # builds artex from this checkout + starts postgres
|
||||
# → http://localhost:8787
|
||||
```
|
||||
|
||||
> The image is built locally. There is no published image to pull: the previous
|
||||
> default pointed at a third-party namespace that no longer hosts this project.
|
||||
|
||||
The image already includes common tools (ripgrep/curl/vim/npm/nmap…); `./skills` and `./data` are persisted via bind mounts.
|
||||
|
||||
For remote MCP, you can choose `http` (Streamable HTTP) or `sse` (legacy SSE) in system settings.
|
||||
@@ -175,6 +182,19 @@ Daily cron example (03:20, operator crontab):
|
||||
|
||||
> Upgrading only swaps the program and never touches data: the Postgres volume `pgdata`, `./data` (jwt.key / SQLite, etc.), and `./skills` are all preserved. **No manual database migration is needed**: `artex` idempotently re-runs `schema.sql` on every start (with `ADD COLUMN` / `CREATE INDEX IF NOT EXISTS`), i.e. "restart equals migrate". It is still recommended to back up `./data` and the database before upgrading.
|
||||
|
||||
### Upgrading from a clone made before 2026-10-23
|
||||
|
||||
The repository history was rewritten that day to drop a personal email address from the commit metadata. A clone made before then cannot fast-forward, and `./update.sh` now stops instead of silently rebuilding stale code. Recover once, then upgrade normally:
|
||||
|
||||
```bash
|
||||
git stash # only if you have local changes
|
||||
git fetch origin
|
||||
git reset --hard origin/main
|
||||
git stash pop # only if you stashed
|
||||
```
|
||||
|
||||
Your data is not affected: `./data`, the `pgdata` volume and `./skills` are not part of the git history.
|
||||
|
||||
### Option 1: one-click update script
|
||||
|
||||
```bash
|
||||
@@ -184,17 +204,16 @@ cd ARTEX
|
||||
|
||||
The script first optionally `git pull`s the latest code, then lets you choose **① Docker update** or **② local build update** (matching `install.sh`):
|
||||
|
||||
- **① Docker**: you can specify a target image tag (press Enter to keep `.env`'s `ARTEX_TAG`, default `latest`) → `docker compose pull` → `docker compose up -d` (restarting with the new image migrates automatically).
|
||||
- **① Docker**: rebuilds the image from this checkout → `docker compose build artex` → `docker compose up -d artex` (restarting migrates the schema automatically).
|
||||
- **② Local**: rebuild the frontend static assets → recompile `./artex` (restart the process for it to take effect).
|
||||
|
||||
### Option 2: Docker Compose (manual)
|
||||
|
||||
```bash
|
||||
cd ARTEX
|
||||
git pull # update compose / scripts (optional)
|
||||
# pin a version: set ARTEX_TAG=v0.2.0 in .env; otherwise latest is used
|
||||
docker compose pull artex
|
||||
docker compose up -d artex # restart with the new image → schema migrates automatically
|
||||
git pull # update compose / scripts
|
||||
docker compose build artex # rebuild the image from this checkout
|
||||
docker compose up -d artex # recreate the container → schema migrates automatically
|
||||
docker image prune -f # clean up old images (optional)
|
||||
```
|
||||
|
||||
@@ -233,8 +252,8 @@ CGO_ENABLED=0 go build -tags embedui -o artex ./cmd/artex
|
||||
}
|
||||
```
|
||||
|
||||
**LLM**: `export ANTHROPIC_API_KEY=sk-...` (or `OPENAI_API_KEY`); you can also fill it in on the UI's "LLM Configuration" page.
|
||||
Optional: `ARTEX_LLM_PROVIDER` / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ARTEX_LLM_PROXY`.
|
||||
**LLM**: `export ANTHROPIC_API_KEY=sk-...` (or `OPENAI_API_KEY`); you can also fill it in on the UI's "LLM Configuration" page — which can add a local Ollama server as a keyless provider (format "Ollama", base URL default `http://127.0.0.1:11434/v1`, models loaded from the server).
|
||||
Optional: `ARTEX_LLM_PROVIDER` (anthropic | openai | openai-responses | ollama) / `ARTEX_LLM_MODEL` / `ARTEX_LLM_BASE_URL` / `ARTEX_LLM_PROXY`.
|
||||
|
||||
**Concurrency**: the number of work agents per task is configured in "System Settings" (default 3).
|
||||
|
||||
|
||||
+94
-10
@@ -3,8 +3,10 @@
|
||||
// §4.3 (planner) and §4.4 (work agent).
|
||||
//
|
||||
// Provider configuration is read from the environment so the system runs with
|
||||
// any Anthropic- or OpenAI-format endpoint. If no key is configured, FromEnv
|
||||
// returns ok=false and the exploration engine stays idle (an LLM is required).
|
||||
// any Anthropic-, OpenAI- or Ollama-format endpoint (the last being keyless).
|
||||
// If no usable configuration is found — a key for the cloud formats, a base
|
||||
// URL for keyless Ollama — FromEnv returns ok=false and the exploration engine
|
||||
// stays idle (an LLM is required).
|
||||
package agent
|
||||
|
||||
import (
|
||||
@@ -125,6 +127,47 @@ const (
|
||||
maxWindowK = 1000 // cap at 1M tokens (user request)
|
||||
)
|
||||
|
||||
// FormatOllama marks a LOCAL Ollama server profile. Ollama exposes an
|
||||
// OpenAI-compatible Chat Completions endpoint (/v1), so on the wire it speaks
|
||||
// llm.FormatOpenAI; the distinct sentinel matters everywhere a profile must be
|
||||
// treated as usable WITHOUT an API key — Ollama runs keyless on localhost/LAN.
|
||||
// See NeedsAPIKey and NewProvider (which maps it onto FormatOpenAI while
|
||||
// forcing our base-URL/key defaults so an empty key never falls back to
|
||||
// OpenAI's public endpoint).
|
||||
const FormatOllama llm.Format = "ollama"
|
||||
|
||||
// DefaultOllamaBaseURL is the base of Ollama's OpenAI-compatible endpoint the
|
||||
// provider appends /chat/completions to. Must never end up empty: with an
|
||||
// empty BaseURL, the SDK would fall back to OpenAI's public endpoint.
|
||||
const DefaultOllamaBaseURL = "http://127.0.0.1:11434/v1"
|
||||
|
||||
// ollamaPlaceholderKey is sent as the Bearer token when no key is configured.
|
||||
// Ollama ignores authorization, but a concrete value keeps the SDK from
|
||||
// falling back to OPENAI_API_KEY (which would silently ship a real cloud key
|
||||
// to the local server) and keeps the header stable.
|
||||
const ollamaPlaceholderKey = "ollama"
|
||||
|
||||
// normalizeOllamaBaseURL maps whatever the operator typed onto the /v1 root the
|
||||
// OpenAI-compatible provider expects: it strips a pasted "/chat/completions"
|
||||
// suffix, an empty value becomes the local default, and a missing "/v1" is
|
||||
// appended so the natural "http://host:11434" also works. Shared by the saved
|
||||
// profile path (ConfigFrom) and the environment path (FromEnv): both used to
|
||||
// differ, and only the profile path normalized.
|
||||
func normalizeOllamaBaseURL(raw string) string {
|
||||
b := strings.TrimRight(strings.TrimSuffix(strings.TrimSpace(raw), "/chat/completions"), "/")
|
||||
if b == "" {
|
||||
return DefaultOllamaBaseURL
|
||||
}
|
||||
if !strings.HasSuffix(b, "/v1") {
|
||||
b += "/v1"
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// NeedsAPIKey reports whether this provider format requires an API key to be
|
||||
// usable. Ollama (local server) is keyless; every cloud format needs one.
|
||||
func (c Config) NeedsAPIKey() bool { return c.Format != FormatOllama }
|
||||
|
||||
// CompactionWindow returns the model context window in TOKENS for compaction
|
||||
// thresholds, resolved from the user-configured size (ContextWindowK). 0/unset →
|
||||
// a 200K default; otherwise clamped to [32K, 1M] so compaction stays effective.
|
||||
@@ -154,11 +197,11 @@ func compactionConfig(windowTokens int) *compaction.Config {
|
||||
|
||||
// FromEnv resolves the LLM provider config:
|
||||
//
|
||||
// ARTEX_LLM_PROVIDER = anthropic|openai (default: inferred from keys)
|
||||
// ARTEX_LLM_PROVIDER = anthropic|openai|openai-responses|ollama (default: inferred from keys)
|
||||
// ARTEX_LLM_MODEL = model id (default: per provider)
|
||||
// ARTEX_LLM_BASE_URL = endpoint (optional)
|
||||
// ARTEX_LLM_BASE_URL = endpoint (optional; ollama defaults to localhost:11434/v1)
|
||||
// ARTEX_LLM_PROXY = proxy URL (optional; http/https/socks5)
|
||||
// ANTHROPIC_API_KEY / OPENAI_API_KEY = credentials
|
||||
// ANTHROPIC_API_KEY / OPENAI_API_KEY = credentials (unused for ollama)
|
||||
func FromEnv() (Config, bool) {
|
||||
prov := os.Getenv("ARTEX_LLM_PROVIDER")
|
||||
anthKey := os.Getenv("ANTHROPIC_API_KEY")
|
||||
@@ -196,6 +239,16 @@ func FromEnv() (Config, bool) {
|
||||
if c.Model == "" {
|
||||
c.Model = "gpt-5"
|
||||
}
|
||||
case "ollama":
|
||||
c.Format = FormatOllama
|
||||
// Deliberately NOT inheriting OPENAI_API_KEY: a local Ollama ignores the
|
||||
// bearer token, so the only thing the inheritance could do is ship a real
|
||||
// cloud key to the endpoint. A gateway that does require a key can still
|
||||
// be configured as a saved profile with an explicit one.
|
||||
c.BaseURL = normalizeOllamaBaseURL(c.BaseURL)
|
||||
if c.Model == "" {
|
||||
c.Model = "llama3.2"
|
||||
}
|
||||
default:
|
||||
c.Format = llm.FormatAnthropic
|
||||
c.APIKey = anthKey
|
||||
@@ -203,7 +256,8 @@ func FromEnv() (Config, bool) {
|
||||
c.Model = "claude-opus-4-8"
|
||||
}
|
||||
}
|
||||
if c.APIKey == "" {
|
||||
// Ollama runs without a key: only the cloud formats gate on one.
|
||||
if c.APIKey == "" && c.NeedsAPIKey() {
|
||||
return Config{}, false
|
||||
}
|
||||
return c, true
|
||||
@@ -236,6 +290,16 @@ func ConfigFrom(provider, model, baseURL, apiKey, proxy string) Config {
|
||||
if c.Model == "" {
|
||||
c.Model = "gpt-5"
|
||||
}
|
||||
case "ollama":
|
||||
c.Format = FormatOllama
|
||||
// Ollama's OpenAI-compatible endpoint lives at /v1 and the provider
|
||||
// appends "/chat/completions". Never leave it empty: an empty BaseURL
|
||||
// would let the SDK fall back to OpenAI's public endpoint with this
|
||||
// profile's request.
|
||||
c.BaseURL = normalizeOllamaBaseURL(c.BaseURL)
|
||||
if c.Model == "" {
|
||||
c.Model = "llama3.2"
|
||||
}
|
||||
default:
|
||||
c.Format = llm.FormatAnthropic
|
||||
// provider appends "/v1/messages".
|
||||
@@ -257,13 +321,16 @@ func isFalsy(s string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// Provider returns the short provider name ("anthropic"/"openai").
|
||||
// Provider returns the short provider name
|
||||
// ("anthropic"/"openai"/"openai-responses"/"ollama").
|
||||
func (c Config) Provider() string {
|
||||
switch c.Format {
|
||||
case llm.FormatOpenAI:
|
||||
return "openai"
|
||||
case llm.FormatOpenAIResponses:
|
||||
return "openai-responses"
|
||||
case FormatOllama:
|
||||
return "ollama"
|
||||
}
|
||||
return "anthropic"
|
||||
}
|
||||
@@ -276,10 +343,27 @@ func (c Config) NewProvider() (llm.Provider, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
wireFormat := c.Format
|
||||
baseURL := c.BaseURL
|
||||
apiKey := c.APIKey
|
||||
// Ollama speaks OpenAI's Chat Completions on the wire. Bundle the keyless
|
||||
// defaults here so an Ollama config can never leak toward the SDK's env-
|
||||
// driven fallbacks: BaseURL to OpenAI's public endpoint, APIKey to
|
||||
// OPENAI_API_KEY. A local Ollama ignores the placeholder token; a gateway
|
||||
// that requires one carries a real key in the profile.
|
||||
if wireFormat == FormatOllama {
|
||||
wireFormat = llm.FormatOpenAI
|
||||
if strings.TrimSpace(baseURL) == "" {
|
||||
baseURL = DefaultOllamaBaseURL
|
||||
}
|
||||
if strings.TrimSpace(apiKey) == "" {
|
||||
apiKey = ollamaPlaceholderKey
|
||||
}
|
||||
}
|
||||
lc := llm.Config{
|
||||
Format: c.Format,
|
||||
BaseURL: c.BaseURL,
|
||||
APIKey: c.APIKey,
|
||||
Format: wireFormat,
|
||||
BaseURL: baseURL,
|
||||
APIKey: apiKey,
|
||||
Model: c.Model,
|
||||
HTTPClient: client,
|
||||
}
|
||||
|
||||
+6
-2
@@ -88,12 +88,16 @@ func run() int {
|
||||
}
|
||||
defer mgr.Close()
|
||||
|
||||
skillDir := config.SkillDir()
|
||||
skillDir, err := config.SkillDir()
|
||||
if err != nil {
|
||||
log.Printf("[config] %v", err)
|
||||
return 1
|
||||
}
|
||||
if abs, err := filepath.Abs(skillDir); err == nil {
|
||||
skillDir = abs
|
||||
}
|
||||
log.Printf("[config] skill directory: %s", skillDir)
|
||||
srv := server.New(ctx, mgr, skillDir, *dataDir, config.BaseDir())
|
||||
srv := server.New(ctx, mgr, skillDir, *dataDir, config.BaseDir(), server.WithSecretScrub())
|
||||
httpSrv := &http.Server{
|
||||
Addr: *addr,
|
||||
Handler: srv.Handler(),
|
||||
|
||||
@@ -7,6 +7,8 @@ import (
|
||||
"github.com/Autumn-27/artex/agent"
|
||||
)
|
||||
|
||||
// The shutdown cause must survive the parent context being cancelled first;
|
||||
// losing it would drop the diagnostic on every graceful stop.
|
||||
func TestShutdownContextPreservesNamedCause(t *testing.T) {
|
||||
signalCtx, signalCancel := context.WithCancel(context.Background())
|
||||
ctx, shutdown := shutdownContext(signalCtx)
|
||||
|
||||
+33
-14
@@ -4,7 +4,9 @@ package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -96,16 +98,23 @@ func Path() string {
|
||||
return candidates[0]
|
||||
}
|
||||
|
||||
// Load reads and parses the config file. A missing/unreadable file yields a zero
|
||||
// Config (so callers fall back to defaults) rather than an error.
|
||||
func Load() Config {
|
||||
// Load reads and parses the config file. A missing file yields a zero Config (so
|
||||
// callers fall back to defaults). A file that exists but cannot be read or parsed
|
||||
// is an error: ignoring it would run with defaults the operator did not choose.
|
||||
func Load() (Config, error) {
|
||||
var c Config
|
||||
b, err := os.ReadFile(Path())
|
||||
if err != nil {
|
||||
return c
|
||||
path := Path()
|
||||
b, err := os.ReadFile(path)
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return c, nil
|
||||
}
|
||||
_ = json.Unmarshal(b, &c)
|
||||
return c
|
||||
if err != nil {
|
||||
return c, fmt.Errorf("read config file %s: %w", path, err)
|
||||
}
|
||||
if err := json.Unmarshal(b, &c); err != nil {
|
||||
return Config{}, fmt.Errorf("parse config file %s: %w", path, err)
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// SkillDir returns the skill root directory with precedence:
|
||||
@@ -113,17 +122,23 @@ func Load() Config {
|
||||
// env ARTEX_SKILL_DIR > config file (skill_dir) > BaseDir()/skills
|
||||
//
|
||||
// The directory is created if it does not exist.
|
||||
func SkillDir() string {
|
||||
func SkillDir() (string, error) {
|
||||
var d string
|
||||
if v := strings.TrimSpace(os.Getenv("ARTEX_SKILL_DIR")); v != "" {
|
||||
d = v
|
||||
} else if v := strings.TrimSpace(Load().SkillDir); v != "" {
|
||||
d = v
|
||||
} else {
|
||||
d = filepath.Join(BaseDir(), "skills")
|
||||
c, err := Load()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if v := strings.TrimSpace(c.SkillDir); v != "" {
|
||||
d = v
|
||||
} else {
|
||||
d = filepath.Join(BaseDir(), "skills")
|
||||
}
|
||||
}
|
||||
_ = os.MkdirAll(d, 0o755)
|
||||
return d
|
||||
return d, nil
|
||||
}
|
||||
|
||||
// PostgresDSN resolves the connection string with precedence:
|
||||
@@ -138,7 +153,11 @@ func PostgresDSN() (dsn, source string, err error) {
|
||||
if v := strings.TrimSpace(os.Getenv("ARTEX_PG_DSN")); v != "" {
|
||||
return v, "environment variable ARTEX_PG_DSN", nil
|
||||
}
|
||||
db := Load().Database
|
||||
c, err := Load()
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
db := c.Database
|
||||
if d := strings.TrimSpace(db.DSN); d != "" {
|
||||
return d, "config file " + Path() + " (database.dsn)", nil
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package config
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -40,3 +41,24 @@ func TestPostgresDSNPrecedence(t *testing.T) {
|
||||
t.Fatalf("file dsn verbatim, got %q err %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
// A config file that exists but is not valid JSON must fail loudly, not run with
|
||||
// defaults (or report "no database config") as if the file were absent.
|
||||
func TestMalformedConfigIsAnError(t *testing.T) {
|
||||
cfgPath := filepath.Join(t.TempDir(), "config.json")
|
||||
// What install.sh used to write for a password containing a double quote.
|
||||
os.WriteFile(cfgPath, []byte(`{"database":{"host":"h","password":"a"b","dbname":"d"},"skill_dir":"/s"}`), 0o644)
|
||||
t.Setenv("ARTEX_CONFIG", cfgPath)
|
||||
t.Setenv("ARTEX_PG_DSN", "")
|
||||
t.Setenv("ARTEX_SKILL_DIR", "")
|
||||
|
||||
if _, _, err := PostgresDSN(); err == nil || !strings.Contains(err.Error(), "parse config file") {
|
||||
t.Fatalf("PostgresDSN: want a parse error, got %v", err)
|
||||
}
|
||||
if d, err := SkillDir(); err == nil {
|
||||
t.Fatalf("SkillDir: want a parse error, got dir %q", d)
|
||||
}
|
||||
if _, err := Load(); err == nil {
|
||||
t.Fatal("Load: want a parse error")
|
||||
}
|
||||
}
|
||||
|
||||
+5
-4
@@ -143,13 +143,14 @@ func (d *DB) ProfileByID(id int64) (*LLMProfile, error) {
|
||||
}
|
||||
|
||||
// PoolProfiles returns the failover chain in run order, api keys included: the
|
||||
// active profile first, then every other keyed profile that isn't excluded, by
|
||||
// active profile first, then every other usable profile that isn't excluded, by
|
||||
// priority DESC (id ASC to stay stable). Profiles without an api key can't serve
|
||||
// a request, so they never enter the chain. The ordering IS the policy — callers
|
||||
// walk the slice front to back.
|
||||
// a request — except Ollama ('ollama'), which is a keyless local server — so
|
||||
// they never enter the chain. The ordering IS the policy — callers walk the
|
||||
// slice front to back.
|
||||
func (d *DB) PoolProfiles() ([]*LLMProfile, error) {
|
||||
rows, err := d.Query(`SELECT ` + profileColsKey + ` FROM llm_profiles
|
||||
WHERE COALESCE(api_key,'') <> '' AND (is_default OR NOT pool_exclude)
|
||||
WHERE (COALESCE(api_key,'') <> '' OR format='ollama') AND (is_default OR NOT pool_exclude)
|
||||
ORDER BY is_default DESC, priority DESC, id ASC`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
+5
-4
@@ -363,7 +363,7 @@ CREATE TABLE IF NOT EXISTS settings (
|
||||
CREATE TABLE IF NOT EXISTS llm_profiles (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
format TEXT NOT NULL CHECK (format IN ('openai','anthropic','openai-responses')),
|
||||
format TEXT NOT NULL CHECK (format IN ('openai','anthropic','openai-responses','ollama')),
|
||||
base_url TEXT,
|
||||
proxy TEXT,
|
||||
model TEXT NOT NULL,
|
||||
@@ -416,11 +416,12 @@ ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS priority INTEGER NOT NULL
|
||||
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS pool_exclude BOOLEAN NOT NULL DEFAULT false;
|
||||
-- Streaming toggle; backfills old databases. Default true = keep existing streaming behavior; old configs upgrade transparently.
|
||||
ALTER TABLE llm_profiles ADD COLUMN IF NOT EXISTS streaming BOOLEAN NOT NULL DEFAULT true;
|
||||
-- Relax the format constraint to allow openai-responses (OpenAI Responses API); backfills old databases.
|
||||
-- Runs at every startup, idempotent: drop the old CHECK first, then create the new three-value CHECK.
|
||||
-- Relax the format constraint to allow openai-responses (OpenAI Responses API) and
|
||||
-- ollama (local Ollama server, keyless OpenAI-compatible endpoint); backfills old databases.
|
||||
-- Runs at every startup, idempotent: drop the old CHECK first, then create the new four-value CHECK.
|
||||
ALTER TABLE llm_profiles DROP CONSTRAINT IF EXISTS llm_profiles_format_check;
|
||||
ALTER TABLE llm_profiles ADD CONSTRAINT llm_profiles_format_check
|
||||
CHECK (format IN ('openai','anthropic','openai-responses'));
|
||||
CHECK (format IN ('openai','anthropic','openai-responses','ollama'));
|
||||
|
||||
-- Output cap and its field name; backfills old databases. Defaults 0 / '' = send no cap and keep the max_tokens field name;
|
||||
-- old configs behave exactly as before.
|
||||
|
||||
+9
-2
@@ -15,7 +15,14 @@ services:
|
||||
restart: unless-stopped
|
||||
|
||||
artex:
|
||||
image: autumn27/artex:${ARTEX_TAG:-latest}
|
||||
# Built from this checkout. The previous reference was a third-party
|
||||
# namespace whose repository is gone: whoever controls that name could
|
||||
# publish the tag this file would pull.
|
||||
build:
|
||||
context: .
|
||||
args:
|
||||
ARTEX_VERSION: ${ARTEX_VERSION:-dev}
|
||||
image: artex:local
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
@@ -28,7 +35,7 @@ services:
|
||||
ARTEX_LLM_BASE_URL: ${ARTEX_LLM_BASE_URL:-}
|
||||
ARTEX_LLM_PROXY: ${ARTEX_LLM_PROXY:-}
|
||||
ports:
|
||||
- "8787:8787"
|
||||
- "${ARTEX_BIND_ADDR:-127.0.0.1}:8787:8787"
|
||||
- "127.0.0.1:8788:8788"
|
||||
volumes:
|
||||
# Bind the data directory (jwt.key, SQLite, etc.) to data/ in the project root,
|
||||
|
||||
+24
-11
@@ -9,6 +9,16 @@ warn(){ printf '\033[33m[!]\033[0m %s\n' "$*"; }
|
||||
die(){ printf '\033[31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
|
||||
ask(){ local p="$1" d="${2:-}" a; read -rp "$p${d:+ [$d]}: " a; echo "${a:-$d}"; }
|
||||
rand(){ head -c 18 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | head -c 24; }
|
||||
# json_str prints $1 as a JSON string literal: backslash, double quote and the
|
||||
# control characters read -r can return are escaped.
|
||||
json_str(){
|
||||
local s="$1"
|
||||
s="${s//\\/\\\\}"; s="${s//\"/\\\"}"
|
||||
s="${s//$'\t'/\\t}"; s="${s//$'\r'/\\r}"
|
||||
printf '"%s"' "$s"
|
||||
}
|
||||
# sed_repl escapes $1 for the replacement side of s|...|...|: \, | and &.
|
||||
sed_repl(){ printf '%s' "$1" | sed 's/[\\|&]/\\&/g'; }
|
||||
|
||||
# ── docker environment check / auto-install ───────────────────
|
||||
ensure_docker(){
|
||||
@@ -38,16 +48,18 @@ install_docker(){
|
||||
local pw key
|
||||
pw="$(ask 'Postgres password (press Enter to generate randomly)' "$(rand)")"
|
||||
key="$(ask 'ANTHROPIC_API_KEY (may be empty, configure later in the UI)' '')"
|
||||
sed -i.bak "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=${pw}|" .env
|
||||
sed -i.bak "s|^ANTHROPIC_API_KEY=.*|ANTHROPIC_API_KEY=${key}|" .env
|
||||
sed -i.bak "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$(sed_repl "$pw")|" .env
|
||||
sed -i.bak "s|^ANTHROPIC_API_KEY=.*|ANTHROPIC_API_KEY=$(sed_repl "$key")|" .env
|
||||
rm -f .env.bak
|
||||
ok ".env generated (POSTGRES_PASSWORD set)"
|
||||
else
|
||||
info "reusing existing .env"
|
||||
fi
|
||||
info "pulling images and starting..."
|
||||
docker compose pull || true
|
||||
docker compose up -d
|
||||
info "building the image from this checkout and starting..."
|
||||
if [ ! -f "dist/amd64/artex" ] && [ ! -f "dist/arm64/artex" ]; then
|
||||
warn "no prebuilt binary under dist/ - build it first (frontend, then go build -tags embedui); see the Dockerfile header"
|
||||
fi
|
||||
docker compose up -d --build
|
||||
ok "started → http://localhost:8787"
|
||||
info "view logs: docker compose logs -f artex"
|
||||
}
|
||||
@@ -74,16 +86,17 @@ install_local(){
|
||||
DB_SSL="$(ask 'sslmode (disable/require)' disable)" ;;
|
||||
esac
|
||||
|
||||
# generate config.json
|
||||
# generate config.json (string values JSON-escaped: a password may contain " or \)
|
||||
case "$DB_PORT" in (*[!0-9]*|'') die "port must be a number: $DB_PORT" ;; esac
|
||||
cat > config.json <<JSON
|
||||
{
|
||||
"database": {
|
||||
"host": "${DB_HOST}",
|
||||
"host": $(json_str "$DB_HOST"),
|
||||
"port": ${DB_PORT},
|
||||
"user": "${DB_USER}",
|
||||
"password": "${DB_PASS}",
|
||||
"dbname": "${DB_NAME}",
|
||||
"sslmode": "${DB_SSL}"
|
||||
"user": $(json_str "$DB_USER"),
|
||||
"password": $(json_str "$DB_PASS"),
|
||||
"dbname": $(json_str "$DB_NAME"),
|
||||
"sslmode": $(json_str "$DB_SSL")
|
||||
}
|
||||
}
|
||||
JSON
|
||||
|
||||
+39
-1
@@ -2,6 +2,8 @@ package server
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/subtle"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"log"
|
||||
"math/big"
|
||||
@@ -154,7 +156,34 @@ func (s *Server) authStatus(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, 200, map[string]any{"initialized": hash != ""})
|
||||
}
|
||||
|
||||
// initSetupToken generates the one-time setup token when no admin password is set
|
||||
// and writes it to the server log. Without it, anyone who can reach the port
|
||||
// before the operator could claim the instance through POST /api/auth/init.
|
||||
func (s *Server) initSetupToken() {
|
||||
if s.m == nil || s.m.pg == nil {
|
||||
return
|
||||
}
|
||||
if hash, _, err := s.m.pg.GetSetting(authPassKey); err == nil && hash != "" {
|
||||
return
|
||||
}
|
||||
buf := make([]byte, 16)
|
||||
if _, err := rand.Read(buf); err != nil {
|
||||
log.Fatalf("[auth] generate setup token: %v", err)
|
||||
}
|
||||
s.setupToken = hex.EncodeToString(buf)
|
||||
log.Printf("[auth] no admin password is set. Setup token for the /setup page: %s", s.setupToken)
|
||||
}
|
||||
|
||||
// checkSetupToken compares got with the active setup token in constant time.
|
||||
// It fails when no token is active.
|
||||
func (s *Server) checkSetupToken(got string) bool {
|
||||
s.setupMu.Lock()
|
||||
defer s.setupMu.Unlock()
|
||||
return s.setupToken != "" && subtle.ConstantTimeCompare([]byte(got), []byte(s.setupToken)) == 1
|
||||
}
|
||||
|
||||
// POST /api/auth/init — sets the password for the first time; rejected if already set.
|
||||
// Requires the setup token from the server log.
|
||||
func (s *Server) authInit(w http.ResponseWriter, r *http.Request) {
|
||||
pg := s.pg(w)
|
||||
if pg == nil {
|
||||
@@ -170,12 +199,18 @@ func (s *Server) authInit(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Password string `json:"password"`
|
||||
Password string `json:"password"`
|
||||
SetupToken string `json:"setup_token"`
|
||||
}
|
||||
if err := decode(r, &req); err != nil || req.Password == "" {
|
||||
writeErr(w, 400, "Password cannot be empty")
|
||||
return
|
||||
}
|
||||
// 403, not 401: the frontend treats any 401 as an expired session and redirects to /login.
|
||||
if !s.checkSetupToken(req.SetupToken) {
|
||||
writeErr(w, 403, "Invalid setup token. The server log prints it at startup while no password is set")
|
||||
return
|
||||
}
|
||||
if msg := validatePassword(req.Password); msg != "" {
|
||||
writeErr(w, 400, msg)
|
||||
return
|
||||
@@ -197,6 +232,9 @@ func (s *Server) authInit(w http.ResponseWriter, r *http.Request) {
|
||||
writeErr(w, 403, "Password is already set")
|
||||
return
|
||||
}
|
||||
s.setupMu.Lock()
|
||||
s.setupToken = ""
|
||||
s.setupMu.Unlock()
|
||||
tok, err := signJWT(s.jwtKey)
|
||||
if err != nil {
|
||||
writeErr(w, 500, "token generation failed")
|
||||
|
||||
@@ -62,6 +62,62 @@ func TestAuthInitFailsClosedWhenDataSourceUnavailable(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// While no password is set, authInit must reject a missing or wrong setup token
|
||||
// and accept the one generated at startup, once.
|
||||
func TestAuthInitRequiresSetupToken(t *testing.T) {
|
||||
m, err := NewManager(t.TempDir(), "")
|
||||
if err != nil {
|
||||
t.Skipf("postgres unavailable (%v)", err)
|
||||
}
|
||||
defer m.Close()
|
||||
clearPassword := func() {
|
||||
if _, err := m.pg.Exec(`DELETE FROM settings WHERE key=$1`, authPassKey); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
clearPassword()
|
||||
defer clearPassword()
|
||||
|
||||
s := &Server{m: m, jwtKey: []byte("0123456789abcdef0123456789abcdef")}
|
||||
s.initSetupToken()
|
||||
setupToken := s.setupToken
|
||||
if setupToken == "" {
|
||||
t.Fatal("no setup token generated while no password is set")
|
||||
}
|
||||
post := func(body string) *httptest.ResponseRecorder {
|
||||
w := httptest.NewRecorder()
|
||||
s.authInit(w, httptest.NewRequest("POST", "/api/auth/init", strings.NewReader(body)))
|
||||
return w
|
||||
}
|
||||
|
||||
for name, body := range map[string]string{
|
||||
"missing": `{"password":"correct horse battery"}`,
|
||||
"empty": `{"password":"correct horse battery","setup_token":""}`,
|
||||
"wrong": `{"password":"correct horse battery","setup_token":"0000"}`,
|
||||
} {
|
||||
if w := post(body); w.Code != 403 || strings.Contains(w.Body.String(), `"token"`) {
|
||||
t.Fatalf("%s setup token: status=%d want 403 without a token; body=%s", name, w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
if hash, _, err := m.pg.GetSetting(authPassKey); err != nil || hash != "" {
|
||||
t.Fatalf("a rejected request changed the password state: hash set=%v err=%v", hash != "", err)
|
||||
}
|
||||
|
||||
ok := `{"password":"correct horse battery","setup_token":"` + setupToken + `"}`
|
||||
if w := post(ok); w.Code != 200 || !strings.Contains(w.Body.String(), `"token"`) {
|
||||
t.Fatalf("right setup token: status=%d want 200 with a token; body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
if hash, _, err := m.pg.GetSetting(authPassKey); err != nil || hash == "" {
|
||||
t.Fatalf("password not stored: err=%v", err)
|
||||
}
|
||||
|
||||
// The token is spent: even with the password row removed, it is refused.
|
||||
clearPassword()
|
||||
if w := post(ok); w.Code != 403 {
|
||||
t.Fatalf("reused setup token: status=%d want 403; body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatePassword(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name, pw string
|
||||
|
||||
@@ -103,7 +103,7 @@ func (s *Server) pgCreateConversation(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
if req.LLMProfileID != nil {
|
||||
if _, ok := s.loadProfileConfig(*req.LLMProfileID); !ok {
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or has no API Key set")
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or is not usable (no API key for the key-requiring formats)")
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -136,7 +136,7 @@ func (s *Server) pgUpdateConversation(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
if req.LLMProfileID != nil {
|
||||
if _, ok := s.loadProfileConfig(*req.LLMProfileID); !ok {
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or has no API Key set")
|
||||
writeErr(w, 400, "The specified LLM profile does not exist or is not usable (no API key for the key-requiring formats)")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
package server
|
||||
|
||||
import "os"
|
||||
|
||||
// secretEnv lists the credentials read from the environment only at startup:
|
||||
// ARTEX_PG_DSN by NewManager, the API keys by agent.FromEnv inside New.
|
||||
var secretEnv = []string{"ARTEX_PG_DSN", "ANTHROPIC_API_KEY", "OPENAI_API_KEY"}
|
||||
|
||||
// ScrubSecretEnv removes secretEnv from the process environment once startup has
|
||||
// read it. Child processes (agent shells, custom tools, MCP stdio servers) copy
|
||||
// os.Environ() when they are spawned, so they no longer inherit these values.
|
||||
//
|
||||
// It runs inside New, after both readers and before the MCP discovery goroutine
|
||||
// and the task resume loop: both can spawn children, and resuming a task starts
|
||||
// workers that run shells. Scrubbing in main after New returned left that window
|
||||
// open.
|
||||
func ScrubSecretEnv() {
|
||||
for _, k := range secretEnv {
|
||||
_ = os.Unsetenv(k)
|
||||
}
|
||||
}
|
||||
|
||||
// Option configures New. Options keep the test call sites unchanged: only the
|
||||
// binary opts into scrubbing, because tests create several servers in one
|
||||
// process and still need ARTEX_PG_DSN on later calls.
|
||||
type Option func(*Server)
|
||||
|
||||
// WithSecretScrub makes New drop the startup secrets after reading them, before
|
||||
// the MCP discovery goroutine and the task resume loop can spawn children.
|
||||
func WithSecretScrub() Option { return func(s *Server) { s.scrubSecrets = true } }
|
||||
@@ -0,0 +1,36 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A child process spawned after startup must not see the startup secrets, while
|
||||
// other variables still pass through.
|
||||
func TestScrubSecretEnvHidesSecretsFromChildren(t *testing.T) {
|
||||
sh, err := exec.LookPath("sh")
|
||||
if err != nil {
|
||||
t.Skip("no sh in PATH")
|
||||
}
|
||||
for _, k := range secretEnv {
|
||||
t.Setenv(k, "leak-"+k)
|
||||
}
|
||||
t.Setenv("ARTEX_SCRUB_KEEP", "kept")
|
||||
|
||||
ScrubSecretEnv()
|
||||
|
||||
out, err := exec.Command(sh, "-c", "env").Output()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env := "\n" + string(out)
|
||||
for _, k := range secretEnv {
|
||||
if strings.Contains(env, "\n"+k+"=") {
|
||||
t.Errorf("%s still reaches child processes", k)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(env, "\nARTEX_SCRUB_KEEP=kept\n") {
|
||||
t.Errorf("non-secret variable did not reach the child process")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,272 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/artex/agent"
|
||||
"github.com/Autumn-27/artex/db"
|
||||
"github.com/Autumn-27/norma/llm"
|
||||
)
|
||||
|
||||
// ConfigFrom("ollama", ...) must normalize to the keyless sentinel with our own
|
||||
// defaults, never the SDK's OpenAI-public-endpoint fallback.
|
||||
func TestOllamaConfigFromKeylessDefaults(t *testing.T) {
|
||||
c := agent.ConfigFrom("ollama", "", "", "", "")
|
||||
if c.Format != agent.FormatOllama {
|
||||
t.Fatalf("format=%v, want FormatOllama", c.Format)
|
||||
}
|
||||
if c.Provider() != "ollama" {
|
||||
t.Fatalf("Provider()=%q", c.Provider())
|
||||
}
|
||||
if c.BaseURL != agent.DefaultOllamaBaseURL {
|
||||
t.Fatalf("BaseURL=%q, want default %q", c.BaseURL, agent.DefaultOllamaBaseURL)
|
||||
}
|
||||
if c.Model == "" {
|
||||
t.Fatal("model must default to a non-empty value")
|
||||
}
|
||||
if c.NeedsAPIKey() {
|
||||
t.Fatal("ollama must not require an API key")
|
||||
}
|
||||
// A full endpoint URL is tolerated (the provider appends /chat/completions).
|
||||
e := agent.ConfigFrom("ollama", "llama3.2", "http://box:11434/v1/chat/completions", "", "")
|
||||
if e.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("full endpoint URL not normalized: %q", e.BaseURL)
|
||||
}
|
||||
// A custom key is preserved (e.g. an authenticated Ollama gateway).
|
||||
k := agent.ConfigFrom("ollama", "m", "http://box:11434", "ollama-secret", "")
|
||||
if k.APIKey != "ollama-secret" {
|
||||
t.Fatalf("custom key lost: %q", k.APIKey)
|
||||
}
|
||||
// The natural Ollama URL has no /v1, but the provider appends
|
||||
// /chat/completions, so the base must be normalized to the /v1 root.
|
||||
n := agent.ConfigFrom("ollama", "m", "http://box:11434", "", "")
|
||||
if n.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("bare host URL not normalized: %q", n.BaseURL)
|
||||
}
|
||||
}
|
||||
|
||||
// The provider built from an Ollama config speaks OpenAI Chat Completions on
|
||||
// the wire (the SDK appends /chat/completions) with the placeholder Bearer (no
|
||||
// real key was configured, no OPENAI_API_KEY env leak).
|
||||
func TestOllamaProviderWiresOpenAIProtocol(t *testing.T) {
|
||||
var gotPath, gotAuth, gotModel atomic.Value
|
||||
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotPath.Store(r.URL.Path)
|
||||
gotAuth.Store(r.Header.Get("Authorization"))
|
||||
var body struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
gotModel.Store(body.Model)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"id":"1","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`))
|
||||
}))
|
||||
t.Cleanup(fake.Close)
|
||||
|
||||
cfg := agent.ConfigFrom("ollama", "llama3.2", fake.URL, "", "")
|
||||
cfg.Stream = false // non-streaming: the fake answers one plain JSON body
|
||||
prov, err := cfg.NewProvider()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
req := llm.CompletionRequest{
|
||||
System: []string{"You are a connection test. Output OK."},
|
||||
Messages: []llm.Message{llm.UserText("ping")},
|
||||
}
|
||||
if _, _, _, err = prov.Complete(ctx, req); err != nil {
|
||||
t.Fatalf("Complete: %v", err)
|
||||
}
|
||||
// ConfigFrom normalizes the fake server URL (no /v1) onto the OpenAI root,
|
||||
// so the request must land on /v1/chat/completions. A path of /chat/completions
|
||||
// would mean the base URL leaked through unnormalized (or fell back to OpenAI).
|
||||
if p, _ := gotPath.Load().(string); p != "/v1/chat/completions" {
|
||||
t.Fatalf("path=%q, want /v1/chat/completions", p)
|
||||
}
|
||||
if a, _ := gotAuth.Load().(string); a != "Bearer ollama" {
|
||||
t.Fatalf("Authorization=%q, want placeholder Bearer ollama", a)
|
||||
}
|
||||
if m, _ := gotModel.Load().(string); m != "llama3.2" {
|
||||
t.Fatalf("model on wire=%q", m)
|
||||
}
|
||||
}
|
||||
|
||||
// A keyless Ollama profile must be usable end to end: stored round-trips,
|
||||
// loadProfileConfig accepts it, the task chain validates it, the failover pool
|
||||
// includes it, and the connection test completes against a fake Ollama.
|
||||
func TestOllamaKeylessProfileLifecycle(t *testing.T) {
|
||||
m, err := NewManager(t.TempDir(), "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { m.Close() })
|
||||
if m.pg == nil {
|
||||
t.Skip("no postgres for this run")
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
|
||||
|
||||
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"id":"1","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`))
|
||||
}))
|
||||
t.Cleanup(fake.Close)
|
||||
|
||||
pid, err := m.pg.SaveProfile(&db.LLMProfile{
|
||||
Name: "ollama-local", Format: "ollama", Model: "llama3.2",
|
||||
BaseURL: fake.URL, APIKey: "", IsDefault: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _, _ = m.pg.Exec("DELETE FROM llm_profiles WHERE id=$1", pid) })
|
||||
|
||||
stored, err := m.pg.ProfileByID(pid)
|
||||
if err != nil || stored == nil {
|
||||
t.Fatalf("ProfileByID: %v %v", stored, err)
|
||||
}
|
||||
|
||||
if _, ok := s.loadProfileConfig(pid); !ok {
|
||||
t.Fatal("keyless ollama profile must load")
|
||||
}
|
||||
if err := s.validateTaskProfileIDs([]int64{pid}); err != nil {
|
||||
t.Fatalf("task chain must accept keyless ollama: %v", err)
|
||||
}
|
||||
pool, err := m.pg.PoolProfiles()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found := false
|
||||
for _, p := range pool {
|
||||
if p.ID == pid {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("keyless ollama profile must join the failover pool")
|
||||
}
|
||||
|
||||
// Connection test: no api_key in the form either.
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"profile_id": pid, "provider": "ollama", "model": "llama3.2",
|
||||
"base_url": fake.URL, "api_key": "", "streaming": false,
|
||||
})
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/llm/test", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
s.testLLM(rec, req)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("handler status %d", rec.Code)
|
||||
}
|
||||
var resp struct {
|
||||
OK bool `json:"ok"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
|
||||
if !resp.OK {
|
||||
t.Fatalf("connection test on keyless ollama failed: %s", resp.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// Model listing for an Ollama server works without an API key and tolerates a
|
||||
// base URL with or without the /v1 suffix.
|
||||
func TestOllamaListModels(t *testing.T) {
|
||||
var calls atomic.Value // []string paths
|
||||
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
paths, _ := calls.Load().([]string)
|
||||
calls.Store(append(paths, r.URL.Path))
|
||||
_, _ = w.Write([]byte(`{"object":"list","data":[{"id":"llama3.2"},{"id":"qwen2.5:14b"}]}`))
|
||||
}))
|
||||
t.Cleanup(fake.Close)
|
||||
|
||||
m, err := NewManager(t.TempDir(), "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { m.Close() })
|
||||
if m.pg == nil {
|
||||
t.Skip("no postgres for this run")
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
s := New(ctx, m, t.TempDir(), t.TempDir(), t.TempDir())
|
||||
|
||||
call := func(baseURL string) (bool, []string) {
|
||||
body, _ := json.Marshal(map[string]any{"provider": "ollama", "base_url": baseURL, "api_key": ""})
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/llm/models", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
s.pgListModels(rec, req)
|
||||
var resp struct {
|
||||
OK bool `json:"ok"`
|
||||
Models []string `json:"models"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &resp)
|
||||
return resp.OK, resp.Models
|
||||
}
|
||||
// Base WITH /v1.
|
||||
ok, models := call(fake.URL + "/v1")
|
||||
if !ok || len(models) != 2 {
|
||||
t.Fatalf("listing with /v1 base failed: ok=%v models=%v", ok, models)
|
||||
}
|
||||
// Base WITHOUT /v1 must still be normalized onto the OpenAI-compatible path
|
||||
// (/v1/models); both calls hit the same endpoint.
|
||||
ok2, models2 := call(fake.URL)
|
||||
if !ok2 || len(models2) != 2 {
|
||||
t.Fatalf("listing without /v1 base failed: ok=%v models=%v", ok2, models2)
|
||||
}
|
||||
if paths, _ := calls.Load().([]string); len(paths) != 2 || paths[0] != "/v1/models" || paths[1] != "/v1/models" {
|
||||
t.Fatalf("both calls must hit /v1/models, got %v", paths)
|
||||
}
|
||||
}
|
||||
|
||||
// The environment path must normalize the base URL too: ARTEX_LLM_BASE_URL is
|
||||
// the natural way to configure a LAN Ollama server, and it used to send chat to
|
||||
// /chat/completions instead of /v1/chat/completions.
|
||||
func TestOllamaFromEnvNormalizesBaseURL(t *testing.T) {
|
||||
t.Setenv("ARTEX_LLM_PROVIDER", "ollama")
|
||||
t.Setenv("OPENAI_API_KEY", "")
|
||||
t.Setenv("ARTEX_LLM_BASE_URL", "http://box:11434")
|
||||
c, ok := agent.FromEnv()
|
||||
if !ok {
|
||||
t.Fatal("ollama env config must be usable")
|
||||
}
|
||||
if c.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("env base URL not normalized: %q", c.BaseURL)
|
||||
}
|
||||
t.Setenv("ARTEX_LLM_BASE_URL", "http://box:11434/v1/chat/completions")
|
||||
c, ok = agent.FromEnv()
|
||||
if !ok {
|
||||
t.Fatal("ollama env config must be usable")
|
||||
}
|
||||
if c.BaseURL != "http://box:11434/v1" {
|
||||
t.Fatalf("full endpoint URL not normalized: %q", c.BaseURL)
|
||||
}
|
||||
}
|
||||
|
||||
// An Ollama config resolved from the environment must not inherit OPENAI_API_KEY:
|
||||
// Ollama ignores the bearer token, so inheritance would only ship the cloud key
|
||||
// to the local or LAN endpoint.
|
||||
func TestOllamaFromEnvDoesNotInheritOpenAIKey(t *testing.T) {
|
||||
t.Setenv("ARTEX_LLM_PROVIDER", "ollama")
|
||||
t.Setenv("OPENAI_API_KEY", "sk-cloud-secret")
|
||||
t.Setenv("ARTEX_LLM_BASE_URL", "")
|
||||
c, ok := agent.FromEnv()
|
||||
if !ok {
|
||||
t.Fatal("ollama env config must be usable without a key")
|
||||
}
|
||||
if c.APIKey != "" {
|
||||
t.Fatalf("ollama inherited a cloud key: %q", c.APIKey)
|
||||
}
|
||||
}
|
||||
@@ -293,7 +293,7 @@ func (s *Server) toolSpawnTask() actool.CoreTool {
|
||||
var pin *int64
|
||||
if id := parseProfileID(a.LLMProfileID); id > 0 {
|
||||
if _, ok := s.loadProfileConfig(id); !ok {
|
||||
return actool.Errorf(fmt.Sprintf("LLM profile #%d not found or has no API key set", id)), nil
|
||||
return actool.Errorf(fmt.Sprintf("LLM profile #%d not found or not usable (no API key for the key-requiring formats)", id)), nil
|
||||
}
|
||||
pin = &id
|
||||
} else if a.ParentRef != "" {
|
||||
|
||||
+47
-29
@@ -46,6 +46,11 @@ type Server struct {
|
||||
skillDir string // root directory for skill subdirectories
|
||||
jwtKey []byte // HS256 signing key loaded from / generated into dataDir/jwt.key
|
||||
|
||||
scrubSecrets bool // drop ARTEX_PG_DSN and the API keys from the environment once read (binary only)
|
||||
|
||||
setupMu sync.Mutex
|
||||
setupToken string // one-time token POST /api/auth/init requires; empty when none is active
|
||||
|
||||
// concMu serializes concurrency-cap decisions (admission + reconcile) so a
|
||||
// scheduler tick and an HTTP settings change / task creation can't both count
|
||||
// free slots off the same snapshot and over-promote past the limit.
|
||||
@@ -140,7 +145,7 @@ type triggeredRun struct {
|
||||
mergeable bool // true for finding/goal event triggers (merge by taskID)
|
||||
}
|
||||
|
||||
func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDir string) *Server {
|
||||
func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDir string, opts ...Option) *Server {
|
||||
key, err := loadOrCreateJWTKey(keyDir, dataDir)
|
||||
if err != nil {
|
||||
log.Fatalf("[auth] JWT key: %v", err)
|
||||
@@ -151,6 +156,7 @@ func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDi
|
||||
profChatAgents: map[int64]*agent.ChatAgent{},
|
||||
provByProfile: map[int64]*provEntry{}, llmHealth: newLLMHealthRegistry(m.pg),
|
||||
taskAgents: map[string]*taskAgentBundle{}, archiveWake: make(chan struct{}, 1)}
|
||||
s.initSetupToken()
|
||||
s.initSideQuestions()
|
||||
// Circuit-breaker thresholds/cooldowns are hot parameters on the failure path; push the global retry policy to the Registry once at startup,
|
||||
// and again on every policy save (saveLLMRetryPolicy).
|
||||
@@ -232,9 +238,6 @@ func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDi
|
||||
// Vulnerability IM push delivery engine. Sits beside the Scheduler but independent: push latency demands (3s)
|
||||
// differ from the triggers' business cadence, and neither's failures drag the other down — a stuck push must not affect agent triggers.
|
||||
go newNotifier(s).Run(s.ctx)
|
||||
// Fill the tool cache for any enabled MCP that has none yet (notably the
|
||||
// seeded browser MCP on first run). Async so it never blocks startup.
|
||||
go s.discoverEmptyMCPsOnStartup()
|
||||
logSink.SetDB(ctx, m.pg) // restore last 100 log rows and enable async persistence
|
||||
}
|
||||
// precedence: persisted DB config > env.
|
||||
@@ -253,6 +256,16 @@ func New(ctx context.Context, m *Manager, skillDir string, dataDir string, keyDi
|
||||
} else {
|
||||
log.Printf("[engine] no LLM provider configured — engine idle until set via /api/llm or env")
|
||||
}
|
||||
// Both env readers have now run (NewManager for the DSN, FromEnv above for the
|
||||
// keys). Drop the secrets before anything below can spawn a child process:
|
||||
// restoring runtimes resumes workers that run shells, and MCP discovery starts
|
||||
// stdio servers.
|
||||
if s.scrubSecrets {
|
||||
ScrubSecretEnv()
|
||||
}
|
||||
// Fill the tool cache for any enabled MCP that has none yet (notably the
|
||||
// seeded browser MCP on first run). Async so it never blocks startup.
|
||||
go s.discoverEmptyMCPsOnStartup()
|
||||
s.restoreTaskRuntimes()
|
||||
go s.reconcileConcurrency()
|
||||
s.startTaskArchiveWorker()
|
||||
@@ -335,7 +348,8 @@ func (s *Server) loadLLMConfig() (agent.Config, bool) {
|
||||
cfg.MaxTokens, cfg.MaxTokensField = p.MaxTokens, p.MaxTokensField
|
||||
cfg.SessionHeaderKey = p.SessionHeaderKey
|
||||
s.applyProfileRetry(&cfg, p)
|
||||
if cfg.APIKey == "" {
|
||||
// Ollama profiles run keyless (local server); the other formats need a key.
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
return cfg, false
|
||||
}
|
||||
s.cfgMu.Lock()
|
||||
@@ -491,7 +505,8 @@ func (s *Server) applyLLM(cfg agent.Config) error {
|
||||
}
|
||||
|
||||
// loadProfileConfig builds an agent.Config from a specific profile id (with its key).
|
||||
// ok=false when the profile is missing or has no api key.
|
||||
// ok=false when the profile is missing or unusable (no api key for the key-requiring
|
||||
// formats).
|
||||
func (s *Server) loadProfileConfig(id int64) (agent.Config, bool) {
|
||||
p, err := s.m.pg.ProfileByID(id)
|
||||
if err != nil || p == nil {
|
||||
@@ -506,7 +521,7 @@ func (s *Server) loadProfileConfig(id int64) (agent.Config, bool) {
|
||||
cfg.MaxTokens, cfg.MaxTokensField = p.MaxTokens, p.MaxTokensField
|
||||
cfg.SessionHeaderKey = p.SessionHeaderKey
|
||||
s.applyProfileRetry(&cfg, p)
|
||||
if cfg.APIKey == "" {
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
return cfg, false
|
||||
}
|
||||
return cfg, true
|
||||
@@ -1343,12 +1358,13 @@ func (s *Server) setLLM(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
cfg.ContextWindowK = k
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
// Keyless formats (Ollama) need no key; the others keep/require one.
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
s.cfgMu.Lock()
|
||||
cfg.APIKey = s.llmCfg.APIKey // keep existing key if not re-entered
|
||||
s.cfgMu.Unlock()
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
if cfg.APIKey == "" && cfg.NeedsAPIKey() {
|
||||
writeErr(w, 400, "api_key required")
|
||||
return
|
||||
}
|
||||
@@ -1441,28 +1457,30 @@ func (s *Server) testLLM(w http.ResponseWriter, r *http.Request) {
|
||||
// see TestConnection). Endpoints like opencode zen that mandate x-opencode-session return
|
||||
// 400 without it, so the test path must send it too — otherwise "chat works, test 400s".
|
||||
cfg.SessionHeaderKey = req.SessionHeaderKey
|
||||
// API key resolution precedence: form input > the named profile's stored key > the global config's key.
|
||||
// A stored profile's key is never sent back to the browser, so the form is empty when testing a stored profile and the key comes from the DB.
|
||||
// Same for the session header name: when the form omits it, the stored profile's value is the fallback.
|
||||
if stored != nil && (cfg.APIKey == "" || cfg.SessionHeaderKey == "") {
|
||||
// Trim like ConfigFrom does for form values: a key stored with
|
||||
// surrounding whitespace runs fine in conversations (the runtime
|
||||
// path re-trims) but would be sent raw here and fail auth.
|
||||
if cfg.APIKey == "" {
|
||||
// The custom session header name follows that profile regardless of format:
|
||||
// when the form omits it, the stored profile's value is the fallback.
|
||||
if stored != nil && cfg.SessionHeaderKey == "" {
|
||||
cfg.SessionHeaderKey = strings.TrimSpace(stored.SessionHeaderKey)
|
||||
}
|
||||
// Keyless formats (Ollama) skip the key resolution entirely.
|
||||
if cfg.NeedsAPIKey() {
|
||||
// API key resolution precedence: form input > the named profile's stored key > the global config's key.
|
||||
// A stored profile's key is never sent back to the browser, so the form is empty when testing a stored profile and the key comes from the DB.
|
||||
if stored != nil && cfg.APIKey == "" {
|
||||
// Trim like ConfigFrom does for form values: a key stored with
|
||||
// surrounding whitespace runs fine in conversations (the runtime
|
||||
// path re-trims) but would be sent raw here and fail auth.
|
||||
cfg.APIKey = strings.TrimSpace(stored.APIKey)
|
||||
}
|
||||
if cfg.SessionHeaderKey == "" {
|
||||
cfg.SessionHeaderKey = strings.TrimSpace(stored.SessionHeaderKey)
|
||||
if cfg.APIKey == "" {
|
||||
s.cfgMu.Lock()
|
||||
cfg.APIKey = s.llmCfg.APIKey
|
||||
s.cfgMu.Unlock()
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
writeJSON(w, 200, map[string]any{"ok": false, "error": "no API key provided"})
|
||||
return
|
||||
}
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
s.cfgMu.Lock()
|
||||
cfg.APIKey = s.llmCfg.APIKey
|
||||
s.cfgMu.Unlock()
|
||||
}
|
||||
if cfg.APIKey == "" {
|
||||
writeJSON(w, 200, map[string]any{"ok": false, "error": "no API key provided"})
|
||||
return
|
||||
}
|
||||
// Retry params do [not] go into the connection test: the test has a 30s hard timeout, and stacking the configured retry counts/long intervals on top
|
||||
// would only turn a perfectly usable endpoint into a "timeout failure". The test asks "does this endpoint work at all"; retry pacing
|
||||
@@ -1621,7 +1639,7 @@ func (s *Server) validateTaskProfileIDs(ids []int64) error {
|
||||
}
|
||||
seen[id] = true
|
||||
if _, ok := s.loadProfileConfig(id); !ok {
|
||||
return fmt.Errorf("LLM profile #%d not found or has no API key set", id)
|
||||
return fmt.Errorf("LLM profile #%d not found or not usable (no API key for the key-requiring formats)", id)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
|
||||
+22
-4
@@ -2037,12 +2037,13 @@ func (s *Server) pgLLMPoolReset(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
// pgListModels fetches available models from the provider's API endpoint.
|
||||
// Supports OpenAI-format (GET /models) and Anthropic-format (GET /v1/models); for
|
||||
// Supports OpenAI-format (GET /models), Anthropic-format (GET /v1/models) and
|
||||
// Ollama (keyless OpenAI-compatible GET /v1/models on the local server); for
|
||||
// Anthropic-compatible third parties (e.g. DeepSeek) whose model list lives only on
|
||||
// the OpenAI path, it falls back to the OpenAI endpoint at the stripped root.
|
||||
func (s *Server) pgListModels(w http.ResponseWriter, r *http.Request) {
|
||||
var req struct {
|
||||
Provider string `json:"provider"` // "openai" | "anthropic"
|
||||
Provider string `json:"provider"` // "openai" | "anthropic" | "ollama"
|
||||
BaseURL string `json:"base_url"`
|
||||
APIKey string `json:"api_key"`
|
||||
Proxy string `json:"proxy"`
|
||||
@@ -2059,13 +2060,14 @@ func (s *Server) pgListModels(w http.ResponseWriter, r *http.Request) {
|
||||
apiKey = p.APIKey
|
||||
}
|
||||
}
|
||||
if apiKey == "" {
|
||||
provider := strings.TrimSpace(req.Provider)
|
||||
// Ollama serves its model list keyless on the local server: no key gate.
|
||||
if apiKey == "" && provider != "ollama" {
|
||||
writeJSON(w, 200, map[string]any{"ok": false, "error": "no API Key provided"})
|
||||
return
|
||||
}
|
||||
|
||||
baseURL := strings.TrimRight(strings.TrimSpace(req.BaseURL), "/")
|
||||
provider := strings.TrimSpace(req.Provider)
|
||||
|
||||
// Candidate endpoints to try in order. Some Anthropic-compatible providers
|
||||
// (e.g. DeepSeek) implement /v1/messages under an /anthropic path but expose
|
||||
@@ -2097,6 +2099,22 @@ func (s *Server) pgListModels(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
b := strings.TrimRight(strings.TrimSuffix(strings.TrimSuffix(baseURL, "/chat/completions"), "/responses"), "/")
|
||||
candidates = append(candidates, candidate{b + "/models", bearerHdr()})
|
||||
case "ollama":
|
||||
// OpenAI-compatible list at {base}/v1/models. Tolerate base URLs with or
|
||||
// without the /v1 suffix (and a full endpoint URL). No auth needed; when a
|
||||
// key was provided anyway, send it as Bearer (Ollama ignores it).
|
||||
if baseURL == "" {
|
||||
baseURL = agent.DefaultOllamaBaseURL
|
||||
}
|
||||
b := strings.TrimSuffix(baseURL, "/chat/completions")
|
||||
if !strings.HasSuffix(b, "/v1") {
|
||||
b += "/v1"
|
||||
}
|
||||
hdr := http.Header{}
|
||||
if apiKey != "" {
|
||||
hdr.Set("Authorization", "Bearer "+apiKey)
|
||||
}
|
||||
candidates = append(candidates, candidate{b + "/models", hdr})
|
||||
default: // anthropic
|
||||
if baseURL == "" {
|
||||
baseURL = "https://api.anthropic.com"
|
||||
|
||||
@@ -29,7 +29,8 @@ func (s *Server) resolutionFromProfile(p *db.LLMProfile, source string) taskLLMR
|
||||
Model: p.Model,
|
||||
Source: source,
|
||||
}
|
||||
if p.APIKey == "" {
|
||||
// Ollama profiles are keyless by design; the other formats need a key.
|
||||
if p.APIKey == "" && p.Format != "ollama" {
|
||||
result.Reason = "LLM profile has no API Key set"
|
||||
return result
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# ARTEX update script: (1) Docker update (pull new image and rebuild) (2) local build update (rebuild the binary)
|
||||
# ARTEX update script: (1) Docker update (rebuild the image from this checkout) (2) local build update (rebuild the binary)
|
||||
# Counterpart of install.sh: install handles the first-time setup, update upgrades to a new version.
|
||||
# No manual DB migration needed — artex idempotently re-runs schema.sql on every start (with ADD COLUMN/CREATE
|
||||
# INDEX IF NOT EXISTS), so "restart equals migrate". Data (pgdata volume, ./data, ./skills) is not affected.
|
||||
@@ -17,7 +17,16 @@ sync_repo(){
|
||||
[ -d .git ] && command -v git >/dev/null 2>&1 || { warn "not a git working copy, skipping git pull"; return; }
|
||||
[ "$(ask 'Pull the latest code (git pull --ff-only)? (y/n)' y)" = y ] || return
|
||||
if ! git pull --ff-only; then
|
||||
warn "git pull could not fast-forward (local changes or diverged branches) — resolve manually and retry; continuing with the current code"
|
||||
# Continuing would rebuild the OLD tree and still print "Build complete":
|
||||
# exactly the silent stale upgrade this script exists to prevent.
|
||||
cat >&2 <<'RECOVER'
|
||||
[!] git pull could not fast-forward. This happens when the checkout has local
|
||||
changes, or when it was cloned before the history rewrite of 2026-10-23.
|
||||
Refusing to rebuild stale code. Recover, then rerun this script:
|
||||
keep your changes: git stash && git fetch origin && git reset --hard origin/main && git stash pop
|
||||
discard them: git fetch origin && git reset --hard origin/main
|
||||
RECOVER
|
||||
die "aborting: nothing was rebuilt"
|
||||
fi
|
||||
}
|
||||
|
||||
@@ -27,23 +36,12 @@ update_docker(){
|
||||
|| die "docker / docker compose not detected; install and deploy with ./install.sh first"
|
||||
[ -f .env ] || die ".env not found; run ./install.sh first to complete the initial deployment"
|
||||
|
||||
# Optional: upgrade to a specific version tag (empty = keep ARTEX_TAG from .env, default latest)
|
||||
local tag; tag="$(ask 'Target image tag (press Enter to keep .env / latest)' '')"
|
||||
if [ -n "$tag" ]; then
|
||||
if grep -q '^ARTEX_TAG=' .env; then
|
||||
sed -i.bak "s|^ARTEX_TAG=.*|ARTEX_TAG=${tag}|" .env && rm -f .env.bak
|
||||
else
|
||||
printf '\nARTEX_TAG=%s\n' "$tag" >> .env
|
||||
fi
|
||||
ok "ARTEX_TAG set to ${tag}"
|
||||
fi
|
||||
|
||||
# Touch only artex: postgres is pinned to 16-alpine and does not need upgrading along
|
||||
# (pulling it wastes bandwidth, and a major version bump risks compatibility). artex
|
||||
# declares depends_on postgres, so an up with the service name starts pg if needed and leaves a running one untouched.
|
||||
info "Pulling new image (artex only)…"
|
||||
docker compose pull artex
|
||||
info "Rebuilding and starting (schema migrates automatically on artex restart)…"
|
||||
info "Building the image from this checkout (artex only)…"
|
||||
docker compose build artex
|
||||
info "Recreating artex (schema migrates automatically on restart)…"
|
||||
docker compose up -d artex
|
||||
ok "Update complete → http://localhost:8787"
|
||||
info "View logs: docker compose logs -f artex"
|
||||
|
||||
@@ -14,6 +14,7 @@ export default function SetupPage() {
|
||||
const router = useRouter();
|
||||
const [password, setPassword] = useState("");
|
||||
const [confirm, setConfirm] = useState("");
|
||||
const [setupToken, setSetupToken] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [checking, setChecking] = useState(true);
|
||||
@@ -47,7 +48,7 @@ export default function SetupPage() {
|
||||
setLoading(true);
|
||||
setError("");
|
||||
try {
|
||||
const { token } = await api.initPassword(password);
|
||||
const { token } = await api.initPassword(password, setupToken.trim());
|
||||
auth.setToken(token);
|
||||
router.replace("/function/tasks");
|
||||
} catch (err) {
|
||||
@@ -80,7 +81,7 @@ export default function SetupPage() {
|
||||
<p className="mx-auto max-w-xl text-muted-foreground">
|
||||
{unavailable
|
||||
? "The backend or database is temporarily unavailable. To avoid overwriting a password already set on this instance, setup is temporarily closed — restore the service and try again."
|
||||
: "First time using ARTEX? Set a login password for the account (at least 8 characters)"}
|
||||
: "First time using ARTEX? Enter the setup token from the server log, then set a login password for the account (at least 8 characters)"}
|
||||
</p>
|
||||
</div>
|
||||
{unavailable ? (
|
||||
@@ -92,6 +93,17 @@ export default function SetupPage() {
|
||||
</div>
|
||||
) : (
|
||||
<form onSubmit={handleSubmit} className="flex flex-col gap-4">
|
||||
<div className="space-y-1.5">
|
||||
<Label htmlFor="setup-token">Setup token</Label>
|
||||
<Input
|
||||
id="setup-token"
|
||||
value={setupToken}
|
||||
onChange={(e) => setSetupToken(e.target.value)}
|
||||
placeholder="Printed in the server log at startup"
|
||||
autoFocus
|
||||
autoComplete="off"
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-1.5">
|
||||
<Label htmlFor="password">New password</Label>
|
||||
<Input
|
||||
@@ -100,7 +112,6 @@ export default function SetupPage() {
|
||||
value={password}
|
||||
onChange={(e) => setPassword(e.target.value)}
|
||||
placeholder="At least 8 characters"
|
||||
autoFocus
|
||||
autoComplete="new-password"
|
||||
/>
|
||||
</div>
|
||||
@@ -116,7 +127,11 @@ export default function SetupPage() {
|
||||
/>
|
||||
</div>
|
||||
{error && <p className="text-sm text-destructive">{error}</p>}
|
||||
<Button type="submit" className="w-full" disabled={loading || !password || !confirm}>
|
||||
<Button
|
||||
type="submit"
|
||||
className="w-full"
|
||||
disabled={loading || !setupToken.trim() || !password || !confirm}
|
||||
>
|
||||
{loading ? "Saving..." : "Set password and sign in"}
|
||||
</Button>
|
||||
</form>
|
||||
|
||||
@@ -49,13 +49,18 @@ const MAX_TOKENS_FIELDS: { value: string; label: string }[] = [
|
||||
{ value: NONE, label: "max_tokens (default)" },
|
||||
{ value: "max_completion_tokens", label: "max_completion_tokens" },
|
||||
];
|
||||
// The other two formats hard-code their field names, so the option is meaningless for them; the hint texts spell this out.
|
||||
// The other formats hard-code their field names, so the option is meaningless for them; the hint texts spell this out.
|
||||
const MAX_TOKENS_FIELD_HINTS: Record<string, string> = {
|
||||
openai:
|
||||
"Which key carries the cap. max_tokens is the default and the vast majority of compatible gateways only accept it; OpenAI's official reasoning models (the o series / GPT-5) conversely only accept max_completion_tokens and fail outright with unsupported_parameter when they receive max_tokens.",
|
||||
anthropic: "Only selectable for the openai format. Anthropic's field name is fixed as max_tokens.",
|
||||
"openai-responses": "Only selectable for the openai format. The Responses API's field name is fixed as max_output_tokens.",
|
||||
ollama: "Only selectable for the openai format. Ollama's field name is fixed as max_tokens.",
|
||||
};
|
||||
// LLM profile formats. The default in the create form is "anthropic"; "ollama" is the
|
||||
// keyless local-server format (base URL defaults to http://127.0.0.1:11434/v1).
|
||||
type ProfileFormat = "anthropic" | "openai" | "openai-responses" | "ollama";
|
||||
const OLLAMA_DEFAULT_BASE_URL = "http://127.0.0.1:11434/v1";
|
||||
const EFFORT_LEVELS: { value: string; label: string }[] = [
|
||||
{ value: NONE, label: "Not sent (default)" },
|
||||
{ value: "low", label: "low" },
|
||||
@@ -72,10 +77,11 @@ function cooldownText(secs: number) {
|
||||
}
|
||||
|
||||
// The "healthy or not" badge a profile shows on its card. A profile without a Key cannot send any request at all — say that before the breaker;
|
||||
// the other states come from the pool's breaker records (with polling off no new records appear, so "healthy" = no known failure).
|
||||
// the exception is Ollama, a keyless local server that IS callable without a key. The other states come from the pool's breaker
|
||||
// records (with polling off no new records appear, so "healthy" = no known failure).
|
||||
type Health = { label: string; cls: string; hint?: string };
|
||||
function healthOf(p: LLMProfile, m?: LLMPoolMember): Health {
|
||||
if (!p.api_key_hint) {
|
||||
if (!p.api_key_hint && p.format !== "ollama") {
|
||||
return {
|
||||
label: "No Key configured",
|
||||
cls: "border-muted-foreground/40 text-muted-foreground",
|
||||
@@ -215,7 +221,8 @@ function PoolSheet({
|
||||
</div>
|
||||
{inChain.length < 2 && (
|
||||
<p className="text-muted-foreground text-xs">
|
||||
Only {inChain.length} usable profile(s) right now, so the pool has no effect — at least 2 profiles with an API Key set and pool participation are required.
|
||||
Only {inChain.length} usable profile(s) right now, so the pool has no effect — at least 2 usable profiles (API Key set, or Ollama) with pool participation are
|
||||
required.
|
||||
</p>
|
||||
)}
|
||||
{chain.map((m) => {
|
||||
@@ -311,7 +318,7 @@ function ProfileSheet({
|
||||
}) {
|
||||
const isNew = !profile;
|
||||
const [name, setName] = React.useState("");
|
||||
const [format, setFormat] = React.useState<"anthropic" | "openai" | "openai-responses">("anthropic");
|
||||
const [format, setFormat] = React.useState<ProfileFormat>("anthropic");
|
||||
const [model, setModel] = React.useState("");
|
||||
const [baseUrl, setBaseUrl] = React.useState("");
|
||||
const [proxy, setProxy] = React.useState("");
|
||||
@@ -340,7 +347,11 @@ function ProfileSheet({
|
||||
React.useEffect(() => {
|
||||
if (!open) return;
|
||||
setName(profile?.name ?? "");
|
||||
setFormat(profile?.format === "openai" || profile?.format === "openai-responses" ? profile.format : "anthropic");
|
||||
setFormat(
|
||||
profile?.format === "openai" || profile?.format === "openai-responses" || profile?.format === "ollama"
|
||||
? profile.format
|
||||
: "anthropic",
|
||||
);
|
||||
setModel(profile?.model ?? "");
|
||||
setBaseUrl(profile?.base_url ?? "");
|
||||
setProxy(profile?.proxy ?? "");
|
||||
@@ -457,6 +468,13 @@ function ProfileSheet({
|
||||
}
|
||||
}
|
||||
|
||||
let keyPlaceholder = "sk-…";
|
||||
if (format === "ollama") {
|
||||
keyPlaceholder = "optional — a local Ollama server runs keyless";
|
||||
} else if (keyHint) {
|
||||
keyPlaceholder = `Set (${keyHint}); leave empty to keep it`;
|
||||
}
|
||||
|
||||
return (
|
||||
<Sheet open={open} onOpenChange={onOpenChange}>
|
||||
<SheetContent
|
||||
@@ -492,7 +510,7 @@ function ProfileSheet({
|
||||
</div>
|
||||
<div className="grid gap-2">
|
||||
<Label>Format</Label>
|
||||
<Select value={format} onValueChange={(v) => setFormat(v as "anthropic" | "openai" | "openai-responses")}>
|
||||
<Select value={format} onValueChange={(v) => setFormat(v as ProfileFormat)}>
|
||||
<SelectTrigger>
|
||||
<SelectValue placeholder="Select a format" />
|
||||
</SelectTrigger>
|
||||
@@ -500,6 +518,7 @@ function ProfileSheet({
|
||||
<SelectItem value="anthropic">Anthropic</SelectItem>
|
||||
<SelectItem value="openai">OpenAI (Chat Completions)</SelectItem>
|
||||
<SelectItem value="openai-responses">OpenAI (Responses API)</SelectItem>
|
||||
<SelectItem value="ollama">Ollama (local, no key needed)</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
@@ -511,7 +530,7 @@ function ProfileSheet({
|
||||
<Input
|
||||
id="p-model"
|
||||
className="font-mono"
|
||||
placeholder="claude-opus-4-8"
|
||||
placeholder={format === "ollama" ? "llama3.2" : "claude-opus-4-8"}
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
/>
|
||||
@@ -557,7 +576,7 @@ function ProfileSheet({
|
||||
<Input
|
||||
id="p-base-url"
|
||||
className="font-mono"
|
||||
placeholder="https://api.openai.com/v1"
|
||||
placeholder={format === "ollama" ? OLLAMA_DEFAULT_BASE_URL : "https://api.openai.com/v1"}
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
/>
|
||||
@@ -595,11 +614,13 @@ function ProfileSheet({
|
||||
</div>
|
||||
|
||||
<div className="grid gap-2">
|
||||
<Label htmlFor="p-api-key">API Key</Label>
|
||||
<Label htmlFor="p-api-key">
|
||||
{format === "ollama" ? "API Key (optional for Ollama)" : "API Key"}
|
||||
</Label>
|
||||
<Input
|
||||
id="p-api-key"
|
||||
type="password"
|
||||
placeholder={keyHint ? `Set (${keyHint}); leave empty to keep it` : "sk-…"}
|
||||
placeholder={keyPlaceholder}
|
||||
value={apiKey}
|
||||
onChange={(e) => setApiKey(e.target.value)}
|
||||
/>
|
||||
|
||||
+2
-1
@@ -240,7 +240,8 @@ export const api = {
|
||||
// ---- auth ----
|
||||
authStatus: () => get<{ initialized: boolean }>("/auth/status"),
|
||||
login: (username: string, password: string) => post<{ token: string }>("/auth/login", { username, password }),
|
||||
initPassword: (password: string) => post<{ token: string }>("/auth/init", { password }),
|
||||
initPassword: (password: string, setupToken: string) =>
|
||||
post<{ token: string }>("/auth/init", { password, setup_token: setupToken }),
|
||||
changePassword: (oldPassword: string, newPassword: string) =>
|
||||
post<{ ok: boolean }>("/auth/change-password", { old_password: oldPassword, new_password: newPassword }),
|
||||
|
||||
|
||||
@@ -1028,7 +1028,7 @@ export interface NotificationDelivery {
|
||||
export interface LLMProfile {
|
||||
id: string;
|
||||
name: string;
|
||||
format: "openai" | "anthropic" | "openai-responses";
|
||||
format: "openai" | "anthropic" | "openai-responses" | "ollama";
|
||||
base_url?: string;
|
||||
proxy?: string;
|
||||
model: string;
|
||||
|
||||
Reference in New Issue
Block a user