Compare commits

..
Author SHA1 Message Date
BaaRai b42bffc210 server: probe every database host of the DSN; compose: SCRAM for TCP
The self-test probe tried only the first host of the DSN. It now tries
each host and port the DSN lists, fallbacks included, and nothing else,
and a DSN it cannot parse fails the self-test instead of skipping the
database check.

docker-compose.yml sets POSTGRES_HOST_AUTH_METHOD=scram-sha-256: a TCP
connection from another container, artex's included, needs the
password. It takes effect when a new pgdata volume is initialised.
2026-11-22 09:50:09 +01:00
BaaRai f83659cd52 server: start degraded where agent isolation cannot be built
The isolation self-test stopped every start it failed, so a non-root
server (the systemd unit), dev.sh, Windows and macOS no longer started
without -dev-insecure-no-isolation.

EnforceIsolation now decides from the platform and the privilege drop:

- self-test passed: isolated;
- Linux with the privilege drop on (root, the Docker image): a failure
  stops the start, unless -dev-insecure-no-isolation is set;
- elsewhere (not Linux, not root, -exec-uid=-1): the server starts
  degraded, without a flag.

A degraded start logs a banner with the self-test failure and what is
exposed, and GET /api/health reports the isolation status to a
signed-in caller; the web UI shows a red banner on every page while it
is degraded. The systemd deployment guide and dev.sh describe the mode.
2026-11-22 09:50:09 +01:00
BaaRai dd80fea140 agent: test that the file tool identity stays on its own thread
The new root tests hold a file tool call open and check, at that moment,
that only the call's thread has file system uid 10001: the caller, 32
goroutines locked on other threads and every other thread listed in
/proc keep uid 0, root's groups and root's access, and goroutines the
call starts do not inherit the ids. A panic and runtime.Goexit inside
the call reach the caller and end the thread instead of returning it
to the runtime. Calls under load with live and cancelled contexts never
show the executor's ids on an observer thread, and no thread keeps
them afterwards.

Both mutations that would leak the identity are caught: unlocking the
thread without restoring its ids, and a process-wide setgroups.

The comment of runWithFileIdentity now states the limits: the ids cover
fn's goroutine only, and fn must not start goroutines or programs.
2026-11-22 09:50:09 +01:00
BaaRai fd1ccb2f6c server: fail the isolation self-test on a skill tree the agent can write
The six platform tools that durably change skills, custom tools and MCP
servers already need operator approval (intercept/persistence.go, the
built-in ask rules of db/intercept_persistence_seed.go, guard.go). Custom
tools and MCP servers live in the database, which the agent's shell
cannot reach. Skills live on disk: an agent process that can write the
skill directory adds or changes a skill without create_skill or
update_skill_file, and every agent then loads it as instructions.

The startup self-test now checks the skill directory: the probe, as the
executor, must not be able to create an entry in it, its parent must not
let the executor replace it, and no file or directory below it may be
owned by the executor uid or writable by the executor gid or by others
(judged from owners and mode bits, symlinks by their target, nothing
written).
2026-11-22 09:50:09 +01:00
BaaRai 7f145965b2 server: keep the keys, config and data out of the executor's reach
PrepareExecPrivilege chmodded jwt.key, secret.key and the config file to
0600 but left their owner as it was: a file owned by the executor uid
(created by an agent process, or on the host under the uid the executor
has in the container) stayed readable by the agent. It now makes root
their owner first.

The startup self-test also checks the directories: the probe, as the
executor, must not be able to create an entry in the data directory, the
key directory or the config file's directory (it could plant a key, a
config or a data tree for the next start), and none of their parents may
let the executor replace them (owned by it, or writable by it without the
sticky bit). For the database, a test against PostgreSQL shows the
server's role refused without its password (SQLSTATE 28P01); the probe
already fails the start when the database lets that role in.

No Unix socket is introduced: in the compose deployment PostgreSQL runs
in its own container and is reached over TCP with a password that the
executor cannot read (sealed environment, root-only config).
2026-11-22 09:50:09 +01:00
BaaRai d86eab4c3a server: self-test agent isolation at startup, fail closed
When the privilege drop could not be applied (the server not root, or
-exec-uid negative), the start went on with one log line, and nothing
checked that an applied drop really isolated the agent. Before the stores
open, the server now runs a self-test: a probe started the way the
agent's Bash starts ("bash" on PATH, so through the shim) checks from
inside the executor process that it runs as the executor uid, gid and
only group, cannot read jwt.key, secret.key, the config file or the MITM
CA key, cannot read the server's /proc environ or memory, sees only the
allowlisted environment, and cannot open a session on the server's
database with the server's user and no password (only host, port, user
and database name reach the probe). The file tools are checked too: the
secrets must not open through their identity.

A failed or impossible self-test stops the start. The development escape
-dev-insecure-no-isolation, off by default, lets the start go on and logs
a banner that says what is exposed. This changes the behaviour of a
non-root start (go run, the systemd example unit, Windows, macOS): it now
needs that flag.
2026-11-22 09:50:09 +01:00
BaaRai 8dc881e6eb server: give agent processes a fixed allowlisted environment
An agent process inherited the server's environment minus the names that
ScrubSecretEnv removes: a deny list, so a credential under any other name
(one an operator or an image adds) reached the agent's shell, its custom
tools and its MCP stdio servers. When a process crosses from the server's
identity to the executor's, its environment is now rebuilt from a fixed
allowlist: the search path and locale, the temp directory, the
non-interactive defaults, the proxy and CA trust variables the agents pass
to their shells, PLAYWRIGHT_BROWSERS_PATH and the shim's own variables.
The Bash shim and the MCP drop shim rebuild it before the drop; a custom
tool starts from it. What the caller passes on purpose still crosses: the
agent's Bash env, the TOOL_* parameters, and the configured env of an MCP
server, which the drop shim keeps because connectMCP names it in
ARTEX_EXEC_KEEP. A nested shell is not filtered again.

The startup scrub and the environment seal stay as they are; they are no
longer the boundary for agent processes. With the drop disabled nothing
changes.
2026-11-22 09:50:09 +01:00
BaaRai 5358ce350e agent: run the file tools as the executor identity
The confined file tools (Read, Write, Edit, MultiEdit, LS, Glob, Grep) and
the opens of ingest_bbot and ingest_scan_output ran inside the server, as
root, and relied on the os.Root path confinement alone. With the executor
privilege drop active, PrepareExecPrivilege now gives them the executor
uid and gid: each call runs on a thread of its own whose file system uid,
file system gid and supplementary groups are the executor's (setfsuid,
setfsgid and the per-thread setgroups of x/sys/unix). The thread is never
unlocked, so the runtime ends it with the call and no other goroutine
runs with those ids. The kernel then checks every open, create and
directory read against the rights of the agent's shell, on top of the
path confinement. Output over the cap is still spilled, after the call,
with the server's identity.

Linux only. Elsewhere the identity is not set and a log line says the
file tools keep the server's identity. golang.org/x/sys becomes a direct
dependency (already in the module graph at the same version).
2026-11-22 09:50:09 +01:00
33 changed files with 2377 additions and 91 deletions
+1 -65
View File
@@ -473,7 +473,7 @@ runtime supports it.
## Configuration
**Database** (`config.json`, `ARTEX_PG_DSN`, or `ARTEX_PG_*` environment variables — see [External PostgreSQL](#external-postgresql)):
**Database** (`config.json`, or the `ARTEX_PG_DSN` environment variable):
```json
{
@@ -558,70 +558,6 @@ from the defaults until a worker finishes in this process.
**Common flags**: `./start.sh -addr :8787 -proxy :8788` (`-addr` frontend/API, `-proxy` traffic
recording proxy).
### External PostgreSQL
ARTEX already supports an external PostgreSQL 16+ instance — the `postgres` service in
`docker-compose.yml` is only the bundled default. Because the database is startup configuration and
the system's single source of truth, there is no runtime database switcher in the UI: change the
environment variables and restart the server to point at another instance.
Configure the connection with either a full connection string or individual parts:
- **`ARTEX_PG_DSN`**: full PostgreSQL connection URL (for example
`postgres://artex:pass@db.example.internal:5432/artex?sslmode=require`). When set, it takes
precedence over the `ARTEX_PG_*` variables and you are responsible for percent-encoding any
URL-special characters in the username or password.
- **`ARTEX_PG_*` parts** (recommended when passwords contain `/`, `?`, `%`, `@`, `:`, or spaces —
ARTEX percent-encodes the user and password and removes `ARTEX_PG_PASSWORD` from the environment
at startup):
- `ARTEX_PG_HOST` — PostgreSQL host
- `ARTEX_PG_PORT` — PostgreSQL port (default `5432`)
- `ARTEX_PG_USER` — database user
- `ARTEX_PG_PASSWORD` — database password (raw, not URL-encoded)
- `ARTEX_PG_DBNAME` — database name
- `ARTEX_PG_SSLMODE` — TLS mode (`disable`, `require`, `verify-ca`, `verify-full`)
#### Compose without the bundled `postgres` service
To run ARTEX against an external PostgreSQL server with Docker Compose, remove the `postgres`
service, the `depends_on` block, and the `pgdata` volume from `docker-compose.yml`, and point
`artex` at your database:
```yaml
services:
artex:
image: ghcr.io/baarai/artex:latest
security_opt:
- "no-new-privileges:true"
environment:
# Option A: individual parts (recommended; percent-encodes special characters)
ARTEX_PG_HOST: ${ARTEX_PG_HOST:?Set ARTEX_PG_HOST in .env}
ARTEX_PG_PORT: ${ARTEX_PG_PORT:-5432}
ARTEX_PG_USER: ${ARTEX_PG_USER:-artex}
ARTEX_PG_PASSWORD: ${ARTEX_PG_PASSWORD:?Set ARTEX_PG_PASSWORD in .env}
ARTEX_PG_DBNAME: ${ARTEX_PG_DBNAME:-artex}
ARTEX_PG_SSLMODE: ${ARTEX_PG_SSLMODE:-require}
# Option B: full DSN (overrides ARTEX_PG_* when set)
# ARTEX_PG_DSN: ${ARTEX_PG_DSN}
ARTEX_JWT_KEY_DIR: /app/keys
ports:
- "${ARTEX_BIND_ADDR:-127.0.0.1}:8787:8787"
- "127.0.0.1:8788:8788"
volumes:
- ./data:/app/data
- ./keys:/app/keys
restart: unless-stopped
```
#### Local disk usage (`data/`)
Moving PostgreSQL to an external host does **not** move the runtime `data/` directory off local
disk. `data/` still stores spilled HTTP/TCP traffic blobs (`data/traffic/_blobs`), LLM
transcripts, task working files, and local task archives, so heavy workloads can still fill the
host disk if `data/` is on a small volume. Mount `data/` on a suitably sized volume, and consider
setting `ARTEX_ARCHIVE_STORAGE=s3` (see [Task archive storage](docs/task-archive-storage.md)) to
offload cold task archives to object storage.
### Reverse proxy deployment (HTTPS / exposing only 443)
The frontend, API and SSE stream share port 8787 (the binary listens on `127.0.0.1:8787` by
+10 -1
View File
@@ -56,10 +56,19 @@ func bbotIntelFact(ev bbotEvent) factItem {
// reads the file as root while the agent names the path, so an absolute path
// outside the working directory, "..", a symlink and a hard-linked file are
// refused. Without a data directory or a working directory it fails closed.
func openBbotOutput(base, dir, p string) (*os.File, error) {
func openBbotOutput(base, dir, p string) (f *os.File, err error) {
if strings.TrimSpace(base) == "" {
return nil, errors.New("no data directory for this run")
}
// The open runs as the executor when the privilege drop is active
// (filetools_identity.go); the descriptor is then read by the server.
if idErr := RunAsFileToolIdentity(func() { f, err = openBbotOutputIn(base, dir, p) }); idErr != nil {
return nil, idErr
}
return f, err
}
func openBbotOutputIn(base, dir, p string) (*os.File, error) {
w, err := openWorkRoot(base, dir)
if err != nil {
return nil, err
+37 -8
View File
@@ -69,12 +69,32 @@ func (t *confinedFileTool) Call(_ context.Context, in json.RawMessage, tc *actoo
if tc != nil {
dir = tc.WorkingDir
}
w, err := openWorkRoot(t.base, dir)
if err != nil {
return actool.Errorf(fmt.Sprintf("Error: %s cannot run: %v", t.Name(), err)), nil
var (
res actool.Result
err error
capture bool
)
// Every file system access of the call, from opening the working directory on,
// runs as the executor when the privilege drop is active (filetools_identity.go).
idErr := RunAsFileToolIdentity(func() {
w, oerr := openWorkRoot(t.base, dir)
if oerr != nil {
res = actool.Errorf(fmt.Sprintf("Error: %s cannot run: %v", t.Name(), oerr))
return
}
defer w.Close()
res, err = t.run(w, in, tc)
capture = w.captured
})
if idErr != nil {
return actool.Errorf(fmt.Sprintf("Error: %s cannot run: %v", t.Name(), idErr)), nil
}
defer w.Close()
return t.run(w, in, tc)
if capture && err == nil {
// Bounded here, with the server's identity: an output over the cap is spilled
// to the root-owned spill directory, which the executor cannot write.
res = actool.Text(actool.Capture(tc, res.Flatten()))
}
return res, err
}
// workRoot is the run's working directory opened as an os.Root. dir is the path the
@@ -82,6 +102,15 @@ func (t *confinedFileTool) Call(_ context.Context, in json.RawMessage, tc *actoo
type workRoot struct {
*os.Root
dir string
// captured is set by capture: the result text still has to go through
// actool.Capture once the call has left the file tool identity.
captured bool
}
// capture returns s unbounded and marks the call for actool.Capture in Call.
func (w *workRoot) capture(s string) string {
w.captured = true
return s
}
var (
@@ -480,7 +509,7 @@ func runConfinedLS(w *workRoot, input json.RawMessage, tc *actool.ToolContext) (
}
fmt.Fprintf(&b, "%s\t%d bytes\n", e.Name(), size)
}
return actool.Text(actool.Capture(tc, b.String())), nil
return actool.Text(w.capture(b.String())), nil
}
// ---------- Glob ----------
@@ -544,7 +573,7 @@ func runConfinedGlob(w *workRoot, input json.RawMessage, tc *actool.ToolContext)
b.WriteString(m.path)
b.WriteByte('\n')
}
return actool.Text(actool.Capture(tc, b.String())), nil
return actool.Text(w.capture(b.String())), nil
}
// globMatch is the SDK Glob matcher: a pattern without "/" matches the base name,
@@ -707,7 +736,7 @@ func runConfinedGrep(w *workRoot, input json.RawMessage, tc *actool.ToolContext)
if more > 0 {
out += fmt.Sprintf("... [%d more result(s) — page with offset=%d, or head_limit=0 for all] ...\n", more, in.Offset+len(kept))
}
return actool.Text(actool.Capture(tc, out)), nil
return actool.Text(w.capture(out)), nil
}
// grepFileTypes maps a ripgrep-style file type to its globs (the SDK's subset).
+36
View File
@@ -0,0 +1,36 @@
package agent
import "sync/atomic"
// File tool identity. The confined file tools (filetools.go) and the ingest tools
// open the paths the agent names inside the server process, which is root in the
// Docker image. With the executor privilege drop active (server/execpriv.go) the
// server sets the executor identity here, and each of those calls then reaches the
// file system as that uid and gid, with that gid as its only group: the kernel
// checks every open, create and directory read against the executor's rights, the
// rights of the agent's shell, on top of the os.Root confinement. Linux only; see
// filetools_identity_linux.go.
// FileToolIdentity is the uid and gid the file tools access files as.
type FileToolIdentity struct{ UID, GID uint32 }
var fileToolIdentity atomic.Pointer[FileToolIdentity]
// SetFileToolIdentity sets the identity of the file tools; nil restores the
// server's own identity.
func SetFileToolIdentity(id *FileToolIdentity) { fileToolIdentity.Store(id) }
// FileToolIdentitySupported reports whether the file tools can run as another
// identity on this platform.
func FileToolIdentitySupported() bool { return fileIdentitySupported }
// RunAsFileToolIdentity runs fn with the file tool identity, or directly when none
// is set. When it returns an error, fn did not run.
func RunAsFileToolIdentity(fn func()) error {
id := fileToolIdentity.Load()
if id == nil {
fn()
return nil
}
return runWithFileIdentity(*id, fn)
}
+71
View File
@@ -0,0 +1,71 @@
//go:build linux
package agent
import (
"fmt"
"runtime"
"golang.org/x/sys/unix"
)
const fileIdentitySupported = true
// runWithFileIdentity runs fn on a thread of its own whose file system uid, file
// system gid and supplementary groups are those of id. On Linux these ids belong to
// the thread: setfsuid, setfsgid and the raw setgroups of x/sys/unix change the
// calling thread only (the syscall package's Setgroups changes every thread and is
// not used). The thread is locked to its goroutine and never unlocked, so the Go
// runtime ends it when the goroutine returns, on a panic or runtime.Goexit in fn
// too (the process's main thread is parked for good instead): no other goroutine
// ever runs with the changed ids, and nothing has to be restored. While it is
// locked the runtime never clones a new thread from it (the template thread does
// that), so no thread inherits the ids. A file system uid other than 0 also
// clears the file capabilities of the thread (CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH,
// CAP_FOWNER, CAP_CHOWN and the others of capabilities(7)), so root's override does
// not apply to fn. A panic in fn is raised again in the caller.
//
// The ids cover fn's own goroutine only: a goroutine fn starts runs on another
// thread with the server's ids, and a program fn would start gets its file system
// ids reset to root's by execve. fn must open everything itself, and start nothing.
// The scope is tested in filetools_identity_scope_root_linux_test.go.
func runWithFileIdentity(id FileToolIdentity, fn func()) error {
done := make(chan error, 1)
var panicked any
go func() {
runtime.LockOSThread()
if err := enterFileIdentity(id); err != nil {
done <- err
return
}
defer func() {
panicked = recover()
done <- nil
}()
fn()
}()
err := <-done
if panicked != nil {
panic(panicked)
}
return err
}
// enterFileIdentity sets the ids of the calling thread and reads them back.
// setfsuid and setfsgid report no error: each returns the previous id, and an
// invalid id (-1) changes nothing and returns the current one.
func enterFileIdentity(id FileToolIdentity) error {
uid, gid := int(id.UID), int(id.GID)
if err := unix.Setgroups([]int{gid}); err != nil {
return fmt.Errorf("file tool identity: setgroups: %w", err)
}
_, _ = unix.SetfsgidRetGid(gid)
_, _ = unix.SetfsuidRetUid(uid)
curGID, _ := unix.SetfsgidRetGid(-1)
curUID, _ := unix.SetfsuidRetUid(-1)
groups, err := unix.Getgroups()
if err != nil || curUID != uid || curGID != gid || len(groups) != 1 || groups[0] != gid {
return fmt.Errorf("file tool identity %d:%d did not take effect (fsuid %d, fsgid %d, groups %v)", uid, gid, curUID, curGID, groups)
}
return nil
}
+14
View File
@@ -0,0 +1,14 @@
//go:build !linux
package agent
import "errors"
const fileIdentitySupported = false
// runWithFileIdentity: per-thread file system ids exist on Linux only. The server
// sets no identity where FileToolIdentitySupported is false; one set anyway fails
// closed.
func runWithFileIdentity(FileToolIdentity, func()) error {
return errors.New("the file tools cannot run as another identity on this platform")
}
+193
View File
@@ -0,0 +1,193 @@
//go:build linux
package agent
import (
"context"
"encoding/json"
"errors"
"io/fs"
"os"
"path/filepath"
"strings"
"sync"
"syscall"
"testing"
actool "github.com/Autumn-27/norma/tool"
)
// The executor identity of the server's default (server.DefaultExecUID/GID).
const testExecID = 10001
// fileIdentityFixture lays out a data directory like the server's with the drop
// active: tasks/1/main is a run directory root:<exec gid> 2775 below root:root 0755
// parents, and holds a file the server wrote root:root 0600.
type fileIdentityFixture struct {
base, work, rootOnly string
tools map[string]actool.CoreTool
tc *actool.ToolContext
}
func newFileIdentityFixture(t *testing.T) *fileIdentityFixture {
t.Helper()
if os.Geteuid() != 0 {
t.Skip("needs root")
}
// Not t.TempDir(): the executor has to traverse the directory. TestMain made the
// temp directory 1777.
base, err := os.MkdirTemp("", "artex-fileid-")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(base) })
work := filepath.Join(base, "tasks", "1", "main")
for _, d := range []string{base, filepath.Join(base, "tasks"), filepath.Join(base, "tasks", "1")} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Chmod(d, 0o755); err != nil {
t.Fatal(err)
}
}
if err := os.Mkdir(work, 0o775); err != nil {
t.Fatal(err)
}
if err := os.Chown(work, 0, testExecID); err != nil {
t.Fatal(err)
}
if err := os.Chmod(work, 0o775|os.ModeSetgid); err != nil {
t.Fatal(err)
}
rootOnly := filepath.Join(work, "root-only.txt")
if err := os.WriteFile(rootOnly, []byte(confinedSecret+"\n"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.Chown(rootOnly, 0, 0); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(work, "shared.txt"), []byte("hello shared\n"), 0o664); err != nil {
t.Fatal(err)
}
if err := os.Chown(filepath.Join(work, "shared.txt"), 0, testExecID); err != nil {
t.Fatal(err)
}
tools := map[string]actool.CoreTool{}
for _, tl := range defaultTools(base) {
tools[tl.Name()] = tl
}
t.Cleanup(func() { SetFileToolIdentity(nil) })
return &fileIdentityFixture{base: base, work: work, rootOnly: rootOnly, tools: tools, tc: &actool.ToolContext{WorkingDir: work}}
}
func (f *fileIdentityFixture) call(t *testing.T, tool string, in map[string]any) actool.Result {
t.Helper()
raw, _ := json.Marshal(in)
res, err := f.tools[tool].Call(context.Background(), raw, f.tc)
if err != nil {
t.Fatalf("%s(%s): %v", tool, raw, err)
}
return res
}
// With the executor identity set, the file tools reach the run directory with the
// executor's rights: a root-only file inside it is refused by the kernel, Grep does
// not see it, a file the tools create belongs to the executor, and the shared file
// stays readable. Without the identity (the server's root) the same file is read,
// which is the state the identity changes.
func TestFileToolsRunAsExecutorAsRoot(t *testing.T) {
f := newFileIdentityFixture(t)
// Root, no identity: the root-only file is read.
if res := f.call(t, "Read", map[string]any{"file_path": "root-only.txt"}); res.IsError || !strings.Contains(res.Flatten(), confinedSecret) {
t.Fatalf("without the identity root reads the file: %s", res.Flatten())
}
SetFileToolIdentity(&FileToolIdentity{UID: testExecID, GID: testExecID})
res := f.call(t, "Read", map[string]any{"file_path": "root-only.txt"})
if !res.IsError || strings.Contains(res.Flatten(), confinedSecret) || !strings.Contains(res.Flatten(), "permission denied") {
t.Errorf("Read of a root-only file as the executor: error=%v %s", res.IsError, res.Flatten())
}
for _, in := range []map[string]any{
{"pattern": "TOPSECRET", "output_mode": "content"},
{"pattern": "TOPSECRET", "path": "root-only.txt", "output_mode": "content"},
} {
if out := f.call(t, "Grep", in).Flatten(); strings.Contains(out, confinedSecret) {
t.Errorf("Grep %v printed the root-only file: %s", in, out)
}
}
res = f.call(t, "Edit", map[string]any{"file_path": "root-only.txt", "old_string": confinedSecret, "new_string": "pwned"})
if !res.IsError {
t.Errorf("Edit of a root-only file as the executor was not refused: %s", res.Flatten())
}
if res := f.call(t, "Read", map[string]any{"file_path": "shared.txt"}); res.IsError || !strings.Contains(res.Flatten(), "hello shared") {
t.Errorf("Read of a file shared with the executor group: %s", res.Flatten())
}
if res := f.call(t, "Write", map[string]any{"file_path": "made.txt", "content": "x\n"}); res.IsError {
t.Fatalf("Write as the executor: %s", res.Flatten())
}
fi, err := os.Stat(filepath.Join(f.work, "made.txt"))
if err != nil {
t.Fatal(err)
}
if st := fi.Sys().(*syscall.Stat_t); st.Uid != testExecID || st.Gid != testExecID {
t.Errorf("made.txt belongs to %d:%d, want the executor %d:%d", st.Uid, st.Gid, testExecID, testExecID)
}
if b, err := os.ReadFile(f.rootOnly); err != nil || string(b) != confinedSecret+"\n" {
t.Fatalf("root-only file changed or unreadable by the test: %q %v", b, err)
}
}
// The ingest tools open the agent's file the same way.
func TestIngestOpenRunsAsExecutorAsRoot(t *testing.T) {
f := newFileIdentityFixture(t)
SetFileToolIdentity(&FileToolIdentity{UID: testExecID, GID: testExecID})
if file, err := openBbotOutput(f.base, f.work, "root-only.txt"); err == nil || !errors.Is(err, fs.ErrPermission) {
if file != nil {
file.Close()
}
t.Errorf("ingest_bbot open of a root-only file: %v, want permission denied", err)
}
if file, err := openScanOutputInBase(f.base, f.work, "root-only.txt"); err == nil || !errors.Is(err, fs.ErrPermission) {
if file != nil {
file.Close()
}
t.Errorf("ingest_scan_output open of a root-only file: %v, want permission denied", err)
}
file, err := openBbotOutput(f.base, f.work, "shared.txt")
if err != nil {
t.Fatalf("ingest_bbot open of a shared file: %v", err)
}
file.Close()
}
// The identity never leaks to the calling goroutine or to other goroutines: after
// many concurrent calls the test, still root, reads the root-only file, and its own
// thread keeps file system uid 0.
func TestFileToolIdentityDoesNotLeakAsRoot(t *testing.T) {
f := newFileIdentityFixture(t)
SetFileToolIdentity(&FileToolIdentity{UID: testExecID, GID: testExecID})
var wg sync.WaitGroup
for i := 0; i < 64; i++ {
wg.Add(1)
go func() {
defer wg.Done()
raw, _ := json.Marshal(map[string]any{"file_path": "root-only.txt"})
res, err := f.tools["Read"].Call(context.Background(), raw, f.tc)
if err != nil || !res.IsError {
t.Errorf("concurrent Read as the executor: err=%v error=%v %s", err, res.IsError, res.Flatten())
}
}()
}
wg.Wait()
for i := 0; i < 16; i++ {
if _, err := os.ReadFile(f.rootOnly); err != nil {
t.Fatalf("the server lost root's access after the file tool calls: %v", err)
}
}
SetFileToolIdentity(nil)
if res := f.call(t, "Read", map[string]any{"file_path": "root-only.txt"}); res.IsError {
t.Fatalf("with the identity cleared root reads the file again: %s", res.Flatten())
}
}
@@ -0,0 +1,367 @@
//go:build linux
package agent
import (
"context"
"encoding/json"
"fmt"
"os"
"runtime"
"slices"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"golang.org/x/sys/unix"
)
// ownFSIDs returns the file system uid, file system gid and supplementary groups of
// the calling thread. setfsuid(-1) and setfsgid(-1) change nothing and return the
// current id.
func ownFSIDs() (fsuid, fsgid int, groups []int) {
fsuid, _ = unix.SetfsuidRetUid(-1)
fsgid, _ = unix.SetfsgidRetGid(-1)
groups, _ = unix.Getgroups()
return fsuid, fsgid, groups
}
// hasExecutorID reports whether the ids of a thread carry any part of the executor
// identity. A process-wide setgroups (the syscall package's) would give the executor
// gid to every thread: the groups are checked as well as the file system ids.
func hasExecutorID(fsuid, fsgid int, groups []int) bool {
return fsuid == testExecID || fsgid == testExecID || slices.Contains(groups, testExecID)
}
// threadFSUID reads the file system uid of thread tid of this process from
// /proc/self/task/<tid>/status (the fourth id of the Uid line).
func threadFSUID(tid int) (int, error) {
b, err := os.ReadFile("/proc/self/task/" + strconv.Itoa(tid) + "/status")
if err != nil {
return -1, err
}
for _, line := range strings.Split(string(b), "\n") {
if f := strings.Fields(line); len(f) == 5 && f[0] == "Uid:" {
return strconv.Atoi(f[4])
}
}
return -1, fmt.Errorf("no Uid line for thread %d", tid)
}
// threadsWithFSUID lists the threads of this process whose file system uid is uid.
func threadsWithFSUID(t *testing.T, uid int) []int {
t.Helper()
entries, err := os.ReadDir("/proc/self/task")
if err != nil {
t.Fatal(err)
}
var tids []int
for _, e := range entries {
tid, err := strconv.Atoi(e.Name())
if err != nil {
continue
}
if got, err := threadFSUID(tid); err == nil && got == uid {
tids = append(tids, tid)
}
}
return tids
}
// waitNoExecutorThread waits until no thread of the process has the executor's file
// system uid, apart from the main thread when a file tool goroutine ran there: the
// runtime parks a locked main thread for good instead of ending it, and no goroutine
// runs on it again. A thread ends shortly after its goroutine returns, not at once.
func waitNoExecutorThread(t *testing.T) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for {
var left []int
for _, tid := range threadsWithFSUID(t, testExecID) {
if tid != os.Getpid() {
left = append(left, tid)
}
}
if len(left) == 0 {
return
}
if time.Now().After(deadline) {
t.Fatalf("threads %v still have the executor's file system uid after the file tool calls returned", left)
}
time.Sleep(10 * time.Millisecond)
}
}
// waitThreadGone waits until thread tid has ended. The main thread is parked instead
// (see waitNoExecutorThread) and is accepted as is.
func waitThreadGone(t *testing.T, tid int) {
t.Helper()
if tid == os.Getpid() {
t.Logf("the file tool goroutine ran on the main thread %d, which the runtime parks for good", tid)
return
}
deadline := time.Now().Add(5 * time.Second)
for {
if _, err := os.Stat("/proc/self/task/" + strconv.Itoa(tid)); os.IsNotExist(err) {
return
}
if time.Now().After(deadline) {
fsuid, _ := threadFSUID(tid)
t.Fatalf("the file tool thread %d is still alive (file system uid %d): it may be reused", tid, fsuid)
}
time.Sleep(10 * time.Millisecond)
}
}
// While a file tool call holds the executor identity, it is the identity of that
// call's thread only. The call itself sees file system uid and gid 10001, the
// executor gid as its only group, and is refused the root-only file. At the same
// moment the caller's thread, 32 goroutines locked on other threads and every other
// thread of the process listed in /proc keep uid 0 and root's access. Goroutines the
// call starts run on other threads (new threads are cloned by the runtime's template
// thread, never from a locked one) and do not get the identity either: everything a
// file tool opens has to be opened on the call's own goroutine, which filetools.go,
// bbot_ingest_tool.go and scan_ingest_tool.go do. After the call the thread ends.
func TestFileToolIdentityScopedToItsThreadAsRoot(t *testing.T) {
f := newFileIdentityFixture(t)
runtime.LockOSThread()
defer runtime.UnlockOSThread()
callerTid := unix.Gettid()
callerUID, callerGID, callerGroups := ownFSIDs()
if callerUID != 0 || hasExecutorID(callerUID, callerGID, callerGroups) {
t.Fatalf("the test thread starts with fsuid %d fsgid %d groups %v, want root's without the executor's", callerUID, callerGID, callerGroups)
}
SetFileToolIdentity(&FileToolIdentity{UID: testExecID, GID: testExecID})
var (
toolTid, toolUID, toolGID, toolProcUID int
toolGroups []int
toolReadErr error
childMu sync.Mutex
childUIDs = map[int]int{} // tid -> fsuid
)
inside, release := make(chan struct{}), make(chan struct{})
callErr := make(chan error, 1)
go func() {
callErr <- RunAsFileToolIdentity(func() {
toolTid = unix.Gettid()
toolUID, toolGID, toolGroups = ownFSIDs()
toolProcUID, _ = threadFSUID(toolTid)
_, toolReadErr = os.ReadFile(f.rootOnly)
const children = 8
var ready, done sync.WaitGroup
ready.Add(children)
done.Add(children)
for i := 0; i < children; i++ {
go func() {
defer done.Done()
runtime.LockOSThread()
defer runtime.UnlockOSThread()
ready.Done()
ready.Wait() // all eight locked at once: at least some are new threads
uid, _, _ := ownFSIDs()
childMu.Lock()
childUIDs[unix.Gettid()] = uid
childMu.Unlock()
}()
}
done.Wait()
close(inside)
<-release
})
}()
<-inside
// The identity is held now.
if toolUID != testExecID || toolGID != testExecID || len(toolGroups) != 1 || toolGroups[0] != testExecID || toolProcUID != testExecID {
t.Errorf("inside the call: fsuid %d (proc %d) fsgid %d groups %v, want %d everywhere", toolUID, toolProcUID, toolGID, toolGroups, testExecID)
}
if !os.IsPermission(toolReadErr) {
t.Errorf("inside the call the root-only file: %v, want permission denied", toolReadErr)
}
if toolTid == callerTid {
t.Errorf("the call ran on the caller's thread %d", toolTid)
}
for tid, uid := range childUIDs {
if tid == toolTid || uid != 0 {
t.Errorf("a goroutine started inside the call ran on thread %d with file system uid %d, want another thread with 0", tid, uid)
}
}
if uid, gid, groups := ownFSIDs(); uid != callerUID || gid != callerGID || fmt.Sprint(groups) != fmt.Sprint(callerGroups) || hasExecutorID(uid, gid, groups) {
t.Errorf("the caller's thread changed during the call: fsuid %d fsgid %d groups %v", uid, gid, groups)
}
const observers = 32
var wg sync.WaitGroup
for i := 0; i < observers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
runtime.LockOSThread()
defer runtime.UnlockOSThread()
tid := unix.Gettid()
uid, gid, groups := ownFSIDs()
if tid == toolTid || uid != 0 || gid != callerGID || fmt.Sprint(groups) != fmt.Sprint(callerGroups) || hasExecutorID(uid, gid, groups) {
t.Errorf("observer on thread %d (call thread %d) during the call: fsuid %d fsgid %d groups %v", tid, toolTid, uid, gid, groups)
}
if _, err := os.ReadFile(f.rootOnly); err != nil {
t.Errorf("observer on thread %d lost root's access during the call: %v", tid, err)
}
}()
}
wg.Wait()
var holders []int
for _, tid := range threadsWithFSUID(t, testExecID) {
if tid != os.Getpid() || tid == toolTid { // a main thread parked by an earlier call is not counted
holders = append(holders, tid)
}
}
if len(holders) != 1 || holders[0] != toolTid {
t.Errorf("threads with the executor's file system uid during the call: %v, want only the call's thread %d", holders, toolTid)
}
close(release)
if err := <-callErr; err != nil {
t.Fatal(err)
}
waitThreadGone(t, toolTid)
waitNoExecutorThread(t)
}
// A panic inside a file tool call is raised again in the caller, which keeps uid 0;
// the thread that held the identity ends instead of going back to the runtime's
// pool. runtime.Goexit inside the call (t.FailNow, for one) ends it the same way.
// The identity setting itself is untouched: the next call runs as the executor.
func TestFileToolIdentityPanicAsRoot(t *testing.T) {
f := newFileIdentityFixture(t)
runtime.LockOSThread()
defer runtime.UnlockOSThread()
SetFileToolIdentity(&FileToolIdentity{UID: testExecID, GID: testExecID})
var panicTid, panicUID int
func() {
defer func() {
if r := recover(); r != "boom inside the file tool" {
t.Fatalf("recovered %v, want the panic of the call", r)
}
}()
_ = RunAsFileToolIdentity(func() {
panicTid = unix.Gettid()
panicUID, _, _ = ownFSIDs()
panic("boom inside the file tool")
})
t.Fatal("the panic did not reach the caller")
}()
if panicUID != testExecID {
t.Fatalf("the call that panicked ran with file system uid %d, want %d", panicUID, testExecID)
}
if uid, gid, groups := ownFSIDs(); uid != 0 || hasExecutorID(uid, gid, groups) {
t.Fatalf("after the panic the caller's thread has fsuid %d fsgid %d groups %v, want root's", uid, gid, groups)
}
if _, err := os.ReadFile(f.rootOnly); err != nil {
t.Fatalf("after the panic the caller lost root's access: %v", err)
}
waitThreadGone(t, panicTid)
var exitTid int
if err := RunAsFileToolIdentity(func() {
exitTid = unix.Gettid()
runtime.Goexit()
}); err != nil {
t.Fatalf("Goexit inside the call: %v", err)
}
waitThreadGone(t, exitTid)
waitNoExecutorThread(t)
if res := f.call(t, "Read", map[string]any{"file_path": "root-only.txt"}); !res.IsError {
t.Fatalf("after a panic the next call no longer runs as the executor: %s", res.Flatten())
}
}
// File tool calls ignore their context: a call made with a cancelled context still
// runs, and runs as the executor. Under load (real tool calls with live and cancelled
// contexts on many goroutines) goroutines that sample their own thread never see the
// executor's uid, every caller keeps uid 0, and no thread keeps the identity once
// the calls are done.
func TestFileToolIdentityUnderLoadAndCancelAsRoot(t *testing.T) {
f := newFileIdentityFixture(t)
SetFileToolIdentity(&FileToolIdentity{UID: testExecID, GID: testExecID})
cancelled, cancel := context.WithCancel(context.Background())
cancel()
raw, _ := json.Marshal(map[string]any{"file_path": "root-only.txt"})
res, err := f.tools["Read"].Call(cancelled, raw, f.tc)
if err != nil || !res.IsError || !strings.Contains(res.Flatten(), "permission denied") {
t.Fatalf("Read with a cancelled context: err=%v %s, want permission denied as the executor", err, res.Flatten())
}
stop := make(chan struct{})
var samples, leaks atomic.Int64
var observers sync.WaitGroup
for i := 0; i < 8; i++ {
observers.Add(1)
go func() {
defer observers.Done()
runtime.LockOSThread()
defer runtime.UnlockOSThread()
for {
select {
case <-stop:
return
default:
}
if uid, gid, groups := ownFSIDs(); uid != 0 || hasExecutorID(uid, gid, groups) {
leaks.Add(1)
}
samples.Add(1)
runtime.Gosched()
}
}()
}
calls := []struct {
tool string
in map[string]any
}{
{"Read", map[string]any{"file_path": "root-only.txt"}},
{"Read", map[string]any{"file_path": "shared.txt"}},
{"Grep", map[string]any{"pattern": "TOPSECRET", "output_mode": "content"}},
{"LS", map[string]any{"path": "."}},
}
var callers sync.WaitGroup
for i := 0; i < 200; i++ {
callers.Add(1)
go func(i int) {
defer callers.Done()
runtime.LockOSThread()
defer runtime.UnlockOSThread()
c := calls[i%len(calls)]
ctx := context.Background()
if i%2 == 1 {
ctx = cancelled
}
raw, _ := json.Marshal(c.in)
res, err := f.tools[c.tool].Call(ctx, raw, f.tc)
if err != nil {
t.Errorf("%s: %v", c.tool, err)
return
}
if strings.Contains(res.Flatten(), confinedSecret) {
t.Errorf("%s printed the root-only file: %s", c.tool, res.Flatten())
}
if uid, gid, groups := ownFSIDs(); uid != 0 || hasExecutorID(uid, gid, groups) {
t.Errorf("after %s the caller's thread has fsuid %d fsgid %d groups %v", c.tool, uid, gid, groups)
}
}(i)
}
callers.Wait()
close(stop)
observers.Wait()
if leaks.Load() != 0 {
t.Fatalf("%d of %d samples on observer threads saw a file system uid other than 0", leaks.Load(), samples.Load())
}
t.Logf("%d samples on observer threads during 200 calls, all file system uid 0", samples.Load())
waitNoExecutorThread(t)
}
+34
View File
@@ -0,0 +1,34 @@
package agent
import (
"os"
"runtime"
"testing"
)
// Without an identity fn runs directly.
func TestRunAsFileToolIdentityWithoutIdentity(t *testing.T) {
SetFileToolIdentity(nil)
ran := false
if err := RunAsFileToolIdentity(func() { ran = true }); err != nil || !ran {
t.Fatalf("err=%v ran=%v", err, ran)
}
}
// A process that cannot change its ids (not root, or not Linux) gets an error and fn
// does not run: the file tools fail closed instead of running as the server.
func TestRunAsFileToolIdentityFailsClosed(t *testing.T) {
if runtime.GOOS == "linux" && os.Geteuid() == 0 {
t.Skip("root can change its ids")
}
SetFileToolIdentity(&FileToolIdentity{UID: 10001, GID: 10001})
t.Cleanup(func() { SetFileToolIdentity(nil) })
ran := false
if err := RunAsFileToolIdentity(func() { ran = true }); err == nil || ran {
t.Fatalf("err=%v ran=%v, want an error and fn not run", err, ran)
}
f := newConfinedFixture(t)
if res := f.call(t, "Read", map[string]any{"file_path": "in.txt"}); !res.IsError {
t.Fatalf("Read ran without its identity: %s", res.Flatten())
}
}
+15 -6
View File
@@ -115,16 +115,25 @@ func openScanOutput(root, path string) (*os.File, error) {
return openScanOutputFromWorkRoot(&workRoot{Root: r, dir: absRoot}, path)
}
func openScanOutputInBase(base, dir, path string) (*os.File, error) {
func openScanOutputInBase(base, dir, path string) (f *os.File, err error) {
if strings.TrimSpace(base) == "" {
return openScanOutput(dir, path)
}
w, err := openWorkRoot(base, dir)
if err != nil {
return nil, err
// The open runs as the executor when the privilege drop is active
// (filetools_identity.go); the descriptor is then read by the server.
idErr := RunAsFileToolIdentity(func() {
w, oerr := openWorkRoot(base, dir)
if oerr != nil {
err = oerr
return
}
defer w.Close()
f, err = openScanOutputFromWorkRoot(w, path)
})
if idErr != nil {
return nil, idErr
}
defer w.Close()
return openScanOutputFromWorkRoot(w, path)
return f, err
}
// ingestScanOutput parses a scanner output file written by the agent's own
+14 -1
View File
@@ -105,6 +105,8 @@ func run() int {
// A negative uid turns the drop off; 0 (root) is refused.
execUID = flag.Int("exec-uid", server.DefaultExecUID, "uid that agent processes run as (Bash, shell, custom tools, MCP stdio); -1 turns the drop off")
execGID = flag.Int("exec-gid", server.DefaultExecGID, "gid that agent processes run as")
// Development escape of the isolation self-test, off by default.
devNoIsolation = flag.Bool(server.DevNoIsolationFlag, false, "DEVELOPMENT ONLY: on Linux as root, start even when the agent isolation self-test fails; agent processes and file tools may then read the server's keys, config and database credentials")
)
flag.Parse()
@@ -187,6 +189,17 @@ func run() int {
log.Printf("[config] %v", err)
return 1
}
// Before the stores open and any agent runs: check that the isolation above holds
// (server/isolation.go). On Linux as root a failure stops the start unless the
// escape is set; elsewhere the server starts degraded and says so.
selfTest := server.IsolationSelfTest(priv, server.IsolationTargets{
DataDir: *dataDir, KeyDir: keyDir, ConfigPath: cfgPath, SkillDir: skillDir, DSN: dsn,
})
isolation, err := server.EnforceIsolation(runtime.GOOS, priv, selfTest, *devNoIsolation)
if err != nil {
log.Printf("[config] %v", err)
return 1
}
sigCtx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
@@ -199,7 +212,7 @@ func run() int {
}
defer mgr.Close()
srv := server.New(ctx, mgr, skillDir, *dataDir, keyDir, server.WithSecretScrub(), server.WithTrustedProxies(trusted), server.WithExecPrivilege(priv))
srv := server.New(ctx, mgr, skillDir, *dataDir, keyDir, server.WithSecretScrub(), server.WithTrustedProxies(trusted), server.WithExecPrivilege(priv), server.WithIsolationStatus(isolation))
// ReadTimeout bounds how long the request headers and body may take to arrive.
// It does not limit how long a handler runs, so an event stream stays open.
// WriteTimeout is left unset: a write deadline would cut that stream.
Executable → Regular
+4
View File
@@ -11,6 +11,10 @@ cleanup() { kill 0 2>/dev/null || true; }
trap cleanup EXIT INT TERM
# Backend (plain go run, frontend not embedded); the concurrent work agent count is configured under "System Settings".
# Run this script as your own user: the backend then starts with agent isolation degraded, which it logs in a
# banner and shows in the web UI (agents run as you and can read the keys, config and database credentials).
# Agent isolation needs Linux and a root server; use the Docker image for it. As root, a failed isolation
# self-test stops the start (server/isolation.go).
go run ./cmd/artex -addr :8787 -proxy 127.0.0.1:8788 &
# Frontend hot reload (Vite/Next dev server, /api reverse-proxied to :8787).
+6
View File
@@ -5,6 +5,12 @@ services:
POSTGRES_USER: ${POSTGRES_USER:-artex}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Please set POSTGRES_PASSWORD in .env}
POSTGRES_DB: ${POSTGRES_DB:-artex}
# Password (SCRAM) for every TCP connection from another container, artex's
# included; the server's isolation self-test checks that the agent cannot
# connect without it. Applied when the database is first initialised (a new
# pgdata volume); the image keeps trust for connections from inside this
# container only, which no agent process reaches.
POSTGRES_HOST_AUTH_METHOD: scram-sha-256
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
+9
View File
@@ -112,6 +112,15 @@ The unit runs `/opt/artex/artex` directly without an extra supervisor script. It
Because `docs/examples/artex.service` runs the server as the non-root `<artex-user>` (`User=artex`), the `-exec-uid`/`-exec-gid` privilege drop cannot be applied: `ResolveExecPrivilege` disables the drop when the server is not root (and errors if those flags are passed explicitly). The example unit deliberately omits `-exec-uid`/`-exec-gid`, so agents and spawned tools run under the `<artex-user>` service identity rather than a separate executor account. The root-owned run-directory layout and executor-group containment exist only when the server runs as root with explicit `-exec-uid`/`-exec-gid`; if you require unprivileged-executor containment, use the Docker deployment, which is the supported path for that isolation model.
At every start the server runs an agent isolation self-test before it opens its stores. Under this unit the self-test reports that the privilege drop is off, and the server starts in **degraded** isolation mode; no flag is needed and the unit needs no change. A degraded start:
- logs a banner that begins with `[isolation] !!!` and contains `AGENT ISOLATION IS DEGRADED` together with the self-test failure (see `journalctl -u artex.service`);
- shows a red banner on every page of the web UI (from the `isolation` field of `GET /api/health`, returned to signed-in users only).
In degraded mode agent shells, custom tools, MCP stdio servers and the file tools run as `<artex-user>`, which owns `config.json`, `keys/` and `data/`: they can read the JWT signing key, the secret key and the database password. Do not use this deployment on a server that holds real targets or credentials.
A Linux server that runs as root with the privilege drop on (the Docker image) is held to the self-test instead: when isolation is broken there, the server refuses to start. `-dev-insecure-no-isolation` lets it start degraded anyway, for development only. The same rule leaves Windows, macOS and `dev.sh` (run as your own user) in degraded mode, with the same banners.
## Web setup
On a new database, visit `http://127.0.0.1:8787/setup`, provide the one-time setup token from the private startup log or journal (`journalctl -u artex.service`), and choose a unique password of at least 12 characters. The username is `ARTEX`. Store the password in your password manager rather than in a plaintext file on disk.
+1 -1
View File
@@ -72,7 +72,7 @@ require (
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 // indirect
golang.org/x/mod v0.42.0 // indirect
golang.org/x/sync v0.25.0 // indirect
golang.org/x/sys v0.50.0 // indirect
golang.org/x/sys v0.50.0
golang.org/x/tools v0.52.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/libc v1.72.3 // indirect
+3 -2
View File
@@ -525,8 +525,9 @@ func execPython(ctx context.Context, interp, key, code string, params map[string
defer cancel()
c := exec.CommandContext(runCtx, interp, tmp)
c.Dir = workDir
c.Env = append(os.Environ(), sessionEnv...) // session proxy env
for k, v := range params { // scalar params mirrored as TOOL_<NAME>
// The allowlist as the executor (execenv_unix.go), then the session proxy env.
c.Env = append(agentProcessEnv(priv), sessionEnv...)
for k, v := range params { // scalar params mirrored as TOOL_<NAME>
if sv, ok := scalarStr(v); ok {
c.Env = append(c.Env, "TOOL_"+strings.ToUpper(k)+"="+sv)
}
+128
View File
@@ -0,0 +1,128 @@
//go:build unix
package server
import (
"os"
"sort"
"strings"
)
// Environment of the agent processes. When a process crosses from the server's
// identity to the executor's (the Bash shim, the MCP drop shim, a custom tool), its
// environment is rebuilt from the fixed allowlist below instead of being inherited
// from the server. ScrubSecretEnv and SealProcessEnv still remove the known secrets
// from the server's own environment; they are a deny list, and a credential under
// any other name (a variable an operator or an image adds) would reach the agent.
// The allowlist is the boundary: nothing outside it crosses, whatever its name.
//
// What an agent process sets for its own children afterwards (a nested bash, an
// exported variable) is its own business: the environment is rebuilt only when the
// identity changes.
// executorEnvNames are the server variables an agent process keeps: the search
// path and locale, the temp directory, the non-interactive defaults Norma sets, the
// proxy and CA trust variables the agents pass to their shells (agent.proxyEnv),
// the browser location of the image (Dockerfile), and the shim's own non-secret
// variables. HOME and XDG_CACHE_HOME are replaced by the executor's home.
var executorEnvNames = nameSet(
"PATH", "HOME", "XDG_CACHE_HOME",
"LANG", "LANGUAGE", "TERM", "TZ", "TMPDIR",
"PAGER", "GIT_PAGER", "SYSTEMD_PAGER", "DEBIAN_FRONTEND",
"HTTP_PROXY", "HTTPS_PROXY", "http_proxy", "https_proxy",
"ALL_PROXY", "all_proxy", "NO_PROXY", "no_proxy", "NODE_USE_ENV_PROXY",
"SSL_CERT_FILE", "CURL_CA_BUNDLE", "REQUESTS_CA_BUNDLE", "GIT_SSL_CAINFO", "NODE_EXTRA_CA_CERTS",
"PLAYWRIGHT_BROWSERS_PATH",
envExecBash, envExecUID, envExecGID, envExecHome,
)
func nameSet(names ...string) map[string]bool {
m := make(map[string]bool, len(names))
for _, n := range names {
m[n] = true
}
return m
}
// executorEnvPrefixes are kept by prefix: the locale categories.
var executorEnvPrefixes = []string{"LC_"}
// envExecKeep lists, comma-separated, names the drop shim keeps besides the
// allowlist: the variables an MCP stdio server is given in its configuration. It is
// not kept itself.
const envExecKeep = "ARTEX_EXEC_KEEP"
// executorEnvAllowed reports whether the variable name crosses to an agent process.
func executorEnvAllowed(name string) bool {
if executorEnvNames[name] {
return true
}
for _, p := range executorEnvPrefixes {
if strings.HasPrefix(name, p) {
return true
}
}
return false
}
// executorEnv returns the entries of env an agent process keeps: the allowlisted
// names and the names in keep, in their order. Malformed entries are dropped.
func executorEnv(env []string, keep map[string]bool) []string {
out := make([]string, 0, len(env))
for _, kv := range env {
name, _, ok := strings.Cut(kv, "=")
if !ok || name == "" {
continue
}
if executorEnvAllowed(name) || keep[name] {
out = append(out, kv)
}
}
return out
}
// sealExecutorEnv replaces the environment of the calling process, a shim about to
// become an agent process, with executorEnv of it and the names in envExecKeep.
func sealExecutorEnv() error {
keep := map[string]bool{}
for _, n := range strings.Split(os.Getenv(envExecKeep), ",") {
if n = strings.TrimSpace(n); n != "" {
keep[n] = true
}
}
env := executorEnv(os.Environ(), keep)
os.Clearenv()
for _, kv := range env {
k, v, _ := strings.Cut(kv, "=")
if err := os.Setenv(k, v); err != nil {
return err
}
}
return nil
}
// agentProcessEnv is the base environment of a custom tool: the allowlist when it
// runs as the executor, the server's environment otherwise.
func agentProcessEnv(p ExecPrivilege) []string {
if !p.Enabled {
return os.Environ()
}
return executorEnv(os.Environ(), nil)
}
// withExecKeep adds envExecKeep to the configured environment of an MCP stdio
// server that runs as the executor, so the drop shim keeps those variables.
func withExecKeep(p ExecPrivilege, env map[string]string) map[string]string {
if !p.Enabled || len(env) == 0 {
return env
}
names := make([]string, 0, len(env))
out := make(map[string]string, len(env)+1)
for k, v := range env {
names = append(names, k)
out[k] = v
}
sort.Strings(names)
out[envExecKeep] = strings.Join(names, ",")
return out
}
+75
View File
@@ -0,0 +1,75 @@
//go:build unix
package server
import (
"os"
"strings"
"testing"
)
// The allowlist keeps the named and prefixed variables and the extra names, and
// drops everything else, secrets under known and unknown names alike.
func TestExecutorEnvAllowlist(t *testing.T) {
env := []string{
"PATH=/usr/bin", "LANG=C.UTF-8", "LC_ALL=C", "HTTP_PROXY=http://127.0.0.1:8788",
"SSL_CERT_FILE=/ca.pem", "PLAYWRIGHT_BROWSERS_PATH=/pw", envExecHome + "=/data/exec-home",
"ARTEX_PG_DSN=postgres://artex:pw@postgres/artex", "OPENAI_API_KEY=sk-x",
"MY_COMPANY_TOKEN=t0k3n", "ARTEX_NOISE_PROFILE=quiet", "MCP_TOKEN=abc",
envExecKeep + "=MCP_TOKEN", "malformed", "=empty-name",
}
got := strings.Join(executorEnv(env, map[string]bool{"MCP_TOKEN": true}), "|")
want := strings.Join([]string{
"PATH=/usr/bin", "LANG=C.UTF-8", "LC_ALL=C", "HTTP_PROXY=http://127.0.0.1:8788",
"SSL_CERT_FILE=/ca.pem", "PLAYWRIGHT_BROWSERS_PATH=/pw", envExecHome + "=/data/exec-home",
"MCP_TOKEN=abc",
}, "|")
if got != want {
t.Fatalf("executorEnv =\n%s\nwant\n%s", got, want)
}
for _, k := range secretEnv {
if executorEnvAllowed(k) {
t.Errorf("secret variable %s is on the allowlist", k)
}
}
}
// withExecKeep names the configured variables for the shim, only for an executor.
func TestWithExecKeep(t *testing.T) {
env := map[string]string{"B_TOKEN": "2", "A_TOKEN": "1"}
got := withExecKeep(ExecPrivilege{Enabled: true, UID: 10001, GID: 10001}, env)
if got[envExecKeep] != "A_TOKEN,B_TOKEN" || got["A_TOKEN"] != "1" || got["B_TOKEN"] != "2" || len(got) != 3 {
t.Fatalf("withExecKeep = %v", got)
}
if _, ok := env[envExecKeep]; ok {
t.Fatal("withExecKeep changed its argument")
}
if got := withExecKeep(ExecPrivilege{}, env); len(got) != 2 {
t.Fatalf("a disabled drop must leave the env as is: %v", got)
}
if got := withExecKeep(ExecPrivilege{Enabled: true}, nil); got != nil {
t.Fatalf("no configured env: %v", got)
}
}
// agentProcessEnv filters only for an executor.
func TestAgentProcessEnv(t *testing.T) {
t.Setenv("ARTEX_ISOLATION_CANARY", "canary")
has := func(env []string) bool {
for _, kv := range env {
if strings.HasPrefix(kv, "ARTEX_ISOLATION_CANARY=") {
return true
}
}
return false
}
if !has(agentProcessEnv(ExecPrivilege{})) {
t.Fatal("with the drop disabled the server environment is passed as before")
}
if has(agentProcessEnv(ExecPrivilege{Enabled: true, UID: 10001, GID: 10001})) {
t.Fatal("an executor process must not inherit a variable outside the allowlist")
}
if len(agentProcessEnv(ExecPrivilege{Enabled: true})) == 0 && os.Getenv("PATH") != "" {
t.Fatal("PATH must cross")
}
}
+5
View File
@@ -4,6 +4,7 @@ package server
import (
"errors"
"os"
"os/exec"
)
@@ -19,6 +20,10 @@ func serverIsRoot() bool { return false }
func applyExecCredential(c *exec.Cmd, p ExecPrivilege) {}
func agentProcessEnv(p ExecPrivilege) []string { return os.Environ() }
func withExecKeep(p ExecPrivilege, env map[string]string) map[string]string { return env }
func prepareExecScript(p ExecPrivilege, path string) error { return nil }
func shareRestoredTaskDir(p ExecPrivilege, taskDir string) error { return nil }
+122 -1
View File
@@ -74,7 +74,7 @@ func newExecRootFixture(t *testing.T) *execRootFixture {
t.Fatal(err)
}
// PrepareExecPrivilege sets them process-wide.
t.Cleanup(func() { agent.SetRunDirMaker(nil); agent.SetSpillDirMaker(nil, nil) })
t.Cleanup(func() { agent.SetRunDirMaker(nil); agent.SetSpillDirMaker(nil, nil); agent.SetFileToolIdentity(nil) })
if err := PrepareExecPrivilege(&priv, dataDir, "", ""); err != nil {
t.Fatal(err)
}
@@ -738,6 +738,127 @@ func TestNormaTaskDirPlantedByExecutorAsRoot(t *testing.T) {
}
}
// envNames returns the variable names of an `env` listing.
func envNames(out string) map[string]string {
m := map[string]string{}
for _, line := range strings.Split(out, "\n") {
if k, v, ok := strings.Cut(line, "="); ok && k != "" && !strings.ContainsAny(k, " \t") {
m[k] = v
}
}
return m
}
// checkExecutorEnv asserts what crosses to an executor process: no variable
// outside the allowlist (the canary and a secret name set in the server), the
// executor's HOME, and the extra names wanted.
func checkExecutorEnv(t *testing.T, path, out, home string, want ...string) {
t.Helper()
got := envNames(out)
for k := range got {
if !executorEnvAllowed(k) && !strings.HasPrefix(k, "TOOL_") && k != "PWD" && k != "SHLVL" && k != "_" && k != "OLDPWD" && !slicesContains(want, k) {
t.Errorf("%s: %s crossed to the executor", path, k)
}
}
for _, k := range []string{"ARTEX_ISOLATION_CANARY", "OPENAI_API_KEY", envExecKeep} {
if _, ok := got[k]; ok {
t.Errorf("%s: %s reached the executor", path, k)
}
}
if got["HOME"] != home {
t.Errorf("%s: HOME=%q, want %q", path, got["HOME"], home)
}
for _, k := range append([]string{"PATH"}, want...) {
if _, ok := got[k]; !ok {
t.Errorf("%s: %s missing from the executor environment:\n%s", path, k, out)
}
}
}
func slicesContains(s []string, v string) bool {
for _, x := range s {
if x == v {
return true
}
}
return false
}
// Each executor path starts its process with the allowlisted environment: a
// variable of the server outside the allowlist does not cross, whatever its name;
// what the caller passes on purpose (Bash env, custom-tool parameters, the
// configured env of an MCP server) does. A nested shell keeps what the agent set.
func TestExecutorEnvAllowlistAsRoot(t *testing.T) {
f := newExecRootFixture(t)
t.Setenv("ARTEX_ISOLATION_CANARY", "canary-value")
t.Setenv("OPENAI_API_KEY", "sk-canary")
if err := InstallBashShim(f.priv); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(filepath.SplitList(os.Getenv("PATH"))[0]) })
// Bash (Norma: exec.Command("bash") with the server env and the agent's BashEnv).
in, _ := json.Marshal(map[string]string{"command": "env; echo NESTED=$(AGENT_SET=1 bash -c 'echo $AGENT_SET')"})
res, err := actool.NewBash().Call(context.Background(), in, &actool.ToolContext{WorkingDir: f.work, Env: []string{"HTTP_PROXY=http://127.0.0.1:9"}})
if err != nil {
t.Fatal(err)
}
out := res.Flatten()
checkExecutorEnv(t, "bash shim", out, f.home, "HTTP_PROXY", "PAGER", "NESTED")
if !strings.Contains(out, "NESTED=1") {
t.Errorf("a nested shell lost the variable the agent set:\n%s", out)
}
// Custom tool.
py, err := exec.LookPath("python3")
if err != nil {
t.Fatalf("python3 is needed for the custom-tool path: %v", err)
}
out, err = execPython(context.Background(), py, "probe", `import os
for k, v in os.environ.items():
print(k + "=" + v)`, map[string]any{"x": "1"}, f.work, []string{"HTTP_PROXY=http://127.0.0.1:9"}, 30*time.Second, f.priv)
if err != nil {
t.Fatal(err)
}
checkExecutorEnv(t, "custom tool", out, f.home, "HTTP_PROXY", "TOOL_X")
// MCP stdio: mcp.NewStdioClient sets the server env plus the configured one.
command, args, err := execDropCommand(f.priv, "env", nil)
if err != nil {
t.Fatal(err)
}
c := exec.Command(command, args...)
c.Dir = f.work
c.Env = os.Environ()
for k, v := range withExecKeep(f.priv, map[string]string{"MCP_TOKEN": "configured"}) {
c.Env = append(c.Env, k+"="+v)
}
b, err := c.CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, b)
}
checkExecutorEnv(t, "mcp stdio", string(b), f.home, "MCP_TOKEN")
if envNames(string(b))["MCP_TOKEN"] != "configured" {
t.Errorf("mcp stdio: the configured MCP_TOKEN did not cross:\n%s", b)
}
}
// PrepareExecPrivilege gives the agent's file tools the executor identity: through
// it, a root-only file is refused by the kernel.
func TestPrepareExecPrivilegeSetsFileToolIdentityAsRoot(t *testing.T) {
f := newExecRootFixture(t)
var readErr error
if err := agent.RunAsFileToolIdentity(func() { _, readErr = os.ReadFile(f.secret) }); err != nil {
t.Fatal(err)
}
if !errors.Is(readErr, fs.ErrPermission) {
t.Fatalf("read of a root-only file with the file tool identity: %v, want permission denied", readErr)
}
if _, err := os.ReadFile(f.secret); err != nil {
t.Fatalf("the server itself must still read the file: %v", err)
}
}
// applyExecCredential sets the credential and the executor's HOME; the later env
// entry wins in exec.Cmd. Runs without root: nothing is started.
func TestApplyExecCredentialSetsExecutorHome(t *testing.T) {
+46 -3
View File
@@ -277,6 +277,13 @@ func PrepareExecPrivilege(p *ExecPrivilege, dataDir, keyDir, configPath string)
func(base, dir string) error { return makeSpillDir(base, dir, gid) },
func(dir string) error { return shareSpillFiles(dir, gid) },
)
// The file tools open the paths the agent names inside the server: they do it as
// the executor too (agent/filetools_identity.go).
if agent.FileToolIdentitySupported() {
agent.SetFileToolIdentity(&agent.FileToolIdentity{UID: p.UID, GID: p.GID})
} else {
log.Printf("[exec] the agent's file tools keep the server's identity on this platform: only their path confinement applies")
}
if err := prepareExecHome(*p, dataDir); err != nil {
return fmt.Errorf("prepare the executor home: %w", err)
}
@@ -295,12 +302,20 @@ func PrepareExecPrivilege(p *ExecPrivilege, dataDir, keyDir, configPath string)
}
path := filepath.Join(keyDir, name)
if _, err := os.Stat(path); err == nil {
if err := ownByRoot(path); err != nil {
return fmt.Errorf("restrict %s to root: %w", path, err)
}
if err := os.Chmod(path, 0o600); err != nil {
return fmt.Errorf("restrict %s to root: %w", path, err)
}
}
}
if configPath != "" {
if _, err := os.Stat(configPath); err == nil {
if err := ownByRoot(configPath); err != nil {
return fmt.Errorf("restrict %s to root: %w", configPath, err)
}
}
if fi, err := os.Stat(configPath); err == nil && fi.Mode().Perm()&0o077 != 0 {
if err := os.Chmod(configPath, 0o600); err != nil {
return fmt.Errorf("restrict %s to root: %w", configPath, err)
@@ -315,6 +330,26 @@ func PrepareExecPrivilege(p *ExecPrivilege, dataDir, keyDir, configPath string)
return nil
}
// ownByRoot makes root the owner of a key or config file that another user owns:
// mode 0600 alone leaves the file readable by its owner, which could be the
// executor uid (a file created by an agent process, or on the host under the uid
// the executor has in the container).
func ownByRoot(path string) error {
fi, err := os.Stat(path)
if err != nil {
return err
}
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok || st.Uid == 0 {
return nil
}
if err := os.Chown(path, 0, -1); err != nil {
return err
}
log.Printf("[exec] %s was owned by uid %d; it is now owned by root", path, st.Uid)
return nil
}
// runDirMode is the mode of a run directory: group-writable, and setgid so what is
// created in it keeps the executor group.
const runDirMode = 0o775 | os.ModeSetgid
@@ -576,7 +611,8 @@ func InstallBashShim(p ExecPrivilege) error {
// RunExecShimIfRequested makes the process a privilege-drop shim when it was started
// as one: as bash through the PATH shim, or as artex __artex-exec-drop for an MCP
// stdio command. It exits or execs the target; it returns only for an ordinary start.
// stdio command; or the isolation self-test probe (isolation_unix.go). It exits or
// execs the target; it returns only for an ordinary start.
func RunExecShimIfRequested(argv []string) {
if len(argv) == 0 {
return
@@ -587,6 +623,9 @@ func RunExecShimIfRequested(argv []string) {
if len(argv) >= 2 && argv[1] == execDropArg {
runDropShim(argv[2:])
}
if len(argv) >= 2 && argv[1] == isolationProbeArg {
runIsolationProbeMain()
}
}
func runBashShim(argv []string) {
@@ -601,10 +640,14 @@ func runBashShim(argv []string) {
}
// enterExecutor drops to uid and gid. When this call performs the switch (not in a
// nested shell already running as the executor, whose HOME the agent may have set),
// HOME and XDG_CACHE_HOME move to the executor's home first.
// nested shell already running as the executor, whose environment the agent may
// have set), the environment is first rebuilt from the allowlist (execenv_unix.go)
// and HOME and XDG_CACHE_HOME move to the executor's home.
func enterExecutor(uid, gid int) error {
if os.Getuid() != uid || os.Getgid() != gid {
if err := sealExecutorEnv(); err != nil {
return err
}
for _, kv := range execHomeEnv() {
k, v, _ := strings.Cut(kv, "=")
if err := os.Setenv(k, v); err != nil {
+112
View File
@@ -0,0 +1,112 @@
package server
import (
"fmt"
"log"
"strings"
)
// Startup self-test of agent isolation. Before the stores open, the server checks
// that what an agent can run really is isolated from it: IsolationSelfTest
// (isolation_unix.go) starts a probe through the Bash shim, the way the agent's Bash
// starts, and checks from inside it that it runs as the executor, cannot read the
// server's secrets or its process, sees only the allowlisted environment and cannot
// open a database session without the server's password; it also checks that the
// file tools cannot read the secrets. EnforceIsolation then decides the start:
//
// - The self-test passes: the server starts isolated.
// - Linux with the privilege drop on (the server is root, as in the Docker image):
// isolation is expected there and the self-test shows it is broken, so the start
// stops. The development escape -dev-insecure-no-isolation, off by default, lets
// it go on degraded.
// - Anywhere else (not Linux, the server is not root, or -exec-uid=-1): isolation
// cannot be built on that host, so the server starts degraded without any flag.
// The systemd unit (non-root), dev.sh, Windows and macOS start as before.
//
// A degraded start logs a warning banner that says what is exposed, and reports the
// state in GET /api/health to a signed-in caller, which the web UI shows as a
// banner on every page.
// DevNoIsolationFlag is the command-line flag of the development escape.
const DevNoIsolationFlag = "dev-insecure-no-isolation"
// The isolation modes the server reports.
const (
IsolationIsolated = "isolated"
IsolationDegraded = "degraded"
)
// IsolationStatus is the agent isolation the server runs with.
type IsolationStatus struct {
Mode string `json:"mode"`
// Reason is the self-test failure of a degraded start.
Reason string `json:"reason,omitempty"`
}
// WithIsolationStatus records the result of EnforceIsolation for GET /api/health.
func WithIsolationStatus(st IsolationStatus) Option {
return func(s *Server) { s.isolation = st }
}
// IsolationTargets are the server's paths and database the self-test checks the
// executor cannot reach.
type IsolationTargets struct {
DataDir string
KeyDir string
ConfigPath string
SkillDir string
// DSN is the server's database DSN; only its hosts, ports, user and database
// name reach the probe, never its password.
DSN string
}
// EnforceIsolation decides the start from the self-test result, the platform goos
// and the privilege drop p (see the rules above). It returns the status to report,
// or an error when the start must stop.
func EnforceIsolation(goos string, p ExecPrivilege, selfTest error, devEscape bool) (IsolationStatus, error) {
if selfTest == nil {
log.Printf("[isolation] self-test passed: agent processes and file tools run as the executor and cannot reach the server's secrets")
if devEscape {
log.Printf("[isolation] -%s is set and not needed: the self-test passed", DevNoIsolationFlag)
}
return IsolationStatus{Mode: IsolationIsolated}, nil
}
if isolationRequired(goos, p) && !devEscape {
return IsolationStatus{}, fmt.Errorf("agent isolation self-test failed: %v. A Linux server running as root with the privilege drop does not start without agent isolation: fix the cause above, or pass -%s for development only", selfTest, DevNoIsolationFlag)
}
logDegradedBanner(selfTest, isolationRequired(goos, p))
return IsolationStatus{Mode: IsolationDegraded, Reason: selfTest.Error()}, nil
}
// isolationRequired reports whether the host can isolate agents, so that a failed
// self-test means broken isolation: Linux, with the privilege drop on (root).
func isolationRequired(goos string, p ExecPrivilege) bool {
return goos == "linux" && p.Enabled
}
// degradedBanner is the warning a degraded start logs. escaped is set when the start
// went on only through the development escape.
func degradedBanner(cause error, escaped bool) []string {
bar := strings.Repeat("!", 78)
why := "!! This host cannot isolate agents: that needs Linux and a root server with the" +
"\n!! privilege drop (-exec-uid), as in the Docker image."
if escaped {
why = "!! -" + DevNoIsolationFlag + " is set. DEVELOPMENT ONLY."
}
lines := []string{bar, "!! AGENT ISOLATION IS DEGRADED."}
lines = append(lines, strings.Split(why, "\n")...)
return append(lines,
"!! Self-test failure: "+cause.Error(),
"!! Agent shells, custom tools, MCP stdio servers and the file tools may read the",
"!! server's keys (JWT signing key, secret key), its config, its database",
"!! credentials and its process, and may change skills and tools on disk.",
"!! Do not use this mode on a server that holds real targets or credentials.",
bar,
)
}
func logDegradedBanner(cause error, escaped bool) {
for _, line := range degradedBanner(cause, escaped) {
log.Print("[isolation] " + line)
}
}
+9
View File
@@ -0,0 +1,9 @@
//go:build !unix
package server
// IsolationSelfTest fails: agent processes cannot run as another identity on this
// platform, so they are never isolated from the server.
func IsolationSelfTest(p ExecPrivilege, t IsolationTargets) error {
return errExecUnsupported
}
+240
View File
@@ -0,0 +1,240 @@
//go:build linux
package server
import (
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
)
// isolationRootFixture is newExecRootFixture with the Bash shim installed, a key
// directory like /app/keys (0700, jwt.key and secret.key 0600) and a 0600 config.
type isolationRootFixture struct {
*execRootFixture
keyDir, config string
}
func newIsolationRootFixture(t *testing.T) *isolationRootFixture {
t.Helper()
f := newExecRootFixture(t) // skips without root
if err := InstallBashShim(f.priv); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(filepath.SplitList(os.Getenv("PATH"))[0]) })
keyDir := filepath.Join(f.data, "..", filepath.Base(f.data)+"-keys")
if err := os.Mkdir(keyDir, 0o700); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(keyDir) })
for _, name := range []string{"jwt.key", "secret.key"} {
if err := os.WriteFile(filepath.Join(keyDir, name), []byte("key-"+name), 0o600); err != nil {
t.Fatal(err)
}
}
// The config file lives in a root-only directory too (/app in the image).
config := filepath.Join(keyDir, "config.json")
if err := os.WriteFile(config, []byte(`{"database":{"dsn":"postgres://u:pw@h/db"}}`), 0o600); err != nil {
t.Fatal(err)
}
return &isolationRootFixture{execRootFixture: f, keyDir: keyDir, config: config}
}
func (f *isolationRootFixture) targets() IsolationTargets {
return IsolationTargets{DataDir: f.data, KeyDir: f.keyDir, ConfigPath: f.config, DSN: os.Getenv("ARTEX_PG_DSN")}
}
// The deployment the self-test is for: root server, executor drop, shim, root-only
// keys and config. It passes, through the real Bash shim, and with ARTEX_PG_DSN set
// it also finds that the executor cannot open a session on that database without
// the password (PostgreSQL asks for one on TCP).
func TestIsolationSelfTestPassesAsRoot(t *testing.T) {
f := newIsolationRootFixture(t)
if os.Getenv("ARTEX_PG_DSN") == "" {
t.Log("ARTEX_PG_DSN unset: the database check is not exercised")
}
if err := IsolationSelfTest(f.priv, f.targets()); err != nil {
t.Fatalf("self-test on an isolated setup: %v", err)
}
}
// A secret the executor can read fails the self-test, for the Bash path and the
// file tool path both.
func TestIsolationSelfTestDetectsReadableSecretAsRoot(t *testing.T) {
f := newIsolationRootFixture(t)
key := filepath.Join(f.keyDir, "jwt.key")
if err := os.Chmod(f.keyDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Chmod(key, 0o644); err != nil {
t.Fatal(err)
}
err := IsolationSelfTest(f.priv, f.targets())
if err == nil || !strings.Contains(err.Error(), "Bash can read "+key) || !strings.Contains(err.Error(), "the file tools can read "+key) {
t.Fatalf("self-test with a world-readable key: %v", err)
}
}
// Without the shim on PATH, "bash" is the real bash and runs as root: the self-test
// sees it.
func TestIsolationSelfTestDetectsMissingShimAsRoot(t *testing.T) {
f := newExecRootFixture(t)
err := IsolationSelfTest(f.priv, IsolationTargets{DataDir: f.data})
if err == nil || !strings.Contains(err.Error(), "Bash runs as uid 0") {
t.Fatalf("self-test without the Bash shim: %v", err)
}
}
// A key file and a config file owned by the executor uid stay readable by it after a
// chmod 0600: PrepareExecPrivilege makes root their owner, and the executor can then
// read neither (the self-test passes, and a direct read as the executor is denied).
func TestPrepareExecPrivilegeOwnsKeysAndConfigAsRoot(t *testing.T) {
f := newExecRootFixture(t)
keyDir := filepath.Join(f.tmp, "keys")
if err := os.Mkdir(keyDir, 0o755); err != nil {
t.Fatal(err)
}
key, config := filepath.Join(keyDir, "jwt.key"), filepath.Join(keyDir, "config.json")
for _, p := range []string{key, config} {
if err := os.WriteFile(p, []byte("secret-"+filepath.Base(p)), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chown(p, DefaultExecUID, DefaultExecGID); err != nil {
t.Fatal(err)
}
}
if err := PrepareExecPrivilege(&f.priv, f.data, keyDir, config); err != nil {
t.Fatal(err)
}
for _, p := range []string{key, config} {
fi, err := os.Stat(p)
if err != nil {
t.Fatal(err)
}
if st := fi.Sys().(*syscall.Stat_t); st.Uid != 0 || fi.Mode().Perm() != 0o600 {
t.Errorf("%s: uid %d mode %v, want root 0600", p, st.Uid, fi.Mode().Perm())
}
}
c := exec.Command("sh", "-c", `for f in "$@"; do if cat "$f" >/dev/null 2>&1; then echo "read $f"; fi; done; echo done`, "sh", key, config)
applyExecCredential(c, f.priv)
out, err := c.CombinedOutput()
if err != nil || string(out) != "done\n" {
t.Fatalf("executor read of the key and config: %v\n%s", err, out)
}
if err := InstallBashShim(f.priv); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(filepath.SplitList(os.Getenv("PATH"))[0]) })
if err := IsolationSelfTest(f.priv, IsolationTargets{DataDir: f.data, KeyDir: keyDir, ConfigPath: config}); err != nil {
t.Fatalf("self-test after PrepareExecPrivilege: %v", err)
}
}
// A data, key or config directory the executor can write fails the self-test: an
// agent process could plant a key, a config or a data tree for the next start.
func TestIsolationSelfTestDetectsWritableDirsAsRoot(t *testing.T) {
f := newIsolationRootFixture(t)
if err := os.Chmod(f.data, 0o777); err != nil {
t.Fatal(err)
}
err := IsolationSelfTest(f.priv, f.targets())
if err == nil || !strings.Contains(err.Error(), "Bash can create files in "+f.data) {
t.Fatalf("self-test with a world-writable data directory: %v", err)
}
if err := os.Chmod(f.data, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Chown(f.keyDir, DefaultExecUID, 0); err != nil {
t.Fatal(err)
}
err = IsolationSelfTest(f.priv, f.targets())
if err == nil || !strings.Contains(err.Error(), "Bash can create files in "+f.keyDir) {
t.Fatalf("self-test with a key directory owned by the executor: %v", err)
}
if entries, _ := os.ReadDir(f.keyDir); len(entries) != 3 {
t.Fatalf("the probe left an entry in the key directory: %v", entries)
}
}
// A data directory whose parent the executor can write (not sticky) can be renamed
// and replaced: the self-test reports it.
func TestIsolationSelfTestDetectsReplaceableParentAsRoot(t *testing.T) {
f := newIsolationRootFixture(t)
open := filepath.Join(f.tmp, "open-parent")
if err := os.Mkdir(open, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Chmod(open, 0o777); err != nil {
t.Fatal(err)
}
data := filepath.Join(open, "data")
if err := os.Mkdir(data, 0o755); err != nil {
t.Fatal(err)
}
tg := f.targets()
tg.DataDir = data
err := IsolationSelfTest(f.priv, tg)
if err == nil || !strings.Contains(err.Error(), "an agent process can replace "+data) {
t.Fatalf("self-test with a replaceable data directory: %v", err)
}
}
// A skill tree an agent process can write fails the self-test: a skill written there
// directly would skip the approval of create_skill and update_skill_file. A tree
// only root can write passes.
func TestIsolationSelfTestDetectsWritableSkillsAsRoot(t *testing.T) {
f := newIsolationRootFixture(t)
skills := filepath.Join(f.tmp, "skills")
skill := filepath.Join(skills, "recon")
if err := os.MkdirAll(skill, 0o755); err != nil {
t.Fatal(err)
}
md := filepath.Join(skill, "SKILL.md")
if err := os.WriteFile(md, []byte("---\nname: recon\n---\n"), 0o644); err != nil {
t.Fatal(err)
}
tg := f.targets()
tg.SkillDir = skills
if err := IsolationSelfTest(f.priv, tg); err != nil {
t.Fatalf("self-test with a root-only skill tree: %v", err)
}
if err := os.Chown(md, DefaultExecUID, DefaultExecGID); err != nil {
t.Fatal(err)
}
err := IsolationSelfTest(f.priv, tg)
if err == nil || !strings.Contains(err.Error(), "can change the skill entry "+md) {
t.Fatalf("self-test with a skill file owned by the executor: %v", err)
}
if err := os.Chown(md, 0, 0); err != nil {
t.Fatal(err)
}
if err := os.Chmod(skills, 0o777); err != nil {
t.Fatal(err)
}
err = IsolationSelfTest(f.priv, tg)
if err == nil || !strings.Contains(err.Error(), "Bash can create files in "+skills) || !strings.Contains(err.Error(), "can change the skill entry "+skills) {
t.Fatalf("self-test with a world-writable skill directory: %v", err)
}
if entries, _ := os.ReadDir(skills); len(entries) != 1 {
t.Fatalf("the probe left an entry in the skill directory: %v", entries)
}
}
// A database that lets the server's user in without a password fails the self-test:
// the probe, as the executor, opens a session on it.
func TestIsolationSelfTestDetectsPasswordlessDatabaseAsRoot(t *testing.T) {
f := newIsolationRootFixture(t)
host, port := fakeTrustPG(t)
tg := f.targets()
tg.DSN = "postgres://artex:server-password@" + host + ":" + strconv.Itoa(int(port)) + "/artex"
err := IsolationSelfTest(f.priv, tg)
if err == nil || !strings.Contains(err.Error(), `database session as "artex"`) {
t.Fatalf("self-test against a passwordless database: %v", err)
}
}
+122
View File
@@ -0,0 +1,122 @@
package server
import (
"bytes"
"encoding/json"
"errors"
"log"
"net/http/httptest"
"strings"
"testing"
)
// captureLog collects what the standard logger writes while fn runs.
func captureLog(t *testing.T, fn func()) string {
t.Helper()
var buf bytes.Buffer
prev, flags := log.Writer(), log.Flags()
log.SetOutput(&buf)
log.SetFlags(0)
defer func() { log.SetOutput(prev); log.SetFlags(flags) }()
fn()
return buf.String()
}
// The start decision, branch by branch.
func TestEnforceIsolation(t *testing.T) {
cause := errors.New("Bash runs as uid 0, not the executor uid 10001")
root := ExecPrivilege{Enabled: true, UID: 10001, GID: 10001} // drop on: the server is root
off := ExecPrivilege{} // not root, or -exec-uid=-1
const banner = "AGENT ISOLATION IS DEGRADED"
var (
st IsolationStatus
err error
)
// Hard failure: Linux, root, self-test failed, no escape. No banner: no start.
out := captureLog(t, func() { st, err = EnforceIsolation("linux", root, cause, false) })
if err == nil || !strings.Contains(err.Error(), cause.Error()) || !strings.Contains(err.Error(), "-"+DevNoIsolationFlag) {
t.Fatalf("linux root: err = %v, want the cause and the flag named", err)
}
if st.Mode != "" || strings.Contains(out, banner) {
t.Fatalf("linux root: status %+v, banner in\n%s", st, out)
}
// Linux, root, with the development escape: degraded, banner names the flag.
out = captureLog(t, func() { st, err = EnforceIsolation("linux", root, cause, true) })
if err != nil || st.Mode != IsolationDegraded || st.Reason != cause.Error() {
t.Fatalf("linux root with the escape: status %+v err %v, want degraded", st, err)
}
for _, want := range []string{banner, "-" + DevNoIsolationFlag, "DEVELOPMENT ONLY", cause.Error(), "keys", "database"} {
if !strings.Contains(out, want) {
t.Errorf("escape banner lacks %q:\n%s", want, out)
}
}
// Degraded without any flag: Linux not root (the systemd unit, dev.sh), Windows,
// macOS (root or not). The banner says why and what is exposed.
for _, c := range []struct {
goos string
p ExecPrivilege
}{{"linux", off}, {"windows", off}, {"darwin", off}, {"darwin", root}} {
out = captureLog(t, func() { st, err = EnforceIsolation(c.goos, c.p, cause, false) })
if err != nil || st.Mode != IsolationDegraded || st.Reason != cause.Error() {
t.Fatalf("%s drop=%v: status %+v err %v, want a degraded start", c.goos, c.p.Enabled, st, err)
}
for _, want := range []string{banner, "cannot isolate agents", cause.Error(), "keys", "database"} {
if !strings.Contains(out, want) {
t.Errorf("%s drop=%v: banner lacks %q:\n%s", c.goos, c.p.Enabled, want, out)
}
}
if strings.Contains(out, "DEVELOPMENT ONLY") {
t.Errorf("%s drop=%v: the degraded banner is not the escape's:\n%s", c.goos, c.p.Enabled, out)
}
}
// A passing self-test starts isolated, escape or not, without a banner.
for _, escape := range []bool{false, true} {
out = captureLog(t, func() { st, err = EnforceIsolation("linux", root, nil, escape) })
if err != nil || st.Mode != IsolationIsolated || !strings.Contains(out, "self-test passed") || strings.Contains(out, banner) {
t.Fatalf("passing self-test (escape %v): status %+v err=%v\n%s", escape, st, err, out)
}
}
}
// GET /api/health reports the isolation status to a signed-in caller only: the web
// UI shows the degraded banner from it.
func TestHealthReportsIsolation(t *testing.T) {
key := []byte("0123456789abcdef0123456789abcdef")
s := &Server{jwtKey: key}
WithIsolationStatus(IsolationStatus{Mode: IsolationDegraded, Reason: "the server is not root"})(s)
w := httptest.NewRecorder()
s.health(w, httptest.NewRequest("GET", "/api/health", nil))
if got := strings.TrimSpace(w.Body.String()); got != `{"ok":true}` {
t.Fatalf("no token: %s, want no isolation status", got)
}
tok, err := signJWT(key)
if err != nil {
t.Fatal(err)
}
r := httptest.NewRequest("GET", "/api/health", nil)
r.Header.Set("Authorization", "Bearer "+tok)
w = httptest.NewRecorder()
s.health(w, r)
var body struct {
Isolation *IsolationStatus `json:"isolation"`
}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil || body.Isolation == nil ||
body.Isolation.Mode != IsolationDegraded || body.Isolation.Reason != "the server is not root" {
t.Fatalf("signed in: %s, want the degraded status", w.Body.String())
}
}
// With the drop disabled (the server is not root, -exec-uid=-1, or no credentials
// on this platform) the self-test reports isolation as unavailable.
func TestIsolationSelfTestUnavailableWithoutDrop(t *testing.T) {
if err := IsolationSelfTest(ExecPrivilege{}, IsolationTargets{}); err == nil {
t.Fatal("a disabled drop must fail the self-test")
}
}
+381
View File
@@ -0,0 +1,381 @@
//go:build unix
package server
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"os/exec"
"path/filepath"
"slices"
"strconv"
"strings"
"syscall"
"time"
"github.com/jackc/pgx/v5"
"github.com/Autumn-27/artex/agent"
)
// isolationProbeArg is the argv marker of the self-test probe:
// artex __artex-isolation-probe, with an isolationProbeSpec on stdin.
const isolationProbeArg = "__artex-isolation-probe"
// isolationProbeTimeout bounds the probe, database attempt included.
const isolationProbeTimeout = 30 * time.Second
// isolationProbeSpec is what the probe checks from inside an executor process.
type isolationProbeSpec struct {
UID uint32 `json:"uid"`
GID uint32 `json:"gid"`
ServerPID int `json:"server_pid"`
// Unreadable are files no agent process may open for reading.
Unreadable []string `json:"unreadable,omitempty"`
// Unwritable are directories no agent process may create an entry in.
Unwritable []string `json:"unwritable,omitempty"`
// DB names the server's database at each host its DSN lists, fallbacks included:
// the real address the server connects to, never one the self-test picks. The
// probe connects to each without a password.
DB []isolationProbeDB `json:"db,omitempty"`
}
type isolationProbeDB struct {
Host string `json:"host"`
Port uint16 `json:"port"`
User string `json:"user"`
Database string `json:"database"`
}
// isolationProbeReport is the probe's answer on stdout.
type isolationProbeReport struct {
Failures []string `json:"failures"`
}
// IsolationSelfTest checks that agent processes and file tools are isolated from
// the server. It returns nil when every check passes, otherwise an error that names
// each failure. With the privilege drop disabled there is nothing to test: agent
// processes run as the server, and that is reported as the failure.
func IsolationSelfTest(p ExecPrivilege, t IsolationTargets) error {
if !p.Enabled {
return errors.New("agent processes run with the server's identity: the privilege drop is off (the server is not root, or -exec-uid is negative)")
}
spec := isolationSpec(p, t)
failures := fileToolIsolationFailures(spec.Unreadable)
if t.DSN != "" && len(spec.DB) == 0 {
failures = append(failures, "the database DSN names no host the self-test can check")
}
failures = append(failures, replaceableDirFailures(spec.Unwritable, p.UID, p.GID)...)
failures = append(failures, skillTreeFailures(t.SkillDir, p.UID, p.GID)...)
rep, err := runIsolationProbe(spec)
if err != nil {
failures = append(failures, fmt.Sprintf("the probe did not run through the Bash shim: %v", err))
} else {
failures = append(failures, rep.Failures...)
}
if len(failures) > 0 {
return errors.New(strings.Join(failures, "; "))
}
return nil
}
// isolationSpec builds the probe's checks from the server's paths.
func isolationSpec(p ExecPrivilege, t IsolationTargets) isolationProbeSpec {
spec := isolationProbeSpec{UID: p.UID, GID: p.GID, ServerPID: os.Getpid()}
if t.KeyDir != "" {
spec.Unreadable = append(spec.Unreadable, filepath.Join(t.KeyDir, "jwt.key"), filepath.Join(t.KeyDir, "secret.key"))
}
if t.ConfigPath != "" {
spec.Unreadable = append(spec.Unreadable, t.ConfigPath)
}
if t.DataDir != "" {
spec.Unreadable = append(spec.Unreadable, filepath.Join(t.DataDir, "traffic", "_ca", "mitmproxy-ca.pem"))
}
// The directories of the keys, the config and the data: an agent process that
// can create an entry there can plant a key, a config or a whole data tree that
// the server uses at its next start.
// The skill directory too: a skill an agent process writes there directly would
// skip the approval of create_skill (see skillTreeFailures).
for _, dir := range []string{t.DataDir, t.KeyDir, configDir(t.ConfigPath), t.SkillDir} {
if dir != "" && !slices.Contains(spec.Unwritable, dir) {
spec.Unwritable = append(spec.Unwritable, dir)
}
}
if t.DSN != "" {
if cfg, err := pgx.ParseConfig(t.DSN); err == nil {
spec.DB = dsnProbeHosts(cfg)
}
}
return spec
}
// dsnProbeHosts lists each distinct host and port of a parsed DSN, the first one
// and its fallbacks (pgx adds a fallback per TLS attempt, so hosts repeat).
func dsnProbeHosts(cfg *pgx.ConnConfig) []isolationProbeDB {
var out []isolationProbeDB
add := func(host string, port uint16) {
if host == "" {
return
}
for _, db := range out {
if db.Host == host && db.Port == port {
return
}
}
out = append(out, isolationProbeDB{Host: host, Port: port, User: cfg.User, Database: cfg.Database})
}
add(cfg.Host, cfg.Port)
for _, fb := range cfg.Fallbacks {
add(fb.Host, fb.Port)
}
return out
}
func configDir(configPath string) string {
if configPath == "" {
return ""
}
return filepath.Dir(configPath)
}
// replaceableDirFailures reports each directory whose parent lets the executor
// rename it and put another in its place: the parent is owned by the executor, or
// the executor can write in it and it is not sticky. The probe checks the
// directories themselves; their parents are checked here from the owners and mode
// bits, since a sticky world-writable parent such as /tmp is writable yet safe.
func replaceableDirFailures(dirs []string, execUID, execGID uint32) []string {
var failures []string
for _, dir := range dirs {
abs, err := filepath.Abs(dir)
if err != nil {
continue
}
parent := filepath.Dir(abs)
fi, err := os.Stat(parent)
if err != nil {
continue
}
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
continue
}
if why := parentReplaceable(fi.Mode(), st.Uid, st.Gid, execUID, execGID); why != "" {
failures = append(failures, fmt.Sprintf("an agent process can replace %s: its parent %s %s", dir, parent, why))
}
}
return failures
}
// parentReplaceable is the decision of replaceableDirFailures for one parent.
func parentReplaceable(mode fs.FileMode, uid, gid, execUID, execGID uint32) string {
if uid == execUID {
return fmt.Sprintf("is owned by the executor uid %d", execUID)
}
if mode&fs.ModeSticky == 0 {
if why := normaTaskDirExecWrite(mode.Perm(), uid, gid, execUID, execGID); why != "" {
return "is writable by the executor and not sticky (" + why + ")"
}
}
return ""
}
// skillTreeMaxFailures caps what skillTreeFailures reports for one tree.
const skillTreeMaxFailures = 20
// skillTreeFailures reports each file and directory under the skill directory that
// an agent process could change: agents load skill files as instructions, so a skill
// written there directly would bypass the operator approval that create_skill and
// update_skill_file need (intercept/persistence.go). It decides from the owners and
// mode bits, without writing anything: an entry owned by the executor uid, or
// writable by the executor gid or by others, is reported. A symlink is judged by its
// target and not descended. The probe checks the directory itself for writes and
// replaceableDirFailures its parent.
func skillTreeFailures(root string, execUID, execGID uint32) []string {
if root == "" {
return nil
}
if _, err := os.Stat(root); err != nil {
return nil
}
var failures []string
_ = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return nil
}
fi, err := os.Stat(path)
if err != nil {
return nil
}
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
if why := normaTaskDirExecWrite(fi.Mode().Perm(), st.Uid, st.Gid, execUID, execGID); why != "" {
failures = append(failures, fmt.Sprintf("an agent process can change the skill entry %s, bypassing the skill approval: %s", path, why))
}
if len(failures) >= skillTreeMaxFailures {
return fs.SkipAll
}
return nil
})
return failures
}
// fileToolIsolationFailures opens each existing secret through the file tool
// identity (agent/filetools_identity.go): the open must be refused.
func fileToolIsolationFailures(secrets []string) []string {
if !agent.FileToolIdentitySupported() {
return []string{"the agent's file tools run with the server's identity on this platform"}
}
var failures []string
for _, path := range secrets {
if _, err := os.Stat(path); err != nil {
continue
}
var openErr error
if err := agent.RunAsFileToolIdentity(func() {
f, err := os.Open(path)
if err == nil {
f.Close()
}
openErr = err
}); err != nil {
failures = append(failures, fmt.Sprintf("the file tool identity cannot be applied: %v", err))
break
}
if openErr == nil {
failures = append(failures, fmt.Sprintf("the file tools can read %s", path))
}
}
return failures
}
// runIsolationProbe starts the probe the way the agent's Bash starts: "bash" looked
// up on PATH, where InstallBashShim put the shim that drops to the executor. bash
// then execs this binary as the probe, which gets the spec on stdin.
func runIsolationProbe(spec isolationProbeSpec) (isolationProbeReport, error) {
var rep isolationProbeReport
self, err := os.Executable()
if err != nil {
return rep, err
}
in, err := json.Marshal(spec)
if err != nil {
return rep, err
}
ctx, cancel := context.WithTimeout(context.Background(), isolationProbeTimeout)
defer cancel()
c := exec.CommandContext(ctx, "bash", "-c", `exec "$0" `+isolationProbeArg, self)
c.Dir = "/"
c.Stdin = bytes.NewReader(in)
var stderr bytes.Buffer
c.Stderr = &stderr
out, err := c.Output()
if err != nil {
return rep, fmt.Errorf("%v: %s", err, strings.TrimSpace(stderr.String()))
}
if err := json.Unmarshal(out, &rep); err != nil {
return rep, fmt.Errorf("unreadable probe report %q: %v", out, err)
}
return rep, nil
}
// runIsolationProbeMain is the probe process: it reads the spec, checks, writes
// the report and exits.
func runIsolationProbeMain() {
var spec isolationProbeSpec
rep := isolationProbeReport{Failures: []string{}}
if err := json.NewDecoder(os.Stdin).Decode(&spec); err != nil {
rep.Failures = append(rep.Failures, fmt.Sprintf("probe: bad spec: %v", err))
} else {
rep.Failures = append(rep.Failures, checkIsolationAsExecutor(spec)...)
}
_ = json.NewEncoder(os.Stdout).Encode(rep)
os.Exit(0)
}
// bashOwnEnv are the variables bash itself exports to the programs it starts.
var bashOwnEnv = nameSet("PWD", "OLDPWD", "SHLVL", "_")
// checkIsolationAsExecutor runs the checks of spec in the calling process and
// returns what failed.
func checkIsolationAsExecutor(spec isolationProbeSpec) []string {
var failures []string
fail := func(format string, args ...any) { failures = append(failures, fmt.Sprintf(format, args...)) }
if uid, euid := os.Getuid(), os.Geteuid(); uid != int(spec.UID) || euid != int(spec.UID) {
fail("Bash runs as uid %d (effective %d), not the executor uid %d", uid, euid, spec.UID)
}
if gid := os.Getgid(); gid != int(spec.GID) {
fail("Bash runs as gid %d, not the executor gid %d", gid, spec.GID)
}
if groups, err := os.Getgroups(); err != nil || len(groups) != 1 || groups[0] != int(spec.GID) {
fail("Bash has the supplementary groups %v, want only the executor gid %d", groups, spec.GID)
}
for _, path := range spec.Unreadable {
if f, err := os.Open(path); err == nil {
f.Close()
fail("Bash can read %s", path)
}
}
if spec.ServerPID > 0 {
if _, err := os.Stat("/proc/self"); err == nil {
for _, name := range []string{"environ", "mem"} {
path := "/proc/" + strconv.Itoa(spec.ServerPID) + "/" + name
if f, err := os.Open(path); err == nil {
f.Close()
fail("Bash can read the server's %s", path)
}
}
}
}
for _, dir := range spec.Unwritable {
if f, err := os.CreateTemp(dir, ".artex-isolation-probe-*"); err == nil {
name := f.Name()
f.Close()
os.Remove(name)
fail("Bash can create files in %s", dir)
}
}
for _, kv := range os.Environ() {
name, _, _ := strings.Cut(kv, "=")
if !executorEnvAllowed(name) && !bashOwnEnv[name] {
fail("the variable %s reached Bash: it is not on the agent environment allowlist", name)
}
}
for i := range spec.DB {
db := &spec.DB[i]
if err := connectWithoutPassword(db); err == nil {
fail("Bash opened a database session as %q on %s:%d without the server's password", db.User, db.Host, db.Port)
}
}
return failures
}
// connectWithoutPassword tries a session on the server's database with its user
// and no password. nil means the database let the executor in.
// The default sslmode (prefer) is kept: TLS first, then plain, as a client in the
// agent's shell would try.
func connectWithoutPassword(db *isolationProbeDB) error {
quote := strings.NewReplacer(`\`, `\\`, `'`, `\'`).Replace
dsn := fmt.Sprintf("host='%s' port=%d user='%s' dbname='%s' connect_timeout=5",
quote(db.Host), db.Port, quote(db.User), quote(db.Database))
cfg, err := pgx.ParseConfig(dsn)
if err != nil {
return err
}
cfg.Password = "" // not even one from a password file in the executor's home
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
conn, err := pgx.ConnectConfig(ctx, cfg)
if err != nil {
return err
}
conn.Close(context.Background())
return nil
}
+283
View File
@@ -0,0 +1,283 @@
//go:build unix
package server
import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"net"
"os"
"path/filepath"
"strings"
"testing"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgproto3"
)
// fakeTrustPG is a PostgreSQL wire endpoint that lets any user in without a
// password, like a server with trust authentication. It returns its address.
func fakeTrustPG(t *testing.T) (host string, port uint16) {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
go func(conn net.Conn) {
defer conn.Close()
be := pgproto3.NewBackend(conn, conn)
msg, err := be.ReceiveStartupMessage()
for err == nil {
switch msg.(type) {
case *pgproto3.SSLRequest, *pgproto3.GSSEncRequest:
if _, err = conn.Write([]byte("N")); err == nil {
msg, err = be.ReceiveStartupMessage()
}
continue
}
break
}
if err != nil {
return
}
be.Send(&pgproto3.AuthenticationOk{})
be.Send(&pgproto3.ParameterStatus{Name: "server_version", Value: "16.0"})
be.Send(&pgproto3.BackendKeyData{ProcessID: 1, SecretKey: []byte{0, 0, 0, 2}})
be.Send(&pgproto3.ReadyForQuery{TxStatus: 'I'})
if be.Flush() != nil {
return
}
for {
if _, err := be.Receive(); err != nil {
return
}
}
}(conn)
}
}()
addr := ln.Addr().(*net.TCPAddr)
return addr.IP.String(), uint16(addr.Port)
}
func hasFailure(failures []string, sub string) bool {
for _, f := range failures {
if strings.Contains(f, sub) {
return true
}
}
return false
}
// The probe's checks, run in the test process (not as the executor): each kind of
// exposure is reported. The process can read the file, write the directory, carries
// a variable outside the allowlist, is not the executor and reaches a database that
// takes no password.
func TestCheckIsolationReportsEachExposure(t *testing.T) {
dir := t.TempDir()
secret := filepath.Join(dir, "jwt.key")
if err := os.WriteFile(secret, []byte("k"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("ARTEX_ISOLATION_CANARY", "canary")
host, port := fakeTrustPG(t)
failures := checkIsolationAsExecutor(isolationProbeSpec{
UID: 10001, GID: 10001, ServerPID: os.Getpid(),
Unreadable: []string{secret, filepath.Join(dir, "absent.key")},
Unwritable: []string{dir},
DB: []isolationProbeDB{{Host: host, Port: port, User: "artex", Database: "artex"}},
})
for _, want := range []string{
"not the executor uid 10001",
"Bash can read " + secret,
"Bash can create files in " + dir,
"ARTEX_ISOLATION_CANARY reached Bash",
`database session as "artex"`,
} {
if !hasFailure(failures, want) {
t.Errorf("no failure %q in %q", want, failures)
}
}
if hasFailure(failures, "absent.key") {
t.Errorf("a missing file is not an exposure: %q", failures)
}
if entries, _ := os.ReadDir(dir); len(entries) != 1 {
t.Errorf("the write check left a file behind: %v", entries)
}
}
// A database that asks for a password, or none listening, is not an exposure.
func TestConnectWithoutPasswordRefused(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
port := uint16(ln.Addr().(*net.TCPAddr).Port)
ln.Close() // nothing listens there now
if err := connectWithoutPassword(&isolationProbeDB{Host: "127.0.0.1", Port: port, User: "artex", Database: "artex"}); err == nil {
t.Fatal("no database listening, yet a session opened")
}
}
// Against the real test database: the server's user without its password is
// refused by PostgreSQL's authentication (it asks for the password on TCP).
func TestConnectWithoutPasswordRealPG(t *testing.T) {
dsn := os.Getenv("ARTEX_PG_DSN")
if dsn == "" {
t.Skip("ARTEX_PG_DSN not set")
}
spec := isolationSpec(ExecPrivilege{Enabled: true, UID: 10001, GID: 10001}, IsolationTargets{DSN: dsn})
if len(spec.DB) == 0 {
t.Fatal("no database in the spec")
}
for i := range spec.DB {
err := connectWithoutPassword(&spec.DB[i])
var pgErr *pgconn.PgError
if !errors.As(err, &pgErr) || (pgErr.Code != "28P01" && pgErr.Code != "28000") {
t.Fatalf("connection without password to %s:%d: %v, want an authentication failure (28P01/28000)", spec.DB[i].Host, spec.DB[i].Port, err)
}
t.Logf("%s:%d refused as expected: SQLSTATE %s", spec.DB[i].Host, spec.DB[i].Port, pgErr.Code)
}
}
// The probe tries the hosts the DSN names, every one of them, and nothing else: no
// loopback of its own. A DSN it cannot use fails the self-test instead of skipping
// the database check.
func TestIsolationSpecUsesTheDSNHosts(t *testing.T) {
p := ExecPrivilege{Enabled: true, UID: 10001, GID: 10001}
for _, c := range []struct {
dsn string
want string
}{
{"postgres://artex:pw@db.internal:6543/artex?sslmode=disable", "db.internal:6543"},
{"postgres://artex:pw@db1.internal:6543,db2.internal:6544/artex?sslmode=prefer", "db1.internal:6543 db2.internal:6544"},
{"host=10.20.30.40 port=5433 user=artex password=pw dbname=artex", "10.20.30.40:5433"},
{"host=/var/run/postgresql user=artex dbname=artex", "/var/run/postgresql:5432"},
} {
var got []string
for _, db := range isolationSpec(p, IsolationTargets{DSN: c.dsn}).DB {
got = append(got, fmt.Sprintf("%s:%d", db.Host, db.Port))
if db.User != "artex" || db.Database != "artex" {
t.Errorf("%s: user %q database %q", c.dsn, db.User, db.Database)
}
}
if strings.Join(got, " ") != c.want {
t.Errorf("%s: probe hosts %v, want %s", c.dsn, got, c.want)
}
}
err := IsolationSelfTest(p, IsolationTargets{DSN: "postgres://%zz"})
if err == nil || !strings.Contains(err.Error(), "names no host the self-test can check") {
t.Fatalf("self-test with an unusable DSN: %v", err)
}
}
// The parent rule: owned by the executor, or writable by it without the sticky bit,
// lets it replace the directory; a sticky world-writable parent (/tmp) does not.
func TestParentReplaceable(t *testing.T) {
const u, g = 10001, 10001
for _, c := range []struct {
name string
mode fs.FileMode
uid, gid uint32
want bool
}{
{"root 0755", fs.ModeDir | 0o755, 0, 0, false},
{"owned by the executor", fs.ModeDir | 0o700, u, 0, true},
{"group-writable executor gid", fs.ModeDir | 0o775, 0, g, true},
{"group-writable other gid", fs.ModeDir | 0o775, 0, 0, false},
{"world-writable", fs.ModeDir | 0o777, 0, 0, true},
{"world-writable sticky", fs.ModeDir | fs.ModeSticky | 0o777, 0, 0, false},
{"host user 0755", fs.ModeDir | 0o755, 1000, 1000, false},
} {
if got := parentReplaceable(c.mode, c.uid, c.gid, u, g) != ""; got != c.want {
t.Errorf("%s: replaceable=%v, want %v", c.name, got, c.want)
}
}
}
// A skill tree is judged from owners and mode bits: an entry the executor owns, or
// can write through its group or as other, is reported; a symlink by its target.
// Runs without root: the test user stands in for the executor or for root.
func TestSkillTreeFailures(t *testing.T) {
root := t.TempDir()
skill := filepath.Join(root, "recon")
if err := os.Mkdir(skill, 0o755); err != nil {
t.Fatal(err)
}
md := filepath.Join(skill, "SKILL.md")
if err := os.WriteFile(md, []byte("---\nname: recon\n---\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chmod(root, 0o755); err != nil {
t.Fatal(err)
}
me := uint32(os.Getuid())
other := me + 54321 // an executor that owns nothing here
if got := skillTreeFailures(root, other, other); len(got) != 0 {
t.Fatalf("a tree only its owner can write: %q", got)
}
if got := skillTreeFailures(root, me, other); len(got) != 3 || !hasFailure(got, md) {
t.Fatalf("a tree owned by the executor uid: %q, want all three entries", got)
}
if err := os.Chmod(md, 0o666); err != nil {
t.Fatal(err)
}
if got := skillTreeFailures(root, other, other); len(got) != 1 || !hasFailure(got, md) || !hasFailure(got, "writable by others") {
t.Fatalf("a world-writable skill file: %q", got)
}
if err := os.Chmod(md, 0o644); err != nil {
t.Fatal(err)
}
outside := filepath.Join(t.TempDir(), "evil.md")
if err := os.WriteFile(outside, []byte("x"), 0o666); err != nil {
t.Fatal(err)
}
if err := os.Chmod(outside, 0o666); err != nil {
t.Fatal(err)
}
link := filepath.Join(skill, "linked.md")
if err := os.Symlink(outside, link); err != nil {
t.Fatal(err)
}
if got := skillTreeFailures(root, other, other); len(got) != 1 || !hasFailure(got, link) {
t.Fatalf("a symlink to a world-writable file: %q", got)
}
if got := skillTreeFailures(filepath.Join(root, "absent"), me, me); got != nil {
t.Fatalf("a missing skill directory: %q", got)
}
}
// The spec carries the server's paths and the database name, never its password.
func TestIsolationSpecHasNoPassword(t *testing.T) {
spec := isolationSpec(ExecPrivilege{Enabled: true, UID: 10001, GID: 10002}, IsolationTargets{
DataDir: "/app/data", KeyDir: "/app/keys", ConfigPath: "/app/config.json",
DSN: "postgres://artex:s3cr3t-pw@postgres:5432/artex?sslmode=disable",
})
for _, want := range []string{"/app/keys/jwt.key", "/app/keys/secret.key", "/app/config.json", "/app/data/traffic/_ca/mitmproxy-ca.pem"} {
found := false
for _, p := range spec.Unreadable {
found = found || p == want
}
if !found {
t.Errorf("%s missing from the unreadable files %v", want, spec.Unreadable)
}
}
if len(spec.DB) != 1 || spec.DB[0].Host != "postgres" || spec.DB[0].Port != 5432 || spec.DB[0].User != "artex" || spec.DB[0].Database != "artex" {
t.Fatalf("db = %+v", spec.DB)
}
if spec.UID != 10001 || spec.GID != 10002 || spec.ServerPID != os.Getpid() {
t.Fatalf("ids = %d:%d pid %d", spec.UID, spec.GID, spec.ServerPID)
}
if b, _ := json.Marshal(spec); strings.Contains(string(b), "s3cr3t") {
t.Fatal("the password reached the spec")
}
}
+3 -1
View File
@@ -32,7 +32,9 @@ func connectMCP(ctx context.Context, m *db.MCPServer, priv ExecPrivilege) (mcpCl
if err != nil {
return nil, err
}
return mcp.NewStdioClient(ctx, m.Name, command, jsonStrMap(m.Env), args...)
// The drop shim rebuilds the environment from the allowlist; the configured
// variables of the server are named so it keeps them.
return mcp.NewStdioClient(ctx, m.Name, command, withExecKeep(priv, jsonStrMap(m.Env)), args...)
case "http":
if m.URL == "" {
return nil, fmt.Errorf("http transport is missing a URL")
+6 -1
View File
@@ -54,7 +54,8 @@ type Server struct {
scrubSecrets bool // drop ARTEX_PG_DSN and the API keys from the environment once read (binary only)
execPriv ExecPrivilege // identity agent processes run under; zero keeps the server's own
execPriv ExecPrivilege // identity agent processes run under; zero keeps the server's own
isolation IsolationStatus // result of the startup isolation self-test (isolation.go); zero when not run
setupMu sync.Mutex
setupToken string // one-time token POST /api/auth/init requires; empty when none is active
@@ -1101,6 +1102,10 @@ func (s *Server) health(w http.ResponseWriter, r *http.Request) {
return
}
out := map[string]any{"ok": true, "service": "artex", "version": BuildVersion}
// The web UI shows a banner on every page for a degraded isolation (isolation.go).
if s.isolation.Mode != "" {
out["isolation"] = s.isolation
}
// Opt-in only (ARTEX_UPDATE_CHECK); off, this returns "" without any request.
if latest := updates.newerVersion(r.Context(), time.Now(), BuildVersion); latest != "" {
out["latest_version"] = latest
@@ -43,6 +43,8 @@ export function MainContent({ children }: { children: ReactNode }) {
const [version, setVersion] = useState("");
// Only set when the backend has the opt-in update check on (ARTEX_UPDATE_CHECK) and found a newer release.
const [latest, setLatest] = useState("");
// Set when the server started with agent isolation degraded (server/isolation.go): the self-test failure.
const [degraded, setDegraded] = useState<string | null>(null);
useEffect(() => {
api
@@ -51,10 +53,25 @@ export function MainContent({ children }: { children: ReactNode }) {
setVersion((h.version ?? "").replace(/^v(?=\d)/, ""));
const l = h.latest_version ?? "";
setLatest(l && getLocalStorageValue(UPDATE_DISMISSED_KEY) !== l ? l : "");
setDegraded(h.isolation?.mode === "degraded" ? (h.isolation.reason ?? "") : null);
})
.catch(() => setVersion(""));
}, []);
// Not dismissible: it stays as long as the server runs without agent isolation.
const isolationBanner =
degraded !== null ? (
<div
role="alert"
title={degraded}
className="flex shrink-0 items-center justify-center gap-2 border-b border-destructive/40 bg-destructive/10 px-4 py-1 text-destructive text-xs"
>
{t(
"Agent isolation is degraded: agent shells, tools and file tools run with the server's identity and may read its keys, config and database credentials.",
)}
</div>
) : null;
const updateBanner = latest ? (
<div className="flex shrink-0 items-center justify-center gap-2 border-b bg-muted/50 px-4 py-1 text-xs">
<a href={RELEASES_URL} target="_blank" rel="noreferrer" className="underline-offset-2 hover:underline">
@@ -77,6 +94,7 @@ export function MainContent({ children }: { children: ReactNode }) {
if (isFullBleed(pathname)) {
return (
<>
{isolationBanner}
{updateBanner}
{children}
</>
@@ -85,6 +103,7 @@ export function MainContent({ children }: { children: ReactNode }) {
return (
<>
{isolationBanner}
{updateBanner}
<header
className={cn(
+1
View File
@@ -214,6 +214,7 @@
"Agent": "Ajan",
"Agent \"{name}\" created": "\"{name}\" ajanı oluşturuldu",
"Agent \"{name}\" deleted": "\"{name}\" ajanı silindi",
"Agent isolation is degraded: agent shells, tools and file tools run with the server's identity and may read its keys, config and database credentials.": "Ajan yalıtımı düşürüldü: ajan kabukları, araçları ve dosya araçları sunucunun kimliğiyle çalışıyor ve anahtarlarını, yapılandırmasını ve veritabanı kimlik bilgilerini okuyabilir.",
"Agent is replying; type /btw to ask a side question…": "Ajan yanıt veriyor; yan soru sormak için /btw yazın…",
"Agent is running": "Ajan çalışıyor",
"Agent list": "Ajan listesi",
+1
View File
@@ -214,6 +214,7 @@
"Agent": "智能体",
"Agent \"{name}\" created": "已创建智能体“{name}”",
"Agent \"{name}\" deleted": "已删除智能体“{name}”",
"Agent isolation is degraded: agent shells, tools and file tools run with the server's identity and may read its keys, config and database credentials.": "Agent 隔离已降级:Agent 的 shell、工具和文件工具以服务器身份运行,可能读取其密钥、配置和数据库凭据。",
"Agent is replying; type /btw to ask a side question…": "Agent 正在回复,可输入 /btw 提问…",
"Agent is running": "Agent 正在运行",
"Agent list": "Agent 清单",
+9 -1
View File
@@ -271,7 +271,15 @@ function interceptPageQuery(page: number, size: number, filter: InterceptApprova
export const api = {
// Backend application version (injected via ldflags at release, default "dev").
health: () => get<{ ok: boolean; service: string; version: string; latest_version?: string }>("/health"),
// isolation: result of the startup agent isolation self-test; "degraded" shows a banner on every page.
health: () =>
get<{
ok: boolean;
service: string;
version: string;
latest_version?: string;
isolation?: { mode: "isolated" | "degraded"; reason?: string };
}>("/health"),
// ---- auth ----
authStatus: () => get<{ initialized: boolean }>("/auth/status"),