mirror of
https://github.com/BaaRai/ARTEX.git
synced 2026-11-23 00:20:55 +08:00
Compare commits
29
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4f6aaef02a | ||
|
|
2677ff32ef | ||
|
|
8bacfc0245 | ||
|
|
90cd290d42 | ||
|
|
449065aaaa | ||
|
|
5df46e22c1 | ||
|
|
83b35f844b | ||
|
|
9d7983f1a2 | ||
|
|
3ca8f217a3 | ||
|
|
132af98029 | ||
|
|
6ef01d4f24 | ||
|
|
f72311994e | ||
|
|
4171a041e6 | ||
|
|
db867f63a4 | ||
|
|
d59227c5d4 | ||
|
|
f37d09e365 | ||
|
|
5ce7ae6006 | ||
|
|
7b8be967cc | ||
|
|
09eb73c547 | ||
|
|
43ace17ad7 | ||
|
|
1e7696c572 | ||
|
|
f4c7184dcc | ||
|
|
0c55774613 | ||
|
|
a133b79a0e | ||
|
|
19110a4727 | ||
|
|
c8b771881e | ||
|
|
0eb66f4c44 | ||
|
|
9a8f366574 | ||
|
|
25217afdce |
@@ -6,6 +6,14 @@ All notable changes to this project are documented in this file. The format is b
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Live log subscribers can no longer crash the process**: a client disconnect closes its log channel. The sink now delivers a line while it still holds the subscriber lock, so that close cannot land in the middle of the send.
|
||||
- **Notification delivery ignores HTTP_PROXY**: the dial guard checks the address it connects to. An environment proxy made that address the proxy, so a link-local target could still be reached. Notification HTTP now connects directly.
|
||||
- **Captured traffic blobs are owner-only**: HTTP and raw-stream bodies spilled out of the database are written `0600`, and their bucket directory is `0700`. An existing blob is tightened when it is seen again.
|
||||
- **Asset URLs drop userinfo when parsing fails**: a URL `url.Parse` rejects is no longer stored unchanged. The `scheme://userinfo@` prefix is removed first, so a password in that prefix does not land in the asset row.
|
||||
- **Hook `range` counts against the memory budget**: ranging over a body, slice, string, or map charges the iteration before the loop. A body larger than the hook budget now stops with the memory limit instead of allocating one element at a time with no check.
|
||||
- **Chat upload name search stops**: choosing `name`, `name-1`, `name-2`, … now returns an error when the filesystem rejects the path or after 10000 collisions, instead of looping while `Stat` keeps failing.
|
||||
- **Uploads do not follow a symlink**: chat and workspace uploads `Lstat` the destination and refuse to write when it is already a symlink. A dangling link is no longer treated as a free name.
|
||||
- **Hardening follow-up for the upload and capture fixes**: existing spill blob directories and files are tightened on startup, hook `range` charges 32 bytes per pair, and uploads open with `O_NOFOLLOW` on Unix. The Windows upload path keeps the `Lstat` check.
|
||||
- **Docker Compose database passwords with URL-special characters**: `docker-compose.yml` no longer pastes `POSTGRES_PASSWORD` into `ARTEX_PG_DSN`. The server builds the URL from `ARTEX_PG_HOST`, `ARTEX_PG_PORT`, `ARTEX_PG_USER`, `ARTEX_PG_PASSWORD`, `ARTEX_PG_DBNAME` and `ARTEX_PG_SSLMODE`, percent-encoding the user and password, so `/`, `?`, `%`, `@`, `:` and spaces connect. An explicit `ARTEX_PG_DSN` still wins and is not rewritten. `ARTEX_PG_PASSWORD` is removed from the environment with the other startup secrets (#450).
|
||||
|
||||
## [0.7.0] - 2026-11-21
|
||||
|
||||
@@ -473,7 +473,7 @@ runtime supports it.
|
||||
|
||||
## Configuration
|
||||
|
||||
**Database** (`config.json`, or the `ARTEX_PG_DSN` environment variable):
|
||||
**Database** (`config.json`, `ARTEX_PG_DSN`, or `ARTEX_PG_*` environment variables — see [External PostgreSQL](#external-postgresql)):
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -558,6 +558,70 @@ from the defaults until a worker finishes in this process.
|
||||
**Common flags**: `./start.sh -addr :8787 -proxy :8788` (`-addr` frontend/API, `-proxy` traffic
|
||||
recording proxy).
|
||||
|
||||
### External PostgreSQL
|
||||
|
||||
ARTEX already supports an external PostgreSQL 16+ instance — the `postgres` service in
|
||||
`docker-compose.yml` is only the bundled default. Because the database is startup configuration and
|
||||
the system's single source of truth, there is no runtime database switcher in the UI: change the
|
||||
environment variables and restart the server to point at another instance.
|
||||
|
||||
Configure the connection with either a full connection string or individual parts:
|
||||
|
||||
- **`ARTEX_PG_DSN`**: full PostgreSQL connection URL (for example
|
||||
`postgres://artex:pass@db.example.internal:5432/artex?sslmode=require`). When set, it takes
|
||||
precedence over the `ARTEX_PG_*` variables and you are responsible for percent-encoding any
|
||||
URL-special characters in the username or password.
|
||||
- **`ARTEX_PG_*` parts** (recommended when passwords contain `/`, `?`, `%`, `@`, `:`, or spaces —
|
||||
ARTEX percent-encodes the user and password and removes `ARTEX_PG_PASSWORD` from the environment
|
||||
at startup):
|
||||
- `ARTEX_PG_HOST` — PostgreSQL host
|
||||
- `ARTEX_PG_PORT` — PostgreSQL port (default `5432`)
|
||||
- `ARTEX_PG_USER` — database user
|
||||
- `ARTEX_PG_PASSWORD` — database password (raw, not URL-encoded)
|
||||
- `ARTEX_PG_DBNAME` — database name
|
||||
- `ARTEX_PG_SSLMODE` — TLS mode (`disable`, `require`, `verify-ca`, `verify-full`)
|
||||
|
||||
#### Compose without the bundled `postgres` service
|
||||
|
||||
To run ARTEX against an external PostgreSQL server with Docker Compose, remove the `postgres`
|
||||
service, the `depends_on` block, and the `pgdata` volume from `docker-compose.yml`, and point
|
||||
`artex` at your database:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
artex:
|
||||
image: ghcr.io/baarai/artex:latest
|
||||
security_opt:
|
||||
- "no-new-privileges:true"
|
||||
environment:
|
||||
# Option A: individual parts (recommended; percent-encodes special characters)
|
||||
ARTEX_PG_HOST: ${ARTEX_PG_HOST:?Set ARTEX_PG_HOST in .env}
|
||||
ARTEX_PG_PORT: ${ARTEX_PG_PORT:-5432}
|
||||
ARTEX_PG_USER: ${ARTEX_PG_USER:-artex}
|
||||
ARTEX_PG_PASSWORD: ${ARTEX_PG_PASSWORD:?Set ARTEX_PG_PASSWORD in .env}
|
||||
ARTEX_PG_DBNAME: ${ARTEX_PG_DBNAME:-artex}
|
||||
ARTEX_PG_SSLMODE: ${ARTEX_PG_SSLMODE:-require}
|
||||
# Option B: full DSN (overrides ARTEX_PG_* when set)
|
||||
# ARTEX_PG_DSN: ${ARTEX_PG_DSN}
|
||||
ARTEX_JWT_KEY_DIR: /app/keys
|
||||
ports:
|
||||
- "${ARTEX_BIND_ADDR:-127.0.0.1}:8787:8787"
|
||||
- "127.0.0.1:8788:8788"
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./keys:/app/keys
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
#### Local disk usage (`data/`)
|
||||
|
||||
Moving PostgreSQL to an external host does **not** move the runtime `data/` directory off local
|
||||
disk. `data/` still stores spilled HTTP/TCP traffic blobs (`data/traffic/_blobs`), LLM
|
||||
transcripts, task working files, and local task archives, so heavy workloads can still fill the
|
||||
host disk if `data/` is on a small volume. Mount `data/` on a suitably sized volume, and consider
|
||||
setting `ARTEX_ARCHIVE_STORAGE=s3` (see [Task archive storage](docs/task-archive-storage.md)) to
|
||||
offload cold task archives to object storage.
|
||||
|
||||
### Reverse proxy deployment (HTTPS / exposing only 443)
|
||||
|
||||
The frontend, API and SSE stream share port 8787 (the binary listens on `127.0.0.1:8787` by
|
||||
|
||||
@@ -20,10 +20,6 @@ type TrafficCaptureChecker interface {
|
||||
TrafficExists(ctx context.Context, trafficID string) (bool, error)
|
||||
}
|
||||
|
||||
// Tool-use guidance is appended without replacing the user's editable prompt.
|
||||
// It does not require capture or claim that unavailable traffic tools exist.
|
||||
const findingTrafficGuidance = "\n\n**Vulnerability traffic evidence**: when reporting a vulnerability via report_finding, bind real HTTP request/response IDs via traffic_refs in reproduction order; domains and times are only candidate filters and do not imply association. A finding without bound traffic stays recorded and visible, but cannot close a goal via prove_goal until verified traffic matching its asset host is bound (via report_finding or bind_finding_traffic). For TCP and other non-HTTP vulnerabilities, when nothing was captured, or when there is no exact match, omit traffic_refs or pass [], keep verifiable command output and logs in evidence, and close non-HTTP goals with an observed tool_output fact. Do not guess IDs, and do not re-probe just to capture packets."
|
||||
|
||||
func (t *ToolSet) SetFindingRecorder(r FindingRecorder) { t.findingRecorder = r }
|
||||
func (w *Worker) SetFindingRecorder(r FindingRecorder) { w.findingRecorder = r }
|
||||
func (p *Planner) SetFindingRecorder(r FindingRecorder) { p.findingRecorder = r }
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFindingRecorderHasNoLegacyTrafficGuidance(t *testing.T) {
|
||||
// Check the declaration directly: an unused constant cannot affect prompt output.
|
||||
f, err := parser.ParseFile(token.NewFileSet(), "finding_recorder.go", nil, parser.SkipObjectResolution)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
if spec, ok := n.(*ast.ValueSpec); ok {
|
||||
for _, name := range spec.Names {
|
||||
if name.Name == "findingTrafficGuidance" {
|
||||
t.Fatal("finding_recorder.go declares the obsolete findingTrafficGuidance constant")
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
}
|
||||
+3
-1
@@ -1692,7 +1692,9 @@ func factEvidenceLines(evidence string) []string {
|
||||
}
|
||||
}
|
||||
return out
|
||||
} // factProvenance classifies one fact against the intent's recorded activity.
|
||||
}
|
||||
|
||||
// factProvenance classifies one fact against the intent's recorded activity.
|
||||
// Beyond the activity tie it checks whether the quoted bytes were captured
|
||||
// coming back from the target through the proxy, in which case the target
|
||||
// authored them and the fact is classified target_response even when the model
|
||||
|
||||
+20
-1
@@ -165,7 +165,7 @@ func calcCSegment(ipStr string) string {
|
||||
func normalizeURL(raw string) string {
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
return raw
|
||||
return stripURLUserinfo(raw)
|
||||
}
|
||||
u.Scheme = strings.ToLower(u.Scheme)
|
||||
u.Host = strings.ToLower(u.Host)
|
||||
@@ -177,6 +177,25 @@ func normalizeURL(raw string) string {
|
||||
return s
|
||||
}
|
||||
|
||||
// stripURLUserinfo removes scheme://userinfo@ when url.Parse cannot. The
|
||||
// userinfo is the span before the last @ that still sits in the authority.
|
||||
func stripURLUserinfo(raw string) string {
|
||||
i := strings.Index(raw, "://")
|
||||
if i < 0 {
|
||||
return raw
|
||||
}
|
||||
rest := raw[i+3:]
|
||||
authEnd := len(rest)
|
||||
if j := strings.IndexAny(rest, "/?#"); j >= 0 {
|
||||
authEnd = j
|
||||
}
|
||||
at := strings.LastIndex(rest[:authEnd], "@")
|
||||
if at < 0 {
|
||||
return raw
|
||||
}
|
||||
return raw[:i+3] + rest[at+1:]
|
||||
}
|
||||
|
||||
// parseURL extracts domain, port, service_name from a URL.
|
||||
func parseURL(raw string) (domain string, port int, serviceName string) {
|
||||
u, err := url.Parse(raw)
|
||||
|
||||
@@ -908,6 +908,7 @@ func TestNormalizeURL(t *testing.T) {
|
||||
{"https://user@example.com/", "https://example.com"},
|
||||
{"https://u:p@example.com:8443/users/@alice?x=1", "https://example.com:8443/users/@alice?x=1"},
|
||||
{"https://example.com/@me?r=http://a:b@x/", "https://example.com/@me?r=http://a:b@x/"},
|
||||
{"http://user:secret@ex ample.com/a", "http://ex ample.com/a"},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
got := normalizeURL(tc.raw)
|
||||
|
||||
@@ -104,6 +104,10 @@ var notifyTransport = func() *http.Transport {
|
||||
return &http.Transport{}
|
||||
}
|
||||
clone := t.Clone()
|
||||
// Clone keeps ProxyFromEnvironment. The dial would then be to the proxy, so
|
||||
// blockInternalDial would see the proxy address and a link-local target
|
||||
// could still be reached through it. Notifications go direct.
|
||||
clone.Proxy = nil
|
||||
clone.DialContext = (&net.Dialer{Timeout: 10 * time.Second, Control: blockInternalDial}).DialContext
|
||||
return clone
|
||||
}()
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
package notify
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestNotifyTransportIgnoresEnvironmentProxy(t *testing.T) {
|
||||
if notifyTransport.Proxy != nil {
|
||||
t.Fatal("notification transport must dial the target, not an environment proxy")
|
||||
}
|
||||
}
|
||||
+31
-8
@@ -21,6 +21,10 @@ const (
|
||||
// Allow multipart headers and boundaries in addition to the file data.
|
||||
chatUploadMultipartOverhead = 1 << 20 // 1 MiB
|
||||
chatUploadLimitMessage = "files exceed the 512 MiB upload limit"
|
||||
// maxUploadNameTries bounds the name-1, name-2, … search. A Stat error that
|
||||
// is not "not exist" (a component the filesystem rejects) must stop the
|
||||
// search instead of spinning.
|
||||
maxUploadNameTries = 10000
|
||||
)
|
||||
|
||||
// safeChatID guards the {id} path segment against traversal — task ids are numeric,
|
||||
@@ -123,7 +127,11 @@ func (s *Server) chatUpload(w http.ResponseWriter, r *http.Request) {
|
||||
if name == "" || name == "." || name == ".." || strings.ContainsAny(name, `/\`) {
|
||||
continue
|
||||
}
|
||||
dest := uniqueUploadPath(dir, name)
|
||||
dest, err := uniqueUploadPath(dir, name)
|
||||
if err != nil {
|
||||
writeErr(w, 500, "Save failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
if err := saveUpload(hdr, dest); err != nil {
|
||||
writeErr(w, 500, "Save failed: "+err.Error())
|
||||
return
|
||||
@@ -136,19 +144,34 @@ func (s *Server) chatUpload(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
// uniqueUploadPath returns dir/name, or dir/name-1, dir/name-2… when it already exists,
|
||||
// so re-uploading the same filename never clobbers a prior attachment.
|
||||
func uniqueUploadPath(dir, name string) string {
|
||||
dest := filepath.Join(dir, name)
|
||||
if _, err := os.Stat(dest); os.IsNotExist(err) {
|
||||
return dest
|
||||
func uniqueUploadPath(dir, name string) (string, error) {
|
||||
if dest, ok, err := uploadPathFree(filepath.Join(dir, name)); err != nil || ok {
|
||||
return dest, err
|
||||
}
|
||||
ext := filepath.Ext(name)
|
||||
stem := strings.TrimSuffix(name, ext)
|
||||
for i := 1; ; i++ {
|
||||
for i := 1; i <= maxUploadNameTries; i++ {
|
||||
cand := filepath.Join(dir, fmt.Sprintf("%s-%d%s", stem, i, ext))
|
||||
if _, err := os.Stat(cand); os.IsNotExist(err) {
|
||||
return cand
|
||||
dest, ok, err := uploadPathFree(cand)
|
||||
if err != nil || ok {
|
||||
return dest, err
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("no free name for upload %q", name)
|
||||
}
|
||||
|
||||
// uploadPathFree reports whether path can be created. ok is false when a file
|
||||
// or a symlink (including a dangling one) is already there. Any other Lstat
|
||||
// error is returned so the caller stops.
|
||||
func uploadPathFree(path string) (dest string, ok bool, err error) {
|
||||
_, err = os.Lstat(path)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return path, true, nil
|
||||
}
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
// composeAgentMessage appends an attachment manifest to the user's message so the agent
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestUniqueUploadPathSuffixesCollisions(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("a"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "note-1.txt"), []byte("b"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := uniqueUploadPath(dir, "note.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if filepath.Base(got) != "note-2.txt" {
|
||||
t.Fatalf("got %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUniqueUploadPathStopsOnUnstorableName(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
name := strings.Repeat("a", 300) + ".txt"
|
||||
if _, err := uniqueUploadPath(dir, name); err == nil {
|
||||
t.Fatal("an overlong file name must not be searched forever")
|
||||
}
|
||||
}
|
||||
+8
-9
@@ -152,9 +152,15 @@ func (s *logSinkT) add(l LogLine) {
|
||||
s.ring = s.ring[len(s.ring)-s.cap:]
|
||||
}
|
||||
ch := s.dbCh
|
||||
subs := make([]chan LogLine, 0, len(s.subs))
|
||||
// Fan-out stays under the lock. subscribe's cancel removes the channel and
|
||||
// then closes it; a send on a closed channel panics even inside select, so
|
||||
// the send and the close must not interleave. The send does not block: a
|
||||
// full buffer drops the line.
|
||||
for sub := range s.subs {
|
||||
subs = append(subs, sub)
|
||||
select {
|
||||
case sub <- l:
|
||||
default:
|
||||
}
|
||||
}
|
||||
s.mu.Unlock()
|
||||
|
||||
@@ -165,13 +171,6 @@ func (s *logSinkT) add(l LogLine) {
|
||||
default:
|
||||
}
|
||||
}
|
||||
|
||||
for _, sub := range subs {
|
||||
select {
|
||||
case sub <- l:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// recent returns ring lines with Seq > since (capped to limit), and the latest seq.
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"io"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLogSinkUnsubscribeDuringAddDoesNotPanic(t *testing.T) {
|
||||
s := &logSinkT{cap: 8, subs: map[chan LogLine]struct{}{}, out: io.Discard}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 32; i++ {
|
||||
_, unsub := s.subscribe()
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
unsub()
|
||||
}()
|
||||
s.add(LogLine{Text: "line"})
|
||||
}
|
||||
wg.Wait()
|
||||
s.add(LogLine{Text: "after"})
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
//go:build !windows
|
||||
|
||||
package server
|
||||
|
||||
import (
|
||||
"os"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// openUploadFileNoFollow opens dest for writing without following a symlink at
|
||||
// the final path component, closing the TOCTOU window after refuseUploadSymlink.
|
||||
func openUploadFileNoFollow(dest string) (*os.File, error) {
|
||||
return os.OpenFile(dest, os.O_WRONLY|os.O_CREATE|os.O_TRUNC|syscall.O_NOFOLLOW, 0o644)
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
//go:build windows
|
||||
|
||||
package server
|
||||
|
||||
import "os"
|
||||
|
||||
func openUploadFileNoFollow(dest string) (*os.File, error) {
|
||||
return os.OpenFile(dest, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRefuseUploadSymlink(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
target := filepath.Join(dir, "secret")
|
||||
if err := os.WriteFile(target, []byte("keep"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
link := filepath.Join(dir, "upload")
|
||||
if err := os.Symlink(target, link); err != nil {
|
||||
t.Skipf("symlink: %v", err)
|
||||
}
|
||||
if err := refuseUploadSymlink(link); err == nil {
|
||||
t.Fatal("symlink destination was accepted")
|
||||
}
|
||||
body, err := os.ReadFile(target)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(body) != "keep" {
|
||||
t.Fatalf("target changed to %q", body)
|
||||
}
|
||||
missing := filepath.Join(dir, "fresh")
|
||||
if err := refuseUploadSymlink(missing); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenUploadFileNoFollowRejectsSymlink(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("O_NOFOLLOW is Unix-only")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
target := filepath.Join(dir, "secret")
|
||||
if err := os.WriteFile(target, []byte("keep"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
link := filepath.Join(dir, "link")
|
||||
if err := os.Symlink(target, link); err != nil {
|
||||
t.Skipf("symlink: %v", err)
|
||||
}
|
||||
if f, err := openUploadFileNoFollow(link); err == nil {
|
||||
_ = f.Close()
|
||||
t.Fatal("openUploadFileNoFollow accepted a symlink")
|
||||
}
|
||||
body, err := os.ReadFile(target)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(body) != "keep" {
|
||||
t.Fatalf("target changed to %q", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUniqueUploadPathSkipsDanglingSymlink(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
dangling := filepath.Join(dir, "note.txt")
|
||||
if err := os.Symlink(filepath.Join(dir, "nonexistent"), dangling); err != nil {
|
||||
t.Skipf("symlink: %v", err)
|
||||
}
|
||||
got, err := uniqueUploadPath(dir, "note.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if filepath.Base(got) != "note-1.txt" {
|
||||
t.Fatalf("got %s, want note-1.txt", got)
|
||||
}
|
||||
}
|
||||
+23
-1
@@ -2,6 +2,7 @@ package server
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
@@ -340,12 +341,15 @@ func (s *Server) wsUpload(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
func saveUpload(hdr *multipart.FileHeader, dest string) error {
|
||||
if err := refuseUploadSymlink(dest); err != nil {
|
||||
return err
|
||||
}
|
||||
src, err := hdr.Open()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer src.Close()
|
||||
out, err := os.Create(dest)
|
||||
out, err := openUploadFileNoFollow(dest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -354,6 +358,24 @@ func saveUpload(hdr *multipart.FileHeader, dest string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// refuseUploadSymlink rejects a destination that is already a symlink. os.Create
|
||||
// follows one, including a dangling link that Stat reports as missing, so an
|
||||
// agent that can write the uploads directory could aim the server's write at
|
||||
// another path. Lstat sees the link itself.
|
||||
func refuseUploadSymlink(dest string) error {
|
||||
fi, err := os.Lstat(dest)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if fi.Mode()&os.ModeSymlink != 0 {
|
||||
return fmt.Errorf("refusing to write through a symlink")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sanitizeFilename strips characters unsafe for a Content-Disposition filename token.
|
||||
func sanitizeFilename(name string) string {
|
||||
name = strings.ReplaceAll(name, "\"", "")
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package traffic
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSpillBlobIsOwnerOnly(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
tr := &Traffic{dir: dir}
|
||||
body := bytes.Repeat([]byte("a"), maxInlineBody+1)
|
||||
got := tr.spill(body, "text/plain")
|
||||
if got.hash == "" {
|
||||
t.Fatal("expected a spilled blob")
|
||||
}
|
||||
blobDir := filepath.Join(dir, "_blobs", "sha256", got.hash[:2])
|
||||
blobPath := filepath.Join(blobDir, got.hash+".bin")
|
||||
info, err := os.Stat(blobPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dirInfo, err := os.Stat(blobDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if runtime.GOOS == "windows" {
|
||||
return
|
||||
}
|
||||
if info.Mode().Perm() != 0o600 {
|
||||
t.Fatalf("blob mode %o, want 0600", info.Mode().Perm())
|
||||
}
|
||||
if dirInfo.Mode().Perm() != 0o700 {
|
||||
t.Fatalf("blob dir mode %o, want 0700", dirInfo.Mode().Perm())
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenTightensExistingBlobPerms(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("POSIX file permissions not enforced on Windows")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
blobsRoot := filepath.Join(dir, "_blobs")
|
||||
shaDir := filepath.Join(blobsRoot, "sha256")
|
||||
bucket := filepath.Join(shaDir, "ab")
|
||||
if err := os.MkdirAll(bucket, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, d := range []string{blobsRoot, shaDir, bucket} {
|
||||
if err := os.Chmod(d, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
blob := filepath.Join(bucket, "abcdef.bin")
|
||||
if err := os.WriteFile(blob, []byte("secret-body"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(blob, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
tr, err := Open(dir, "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer tr.Close()
|
||||
|
||||
for _, d := range []string{blobsRoot, shaDir, bucket} {
|
||||
info, err := os.Stat(d)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.Mode().Perm() != 0o700 {
|
||||
t.Fatalf("%s mode %o, want 0700", d, info.Mode().Perm())
|
||||
}
|
||||
}
|
||||
info, err := os.Stat(blob)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.Mode().Perm() != 0o600 {
|
||||
t.Fatalf("%s mode %o, want 0600", blob, info.Mode().Perm())
|
||||
}
|
||||
}
|
||||
@@ -330,24 +330,30 @@ func (e *vm) execRange(s *ast.RangeStmt, sc *env) (flowSignal, any) {
|
||||
var items []kvPair
|
||||
switch x := e.eval(s.X, sc).(type) {
|
||||
case []byte:
|
||||
e.alloc(int64(len(x)) * 32)
|
||||
for i, v := range x {
|
||||
items = append(items, kvPair{int64(i), int64(v)})
|
||||
}
|
||||
case []any:
|
||||
e.alloc(int64(len(x)) * 32)
|
||||
for i, v := range x {
|
||||
items = append(items, kvPair{int64(i), v})
|
||||
}
|
||||
case []string:
|
||||
e.alloc(int64(len(x)) * 32)
|
||||
for i, v := range x {
|
||||
items = append(items, kvPair{int64(i), v})
|
||||
}
|
||||
case string:
|
||||
e.alloc(int64(len(x)) * 32)
|
||||
for i, r := range x {
|
||||
items = append(items, kvPair{int64(i), int64(r)})
|
||||
}
|
||||
case map[string]any:
|
||||
e.alloc(int64(len(x)) * 32)
|
||||
items = sortedMapPairs(x)
|
||||
case map[string][]string:
|
||||
e.alloc(int64(len(x)) * 32)
|
||||
items = sortedMapPairs(x)
|
||||
case nil:
|
||||
return flowNone, nil
|
||||
|
||||
@@ -204,6 +204,25 @@ func OnResponse(req *Request, resp *Response) error {
|
||||
}
|
||||
|
||||
func TestResourceBoundsMemoryStepTimeoutPanicAndRegexp(t *testing.T) {
|
||||
t.Run("range_over_body_stopped_by_memory_budget", func(t *testing.T) {
|
||||
src := `package hook
|
||||
func OnRequest(req *Request) error {
|
||||
for _, b := range req.Body {
|
||||
_ = b
|
||||
}
|
||||
return nil
|
||||
}`
|
||||
prog, err := Compile("request", src)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body := make([]byte, 256<<10)
|
||||
err = prog.RunRequest(context.Background(), &Request{Body: body}, RunOptions{MaxMemory: 64 << 10})
|
||||
if !errors.Is(err, ErrMemoryLimit) {
|
||||
t.Fatalf("err = %v, want ErrMemoryLimit", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("make_100MiB_stopped_by_memory_budget", func(t *testing.T) {
|
||||
src := `package hook
|
||||
func OnRequest(req *Request) error {
|
||||
|
||||
+2
-2
@@ -63,7 +63,7 @@ func (t *Traffic) spillRawDir(data []byte, previewCap int) (preview []byte, blob
|
||||
sum := sha256.Sum256(data)
|
||||
h := hex.EncodeToString(sum[:])
|
||||
blobDir := filepath.Join(t.dir, "_blobs", "sha256", h[:2])
|
||||
if err := os.MkdirAll(blobDir, 0o755); err != nil {
|
||||
if err := os.MkdirAll(blobDir, 0o700); err != nil {
|
||||
log.Printf("[traffic/socks5] failed to create blob directory: %v", err)
|
||||
return append([]byte(nil), data[:previewCap]...), ""
|
||||
}
|
||||
@@ -96,7 +96,7 @@ func writeBlobAtomic(blobDir, blobPath, hash string, data []byte) error {
|
||||
_ = tmp.Close()
|
||||
return err
|
||||
}
|
||||
if err := tmp.Chmod(0o644); err != nil {
|
||||
if err := tmp.Chmod(0o600); err != nil {
|
||||
_ = tmp.Close()
|
||||
return err
|
||||
}
|
||||
|
||||
+45
-3
@@ -295,11 +295,16 @@ type Traffic struct {
|
||||
|
||||
// Open initializes the traffic tree, blob store and SQLite index under dir.
|
||||
func Open(dir, addr string) (*Traffic, error) {
|
||||
for _, d := range []string{dir, filepath.Join(dir, "_index"), filepath.Join(dir, "_blobs"), filepath.Join(dir, "_ca")} {
|
||||
for _, d := range []string{dir, filepath.Join(dir, "_index"), filepath.Join(dir, "_ca")} {
|
||||
if err := os.MkdirAll(d, 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
blobsRoot := filepath.Join(dir, "_blobs")
|
||||
if err := os.MkdirAll(blobsRoot, 0o700); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tightenBlobPerms(blobsRoot)
|
||||
// busy_timeout is a per-connection setting, so it belongs in the DSN rather
|
||||
// than in a one-off Exec: the pool opens connections on demand, and an Exec
|
||||
// only configures whichever one happened to serve it — leaving every other
|
||||
@@ -847,6 +852,38 @@ type storedBody struct {
|
||||
// reader can identify them without fetching the blob. Either way, text bodies
|
||||
// are handed to the full-text index in full (up to maxIndexBody) — indexing is
|
||||
// independent of where the bytes end up.
|
||||
// tightenBlobPerms restricts existing spilled blob directories to 0700 and
|
||||
// regular blob files to 0600 on startup so pre-existing 0755/0644 artifacts
|
||||
// from earlier versions do not stay world-readable.
|
||||
func tightenBlobPerms(root string) {
|
||||
_ = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
if d.Type()&os.ModeSymlink != 0 {
|
||||
return nil
|
||||
}
|
||||
info, err := d.Info()
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
if d.IsDir() {
|
||||
if info.Mode().Perm() != 0o700 {
|
||||
if err := os.Chmod(path, 0o700); err != nil {
|
||||
log.Printf("[traffic] failed to restrict blob directory %s: %v", path, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if info.Mode().IsRegular() && info.Mode().Perm() != 0o600 {
|
||||
if err := os.Chmod(path, 0o600); err != nil {
|
||||
log.Printf("[traffic] failed to restrict blob %s: %v", path, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (t *Traffic) spill(body []byte, contentType string) storedBody {
|
||||
return t.spillAt(body, contentType, maxInlineBody)
|
||||
}
|
||||
@@ -873,16 +910,21 @@ func (t *Traffic) spillAt(body []byte, contentType string, maxInline int) stored
|
||||
// One bucket level (256 buckets) is enough to keep any single directory small;
|
||||
// the store only ever holds bodies above maxInlineBody, deduplicated by hash.
|
||||
blobDir := filepath.Join(t.dir, "_blobs", "sha256", h[:2])
|
||||
if err := os.MkdirAll(blobDir, 0o755); err != nil {
|
||||
if err := os.MkdirAll(blobDir, 0o700); err != nil {
|
||||
log.Printf("[traffic] failed to create blob directory: %v", err)
|
||||
return storedBody{inline: clipBytes(body, blobPreview), index: indexText()}
|
||||
}
|
||||
if err := os.Chmod(blobDir, 0o700); err != nil {
|
||||
log.Printf("[traffic] failed to restrict blob directory: %v", err)
|
||||
}
|
||||
blobPath := filepath.Join(blobDir, h+".bin")
|
||||
if _, err := os.Stat(blobPath); os.IsNotExist(err) {
|
||||
if err := os.WriteFile(blobPath, body, 0o644); err != nil {
|
||||
if err := os.WriteFile(blobPath, body, 0o600); err != nil {
|
||||
log.Printf("[traffic] failed to write blob %s: %v", h, err)
|
||||
return storedBody{inline: clipBytes(body, blobPreview), index: indexText()}
|
||||
}
|
||||
} else if err := os.Chmod(blobPath, 0o600); err != nil {
|
||||
log.Printf("[traffic] failed to restrict blob %s: %v", h, err)
|
||||
}
|
||||
sb := storedBody{hash: h, index: indexText()}
|
||||
if text {
|
||||
|
||||
Reference in New Issue
Block a user