Replaces "runtime npm dependency that gets `npm install`-ed into
dist/" with "TS wrapper that loads the native `.node` from
`dist/native/` like every other native binding in the project".
Why the change
--------------
The previous approach (better-sqlite3 declared in
packages/runtime/package.json, materialized by `npm install
--omit=dev` in dist/) worked but produced an artifact that needed
runtime npm install — a step the rest of SnowLuma's native bindings
don't need (websocket / snowluma-hook / ffmpeg are all standalone
`.node` files copied into `dist/native/` by vite-plugin-cp). The
asymmetry was friction: dev mode required `pnpm dist:install`,
build artifacts carried a `dist/node_modules/` tree the other
modules didn't need, and Docker images had to special-case the
glibc-vs-musl ABI lookup that prebuild-install runs.
The new shape:
- `@snowluma/sqlite` (packages/sqlite/) — TS wrapper. Has
`better-sqlite3` as a devDep (for the JS-layer types + the
actual JS wrapper code that vite inlines). At runtime, calls
`createRequire().require()` against
`<dist|repo>/native/better-sqlite3-v<abi>-<platform>-<arch>.node`
and passes the resolved addon to `new Database(path, {
nativeBinding })`.
- `packages/runtime/native/better-sqlite3-v<abi>-<triplet>.node`
— prebuilt binaries vendored into the repo. Two ABIs ship side-
by-side (v127 = Node 22 LTS used by CI, v137 = Node 24 used by
dev boxes) so `process.versions.modules` picks the matching one
at load time.
- `tools/fetch-sqlite-prebuilts.mjs` — fetches all 5 platforms ×
N ABIs from WiseLibs's GitHub releases. Idempotent: existing
`.node` files skip. Accepts extra ABI args:
`node tools/fetch-sqlite-prebuilts.mjs 12.10.0 v127 v137 v115`.
What was unwound
----------------
- packages/runtime/package.json — dependencies removed (runtime
artifact is now self-contained, no `npm install` step needed).
- root package.json — `dist:install` script removed (no node_modules
to install).
- .github/workflows/dev-build.yml — `Bundle runtime dependencies`
step removed.
- .github/workflows/release.yml — kept the `Bundle runtime
dependencies` step (was a no-op already, but harmless safety net
in case a future runtime dep shows up).
- packages/core/vite.config.ts — `better-sqlite3` no longer
external; vite inlines the JS wrapper (+~33 KB to bundle, ~2.84
MB → 2.87 MB total).
Naming
------
Mirrors the existing `websocket-${triplet}.node` convention (flat
dir under `runtime/native/`, dash-separated `<platform>-<arch>`).
Adds a `v<abi>` segment because better-sqlite3 is NOT N-API — one
binary per Node ABI. Naming:
better-sqlite3-v127-darwin-arm64.node Node 22 / macOS arm
better-sqlite3-v127-darwin-x64.node Node 22 / macOS intel
better-sqlite3-v127-linux-arm64.node Node 22 / Linux arm
better-sqlite3-v127-linux-x64.node Node 22 / Linux x64
better-sqlite3-v127-win32-x64.node Node 22 / Windows x64
better-sqlite3-v137-... (5) Node 24 / same 5 hosts
Total: 10 binaries × ~1.9 MB = ~19 MB committed. Aligned with the
existing native-vendoring footprint (websocket ~2 MB × 5, snowluma-
hook ~10 MB × Windows + Linux, ffmpeg ~10-30 MB × 4).
What changes for consumers
--------------------------
Source-side: nothing. `import Database, { type Database, type
Statement } from '@snowluma/sqlite'` is a drop-in for the previous
`import Database from 'better-sqlite3'`. The wrapper re-exports the
same types and the `Database` callable factory has the same
overload set.
Runtime: `node` finds the binding at
`<dist|repo>/native/better-sqlite3-v${process.versions.modules}-
${process.platform}-${process.arch}.node`. If the ABI doesn't
match a vendored binary, the wrapper throws a clear error pointing
at `tools/fetch-sqlite-prebuilts.mjs` with the missing ABI/triplet.
Bumped better-sqlite3 from 11.10.0 → 12.10.0 because v11 only had
prebuilt up to Node ABI v127 (no v137 = no Node 24). v12.10.0
ships both. Both versions are API-compat (the breaking changes
between v11 and v12 were either internal or in error-handling
paths SnowLuma doesn't exercise).
Verification
------------
- pnpm -r typecheck (12 projects): clean
- @snowluma/core test: 302/302
- @snowluma/onebot test: 250/250 (552/552 combined, unchanged from
the perf-series baseline)
- Bench numbers preserved — wrapper indirection is sub-μs:
storeMeta ~11 μs, storeEvent ~17 μs, findMeta hit ~1.1 μs (all
within noise of the post-RoadMap-#5 measurements).
- Vite build clean, bundle = 2.87 MB / 230 KB gzip
- `dist/index.mjs` no longer contains a `node:sqlite` import (it
did before — media-store hadn't migrated; now neither it nor
message-store reach into the Node builtin).
Bonus side-fix: `packages/onebot/src/media-store.ts` also moved
off `node:sqlite` (caught by the post-build bundle grep — the
file was using `DatabaseSync` + `StatementSync` from the builtin
even though it had already been written to cache its prepared
statements). Its hot paths now go through the same wrapper.
Followup to 0b6541c. That commit added `better-sqlite3` as a
workspace dependency and marked it as a vite external — meaning
the bundled `dist/index.mjs` does `require('better-sqlite3')` at
runtime instead of inlining its JS wrapper. Without the binding
+ wrapper present in the production archive, every shipped
SnowLuma would crash on first SQLite op.
The fix is in three small pieces:
1. `packages/runtime/package.json` — declare `better-sqlite3` as a
runtime dep. `vite-plugin-cp` copies this manifest into
`dist/package.json` during build (see `runtimeDistFiles` in
`packages/core/vite.config.ts`), so the release workflow's
existing `npm install --omit=dev` step inside `dist/` picks it
up and compiles the native binding for the matrix target via
prebuild-install (with node-gyp fallback). This is the same
mechanism release.yml has used since v1.x — it was just empty
until now because the previous vite config bundled everything.
2. `.github/workflows/dev-build.yml` — mirror the
`Bundle runtime dependencies` step from release.yml. Dev-build
was packaging `dist/` straight after vite, which would have
shipped a broken artifact (no `dist/node_modules/`). Added the
`npm install --omit=dev` step before the tar/zip packaging
steps so dev-build archives carry the binding too.
3. `package.json` root — add `dist:install` script (just
`cd dist && npm install --omit=dev`) so local prod-mimic flows
work: `pnpm build && pnpm dist:install && pnpm dev:mode`. Not
chained into `pnpm build` itself to keep the watch-cycle fast
for users who only need the vite build output; the install
step adds ~17s.
Note on dev mode: `pnpm dev` continues to work unchanged — it
runs via tsx against workspace `node_modules/`, which already has
`better-sqlite3` compiled (from the workspace-level pnpm install).
Only the dist/-based prod-mimic path needed the explicit install.
Verification
------------
- `pnpm build` (with stubbed native binary): clean, dist/ has the
updated `dist/package.json` with the dependency declared
- `pnpm dist:install`: installs 38 packages, compiles
`dist/node_modules/better-sqlite3/build/Release/better_sqlite3.node`
- Smoke-test from `dist/`: `node -e "import('better-sqlite3')..."`
prints `OK: [ { id: 42 } ]`, confirming the binding loads + a
basic INSERT/SELECT roundtrip works.
CI matrix coverage: win-x64, linux-x64, linux-arm64 (release.yml +
dev-build.yml share the same matrix). Each builds its own native
binding during the install step — prebuild-install hits
better-sqlite3's prebuilt-binary registry for these three targets
(no source compile needed in the common case, falls back to
node-gyp if a prebuilt isn't available).
Replaces the experimental `node:sqlite` builtin with the stable
`better-sqlite3` wrapper across the two hot SQLite users — the
OneBot message-store and the bridge identity-service. While the API
surface is essentially identical (sync, prepared-statement-driven,
`Database`/`Statement` shape), better-sqlite3 (a) doesn't carry the
"ExperimentalWarning: SQLite is an experimental feature" baggage
that ships with `node:sqlite`, (b) has a more mature C++ binding
that consistently outperforms the builtin by ~30-50%, and (c)
encourages reusing `Statement` objects by design — exactly the win
the second half of this commit is after.
Why both at once
----------------
Caching prepared statements (was P1 in the RoadMap-#5 analysis) is
the largest single optimization opportunity on the SQLite path. The
old shape was the textbook SQLite anti-pattern:
// every call re-parsed the SQL
this.db.prepare(BIG_SQL_STRING).run(...);
Each `prepare(sql)` runs the full SQL parser + planner — for simple
inserts/updates/selects that dominates wall-clock cost. Doing both
the driver swap AND the statement caching in one commit lets the
benchmark numbers attribute the combined effect honestly; splitting
them would have double-charged the same disruption budget.
What changed code-wise
----------------------
**messageStore** — 8 distinct SQL templates, all now `readonly`
fields prepared once in the constructor:
- stmtStoreEvent — INSERT/UPDATE event row + JSON payload
- stmtStoreMeta — INSERT/UPDATE meta-only row
- stmtFindEvent — single-row SELECT by message_hash
- stmtFindMeta — single-row SELECT (full meta columns)
- stmtResolveReplyGroup, stmtResolveReplyPrivate
— 2 variants for the reply-id lookup
- stmtListEventsAnchored, stmtListEventsLatest
— 2 variants for /get_history_msg
**identity-service** — 25+ distinct statements. Used a Map-based
`pstmt(sql)` memoization helper rather than enumerating all 25 as
named fields (call sites stay readable; Map lookup is O(1) hash +
string-eq, negligible next to the SQL parse it replaces). The
helper short-circuits to the cached `Statement` on second-and-later
calls.
API differences from node:sqlite (handled in this commit):
- `import { DatabaseSync } from 'node:sqlite'`
→ `import Database from 'better-sqlite3'`
- `new DatabaseSync(path)` → `new Database(path)`
- `db.exec('PRAGMA journal_mode = WAL')`
→ `db.pragma('journal_mode = WAL')`
- `db.exec('BEGIN' / 'COMMIT' / 'ROLLBACK')` — kept as-is; both
drivers accept raw `exec()` for transaction control. The
`db.transaction()` higher-order wrapper is more idiomatic
but the existing manual control already works and changing
it would invalidate the `transaction()` helper's matched
try/catch shape.
vite build wiring:
- `node:sqlite` removed from the externals list
- `better-sqlite3` added — must stay external because it has a
native binding (`require('./build/Release/better_sqlite3.node')`)
that Vite can't bundle. Vendoring the prebuilt `.node` into
`packages/runtime/native/` is a follow-up packaging task; the
workspace `pnpm install` already places the binding under
`node_modules` so dev + tests work today.
Bench results (RoadMap #5 baseline → after)
-------------------------------------------
@snowluma/bridge — recv pipeline (with SQLite identity writes):
group_member_join 17.5 μs/op → 4.7 μs/op (~3.7x faster)
group_message text 0.42→0.42 μs (no path change)
group_message mixed 0.67→0.67 μs (no path change)
@snowluma/onebot — send pipeline:
messageStore.storeMeta 17.0 μs/op → 10.9 μs/op (~1.6x)
messageStore.storeEvent 23.2 μs/op → 18.2 μs/op (~1.3x)
messageStore.findMeta hit 5.4 μs/op → 1.3 μs/op (~4x)
messageStore.findMeta miss 3.7 μs/op → 0.7 μs/op (~5x)
messageStore.findEvent 3.8 μs/op → 1.7 μs/op (~2.3x)
Read paths gained the most because they're parse-bound (small SQL
+ short execution → the parser was most of the cost). Writes gained
less because the fsync/journal overhead on each commit still
dominates the parse savings. Combined send hot path (storeMeta +
storeEvent, called on every outbound message + the /get_msg-fix
self-event store) drops from ~40 μs to ~29 μs — about a 27%
reduction per message sent.
Verification
------------
- pnpm -r typecheck: all 11 projects clean
- @snowluma/onebot test suite: 250/250
- @snowluma/core test suite: 302/302
- Combined: 552/552 (unchanged baseline)
- Vite build (with stubbed native binary): clean, 2.84 MB / 223 KB
gzip (matches pre-Phase-2 envelope after removing the better-
sqlite3 inline-bundle attempt — externalizing it shaves the
~30 KB regression we saw mid-flight)
Follow-up (next commits in this series)
---------------------------------------
- Skip rememberGroupMemberIdentity DB write when the (uid, uin,
nickname, card) tuple is unchanged in the in-memory cache.
- Drop the `Buffer.from(pkt.body)` allocation in msg-push context.
- Synchronous fast-path for elementsToJson when no async resolver
is wired.
- Vendor `better-sqlite3` prebuilt `.node` into the runtime/native
layout for the production package step.
Add a repository .editorconfig and a root eslint.config.ts to standardize formatting and linting. Migrate packages/webui ESLint config to TypeScript (eslint.config.ts) with project-specific rules and tsconfigRootDir handling, and remove the old JS config. Update package.json to run ESLint directly (add lint and lint:fix) and add required ESLint/dev dependencies; also add jiti to packages/webui/package.json. Include a small formatting tweak to tsconfig.base.json.
Bumps version, pushes dev with chore(release): prefix to trigger
Promote, waits for the PR to merge, then tags main and pushes
the tag to kick off the release workflow.
- WebUI sidebar/login display and OneBot get_version_info now read the
root package.json version via Vite `define`, removing hardcoded
literals in webui/src/types.ts and core/onebot/actions/info.ts.
- tools/bump-version.mjs (exposed as `pnpm bump <ver>`) syncs the
version field across the root and all five workspace packages.
- Bump everything to 1.7.8.
Introduce a development-only WebUI auth mode controlled by SNOWLUMA_DEV_MODE. When enabled, a fixed in-memory password ("snowluma-dev") is used, config/webui.json is not read or written, and password changes are disabled. Expose WebuiAuth.isDevMode() and WebuiAuth.devPassword, add startup logs when dev mode is active, and wire up dev:mode npm scripts (root and packages/core) using cross-env to set the env flags.
Restructure the repository into a pnpm workspace under packages/ (core, webui, runtime). Move source files for core and webui into packages/*, add package manifests for @snowluma/core and @snowluma/runtime, and remove the old launcher package.json. Update root package.json to be a monorepo controller (scripts delegate to pnpm filters, set packageManager and Node engine). Configure build outputs to emit into the repo-root dist/ (vite config changes) so the existing release pipeline continues to work. Add .npmrc with mirrors, expand .gitignore for monorepo layout, add tsconfig base and package-level tsconfigs, and add vitest config for the core package. Adjust injector native search paths and update tests import paths to the new package layout. Small CI/workflow comment added to clarify build behavior and README updated with monorepo structure and common commands.
DrSmoothl