dev2
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 手瓜一十雪
|
adf0744f75 |
Rename bridge → channel and reorganize packages
Move/rename the transport/bridge code into a new @snowluma/channel layout and extract hook implementations into channel-specific packages. Many bridge sources were deleted and equivalent channel files added (adapter/ctx/interface/channel/hook/socket pieces moved/renamed), along with tests and configs relocated under packages/channel. RoadMap.md replaced with a detailed v3 architecture plan describing Channel/ChannelCtx/Core/CoreCtx/Hub boundaries and migration steps. Also apply related core package updates (account/core reshaping and package.json adjustments) to match the new architecture. |
||
|
手瓜一十雪
|
652bafd75a |
Extract bridge transport into @snowluma/bridge
Move transport-layer bridge code out of @snowluma/core into a dedicated @snowluma/bridge package and introduce a clear account layer in core. Adds Bridge, BridgeInterface, InjectBridge, ProtocolBridge (stub), adapters, and manager files under packages/bridge and updates package.json exports. In @snowluma/core the former bridge surface is replaced by Account, AccountContext, AccountInterface and AccountManager; apis were moved/renamed to account/apis and now consume AccountContext. Adjustments include packet handling plumbing (use deliverPacket / setPacketHandler), updated sendRawPacket wiring, uploaded-file cache and lifecycle changes (InjectBridge.dispose, ProtocolBridge returns a structured not-implemented result). Tests and imports were updated accordingly. |
||
|
手瓜一十雪
|
2c565fbd8d |
Switch from @snowluma/sqlite to node:sqlite (sync)
Remove the in-tree @snowluma/sqlite wrapper and vendored better-sqlite3 prebuilts; migrate consumers to node:sqlite's synchronous API. Replaced imports and types (Database/Statement -> DatabaseSync/StatementSync), updated PRAGMA calls to use exec, and adjusted code in onebot (media/message/reaction stores) and protocol (identity service). Vite config no longer bundles sqlite native binaries and references to the sqlite workspace were removed. Also delete the packages/sqlite package and its native prebuild artifacts, and add a couple of unrelated AST improvements in proton: use a module-level shared wrapper cache and add an inheritance cache to avoid repeated scans. Minor import reordering/cleanup applied where needed. |
||
|
DrSmoothl
|
5924346823 | chore(release): v1.9.2 | ||
|
DrSmoothl
|
c16064d7e3 |
refactor(packages): rename @snowluma/bridge → @snowluma/protocol
The package previously named "bridge" houses the protocol-byte layer (OIDB cmds, msg-push decoders, highway uploaders, element-builder, event bus). The name "bridge" suggested transport/lifecycle, which it isn't — that role lives in core/hook and core/bridge/manager. Rename to match what's actually inside. This is a pure rename: zero logic changes, all 73 import sites updated mechanically and TS-verified. core/onebot dependencies key updated. ESLint override path adjusted for the moved oidb-services directory. Rename detection threshold: git log --find-renames=20% recommended when following history across this commit — content-replacement (@snowluma/bridge → @snowluma/protocol) dropped per-file similarity below git's default 50% on import-heavy files. |
||
|
DrSmoothl
|
ee0f159b7e |
fix(deps): drop dead hono devDeps from @snowluma/onebot, align core to ^4.12.18
`packages/onebot` declared `hono` and `@hono/node-server` as devDeps but
never imported either — they were leftover scaffolding from when onebot
was carved out of `@snowluma/core`. Combined with dependabot's hono
bump only landing on core's pin (4.12.14 → 4.12.18 in
|
||
|
DrSmoothl
|
607a7b319b | chore(release): v1.9.1 | ||
|
DrSmoothl
|
4e51b4bd7d |
build(sqlite): wrap better-sqlite3 with @snowluma/sqlite + vendor prebuilts
Replaces "runtime npm dependency that gets `npm install`-ed into
dist/" with "TS wrapper that loads the native `.node` from
`dist/native/` like every other native binding in the project".
Why the change
--------------
The previous approach (better-sqlite3 declared in
packages/runtime/package.json, materialized by `npm install
--omit=dev` in dist/) worked but produced an artifact that needed
runtime npm install — a step the rest of SnowLuma's native bindings
don't need (websocket / snowluma-hook / ffmpeg are all standalone
`.node` files copied into `dist/native/` by vite-plugin-cp). The
asymmetry was friction: dev mode required `pnpm dist:install`,
build artifacts carried a `dist/node_modules/` tree the other
modules didn't need, and Docker images had to special-case the
glibc-vs-musl ABI lookup that prebuild-install runs.
The new shape:
- `@snowluma/sqlite` (packages/sqlite/) — TS wrapper. Has
`better-sqlite3` as a devDep (for the JS-layer types + the
actual JS wrapper code that vite inlines). At runtime, calls
`createRequire().require()` against
`<dist|repo>/native/better-sqlite3-v<abi>-<platform>-<arch>.node`
and passes the resolved addon to `new Database(path, {
nativeBinding })`.
- `packages/runtime/native/better-sqlite3-v<abi>-<triplet>.node`
— prebuilt binaries vendored into the repo. Two ABIs ship side-
by-side (v127 = Node 22 LTS used by CI, v137 = Node 24 used by
dev boxes) so `process.versions.modules` picks the matching one
at load time.
- `tools/fetch-sqlite-prebuilts.mjs` — fetches all 5 platforms ×
N ABIs from WiseLibs's GitHub releases. Idempotent: existing
`.node` files skip. Accepts extra ABI args:
`node tools/fetch-sqlite-prebuilts.mjs 12.10.0 v127 v137 v115`.
What was unwound
----------------
- packages/runtime/package.json — dependencies removed (runtime
artifact is now self-contained, no `npm install` step needed).
- root package.json — `dist:install` script removed (no node_modules
to install).
- .github/workflows/dev-build.yml — `Bundle runtime dependencies`
step removed.
- .github/workflows/release.yml — kept the `Bundle runtime
dependencies` step (was a no-op already, but harmless safety net
in case a future runtime dep shows up).
- packages/core/vite.config.ts — `better-sqlite3` no longer
external; vite inlines the JS wrapper (+~33 KB to bundle, ~2.84
MB → 2.87 MB total).
Naming
------
Mirrors the existing `websocket-${triplet}.node` convention (flat
dir under `runtime/native/`, dash-separated `<platform>-<arch>`).
Adds a `v<abi>` segment because better-sqlite3 is NOT N-API — one
binary per Node ABI. Naming:
better-sqlite3-v127-darwin-arm64.node Node 22 / macOS arm
better-sqlite3-v127-darwin-x64.node Node 22 / macOS intel
better-sqlite3-v127-linux-arm64.node Node 22 / Linux arm
better-sqlite3-v127-linux-x64.node Node 22 / Linux x64
better-sqlite3-v127-win32-x64.node Node 22 / Windows x64
better-sqlite3-v137-... (5) Node 24 / same 5 hosts
Total: 10 binaries × ~1.9 MB = ~19 MB committed. Aligned with the
existing native-vendoring footprint (websocket ~2 MB × 5, snowluma-
hook ~10 MB × Windows + Linux, ffmpeg ~10-30 MB × 4).
What changes for consumers
--------------------------
Source-side: nothing. `import Database, { type Database, type
Statement } from '@snowluma/sqlite'` is a drop-in for the previous
`import Database from 'better-sqlite3'`. The wrapper re-exports the
same types and the `Database` callable factory has the same
overload set.
Runtime: `node` finds the binding at
`<dist|repo>/native/better-sqlite3-v${process.versions.modules}-
${process.platform}-${process.arch}.node`. If the ABI doesn't
match a vendored binary, the wrapper throws a clear error pointing
at `tools/fetch-sqlite-prebuilts.mjs` with the missing ABI/triplet.
Bumped better-sqlite3 from 11.10.0 → 12.10.0 because v11 only had
prebuilt up to Node ABI v127 (no v137 = no Node 24). v12.10.0
ships both. Both versions are API-compat (the breaking changes
between v11 and v12 were either internal or in error-handling
paths SnowLuma doesn't exercise).
Verification
------------
- pnpm -r typecheck (12 projects): clean
- @snowluma/core test: 302/302
- @snowluma/onebot test: 250/250 (552/552 combined, unchanged from
the perf-series baseline)
- Bench numbers preserved — wrapper indirection is sub-μs:
storeMeta ~11 μs, storeEvent ~17 μs, findMeta hit ~1.1 μs (all
within noise of the post-RoadMap-#5 measurements).
- Vite build clean, bundle = 2.87 MB / 230 KB gzip
- `dist/index.mjs` no longer contains a `node:sqlite` import (it
did before — media-store hadn't migrated; now neither it nor
message-store reach into the Node builtin).
Bonus side-fix: `packages/onebot/src/media-store.ts` also moved
off `node:sqlite` (caught by the post-build bundle grep — the
file was using `DatabaseSync` + `StatementSync` from the builtin
even though it had already been written to cache its prepared
statements). Its hot paths now go through the same wrapper.
|
||
|
DrSmoothl
|
0b6541ca36 |
perf(sqlite): swap node:sqlite for better-sqlite3 + cache prepared stmts
Replaces the experimental `node:sqlite` builtin with the stable
`better-sqlite3` wrapper across the two hot SQLite users — the
OneBot message-store and the bridge identity-service. While the API
surface is essentially identical (sync, prepared-statement-driven,
`Database`/`Statement` shape), better-sqlite3 (a) doesn't carry the
"ExperimentalWarning: SQLite is an experimental feature" baggage
that ships with `node:sqlite`, (b) has a more mature C++ binding
that consistently outperforms the builtin by ~30-50%, and (c)
encourages reusing `Statement` objects by design — exactly the win
the second half of this commit is after.
Why both at once
----------------
Caching prepared statements (was P1 in the RoadMap-#5 analysis) is
the largest single optimization opportunity on the SQLite path. The
old shape was the textbook SQLite anti-pattern:
// every call re-parsed the SQL
this.db.prepare(BIG_SQL_STRING).run(...);
Each `prepare(sql)` runs the full SQL parser + planner — for simple
inserts/updates/selects that dominates wall-clock cost. Doing both
the driver swap AND the statement caching in one commit lets the
benchmark numbers attribute the combined effect honestly; splitting
them would have double-charged the same disruption budget.
What changed code-wise
----------------------
**messageStore** — 8 distinct SQL templates, all now `readonly`
fields prepared once in the constructor:
- stmtStoreEvent — INSERT/UPDATE event row + JSON payload
- stmtStoreMeta — INSERT/UPDATE meta-only row
- stmtFindEvent — single-row SELECT by message_hash
- stmtFindMeta — single-row SELECT (full meta columns)
- stmtResolveReplyGroup, stmtResolveReplyPrivate
— 2 variants for the reply-id lookup
- stmtListEventsAnchored, stmtListEventsLatest
— 2 variants for /get_history_msg
**identity-service** — 25+ distinct statements. Used a Map-based
`pstmt(sql)` memoization helper rather than enumerating all 25 as
named fields (call sites stay readable; Map lookup is O(1) hash +
string-eq, negligible next to the SQL parse it replaces). The
helper short-circuits to the cached `Statement` on second-and-later
calls.
API differences from node:sqlite (handled in this commit):
- `import { DatabaseSync } from 'node:sqlite'`
→ `import Database from 'better-sqlite3'`
- `new DatabaseSync(path)` → `new Database(path)`
- `db.exec('PRAGMA journal_mode = WAL')`
→ `db.pragma('journal_mode = WAL')`
- `db.exec('BEGIN' / 'COMMIT' / 'ROLLBACK')` — kept as-is; both
drivers accept raw `exec()` for transaction control. The
`db.transaction()` higher-order wrapper is more idiomatic
but the existing manual control already works and changing
it would invalidate the `transaction()` helper's matched
try/catch shape.
vite build wiring:
- `node:sqlite` removed from the externals list
- `better-sqlite3` added — must stay external because it has a
native binding (`require('./build/Release/better_sqlite3.node')`)
that Vite can't bundle. Vendoring the prebuilt `.node` into
`packages/runtime/native/` is a follow-up packaging task; the
workspace `pnpm install` already places the binding under
`node_modules` so dev + tests work today.
Bench results (RoadMap #5 baseline → after)
-------------------------------------------
@snowluma/bridge — recv pipeline (with SQLite identity writes):
group_member_join 17.5 μs/op → 4.7 μs/op (~3.7x faster)
group_message text 0.42→0.42 μs (no path change)
group_message mixed 0.67→0.67 μs (no path change)
@snowluma/onebot — send pipeline:
messageStore.storeMeta 17.0 μs/op → 10.9 μs/op (~1.6x)
messageStore.storeEvent 23.2 μs/op → 18.2 μs/op (~1.3x)
messageStore.findMeta hit 5.4 μs/op → 1.3 μs/op (~4x)
messageStore.findMeta miss 3.7 μs/op → 0.7 μs/op (~5x)
messageStore.findEvent 3.8 μs/op → 1.7 μs/op (~2.3x)
Read paths gained the most because they're parse-bound (small SQL
+ short execution → the parser was most of the cost). Writes gained
less because the fsync/journal overhead on each commit still
dominates the parse savings. Combined send hot path (storeMeta +
storeEvent, called on every outbound message + the /get_msg-fix
self-event store) drops from ~40 μs to ~29 μs — about a 27%
reduction per message sent.
Verification
------------
- pnpm -r typecheck: all 11 projects clean
- @snowluma/onebot test suite: 250/250
- @snowluma/core test suite: 302/302
- Combined: 552/552 (unchanged baseline)
- Vite build (with stubbed native binary): clean, 2.84 MB / 223 KB
gzip (matches pre-Phase-2 envelope after removing the better-
sqlite3 inline-bundle attempt — externalizing it shaves the
~30 KB regression we saw mid-flight)
Follow-up (next commits in this series)
---------------------------------------
- Skip rememberGroupMemberIdentity DB write when the (uid, uin,
nickname, card) tuple is unchanged in the in-memory cache.
- Drop the `Buffer.from(pkt.body)` allocation in msg-push context.
- Synchronous fast-path for elementsToJson when no async resolver
is wired.
- Vendor `better-sqlite3` prebuilt `.node` into the runtime/native
layout for the production package step.
|
||
|
DrSmoothl
|
f3f5d27c65 |
perf(bench): bench harness for receive/send chains (RoadMap #5 baseline)
vitest-driven benches at `packages/bridge/bench/recv.bench.ts` and
`packages/onebot/bench/send.bench.ts`. Picks up proton's vite plugin
so call sites get the inlined codecs (running via tsx + proton
runtime-fallback would throw without a pre-built runtime map).
Run via `pnpm -F @snowluma/bridge bench` and
`pnpm -F @snowluma/onebot bench`.
What's measured
---------------
Receive chain:
- parseMsgPush: pure decode + dispatch (no SQLite, no event emit)
* group_message text ~ 0.29 μs/op (3.4M ops/sec)
* group_message 5 mixed ~ 0.50 μs/op (2.0M ops/sec)
* group_member_join ~ 0.25 μs/op (4.1M ops/sec)
- IncomingPacketPipeline.process: full pipeline (with SQLite via
rememberGroupMemberIdentity, BridgeEventBus emit)
* group_message text ~ 0.42 μs/op (2.4M ops/sec)
* group_message 5 mixed ~ 0.67 μs/op (1.5M ops/sec)
* group_member_join ~17.5 μs/op ( 57K ops/sec) ⚠
Send chain:
- parseMessage (OneBot input → MessageElement[])
* CQ string text ~ 0.31 μs/op
* CQ string mixed ~ 1.22 μs/op
* segment array text ~ 0.32 μs/op
* segment array 5 ~ 1.07 μs/op
- messageStore SQLite writes:
* storeMeta ~17.0 μs/op ( 59K ops/sec) ⚠
* storeEvent (~250B JSON) ~23.2 μs/op ( 43K ops/sec) ⚠
- messageStore reads:
* findMeta hit ~ 5.4 μs/op (186K ops/sec)
* findMeta miss ~ 3.7 μs/op (272K ops/sec)
* findEvent hit ~ 3.8 μs/op (262K ops/sec)
Headline finding
----------------
Pure decode/encode is already cheap (sub-μs). The real cost per
sent message is the SQLite write pair (storeMeta + storeEvent =
~40 μs). On receive, group_message itself does NOT trigger
identity writes; only join/leave/admin events do — but those go
through 4-6 db.prepare(sql).run/get(...) calls each, all re-parsing
SQL on every invocation (the canonical SQLite anti-pattern).
The follow-up commits in this series target:
1. Replace `node:sqlite` (experimental, API may shift) with
`better-sqlite3` (stable, sync, ~30-50% faster wire) + cache
prepared statements as readonly fields. Single commit, both
wins together — better-sqlite3's API encourages stmt caching
by design. Expected: storeMeta / storeEvent drop to ~3-5 μs,
group_member_join drops to ~3-5 μs.
2. Skip rememberGroupMemberIdentity DB write when the in-memory
uidByUin / uinByUid cache already has the mapping unchanged.
3. Drop the `Buffer.from(pkt.body)` allocation in msg-push
context (pkt.body is already Uint8Array; proton accepts it).
4. Make `elementsToJson` sync-fast-path when no async resolvers
are wired (saves per-element microtask overhead).
Each commit re-runs these benches and the message should quote the
before/after numbers so the deltas are auditable.
Note: tsx is NOT involved — vitest's bench mode owns the run, and
the proton vite plugin is configured in each `vitest.config.ts`.
Absolute numbers reflect the production code path.
|
||
|
DrSmoothl
|
9a30df9907 |
refactor(onebot): extract OneBot protocol translation into @snowluma/onebot
Phase 3 of the 4-package split (RoadMap #3). Moves the entire `packages/core/src/onebot/` subtree (43 source files + 20 test files spanning network adapters, action handlers, event converters, message parser, message store, media store, instance + manager) into the new `@snowluma/onebot` workspace package. The remaining `@snowluma/core` now hosts only the api-layer (Bridge class + 13 Api classes), the hook tooling, the WebUI server, and the runnable `index.ts`. Why --- - The OneBot adapter is the public-facing OneBot11 protocol surface; it has its own version cadence and its own consumers (third-party bots talking OneBot WS / HTTP). Splitting it lets that cadence move independently of the Api-on-ctx layer. - The two were sharing the `packages/core/` build pipeline only because they happened to live in the same directory. Nothing in the api-layer reaches into OneBot internals. - Mirrors NapCat / Lagrange.OneBot's repo split (their OneBot adapter is a separate assembly from the Core protocol library). Layout ------ Moved via `git mv` (history preserved): packages/core/src/onebot/* → @snowluma/onebot/src/<x> packages/core/tests/onebot/* (6 files) → @snowluma/onebot/tests/ packages/core/tests/{message-parser,message-store,event-pipeline, media-store,media-url-resolver,event-converter, http-server-hot-reload,http-server-content-type, api-handler-logging,message-id,group-msg-emoji-like, config,media-indexer,network-manager}.test.ts (14 files) → @snowluma/onebot/tests/ What stays in @snowluma/core ---------------------------- - `bridge/{bridge,bridge-interface,bridge-context,manager}.ts` - `bridge/apis/*` (the 13 typed Api classes) - `hook/*` (debugger/injector — tooling for the core protocol layer) - `webui/*` (admin UI server — application wiring, not the OneBot adapter itself) - `index.ts` (application entry that wires BridgeManager + OneBotManager + HookManager + WebUI together) Changes ------- - New package `packages/onebot/` (package.json with subpath exports mirroring legacy `core/src/onebot/<x>` 1:1, tsconfig.json, vitest.config.ts with `SNOWLUMA_LOG_LEVEL=debug` env so the action- dispatch logging tests see debug-level subscribers — copied verbatim from core's vitest.config.ts). - @snowluma/core grows new subpath exports for the api-layer files OneBot's type-only imports need: `./bridge`, `./bridge-interface`, `./bridge-context`, `./manager`, `./apis`. - `@snowluma/onebot` depends on `@snowluma/bridge`, `@snowluma/common`, `@snowluma/core`, `@snowluma/proto-defs`, `@snowluma/proton`, `@snowluma/websocket`. - `@snowluma/core` depends on `@snowluma/onebot` (for index.ts + webui/server.ts wiring). - 100+ import sites rewritten via sed: * OneBot → core: `'../bridge/bridge'` → `'@snowluma/core/bridge'`, `'../bridge/manager'` → `'@snowluma/core/manager'`, `'../bridge/bridge-interface'` → `'@snowluma/core/bridge-interface'`, `'@/bridge/bridge'` (vite path alias) → `'@snowluma/core/bridge'`. * Core → OneBot: `'./onebot/manager'` → `'@snowluma/onebot/manager'`, `'../onebot/config'` → `'@snowluma/onebot/config'`, etc. * Test relative paths: `'../../src/onebot/<x>'` → `'../src/<x>'` after the test file move (one level shallower). - New `packages/onebot/src/env.d.ts` declaring `__APP_VERSION__` and `__BUILD_WEBUI__` (the vite-injected globals `info.ts` reads). Workspace cycle --------------- core ↔ onebot is allowed by pnpm workspaces. The runtime direction is one-way: core → onebot (index.ts constructs OneBotManager, webui/server reads OneBot config). OneBot only reaches into core for TYPES (Bridge, BridgeInterface, BridgeManager, ClientKeyInfo, DownloadRKeyInfo); those imports use `import type` so they compile out at runtime and don't introduce a real cycle. Verification ------------ - Full workspace `pnpm -r typecheck`: all 11 projects clean. - `@snowluma/core` test suite: 302/302 pass (33 test files). - `@snowluma/onebot` test suite: 250/250 pass (20 test files; pre-existing `network-manager.test.ts` native-addon failure unchanged — same baseline as before the split). - Combined: 552/552 — identical pass count to the pre-Phase-3 baseline. - Vite build with stubbed native binary: clean, 2.84 MB bundle / 224 KB gzip (identical envelope). |