codex/async-sqlite-node24
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DrSmoothl
|
7c0519f75f | lint | ||
|
DrSmoothl
|
87a604ecbd |
feat(storage): add managed storage controls
Enforce a global log quota with configurable retention and observable degradation. Add authenticated storage reporting and safe cleanup controls to the settings UI. Protect active stream transfers and isolate parallel test resources. |
||
|
DrSmoothl
|
f3f5d27c65 |
perf(bench): bench harness for receive/send chains (RoadMap #5 baseline)
vitest-driven benches at `packages/bridge/bench/recv.bench.ts` and
`packages/onebot/bench/send.bench.ts`. Picks up proton's vite plugin
so call sites get the inlined codecs (running via tsx + proton
runtime-fallback would throw without a pre-built runtime map).
Run via `pnpm -F @snowluma/bridge bench` and
`pnpm -F @snowluma/onebot bench`.
What's measured
---------------
Receive chain:
- parseMsgPush: pure decode + dispatch (no SQLite, no event emit)
* group_message text ~ 0.29 μs/op (3.4M ops/sec)
* group_message 5 mixed ~ 0.50 μs/op (2.0M ops/sec)
* group_member_join ~ 0.25 μs/op (4.1M ops/sec)
- IncomingPacketPipeline.process: full pipeline (with SQLite via
rememberGroupMemberIdentity, BridgeEventBus emit)
* group_message text ~ 0.42 μs/op (2.4M ops/sec)
* group_message 5 mixed ~ 0.67 μs/op (1.5M ops/sec)
* group_member_join ~17.5 μs/op ( 57K ops/sec) ⚠
Send chain:
- parseMessage (OneBot input → MessageElement[])
* CQ string text ~ 0.31 μs/op
* CQ string mixed ~ 1.22 μs/op
* segment array text ~ 0.32 μs/op
* segment array 5 ~ 1.07 μs/op
- messageStore SQLite writes:
* storeMeta ~17.0 μs/op ( 59K ops/sec) ⚠
* storeEvent (~250B JSON) ~23.2 μs/op ( 43K ops/sec) ⚠
- messageStore reads:
* findMeta hit ~ 5.4 μs/op (186K ops/sec)
* findMeta miss ~ 3.7 μs/op (272K ops/sec)
* findEvent hit ~ 3.8 μs/op (262K ops/sec)
Headline finding
----------------
Pure decode/encode is already cheap (sub-μs). The real cost per
sent message is the SQLite write pair (storeMeta + storeEvent =
~40 μs). On receive, group_message itself does NOT trigger
identity writes; only join/leave/admin events do — but those go
through 4-6 db.prepare(sql).run/get(...) calls each, all re-parsing
SQL on every invocation (the canonical SQLite anti-pattern).
The follow-up commits in this series target:
1. Replace `node:sqlite` (experimental, API may shift) with
`better-sqlite3` (stable, sync, ~30-50% faster wire) + cache
prepared statements as readonly fields. Single commit, both
wins together — better-sqlite3's API encourages stmt caching
by design. Expected: storeMeta / storeEvent drop to ~3-5 μs,
group_member_join drops to ~3-5 μs.
2. Skip rememberGroupMemberIdentity DB write when the in-memory
uidByUin / uinByUid cache already has the mapping unchanged.
3. Drop the `Buffer.from(pkt.body)` allocation in msg-push
context (pkt.body is already Uint8Array; proton accepts it).
4. Make `elementsToJson` sync-fast-path when no async resolvers
are wired (saves per-element microtask overhead).
Each commit re-runs these benches and the message should quote the
before/after numbers so the deltas are auditable.
Note: tsx is NOT involved — vitest's bench mode owns the run, and
the proton vite plugin is configured in each `vitest.config.ts`.
Absolute numbers reflect the production code path.
|
||
|
DrSmoothl
|
9a30df9907 |
refactor(onebot): extract OneBot protocol translation into @snowluma/onebot
Phase 3 of the 4-package split (RoadMap #3). Moves the entire `packages/core/src/onebot/` subtree (43 source files + 20 test files spanning network adapters, action handlers, event converters, message parser, message store, media store, instance + manager) into the new `@snowluma/onebot` workspace package. The remaining `@snowluma/core` now hosts only the api-layer (Bridge class + 13 Api classes), the hook tooling, the WebUI server, and the runnable `index.ts`. Why --- - The OneBot adapter is the public-facing OneBot11 protocol surface; it has its own version cadence and its own consumers (third-party bots talking OneBot WS / HTTP). Splitting it lets that cadence move independently of the Api-on-ctx layer. - The two were sharing the `packages/core/` build pipeline only because they happened to live in the same directory. Nothing in the api-layer reaches into OneBot internals. - Mirrors NapCat / Lagrange.OneBot's repo split (their OneBot adapter is a separate assembly from the Core protocol library). Layout ------ Moved via `git mv` (history preserved): packages/core/src/onebot/* → @snowluma/onebot/src/<x> packages/core/tests/onebot/* (6 files) → @snowluma/onebot/tests/ packages/core/tests/{message-parser,message-store,event-pipeline, media-store,media-url-resolver,event-converter, http-server-hot-reload,http-server-content-type, api-handler-logging,message-id,group-msg-emoji-like, config,media-indexer,network-manager}.test.ts (14 files) → @snowluma/onebot/tests/ What stays in @snowluma/core ---------------------------- - `bridge/{bridge,bridge-interface,bridge-context,manager}.ts` - `bridge/apis/*` (the 13 typed Api classes) - `hook/*` (debugger/injector — tooling for the core protocol layer) - `webui/*` (admin UI server — application wiring, not the OneBot adapter itself) - `index.ts` (application entry that wires BridgeManager + OneBotManager + HookManager + WebUI together) Changes ------- - New package `packages/onebot/` (package.json with subpath exports mirroring legacy `core/src/onebot/<x>` 1:1, tsconfig.json, vitest.config.ts with `SNOWLUMA_LOG_LEVEL=debug` env so the action- dispatch logging tests see debug-level subscribers — copied verbatim from core's vitest.config.ts). - @snowluma/core grows new subpath exports for the api-layer files OneBot's type-only imports need: `./bridge`, `./bridge-interface`, `./bridge-context`, `./manager`, `./apis`. - `@snowluma/onebot` depends on `@snowluma/bridge`, `@snowluma/common`, `@snowluma/core`, `@snowluma/proto-defs`, `@snowluma/proton`, `@snowluma/websocket`. - `@snowluma/core` depends on `@snowluma/onebot` (for index.ts + webui/server.ts wiring). - 100+ import sites rewritten via sed: * OneBot → core: `'../bridge/bridge'` → `'@snowluma/core/bridge'`, `'../bridge/manager'` → `'@snowluma/core/manager'`, `'../bridge/bridge-interface'` → `'@snowluma/core/bridge-interface'`, `'@/bridge/bridge'` (vite path alias) → `'@snowluma/core/bridge'`. * Core → OneBot: `'./onebot/manager'` → `'@snowluma/onebot/manager'`, `'../onebot/config'` → `'@snowluma/onebot/config'`, etc. * Test relative paths: `'../../src/onebot/<x>'` → `'../src/<x>'` after the test file move (one level shallower). - New `packages/onebot/src/env.d.ts` declaring `__APP_VERSION__` and `__BUILD_WEBUI__` (the vite-injected globals `info.ts` reads). Workspace cycle --------------- core ↔ onebot is allowed by pnpm workspaces. The runtime direction is one-way: core → onebot (index.ts constructs OneBotManager, webui/server reads OneBot config). OneBot only reaches into core for TYPES (Bridge, BridgeInterface, BridgeManager, ClientKeyInfo, DownloadRKeyInfo); those imports use `import type` so they compile out at runtime and don't introduce a real cycle. Verification ------------ - Full workspace `pnpm -r typecheck`: all 11 projects clean. - `@snowluma/core` test suite: 302/302 pass (33 test files). - `@snowluma/onebot` test suite: 250/250 pass (20 test files; pre-existing `network-manager.test.ts` native-addon failure unchanged — same baseline as before the split). - Combined: 552/552 — identical pass count to the pre-Phase-3 baseline. - Vite build with stubbed native binary: clean, 2.84 MB bundle / 224 KB gzip (identical envelope). |