DrSmoothl 0b6541ca36 perf(sqlite): swap node:sqlite for better-sqlite3 + cache prepared stmts
Replaces the experimental `node:sqlite` builtin with the stable
`better-sqlite3` wrapper across the two hot SQLite users — the
OneBot message-store and the bridge identity-service. While the API
surface is essentially identical (sync, prepared-statement-driven,
`Database`/`Statement` shape), better-sqlite3 (a) doesn't carry the
"ExperimentalWarning: SQLite is an experimental feature" baggage
that ships with `node:sqlite`, (b) has a more mature C++ binding
that consistently outperforms the builtin by ~30-50%, and (c)
encourages reusing `Statement` objects by design — exactly the win
the second half of this commit is after.

Why both at once
----------------
Caching prepared statements (was P1 in the RoadMap-#5 analysis) is
the largest single optimization opportunity on the SQLite path. The
old shape was the textbook SQLite anti-pattern:

  // every call re-parsed the SQL
  this.db.prepare(BIG_SQL_STRING).run(...);

Each `prepare(sql)` runs the full SQL parser + planner — for simple
inserts/updates/selects that dominates wall-clock cost. Doing both
the driver swap AND the statement caching in one commit lets the
benchmark numbers attribute the combined effect honestly; splitting
them would have double-charged the same disruption budget.

What changed code-wise
----------------------
**messageStore** — 8 distinct SQL templates, all now `readonly`
fields prepared once in the constructor:
  - stmtStoreEvent      — INSERT/UPDATE event row + JSON payload
  - stmtStoreMeta       — INSERT/UPDATE meta-only row
  - stmtFindEvent       — single-row SELECT by message_hash
  - stmtFindMeta        — single-row SELECT (full meta columns)
  - stmtResolveReplyGroup, stmtResolveReplyPrivate
                        — 2 variants for the reply-id lookup
  - stmtListEventsAnchored, stmtListEventsLatest
                        — 2 variants for /get_history_msg

**identity-service** — 25+ distinct statements. Used a Map-based
`pstmt(sql)` memoization helper rather than enumerating all 25 as
named fields (call sites stay readable; Map lookup is O(1) hash +
string-eq, negligible next to the SQL parse it replaces). The
helper short-circuits to the cached `Statement` on second-and-later
calls.

API differences from node:sqlite (handled in this commit):
  - `import { DatabaseSync } from 'node:sqlite'`
       → `import Database from 'better-sqlite3'`
  - `new DatabaseSync(path)` → `new Database(path)`
  - `db.exec('PRAGMA journal_mode = WAL')`
       → `db.pragma('journal_mode = WAL')`
  - `db.exec('BEGIN' / 'COMMIT' / 'ROLLBACK')` — kept as-is; both
       drivers accept raw `exec()` for transaction control. The
       `db.transaction()` higher-order wrapper is more idiomatic
       but the existing manual control already works and changing
       it would invalidate the `transaction()` helper's matched
       try/catch shape.

vite build wiring:
  - `node:sqlite` removed from the externals list
  - `better-sqlite3` added — must stay external because it has a
    native binding (`require('./build/Release/better_sqlite3.node')`)
    that Vite can't bundle. Vendoring the prebuilt `.node` into
    `packages/runtime/native/` is a follow-up packaging task; the
    workspace `pnpm install` already places the binding under
    `node_modules` so dev + tests work today.

Bench results (RoadMap #5 baseline → after)
-------------------------------------------
@snowluma/bridge — recv pipeline (with SQLite identity writes):
  group_member_join         17.5 μs/op → 4.7 μs/op   (~3.7x faster)
  group_message text         0.42→0.42 μs            (no path change)
  group_message mixed        0.67→0.67 μs            (no path change)

@snowluma/onebot — send pipeline:
  messageStore.storeMeta    17.0 μs/op → 10.9 μs/op  (~1.6x)
  messageStore.storeEvent   23.2 μs/op → 18.2 μs/op  (~1.3x)
  messageStore.findMeta hit  5.4 μs/op →  1.3 μs/op  (~4x)
  messageStore.findMeta miss 3.7 μs/op →  0.7 μs/op  (~5x)
  messageStore.findEvent     3.8 μs/op →  1.7 μs/op  (~2.3x)

Read paths gained the most because they're parse-bound (small SQL
+ short execution → the parser was most of the cost). Writes gained
less because the fsync/journal overhead on each commit still
dominates the parse savings. Combined send hot path (storeMeta +
storeEvent, called on every outbound message + the /get_msg-fix
self-event store) drops from ~40 μs to ~29 μs — about a 27%
reduction per message sent.

Verification
------------
- pnpm -r typecheck: all 11 projects clean
- @snowluma/onebot test suite: 250/250
- @snowluma/core test suite: 302/302
- Combined: 552/552 (unchanged baseline)
- Vite build (with stubbed native binary): clean, 2.84 MB / 223 KB
  gzip (matches pre-Phase-2 envelope after removing the better-
  sqlite3 inline-bundle attempt — externalizing it shaves the
  ~30 KB regression we saw mid-flight)

Follow-up (next commits in this series)
---------------------------------------
- Skip rememberGroupMemberIdentity DB write when the (uid, uin,
  nickname, card) tuple is unchanged in the in-memory cache.
- Drop the `Buffer.from(pkt.body)` allocation in msg-push context.
- Synchronous fast-path for elementsToJson when no async resolver
  is wired.
- Vendor `better-sqlite3` prebuilt `.node` into the runtime/native
  layout for the production package step.
2026-05-21 10:34:37 +08:00
2026-05-03 14:02:29 +08:00

❄️ SnowLuma

Next Remote Protocol Framework.

Release Build @snowluma/sdk @snowluma/ui Stars

Releases · Issues · QQ 群 · Telegram


简介

SnowLuma 在 QQ 客户端与生态侧之间承担协议转换工作,向上以 OneBot v11 标准协议暴露能力。Bridge 层负责会话与事件枢纽,再通过 WebSocket / HTTP / WebUI 等多种适配器对外提供接入。

项目持续迭代中,稳定版本通过 Releases 发布。

特性

  • OneBot v11 兼容 —— 涵盖文本、图片、视频、语音、合并转发、戳一戳、Markdown、JSON、XML 等常用消息段。
  • 多账号并行 —— 同一进程可同时托管多个 QQ 账号,会话彼此独立。
  • 多种适配器 —— WebSocket Server / WebSocket Client / HTTP Server / HTTP Post,HTTP 服务端同时支持 application/json 与 application/x-www-form-urlencoded。
  • WebUI 管理面板 —— 浏览器登录、密码强度校验、登录限流、SSE 实时日志、配置热更新。
  • TypeScript 全链路 —— Bridge / OneBot / SDK / UI 均使用 TypeScript,发布到 npm 的 @snowluma/sdk 与 @snowluma/ui 可直接供下游应用消费。
  • 数据持久化 —— 好友、群、群成员等关系数据落 SQLite,跨重启可用。

快速开始

下载预构建发布包:

# 从 Releases 下载 zip,解压到任意目录
./launcher.bat        # Windows

或从源码开发:

git clone https://github.com/SnowLuma/SnowLuma.git
cd SnowLuma
pnpm install
pnpm build
pnpm start

WebUI 默认监听 5099 端口。首次启动会在控制台打印 initial credentials: user=admin password=<...>,登录后请立即修改。 若未完成首次改密就关闭进程,下次启动会自动重新生成一个新的随机密码。

工作区

本仓库使用 pnpm monorepo 管理,各包职责如下:

包 说明
@snowluma/core 核心:协议层 / Bridge / OneBot / WebUI 全部在此。
@snowluma/sdk OneBot HTTP & WebSocket TypeScript SDK,发布到 npm。
@snowluma/ui Tailwind 4 共享组件库(shadcn 风格),发布到 npm。
@snowluma/websocket OneBot WebSocket 适配器底层。
@snowluma/proton 编译期 protobuf 编/解码器生成器:TS interface 标注 pb<N, T>,Vite 插件在构建时把 protobuf_encode<T> / protobuf_decode<T> 替换为内联 codec,零运行时反射。
@snowluma/runtime 运行时清单与发布包资源。
webui Web 管理面板前端。

架构

flowchart TD
    A([QQ 客户端])
    B["协议层<br/><sub>packet pipeline · cmd 分派</sub>"]
    C["Bridge 层<br/><sub>会话调度 · 花名册缓存 · 消息发送</sub>"]
    D["OneBot 层<br/><sub>Event Converter · Action Handler</sub>"]
    E([WS / HTTP / WebUI])

    A --> B --> C --> D --> E

文档

开发命令

pnpm dev          # 启动 core dev server
pnpm dev:web      # 启动 webui dev server
pnpm test         # 运行 core 测试
pnpm typecheck    # 全仓库 type check
pnpm build:all    # 构建 core + sdk + webui

鸣谢

  • LagrangeV2 —— Proto 定义参考。
  • NapCatQQ —— Scanner 与 Packet 处理实现参考。

贡献者

感谢每一位为 SnowLuma 提交过代码的伙伴。

Contributors

Star 趋势

Star History Chart

社区

S
Description
No description provided
Readme
64 MiB
Languages
TypeScript 98.9%
CSS 0.5%
JavaScript 0.5%