Reject malformed nested payloads, cap compressed card expansion, and enforce per-message decode budgets. Preserve unknown-wire metadata in diagnostics so unsupported QQ elements remain observable without exhausting memory.
Reject malformed varints, out-of-bounds fields, and invalid groups before they escape their parent frame. Aggregate unknown-field metadata and decode signed int32 values while preserving SnowLuma's existing outbound encoding bytes.
send_private_msg / send_msg can now reply to a group temp-session message
by passing the source group_id. Inbound temp messages surface their
source group on sender.group_id; a reply is only made into a temp session
the bot has an inbound record of. Group temp replies route through
RoutingHead.grpTmp (field 3).
9b470d0 added ctx.identity.findGroup() to the group-message decoder for
the group_name cache fallback, but several suites mock a minimal identity
without findGroup, so decodeGroupMessage threw at runtime once the
typecheck gate (fixed in 5e73d69) let CI reach the test phase — forward /
packet-pipeline-stranger-resolve / msg-push-handler. Optional-chain the
call: the real IdentityService always has findGroup; only test mocks omit
it, and an absent cache simply yields no fallback name.
core's typecheck covers its own tests (protocol/onebot typecheck src
only), so the newly-required GroupMessage.groupName field failed the
Dev Build typecheck on test fixtures that construct group_message
events. Add groupName to every such fixture across core/onebot/protocol
tests.
Group message events (message + message_sent) now carry group_name — a
non-standard but common OneBot extension. The name comes from the
message's own wire field (ResponseGrp field 7, accurate even right after
a rename), falling back to the identity group cache, else '' when neither
is available. It's resolved in the group-message decoder, where both the
wire head and the identity cache are already at hand, so the converter
just passes it through (no ConverterContext change).
Verified live (tempserver): real inbound group messages from a non-self
account carry group_name set to the actual group name.
The #211 concurrency attempt (c17fff6) was disproven on a live endpoint.
QQ's highway server requires blocks to arrive in offset order and rejects
an out-of-order PUT with error_code=102902 — ~20% failure under a real
account even after deferring the finalize block. Parallel connections
can't guarantee arrival order, so that approach is unshippable.
Replace it with a single persistent connection that PUTs blocks in strict
offset order, reusing the socket across the whole file via HTTP keep-alive:
- In order: the server never sees an out-of-order block, so no 102902.
- One warm connection: the TCP congestion window stays open across the
file instead of paying slow-start on a fresh connection per 1 MiB block.
That per-block cold start was the real throughput cap behind #211.
Adaptive to QQ's edge nodes FIN'ing between chunks (the reason #118 went
one-connection-per-chunk): on a transport failure, or a peer half-close
after a response, drop the socket, reconnect, and re-send the SAME block
(re-PUTting an offset is idempotent). Worst case degrades to one
connection per block — no worse than #118, and still strictly in order.
Also fixes a latent bug on the reconnect path, surfaced by review: a
response truncated mid-stream (header arrived, Content-Length unmet, then
FIN) was resolved as a partial frame and then fatally rejected by
unpackHighwayFrame OUTSIDE the retry loop, turning a recoverable mid-body
FIN into a failed upload. A truncated response now rejects as a retryable
transport failure so the block is re-sent. Added an idle read timeout so
a silent peer can't hang the upload forever (the persistent socket widens
that exposure).
Verified live (tempserver, repeated 48 MiB group-file uploads): 1
connection per file, 0 failures across dense repeats, ~1.4x throughput
over the old per-chunk-connection serial path in a low-RTT datacenter —
the win is larger on high-RTT home links where per-block slow-start hurt
most (the original #211 report). Unit tests cover keep-alive reuse,
FIN-after-each-response degradation, mid-stream reconnect+resume, body-
truncation retry, in-order exactly-once coverage, and close-once.
Group/private file uploads (and all highway media) topped out around
500 KB/s regardless of link speed. Two costs stacked up in
uploadHighwayHttp: chunks were sent strictly one at a time (throughput
capped at one 1 MiB block per round-trip), and each block opened a fresh
connection that paid TCP slow-start from a cold congestion window — so a
1 MiB block finished before the window ramped and never reached line
rate.
Drive the (already offset-addressed, self-contained) blocks through a
small worker pool instead of a sequential loop. Highway frames carry
their own dataOffset/chunkMd5/fileMd5, so blocks are independent and the
server reassembles by offset; a handful of connections overlap the
round-trip stalls and ramp their windows together. Concurrency is kept
modest (4) to stay friendly to the edge nodes' rate limiting, and the
one-connection-per-block invariant from #118 is preserved — parallelism
just puts several of those short-lived connections in flight at once.
Correctness preserved and covered by new tests:
- offset claim is synchronous (no await between read and advance of
nextOffset), so every byte range is uploaded exactly once — no skip,
no duplicate, ragged final block correct;
- concurrent ChunkSource reads are race-free (positional FileHandle
reads into per-call buffers; read-only Buffer subarrays);
- first permanent block failure aborts the whole upload and rejects;
peers stop claiming new blocks and drain the one in flight;
- workers never reject, so Promise.all settles all in-flight
reads/sockets before the source is closed exactly once — no
close-during-read, no masked upload error;
- 0-byte source still opens zero connections; peak memory bounded to
concurrency x ~1 MiB, never the whole file.
Validated the 0x210/39 ProfileLikeTip field layout byte-exact against three
real captured like pushes (detail.txt='赞了我的资料卡N次', f3=uin, f5=nick).
Drop the pending-validation caveat; test now uses the real txt shape.
Decode the multiplexed 0x210 subType-39 push: when body.msgContent is a
ProfileLikeTip with inner msgType==0/subType==203, emit notify/profile_like
{operator_id, operator_nick, times}. times is parsed from detail.txt.
Proto shape from NapCat's raw decoder; wiring + decode verified, but the
byte-exact wire is pending a real like capture to confirm.
Decode the 0x2DC subType-16 field13==6 push: eventParam carries the member
uin (f5) and a gray-tip template (f2) whose last rich token holds the title
text. Emit notify/title {group_id, user_id, title}, matching NapCat. Field
layout taken from a real on-wire sample, not ported blindly.
Detect a group member's card change from message traffic (sender card vs
cached roster card) and emit group_card {group_id, user_id, card_new,
card_old}, matching NapCat's parseCardChangedEvent. Fires only on a real
change of a known member; first-contact and unchanged cards are suppressed.
Decode the 0x210/0x115 input-state sys_msg (RE'd from wrapper.linux.node
aio_input_state_worker) into a friend_input_status event and map it to a
notify/input_status notice, matching NapCat's onInputStatusPush.
event_type 1 = typing, 3 = recording voice; status_text synthesised to
match the client's own strings.
The width/height=0 → 文件已过期 fix only covered the normal upload path, which
ffprobes real dimensions from the video bytes. The fast-upload (fingerprint /
forward) path has NO bytes to probe, so every field it ships is caller-supplied
and untrusted — a video that arrived on the wire without metadata caches 0s, and
re-forwarding it then shipped width/height=0 (receiver renders 文件已过期) and
time=0 (every client renders 00:00).
Harden videoPayloadFromFingerprint so no degenerate value reaches the wire:
- width/height fall back with `||` (not `??`, so a cached 0 is replaced too);
- duration falls back the same way (0 → 1, never 00:00);
- the fallback cover is now a real 720x1280 image, and the fallback video/cover
dimensions are derived FROM it (detectImageFormat), so the declared video size,
the declared cover size and the actual cover pixels are the same numbers by
construction — never a 1x1 image lying about its size.
Real cached values always win; c2c is unaffected (the wire builder forces 0 for
c2c). Verified on-target by forwarding a zeroed video (renders, not expired).
The width/height=0 → 文件已过期 fix only covered the normal upload path, which
ffprobes real dimensions. The fast-upload (fingerprint / forward) path has no
bytes to probe and takes dimensions from cached metadata, which is 0 when the
source video arrived on the wire without dimensions (e.g. forwarded from another
client that itself sent 0x0). Re-forwarding such a video to a group then shipped
0x0 and the receiver rendered a stale "expired" placeholder.
videoPayloadFromFingerprint now substitutes a neutral 720x1280 portrait when a
dimension is absent or zero (|| not ??, so a cached 0 is replaced too). Real
cached dimensions always win; c2c is unaffected (the wire builder forces 0 there
regardless). A forwarded dimensionless video now renders and plays instead of
showing 文件已过期; verified on-target by forwarding a zeroed-dim video.
A member's cached group card could stay stale for the lifetime of a long-running
bot: nothing refreshes it on a card change (no push), and the member-list refetch
only fires on join/leave/admin events (+60s TTL), never periodically — so a quiet
group freezes at warmup. get_group_msg_history / live events then showed an empty
or outdated sender.card even though the current card was in the message.
Every group message carries the sender's current display name in grp.memberCard
(field 4 = group card if set, else base nickname). The group decoder now prefers
it over the possibly-stale cache when it differs from the base nickname (a real
group card); and packet-pipeline self-heals it back into the in-memory member
cache, gated so it writes only on an actual change. This keeps active members'
cards fresh for all cache consumers off message traffic alone — no extra fetch,
no rate-limit exposure. Verified on-target: a member's card change now reflects
in the next message without a member-list refetch.
Known, accepted edge: a member changing only their base nickname (no card) is
mislabeled as a card until the next member-list refetch corrects it — field 4
alone can't distinguish the two, and the mislabeled value is still the member's
own current name.
get_forward_msg returned empty sender.nickname for group merged-forward nodes.
The reporter blamed a mis-parsed group_id + a failing member-list enrichment
(0xfe7_3 0x899_AUTHORITY_FAIL), but on-target capture shows a mislabeled proto
field: the group response-head's sender display name lives in field 4, which we
had labeled `groupName`. It's populated on live pushes ("星屿" / the group card
"墨梓柒111") AND on merged-forward nodes (per-node sender name), while field 2
(`memberName`) is empty on live/forward and only set by the group-history fetch,
and the real group name is field 7. Lagrange labels field 4 `MemberName`.
Relabel ResponseGrp: field 4 `memberCard` (sender display name = card||nick),
field 7 `groupName` (the real group name); keep field 2 `memberName` for the
group-history slot. The group decoder now resolves the sender name by priority:
field-2 name → local member cache (clean nick/card split) → field-4 memberCard.
The last step is what lets a forward node from a group the bot isn't in keep its
per-node sender name with no member-list lookup, so the forward-specific fallback
and the AUTHORITY_FAIL enrichment path are no longer needed for the common case.
Verified on-target across scenarios: live message senderNick unchanged; a plain
forward, a forward from a never-joined group (placeholder uin 1094950020, four
distinct senders 金橘雪梨/Crel/503/Xiing0), and a nested forward all keep their
nicknames; no member-list call fires. group_id stays the payload's value.
Wire the streaming primitives into the video send path. `loadVideo` now stages
the source onto a local disk path (stageSourceToDisk — hardlink/download),
hashes it in one streaming pass (hashFileStreaming), and returns a `fileSource`
instead of a full in-memory buffer; the main video sub-file PUTs through a
`FileChunkSource`, so a multi-GiB video never holds more than one 1 MiB chunk in
RAM. The thumbnail and the fingerprint (noByteFallback) fast path stay
byte-based and unchanged.
`runPuts` now sizes its fast-path / fast-only checks off `fileSource ?? bytes`
and resolves the highway session before opening the file handle (the handle's
owner, uploadHighwayHttp, must be the next call so its finally-close always
runs). A size+mtime guard re-checks the staged file before the PUT so a source
mutated mid-send fails cleanly rather than shipping bytes that disagree with the
already-sent hashes. The staged temp is released in every exit path.
Output is byte-identical to the buffered path (the OIDB fileInfo md5/sha1/size
and the uploaded bytes are unchanged); the old readFileSync-based
stageVideoSource is removed. Tests cover the disk path (fileSource not bytes,
streamed hash == buffered hash), cleanup on upload failure, and empty sources.
Stage any upload source onto a local disk path so the uploader can stream it
without buffering the whole file. `stageSourceToDisk(source, maxBytes)` unifies:
local file:// / raw path → hardlink into an owned stage dir (O(1); the inode
link pins the bytes against a caller-side unlink/retention), streamed-copy
fallback on any link() failure; http(s):// → stream to a .part file; base64://
→ maxBytes-checked decode to a .part file. Temps are finalized atomically
(fsync file → rename → fsync dir); a TTL reaper reaps crash-orphaned temps while
an in-use `active` set protects live uploads (a hardlink keeps the source's old
mtime, so the staged path is reserved in `active` before it appears on disk).
To avoid divergence, the existing `loadBinarySource` HTTP transport is extracted
into a shared `downloadHttp(source, name, maxBytes, makeSink)` engine (browser
UA, redirect, timeout, Referer retry, declared + streamed size caps, noRetry);
`loadBinarySource` now drives it with an in-memory sink — behavior unchanged.
Additive only — not yet wired into video/file uploads. Tests cover the
retention race (hardlink survives the original's deletion), the reaper's
active-set protection, size caps, symlink resolution, the copy fallback, the
Referer retry reusing the .part, and streamed-overflow .part cleanup.
Single-pass, disk-streaming replacement for the buffered
`computeHashes` + `computeVideoSha1Blocks` (+ offline-file head md5): reads the
file 1 MiB at a time so hashing a multi-GiB upload never buffers the whole file.
Reproduces the buffered helpers exactly — whole-file md5/sha1, the per-1 MiB
intermediate SHA1 state snapshots plus the finalized whole-file SHA1, and an
optional md5 over the first N bytes (`headLimit`, for `md510MCheckSum`).
Additive only — not wired into any caller yet; the old helpers stay in place.
Lands in a new leaf module to avoid a utils↔highway-client import cycle. A
byte-oracle test asserts identity vs the buffered functions across sizes around
the 1 MiB block boundary (empty, sub-block, exact multiples, ±1, multi-block +
tail) and head limits across chunk boundaries.
Prep for streaming large media uploads from disk instead of buffering the
whole file in RAM. Add a `ChunkSource` abstraction — `BufferChunkSource`
(in-memory, byte-for-byte the old path) and `FileChunkSource` (disk-backed,
reads exact-length chunks via a FileHandle, loops on short reads, throws on
unexpected EOF) — and change `uploadHighwayHttp` to consume a `ChunkSource`
instead of a `Uint8Array`. The uploader now owns the source and closes it once
in `finally` (best-effort; never masks a primary upload error).
Pure structural refactor: no streaming is wired yet. All five call sites
(pipeline runPuts, group + private file upload, personal + group avatar) wrap
their bytes in `BufferChunkSource`, so behavior is unchanged. Adds a
byte-oracle test proving FileChunkSource ≡ BufferChunkSource across block
boundaries, plus short-read/EOF/empty-input coverage.
The 100 MB cap that re-routed larger videos to the file-upload pipeline
was a workaround for the width/height=0 → 已过期 bug (fixed in the prior
commit), not a real QQ size limit. Real-machine verified on tempserver:
88 MB / 149 MB / 299 MB / 497 MB videos all upload, render and play as
normal videos in a group. Raise MAX_VIDEO_SIZE to 1.5 GiB (matching the
file-upload ceiling); only videos above that still fall back to the file
path. Collapse the redundant MAX_VIDEO_SIZE_HARD (was also 1.5 GiB) into
the single constant and drop the now-dead 100 MB warn branches.
Note: the whole video is buffered in RAM for the Highway upload, so a
1.5 GiB send costs ~1.5 GiB+ of process memory — that, not a QQ limit,
is what bounds the ceiling.
A group video uploaded with fileInfo width/height = 0x0 reaches receivers
as 文件已过期 even though the resource is fresh and downloadable: QQ-NT
clients (Android especially) can't lay out the video tile without real
dimensions, so they render the expired placeholder. iOS is lenient — it
shows the expired badge but still opens the video, which is exactly the
"iOS 显示过期可以点开、安卓过期点不开" split users reported.
Real-machine capture on tempserver confirmed a working QQ group video's
MsgInfo carries real dimensions (e.g. 296x640) while ours sent 0x0. Fill
width/height from the probed video for group sends; c2c stays at 0 because
the QQ-NT server rejects non-zero dimensions there with a schema mismatch
(no c2c regression coverage yet). Verified end-to-end: a fresh 11 MB group
video that rendered 已过期 now displays and plays.
Older QQ clients (≤9.9.30) deliver a 闪传 (flash transfer) file as a richui
markdown commonElem (svc=45, busId=FlashTransfer); SnowLuma had no decoder for
it, so the message decoded to 0 elements and was reported empty. (Newer
clients ≥9.9.31 send a plain text+link message that already decodes fine.)
Decode svc=45 markdown → parse the FlashTransfer card JSON → emit a flash_file
element, surfaced as the segment the issue asked for:
{ type: "flash_file", data: { title, file_set_id, scene_type } }
- Field names (fileSetId / sceneType / title) confirmed against QQ NT's
flash-transfer manager in wrapper.node.i64. The card is built by the
sender's client so the exact JSON nesting is unknown — values are pulled by
a bounded recursive search rather than a pinned path (handles either shape).
- Non-FlashTransfer markdown is ignored; missing fields default (never empty).
- Manifest: D/S = yes, P/W = by-design-no (receive-only; sending is the
existing send_flash_msg action). Log preview shows [闪传文件:title].
- Tests: card decode (nested + alias + defaults + non-flash) and the
element→segment mapping.
Note: not end-to-end verified on a live message — the svc=45 format only comes
from ≤9.9.30 senders and no such client was available (every test sender here
is ≥9.9.31 → text format). Logic + field names are unit-tested / IDA-confirmed.
typecheck 12 包; protocol 596 / onebot 529 green.
* fix(protocol): retry qzone msglist via user.qzone route on -10000
emotion_cgi_msglist_v6 through the legacy h5.qzone.qq.com →
taotao.qzone.qq.com gateway can be rejected with `-10000 使用人数过多`
for some accounts (issue #182). Real-device captures show the same CGI
via user.qzone.qq.com → taotao.qq.com is not rate-limited, so retry once
through that route when the legacy route returns -10000.
Behaviour-preserving for accounts the legacy route already serves: the
retry only fires on a -10000 envelope; all other codes, transport
errors, and the missing-msglist case are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(port-probe): fix port probe logic (parse accounts more than 2) and add description for 4302-4310 port.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round out the OneBot v11 group-profile fields the issue asked for.
Group (get_group_info / get_group_list, inherited by get_group_info_ex /
get_group_detail_info):
- group_create_time + group_memo come free from the 0xFE5_2 list
(createdTime / announcement), verified live (createdTime=1729505343,
announcement populated).
- group_level lives only in the 0x88D_0 detail; get_group_info fetches it
(memoized 5min) so a joined group doesn't hit the detail every call.
get_group_list leaves it 0 — a per-group detail fetch across the whole
list would be far too costly.
Member (get_group_member_info / get_group_member_list):
- Add area / unfriendly / title_expire_time / card_changeable. QQ NT does
not expose these, so they're fixed placeholders ('' / false / 0 / true),
matching go-cqhttp / NapCat. sex/age stay unknown/0 (real values need a
per-member profile fetch — out of scope here).
typecheck 12 packages; onebot 523 / core 472 green.
* feat(qzone): add set_qzone_msg_right to change a feed's visibility
Add updateQzoneMsgRight in the protocol layer, porting php-qzone's
updateRight: fetch the feed via emotion_cgi_msgdetail_v6, rebuild the
full publish-shaped payload (richval/pic_bo/con), then override
ugc_right/allow_uins and POST to emotion_cgi_update. Expose it through
QzoneApi.updateRight and the OneBot set_qzone_msg_right action, with the
same ugc_right/target_uins validation as send_qzone_msg. Covered by
protocol, core and onebot tests.
* fix(qzone): make qzoneParse more stable and fixed a test example
Add ugc_right and target_uins to the qzone publish path across protocol,
bridge, and the send_qzone_msg OneBot action, regenerate the MCP catalog,
and cover restricted visibility (16/128) with validation tests.