Commit Graph
3 Commits
Author SHA1 Message Date
DrSmoothl 7c0519f75f lint 2026-10-12 15:43:34 +08:00
DrSmoothl b8b9be83c3 feat(core): stream flash-transfer uploads from disk
create_flash_task loaded every fileset file through loadBinarySource
and kept the bytes through hash, commit, and sliceupload, so a single
file over 1 GiB could not be sent. Stage to disk like group files,
stream MD5/SHA1/Sha1StateV, then POST 1 MiB chunks from FileChunkSource.

Do not reuse video sha1Blocks as Sha1StateV: an exact N-MiB file would
ship an extra un-finalized last-slice state. Local/HTTP sources use the
4 GiB group-file ceiling; inline base64 stays at 1 GiB because stage
still decodes it in RAM. Protocol order, seconds-pass, and multi-file
commit are unchanged.

Fixes #359
2026-08-13 18:34:27 +08:00
DrSmoothl 7b91903f80 feat(highway): add streaming file hasher hashFileStreaming (phase 2)
Single-pass, disk-streaming replacement for the buffered
`computeHashes` + `computeVideoSha1Blocks` (+ offline-file head md5): reads the
file 1 MiB at a time so hashing a multi-GiB upload never buffers the whole file.
Reproduces the buffered helpers exactly — whole-file md5/sha1, the per-1 MiB
intermediate SHA1 state snapshots plus the finalized whole-file SHA1, and an
optional md5 over the first N bytes (`headLimit`, for `md510MCheckSum`).

Additive only — not wired into any caller yet; the old helpers stay in place.
Lands in a new leaf module to avoid a utils↔highway-client import cycle. A
byte-oracle test asserts identity vs the buffered functions across sizes around
the 1 MiB block boundary (empty, sub-block, exact multiples, ±1, multi-block +
tail) and head limits across chunk boundaries.
2026-07-05 02:37:56 +08:00