Stream content-addressed storage so an object can exceed heap

Featurekamo-shared-library
Shipped
August 8, 2026 at 2:36 AM UTC
Author
Kamo
Commit
67391b8

Chat attachments are moving to a 3 GiB ceiling. storeContentAddressed took a byte[], which cannot express that at all — a Java array stops at Integer.MAX_VALUE — and MinIOStorageService.upload then buffered the whole object a second time so it could retry the put against the next node. Adds a streaming path alongside both: HashUtils.computeDigests derives Blake3 and SHA3-256 from one pass, uploadStreaming re-opens an InputStreamSource per node attempt instead of holding the bytes, and the new storeContentAddressed overload hashes, dedups and stores without ever accumulating the content. Peak memory is the MinIO part buffer rather than the file size. It is deliberately not @Transactional: storing gigabytes takes minutes, and the row is only published (isMissing=false) once the object is durably stored, so a crash leaves the same state the buffered path leaves.

All changes

Like what you see shipping?

Every one of these updates lands in your workspace automatically. Start free and watch it grow week after week.

Start Free ForeverView Pricing