KamoCRM

Intake endpoint counters come from a ledger and stop losing updates

Fixkamo-shared-library
Shipped
September 23, 2026 at 1:51 AM UTC
Author
Kamo
Commit
78a3f60

**************** / total_processed / last_received_at were bumped on the endpoint row itself: once per inbound payload (~5/s during a farm import) and once per processing batch, all on one shared row. YugabyteDB aborts concurrent UPDATEs of a row with 40001, so the bumps had been split into their own transactions with their failures swallowed — the counters were lossy by design: "imported" was 437 short across two endpoints (measured against the payload rows 2026-09-22), and every settings save rewrote the row whole, clobbering bumps in flight. 41 rows had grown to 139 MB of versions. Now the payload rows are the record. A trigger on lead_intake_raw_payloads journals each payload and each change of its imported-ness in the payload's own transaction (securityservice create_lead_intake_ledger.sql, applied and backfilled 2026-09-22: 32 endpoints, 0 mismatches), and **************** sums the folded total with the unfolded journal in one statement. **************** fills a list of endpoint DTOs from one read; recordReceipt and recordProcessed are no-ops, kept deprecated so a service built before its callers were removed still compiles. DaemonService folds and recounts this ledger with the others. Tests: LeadIntakeLiveCountsTest (one read per list, zero for an idle endpoint, no endpoint-row writes), LeadLedgerQueryShapeTest (reads the ledger tables, never the endpoint row). The trigger, read, fold and recount SQL were replayed against YugabyteDB (temp tables, 24 checks).

All changes

Like what you see shipping?

All of it arrives in your workspace on its own. Start on the free plan and read this page again in a month.

Start Free ForeverView Pricing