Probe SIP for real, and assert the white-label holds on the wire
A TCP connect to 5060 proves almost nothing — it succeeds against anything listening, and on UDP it cannot fail at all. That ambiguity is exactly what led me to...
Distinct Erlang node names for apps and ecallmgr
One image runs as two nodes. They need different names or they refuse to coexist, and both need the shared cookie that FreeSWITCH's mod_kazoo also uses.
Run the OTP release, not make release
The image now ships a relx release rather than the source tree, so passing 'make release' to the entrypoint failed with 'No rule to make target release'. foreg...
Bootstrap script and operational README
bootstrap.sh creates the master account, promotes it to reseller and writes the non-secret platform configuration. Idempotent: a second run detects the existing...
Swap the inbound greeting to KamoCRM-Main-Intro-3
Third take, 40s (down from 60s). Uploaded alongside the previous two rather than over them, so rolling back to either is a one-line change to KAMO_ATTENDANT_INT...
Scale edge to 0 until its config is rewritten
Kamailio cannot start on 2600Hz's vendored configs — they need 2600Hz's patched build (spec 16). Leaving it at replicas:1 restart-storms k3m1 indefinitely for n...
The NAT is in place; a closed port means no listener
The router forwards for 5060/5061 TCP+UDP and 20001-29999/UDP to 192.168.4.36 are confirmed. My gate reported PHASE 0 BLOCKED against them while Kamailio was cr...
Apps, media, edge manifests and network policy
- config.ini carries a [zone] block from day one so a second region is a config edit, not surgery. Secrets are substituted by an entrypoint at container start...
K3m1 is 47.181.8.87 (pbx.k3.kluster.kamocrm.com)
Reverts an incorrect change of mine. I had propagated 42.181.8.87 and edited coturn/deployment.yaml's comment, which was right all along. Verified by DNS: pbx...
CouchDB and RabbitMQ on k3m1
CouchDB q=8 / n=1, verified live. q is fixed at database creation and KamoPBX creates a database per account plus monthly MODBs, so q is sized now for the event...
Phase 0 infrastructure gate
- TURN over TLS on 5349 with a real cert (turn-kamocrm-com-cert). Without turns:, a customer behind a 443-only egress firewall has no media path. Verified e...
Drop dead rate.loans and priceturbo.com from static provisioning
Both have had ACME challenges stuck pending for 47-54 days. The ratestack estate they proxy to has been scaled to 0 since 2026-08-04, so nothing can answer HTTP...
Bound the Go heap with GOMEMLIMIT and declare a real memory request
Root cause of the crash-loop was not a leak. The 485Mi steady state that kept tripping the old 512Mi limit was two normal things stacked: ~350Mi Go heap at ...
Swap the inbound greeting to KamoCRM-Main-Intro-2
Uploaded as a new file rather than overwriting the old one, so rolling back is a one-line change to KAMO_ATTENDANT_INTRO plus a dialplan reload — no re-upload n...
Raise memory limit to 1Gi and relax liveness probe timeout
Traefik sat at ~485Mi against a 512Mi limit (95%). The cgroup recorded 47836 memory.max hits with oom_kill 0 — never OOM-killed, because enough of the footprint...
Queue timeout and "press 0" both fall through to voicemail
Replaces the hand-rolled Dial()+Goto queue loop with app_queue. The deciding factor is DTMF: digits are NOT matched against the dialplan while Dial() is running...
Hold-music queue ringing 8000 and 8080 simultaneously
Eight hold tracks uploaded as the `kamocrm-hold` MOH class, shuffled. The random ordering comes from `random=1` on the row in FreePBX's `music` table, which is ...
Recorded greeting, dial-by-extension, 1/2 to operator
Replaces the music-on-hold hold pattern with the recorded intro: - **************** plays the greeting AND listens at the same time, so a caller can dial ove...
Inbound auto-attendant — music on hold + dial-by-extension
Inbound calls had nowhere to go: the `incoming` table was completely empty, so even a delivered call would have dead-ended. Adds the attendant context plus the ...
Allowlist Telnyx in both fail2ban jails
Adds the Telnyx US signaling addresses (192.76.120.10, 64.16.250.10 — sip.telnyx.com) to ignoreip now that the carrier trunk is provisioned. Also overrides ign...
Make fail2ban actually run — enable Asterisk's security log
fail2ban has never started on this PBX. Its asterisk-security jail watches /var/log/asterisk/security, a file Asterisk never created: FreePBX generates logger_l...
Bounce a bare visit to the Kamo login, and end the session on tab close
Closing the KamoDesktop tab left a live session sitting on the machine. Guacamole keeps its auth token in localStorage (GUAC_AUTH), which survives closing the t...
The auto-reconnect reload-looped every page into a blank screen
I shipped this and it took the desktop down: a blank white page and a tab loading forever. Disabled in-cluster immediately; this is the real fix. `ng-switch` s...
The auto-reconnect was watching for a class that never appears
It keyed on **************** That class exists in Guacamole's stylesheet and in none of its templates, so the selector matched nothing: the script loaded, ran, ...
Cap Guacamole's heap, and unbreak the deploy that was blocking it
Two faults, one of them mine. MINE FIRST: the `node --test` gate added with the auto-reconnect script pointed at a DIRECTORY, and the runner's node resolves a ...
Stop the canary crying wolf after every legitimate restart
session_durable treated a REJECTED token as a durability failure. That is not the outage it guards: a gateway restart, or Guacamole's own inactivity timeout, in...
Stop every deploy from throwing everyone off their desktop
"An error has occurred and this action cannot be completed" after leaving a desktop open was not an idle timeout and not a browser problem. It was CI. deploy-s...
An access canary that actually logs in, and alerts that reach a person
Two gaps, both proven against the live cluster today. FIRST: every existing KamoDesktop alert measures whether a COMPONENT is alive. All four were green during...
Run a dispatched session in the workspace's common parent
sage's VS Code is a multi-root workspace holding all 63 repositories, so taking the first workspace folder rooted a bug about the login site in aiservice — the ...
Say what a dead KamoDesktop connection actually means
Guacamole's stock CLIENT.ERROR_TUNNEL_204 is "The requested connection does not exist. Please check the connection name and try again." There is no connection n...
Finish removing Roundcube webmail
The Roundcube deployment/service and the mail.kamocrm.com IngressRoute were already dropped from the CI deploy list, but the MySQL bootstrap still created and g...
Submit the bug report without simulating a keystroke
vscode:// URI handler and sent a synthetic Return. Measured on desktop-1, that cannot work: KDE's screen locker holds an exclusive X input grab whenever the scr...
Dispatch a bug report into sage's open VS Code
URI handler that creates a conversation in a RUNNING window and pre-fills the input, which is the only way to reach the editor sage already has open instead of ...
auto-cert: let the junk-cert cleanup actually see the junk it was written for
cleanup_orphan_certificates() listed with '-l auto-cert.kamo/fqdn'. That label is newer than the scanner certs the function exists to remove, so the selector ex...
auto-cert: issue an org's certificates as a batch, not one every 90 seconds
A new organization needs a certificate for all eleven of its hosts, and each one is a separate cert-manager Certificate. The loop waited for each host to become...
Prefer brotli on the theme host, and correct the previous rationale
Two things, one of them a correction to the commit before this. The previous commit justified `encodings: [br, gzip]` with a probe against api.kamocrm.com. Tha...
Prefer brotli, stop compressing what is already compressed
The shared `compress` middleware was `compress: {}`, which is three separate defaults and none of them is the one we want. Traefik picks gzip whenever the clie...
Cache CORS preflights for two hours, not 100 seconds
`accessControlMaxAge: 100` meant browsers re-ran the preflight roughly once per visit, so every page load on the marketing site paid two extra OPTIONS round-tri...
Stop the edge rate limit from 429ing every cold KamoDesktop load
Opening an assigned Linux instance always landed on Guacamole's generic "An error has occurred and this action cannot be completed" modal, and always worked aft...
Provision MARKETING_EVENTS for visitor conversion signals
Marketing tracking codes credit a partner for the signups, organizations and paid accounts their link produced, and those three moments happen in services that ...
Make the HLS playlists immutable, and say what that costs
The playlists were the last entry in Lighthouse's cache-lifetime audit — 300s originally, then an hour, still reported. Everything else in the ladder was alread...
Compress the theme host, which was serving every text asset raw
MinIO returns object bytes exactly as stored — it does not negotiate Content-Encoding — and the two theme IngressRoutes carried `strip-hsts` and `cors` but no c...
Swap the hero reel to kamo13
kamo13.mp4 uploaded to MinIO at public/kamo13.mp4 (30 MB, sha256 174aaec6…, verified byte-identical to the local source after transfer — this link has a history...
Pin and verify the kubectl download instead of falling back to a broken wget
The deploy died at "Install kubectl". Three separate faults lined up: 1. curl aborted mid-stream with (56) "decryption failed or bad record mac" — the corru...
Correct the README to the routes that actually shipped
The WebRTC endpoint moved from /webrtc/<ch>/whep to /live/<ch>/whep when channels were renamed live/<ch> to drop StripPrefix, and the README still described the...
Name channels live/<name>, use Recreate, feed the loop a uniform GOP
Three defects found by exercising the deployed service. 1. The /live/ route 404'd through Traefik. MediaMTX answers the first HLS request with a 302 to `?co...
Move the encode job's decision logic out of the mc container
The plan stage died on `sed: command not found`. The minio/mc image is minimal to the point of having no sed, awk, grep, wget or curl — it can move objects and ...
Upload segments with an explicit glob, not --recursive
mc cp --recursive takes a directory; the segments were being passed as a glob, which is the wrong shape for that flag. Segments and the playlist beside them nee...
Deploy a streaming server and an ABR ladder for the hero reel
MediaMTX serves live channels over HLS, WebRTC, RTSP and RTMP, and loops the marketing reel as an always-available channel via alwaysAvailableFile — no ffmpeg s...
Like what you see shipping?
Every one of these updates lands in your workspace automatically. Start free and watch it grow week after week.