Beam 4 only for labels, beam 2 up to 64 tokens

Shipped

October 22, 2026 at 4:39 PM UTC

Author

Kamo

Commit

b116595

Cost is tokens x beam. Measured in the cluster with the old engines still running: one stream 77 chars/s and two 115 chars/s on short strings at beam 4. Sentence-length text does not need the width that fixed labels (a stray '&' in Tamil), so the wide beam now stops at 12 source tokens and beam 2 at 64. Both thresholds are env settings.