summaryrefslogtreecommitdiff
path: root/tokenizer/bpe/split
Commit message (Collapse)AuthorAgeFilesLines
* Track state in local variableJonas Knobloch2026-04-221-77/+37
|
* Fix FSA staticsJonas Knobloch2026-04-221-1/+1
| | | | * Add missing 've match
* Preserve per-byte error replacementsJonas Knobloch2026-03-244-0/+38
|
* Remove input bufferJonas Knobloch2026-03-242-36/+66
|
* Expose finite state automatonJonas Knobloch2026-03-241-0/+230