summaryrefslogtreecommitdiff
Commit message (Expand)AuthorAgeFilesLines
* Add flags to import contextJonas Knobloch2026-07-134-19/+77
* Clamp perplexity evaluationJonas Knobloch2026-06-271-2/+5
* Pad token bufferJonas Knobloch2026-06-275-18/+136
* Add token buffer configJonas Knobloch2026-06-276-17/+32
* Add stream iterator to token bufferJonas Knobloch2026-06-273-45/+79
* Bump github.com/jonasknobloch/mbpe to v0.1.2Jonas Knobloch2026-06-278-27/+45
* Save segments after actually segmentingJonas Knobloch2026-06-271-9/+15
* Allow garbage collection of finished trainersJonas Knobloch2026-06-271-29/+19
* Add config to lesci experimentJonas Knobloch2026-06-076-7/+57
* Query number of samples used for analysisJonas Knobloch2026-06-051-0/+35
* Clamp rules before filteringJonas Knobloch2026-06-051-2/+2
* Perform regression with all pointsJonas Knobloch2026-06-051-6/+4
* Add flags to lesci commandJonas Knobloch2026-06-031-25/+36
* Print counterfactual merge examplesJonas Knobloch2026-06-032-0/+67
* Annotate plot with cutoff deltaJonas Knobloch2026-06-031-1/+38
* Cache Morfessor segmentationsJonas Knobloch2026-05-192-28/+47
* Refactor profiling helpers into moduleJonas Knobloch2026-05-135-42/+79
* Add layer truncation scriptJonas Knobloch2026-05-111-0/+68
* Add parameter to set hidden dimensionJonas Knobloch2026-05-044-12/+16
* Add extraction from external dataJonas Knobloch2026-05-041-1/+67
* Handle external dataJonas Knobloch2026-05-041-4/+8
* Add hidden dimension to configJonas Knobloch2026-05-041-0/+3
* Rename default configJonas Knobloch2026-05-045-6/+6
* Add config for GPT-2 LargeJonas Knobloch2026-05-041-0/+10
* Update shelf sumJonas Knobloch2026-04-301-0/+282
* Add knobloch moduleJonas Knobloch2026-04-305-0/+236
* Index delimited texts instead of shardsJonas Knobloch2026-04-301-6/+10
* Add MiniPile preparationJonas Knobloch2026-04-302-0/+48
* Manually rename first shardJonas Knobloch2026-04-302-8/+12
* Return number of written shardsJonas Knobloch2026-04-302-12/+14
* Update parameter namesJonas Knobloch2026-04-301-7/+7
* Add utility to count linesJonas Knobloch2026-04-274-2/+267
* Add option to strip delimiterJonas Knobloch2026-04-271-5/+15
* Don't return reader when setting delimiterJonas Knobloch2026-04-271-1/+2
* Limit splitting to single delimiterJonas Knobloch2026-04-272-26/+11
* Fix default delimitersJonas Knobloch2026-04-271-1/+1
* Recover tokenization based on configJonas Knobloch2026-04-2411-21/+64
* Cleanup eval commandJonas Knobloch2026-04-243-135/+0
* Add logits scriptJonas Knobloch2026-04-241-0/+35
* Access artifacts via shelfJonas Knobloch2026-04-2415-30/+1105
* Add shelf moduleJonas Knobloch2026-04-2310-11/+327
* Use platform specific defaults for CUDA device IDJonas Knobloch2026-04-237-8/+30
* Fix output orderJonas Knobloch2026-04-232-29/+95
* Add llmc moduleJonas Knobloch2026-04-2311-0/+706
* Support arbitrary delimitersJonas Knobloch2026-04-221-11/+31
* Implement decodingJonas Knobloch2026-04-221-1/+9
* Skip AppleDouble metadata filesJonas Knobloch2026-04-221-2/+8
* Fix iterator terminationJonas Knobloch2026-04-221-4/+4
* Propagate record reader errorJonas Knobloch2026-04-221-0/+6
* Track state in local variableJonas Knobloch2026-04-221-77/+37
generated by cgit v1.2.3 (git 2.25.1) at 2026-10-06 23:54:31 +0000