summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* WIPwip-frequencyJonas Knobloch2026-09-1114-0/+3267
|
* Merge remote-tracking branch 'origin/wip-entropy' into wip-frequencyJonas Knobloch2026-09-1113-0/+844
|\
| * WIPJonas Knobloch2026-09-108-724/+189
| |
| * WIPJonas Knobloch2026-09-092-0/+143
| |
| * WIPJonas Knobloch2026-09-0911-0/+1073
| |
| * Add dict moduleJonas Knobloch2026-09-094-0/+167
| |
* | WIPJonas Knobloch2026-09-112-0/+28
| |
* | WIPJonas Knobloch2026-09-115-21/+1017
|/
* Add flag to configure estimation windowmainJonas Knobloch2026-07-291-1/+5
|
* Try likely initializersJonas Knobloch2026-07-141-2/+32
|
* Add method to check for initializerJonas Knobloch2026-07-141-0/+12
|
* Refactor ONNX extractionJonas Knobloch2026-07-143-9/+37
|
* Add flags to control evaluatorJonas Knobloch2026-07-141-23/+120
|
* Infer vocab size and cutoffJonas Knobloch2026-07-131-4/+16
|
* Refactor experiment configurationJonas Knobloch2026-07-132-20/+32
| | | | | * Expose token buffer config * Expose evaluator config
* Add flags to import contextJonas Knobloch2026-07-134-19/+77
|
* Clamp perplexity evaluationJonas Knobloch2026-06-271-2/+5
|
* Pad token bufferJonas Knobloch2026-06-275-18/+136
|
* Add token buffer configJonas Knobloch2026-06-276-17/+32
|
* Add stream iterator to token bufferJonas Knobloch2026-06-273-45/+79
|
* Bump github.com/jonasknobloch/mbpe to v0.1.2Jonas Knobloch2026-06-278-27/+45
|
* Save segments after actually segmentingJonas Knobloch2026-06-271-9/+15
|
* Allow garbage collection of finished trainersJonas Knobloch2026-06-271-29/+19
|
* Add config to lesci experimentJonas Knobloch2026-06-076-7/+57
|
* Query number of samples used for analysisJonas Knobloch2026-06-051-0/+35
|
* Clamp rules before filteringJonas Knobloch2026-06-051-2/+2
|
* Perform regression with all pointsJonas Knobloch2026-06-051-6/+4
|
* Add flags to lesci commandJonas Knobloch2026-06-031-25/+36
|
* Print counterfactual merge examplesJonas Knobloch2026-06-032-0/+67
|
* Annotate plot with cutoff deltaJonas Knobloch2026-06-031-1/+38
|
* Cache Morfessor segmentationsJonas Knobloch2026-05-192-28/+47
|
* Refactor profiling helpers into moduleJonas Knobloch2026-05-135-42/+79
|
* Add layer truncation scriptJonas Knobloch2026-05-111-0/+68
|
* Add parameter to set hidden dimensionJonas Knobloch2026-05-044-12/+16
|
* Add extraction from external dataJonas Knobloch2026-05-041-1/+67
|
* Handle external dataJonas Knobloch2026-05-041-4/+8
|
* Add hidden dimension to configJonas Knobloch2026-05-041-0/+3
|
* Rename default configJonas Knobloch2026-05-045-6/+6
|
* Add config for GPT-2 LargeJonas Knobloch2026-05-041-0/+10
|
* Update shelf sumJonas Knobloch2026-04-301-0/+282
|
* Add knobloch moduleJonas Knobloch2026-04-305-0/+236
|
* Index delimited texts instead of shardsJonas Knobloch2026-04-301-6/+10
|
* Add MiniPile preparationJonas Knobloch2026-04-302-0/+48
|
* Manually rename first shardJonas Knobloch2026-04-302-8/+12
|
* Return number of written shardsJonas Knobloch2026-04-302-12/+14
|
* Update parameter namesJonas Knobloch2026-04-301-7/+7
|
* Add utility to count linesJonas Knobloch2026-04-274-2/+267
|
* Add option to strip delimiterJonas Knobloch2026-04-271-5/+15
|
* Don't return reader when setting delimiterJonas Knobloch2026-04-271-1/+2
|
* Limit splitting to single delimiterJonas Knobloch2026-04-272-26/+11
|
generated by cgit v1.2.3 (git 2.25.1) at 2026-10-06 23:27:13 +0000