summaryrefslogtreecommitdiff
path: root/llm
Commit message (Expand)AuthorAgeFilesLines
* Clamp perplexity evaluationJonas Knobloch2026-06-271-2/+5
* Pad token bufferJonas Knobloch2026-06-275-18/+136
* Add token buffer configJonas Knobloch2026-06-275-16/+26
* Add stream iterator to token bufferJonas Knobloch2026-06-273-45/+79
* Bump github.com/jonasknobloch/mbpe to v0.1.2Jonas Knobloch2026-06-272-17/+0
* Rename default configJonas Knobloch2026-05-041-1/+1
* Recover tokenization based on configJonas Knobloch2026-04-241-1/+4
* Cleanup eval commandJonas Knobloch2026-04-243-135/+0
* Access artifacts via shelfJonas Knobloch2026-04-241-3/+4
* Use platform specific defaults for CUDA device IDJonas Knobloch2026-04-231-1/+1
* Add sander moduleJonas Knobloch2026-04-172-21/+14
* Remove execute panic recoveryJonas Knobloch2026-04-111-7/+0
* Destroy session on model teardownJonas Knobloch2026-04-102-0/+8
* Refactor model optionsJonas Knobloch2026-04-081-1/+7
* Add evaluator configJonas Knobloch2026-04-083-3/+14
* Rename default config constructorJonas Knobloch2026-04-081-1/+1
* Batch scoringJonas Knobloch2026-04-072-14/+18
* Run evaluator with scoreJonas Knobloch2026-04-076-87/+19
* Handle environment initialization on package levelJonas Knobloch2026-04-061-0/+8
* Add parameter to control logits output bindingJonas Knobloch2026-04-061-1/+1
* Add score methodJonas Knobloch2026-04-042-1/+2
* Add config structJonas Knobloch2026-04-041-1/+1
* Return document index in text iteratorJonas Knobloch2026-04-032-12/+13
* Scope text column to parquet readerJonas Knobloch2026-04-032-2/+2
* Use token buffer to generate jobsJonas Knobloch2026-04-032-111/+36
* Return relative offset in push iteratorJonas Knobloch2026-04-032-28/+32
* Refactor tail cleanupJonas Knobloch2026-04-031-13/+13
* Track already seen tokensJonas Knobloch2026-04-022-9/+75
* Add token bufferJonas Knobloch2026-04-022-0/+250
* Count skipped textsJonas Knobloch2026-03-311-2/+14
* Refactor llm moduleJonas Knobloch2026-03-252-0/+0
* Expose progress bar titleJonas Knobloch2026-03-254-6/+6
* Add helper to collect resultsJonas Knobloch2026-03-253-44/+39
* Refactor evaluator initializationJonas Knobloch2026-03-254-27/+9
* Sort importsJonas Knobloch2026-03-251-2/+1
* Tidy dependenciesJonas Knobloch2026-03-252-1/+17
* Refactor tokenizer usageJonas Knobloch2026-03-241-12/+7
* Refactor progress bar life cycleJonas Knobloch2026-03-241-6/+3
* Sync dependenciesJonas Knobloch2026-03-232-44/+26
* Add log probs extractorJonas Knobloch2026-03-174-35/+355
* Update perplexity evaluatorJonas Knobloch2026-03-179-208/+304
* Rename dataset readerJonas Knobloch2026-03-162-4/+4
* Use vanity import pathJonas Knobloch2026-03-163-5/+7
* Add ppl Python scriptJonas Knobloch2026-03-091-0/+112
* Fix perplexity calculationJonas Knobloch2026-03-094-12/+31
* Expose window and strideJonas Knobloch2026-03-092-7/+7
* Add ppl commandJonas Knobloch2026-03-091-0/+69
* Use dataset packageJonas Knobloch2026-03-091-16/+13
* Update module pathsJonas Knobloch2026-02-281-1/+1
* Bump Go to 1.25.0Jonas Knobloch2026-02-101-1/+1
generated by cgit v1.2.3 (git 2.25.1) at 2026-10-06 23:41:34 +0000