LiteRT-LM / schema /testdata /tokenizer.json
SeaWolf-AI's picture
Upload full LiteRT-LM codebase
5f923cd verified
{
"version": "1.0",
"truncation": null,
"padding": null,
"added_tokens": [
{
"id": 0,
"content": "<|endoftext|>",
"single_word": false,
"lstrip": false,
"rstrip": false,
"normalized": false,
"special": true
}
]
}