Article
There is no such thing as a tokenizer-free lunch
•
95
Catherine Arnett
catherinearnett
AI & ML interests
multilingual NLP, tokenization
Recent Activity
updated a dataset about 1 month ago
catherinearnett/bilingual-tokenizer-training-data published a dataset about 1 month ago
catherinearnett/bilingual-tokenizer-training-data liked a dataset about 1 month ago
commoncrawl/CommonLIDOrganizations
published an article 6 months ago
published an article 6 months ago
Article
An Analysis of Multilingual Models on Hugging Face
•
4
published an article 11 months ago
Article
Best Practices for Open Multilingual LLM Evaluation
•
6
published an article over 1 year ago
Article
They Said It Couldn’t Be Done
•
90
published an article over 1 year ago
Article
Releasing the largest multilingual open pretraining dataset
•
106
published an article over 1 year ago
Article
Detoxifying the Commons
•
6
published an article over 1 year ago
Article
wHy DoNt YoU jUsT uSe ThE lLaMa ToKeNiZeR??
•
54