Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Log In
Sign Up
Remeinium
/
WWHO
like
0
Follow
Remeinium AI
1
Feature Extraction
Transformers
Remeinium/WWHO_30m
Sinhala
Hindi
English
tokenizer
WWHO
SGPE
linguis_trie
token
tokenization
Syllable
remeinium
transformer
linguistics
NLP
sinhala
hindi
english
BPE
GPE
Eval Results (legacy)
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
main
WWHO
18.5 MB
Ctrl+K
Ctrl+K
1 contributor
History:
7 commits
thekusaldarshana
Update README.md
b3a398c
verified
23 days ago
.gitattributes
Safe
1.52 kB
initial commit
about 1 month ago
EVALUATION.md
Safe
18.9 kB
Seperate Before you Compress
23 days ago
LICENSE
Safe
9.14 kB
Syllable is the Token
about 1 month ago
README.md
5.93 kB
Update README.md
23 days ago
encoder.py
Safe
13.1 kB
Seperate Before you Compress
23 days ago
gpe_trainer.py
Safe
28.4 kB
Seperate Before you Compress
23 days ago
linguis_trie.py
Safe
11.1 kB
WWHO
25 days ago
router.py
Safe
5.75 kB
Seperate Before you Compress
23 days ago
tokenizer.json
Safe
8.07 MB
WWHO
25 days ago
vocab.json
Safe
10.4 MB
WWHO
25 days ago