Update tokenizer files (and add tokenizer.json)
#3
by
Xenova HF Staff - opened
Exported with tokenizer.save_pretrained(...) (latest transformers version)
Quite a large diff, so if anything, we can just update the tokenizer.json file.
Xenova changed pull request status to
merged