Post
86
tokenizers 0.23.3 is out!https://github.com/huggingface/tokenizers/releases/tag/v0.23.3
The release is python-only and fixes a few very annoying issues
hf-inference
Team
non-profit
Inference Provider
VERIFIED
13,267,182 monthly requests
AI & ML interests
None defined yet.
Recent Activity
https://github.com/huggingface/tokenizers/releases/tag/v0.23.3
The release is python-only and fixes a few very annoying issues
Post
246
I put together a Space to showcase tokenization-in-the-browser libs we maintain at HF
Let me know what you think!
sbrandeis/tokenizers-wasm-demo
Let me know what you think!
sbrandeis/tokenizers-wasm-demo
tomaarsen
updated a
collection 21 days ago