Hey all! I am Aritra from Hugging Face. I wanted to share an update on the `tokenizers` library that we have at Hugging Face. It has gone under major changes and we have finally released version 1 of it. Here are what we are most excited about: > multiple language support > multi-thread scaling > mi
