codewithkyrian/tokenizers (2.1.1)
syndicated from packagist.org
on June 17, 2026
Fast, pure-PHP tokenizer library compatible with Hugging Face tokenizers for encoding and decoding text
Fast, pure-PHP tokenizer library compatible with Hugging Face tokenizers for encoding and decoding text