codingninja

llama3-32k-pa-tokenizer

codingninja/llama3-32k-pa-tokenizer

llama3-32k-pa-tokenizer is a machine learning model from codingninja. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

Vichaar ds - indexesranges = [(0, 600000), (2200000, 2200000+1738597), (2200000+1738597, 2200000+1738597+800000), (2200000+1738597+5824922, 2200000+1738597+5824922+700000)]