Yi-34B-GGUF

<img src="data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iNjAwIiBoZWlnaHQ9IjYwMCIgdmlld0JveD0iMCAwIDYwMCA2MDAiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CjxjaXJjbGUgY3g9IjMwMCIgY3k9IjMwMCIgcj0iMzAwIiBmaWxsPSIjMDAzNDI1Ii8+CjxyZWN0IHg9IjQwOS43MzMiIHk9IjM0MC4wMzEiIHdpZHRoPSI0Mi4zODYyIiBoZWlnaHQ9IjE1MS42NDgiIHJ4PSIyMS4xOTMxIiBmaWxsPSJ3aGl0ZSIvPgo8cGF0aCBmaWxsLXJ1bGU9ImV2ZW5vZGQiIGNsaXAtcnVsZT0iZXZlbm9kZCIgZD0iTTQyMi4wMDUgMTMzLjM1NEM0MTMuMDg5IDEyNS43NzEgMzk5LjcxNCAxMjYuODUxIDM5Mi4xMzEgMTM1Ljc2N0wyNzMuNjk5IDI3NS4wMjFDMjcwLjY0MyAyNzguNjE0IDI2OC45OTQgMjgyLjkzMiAyNjguNjk4IDI4Ny4zMDJDMjY4LjUzMiAyODguMzcxIDI2OC40NDYgMjg5LjQ2NiAyNjguNDQ2IDI5MC41ODFWNDY4LjYwM0MyNjguNDQ2IDQ4MC4zMDggMjc3LjkzNCA0ODkuNzk2IDI4OS42MzkgNDg5Ljc5NkMzMDEuMzQ0IDQ4OS43OTYgMzEwLjgzMiA0ODAuMzA4IDMxMC44MzIgNDY4LjYwM1YyOTYuNzg0TDQyNC40MTkgMTYzLjIyOEM0MzIuMDAyIDE1NC4zMTIgNDMwLjkyMSAxNDAuOTM3IDQyMi4wMDUgMTMzLjM1NFoiIGZpbGw9IndoaXRlIi8+CjxyZWN0IHg9IjExMy45NzIiIHk9IjEzNC4yNSIgd2lkdGg9IjQyLjM4NjIiIGhlaWdodD0iMTc0Ljc0NSIgcng9IjIxLjE5MzEiIHRyYW5zZm9ybT0icm90YXRlKC0zOS4zNDQxIDExMy45NzIgMTM0LjI1KSIgZmlsbD0id2hpdGUiLz4KPGNpcmNsZSBjeD0iNDYwLjEyNiIgY3k9IjI3OS4yNzgiIHI9IjI1LjkwMjciIGZpbGw9IiMwMEZGMjUiLz4KPC9zdmc+" />

These are GGUF quantized versions of 01-ai/Yi-34B.

The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.

The IQ2_XXS and IQ2_XS versions are compatible with llama.cpp, version 147b17a or later. The IQ3_XXS requires version f4d7e54 or later.

Some model files above 50GB are split into smaller files. To concatenate them, use the cat command (on Windows, use PowerShell): cat foo-Q6_K.gguf.* > foo-Q6_K.gguf