phatjk

vinallama-7b-AWQ

phatjk/vinallama-7b-AWQ

vinallama-7b-AWQ is a text generation model from phatjk. Use it when you need the model to write or continue text. It is set up for transformers.

quantconfig = { "zeropoint": True, "qgroupsize": 128, "wbit": 4, "version": "GEMM" }