aless2212

Mistral-7B-Instruct-v0.2-onnx-fp16

aless2212/Mistral-7B-Instruct-v0.2-onnx-fp16

Mistral-7B-Instruct-v0.2-onnx-fp16 is a text generation model from aless2212. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

see the instructions there. The fp32 can be probably used with CPU. fp16 is only gpu. all conversion done on an RX7600 8GB with my custom versions of PyTorch and Onnxruntime. https://huggingface.co/aless2212/Mistral-7…