gstaff

TensorRT-LLM-engine-Mistral-7B-instruct-v0.2

gstaff/TensorRT-LLM-engine-Mistral-7B-instruct-v0.2

TensorRT-LLM-engine-Mistral-7B-instruct-v0.2 is a machine learning model from gstaff. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

This repo contains TensorRT-LLM engine files for Mistral-7B-instruct-v0.2 compiled on Windows 11 in single-GPU mode on a RTX 3080Ti GPU.