Downloads · 30 days
0
Llama-3.1-8B-Instruct-QAIC
Hyratek/Llama-3.1-8B-Instruct-QAIC
Llama-3.1-8B-Instruct-QAIC is a text generation model from Hyratek. Use it when you need the model to write or continue text. It is set up for transformers.
This model, derived from Meta’s Llama-3.1-8B-Instruct, has been converted and optimized to run efficiently on Qualcomm Cloud AI 100 hardware. Leveraging Qualcomm's developer-centric toolchain, it incorporates reengine…
Access
Public
Updated Nov 4, 2024
Repo size
7.1 GB
Likes
9
Public
Repo makeup
Click a slice to open those files.
.bin7.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Access
- Public
- Created
- Oct 31, 2024
- Updated
- Nov 4, 2024
- SHA
- 4a96ddcc
Try a prompt
Base models
Model card
From the Hugging Face model README
Model Information
This model, derived from Meta’s Llama-3.1-8B-Instruct, has been converted and optimized to run efficiently on Qualcomm Cloud AI 100 hardware. Leveraging Qualcomm's developer-centric toolchain, it incorporates reengineered Transformer components and precision-optimized graph transformations for enhanced performance on-device.
Key Features
- Optimized LLM Blocks: Includes custom modules to handle intermediate states and precision challenges, ensuring high-performance inference.
- Transformation Tools: Supports graph modifications to retain model accuracy while improving efficiency through mathematical optimizations.
- Export Ready: Compatible with ONNX for easy deployment.
- Comprehensive Testing: Each PR undergoes extensive validation, comparing MSE against the original model.
- Task
- Text Generation
- Library
- transformers
- Created
- Oct 31, 2024
- Updated
- Nov 4, 2024