Downloads · 30 days
6
8% of all-time downloads
BennyWang/Qwen2.5-0.5B-Instruct-cl-24233news-5stg-v4-lr_adj-q4f16_1-MLC
Qwen2.5-0.5B-Instruct-cl-24233news-5stg-v4-lr_adj-q4f16_1-MLC is a machine learning model from BennyWang. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlc-llm.
This is the Qwen2.5-0.5B-Instruct model in MLC format q4f161. The model can be used for projects MLC-LLM and WebLLM.
Downloads · 30 days
6
8% of all-time downloads
All-time downloads
72
Public
Repo size
296 MB
Likes
0
Public
Click a slice to open those files.
.bin278 MB · 96%
From the Hugging Face model README
This is the Qwen2.5-0.5B-Instruct model in MLC format q4f16_1.
The model can be used for projects MLC-LLM and WebLLM.
Model Training: Detailed training procedures, dataset configurations, and curriculum learning stages used to fine-tune this model are documented in the Individual Studies repository
Demo Application: To explore the model's capabilities in a real-world mobile setup, refer to the companion app repository: Individual Studies App
Here are some examples of using this model in MLC LLM. Before running the examples, please install MLC LLM by following the installation documentation.
In command line, run
mlc_llm chat HF://BennyWang/Qwen2.5-0.5B-Instruct-cl-24233news-5stg-v4-lr_adj-q4f16_1-MLC
In command line, run
mlc_llm serve HF://BennyWang/Qwen2.5-0.5B-Instruct-cl-24233news-5stg-v4-lr_adj-q4f16_1-MLC
from mlc_llm import MLCEngine
# Create engine
model = "HF://BennyWang/Qwen2.5-0.5B-Instruct-cl-24233news-5stg-v4-lr_adj-q4f16_1-MLC"
engine = MLCEngine(model)
# Run chat completion in OpenAI API.
for response in engine.chat.completions.create(
messages=[{"role": "user", "content": "What is the meaning of life?"}],
model=model,
stream=True,
):
for choice in response.choices:
print(choice.delta.content, end="", flush=True)
print("\n")
engine.terminate()
For more information on MLC LLM project, please visit our documentation and GitHub repo.