DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-q4f16_1-MLC

DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-q4f16_1-MLC

This is the DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-q4f16_1-MLC model in MLC format q4f16_1.
The model can be used for projects MLC-LLM and WebLLM.

Thank you mitulagr2 for creating the DeepSeek-R1-Distill-Qwen-1.5B-q4f16_1 custom wasm.
The MLC_LLM library from the official sources does not support Distill-Qwen-1.5B-q4f16_1.

感謝mitulagr2 製作DeepSeek-R1-Distill-Qwen-1.5B-q4f16_1專用的wasm。
MLC_LLM 官方lib無法使用Distill-Qwen-1.5B-q4f16_1。
WASM：https://huggingface.co/mitulagr2/DeepSeek-R1-Distill-Qwen-1.5B-q4f16_1-MLC/tree/main

Example Usage

Here are some examples of using this model in MLC LLM. Before running the examples, please install MLC LLM by following the installation documentation.

Chat

In command line, run

mlc_llm chat HF://willopcbeta/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-q4f16_1-MLC

REST Server

In command line, run

mlc_llm serve HF://willopcbeta/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-q4f16_1-MLC

Python API

from mlc_llm import MLCEngine

# Create engine
model = "HF://willopcbeta/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-q4f16_1-MLC"
engine = MLCEngine(model)

# Run chat completion in OpenAI API.
for response in engine.chat.completions.create(
    messages=[{"role": "user", "content": "What is the meaning of life?"}],
    model=model,
    stream=True,
):
    for choice in response.choices:
        print(choice.delta.content, end="", flush=True)
print("\n")

engine.terminate()

Documentation

For more information on MLC LLM project, please visit our documentation and GitHub repo.