Qwen2.5-Math-1.5B-Instruct-GGUF

<!-- header start --> <!-- 200823 --> <div style="width: auto; margin-left: auto; margin-right: auto"> <img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;"> </div> <hr style="margin-top: 1.0em; margin-bottom: 1.0em;"> <!-- header end -->

Qwen2.5-Math-1.5B-Instruct-GGUF

Original Model

Qwen/Qwen2.5-Math-1.5B-Instruct

Run with LlamaEdge

• LlamaEdge version: coming soon

<!-- - LlamaEdge version: [v0.14.3](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.14.3) --> <!-- - Prompt template - Prompt type: coming soon - Prompt string - File-Level Code Completion (Fill in the middle) ```text <|fim_prefix|>{prefix_code}<|fim_suffix|>{suffix_code}<|fim_middle|> ``` *Reference: https://github.com/QwenLM/Qwen2.5-Coder?tab=readme-ov-file#3-file-level-code-completion-fill-in-the-middle* - Repository-Level Code Completion ```text <|repo_name|>{repo_name} <|file_sep|>{file_path1} {file_content1} <|file_sep|>{file_path2} {file_content2} ``` *Reference: https://github.com/QwenLM/Qwen2.5-Coder?tab=readme-ov-file#4-repository-level-code-completion* - Context size: `128000` - Run as LlamaEdge service ```bash wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen2.5-Math-1.5B-Instruct-Q5_K_M.gguf \ llama-api-server.wasm \ --model-name Qwen2.5-Math-1.5B-Instruct \ --prompt-template chatml \ --ctx-size 128000 ``` - Run as LlamaEdge command app ```bash wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen2.5-Math-1.5B-Instruct-Q5_K_M.gguf \ llama-chat.wasm \ --prompt-template chatml \ --ctx-size 128000 ``` -->

Quantized GGUF Models

Quantized with llama.cpp b3751