qwen2-5-14b-korean-coding-assistant-gguf

<!-- DGX_LEGACY_NOTICE_START -->

Legacy / Experimental Release

This repository is kept public as part of the development portfolio.

Latest production release:

https://huggingface.co/koreallmdev/qwen2-5-14b-korean-coding-assistant-gguf

최신 공개 운영 모델은 아래 repo를 사용하세요.

koreallmdev/qwen2-5-14b-korean-coding-assistant-gguf

<!-- DGX_LEGACY_NOTICE_END -->

---

<!-- DGX_README_POLISH_START -->

Qwen2.5 14B Korean Coding Assistant GGUF

DGX 로컬 코딩/운영 비서용 Qwen2.5 14B 계열 GGUF 릴리스입니다.

This repository provides Korean coding and operations assistant GGUF builds based on a Qwen2.5 14B repair workflow.

Available GGUF files

Which one should I use?

Use Q8_0 for best response quality.

Use Q4_K_M when memory usage, loading speed, or smaller download size matters more.

한국어 기준:

• 품질 우선이면 Q8_0을 권장합니다.
• 메모리 사용량과 로딩 속도를 우선하면 Q4_K_M을 권장합니다.

Ollama usage

Q8_0

FROM ./gguf/dgx-coding-prod-q8_0.gguf

PARAMETER temperature 0
PARAMETER top_p 0.82
PARAMETER repeat_penalty 1.06
PARAMETER num_ctx 4096
PARAMETER num_predict 512

ollama create qwen2-5-14b-korean-coding-assistant-q8 -f Modelfile
ollama run qwen2-5-14b-korean-coding-assistant-q8

Q4_K_M

FROM ./gguf/dgx-coding-prod-q4_k_m.gguf

PARAMETER temperature 0
PARAMETER top_p 0.82
PARAMETER repeat_penalty 1.06
PARAMETER num_ctx 4096
PARAMETER num_predict 512

ollama create qwen2-5-14b-korean-coding-assistant-q4 -f Modelfile
ollama run qwen2-5-14b-korean-coding-assistant-q4

Intended behavior

• Korean honorific answers.
• Avoid Chinese/Japanese leakage.
• Prefer executable code and shell commands.
• FastAPI/API examples include server run command and curl test unless explicitly prohibited.
• CUDA memory checks use torch.cuda.mem_get_info().
• systemd answers use daemon-reload -> restart -> status.
• Docker answers use logs -> restart -> ps.
• Ollama Modelfile examples use FROM, PARAMETER, and ollama create.

Local benchmark summary

Q8_0

model              = dgx-repair-v1-q8-prod2-safe:latest
average_score      = 93.9
median_score       = 100.0
pass_70_plus       = 9/10
strong_85_plus     = 9/10
api_failures       = 0
cjk_fail_count     = 0
avg_latency_sec    = 4.42

Q4_K_M

model              = dgx-coding-prod-q4-candidate:latest
average_score      = 94.46
median_score       = 100.0
pass_70_plus       = 20/20
strong_85_plus     = 15/20
api_failures       = 0
cjk_fail_count     = 0
avg_latency_sec    = 11.875

Portfolio note

This is the latest public production GGUF release. Earlier repositories are kept public as portfolio and development history.

<!-- DGX_README_POLISH_END -->

---

Previous model card content

DGX 로컬 코딩/운영 비서용 Qwen2.5 14B 계열 repair 모델입니다.

Deployment model

Recommended Ollama alias:

ollama run dgx-coding-prod

Release GGUF:

gguf/dgx-coding-prod-q8_0.gguf

Intended behavior

• Korean honorific answers.
• Avoid CJK leakage.
• Prefer executable code and shell commands.
• FastAPI/API examples include server run command and curl test unless explicitly prohibited.
• CUDA memory checks use torch.cuda.mem_get_info().
• systemd answers use daemon-reload -> restart -> status.
• Ollama Modelfile examples use FROM, PARAMETER, and ollama create.

Latest local benchmark

Best local candidate before final alias promotion:

dgx-repair-v1-q8-prod2-safe:latest
average_score   = 93.9
median_score    = 100.0
pass_70_plus    = 9/10
strong_85_plus  = 9/10
api_failures    = 0
cjk_fail_count  = 0
avg_latency_sec = 4.42

Final operational alias accepted a flexible Korean honorific rule because 보내 주시면 and 보내주십시오 are valid honorific Korean even when an exact scorer expects 주세요.

Notes

This release is optimized for the user's local DGX/Ollama/Open-WebUI workflow and local benchmark suite. It should be evaluated again if used outside that environment.

Q4_K_M Release

A Q4_K_M GGUF build is now included for lower memory usage.

gguf/dgx-coding-prod-q4_k_m.gguf

Recommended local Ollama alias:

ollama run dgx-coding-prod-q4

Q4 Benchmark

model             = dgx-coding-prod-q4-candidate:latest
average_score     = 94.46
median_score      = 100.0
pass_70_plus      = 20/20
strong_85_plus    = 15/20
api_failures      = 0
cjk_fail_count    = 0
avg_latency_sec   = 11.875

Q4 is provided as a practical, smaller deployment option. For best quality, use Q8; for lower memory and faster loading, use Q4.