TranslateQwen_v2-GGUF

TranslateQwen_v2 GGUF

GGUF conversion of alwaysgood/scp-stage4-oneshot-reverse_dpo_sigmoid for llama.cpp and LM Studio.

Files

The main BF16 and Q4_K_M GGUF files are language/text models. Vision/image tensors are split into the separate mmproj-BF16 GGUF, so image input requires loading both the main model and the matching mmproj file.

LM Studio custom template

This model was trained with a raw DPO/SFT translation prompt, not a ChatML chat prompt. LM Studio's normal User/Assistant split can still be used by putting the fixed Instruction and Source headers in the custom template.

In LM Studio, open the model settings, go to Advanced Configuration -> Prompt Template, choose Custom, and use:

If you type this in LM Studio chat:

The Federal Reserve raised interest rates by 25 basis points.

LM Studio should send this prompt to the model:

### Instruction:
Translate the English source into Korean.

### Source:
The Federal Reserve raised interest rates by 25 basis points.

### Response:

Suggested decoding: greedy or low-temperature translation, e.g. temperature 0.0 to 0.3, context length 8192.

llama.cpp

Text:

llama-cli \
  -m TranslateQwen_v2-Q4_K_M.gguf \
  --color -ngl all -fa -c 8192 \
  -p $'### Instruction:\nTranslate the English source into Korean.\n\n### Source:\nThe Federal Reserve raised interest rates by 25 basis points.\n\n### Response:\n'

Vision:

llama-mtmd-cli \
  -m TranslateQwen_v2-Q4_K_M.gguf \
  --mmproj TranslateQwen_v2-mmproj-BF16.gguf \
  --image /path/to/image.png \
  -p $'### Instruction:\nTranslate the English source into Korean.\n\n### Source:\nDescribe this image in Korean.\n\n### Response:\n' \
  -ngl all -fa -c 8192