Loop-DFS-Qwen-Merged

LOOP DFS Qwen Merged 16-bit Model

Revised README.md

# LOOP DFS Qwen Merged 16-bit Model

This repository contains a merged 16-bit Qwen model fine-tuned for LOOP DFS assistance.

The model is designed to answer questions about LOOP DFS company information, products, services, partnerships, and documented API behavior.

## Generation configuration

The recommended generation settings are:

- `max_new_tokens`: 512
- `temperature`: 0.2
- `top_p`: 0.9
- `repetition_penalty`: 1.05

The complete system prompt is stored in `system_prompt.txt`.

Machine-readable inference settings are stored in `deployment_config.json`.

## Quick start

Install the required packages:

```bash
pip install -U transformers accelerate huggingface_hub sentencepiece torch
```

Load the model and its system prompt directly from this public repository:


```python
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer


MODEL_ID = "RayNene/Loop-DFS-Qwen-Merged"
MAX_NEW_TOKENS = 512


# Load the complete system prompt from the repository.
system_prompt_path = hf_hub_download(
    repo_id=MODEL_ID,
    filename="system_prompt.txt",
)

with open(system_prompt_path, "r", encoding="utf-8") as file:
    SYSTEM_PROMPT = file.read().strip()

if not SYSTEM_PROMPT:
    raise RuntimeError("system_prompt.txt is empty.")


# Select an appropriate inference data type.
if torch.cuda.is_available():
    inference_dtype = (
        torch.bfloat16
        if torch.cuda.is_bf16_supported()
        else torch.float16
    )
else:
    inference_dtype = torch.float32


# Load the tokenizer.
tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token


# Load the merged model.
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=inference_dtype,
    device_map="auto",
    trust_remote_code=True,
    low_cpu_mem_usage=True,
)

model.eval()


# Configure valid end-of-response tokens.
terminator_ids = []

for token_id in [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|im_end|>"),
]:
    if token_id is None:
        continue

    if token_id == tokenizer.unk_token_id:
        continue

    if token_id not in terminator_ids:
        terminator_ids.append(token_id)

if not terminator_ids:
    raise RuntimeError("No valid generation terminator was found.")


def clean_answer(answer):
    """Prevent accidental generation of another conversation turn."""
    stop_markers = [
        "<|im_end|>",
        "<|im_start|>system",
        "<|im_start|>user",
        "<|im_start|>assistant",
        "\nSystem:",
        "\nUser:",
    ]

    positions = [
        answer.find(marker)
        for marker in stop_markers
        if answer.find(marker) >= 0
    ]

    if positions:
        answer = answer[:min(positions)]

    return answer.strip()


def ask(question, history=None):
    """Generate one LOOP DFS assistant response."""
    if not isinstance(question, str) or not question.strip():
        raise ValueError("The question must be a non-empty string.")

    if history is None:
        history = []

    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        *history,
        {
            "role": "user",
            "content": question.strip(),
        },
    ]

    prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
        add_special_tokens=False,
    )

    input_device = model.get_input_embeddings().weight.device

    inputs = {
        name: tensor.to(input_device)
        for name, tensor in inputs.items()
    }

    prompt_length = inputs["input_ids"].shape[1]

    with torch.inference_mode():
        output = model.generate(
            **inputs,
            max_new_tokens=MAX_NEW_TOKENS,
            do_sample=True,
            temperature=0.2,
            top_p=0.9,
            repetition_penalty=1.05,
            eos_token_id=terminator_ids,
            pad_token_id=tokenizer.pad_token_id,
            use_cache=True,
        )

    generated_tokens = output[0, prompt_length:]

    answer = tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True,
    )

    return clean_answer(answer)


answer = ask("Who is the CEO of LOOP DFS Kenya?")
print(answer)
```

## Interactive chat

The following starter code maintains conversation history and uses the complete system prompt loaded from `system_prompt.txt`:

```python
conversation = []

print("LOOP DFS Assistant")
print("Commands: clear | exit | quit")
print()

while True:
    try:
        user_message = input("You: ").strip()

        if not user_message:
            continue

        command = user_message.lower()

        if command in {"exit", "quit"}:
            print("Chat ended.")
            break

        if command == "clear":
            conversation.clear()
            print("Conversation history cleared.")
            print()
            continue

        response = ask(
            question=user_message,
            history=conversation,
        )

        print(f"\nLOOP DFS Assistant: {response}\n")

        conversation.extend(
            [
                {
                    "role": "user",
                    "content": user_message,
                },
                {
                    "role": "assistant",
                    "content": response,
                },
            ]
        )

    except KeyboardInterrupt:
        print("\nChat ended.")
        break
```

## Google Colab starter

In a new Google Colab notebook:

1. Select **Runtime → Change runtime type → GPU**.
2. Run the installation cell:

```python
!pip -q install -U transformers accelerate huggingface_hub sentencepiece
```

3. Run the Python code from the **Quick start** section.
4. Run the **Interactive chat** section.

No Hugging Face token is required because the repository is public.

## Loading the deployment configuration

Applications can load the recommended generation configuration directly from `deployment_config.json`:

```python
import json
from huggingface_hub import hf_hub_download


config_path = hf_hub_download(
    repo_id="RayNene/Loop-DFS-Qwen-Merged",
    filename="deployment_config.json",
)

with open(config_path, "r", encoding="utf-8") as file:
    deployment_config = json.load(file)

print(deployment_config)
```

It can then be applied during generation:

```python
output = model.generate(
    **inputs,
    max_new_tokens=deployment_config.get(
        "max_new_tokens",
        512,
    ),
    do_sample=deployment_config.get(
        "do_sample",
        True,
    ),
    temperature=deployment_config.get(
        "temperature",
        0.2,
    ),
    top_p=deployment_config.get(
        "top_p",
        0.9,
    ),
    repetition_penalty=deployment_config.get(
        "repetition_penalty",
        1.05,
    ),
    eos_token_id=terminator_ids,
    pad_token_id=tokenizer.pad_token_id,
    use_cache=True,
)
```

## Deployment requirements

Current company facts, product terms, fees, eligibility requirements, leadership information, partnerships, and API behavior should be supplied through approved retrieval sources or verified tools.

Authentication and authorization must be handled outside the conversation. The assistant must never collect passwords, PINs, OTPs, CVVs, secret keys, access tokens, or other authentication credentials through chat.

Financial transactions must be validated by application code, summarized for the user, and explicitly confirmed through an approved secure flow. The model must not be used as the transaction authorization layer.

## Repository files

- `system_prompt.txt` — complete production system prompt
- `deployment_config.json` — recommended generation and deployment settings
- `config.json` — model architecture configuration
- `generation_config.json` — model generation configuration, when included
- `tokenizer_config.json` — tokenizer configuration
- `tokenizer.json` — tokenizer vocabulary and rules, when included
- `*.safetensors` — merged model weights