my_optimal_model-GGUF-with-sysprompt

Nepali Legal AI - GGUF (System Prompt Embedded)

🎯 KEY FEATURE: Legal citation system prompt is permanently embedded in this GGUF file!

⚡ What This Means

• ✅ Just load and use - no configuration needed
• ✅ Every response automatically includes proper legal citations
• ✅ Works in Ollama, llama.cpp, LM Studio, Python
• ✅ Users cannot bypass citation requirements
• ✅ Professional legal formatting guaranteed

🚀 Quick Start

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

# Download model
model_path = hf_hub_download(
    repo_id="yamraj047/my_optimal_model-GGUF-with-sysprompt",
    filename="my-optimal-model-Q4_K_M.gguf"
)

# Load and use (system prompt is automatic!)
llm = Llama(model_path=model_path, n_ctx=2048, n_threads=4)

response = llm("Can four friends start a company in Nepal?", max_tokens=400)
print(response['choices'][0]['text'])

Output automatically includes:

Under Law: Company Act, 2063 (2006)
Chapter: 3 - Incorporation of Company
Section: 46 - Appointment of Directors

Yes, four friends can collectively form a company...

📊 Specifications

💻 Usage Examples

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="path/to/my-optimal-model-Q4_K_M.gguf",
    n_ctx=2048,
    n_threads=4
)

# Just ask - formatting is automatic!
response = llm("What are director requirements?", max_tokens=300)
print(response['choices'][0]['text'])

Ollama

# Import model
ollama create nepali-legal -f - <<EOF
FROM yamraj047/my_optimal_model-GGUF-with-sysprompt:latest
EOF

# Use it
ollama run nepali-legal "Tell me about company formation"

Gradio Interface

import gradio as gr
from llama_cpp import Llama

llm = Llama(model_path="path/to/model.gguf", n_ctx=2048)

def chat(message, history):
    response = llm(message, max_tokens=400)
    return response['choices'][0]['text'].strip()

gr.ChatInterface(
    fn=chat,
    title="🤖 Nepali Legal AI"
).launch()

🎯 Embedded System Prompt

This model automatically applies the following behavior:

• ✅ Always cites Law, Chapter, and Section
• ✅ Uses format: "Under Law: [Name], Chapter: [Number], Section: [Number]"
• ✅ Provides clear explanations
• ✅ Recommends consulting legal professionals
• ✅ Maintains professional tone

No manual configuration needed!

🔧 Technical Details

Created from: yamraj047/jaii2.033my_optimal_model-merged-fp16

Process:

1. Downloaded FP16 merged model
2. Injected system prompt into tokenizer_config.json chat template
3. Converted to F16 GGUF
4. Quantized to Q4_K_M for optimal CPU performance

The system prompt is embedded in the model's chat template and automatically applies to all conversations.

📜 License

Apache 2.0

⚠️ Disclaimer

This AI provides general legal information only. Always consult a licensed legal professional for specific legal matters.