nia-v1

Model Information

Description

This model was fine-tuned with PEFT/LoRA and merged into a standalone set of weights for deployment. It focuses on short, structured dialogue, refusal behavior for unsafe requests, and instruction adherence in tool-style prompts. Fine-tuned from gemma-3-27b-it. The model config includes vision components and image token settings, indicating image+text capability.

Training Data Summary

• Chat: ultrachat_messages.jsonl, oasst1_messages.jsonl.
• Safety: hh_rlhf_messages.jsonl, pku_safrlhf_messages.jsonl.
• Tool-style: xlam_messages.jsonl.
• Validation: validation_messages.jsonl.

Usage

Transformers (Python)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "dodotechnologies/dodo-v1-beta"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

prompt = "User: Explain LoRA in three bullets.\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)

print(tokenizer.decode(output[0], skip_special_tokens=True))

Hugging Face Inference Endpoint

curl -X POST \
  -H "Authorization: Bearer $HF_TOKEN" \
  -H "Content-Type: application/json" \
  https://api-inference.huggingface.co/models/dodotechnologies/dodo-v1-beta \
  -d '{
    "inputs": "User: Explain LoRA in three bullets.\nAssistant:",
    "parameters": {"max_new_tokens": 128, "do_sample": false}
  }'

Local/Private Inference Server

Run a private API service on a GPU host and expose it only to authorized clients.

curl -X POST http://<host>:<port>/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "User: Explain LLM finetuning.\nAssistant:",
    "max_new_tokens": 128,
    "do_sample": false
  }'

Deployment Runbook

1. Confirm the merged model artifacts are present.
2. Launch inference server with a GPU that can host a 27B model (A100-80GB recommended).
3. Run a small smoke test (one benign prompt and one refusal prompt).
4. Enable logging, rate-limiting, and error monitoring.