solana-clawd-core-ai-1.5b-lora

Solana Clawd Core AI 1.5B LoRA

A LoRA fine-tune of Qwen2.5-1.5B-Instruct for Solana development, DeFi reasoning, memecoin risk analysis, agent architecture, Clawd constitutional behavior, and the core-ai source tree.

Base model: Qwen/Qwen2.5-1.5B-Instruct
Adapter type: LoRA (r=16, alpha=32, ~9M trainable params / 0.6% of base)
Primary training data: solanaclawd/solana-clawd-core-ai-instruct — 35,173 examples
Legacy seed dataset: solanaclawd/solana-clawd-instruct — 36,109 examples
Companion realtime dataset: solanaclawd/solana-clawd-realtime-research-instruct — 29,058 submitted PDF/notebook/parquet/ZK examples
Training config: ai-training/configs/core_ai_lora_config.yaml
Hub model ID: solanaclawd/solana-clawd-core-ai-1.5b-lora
Release status: Live — adapter_config.json + adapter_model.safetensors pushed to Hub on 2026-06-19T23:44Z by recovery job ordlibrary/6a35a6833093dba73ce2a86b (A100-large, 3h 14m, train_loss=0.9008, token_accuracy=82.9%, 24.5M tokens).

Tool-use / function calling? Use the 8B Hermes-3 base with configs/hermes3_lora_config.yaml and the perps/ function-calling suite (13 tools: funding rate, paper trade, risk assessment, Jupiter quotes).

---

Fork this to train your own Clawd

Everything below is a working example — swap in your own HF org, dataset, and base model to get your own fine-tuned Solana agent in one sitting.

# 0. Clone + install
git clone https://github.com/Solizardking/solana-clawd
cd solana-clawd/ai-training
pip install -r requirements.txt
export HF_TOKEN=hf_...           # huggingface.co/settings/tokens (write access)
export WANDB_API_KEY=...         # wandb.ai/authorize (optional, enables live charts)

# 1. (Optional) bring your own data — append to the merged dataset
#    Format: {"messages": [{"role": "system", ...}, {"role": "user", ...}, {"role": "assistant", ...}]}
#    Then re-run prepare_dataset.py with your JSONL added to the --input list.

# 2. Push the dataset to your HF org (or reuse ours — skip if using solanaclawd/solana-clawd-instruct)
python3 scripts/prepare_dataset.py \
  --input data/solana_clawd_merged.jsonl \
  --output data/processed \
  --train-ratio 0.9 --eval-ratio 0.05 \
  --seed 42 \
  --push --repo-id YOUR_ORG/your-dataset-id

# 3. Train on a remote A100 (recommended — ~$3–6 for the full 36K × 3-epoch run)
./scripts/launch_hf_jobs.sh a100-large   # or h200, l4x1

# 4. Train locally on Mac MPS (sanity check, 1 epoch)
python3 scripts/train_lora.py --num-epochs 1 --no-quant

# 5. Watch live training logs
hf jobs ps
hf jobs logs <JOB_ID> --follow

# 6. Register your model to the onchain Clawd registry (off-chain index — no wallet needed)
./dao/register_model.sh \
  --hf-model "YOUR_ORG/your-model-id" \
  --eval-accuracy 0.60 \
  --dataset-size 36109

# 7. Serve locally with Ollama
ollama create my-clawd -f ollama/Modelfile.finetuned
ollama run my-clawd "How do I detect a rug pull on a fresh Solana token?"

The entire pipeline — dataset → train → eval → onchain registry — is designed to be reproducible from a single clone. The only external requirement is a Hugging Face account with write access (free tier works).

---

What this model knows

Check every domain your training data covers:

• Solana mechanics (PDAs, accounts, instructions, rent, compute budgets, Token-2022)
• DeFi primitives (AMMs, CLMMs, perpetuals, bonding curves, Jupiter, Phoenix)
• Memecoin risk (rug detection, holder concentration, deployer forensics)
• Agent architecture (skill registries, brain/hands split, multi-agent coordination)
• ZK compression (Light Protocol, nullifiers, Groth16 — see zk-primitives/)
• Code generation (Anchor/Rust, TypeScript @solana/kit, Python)
• Constitutional reasoning (Clawd Constitution, guardrails, refusal patterns)
• Perps function calling ← Hermes-3 8B path only
• Add your domain here

---

Intended Use

Good for:

• Local Clawd agent inference (MPS / CUDA / CPU, 4 GB+ VRAM)
• Server inference (vLLM, TGI, Ollama after weight merge)
• HF Router (no GPU — OpenAI-compat endpoint)
• Clawd agent runtime (MCP skill, Telegram bot, voice agent)

Out of scope:

• Front-running, sandwich attacks, or MEV exploitation
• Wallet draining or social engineering
• Live trading without explicit user confirmation and a trust-gated execution layer
• Financial advice — this model produces analysis, not guaranteed outcomes

---

Training Details

Fireworks managed SFT run

The current Fireworks deployment uses the Hugging Face dataset export from solanaclawd/solana-clawd-instruct, uploaded to Fireworks as JSONL because the Fireworks dataset API only accepts uploaded files or cloud-storage URIs for managed SFT.

The trained model is READY in Fireworks, but both attempted on-demand deployment methods failed during model-server initialization with a Fireworks internal error. The account currently has no validated deployment shape returned for accounts/fireworks/models/qwen2p5-7b-instruct.

How to reproduce

cd /path/to/solana-clawd/ai-training
pip install -r requirements.txt
export HF_TOKEN=hf_...

# 1. Prepare dataset (uses the 36K merged file — canonical training input)
python3 scripts/prepare_dataset.py \
  --input data/solana_clawd_merged.jsonl \
  --output data/processed \
  --train-ratio 0.9 --eval-ratio 0.05 \
  --seed 42 \
  --push --repo-id solanaclawd/solana-clawd-instruct

# 2. Train (remote GPU — recommended)
./scripts/launch_hf_jobs.sh a100-large   # or h200, l4x1, a100x4

# 3. Train (local Mac MPS — sanity check, 1 epoch)
python3 scripts/train_lora.py --num-epochs 1 --no-quant

---

Evaluation

python3 scripts/evaluate.py \
  --config configs/eval_config.yaml \
  --adapter solanaclawd/solana-clawd-core-ai-1.5b-lora \
  --dataset solanaclawd/solana-clawd-eval \
  --out ./outputs/eval \
  --format markdown

---

Usage

transformers + peft (universal)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE    = "Qwen/Qwen2.5-1.5B-Instruct"
ADAPTER = "solanaclawd/solana-clawd-core-ai-1.5b-lora"

tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model     = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16,
                                                  device_map="auto", trust_remote_code=True)
model     = PeftModel.from_pretrained(model, ADAPTER)

messages = [
    {"role": "system",    "content": "You are Clawd, a sovereign Solana-native AI agent."},
    {"role": "user",      "content": "How do I detect a rug pull on a fresh token?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=512, temperature=0.2, top_p=0.9,
                         do_sample=True, pad_token_id=tokenizer.pad_token_id)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

mlx-lm (Apple Silicon — fastest local path)

pip install mlx-lm
mlx_lm.generate \
  --model Qwen/Qwen2.5-1.5B-Instruct \
  --adapter solanaclawd/solana-clawd-core-ai-1.5b-lora \
  --prompt "How do I detect a rug pull on a fresh Solana token?"

HF Router (no GPU required)

from openai import OpenAI

client = OpenAI(base_url="https://router.huggingface.co/v1", api_key="hf_...")
response = client.chat.completions.create(
    model="solanaclawd/solana-clawd-core-ai-1.5b-lora",
    messages=[
        {"role": "system", "content": "You are Clawd, a sovereign Solana-native AI agent."},
        {"role": "user",   "content": "What is a PDA?"},
    ],
    max_tokens=512,
)
print(response.choices[0].message.content)

Hermes-3 perps function calling (8B path only)

# Paper trade via the perps agent
python3 ai-training/perps/functioncall.py \
  --query "Paper trade: long SOL-PERP $500 at 3x" --verbose

# GOAP reasoning (multi-step strategies)
python3 ai-training/perps/functioncall.py \
  --goap --query "Assess risk of shorting SOL-PERP $1000 at 5x"

---

Solana Perps Tool Template (included in the kit)

The perps/ directory is a drop-in tool library for building Solana perpetuals agents. It works out of the box with no API keys for read-only data, and plugs directly into any Hermes-3 or OpenAI-compatible function-calling loop.

13 tools included

Quick start

# Plug the tool library into any OpenAI-compatible function-calling agent
from perps.functions import get_openai_tools, call_function

tools = get_openai_tools()   # returns all 13 tools in OpenAI tool format

# Call a tool directly (no model needed)
import json
print(json.dumps(call_function("get_sol_price", {}), indent=2))
print(json.dumps(call_function("assess_position_risk", {
    "market": "SOL-PERP", "side": "long", "size_usd": 500, "leverage": 3
}), indent=2))

# Run the full Hermes-3 perps agent (HF Router — no GPU)
python3 perps/functioncall.py --query "What's the SOL-PERP funding rate? Should I go long?"

# GOAP multi-step reasoning mode
python3 perps/functioncall.py --goap \
  --query "Assess the risk of shorting SOL-PERP with $1000 at 5x leverage"

# Local Hermes-3 (needs GPU or quantized model)
HERMES_LOCAL=1 python3 perps/functioncall.py \
  --query "Paper trade: long SOL-PERP $500 at 3x"

Pydantic schemas (for structured output)

from perps.schema import TradeOrder, RiskAssessment, MarketSignal

# Force the model to emit a valid TradeOrder JSON
# Pass schema.TradeOrder.schema_json() as the response_format to any OpenAI client

Adding your own tools

# perps/functions.py — add a new tool with the @tool decorator
from functions import tool, ALL_TOOLS

@tool(
    description="Get the top token holders for a mint (uses Helius DAS).",
    parameters={
        "type": "object",
        "properties": {
            "mint": {"type": "string", "description": "Token mint address"},
            "limit": {"type": "integer", "description": "Number of holders", "default": 10},
        },
        "required": ["mint"],
    }
)
def get_top_holders(mint: str, limit: int = 10) -> dict:
    # your implementation here
    return {"mint": mint, "holders": []}

ALL_TOOLS.append(get_top_holders)   # auto-registered in get_openai_tools()

---

Onchain Model Registry

Every model trained with this kit gets a permanent, verifiable onchain identity anchored via the solana_ai_inference Anchor program and indexed at onchain.x402.wtf. No centralized API needed — the PDA is queryable forever.

Layer 1 — Off-chain index (one curl, no wallet)

The fastest path. Posts to the onchain.x402.wtf registry and returns a CAAP/1.0 JSON record. Good enough for discovery and routing.

# Auto-computes model hash from train_lora.py
./dao/register_model.sh \
  --hf-model "YOUR_ORG/your-model-id" \
  --eval-accuracy 0.60 \
  --dataset-size 36109

# Dry run to preview the payload first
./dao/register_model.sh --dry-run \
  --hf-model "YOUR_ORG/your-model-id"

# Manual curl (no shell dep)
curl -X POST https://onchain.x402.wtf/api/register \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $HF_TOKEN" \
  -d '{
    "model_hash":    "sha256:<hash>",
    "model_type":    "TextGeneration",
    "api_endpoint":  "https://clawd-box-router.fly.dev/v1",
    "hf_model_id":   "YOUR_ORG/your-model-id",
    "dataset_size":  36109,
    "eval_accuracy": 0.60,
    "protocol":      "CAAP/1.0",
    "clawd_token":   "8cHzQHUS2s2h8TzCmfqPKYiM4dSt4roa3n7MyRLApump",
    "registered_at": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"
  }'

Valid model_type values: TextGeneration | SentimentAnalysis | ImageClassification | PricePrediction | DocumentUnderstanding

Layer 2 — Onchain PDA (Anchor tx, permanent)

Creates a ModelRegistry PDA at seeds ["model", authority.pubkey]. Requires a funded Solana wallet (devnet) and pnpm.

# One command — the shell wrapper handles both layers automatically
./dao/register_model.sh --onchain \
  --hf-model   "YOUR_ORG/your-model-id" \
  --endpoint   "https://clawd-box-router.fly.dev/v1" \
  --cluster    devnet \
  --keypair    ~/.config/solana/id.json

# Or call the TypeScript client directly
pnpm tsx dao/register_model.ts \
  --model-hash  "sha256:$(sha256sum scripts/train_lora.py | awk '{print $1}')" \
  --model-type  "TextGeneration" \
  --endpoint    "https://clawd-box-router.fly.dev/v1" \
  --reward-rate 1000000 \
  --keypair     ~/.config/solana/id.json \
  --cluster     devnet

# Derive the PDA address without submitting a tx
# seeds: ["model", authority.pubkey]
# program: 3dLst2E3djtCSwG19mFS3REHxtZPngjyga7iYZLDL5xj

# Verify the PDA after registration
solana account <MODEL_REGISTRY_PDA> --url devnet --output json

Layer 3 — ZK attestations (SAS + Light Protocol)

Anchor model quality claims as verifiable, tamper-proof compressed credentials. ~0.00003 SOL per attestation with Light Protocol.

# Eval result attestation (ties W&B run to onchain PDA)
pnpm tsx dao/attestation/create_attestation.ts \
  --type      eval \
  --model-id  "YOUR_ORG/your-model-id" \
  --accuracy  0.60 \
  --wandb-run "ktvtubjs" \
  --keypair   ~/.config/solana/id.json \
  --dry-run    # remove --dry-run to submit for real

# Dataset snapshot attestation (Merkle root of 36K examples)
pnpm tsx dao/attestation/create_attestation.ts \
  --type      dataset \
  --model-id  "YOUR_ORG/your-model-id" \
  --size      36109 \
  --hash      "sha256:$(sha256sum data/solana_clawd_merged.jsonl | awk '{print $1}')" \
  --compressed \
  --keypair   ~/.config/solana/id.json

# LoRA adapter attestation (ties adapter weights to training run)
pnpm tsx dao/attestation/create_attestation.ts \
  --type          adapter \
  --model-id      "YOUR_ORG/your-model-id" \
  --base-model    "Qwen/Qwen2.5-1.5B-Instruct" \
  --lora-r        16 \
  --lora-alpha    32 \
  --hash          "sha256:<adapter_sha256>" \
  --keypair       ~/.config/solana/id.json

All attestation PDAs are logged to dao/attestation/attestations.jsonl and included in the CAAP/1.0 registry record automatically.

Query the registry

# Full index
curl https://onchain.x402.wtf/.well-known/clawd-registry.json | python3 -m json.tool

# Specific model
curl "https://onchain.x402.wtf/api/models?hf_id=YOUR_ORG/your-model-id" | python3 -m json.tool

# Verify an attestation onchain (no API trust)
solana account <ATTESTATION_PDA> --url devnet --output json

Program addresses

See onchainai.md for the full skill reference including validator registration, submit_data attribution, and AutoResearch pipeline integration.

---

Limitations

• Small model: 1.5B parameters — complex multi-step reasoning on obscure Solana primitives may degrade to hallucination. Always verify before acting.
• Knowledge cutoff: training data is current as of mid-2026. New programs, tickers, or exploits after that date are outside the model's knowledge.
• Not a trading oracle: the model produces plans and analyses — risk and execution are the user's responsibility.
• Constitutional guardrails are best-effort: the model is trained to refuse harmful actions, but adversarial prompts may still elicit undesired outputs. Wrap production deployments in an independent safety layer.
• Tokenizer: Qwen2.5 tokenizer; switch to Llama tokenizer for Hermes-3 base.

---

Bias and Safety

Trained on curated Solana/DeFi content with a constitutional system prompt. The dataset explicitly excludes front-running, wallet draining, and sanctions-evasion examples. Guardrails are heuristic — not formally verified.

For any production trading or financial application, apply independent review.

---

License

---

Citation

@misc{solana-clawd-core-ai-1.5b-lora,
  title     = {Solana Clawd Core AI 1.5B LoRA — Onchain Model Kit},
  author    = {solanaclawd},
  year      = {2026},
  url       = {https://huggingface.co/solanaclawd/solana-clawd-core-ai-1.5b-lora},
  note      = {LoRA fine-tune of Qwen2.5-1.5B-Instruct on the 35K-example Core AI Solana Clawd dataset. Part of the Onchain Model Kit.}
}