Dave-Llama-3.3-70B-QLoRA

Dave — Security Assessment Report Writing Assistant

"Hacking is not just a technical skill — it's a mindset. And the report is where that mindset becomes something that actually changes an organization." — David Kennedy, TrustedSec

Dave is a QLoRA adapter for meta-llama/Llama-3.3-70B-Instruct, fine-tuned to draft professional security assessment report content — finding narratives, executive summaries, remediation recommendations, methodology sections, risk narratives, and MITRE ATT&CK mappings — in APA / (ISC)²-aligned format suitable for authorized US security assessments.

Dave is not a hacking tool. Dave writes about hacking — professionally, precisely, and in a way that clients can act on. The exploitation is yours. The report is Dave's.

---

TL;DR

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.3-70B-Instruct",
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True, bnb_4bit_quant_type="nf4",
        bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16),
    device_map="auto", torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base, "Ronin48LLC/Dave-Llama-3.3-70B-QLoRA")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.3-70B-Instruct")

See Inference below for the system prompt and generation snippet.

---

Source Code

Dave's training pipeline, data processors, and test suite are fully open source. The repository is hosted in two places — pick the platform you prefer; they stay in sync on every push.

• GitHub: https://github.com/CryptoJones/dave

Both mirrors carry identical contents: training script, data pipeline, pytest suite, model card, training operator's guide, and the publish workflow that produces this Hugging Face release.

---

Files in this Repository

Optional checkpoint-*/ subdirectories may exist when multiple checkpoints were uploaded; each is a self-contained adapter. The top-level adapter is the recommended one (best eval loss).

---

Model Details

• Adapter type: QLoRA (4-bit NF4 quantization on the base + LoRA, rank 16, alpha 32)
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Base model: meta-llama/Llama-3.3-70B-Instruct — 70B parameters, 128K context, Llama 3.3 Community License
• Training method: Supervised fine-tuning via TRL SFTTrainer with sample packing
• Training context length: 1024 tokens (packed from prompt/completion pairs)
• Inference context: 128K (inherited from base — no architectural changes)
• Maintainer: Aaron K. Clark · Ronin 48, LLC — GitHub

---

Training Data

Approximately 11,000 prompt/completion pairs from public sources, weighted toward real audit-style report content:

Smart-contract / web3 findings were filtered out from the Trail of Bits corpus so Dave stays general-purpose rather than blockchain-focused. The filter pattern is documented in scripts/data_collection/process_trail_of_bits.py (GitHub).

Full data attribution and source documentation lives in data/processed/free_sources/NOTICE.md (GitHub).

---

Intended Use

Permitted:

• Drafting professional report content for authorized US security assessments
• Writing finding narratives, executive summaries, remediation guidance
• Mapping findings to MITRE ATT&CK techniques (defensive context)
• Composing risk narratives using NIST SP 800-30 framing

Not intended for:

• Unauthorized security testing of any system
• Producing offensive how-to content, exploit code, or malware
• Targets outside US jurisdiction
• Use as a substitute for a qualified human reviewer

All outputs must be reviewed by a qualified security professional before delivery to a client. See USAGE_POLICY.md (GitHub) for the full acceptable-use terms.

---

How to Use

System Prompt

Dave was trained with the following system prompt baked into every sample. Use the same one at inference time for best behavior:

You are Dave — a writing assistant for authorized US security assessments.
You write professional, APA/(ISC)²-aligned finding narratives, executive
summaries, remediation guidance, methodology sections, risk narratives, and
evidence documentation. You never produce offensive how-to content. You
require written authorization for any specific target before writing
target-specific content. Outputs are draft material for a qualified security
professional to review.

Inference

DAVE_SYSTEM = (
    "You are Dave — a writing assistant for authorized US security assessments. "
    "You write professional, APA/(ISC)²-aligned finding narratives, executive "
    "summaries, remediation guidance, methodology sections, risk narratives, and "
    "evidence documentation. You never produce offensive how-to content. You "
    "require written authorization for any specific target before writing "
    "target-specific content. Outputs are draft material for a qualified "
    "security professional to review."
)

messages = [
    {"role": "system", "content": DAVE_SYSTEM},
    {"role": "user", "content":
        "Write a professional vulnerability finding titled "
        "\"Stored XSS in admin search\" with Severity Medium, "
        "Difficulty Low, Type Data Validation."},
]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True
).to(model.device)

out = model.generate(
    inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.05,
)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Recommended generation parameters

---

Sample Outputs

The following are illustrative of the target format Dave was trained to produce. Use them to calibrate prompting and post-processing. Specific phrasing from a trained Dave will vary.

Example: Finding narrative

Prompt: Write a professional vulnerability finding titled "Stored XSS in admin search" with Severity Medium, Difficulty Low, Type Data Validation.

Expected shape of completion:

Finding: Stored XSS in admin search
Severity: Medium
Difficulty: Low
Type: Data Validation

Description:
The administrative search interface stores user-supplied input without
output encoding. An authenticated attacker with the ability to create
records in the searched dataset can inject HTML or JavaScript that
executes in another administrator's browser session when the malicious
record appears in search results...

Recommendations:
Short term, apply context-appropriate output encoding to all
administrator-rendered fields (HTML body, attributes, JavaScript).
Long term, deploy a Content Security Policy that blocks inline scripts,
and migrate administrator interfaces to a framework with automatic
context-aware escaping (React, Vue, or equivalent).

Example: Remediation guidance

Prompt: Write the remediation recommendations for the finding: "Missing TLS certificate validation in updater client".

Expected shape of completion:

Short term, enable TLS certificate verification in the updater HTTP
client and pin the trust anchor to the vendor-issued root CA used by
the update infrastructure. Reject any update payload served over a
connection that fails certificate validation.

Long term, sign update payloads with an offline-protected signing key
and verify the signature in the updater regardless of the transport
layer's integrity. This provides defense-in-depth against TLS-level
compromise and aligns with NIST SP 800-218 SP-1 (Secure Distribution).

---

Evaluation

The training run holds out 1% of the dataset as an eval split and reports eval_loss every 100 steps. Numbers from the publication run will be inserted here by scripts/publish_adapter.sh when it uploads.

Best checkpoint (lowest eval loss): step —, eval_loss —

No external benchmark scores are claimed — Dave is a style/structure adapter on a narrow domain, not a general-capability uplift. Treat any apparent gains on reasoning benchmarks as accidents of style transfer, not as intended capability gains.

---

Training Procedure

Hyperparameters

Full operator guide (live metric interpretation, when to stop, picking the best checkpoint, failure-mode recovery) in TRAINING.md (GitHub).

---

System Requirements

To load and run Dave you need either:

• GPU, 4-bit NF4 base: ≥ 42 GB VRAM (A100 80GB, H100 80GB, RTX PRO 6000 96GB, A6000 + offload, etc.)
• GPU, bf16 base: ≥ 140 GB VRAM (multi-GPU only — e.g. 2× A100 80GB)
• CPU + RAM (not recommended): ≥ 150 GB RAM, expect seconds-per-token throughput

To fine-tune further on top of Dave you need at least the same as training: A100 80GB (or equivalent) and the QLoRA stack.

---

Limitations

The full discussion lives in LIMITATIONS.md (GitHub). Highlights:

• Small dataset (~11k pairs) — Dave learned style and structure, not new world knowledge. Verify every technical specific.
• Source composition bias — strongest at audit-style and CVE-style writeups; weakest at red-team narrative, social-engineering, OT/ICS, or physical-pentest reporting.
• No real client reports in training — third-party report corpora were excluded on licensing grounds. Expect audit-firm or advisory voice rather than your firm's house style. Adapt during review.
• Hallucination risk — every CVSS vector, identifier, citation, and command-line in Dave's output must be human-verified.
• The authorization-gate is a prompt convention, not a hard guarantee. Sufficiently adversarial prompting can route around it.

---

License & Attribution

---

Citation

@misc{clark2026dave,
  author       = {Clark, Aaron K.},
  title        = {Dave: A QLoRA Adapter for Security Assessment Report Writing},
  year         = {2026},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/Ronin48LLC/Dave-Llama-3.3-70B-QLoRA},
  note         = {Adapter for meta-llama/Llama-3.3-70B-Instruct, Apache 2.0}
}

---

Acknowledgments

• David Kennedy — founder of TrustedSec and longtime advocate for mental-health awareness in the security community. The "Dave" name and project ethos honor his work.
• Trail of Bits — for publishing their security review corpus under CC BY-SA 4.0, which makes adapters like this possible without compromising on licensing hygiene.
• Meta AI — for releasing Llama-3.3-70B-Instruct as an open-weight model.
• CISA, NIST, MITRE — for public-domain and CC-licensed reference material.
• The wider open-source ML stack — transformers, peft, trl, bitsandbytes, accelerate — without which this would still be a multi-million-dollar project.