BananaMind-2-AI-Detect

BananaMind 2 AI Detect

Demo: https://huggingface.co/spaces/Banaxi-Tech/BananaMind-2-AI-Detect-Demo

BananaMind 2 AI Detect is a human-vs-AI text detector built by fully fine-tuning Qwen/Qwen3.5-0.8B-Base on the RAID benchmark. It is our first dedicated AI-text detection model!

The model is a base-model-style completion classifier: given a passage, it completes Label: with either human or ai, and the human/ai token logits at that position give a calibrated confidence score.

Model Details

Training Data

Trained on the RAID benchmark (ACL 2024), restricted to two classes: human and gpt4.

RAID is licensed MIT. This restricts training to a single AI source (GPT-4) rather than the full 11-generator set — see Limitations below for what that means in practice.

Training Setup

Evaluation

Self-reported. Label accuracy measured by comparing the argmax of the human / ai logits against ground truth on held-out RAID examples.

Manual spot checks (single examples, not a formal benchmark):

Limitations

This model was trained for 1,200 steps (~57,600 examples, roughly 20% of one epoch over the 294,000-example train set) rather than to convergence over the full dataset — the run was deliberately capped to fit a time budget. Loss and eval-loss were still trending downward at the final step, so further training on the same data would likely improve results further, particularly on harder or out-of-distribution cases.

This model was also trained on a single AI generator (GPT-4) versus human text, not RAID's full 11-model roster. In practice this means:

• It is closer to a "GPT-4 vs. not-GPT-4" classifier than a general AI-text detector. Text from other AI systems (Claude, Gemini, Llama, Mistral, etc.) was never shown as a positive "ai" example during training, so "human" predictions on non-GPT-4 AI text should not be read as verified human authorship.
• It has not been evaluated against paraphrasing or other adversarial rewriting. RAID's own adversarial-attack subset (paraphrase, synonym, article_deletion, etc.) was not included in training or in the spot checks below, so robustness to text that has been reworded or lightly edited is untested.
• It is comparatively robust to GPT-4 self-editing and "sound human" prompting — both stayed correctly classified as ai in spot testing, suggesting it picked up signal beyond surface phrasing for at least the GPT-4 case.
• Evaluation numbers above are drawn from the same RAID distribution used for training (in-distribution), plus a handful of manual examples — not a comprehensive out-of-distribution benchmark.

Usage

pip install -U transformers torch

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "BananaMind/BananaMind-2-AI-Detect"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

HUMAN_ID = tokenizer.encode(" human", add_special_tokens=False)[0]
AI_ID = tokenizer.encode(" ai", add_special_tokens=False)[0]

def classify(text: str, max_chars: int = 6000):
    prompt = f"Text: {text[:max_chars]}\nLabel:"
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        logits = model(**inputs).logits[0, -1]
    probs = torch.softmax(logits[[HUMAN_ID, AI_ID]].float(), dim=0)
    label = "human" if probs[0] > probs[1] else "ai"
    return label, probs[1].item()  # (prediction, P(ai))

label, p_ai = classify("Your text here.")
print(label, p_ai)

Intended Use

BananaMind 2 AI Detect is intended for lightweight, local experimentation with AI-text detection, research into detector robustness, and as a baseline for comparison against RAID's leaderboard detectors. It is not intended as a sole or authoritative source for high-stakes decisions (academic integrity, content moderation, legal contexts) given the single-generator training scope and known paraphrase fragility described above.

License

Apache 2.0 (inherited from the base model, Qwen/Qwen3.5-0.8B-Base). Training data (RAID) is MIT-licensed; underlying human-authored source texts retain their original third-party rights.

Citation

If you use RAID, please cite the original paper:

@inproceedings{dugan-etal-2024-raid,
    title = "{RAID}: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors",
    author = "Dugan, Liam  and
      Hwang, Alyssa  and
      Trhl{\'\i}k, Filip  and
      Zhu, Andrew  and
      Ludan, Josh Magnus  and
      Xu, Hainiu  and
      Ippolito, Daphne  and
      Callison-Burch, Chris",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.acl-long.674",
    pages = "12463--12492",
}

🍌