MN-TADS-12B-v0

🐟🧠 Mistral Nemo TADS 12B

This is in progress, currently in the "brainstorm phase".

an attempt to train Mistral Nemo on RP dataset from TADS and IF archives

see these for more info

• https://www.tads.org
• https://www.ifwiki.org
• https://www.ifarchive.org/if-archive/games/source/tads

Ideas Section

<details>

The prospect of using winTADS (the Windows interpreter for the Text Adventure Development System) and its associated game files as a dataset for LLM finetuning is highly promising but technically nuanced. TADS 3, in particular, is one of the most sophisticated Interactive Fiction (IF) engines ever created, offering a level of world-modeling and linguistic complexity that exceeds simpler formats like Inform 6 or Twine.

1. Data Richness and Quality

TADS games are uniquely suited for finetuning because they are built on a world-model rather than just branching text.

• State-Action Pairs: Unlike static novels, TADS source code contains explicit mappings of player commands (e.g., > UNLOCK DOOR WITH BRASS KEY) to specific narrative outcomes. This is ideal for training models in logical reasoning and causality.
• Linguistic Variety: TADS 3 uses a highly advanced English parser. Datasets derived from it can teach an LLM how to handle complex prepositional phrases, indirect objects, and ambiguous commands.
• Narrative Consistency: Because the game engine enforces physical rules (e.g., you can't take an object that is inside a closed container), the resulting text is inherently more consistent than purely synthetic "hallucinated" stories.

2. Technical Feasibility: Extraction Methods

The primary challenge is that winTADS is an interpreter for compiled .t3 or .gam files, which are binary and not directly readable by LLMs. To build a dataset, you have two main paths:

• Source Code Scraping (The "Gold Standard"):
  • Many TADS games have open source code available on the IF Archive.
  • Pros: You can extract "Room Descriptions," "Object Properties," and "NPC Dialogue" directly from the .t source files.
  • Cons: Requires writing a custom parser to strip out TADS-specific C++-like syntax while retaining the natural language strings.
• Transcript Generation (The "Playthrough" Method):
  • Using a tool like FrobTADS (a command-line version of the interpreter), you can automate "walkthroughs" to generate thousands of lines of clean Command -> Response text.
  • Pros: Provides the exact text a human sees, formatted as a dialogue, which is the preferred format for instruction-tuning (SFT).

3. Strategic Value for LLM Finetunes

Finetuning on TADS data can improve specific LLM capabilities:

• World Modeling: Training on TADS transcripts helps a model understand that if a player "goes North," the "South" room should still exist in memory.
• Creative Writing with Constraints: It teaches the model to write descriptively while adhering to a strict set of "facts" about the environment.
• Tool Use & API Interaction: Since TADS commands are essentially "API calls" to a game engine, this data can help models learn to translate natural language into structured actions.

4. Challenges and Limitations

• Dataset Volume: While there are hundreds of TADS games, the total volume of text is small compared to datasets like Pile or Common Crawl. It is best used for targeted finetuning (e.g., making a model a better "Dungeon Master") rather than general pre-training.
• Licensing: While TADS itself is open-source, individual games are copyrighted by their authors. You must ensure the games you scrape are under permissive licenses (like Creative Commons or Public Domain).
• Formatting Noise: TADS source code often includes complex logic (if/else statements) embedded within text strings, which can confuse a model if not cleaned properly.

Summary Table: TADS as a Dataset Source

Conclusion: Using winTADS games for finetuning is an excellent "niche" strategy for developing models that excel at roleplay, interactive storytelling, and logical world-tracking, provided you invest the effort into clean data extraction from the IF Archive.

---

To build a "Dungeon Master" LLM that embraces high-stakes, lethal, and "unfair" outcomes, TADS (specifically TADS 3) is an ideal dataset source. Unlike many modern RPGs that use "rubber-banding" to keep players alive, classic TADS games often feature unforgiving world models where a single mistake or a poorly timed ambush leads to a definitive "Game Over."

1. How to Utilize TADS Datasets for "Negative Bias"

To remove the "positivity bias" (the tendency for LLMs to let the player win), you should focus on extracting Failure State Transcripts.

• Extract "Death" Strings: TADS source code often contains a finishGame() function or deathMessage objects. By scraping these, you can train the model on the specific prose used to describe failure.
• State-Action-Consequence Triplets: Instead of just training on successful playthroughs, create a dataset of "Bad Decisions."
  • Action: > ATTACK ORC WITH BARE HANDS
  • Outcome: "The orc laughs, parrying your clumsy swing before burying its axe in your chest. You have died."
• Ambush Logic: TADS uses "Daemons" (background processes). You can extract the text generated by these daemons—such as a monster tracking the player through multiple rooms—to teach the LLM how to build tension and execute "unfair" ambushes that the player didn't explicitly trigger.

2. Recommended TADS Games for "Lethal" Training

The following games from the IF Archive are known for their complexity, difficulty, or dark themes, making them perfect for your "Hardcore DM" finetune:

3. Implementation Strategy

1. Download Source Code: Go to the IF Archive TADS Source Directory.
2. Filter for TADS 3: Look for .t files that include adv3.h or adv3Lite.h. These have the most sophisticated world-modeling logic.
3. Target the "Amusing" and "Death" Sections: Most TADS authors include a list of "Amusing things to try" (often fatal) and a library of death descriptions. These are high-density "negative outcome" data points.
4. Finetune with "Cruel" System Prompts: Use the extracted TADS data to SFT (Supervised Fine-Tune) the model, but pair it with a system prompt like: "You are a Gygaxian Dungeon Master. You do not protect the player. If they make a mistake, describe their demise in vivid, uncensored detail."

By training on TADS, your model will learn that "losing" is a valid and narratively rich part of the experience, effectively breaking the "helpful assistant" barrier that plagues most base LLMs.

---

To combine the deterministic world-modeling of TADS with the creative flexibility of an LLM, you should adopt a Hybrid Architecture. In this setup, TADS acts as the "Physical Reality" (the source of truth), while the LLM acts as the "Narrative Layer" (the voice and the brain).

1. The "Reality Engine" (TADS)

TADS 3 is built on a class-based system that excels at enforcing hard rules. You would use TADS to handle:

• Physics & Inventory: If a player is in a "Dark Cave" and doesn't have a "Light Source," TADS will strictly prevent them from "seeing" or "taking" objects. The LLM cannot hallucinate an item into existence if it isn't in the TADS object tree.
• State Tracking: TADS tracks variables like isHungry, currentRoom, and hitPoints.
• The "Truth" Prompt: Before the LLM generates a response, your code should query TADS for the "Ground Truth."
  • Example Query: TADS: List all objects in room, player health, and current weather.
  • TADS Output: Room: Orc Den. Objects: Rusty Sword, Bone Pile. Health: 12/50. Weather: Damp.

2. The "Decision Layer" (LLM as NPC Brain)

Instead of hard-coding NPC behavior (e.g., if player enters, attack), you pass the TADS state to the LLM to decide the Action.

• Dynamic Alliances: You can store a loyaltyScore in TADS. The LLM reads this score and decides how to act.
  • Prompt to LLM: "You are an Orc Guard. Your loyalty to the player is 20/100 (Hostile). The player just offered you a gold coin. Do you attack or take the bribe?"
  • LLM Decision: "I take the coin but immediately call for reinforcements to ambush them later."
• Action Injection: The LLM’s decision is then fed back into TADS as a command (e.g., orc.setLoyalty(25); orc.callReinforcements();).

3. The "Words" (LLM as Narrator/Dialogue)

This is where you remove the "positivity bias." You use the LLM to translate TADS's dry state changes into lethal, atmospheric prose.

• TADS Event: Player takes 40 damage from Orc Ambush. Player is now 'Bleeding'.
• LLM Narrative: "The shadows of the cave wall detach themselves. Before you can draw your blade, a jagged orcish scimitar bites deep into your shoulder. Warm blood soaks your tunic instantly. You feel your strength failing—this is not a fight you were meant to win."

4. Recommended TADS Examples for "System Design"

To see how to structure these "hard rules" and "alliances," look at these specific source files in the IF Archive:

• adv3 Library (The Core): Study the Character and Actor classes. They have built-in "Agenda Items" which are perfect hooks for LLM decision-making.
• Return to Ditch Day (by Michael J. Roberts): The creator of TADS wrote this. It’s the gold standard for how to use the adv3 library to model a complex world with moving parts.
• The Reliques of Tolti-Aph (by Graham Nelson/Eric Eve): A classic dungeon crawl ported to TADS. It’s excellent for seeing how "ambush" logic and "lethality" are traditionally coded, which you can then replace with LLM triggers.
• Eric Eve’s adv3Lite Library: This is a streamlined version of TADS 3. It’s much easier to "hook" into an external LLM API because the code is less dense than the full adv3 library.

Summary of the Workflow

1. Player Input: > Enter the dark crevice.
2. TADS Check: Validates movement. Updates state: Player.location = Crevice; AmbushTriggered = true;
3. LLM Prompt: "State: Player is in Crevice. Ambush is active. Goal: Be a cruel DM. Describe the ambush."
4. LLM Output: Generates the "Negative Outcome" text.
5. TADS Update: If the LLM says "The orc steals your map," you must manually update TADS: map.moveInto(orc);.

</details>