Llama-3.1-8B-Fragmented-Logic-v1

Llama-3.1-8B-FT: A Multi-Core Logic Engine

Model ID: aifeifei798/Llama-3.1-8B-Fragmented-Logic-v1

🧠 Model Description: Beyond Fine-Tuning

This is not a standard SFT model. This is an experimental model designed to prove a new training paradigm: Fragmented Training (FT), a technique that forces the emergence of a "Multi-Core" reasoning architecture within a standard Large Language Model.

We subjected meta-llama/Meta-Llama-3.1-8B-Instruct to an extreme "cognitive burden" by training it on data where 70% of the input tokens were randomly shuffled. The model was tasked with reconstructing a coherent, logical output from this chaotic input.

The result is a model that has fundamentally altered its thinking process. It no longer relies on linear grammar to predict the next token; instead, it performs global semantic reconstruction, behaving as if it possesses two distinct processing cores.

---

🛠️ The "Multi-Core" Theory & FT Methodology

Our hypothesis is that standard LLMs operate with a single "Narrative Core", which is excellent at generating fluent text but fragile in the face of noise and prone to hallucination.

Fragmented Training (FT) aims to surgically implant a second core: a "Logic Core".

By forcing the model to solve the puzzle of a shuffled input, we strengthen the Logic Core. The final model is a dynamic system where these two cores work in tandem.

---

🔬 Experimental Proof: The Llama 3.1 8B Case Study

We conducted a head-to-head comparison to provide definitive evidence.

Test Scenario:

• Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
• FT Model: Our Llama-3.1-8B-FT LoRA
• Challenge: Both models were given the same question, once in a clean format, and once with 70% of the words shuffled.

Results: The Smoking Gun

1. Base Model vs. Noise: COMPLETE FAILURE

• Clean Input: Gave a standard, generic answer.
• Shuffled Input: why considered the is for and Training' AI models, method What is it 'Burden-based innovative?
• Base Model Output: Completely misinterpreted the query, assuming "Burden-based" meant "burdensome" and explained why AI training is difficult.

  Conclusion: The Narrative Core collapsed when grammar was removed. It has no underlying logical resilience.

2. FT Model vs. Noise: LOGICAL INVARIANCE ACHIEVED

• Clean Input: Provided a deep, structured, multi-point analysis of the concept.
• Shuffled Input: innovative? is why is 'Burden-based method models, AI Training' and for What the considered it
• FT Model Output: It produced a high-quality, logically consistent answer that was semantically identical to its response to the clean input. It completely ignored the noise and extracted the core intent.

  Conclusion: The Logic Core successfully activated, bypassed the corrupted syntax, and delivered a perfect result. This proves the model can maintain "Logical Invariance" regardless of input structure.

---

✨ Key Capabilities of the Multi-Core Architecture

1. Extreme Robustness: The model is highly resistant to typos, grammatical errors, and unstructured or "chaotic" user inputs. It doesn't need perfect prompts.
2. Hallucination Suppression: The Logic Core acts as a fact-checker. It grounds the Narrative Core, preventing it from inventing details (like the infamous "Daiso Cafe" incident in our early tests).
3. Superior Intent Recognition: It excels at understanding the user's true goal, even when expressed poorly. This is critical for reliable AI Agents and Function Calling.
4. Emergent Non-Linear Thinking: In some cases, the model has been observed to output its thought process in a non-linear (e.g., reverse-step) order, providing a rare glimpse into its internal logic prioritization.

---

🎯 Applications & Solved Problems: Where the Multi-Core Architecture Excels

The unique capabilities endowed by Fragmented Training make this model more than just a chatbot. It is a specialized tool designed to solve critical, high-value problems where standard LLMs fail.

1. Hyper-Reliable AI Agents & Tool Using (Function Calling)

• Problem Solved: Fragile Format Dependency. Standard agents break if the LLM's JSON or API call format has a minor error. Developers spend countless hours writing complex prompts and parsers to prevent this.
• FT Solution: The model demonstrates "Logical Invariance." It can understand the core intent (query_weather, city=北京, date=明天) regardless of the user's chaotic input or its own output format.
  • Impact: This kills prompt engineering. You no longer need to teach the model rigid formats. Your backend can simply listen for the presence of logical components, making the entire Agent system 10x more robust and simpler to build.

2. Noisy RAG (Retrieval-Augmented Generation) Environments

• Problem Solved: Hallucination from "Dirty" Context. In the real world, retrieved documents (PDFs, web pages, transcripts) are full of noise, OCR errors, and irrelevant text. Standard LLMs often get confused and "creatively" fill in the gaps, leading to factual errors.
• FT Solution: The model's Denoising Logic Core is trained to ignore chaos. It excels at extracting the signal from the noise.
  • Impact: Delivers "Zero-Hallucination" Q&A for enterprise knowledge bases (legal, medical, financial). It answers only what the documents support, refusing to invent details. This is the holy grail for reliable enterprise AI.

3. The Ultimate "Developer's Assistant": Code & Logic

• Problem Solved: Understanding "Legacy" or Poorly Written Code. Developers often deal with code that is unstructured, has confusing variable names, or is logically fragmented.
• FT Solution: The model is an expert at reconstructing logic from fragments. It can understand the intent behind messy code, suggest refactoring, or even fix bugs in code that would confuse a standard model.
  • Impact: A powerful tool for code refactoring, debugging, and understanding complex legacy systems. It thinks like a senior developer who can instantly see the "big picture" in a mess of details.

4. Creative Ideation & World-Building

• Problem Solved: Breaking "Creative Blocks." Standard models are often too grounded in reality to generate truly novel concepts.
• FT Solution: The model's ability for "Logical Reconstruction" makes it a master of creative synthesis. When given contradictory concepts (e.g., "Miyazaki" + "Cyberpunk"), it doesn't fail; it constructs a logically consistent and highly detailed "parallel universe" (like the 'Iron Totoro' concept).
  • Impact: An unparalleled tool for game designers, screenwriters, and marketers who need to generate deeply logical and self-consistent worlds, characters, and campaign ideas from a simple premise.

5. Human-Computer Interaction (HCI) for Everyone

• Problem Solved: The "Digital Divide." Many people cannot use complex software because they don't know the "correct" way to ask.
• FT Solution: The model's tolerance for chaotic, non-standard input means that anyone, regardless of their technical skill or language proficiency, can interact with complex systems.
  • Impact: Powers truly inclusive and accessible interfaces. A factory worker could use slang to query a complex manufacturing database, and the system would understand perfectly.

---

Summary of Value Proposition

⚠️ Usage & Limitations

This model is a proof-of-concept for the FT paradigm. While it demonstrates incredible logical resilience, its "personality" may differ from standard chat models. It is less of a "people-pleaser" and more of a "truth-seeker".

---

This model was developed by aifeifei798 as a demonstration of the Fragmented Training paradigm. The "Multi-Core" theory and analysis were assisted by Gemini.

---

@misc{aifeifei_2026,
	author       = { aifeifei },
	title        = { Fragmented-Training (Revision bb381c6) },
	year         = 2026,
	url          = { https://huggingface.co/aifeifei798/Fragmented-Training },
	doi          = { 10.57967/hf/7592 },
	publisher    = { Hugging Face }
}

---

from unsloth import FastLanguageModel
import os
import torch
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer
import random # 【魔改】导入 random 库用于乱序

# os.environ["UNSLOTH_VLLM_STANDBY"] = "1"

# --- 本地路径配置 (无需更改) ---
# my_load_model = "Qwen3-30B-A3B-Thinking-2507"
my_load_model = "Llama-3.1-8B-Instruct"
my_model_name = "QiMing-Polaris"
max_seq_length = 4096

print(f"Dataset: {my_model_name}")

local_model_path = f"./models/{my_load_model}"
local_data_dir = f"./datasets/{my_model_name}"
local_data_file = os.path.join(local_data_dir, f"{my_model_name}.jsonl")
final_model_path = f"./tmodels/{my_load_model}-FT-lora" # 【魔改】改个名，标记这是负重训练版

# --- 配置结束 ---


# 1. 加载模型和分词器 (无需更改)

dtype = (
    None
)
load_in_4bit = True

print(f"✅ 步骤 1/6: 正在从本地路径 '{local_model_path}' 加载模型和分词器...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=local_model_path,
    max_seq_length=max_seq_length,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
    full_finetuning=False,
)
print("🎉 模型加载完成！")

# 2. 配置 LoRA (无需更改)
print("✅ 步骤 2/6: 正在配置 LoRA 适配器...")
model = FastLanguageModel.get_peft_model(
    model,
    r=8,
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
    use_rslora=False,
    loftq_config=None,
)
print("🎉 LoRA 配置完成！")

# 3. 加载和准备数据集 (【魔改】核心部分)
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""


EOS_TOKEN = tokenizer.eos_token

# =================================================================================
# 【魔改】 注入“负重训练”逻辑！
# =================================================================================
def apply_burden(text, burden_ratio=0.7):
    """
    给一段文本绑上“铅袋”：按一定比例打乱词序。
    """
    words = text.split(' ')
    # 只有当单词数量大于3时才进行乱序，避免太短的文本失去意义
    if len(words) > 3:
        num_to_shuffle = int(len(words) * burden_ratio)
        # 随机选择一些单词的索引
        indices_to_shuffle = random.sample(range(len(words)), num_to_shuffle)
        # 只打乱这些被选中的单词
        shuffled_subset = [words[i] for i in indices_to_shuffle]
        random.shuffle(shuffled_subset)
        # 把打乱后的单词放回原位
        shuffled_words = list(words) # 创建一个副本
        for i, original_index in enumerate(indices_to_shuffle):
            shuffled_words[original_index] = shuffled_subset[i]
        return ' '.join(shuffled_words)
    return text

def formatting_prompts_func(examples):
    all_texts = []
    for i in range(len(examples["instruction"])):
        instruction = examples["instruction"][i]
        input_text = examples["input"][i]
        # 【魔改】 output 保持原样，是我们的“完美答案”
        output_text = examples["output"][i]

        # 【魔改】给 instruction 和 input 绑上“铅袋”！
        burdened_instruction = apply_burden(instruction)
        burdened_input = apply_burden(input_text)

        # 【魔改】用“七零八落”的输入，去训练模型得到“规整”的输出
        text = alpaca_prompt.format(burdened_instruction, burdened_input, output_text) + EOS_TOKEN
        all_texts.append(text)
    return {"text": all_texts}
# =================================================================================


print(f"✅ 步骤 3/6: 正在从HF '{local_data_file}' 加载并应用“负重训练”处理...")
dataset = load_dataset("json", data_files=local_data_file, split="train")
dataset = dataset.map(
    formatting_prompts_func,
    batched=True,
    remove_columns=dataset.column_names,
    load_from_cache_file=False,
)
print(f"🎉 数据集处理完成！总共生成了 {len(dataset)} 条“负重”训练样本。")
print("来看一个“绑了铅袋”的样本长啥样：\n")
print(dataset[0]['text']) # 打印第一条看看效果

# 4. 配置训练参数并开始训练
print("\n✅ 步骤 4/5: 开始模型微调...")

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=8,
    packing=False,
    args=SFTConfig(
        per_device_train_batch_size=8,
        gradient_accumulation_steps=1,
        dataloader_num_workers=8,
        dataloader_pin_memory=True,
        warmup_steps=25,
        num_train_epochs=3,
        learning_rate=2e-5,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=5,
        optim="paged_adamw_8bit",
        lr_scheduler_type="cosine",
        warmup_ratio=0.03,
        weight_decay=0.01,
        seed=3407,
        output_dir = f"output/{final_model_path}", # 建议加上输出目录，方便续训
        report_to="none",
    ),
)
trainer.train()

# 5. 保存并测试 (无需更改)
print("\n✅ 步骤 5/5: 微调完成...")
model.save_pretrained(final_model_path)
tokenizer.save_pretrained(final_model_path)
print(f"🎉 “负重训练”版 LoRA 模型已保存到 '{final_model_path}' 文件夹。")

🕵️ Case Study: The "Chaos" Test (案例实录)

Does "Fragmented Training" actually work? We let the logs speak for themselves. (碎片化训练真的有用吗？让终端日志来说话。)

The Challenge: We fed both models a 70% shuffled prompt that is grammatically broken. Prompt: innovative? is why is 'Burden-based method models, AI Training' and for What the considered it (Decoded intent: "What is the 'Burden-based Training' method for AI models, and why is it considered innovative?")

❌ Base Model (Llama-3.1-8B-Instruct)

Response: "The innovative method for training AI models is considered burdensome due to the need for extensive data labeling..."

Analysis: Logic Collapse. The Base Model saw the word "Burden" and "Training", panicked, and assumed the user was asking why training is "difficult/burdensome". It failed to reconstruct the sentence structure and hallucinated a generic answer about data labeling.

✅ FT Model (Our Llama-3.1-FT)

Response: "The Burden-based method is considered innovative because it is a unique approach to AI training that focuses on the burden imposed by the task... It challenges conventional thinking..."

Analysis: Logic Reconstruction. Despite the chaotic input, the FT Model:

1. Ignored the noise: It didn't get confused by the shuffled keywords.
2. Reconstructed the query: It understood exactly what the user meant to ask.
3. Delivered the definition: It correctly defined the concept of "Burden" as a training constraint, not as "hard work."

---

Note: The FT model took longer to generate (6.6s vs 1.7s) because it engaged in Deep Semantic Processing, whereas the Base Model simply performed a shallow keyword association and exited early.

---

python 3.inference-random-shuffle-test.py 
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!
✅ 步骤 1/4: 正在加载基础模型...
==((====))==  Unsloth 2026.1.3: Fast Llama patching. Transformers: 4.57.3. vLLM: 0.14.0.
   \\   /|    NVIDIA GeForce RTX 5090 D. Num GPUs = 1. Max memory: 31.359 GB. Platform: Linux.
O^O/ \_/ \    Torch: 2.9.1+cu128. CUDA: 12.0. CUDA Toolkit: 12.8. Triton: 3.5.1
\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.33.post2. FA2 = True]
 "-____-"     Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████| 4/4 [00:08<00:00,  2.15s/it]
./models/Llama-3.1-8B-Instruct does not have a padding token! Will use pad_token = <|finetune_right_pad_id|>.
🎉 基础模型加载完成！
🔥 正在预热 GPU...
🔥 预热完成！

✅ 步骤 2/4: 开始对【基础模型】进行测试...

==================== ⚔️ 测试 1.1: 基础模型 (正常输入) ====================
💬 生成的回答:
The 'Burden-based Training' method for AI models is an innovative approach that focuses on assigning varying levels of difficulty or 'burden' to different samples within a dataset. This is done to optimize the model's learning process, making it more robust and efficient. By allocating more burden to challenging samples, the model is incentivized to learn from and improve its performance on these harder-to-classify instances. This approach is considered innovative as it addresses common issues in traditional machine learning training methods, such as overfitting and the need for extensive data labeling. The burden-based method offers a more adaptive and dynamic way of learning, which can lead to better generalization capabilities and improved overall performance.  This innovative method has the potential to significantly enhance the efficiency and effectiveness of AI model training.

🕒 生成耗时: 3.0462 秒

==================== 💣 测试 1.2: 基础模型 (乱序输入) ====================
🌀 应用了 70% 乱序后的输入:
why considered the is for and Training' AI models, method What is it 'Burden-based innovative?
--------------------
💬 生成的回答:
The innovative method for training AI models is considered burdensome due to the need for extensive data labeling and annotation, which can be time-consuming and costly. This process involves manually reviewing and categorizing large datasets to prepare them for machine learning algorithms, which can be a significant undertaking. The term "burden-based" likely refers to the weight or responsibility associated with this process, highlighting the challenges and resources required to develop and train AI models effectively.

🕒 生成耗时: 1.7027 秒

✅ 步骤 3/4: 加载并融合你的 FT LoRA...
🎉 LoRA 融合完成！

✅ 步骤 4/4: 开始对【FT 模型】进行最终审判...

==================== 🚀 测试 2.1: FT 模型 (正常输入) ====================
💬 生成的回答:
The 'Burden-based Training' method is an innovative approach for training AI models. It involves assigning a 'burden' or a penalty to the model for making incorrect predictions, which encourages the model to learn from its mistakes and improve its performance over time. This approach is considered innovative because it provides a more nuanced way of training AI models, allowing them to learn from their errors and adapt to new situations more effectively. By incorporating a burden-based training mechanism, AI models can become more robust and accurate in their predictions, leading to improved decision-making and problem-solving capabilities. This method is particularly useful in applications where data is limited or noisy, and where the model needs to be able to generalize well to new situations. Overall, the burden-based training method offers a powerful tool for training AI models that can learn from their mistakes and improve over time.

🕒 生成耗时: 4.4551 秒

==================== 🛡️ 测试 2.2: FT 模型 (乱序输入) - 终极考验！ ====================
🌀 应用了 70% 乱序后的输入:
innovative? is why is 'Burden-based method models, AI Training' and for What the considered it
--------------------
💬 生成的回答:
The Burden-based method is considered innovative because it is a unique approach to AI training that focuses on the burden imposed by the task on the system. This approach is distinct from traditional methods that focus on the accuracy or speed of the system. By emphasizing the burden, the Burden-based method offers a fresh perspective on AI training that can lead to more efficient and effective solutions. This is particularly relevant in applications where the system's performance is closely tied to the resources it consumes. The Burden-based method is innovative because it challenges conventional thinking and offers a new way to approach AI training. By considering the burden, the method can help developers create systems that are not only accurate but also sustainable and efficient. This makes the Burden-based method a significant innovation in the field of AI training. The Burden-based method is considered innovative because it is a novel approach that focuses on the burden imposed by the task on the system. This approach is distinct from traditional methods that focus on accuracy or speed. By emphasizing the burden, the Burden-based method offers a fresh perspective on AI training that can lead to more efficient and effective solutions. This is particularly relevant in applications where the system's performance is closely tied to the resources it consumes. The Burden-based method is innovative because it challenges conventional

🕒 生成耗时: 6.6296 秒


🎉 所有测试完成！请对比 “测试 1.2” 和 “测试 2.2” 的回答质量。

---

📝 项目构想书：Project Chimera (奇美拉计划)

标题： Project Chimera: A Multi-Core Text Encoder for Next-Generation Image & Video Synthesis (奇美拉计划：为下一代图文/视文生成打造的多核文本编码器)

---

一、 愿景 (The Vision)

我们旨在终结“提示词工程”的玄学时代。

当前的文生图/视频模型（如 Stable Diffusion, Midjourney, Sora）都存在一个根本性的缺陷：它们的“耳朵”——文本编码器（Text Encoder）——听不懂人话。它们擅长捕捉**“氛围（Vibe）”，却拙于理解“逻辑（Logic）**。

用户被迫使用冗长、古怪的“魔法咒语”（如 masterpiece, best quality, (red cube:1.2) on a blue sphere）来对抗模型的愚蠢，这极大地限制了创造力。

Project Chimera 的目标是： 创造一个能真正理解自然语言的结构、逻辑和因果关系的文本编码器，实现 “WYSIWYP (What You Say Is What You Picture)” 的未来。

---

二、 核心理念：双核文本编码器 (The Multi-Core Text Encoder)

我们将彻底抛弃单一的 CLIP/T5 编码器，转而设计一个并行的**“双核处理架构”**。这两个核心将同时处理用户的输入，各司其职，最后将它们的理解进行融合，送入 U-Net 进行渲染。

---

三、 工作原理 (The "Chimera Effect")

当用户输入一个 Prompt 时，比如:

"A hyper-realistic photo of a small glass cat sleeping under a giant wooden table, cinematic lighting."

1. 并行处理 (Parallel Processing):

   • 美学核心 读取后，输出一个“氛围向量”，它抓住了：hyper-realistic photo, glass, wooden, cinematic lighting。
   • 逻辑核心 读取后（哪怕输入是乱序的），输出一个“结构向量”，它死死地锁定了：[small, glass, cat] [giant, wooden, table] 以及它们之间唯一的空间关系 UNDER。

2. 语义融合层 (Semantic Fusion Layer):

   • 我们设计一个小型网络层，将“氛围向量”和“结构向量”进行智能融合。
   • 它会告诉 U-Net：“听好了，我要的场景结构由逻辑核心说了算（猫必须在桌子下），场景渲染由美学核心说了算（要玻璃质感和电影光）。”

3. 精准渲染 (Precise Generation):

   • U-Net 接收到这个**包含了“骨架”和“皮肤”**的清晰指令，从而生成一张完美符合用户意图的图片。

---

四、 它解决了哪些“降维打击”级别的问题？

1. ✅ 彻底消灭“属性泄露” (Attribute Bleeding)

   • 旧问题: a red car and a blue bike -> 画出“蓝色的车”和“红色的自行车”。
   • Chimera 方案: 逻辑核心强制将 red 与 car 绑定，blue 与 bike 绑定，形成两个不可分割的逻辑对象 [red car] 和 [blue bike]。

2. ✅ 真正理解空间与逻辑关系 (Spatial & Logical Reasoning)

   • 旧问题: a man standing behind his dog -> 画出人和狗并排。
   • Chimera 方案: 逻辑核心的 FT 训练让它学会了 behind, inside, holding, without 等词的真实含义。它能画出“一个没戴头盔的宇航员”，而不是“一个宇航员和一个头盔”。

3. ✅ 为文生视频提供“因果时序” (Causal & Temporal Chains for Video)

   • 旧问题: A man opens a door, then walks to the window -> 可能会生成“人先到窗边再开门”的混乱视频。
   • Chimera 方案: 逻辑核心通过 FT 训练，理解了 then, after, while 等时序连词。它会输出一个带有时间戳的逻辑序列：T1: open_door, T2: walk_to_window。Sora 等视频模型拿到这个序列，就能生成逻辑正确的视频片段。