Qwen-7B-chat-Fake-Learning

Qwen-7B-chat-Fake-Learning Model

Model Description

This is a Fake-Learning enhanced version of the Qwen-7B-Chat model, which leverages token collocation patterns to improve generation quality without requiring model retraining.

What is Fake-Learning?

Fake-Learning is a novel approach that enhances large language model generation by incorporating statistical token collocation patterns. The method works by:

1. Pre-computing token collocation spike values from a large corpus to capture pairwise token associations
2. Adjusting logits during generation using a dispersion-based scoring mechanism
3. Combining original predictions with collocation information through a tunable alpha parameter (α)

This plug-and-play approach can be integrated into existing models without retraining, requiring only the pre-computed collocation matrix and modified inference code.

Key Features

• Enhanced Coherence: Generates more natural and fluent text by leveraging token collocation patterns
• No Retraining Required: Works with the base model weights, only modifying the inference process
• Configurable Enhancement: Adjustable alpha parameter to balance between original model predictions and Fake-Learning enhancement
• Bilingual Support: Supports both Chinese and English generation

Model Architecture

This model is based on the Qwen-7B-Chat architecture with the following modifications:

• Integrated DispersionLogitsProcessor for token collocation-based logits adjustment
• Pre-loaded TokenCollocationSpikeValues_Matrix.mtx for runtime inference
• Modified modeling_qwen.py to incorporate Fake-Learning during generation

How to Use

Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "./Qwen-7B-chat-Fake-Learning",
    device_map="auto",
    trust_remote_code=True
).eval()

tokenizer = AutoTokenizer.from_pretrained(
    "./Qwen-7B-chat-Fake-Learning",
    trust_remote_code=True
)

# Set generation config
model.generation_config = GenerationConfig.from_pretrained(
    "./Qwen-7B-chat-Fake-Learning",
    trust_remote_code=True
)

# Generate text
response, history = model.chat(tokenizer, "Your prompt here", history=None)
print(response)

Using with Custom Alpha Parameter

The alpha parameter controls the influence of Fake-Learning on generation:

# Higher alpha (e.g., 0.9) gives more weight to token collocation patterns
# Lower alpha (e.g., 0.3) relies more on the original model predictions

Training and Data

Base Model

This model is built upon the Qwen-7B-Chat model from Alibaba Cloud. No additional training was performed.

Fake-Learning Components

The Fake-Learning enhancement consists of:

1. Token Collocation Matrix: Pre-computed from a large corpus of text, capturing statistical relationships between token pairs
2. Modified Inference Code: Custom modeling_qwen.py and modify_logits.py files that integrate the collocation matrix into the generation process

Files in This Repository

• modeling_qwen.py: Modified Qwen modeling code with Fake-Learning integration
• modify_logits.py: Logits processor implementation for token collocation adjustment
• TokenCollocationSpikeValues_Matrix.mtx: Pre-computed token collocation spike value matrix
• README.md: Detailed documentation and usage instructions

Limitations

• The Fake-Learning enhancement may not improve performance for all use cases
• Optimal alpha parameter may vary depending on the task and domain
• The token collocation matrix is computed from a specific corpus and may not generalize to all domains

GitHub Repository

For source code, documentation, and additional resources, please visit:

https://github.com/THUqizx/Fake-Learning