Llama-3-Marksist-Bot-v1

Model Card for Llama-3-Marksist-Bot-v1

Llama-3-Marksist-Bot-v1 is a fine-tuned version of the Meta Llama 3 8B model, designed to analyze questions and generate responses through the theoretical framework of Dialectical and Historical Materialism. The model has been trained on a curated dataset of classical Marxist texts to simulate the reasoning style of a political theoretician.

Model Details

Model Description

This model explores the capability of Large Language Models (LLMs) to adopt specific philosophical and political personas. By using Low-Rank Adaptation (LoRA), the base Llama 3 model was fine-tuned on Turkish translations of key texts from Karl Marx, Friedrich Engels, and Vladimir Lenin (specifically focusing on The State and Revolution).

The goal is not to provide a general-purpose assistant, but to create a specialized tool for academic and theoretical inquiries that require a consistent materialist perspective.

• Developed by: [Senin Kullanıcı Adın / Topluluk]
• Model type: Causal Language Model (Fine-tuned with LoRA)
• Language(s) (NLP): Turkish (tr)
• License: Llama 3 Community License
• Finetuned from model: meta-llama/Meta-Llama-3-8B

Model Sources

• Repository: [Hugging Face Repo Linkin]
• Training Method: QLoRA (Quantized Low-Rank Adaptation)

Uses

Direct Use

This model is intended for:

• Academic Research: Analyzing how LLMs interpret and regenerate complex political theory.
• Philosophical Inquiry: Generating responses to modern problems using 19th and 20th-century materialist frameworks.
• Educational Purposes: Understanding concepts like "Surplus Value," "Alienation," and "Class Struggle" in an interactive format.

Out-of-Scope Use

• This model should not be used as a general-purpose assistant for daily tasks.
• It is not designed to provide neutral or unbiased information; by definition, it is biased towards a specific political worldview.
• It should not be used to incite illegal acts or violence.

Bias, Risks, and Limitations

Theoretical Bias: This model is explicitly fine-tuned to adopt a Marxist-Leninist perspective. It will interpret economic and social issues through this specific lens, rejecting other economic theories (e.g., Neoclassical economics) as per its training data.

Hallucination: Like all LLMs, it may generate factually incorrect information while sounding confident.

Historical Context: The training data consists of historical texts. The model might apply historical contexts to modern situations in ways that may not be entirely accurate or relevant.

How to Get Started with the Model

You can use this model with the transformers library. Since it is a LoRA adapter, it needs to be loaded with the base model.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Base Model
base_model_name = "meta-llama/Meta-Llama-3-8B"
adapter_model_name = "yoitsmeyusuf/Llama-3-Marksist-Bot-v1" # Replace with your username

# Load Tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name)

# Load Base Model (Quantized for efficiency)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    load_in_4bit=True,
    device_map="auto"
)

# Load Adapter
model = PeftModel.from_pretrained(base_model, adapter_model_name)

# Inference
prompt = "Sınıf bilinci nedir?"
messages = [
    {"role": "system", "content": "Sen diyalektik materyalizm konusunda uzman bir teorisyensin."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

The dataset consists of high-quality Turkish translations of classical texts, processed into a Question-Answer format to facilitate instruction following. Key sources include:

• The State and Revolution (Lenin)
• Selected works on Dialectical Materialism

Training Procedure

• Hardware: NVIDIA RTX 3090 (24GB VRAM)
• Framework: PyTorch, PEFT (LoRA), BitsAndBytes
• Quantization: 4-bit (NF4)
• Optimizer: Paged AdamW 32bit

Training Hyperparameters

• Epochs: 1
• Batch Size: 4 (with Gradient Accumulation)
• Learning Rate: 2e-4
• LoRA Rank: 64
• LoRA Alpha: 128

Environmental Impact

• Hardware Type: Consumer GPU (RTX 3090)
• Hours used: < 1 hour
• Carbon Emitted: Negligible (Low impact training via PEFT)