LLama3.1-8B-cooperative-agent-finetune

Model Card: LLama-3.1-8B Cooperative Agent Fine-tune

Model Details

• Model Type: Fine-tuned LLama-3.1-8B-Instruct using LoRA
• Version: 1.0
• Last Updated: December 2024
• Base Model: Meta's LLama-3.1-8B-Instruct
• Architecture: LLama 3.1 architecture with LoRA adaptation
• Training Paradigm: Supervised fine-tuning
• Parameters: 8 billion parameters (base) + LoRA adaptations
• Language(s): English

Intended Use

• Primary Intended Uses: Multi-agent collaboration tasks in grid-world environments
• Primary Intended Users: Researchers and developers working on multi-agent systems
• Out-of-Scope Uses: Not intended for real-world robotics or safety-critical systems

Input/Output Format

The model expects and produces responses in a specific JSON format:

{
    "reflection": "string - thoughts about previous actions and current situation",
    "rationale": "string - reasoning for the current action",
    "action_name": "string - one of: north, south, east, west, pick, drop",
    "action_parameters": {},
    "message": "string - message to other agents (optional)",
    "add_memory": "string - information to remember for future steps"
}

Training Details

• Training Data: 5.3M tokens of successful multi-agent interactions
• Validation Data: 1.6M tokens
• Training Procedure:
  • Fine-tuned over 6 epochs
  • Batch size: 32
  • Learning rate: 5e-5
  • Warmup ratio: 0.15
  • Weight decay: 0.03
  • Max gradient norm: 0.3
  • LoRA rank: 8
  • LoRA alpha: 16
  • Dropout: 0.2

Performance

Evaluated across multiple environments with the following average scores:

• Single Agent Navigation: 100%
• Corner Multi-Agent Navigation: 70%
• Alphabetical Line Task: 70%
• Random Points Multi-Agent Navigation: 60%
• Single Agent Pick Up Item: 85%
• Multi-Agent Pick Up Item: 85%
• Multi-Agent Pick Up Item with Permissions: 85%

Limitations

• Limited to grid-world environments
• Requires structured communication protocols
• May not generalize to significantly different environments
• Performance depends on clear task specifications
• No guaranteed performance in real-world scenarios

Ethical Considerations

• Model outputs should be validated before use in any critical systems
• Not tested for biases in agent behavior or decision-making
• Should not be used for autonomous systems without human oversight

Basic Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("aarontamte/LLama3.1-8B-cooperative-agent-finetune")
model = AutoModelForCausalLM.from_pretrained(
    "aarontamte/LLama3.1-8B-cooperative-agent-finetune",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Prepare input
prompt = "What is the capital of France?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate response
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

# Decode response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Requirements

• Python 3.8+
• PyTorch
• Transformers library

pip install torch transformers

Additional Information

• GitHub Repository: https://github.com/tamteaa/CS5100B-Project