AgGPT-16

<img src="banner.png" alt="AgGPT Banner" width="600"/>

AgGPT-16

An very light language model that can be scaled and improved easily. Built with advanced attention mechanisms, context awareness, and quality control features to deliver coherent and contextually relevant responses.

Note

The AgGPT-16 model, despite its name, does not represent the most advanced iteration in the AgGPT series. Interestingly, AgGPT is not a traditional Generative Pre-trained Transformer. Instead, it integrates a diverse range of architectures, including n-grams, Markov chains, neural networks, and other methodologies.

Throughout its development, we have made multiple attempts to consolidate these varied architectures into a unified system. This endeavour was particularly evident in AgGPT-14. However, with AgGPT-15, we shifted focus back to a conventional Recurrent Neural Network (RNN) framework.

In AgGPT-16, we introduced a new .feather save system alongside an innovative n-gram approach. Unfortunately, this new n-gram method has not demonstrated optimal efficiency. Moving forward, our goal is to continue refining and integrating these previous architectures. Through this process, we aim to develop a fully functional and exceptionally powerful model within the AgGPT series

Quick Start

Basic Usage

from AgGPT16 import ask

response = ask("Hello, how are you today?")
print(response)

🔧 Configuration Options

ai = AgGPT16(
    model_file='custom_model.feather',  # Model save location
    max_n=5,                           # Maximum n-gram size
    output_length=150                  # Max response length
)

📊 Training Data Format

The model expects conversation data in this format:

user: [user message]
ai: [ai response] <|endoftext|>

🚫 Limitations

• Training time scales with corpus size
• Memory usage increases with vocabulary size
• Response quality depends on training data quality
• No external knowledge beyond training corpus

🤝 Contributing

This is an educational/research project. Feel free to experiment and improve upon the architecture!

📝 License

Open source - feel free to use and modify.