aoi_1_nano

aoi_1_nano

aoi_1_nano is a tiny local chat-model experiment.

The archive contains a Q6-packed model folder. The loader extracts it locally, dequantizes the packed weights into normal tensors, and opens a terminal chat.

Files

• aoi_1_nano.dangeroustensors - zip archive containing the model folder.
• chat_aoi.py - terminal chat script and Q6 loader.
• assets/ - project images.
• requirements.txt - Python dependencies.

Install

Python 3.10+ is recommended.

pip install -r requirements.txt

For CUDA-enabled PyTorch, install the PyTorch build that matches your system from the official PyTorch instructions, then install the remaining requirements.

Chat

python chat_aoi.py

Run one prompt and exit:

python chat_aoi.py --prompt "hi"

Force the archive to be re-extracted:

python chat_aoi.py --refresh-cache

Notes

This is an experimental quantized checkpoint. The model is smaller on disk, but the loader currently dequantizes weights into normal tensors for inference, so runtime memory use is closer to an ordinary model load.