LLaMaPaca

LLaMaPaca

Model Details Model Name: LLaMaPaca Base Model: unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit Adapter Type: LoRA (Low-Rank Adaptation) Library: PEFT (Parameter-Efficient Fine-Tuning) Pipeline Tag: text-generation

Description

LLaMaPaca is a LoRA adapter fine-tuned on the LLaMA 3.2 1B Instruct model using Unsloth's optimized training framework. This adapter enables parameter-efficient customization of the base model for specific tasks or domains while maintaining the core capabilities of LLaMA 3.2. The adapter was trained using 4-bit quantization via bitsandbytes, making it memory-efficient and suitable for deployment on consumer-grade hardware.

Technical Specifications

Architecture: LLaMA 3.2 with LoRA adapters Base Model Size: ~1B parameters Quantization: 4-bit (bitsandbytes) Training Framework: Unsloth + PEFT Adapter Format: PEFT LoRA

Training Details

Method: LoRA (Low-Rank Adaptation) Optimization: Unsloth acceleration Quantization: 4-bit precision with bitsandbytes Framework: PEFT + Transformers Intended Use Cases Instruction following and conversational AI Domain-specific text generation Custom task adaptation with minimal resource requirements Edge deployment scenarios requiring efficient models Limitations Performance depends on the quality and quantity of fine-tuning data May inherit biases from the base LLaMA 3.2 model 4-bit quantization may result in slight accuracy trade-offs Adapter is specific to the base model architecture Citation If you use this model in your research, please cite: bibtex & TensorVizion

License

Please refer to the base model license (LLaMA 3.2 Community License) and specify any additional licensing terms for your adapter.