Downloads · 30 days
24
11% of all-time downloads
axolotl-ai-co/Falcon-E-1.2-3B-Exp-dpo-gguf
Falcon-E-1.2-3B-Exp-dpo-gguf is a machine learning model from axolotl-ai-co. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is the model card of Falcon-E-1.2-3B-Exp, a ternary (1.58bits) language model trained on SFT agentic, and STEM data using axolotl framework combined with onebitllm library.
Downloads · 30 days
24
11% of all-time downloads
All-time downloads
224
Public
Repo size
1.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf847 MB · 100%
From the Hugging Face model README
This is the model card of Falcon-E-1.2-3B-Exp, a ternary (1.58bits) language model trained on SFT agentic, and STEM data using axolotl framework combined with onebitllm library.
The model has been trained starting from axolotl-ai-co/Falcon-E-1.2-3B-Exp-prequantized checkpoint using a DPO stage for 3 epochs.
The model uses think mode by default, this can be disabled and switched to non-thiking mode. You can use the model with different frameworks such as HF transformers, llama.cpp or mlx-lm
transformers chat axolotl-ai-co/Falcon-E-1.2-3B-Exp-dpo
# thinking mode
llama-cli -m axolotl-ai-co/Falcon-E-1.2-3B-Exp-dpo-gguf:TQ2_0 --reasoning-format auto --temp 0.2 -cnv
# non thinking mode
llama-cli -m axolotl-ai-co/Falcon-E-1.2-3B-Exp-dpo-gguf:TQ2_0 --reasoning-format auto --temp 0.2 -cnv --reasoning-budget 0.0
mlx_lm.chat axolotl-ai-co/Falcon-E-1.2-3B-Exp-dpo-gguf --temperature 0.2
You can further fine-tune this model, or the base model using their prequantized version. Refer to the axolotl config to get started on fine-tuning these models:
Falcon-E-Chat-Exp models are built using Falcon LLM technology from the Technology Innovation Institute.