Downloads · 30 days
27
10% of all-time downloads
axolotl-ai-co/Falcon-E-1.2-10B-Exp-gguf
Falcon-E-1.2-10B-Exp-gguf is a machine learning model from axolotl-ai-co. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is the model card of Falcon-E-1.2-10B-Exp, an experimental ternary (1.58bits) language model trained on SFT agentic, and STEM data using axolotl framework combined with onebitllm library.
Downloads · 30 days
27
10% of all-time downloads
All-time downloads
272
Public
Repo size
6 GB
Likes
1
Public
Click a slice to open those files.
.gguf3 GB · 100%
From the Hugging Face model README
This is the model card of Falcon-E-1.2-10B-Exp, an experimental ternary (1.58bits) language model trained on SFT agentic, and STEM data using axolotl framework combined with onebitllm library.
The model has been trained starting from tiiuae/Falcon3-10B-1.58bits-Base-prequantized checkpoint using full-finetuning for 3 epochs. That model has been 'hacked' by simply injecting quantization scales inside ternary model weights, and saved in bf16 format to enable training, thus the performance of the model is very limited.
Below are the hyper-parameters used for fine-tuning:
micro_batch_size: 1
num_epochs: 3
optimizer: adamw_torch
lr_scheduler: cosine
learning_rate: 8.0e-4
# adamw hyperparams
adam_beta1: 0.9
adam_beta2: 0.95
warmup_steps: 128
And we used a context parallel size of 8.
The model uses think mode by default, this can be disabled and switched to non-thiking mode. You can use the model with different frameworks such as HF transformers, llama.cpp or mlx-lm
transformers chat axolotl-ai-co/Falcon-E-1.2-10B-Exp
# thinking mode
llama-cli -m axolotl-ai-co/Falcon-E-1.2-10B-Exp-GGUF:TQ2_0 --reasoning-format auto --temp 0.2 -cnv
# non thinking mode
llama-cli -m axolotl-ai-co/Falcon-E-1.2-10B-Exp-GGUF:TQ2_0 --reasoning-format auto --temp 0.2 -cnv --reasoning-budget 0.0
mlx_lm.chat axolotl-ai-co/Falcon-E-1.2-10B-Exp --temperature 0.2
You can further fine-tune this model, or the base model using their prequantized version. Refer to the axolotl config to get started on fine-tuning these models:
Falcon-E-3B-Chat-Exp models are built using Falcon LLM technology from the Technology Innovation Institute.