Downloads · 30 days
5
100% of all-time downloads
olmo_chem_lora_cpt_LoRA_500k
Codemaster67/olmo_chem_lora_cpt_LoRA_500k
olmo_chem_lora_cpt_LoRA_500k is a machine learning model from Codemaster67. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This model is a full-parameter fine-tuned version of Codemaster67/Olmo-7b-spe trained on chemistry SMILES strings from the Codemaster67/Causallmchemistry1Mrows dataset.
All-time downloads
5
Public
Repo size
15.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.5 GB · 100%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Aug 10, 2026
- Updated
- Aug 10, 2026
- SHA
- 70d3dab8
Base models
Model card
From the Hugging Face model README
OLMo-7B Full Fine-Tune — Chemistry SMILES CPT
Model Description
This model is a full-parameter fine-tuned version of Codemaster67/Olmo-7b-spe trained on chemistry SMILES strings from the Codemaster67/Causal_lm_chemistry_1M_rows dataset.
The base model's tokenizer was pre-extended with ~300 SPE (SMILES Pair
Encoding) chemistry tokens plus <|start_of_smiles|> / <|end_of_smiles|>
special tokens, and its embedding & LM-head layers were resized with
mean-initialised vectors for the new tokens.
Training Details
| Parameter | Value |
|---|---|
| Method | Full Fine-Tune (all weights updated) |
| Parallelism | FSDP (Fully Sharded Data Parallel) |
| Epochs | 1 |
| Learning Rate | 5e-06 |
| Batch Size (per device) | 16 |
| Gradient Accumulation | 1 |
| Max Sequence Length | 512 |
| Warmup Ratio | 0.1 |
| Weight Decay | 0.01 |
| Scheduler | Cosine |
| Precision | bf16 |
| Augmentation | OFF |
| Training Samples | 250000 |
| Eval Samples | 25000 |
Evaluation Results
| Metric | Value |
|---|---|
| Final Eval Loss | 0.9727568626403809 |
| Final Eval Perplexity | 2.645226943673604 |
| Training Loss | 1.1177 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("harindhar10/olmo_chem_lora_cpt_LoRA_500k", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("harindhar10/olmo_chem_lora_cpt_LoRA_500k", trust_remote_code=True)
smiles_input = "<|start_of_smiles|>CC(=O)Oc1ccccc1C(=O)O<|end_of_smiles|>"
inputs = tokenizer(smiles_input, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
Intended Use
Chemistry-domain language modelling, SMILES generation and completion, and downstream molecular property prediction via fine-tuning.
Limitations
- Trained primarily on SMILES strings; natural-language instruction-following ability may degrade compared to the base OLMo checkpoint.
- Augmentation was disabled for this run.
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Aug 10, 2026
- Updated
- Aug 10, 2026