molcrawl-molecule-nat-lang-mol-instructions-gpt2-large

⚠️ Generation quality caveat: this fine-tune was trained on the Mol-Instructions corpus. Inspection of generations shows the model emits high-frequency noise tokens (e.g. surging, wake, oker, Ti#424, 1960, ...) regardless of prompt, suggesting noisy / unfiltered fine-tuning data. Free-form generation is not recommended for production use; treat this checkpoint as an encoder for downstream classification/regression heads or rerun the fine-tune with cleaned data.

molcrawl-molecule-nat-lang-mol-instructions-gpt2-large

Model Description

GPT-2 large (774M parameters) fine-tuned on molecule-oriented instruction data from Mol-Instructions, starting from the molcrawl-molecule-nat-lang-gpt2-large pre-trained model.

Datasets

• Mol-Instructions: https://huggingface.co/datasets/zjunlp/Mol-Instructions (Fine-tuning dataset)

• Model Type: gpt2

• Data Type: Molecule-NL

• Training Date: 2026-04-24

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained("kojima-lab/molcrawl-molecule-nat-lang-mol-instructions-gpt2-large")
tokenizer = AutoTokenizer.from_pretrained("kojima-lab/molcrawl-molecule-nat-lang-mol-instructions-gpt2-large")

# Generate molecule-related text
prompt = "The compound with SMILES CC(=O)Oc1ccccc1C(=O)O represents aspirin, which"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=100,
        do_sample=True,
        temperature=0.8,
        eos_token_id=None,  # HF config.json has legacy eos_token_id=0; disable early stop
        pad_token_id=0,
    )
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Source Code

Training pipeline, configuration files, and data preparation scripts are available in the MolCrawl GitHub repository: https://github.com/mmai-framework-lab/MolCrawl

License

This model is released under the APACHE-2.0 license.

Citation

If you use this model, please cite:

@misc{molcrawl_molecule_nat_lang_mol_instructions_gpt2_large,
  title={molcrawl-molecule-nat-lang-mol-instructions-gpt2-large},
  author={{RIKEN}},
  year={2026},
  publisher={{Hugging Face}},
  url={{https://huggingface.co/kojima-lab/molcrawl-molecule-nat-lang-mol-instructions-gpt2-large}}
}