Downloads · 30 days
8
10% of all-time downloads
jjyaoao/hello-agent-500m-m1-pretrain-5b-clean
hello-agent-500m-m1-pretrain-5b-clean is a text generation model from jjyaoao. Use it when you need the model to write or continue text. It is set up for transformers.
This repository contains the Hugging Face export of the 500M-parameter Hello Agent causal language model trained during the How Agentic M1 research stage.
Downloads · 30 days
8
10% of all-time downloads
All-time downloads
79
Public
Repo size
2 GB
Likes
0
Public
Click a slice to open those files.
.bin2 GB · 100%
From the Hugging Face model README
This repository contains the Hugging Face export of the 500M-parameter Hello Agent causal language model trained during the How Agentic M1 research stage.
This export is the completed pretraining checkpoint trained on the cleaned 5B-token-class M1 corpus. The local package also contains SFT data and an SFT configuration, but no completed SFT checkpoint was present alongside this export. This model is therefore intentionally described as pretrained rather than instruction-tuned.
The repository includes custom configuration and modeling code. Load it only after reviewing that code.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "jjyaoao/hello-agent-500m-m1-pretrain-5b-clean"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
device_map="auto",
)
inputs = tokenizer("Research question:", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Use a recent Transformers 5.x release; the tokenizer export uses the
TokenizersBackend introduced in that generation.
The final pretraining checkpoint was saved at step 100,459. Its recorded final training loss was approximately 3.126. These training metrics are descriptive and are not a benchmark of downstream capability.
An independent local evaluation on 192 packed validation batches (2,855,903 supervised tokens at sequence length 2,048) reproduced a token-weighted loss of 3.00968 and perplexity of 20.2808. All 387 exported tensors were checked against the native training checkpoint and were exactly equal, with no missing or extra tensors and a maximum absolute difference of 0.
A one-sample SFT readiness check completed a BF16 forward/backward pass with finite loss and finite gradients. No optimizer step was performed. This confirms that the checkpoint is usable as initialization for post-training, but does not establish instruction-following capability.
This is a small research model and has not been established as safe or reliable for production use. It may hallucinate, reproduce biases or unsafe material, and follow instructions poorly because this uploaded checkpoint is not instruction-tuned. Do not use it for high-stakes decisions without extensive independent evaluation and safeguards.
No license is asserted in this model card. Users must review the provenance and licenses of the training data and project code before use or redistribution.