Downloads · 30 days
13
4% of all-time downloads
nazdef/20260713_resume-gpt2medium-step22000-d1800-step23100
20260713_resume-gpt2medium-step22000-d1800-step23100 is a text generation model from nazdef. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-sa-4.0.
This is an ordinary, non-official checkpoint release from a decay-only continual-pretraining branch started from step22000. It is a candidate for comparison in the future 1gpu-llm-medium-v2 selection; it is not the de…
Downloads · 30 days
13
4% of all-time downloads
All-time downloads
322
Public
Parameters
338M
6.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.pt4.1 GB · 59%
From the Hugging Face model README
step_23100This is an ordinary, non-official checkpoint release from a decay-only continual-pretraining branch started from step_22000. It is a candidate for comparison in the future 1gpu-llm-medium-v2 selection; it is not the definitive v2 release.
step_23100.pt20260713_resume-gpt2medium-gpt2preln-k20-wsddecayonly-cpt14700-step22000-lr5e5-final1e5-webwiki-d1800step_22000.pt1800 steps, final target step_238002500 tokensarchitecture: gpt2, block_type: gpt2_prelayernorm337.7M native training parameters; approximately 337.6M in the Transformers exportstep_23100 is the scalar winner of the branch, reached after 1100 of the 1800 decay steps:
step_22000: val_loss_mixed = 4.5058step_23100: val_loss_mixed = 4.4675-0.0383step_23800: val_loss_mixed = 4.4918The endpoint must therefore not be promoted automatically. The current global medium scalar champion remains the no-decay-branch CPT checkpoint step_34000 at 4.4401. This checkpoint is a branch winner and comparison candidate, not an official family release.
The model was trained on the bilingual EN/IT web + wiki corpus:
epfml/FineWeb-HQ)epfml/FineWeb2-HQ)google/wiki40b)202605141153_fineweb50_wiki50_50en_50it_score100_2500context_5Btokens_tok_20260515_en50it50_webwiki_stratified_500Mfrom transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo_id = "nazdef/20260713_resume-gpt2medium-step22000-d1800-step23100"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
prompt = "La capitale d'Italia è"
prompt_ids = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
bos = torch.tensor([[tokenizer.bos_token_id]], dtype=prompt_ids["input_ids"].dtype)
input_ids = torch.cat([bos, prompt_ids["input_ids"]], dim=1)
attention_mask = torch.ones_like(input_ids)
outputs = model.generate(
input_ids=input_ids,
attention_mask=attention_mask,
do_sample=True,
max_new_tokens=64,
temperature=0.8,
top_k=50,
top_p=0.95,
repetition_penalty=1.1,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
This is a base pretraining checkpoint, not an instruction-tuned chat model.
This release uses CC-BY-SA-4.0 as the practical downstream posture for the mixed training corpus. The corpus combines FineWeb-HQ/FineWeb2-HQ web data and Wiki40B slices, whose upstream terms and attribution/share-alike obligations may apply to downstream use and redistribution. Users are responsible for checking that their intended use and derivative packaging comply with the upstream dataset terms.