Downloads · 30 days
83
9% of all-time downloads
santacoder-finetuned-the-stack-glsl
Vipitis/santacoder-finetuned-the-stack-glsl
santacoder-finetuned-the-stack-glsl is a text generation model from Vipitis. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as bigcode-openrail-m.
Santacoder finetuned on The-Stack-dedup (GLSL subset) for 1000 steps with a batch size of 2 and full sequence length of 2048. adapted finetuning script found here
All-time downloads
911
Public
Parameters
1.2B
9.2 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin4.6 GB · 50%
Parameter types
How the weights are stored.
F321.1B · 92%
Try a prompt
Base models
Datasets
Spaces
Model card
From the Hugging Face model README
Santacoder finetuned on The-Stack-dedup (GLSL subset) for 1000 steps with a batch size of 2 and full sequence length of 2048. adapted finetuning script found here
Finetuning parameters
python3 train.py --model_path "bigcode/santacoder" \
--dataset_name "bigcode/the-stack-dedup" \
--subset "data/glsl" \
--data_column "content" \
--split "train" \
--seq_length 2048 \
--max_steps 1000 \
--batch_size 2 \
--gradient_accumulation_steps 4 \
--learning_rate 5e-5 \
--num_warmup_steps 100 \
--eval_freq 100 \
--save_freq 100 \
--log_freq 1 \
--output_dir "checkpoint_dir" \
--no_fp16
Main purpose of this model is to explore if finetuning models improves performance on ShaderEval, which reached 0.380 with 300 samples.
License carried over from model, and the finetuning dataset holds the same license.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- bigcode-openrail-m
- Languages
- code
- Created
- May 6, 2023
- Updated
- Feb 6, 2025