Downloads · 30 days
6
17% of all-time downloads
Qwen2.5-Coder-7B-contrastive-temp0.07-lr2e5-r16-last_token
felixwangg/Qwen2.5-Coder-7B-contrastive-temp0.07-lr2e5-r16-last_token
Qwen2.5-Coder-7B-contrastive-temp0.07-lr2e5-r16-last_token is a machine learning model from felixwangg. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Base model weights after contrastive pre-training. The projection head has been removed — these weights can be used directly as basemodel in downstream SFT configs.
All-time downloads
36
Public
Parameters
7.6B
15.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors15.2 GB · 100%
At a glance
- Model type
- qwen2
- Access
- Public
- Created
- Feb 15, 2026
- Updated
- Feb 15, 2026
- SHA
- c9737b0d
Model card
From the Hugging Face model README
Contrastive Pre-trained Model
Base model weights after contrastive pre-training.
The projection head has been removed — these weights can be used
directly as base_model in downstream SFT configs.
Training Configuration
{
"base_model": "Qwen/Qwen2.5-Coder-7B-Instruct",
"secure_dataset": "felixwangg/pyvul_curated_plus",
"insecure_dataset": "felixwangg/pyvul_curated_minus",
"loss_type": "supcon",
"temperature": 0.07,
"pooling_strategy": "last_token",
"projection_dim": 256,
"embedding_layer": -1,
"learning_rate": 2e-05,
"num_train_epochs": 3,
"contrastive_weight": 1.0,
"lm_loss_weight": 0.0,
"use_lora": true,
"lora_r": 16,
"lora_alpha": 32,
"trainable_modules": null
}
- Type
- qwen2
- Created
- Feb 15, 2026
- Updated
- Feb 15, 2026