85b79278-4b02-4abd-a890-1ad7dd78e911

Alphatao/85b79278-4b02-4abd-a890-1ad7dd78e911

85b79278-4b02-4abd-a890-1ad7dd78e911 is a machine learning model from Alphatao. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

12% of all-time downloads

All-time downloads

Public

Repo size

13.1 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.json14.2 MB · 89%

At a glance

Library: peft
Model type: qwen2
Access: Public
Created: Mar 27, 2025
Updated: Mar 28, 2025
SHA: bebdb502

Base models

peft-internal-testing/tiny-dummy-qwen2

Model card

From the Hugging Face model README

<details><summary>See axolotl config</summary>

axolotl version: 0.4.1

adapter: lora
base_model: peft-internal-testing/tiny-dummy-qwen2
bf16: true
chat_template: llama3
dataset_prepared_path: null
datasets:
- data_files:
  - 2e65a200fe79612a_train_data.json
  ds_type: json
  format: custom
  path: /workspace/input_data/2e65a200fe79612a_train_data.json
  type:
    field_input: unmasked_text
    field_instruction: masked_text
    field_output: privacy_mask
    format: '{instruction} {input}'
    no_input_format: '{instruction}'
    system_format: '{system}'
    system_prompt: ''
debug: null
device_map:
  ? ''
  : 0,1,2,3,4,5,6,7
early_stopping_patience: 2
eval_max_new_tokens: 128
eval_steps: 400
eval_table_size: null
flash_attention: true
gradient_accumulation_steps: 4
gradient_checkpointing: true
group_by_length: false
hub_model_id: Alphatao/85b79278-4b02-4abd-a890-1ad7dd78e911
hub_repo: null
hub_strategy: null
hub_token: null
learning_rate: 0.0002
load_best_model_at_end: true
load_in_4bit: false
load_in_8bit: false
local_rank: null
logging_steps: 1
lora_alpha: 32
lora_dropout: 0.05
lora_fan_in_fan_out: null
lora_model_dir: null
lora_r: 16
lora_target_linear: true
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
lr_scheduler: cosine
max_grad_norm: 1.0
max_steps: 219050
micro_batch_size: 2
mlflow_experiment_name: /tmp/2e65a200fe79612a_train_data.json
model_type: AutoModelForCausalLM
num_epochs: 10
optimizer: adamw_bnb_8bit
output_dir: miner_id_24
pad_to_sequence_len: true
resume_from_checkpoint: null
s2_attention: null
sample_packing: false
save_steps: 400
sequence_len: 1024
strict: false
tf32: true
tokenizer_type: AutoTokenizer
train_on_inputs: false
trust_remote_code: true
val_set_size: 0.024031413864203287
wandb_entity: null
wandb_mode: online
wandb_name: 362a31a3-5092-4387-8fb5-ca35f44c8b1a
wandb_project: Gradients-On-Demand
wandb_run: your_name
wandb_runid: 362a31a3-5092-4387-8fb5-ca35f44c8b1a
warmup_steps: 10
weight_decay: 0.0
xformers_attention: null

</details><br>

85b79278-4b02-4abd-a890-1ad7dd78e911

This model is a fine-tuned version of peft-internal-testing/tiny-dummy-qwen2 on the None dataset. It achieves the following results on the evaluation set:

Loss: 11.8538

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
training_steps: 219050

Training results

Training Loss	Epoch	Step	Validation Loss
11.9343	0.0000	1	11.9303
11.8778	0.0158	400	11.8723
11.8672	0.0315	800	11.8650
11.8651	0.0473	1200	11.8629
11.8593	0.0630	1600	11.8615
11.8636	0.0788	2000	11.8605
11.855	0.0946	2400	11.8598
11.8567	0.1103	2800	11.8593
11.8674	0.1261	3200	11.8588
11.8622	0.1418	3600	11.8583
11.8541	0.1576	4000	11.8580
11.8625	0.1733	4400	11.8577
11.8556	0.1891	4800	11.8574
11.8563	0.2049	5200	11.8573
11.8586	0.2206	5600	11.8571
11.8529	0.2364	6000	11.8568
11.8601	0.2521	6400	11.8568
11.8529	0.2679	6800	11.8566
11.8629	0.2837	7200	11.8565
11.8588	0.2994	7600	11.8563
11.864	0.3152	8000	11.8562
11.8638	0.3309	8400	11.8561
11.8506	0.3467	8800	11.8560
11.8658	0.3625	9200	11.8560
11.8564	0.3782	9600	11.8558
11.8512	0.3940	10000	11.8558
11.8631	0.4097	10400	11.8556
11.8509	0.4255	10800	11.8555
11.8613	0.4412	11200	11.8555
11.8521	0.4570	11600	11.8554
11.8584	0.4728	12000	11.8553
11.8576	0.4885	12400	11.8552
11.8553	0.5043	12800	11.8551
11.8577	0.5200	13200	11.8551
11.8486	0.5358	13600	11.8550
11.8647	0.5516	14000	11.8549
11.8507	0.5673	14400	11.8548
11.8569	0.5831	14800	11.8547
11.8564	0.5988	15200	11.8547
11.8536	0.6146	15600	11.8547
11.8625	0.6303	16000	11.8546
11.8549	0.6461	16400	11.8545
11.8607	0.6619	16800	11.8545
11.861	0.6776	17200	11.8544
11.8607	0.6934	17600	11.8544
11.8556	0.7091	18000	11.8543
11.8558	0.7249	18400	11.8543
11.8527	0.7407	18800	11.8543
11.864	0.7564	19200	11.8543
11.8636	0.7722	19600	11.8542
11.863	0.7879	20000	11.8542
11.8592	0.8037	20400	11.8542
11.8636	0.8195	20800	11.8541
11.8623	0.8352	21200	11.8541
11.8496	0.8510	21600	11.8541
11.855	0.8667	22000	11.8541
11.8589	0.8825	22400	11.8540
11.847	0.8982	22800	11.8540
11.8596	0.9140	23200	11.8539
11.8567	0.9298	23600	11.8539
11.852	0.9455	24000	11.8538
11.8608	0.9613	24400	11.8538
11.8691	0.9770	24800	11.8538
11.8529	0.9928	25200	11.8537
12.5381	1.0086	25600	11.8538
13.4554	1.0243	26000	11.8538

Framework versions

PEFT 0.13.2
Transformers 4.46.0
Pytorch 2.5.0+cu124
Datasets 3.0.1
Tokenizers 0.20.1

Library: peft
Type: qwen2
Created: Mar 27, 2025
Updated: Mar 28, 2025