Downloads · 30 days
0
llama3.1-8b-pmt-task-cefr-demarcation
MohammadKhosravi/llama3.1-8b-pmt-task-cefr-demarcation
llama3.1-8b-pmt-task-cefr-demarcation is a machine learning model from MohammadKhosravi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
This model implements Step 1 (Task vs Control Demarcation) using the Pure PrefixMemory-Tuning architecture.
Access
Public
Updated Aug 26, 2026
Repo size
1.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.2 GB · 100%
At a glance
- Library
- peft
- License
- mit
- Access
- Public
- Created
- Aug 26, 2026
- Updated
- Aug 26, 2026
- SHA
- 8bb87df2
Base models
Model card
From the Hugging Face model README
Task-CEFR Demarcation PMT
This model implements Step 1 (Task vs Control Demarcation) using the Pure PrefixMemory-Tuning architecture.
Architecture Details
- Zero Text Prompts: Instructional text strings are completely removed.
- Task Demarcation: Employs a single global 4096-dim Task Embedding combined with a dynamic CEFR Embedding.
- MLP Gating: Concatenates Task and CEFR embeddings into an 8192-dim vector, passing it through a 2-layer MLP to generate a 4096-dim gating vector.
- Uncompressed Memory: Applies the gating vector to the ELU-activated hidden states before querying independent 4096*4096 Memory matrices across all 32 layers.
Training Performance
[
{
"epoch": 1,
"train_loss": 1.5453,
"val_loss": 1.4831,
"val_ppl": 4.41
},
{
"epoch": 2,
"train_loss": 1.2899,
"val_loss": 1.4498,
"val_ppl": 4.26
},
{
"epoch": 3,
"train_loss": 0.9291,
"val_loss": 1.5799,
"val_ppl": 4.85
},
{
"epoch": 4,
"train_loss": 0.4848,
"val_loss": 1.8735,
"val_ppl": 6.51
}
]
- Library
- peft
- License
- mit
- Created
- Aug 26, 2026
- Updated
- Aug 26, 2026