Downloads · 30 days
11
19% of all-time downloads
jepa_proof_bert
ducanhdinh/jepa_proof_bert
jepa_proof_bert is a machine learning model from ducanhdinh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
BERT base pretrained from scratch với hai mục tiêu: - Masked Language Modeling (MLM) — 80/10/10 replacement rule, mask probability 0.15 - Next Sentence Prediction (NSP)
All-time downloads
58
Public
Parameters
110M
440 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors440 MB · 100%
At a glance
- License
- apache-2.0
- Model type
- bert
- Access
- Public
- Created
- Jun 8, 2026
- Updated
- Jun 8, 2026
- SHA
- c86639f7
Model card
From the Hugging Face model README
ducanhdinh/jepa_proof_bert
BERT base pretrained from scratch với hai mục tiêu:
- Masked Language Modeling (MLM) — 80/10/10 replacement rule, mask probability
0.15 - Next Sentence Prediction (NSP)
Thông số huấn luyện
| Tham số | Giá trị |
|---|---|
| Max sequence length | 256 |
| Batch size | 256 |
| Epochs | 10 |
| Learning rate | 0.0001 |
| MLM probability | 0.15 |
Cách dùng
from transformers import BertForPreTraining, BertTokenizerFast
import torch
tokenizer = BertTokenizerFast.from_pretrained("ducanhdinh/jepa_proof_bert")
model = BertForPreTraining.from_pretrained("ducanhdinh/jepa_proof_bert")
encoded = tokenizer("Hello world!", return_tensors="pt")
with torch.no_grad():
output = model(**encoded)
- Type
- bert
- License
- apache-2.0
- Languages
- en
- Created
- Jun 8, 2026
- Updated
- Jun 8, 2026