Downloads · 30 days
3
15% of all-time downloads
DUSFT-llm-model
leftfooted/DUSFT-llm-model
DUSFT-llm-model is a machine learning model from leftfooted. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
The DUS Forty Layer Merged Model leverages a unique layer interlocking strategy, combining layers from the Llama-2-13B and Mistral-7B architectures. This approach optimizes computational efficiency while maintaining c…
All-time downloads
20
Public
Parameters
9B
18 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors18 GB · 100%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Jan 10, 2025
- Updated
- Jan 10, 2025
- SHA
- 64fbae9c
Model card
From the Hugging Face model README
DUS Forty Layer Merged Model
Overview
The DUS Forty Layer Merged Model leverages a unique layer interlocking strategy, combining layers from the Llama-2-13B and Mistral-7B architectures. This approach optimizes computational efficiency while maintaining competitive performance across various natural language processing tasks.
Model Details
- Architecture: Based on Llama-2-13B and Mistral-7B
- Layer Arrangement: The
fortyconfiguration merges layers from both models, interlocking layers 0–20 with layers 12–32. - Tokenizer: Mistral-7B tokenizer is used for encoding and decoding.
Training Details
- Base Models:
- Llama-2-13B: meta-llama/Llama-2-13b-hf
- Mistral-7B: mistralai/Mistral-7B-v0.1
- Type
- llama
- License
- mit
- Created
- Jan 10, 2025
- Updated
- Jan 10, 2025