Downloads · 30 days
19
8% of all-time downloads
SmartMaid-123b
gghfez/SmartMaid-123b
SmartMaid-123b is a machine learning model from gghfez. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
This experimental model is a hybrid creation combining aspects of Mistral-Large-Instruct-2407 and Lumimaid-v0.2-123B using LoRA (Low-Rank Adaptation) on the mlp.downproj module.
All-time downloads
244
Public
Parameters
123B
245 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors245 GB · 100%
At a glance
- License
- other
- Model type
- mistral
- Access
- Public
- Created
- Sep 26, 2024
- Updated
- Sep 28, 2024
- SHA
- 9d2d5049
Base models
Model card
From the Hugging Face model README
SmartMaid-123b
This experimental model is a hybrid creation combining aspects of Mistral-Large-Instruct-2407 and Lumimaid-v0.2-123B using LoRA (Low-Rank Adaptation) on the mlp.down_proj module.
Model Details
- Base Model: Mistral-Large-Instruct-2407
- Influence Model: Lumimaid-v0.2-123B
- Method: LoRA extraction from Lumimaid and targetted application to Mistral-Large
- LoRA Configuration:
- Rank: 32
- Alpha: 64
- Target Module:
mlp.down_proj
Process
- A LoRA was extracted from Lumimaid-v0.2-123B
- This LoRA was then applied to a fresh instance of Mistral-Large-Instruct-2407 targetting only the mlp.down_proj modules.
- The resulting model was merged to create this standalone version.
Purpose
The aim of this model is to incorporate the enhanced prose qualities of Lumimaid-v0.2-123B while retaining the core intelligence and capabilities of Mistral-Large.
By applying the LoRA to the mlp.down_proj module, we sought to influence the model's language generation style without significantly altering its underlying knowledge and reasoning abilities.
Prompt Template
<s>[INST] {input} [/INST] {output}</s>
EXl2 Quants
- Type
- mistral
- License
- other
- Languages
- en
- Created
- Sep 26, 2024
- Updated
- Sep 28, 2024