uoomi-v1-1.78B

Uoomi-ai/uoomi-v1-1.7b

Model Card for uoomi-v1-1.7b This is a small model with 1.78 billion parameters distilled from the Qwen large model. Its small parameter size makes it unsuitable for answering complex questions. The uoomi-v1-1.7b model is designed for personal developers to embed and optimize or further fine-tune. It allows users to create custom dialogue templates.

<div align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/64b81f6d047fa3db940cab35/2uiS-5hICp_PT6c27LLVe.png" width="50%" alt="Uoomi" /> </div> <hr> <div align="center" style="line-height: 1;"> <a href="https://xxx" target="_blank" style="margin: 21px;"> <img src="https://img.shields.io/badge/MMCC-Website-fff?colorA=888" alt="MMCC" style="display: inline-block; vertical-align: middle;"> </a> <a href="https://xxx" target="_blank" style="margin: 21px;"> <img src="https://img.shields.io/badge/uoomi-joinus-fff?colorA=888" alt="NAVF-join" style="display: inline-block; vertical-align: middle;"> </a> <a href="https://xxx" target="_blank" style="margin: 21px;"> <img src="https://img.shields.io/badge/NAVF-Website-00ff24?colorA=638dff" alt="NAVF" style="display: inline-block; vertical-align: middle;"> </a> </div>

Model Description

• Developer: Navf-新世家
• Funder: Mystic Matrix Computer Company (灵兮矩阵)
• Model Type: Transformer
• Supported Languages (NLP): We recommend English (en) and Chinese (zh).
• License Agreement: MIT License

Model Usage

uoomi-v1-1.7b allows developers to perform fine-tuning, especially for output labeling or backend classification. You can try using this model to enhance the recognition of images and audio. The parameters of uoomi-v1-1.7b are relatively clean, and you can use "polluted weights" to overwrite the original data. However, this will increase the original model's weight parameters. Developers can choose to package parameters and weights separately (unpolluted) to customize model weights. If you do not want to lose the original model's capabilities, avoid directly deleting the original weight files.

Inapplicable Use Cases

• Complex ethical issues

Bias

uoomi-v1-1.7b is distilled from the Qwen2 model, and its biases are based on those of Qwen2. Additionally, the biases of uoomi-v1-1.7b mainly originate from biases in the training data. If the training data contains unfair descriptions of specific genders, races, or social groups, the model may exacerbate these biases during prediction.

Risks

When processing personal data and sensitive information, such as medical health record analysis or personalized recommendation systems, privacy protection and security become important considerations.

Resource Constraints

• Computational Complexity: The self-attention mechanism of the uoomi-v1-1.7b model has a computational complexity of O(N²d), where N is the sequence length and d is the token embedding dimension. This means that as the input sequence length increases, the computational complexity grows quadratically, leading to significant consumption of computing resources. Therefore, we do not recommend using this model for long text inputs.
• GPU Requirements: Do not use GPUs with less than 2 GB of VRAM when using the uoomi-v1-1.7b model. According to our tests, the model can operate normally with a minimum of 2 GB VRAM.

Training Hyperparameters

• Training Mode: Mixed Precision
• Batch Size per GPU: 2
• Learning Rate: 4e-4
• Number of Training Epochs: 8

Hardware Recommendations for Fine-Tuning

• Operating System: Windows 10
• GPU: NVIDIA 3090 24GB (Recommended GPU VRAM > 4GB)

Contact Information

If you have any questions or would like to join our development team, please contact us at [email protected]. Include the purpose of your contact in the email subject line. We welcome your suggestions and contributions!