Downloads · 30 days
16
26% of all-time downloads
zhang-yice/Qwen3-4B-sentiment-distillation-v2
Qwen3-4B-sentiment-distillation-v2 is a machine learning model from zhang-yice. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-4.0.
Recent efforts leverage knowledge distillation techniques to develop lightweight and practical sentiment analysis models. These methods are grounded in human-written instructions and large-scale user texts. Despite th…
Downloads · 30 days
16
26% of all-time downloads
All-time downloads
62
Public
Parameters
4B
8.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8 GB · 100%
From the Hugging Face model README
Recent efforts leverage knowledge distillation techniques to develop lightweight and practical sentiment analysis models. These methods are grounded in human-written instructions and large-scale user texts. Despite the promising results, two key challenges remain: (1) manually written instructions are limited in diversity and quantity, making them insufficient to ensure comprehensive coverage of distilled knowledge; (2) large-scale user texts incur high computational cost, hindering the practicality of these methods. To this end, we introduce CompEffDist, a comprehensive and efficient distillation framework for sentiment analysis. Our framework consists of two key modules: attribute-based automatic instruction construction and difficulty-based data filtering, which correspondingly tackle the aforementioned challenges. Applying our method across multiple model series (Llama-3, Qwen-3, and Gemma-3), we enable 3B student models to match the performance of 20x larger teacher models on most tasks. In addition, our approach greatly outperforms baseline methods in data efficiency, attaining the same performance level with only 10% of the data.
@inproceedings{xie-etal-2025-comprehensive,
title = "Comprehensive and Efficient Distillation for Lightweight Sentiment Analysis Models",
author = "Xie, Guangyu and
Zhang, Yice and
Bao, Jianzhu and
Wang, Qianlong and
Sun, Yang and
Wang, Bingbing and
Xu, Ruifeng",
editor = "Christodoulopoulos, Christos and
Chakraborty, Tanmoy and
Rose, Carolyn and
Peng, Violet",
booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.emnlp-main.1122/",
doi = "10.18653/v1/2025.emnlp-main.1122",
pages = "22081--22102",
ISBN = "979-8-89176-332-6",
}