Downloads · 30 days
6
26% of all-time downloads
InternVL3_5-8B-Flash-LLM-NF4-4bit
leewaylicnmsft/InternVL3_5-8B-Flash-LLM-NF4-4bit
InternVL3_5-8B-Flash-LLM-NF4-4bit is a machine learning model from leewaylicnmsft. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
--- tags: - internvl3.5 - 8b - flash - nf4 - 4bit - selective-quantization license: other ---
All-time downloads
23
Public
Parameters
8.8B
17.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors17.6 GB · 100%
At a glance
- Model type
- internvl_chat
- Access
- Public
- Created
- Nov 28, 2025
- Updated
- Nov 28, 2025
- SHA
- 5ecf379d
Model card
From the Hugging Face model README
tags:
- internvl3.5
- 8b
- flash
- nf4
- 4bit
- selective-quantization
license: other
---
# InternVL3_5-8B-Flash with Selective NF4 4-bit (LLM MLP only)
This package contains a copy of the original Flash model with Qwen3 MLP FFN layers (gate_proj, up_proj, down_proj) replaced by bitsandbytes Linear4bit (NF4). Attention projections, vision model, gating, and lm_head remain in FP16. Flash path remains enabled.
Load with:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoProcessor
model = AutoModelForCausalLM.from_pretrained('<repo>', trust_remote_code=True, device_map={'':'cuda'})
tok = AutoTokenizer.from_pretrained('<repo>', trust_remote_code=True, use_fast=False)
proc = AutoProcessor.from_pretrained('<repo>', trust_remote_code=True)
```
- Type
- internvl_chat
- Created
- Nov 28, 2025
- Updated
- Nov 28, 2025