Downloads · 30 days
0
Lara-TRT-EDGE-LLM-Qwen3.5-9B-INT4
LaraAI-Labs/Lara-TRT-EDGE-LLM-Qwen3.5-9B-INT4
Lara-TRT-EDGE-LLM-Qwen3.5-9B-INT4 is a machine learning model from LaraAI-Labs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for tensorrt. The card lists the license as apache-2.0.
Access
Public
Updated Jun 10, 2026
Repo size
15.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.engine13.4 GB · 77%
At a glance
- Library
- tensorrt
- License
- apache-2.0
- Access
- Public
- Created
- Jun 10, 2026
- Updated
- Jun 10, 2026
- SHA
- 44016c63
Base models
Model card
From the Hugging Face model README
Lara-TRT-EDGE-LLM-Qwen3.5-9B-INT4
INT4 TensorRT-Edge-LLM engine (Qwen3.5-9B base) for the Lara64-TRT project — built to push time-to-first-token (TTFT) to the limit for real-time voice conversation.
Overview
| Base model | Qwen3.5-9B |
| Quantization | INT4 (groupwise GEMM / INT4 MoE) |
| Runtime | TensorRT-Edge-LLM |
| Target hardware | NVIDIA Jetson AGX Orin 64GB |
| Stack | JetPack 6.2 · CUDA 12.6 · aarch64 |
| Goal | Real-time conversation, TTFT ≈ 0.8 s |
| Use | Primary LLM backend for the Lara voice assistant (llm.backend: trt_edge) |
Contents
engines/llm/— main INT4 LLM enginellm.engine·embedding.safetensors·tokenizer.json·tokenizer_config.json·config.json·processed_chat_template.json
engines_mtp/— Eagle MTP speculative decoding engines for higher decode throughputeagle_base.engine·eagle_draft.engine·embedding.safetensors· tokenizer/config files
⚠️ Compatibility
TensorRT engines are hardware- and version-locked. These engines run only on Jetson AGX Orin with JetPack 6.2 / CUDA 12.6 and a matching TensorRT-Edge-LLM build. To run on other hardware or TensorRT versions, rebuild the engine from the source ONNX.
License
Apache-2.0, following the Qwen3.5-9B base model license.
- Library
- tensorrt
- License
- apache-2.0
- Languages
- en, ko, zh
- Created
- Jun 10, 2026
- Updated
- Jun 10, 2026