Qwen2-1.5B-Instruct-ONNX-Web

Qwen2-1.5B-Instruct ONNX (INT8 Quantized) for Web

This repository contains a web-ready version of the Qwen/Qwen2-1.5B-Instruct model, converted to ONNX and quantized to INT8 for efficient in-browser inference with transformer.js.

This is a modified version of the original model. The conversion and quantization were performed to optimize for size and speed in WebGPU environments.

Original Model

• Model: Qwen/Qwen2-1.5B-Instruct
• Link: https://huggingface.co/Qwen/Qwen2-1.5B-Instruct

License

This model is governed by the Tongyi Qianwen LICENSE 2.0. The full license is included in this repository. By using this model, you agree to the terms and conditions set forth in the license.

How to Use with transformer.js

You can use this model directly in your JavaScript projects with the @xenova/transformers library.

import { pipeline } from '@xenova/transformers';

// Your Hugging Face repository ID
const modelId = 'HarmanjotSingh/Qwen2-1.5B-Instruct-ONNX-Web';

// Initialize the text generation pipeline
const generator = await pipeline('text-generation', modelId, {
  device: 'webgpu', // Use WebGPU for GPU acceleration
  quantized: true,  // The model is quantized
});

// Generate text
const output = await generator('Hello, my name is', {
  max_new_tokens: 100,
});

console.log(output);