teapotai-teapotllm-onnx

TeapotAI TeapotLLM ONNX

ONNX version of teapotai/teapotllm to use with Transformers.js.

Also includes a JavaScript version of the teapotai python package for interacting with the model using the Transformers.js library.

View the example script index.js.

Check the WebGPU ONNX Playground.

Installation

yarn install

Usage

This library provides a TeapotAI class that helps you interact with language models. Here's a basic example:

import TeapotAI from './TeapotAI.js';

const ai = new TeapotAI({
  pipelineOptions: {
    dtype: 'q4'
  }
});

// Query with context
const response = await ai.query(
  "What is the capital?",
  "France is a country in Western Europe."
);