sentis-phi-3.5-mini-instruct-uint8

Phi-3.5-mini-instruct Sentis Conversion

This repository contains a conversion of Microsoft's Phi-3.5 mini model into Unity's Sentis format. The model has been quantized to uint8 for optimized performance.

Model Information

• Original Model: Microsoft Phi-3.5-mini-instruct
• Conversion: Phi-3.5 mini to Unity Sentis format
• Quantization: uint8

Installation

To use this model in your Unity project:

1. Download the model JSON files and the .sentis model from this repository.
2. Place these files in your Unity project's StreamingAssets folder.
3. Download the Phi3Claude.cs script and put it into your Assets folder.

Requirements

• Unity Sentis Version: 1.6.0-pre.1

To run the LlamaTokenizer in Unity, you need to download and include the Microsoft.ML.Tokenizers.dll DLL from the Microsoft's nuget servers and all its dependencies in your project:

• Google.Protobuf.dll
• Microsoft.Bcl.AsyncInterfaces.dll
• Microsoft.Bcl.HashCode.dll
• Microsoft.ML.Tokenizers.dll
• System.Runtime.CompilerServices.Unsafe.dll
• System.Text.Encodings.Web.dll
• System.Text.Json.dll

Usage

To use the Phi-3.5-mini-instruct model in your Unity project:

1. Ensure you have completed the installation steps and added the required DLLs to your project.
2. In your Unity scene, add the Phi3Claude script to the Camera GameObject.
3. Click the Play button to run your scene.
4. Check the Console window to inspect the generated text output.

The Phi3Claude script will run the text generation using the Sentis-converted Phi-3.5-mini-instruct model. You can modify this script to customize the input or adjust the generation as needed.

Acknowledgements

This model is a conversion of the Microsoft Phi-3.5 mini model. Please refer to the original model repository for more information about the base model.