omni-0-mini-preview

<p align="center"> <img alt="Omni 0 preview models header" src="https://github.com/omniomni-ai/omni-0-preview-models/raw/refs/heads/main/omni-0-preview-models-image.png"> </p> <p align="center"> <a href="https://github.com/omniomni-ai"><strong>GitHub</strong></a> &nbsp <a href="https://omniomni.framer.website/"><strong>Website</strong></a> &nbsp <strong>Paper (Coming Soon)</strong> </p> <br>

Omni 0 Preview Models

Omni 0 preview models are a series of 1.7B parameter LLMs optimized for STEM knowledge consisting of 4 expert models and one final merged output between all experts. Experts include: Science Expert, Technology Expert, Engineering Expert, Math Expert. Through DARE-TIES merging and expert finetuning, Omni is able to achieve state-of-the-art domain benchmark results among alternative optimization techniques, as well as optimal compute-knowledge efficiency across similar models, while also performing comparably with them in tested benchmarks.

<p align="center"> <figure> <img src=" https://github.com/omniomni-ai/omni-0-preview-models/raw/refs/heads/main/compute-knowledge-efficiency-plot.png" alt="Omni compute-knowledge efficiency plot"> <figcaption>Omni achieves optimal compute-knowledge efficiency compared to alternative models</figcaption> </figure> </p>

[!Note] This model card is for omni-0-mini-preview. All other models can be found on Omni's HuggingFace page

Benchmarks

Omni-0-mini-preview Benchmarks

<br>

Expert Model Benchmarks

[!Note] Expert average refers to the average of the expert model for the STEM domain in focus while benchmarking

Models

Omni comes in a total of 5 models:

Merged Model

• omni-0-mini-preview - Merged output of all four experts through DARE-TIES, delivering large improvements in performance in STEM domains compared to its base.

Experts

• omni-0-science-preview - Science expert finetuned on corpora of scientific wikipedia texts and academic papers, as well as a chat-templated scientific Q&A dataset
• omni-0-technology-preview - Technology expert finetuned on chat-templated code generation data and stack exchange questions and top-voted answers
• omni-0-engineering-preview - Engineering expert finetuned on corpora of engineering-related wikipedia texts and academic papers
• omni-0-math-preview - Math expert finetuned on chat-templated math Q&A data

All Omni experts are finetuned from their base model: SmolLM2 1.7B Instruct on H100/Ada 6000/A100 GPUs, improving by 25.32% on average over all tested STEM benchmarks.

Features

Made for all Omni is a series of highly efficient Large Language Models aimed at expanding the accessibility of AI, filling in its gaps among underserved populations.

Efficient Omni operates at optimal compute-knowledge efficiency compared to similar models.

Merged architecture Omni uses merging to provide the collective accuracy of specialized models, leveraging their capabilities to enhance the final merged model.

Multi-disciplinary Omni's first variant achieves state-of-the-art performance across STEM compared to alternative optimization techniques.

---

Inference

Transformers

Transformers is a framework by HuggingFace unifying model development and inference, allowing for simple and seamless interactions with models found on HuggingFace.

To get started with running inference using Omni, install transformers

pip install transformers

After transformers has been installed, run the following code to generate outputs from any Omni model

from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "omniomni/omni-0-mini-preview" # Can be any Omni model

device = "cuda" # For GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

# For multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
# However, Omni is small enough to run on individual commodity GPUs and low-resource devices
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

messages = [{"role": "user", "content": "What is STEM?"}]

input_text=tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)

outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, do_sample=True)

print(tokenizer.decode(outputs[0]))

vLLM

vLLM provides fast LLM inference to models spanning vast amounts of architectures through both server and in-file implementations.

First, install vLLM's package

uv pip install vllm --torch-backend=auto

After that, run this command to start a server with Omni via vLLM

vllm serve omniomni/omni-0-mini-preview

To use Omni with vLLM without creating a server, run this code to generate outputs within a Python file

# vLLM automatically uses a GPU unless built with CPU wheels, so no need to specify a device

from vllm import LLM, SamplingParams

prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The answer to x^2 + 2x + 1 is",
    "The future of AI is",
]

sampling_params = SamplingParams(temperature=0.7)

llm = LLM(model="omniomni/omni-0-mini-preview")

outputs = llm.generate(prompts, sampling_params)

for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")