Huihui-gemma-4-E4B-it-abliterated-LiteRT-LM

Huihui Gemma 4 E4B Abliterated — LiteRT-LM

A LiteRT-LM conversion of huihui-ai/Huihui-gemma-4-E4B-it-abliterated.

This repository contains a ready-to-run .litertlm bundle converted from the original Hugging Face BF16 Safetensors checkpoint for use with Google LiteRT-LM.

This repository is not a new fine-tune and does not claim authorship of the original model weights. It is a runtime-format conversion of the Huihui model.

Model lineage

google/gemma-4-E4B
        ↓
google/gemma-4-E4B-it
        ↓
huihui-ai/Huihui-gemma-4-E4B-it-abliterated
        ↓
BF16 Safetensors
        ↓
litert-torch export_hf
        ↓
Huihui-gemma-4-E4B-it-abliterated.litertlm

About the source model

The source model was published by huihui-ai as an abliterated variant of google/gemma-4-E4B-it.

According to the original model card, abliteration significantly reduces the model's refusal behavior and safety filtering. This conversion does not restore or add safety alignment.

Please read the source model card before use:

• https://huggingface.co/huihui-ai/Huihui-gemma-4-E4B-it-abliterated

Conversion

The model was converted from the original Hugging Face Safetensors checkpoint using the LiteRT Torch Hugging Face exporter.

The conversion command was:

litert-torch export_hf \
  --model="$HOME/Models/huggingface/huihui-gemma-4-e4b" \
  --output_dir="$HOME/Models/litert/huihui-gemma-4-e4b-full" \
  --task=image_text_to_text \
  --cache_length=4096 \
  --prefill_lengths=128,256,512 \
  --externalize_embedder=true \
  --moe_exports_implementation=eager \
  --export_vision_encoder=true \
  --export_audio_encoder=true \
  --quantization_recipe=dynamic_wi8_afp32 \
  --jinja_chat_template_override=litert-community/gemma-4-E4B-it-litert-lm \
  --bundle_litert_lm=true

The LiteRT-compatible Gemma 4 E4B chat template was taken from:

litert-community/gemma-4-E4B-it-litert-lm

The resulting bundle is:

Huihui-gemma-4-E4B-it-abliterated.litertlm

No GGUF or Ollama model was used as an intermediate format.

Download

Install the Hugging Face CLI if needed:

pip install -U huggingface_hub

Download only the LiteRT-LM bundle:

hf download \
  vokash3/Huihui-gemma-4-E4B-it-abliterated-LiteRT-LM \
  Huihui-gemma-4-E4B-it-abliterated.litertlm \
  --local-dir .

This produces:

./Huihui-gemma-4-E4B-it-abliterated.litertlm

Run with LiteRT-LM

Install the current LiteRT-LM CLI:

uv tool install litert-lm

Run the model:

litert-lm run \
  ./Huihui-gemma-4-E4B-it-abliterated.litertlm \
  --prompt="Hello! Tell me briefly who you are."

For an interactive run:

litert-lm run ./Huihui-gemma-4-E4B-it-abliterated.litertlm

Available backends and CLI options can change as LiteRT-LM evolves, so check the installed version before using backend-specific flags:

litert-lm run --help

Benchmarks

The current bundle was benchmarked across several desktop and mobile devices.

Important: the Ollama figures below are not an apples-to-apples backend comparison. Ollama uses a GGUF/Q4_K_M build of huihui_ai/gemma-4-abliterated:e4b, while this repository contains a LiteRT-LM bundle produced by litert-torch export_hf. Quantization, runtime, kernels, prompt processing, and packaging therefore differ.

LiteRT-LM results

Desktop LiteRT-LM results used the built-in benchmark with 256 prefill tokens, 256 decode tokens, and a maximum context size of 4096.

The mobile measurements were taken in Google AI Edge Gallery / Edge Studio using the successfully imported multimodal bundle.

Ollama comparison

For the same Huihui Gemma 4 E4B family, Ollama was benchmarked separately with a natural-language prompt of 243 tokens and 256 generated tokens.

Practical observations

On the tested desktop systems, Ollama/GGUF delivered higher decode throughput than the current LiteRT-LM export. The largest absolute LiteRT-LM result came from the RX 5700 XT at 37.98 tok/s decode, followed by the Apple M4 at 16.53 tok/s.

The refreshed LiteRT-LM bundle also shows notably higher prefill throughput than the earlier text-only export. One relevant change is the explicit export of prefill_lengths=128,256,512, which includes a native 256-token prefill path matching the benchmark workload.

These results should be read as practical deployment measurements for this model on these devices, not as a universal ranking of LiteRT-LM versus Ollama.

Platforms

LiteRT-LM is designed for on-device inference across supported desktop and edge platforms. Consult the upstream project for the current platform/backend support matrix:

• https://github.com/google-ai-edge/LiteRT-LM

The .litertlm file in this repository was created as a portable LiteRT-LM bundle rather than a GGUF model for llama.cpp/Ollama.

Google AI Edge Gallery / Edge Studio compatibility

The refreshed bundle is exported as an image_text_to_text model with the vision encoder and vision adapter included.

The model has been successfully imported and initialized in Google AI Edge Gallery / Edge Studio, including AI Chat mode.

The working export uses:

--task=image_text_to_text
--export_vision_encoder=true
--export_audio_encoder=true
--prefill_lengths=128,256,512

Inspection of the generated bundle confirms the presence of the text inference components plus the Gemma 4 vision encoder and vision adapter.

This supersedes the earlier text-only build that could fail in AI Chat with:

NOT_FOUND: TF_LITE_AUDIO_ENCODER_HW not found in the model

The current uploaded model should therefore be treated as the replacement for that earlier bundle.

Important notes and limitations

• This is a conversion, not a separately trained model.
• Behavioral characteristics of the source Huihui model remain relevant.
• The source model has substantially reduced refusal/safety behavior.
• Output can include sensitive, controversial, unsafe, inaccurate or otherwise undesirable content.
• Users should review outputs before using them in production or public-facing applications.
• Runtime behavior may differ from Transformers or GGUF builds because of differences in quantization, execution backend, sampling, prompt formatting and runtime implementation.
• Exact token-for-token parity with the original Safetensors checkpoint is not expected.
• This repository does not provide independent benchmark or quality claims.
• The conversion is intended for LiteRT-LM multimodal chat / text-and-vision use.

License and upstream terms

The source Huihui repository is published on Hugging Face with the Apache-2.0 license metadata.

Users should also review the terms, notices and usage requirements associated with the upstream Gemma model family and the original Huihui model before redistribution or deployment.

Upstream resources:

• Huihui source model: https://huggingface.co/huihui-ai/Huihui-gemma-4-E4B-it-abliterated
• Google Gemma 4 E4B IT: https://huggingface.co/google/gemma-4-E4B-it
• LiteRT-LM: https://github.com/google-ai-edge/LiteRT-LM
• LiteRT Torch: https://github.com/google-ai-edge/litert-torch
• LiteRT Community Gemma 4 E4B: https://huggingface.co/litert-community/gemma-4-E4B-it-litert-lm

Credits

• Google — Gemma 4 and LiteRT/LiteRT-LM
• huihui-ai — Huihui-gemma-4-E4B-it-abliterated
• LiteRT Community — LiteRT-LM Gemma 4 reference bundle and compatible chat template
• vokash3 — LiteRT-LM conversion and packaging in this repository

Repository contents

.
├── README.md
└── Huihui-gemma-4-E4B-it-abliterated.litertlm

Reproducibility summary

Source:
huihui-ai/Huihui-gemma-4-E4B-it-abliterated

Source format:
BF16 Safetensors

Exporter:
litert-torch export_hf

Options:
--task=image_text_to_text
--cache_length=4096
--prefill_lengths=128,256,512
--externalize_embedder=true
--moe_exports_implementation=eager
--export_vision_encoder=true
--export_audio_encoder=true
--quantization_recipe=dynamic_wi8_afp32
--jinja_chat_template_override=litert-community/gemma-4-E4B-it-litert-lm
--bundle_litert_lm=true

Output:
Huihui-gemma-4-E4B-it-abliterated.litertlm (~7.8 GiB)