EVE-Instruct-GGUF-Q4_K_M

EVE-Instruct-GGUF-Q4_K_M

This repository provides a quantized GGUF version of EVE-Instruct for efficient local inference.

• Base model: eve-esa/EVE-Instruct
• Quantized variant: eve-esa/EVE-Instruct-GGUF-Q4_K_M
• Architecture: Llama (Mistral-compatible)
• Parameters: 24B
• Context length: 128k tokens
• Quantized file size: ~14.3 GB
• Minimum RAM: ~16 GB system or GPU RAM recommended

For full details on training, benchmarks, and capabilities, refer to the main EVE-Instruct model card.

---

Model Description

EVE-Instruct is a fine-tuned version of Mistral-Small-3.2-24B-Instruct-2506 specializing in Earth Intelligence, with particular emphasis on Earth Observation (EO) and Earth Science (ES) domains. It improves domain-specific capabilities while maintaining or exceeding the general capabilities of its base model.

EVE-Instruct-GGUF-Q4_K_M is a compressed (quantized) version designed for running on consumer hardware (CPU or low-VRAM GPUs).

Quantization Details

Q4_K_M is a "type-1" 4-bit quantization in super-blocks containing 8 blocks, each block having 32 weights. Scales and mins are quantized with 6 bits. This results in a good balance between model quality and file size, with modest accuracy loss compared to the original bf16 weights. For accuracy benchmarks, refer to the base model card.

Usage

llama.cpp

# Start a local OpenAI-compatible server:
llama-server -hf eve-esa/EVE-Instruct-GGUF-Q4_K_M:Q4_K_M

# Run inference directly in the terminal:
llama-cli -hf eve-esa/EVE-Instruct-GGUF-Q4_K_M:Q4_K_M

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="eve-esa/EVE-Instruct-GGUF-Q4_K_M",
    filename="EVE-Instruct-Q4_K_M.gguf",
)

llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a helpful Earth Intelligence assistant specializing in Earth Observation and Earth Science."
        },
        {
            "role": "user",
            "content": "What is the Normalized Difference Vegetation Index (NDVI) and how is it used in remote sensing?"
        }
    ]
)

Ollama

ollama run hf.co/eve-esa/EVE-Instruct-GGUF-Q4_K_M:Q4_K_M

Note: This is a GGUF quantized file intended for local inference with llama.cpp-compatible runtimes (llama.cpp, Ollama, LM Studio, etc.). It is not compatible with vLLM, which requires the original safetensors weights from eve-esa/EVE-Instruct.

Funding

This project is supported by the European Space Agency (ESA) Φ-lab through the Large Language Model for Earth Observation and Earth Science project, as part of the Foresight Element within the FutureEO Block 4 programme.

Citation

If you use this model in academic or research settings, please cite the base model:

@misc{eve-instruct-2025,
  title={EVE-Instruct: An Earth Intelligence Language Model},
  author={EVE-ESA},
  year={2025},
  note={arXiv:2508.09494},
  url={https://huggingface.co/eve-esa/EVE-Instruct}
}