Saul-7B-Instruct-v1-GGUF

Saul-7B-Instruct-v1-GGUF

Legal domain language model for case analysis and legal document processing.

Model Details

• Base Model: Equall/Saul-7B-Instruct-v1
• GGUF Source: MaziyarPanahi/Saul-Instruct-v1-GGUF
• Quantization: Q4_K_M (4.37 GB) - Balanced quality and performance
• Architecture: Llama-based, 7B parameters
• Context Length: 32,768 tokens
• License: MIT

Files

Usage

LM Studio

1. Download the .gguf file
2. Load in LM Studio
3. Use chat format: llama-2

llama.cpp

./main -ngl 35 -m Saul-Instruct-v1.Q4_K_M.gguf --color -c 32768 --temp 0.7 --repeat_penalty 1.1

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
  model_path="./Saul-Instruct-v1.Q4_K_M.gguf",
  n_ctx=32768,
  n_threads=8,
  n_gpu_layers=35
)

Citation

@misc{colombo2024saullm7b,
  title={SaulLM-7B: A pioneering Large Language Model for Law},
  author={Pierre Colombo and Telmo Pessoa Pires and Malik Boudiaf and Dominic Culver and Rui Melo and Caio Corro and Andre F. T. Martins and Fabrizio Esposito and Vera Lúcia Raposo and Sofia Morgado and Michael Desa},
  year={2024},
  eprint={2403.03883},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Paper

https://arxiv.org/abs/2403.03883