SahabatAI-Lion-9B-TIES-v1-GGUF

SahabatAI-Lion-9B-TIES-v1

formerly gemma2-9b-cpt-sahabatai-v1-instruct-BaseTIES (model name too long :D )

Based on some research, when a finetuned model is merged with its base model with TIES method, there is possibility the merged model will achieve better output.

UPDATE!!! as 20 November 2024, this model is third best model (number one for Gemma2-9B based model) on HF's Open LLM Leaderboard (with Merge/MoErges hide model unchecked) for LLM model below 10B parameters.

gmonsoon/SahabatAI-Lion-9B-TIES-v1 is a merge of the following models:

• GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
• aisingapore/gemma2-9b-cpt-sea-lionv3-instruct

DEMO Spaces: HERE

🧩 Configuration

models:
  - model: GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
    parameters:
      weight: 1
      density: 1
  - model: GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
    parameters:
      weight: 1
      density: 1
merge_method: ties
base_model: aisingapore/gemma2-9b-cpt-sea-lionv3-instruct
parameters:
  density: 1
  normalize: true
  int8_mask: true
dtype: bfloat16

💻 Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "gmonsoon/SahabatAI-Lion-9B-TIES-v1"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Open LLM Leaderboard Evaluation Results

Detailed results can be found here