xLSTM-7b-Instruct

Model Card for xLSTM-7b-Instruct

[!WARNING] This repository contains experimental models designed strictly for academic evaluation and research purposes.

Critical Constraints:

• No Production Deployment: Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances.
• No Liability: Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.

This model is an experimental fine-tuned version of NX-AI/xLSTM-7b.

The goal is to add support for chat templates (from HuggingFaceTB/SmolLM3-3B) as discussed here: https://huggingface.co/NX-AI/xLSTM-7b/discussions/11

It has been trained using TRL.

Quick start

For text generation you have to to pin specific pytorch versions https://huggingface.co/datasets/John6666/forum1/blob/main/xlstm_1.md

pip install "torch==2.5.1" "torchvision==0.20.1" "torchaudio==2.5.1" --index-url https://download.pytorch.org/whl/cu124
pip install "triton==3.4.0"  # >=3.1 is OK; 3.4.0 current as of Sep 2025
pip install "mlstm-kernels==2.0.1" "xlstm==2.0.5"

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

MERGED_MODEL_PATH = "ethicalabs/xLSTM-7b-Instruct"

# We apply a configuration that uses native, hardware-agnostic kernels.
print("Defining a safe, native kernel configuration for compatibility...")

safe_config = AutoConfig.from_pretrained(MERGED_MODEL_PATH, trust_remote_code=True)
# Use the stable, native parallel kernel
safe_config.chunkwise_kernel = "chunkwise--native_autograd"
safe_config.sequence_kernel = "native_sequence__native"
safe_config.step_kernel = "native"
# This flag is still required for the HF implementation to avoid unpacking errors
safe_config.return_last_states = False

# Load the final, merged model with the safe config (no quantization)
print("Loading the final, merged model in bfloat16 (no quantization for compatibility)...")
final_model = AutoModelForCausalLM.from_pretrained(
    MERGED_MODEL_PATH,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    config=safe_config 
)
final_tokenizer = AutoTokenizer.from_pretrained(MERGED_MODEL_PATH)

# The tokenizer needs to know which token to use for padding.
if final_tokenizer.pad_token is None:
    final_tokenizer.pad_token = final_tokenizer.eos_token
print("Padding token has been set.")

# Set the model to evaluation mode
final_model.eval()

# Run Inference
print("Preparing prompt and running generation...")
messages = [{"role": "user", "content": "Please suggest me some mindfullness relaxation techniques to overcome the frustration I have when I deal with Triton kernels."}]

prompt_string = final_tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False
)

inputs = final_tokenizer(
    prompt_string,
    return_tensors="pt"
).to(final_model.device)

with torch.no_grad():
    outputs = final_model.generate(**inputs, max_new_tokens=512)
    output_ids = outputs[0][len(inputs.input_ids[0]) :]

response_text = final_tokenizer.decode(output_ids, skip_special_tokens=True)
print("\n--- Generated Response ---")
print(response_text)

Example text generation:

--- Generated Response ---
1. Mindful breathing: Take a few deep breaths and focus on your breath. Feel the air entering and leaving your body. This can help you calm down and focus on the present moment.

2. Body scan: Lie down or sit comfortably and focus on each part of your body, starting from your toes and moving up to your head. Notice any tension or discomfort and try to release it.

3. Visualization: Close your eyes and imagine yourself in a peaceful place, such as a beach or forest. Focus on the sights, sounds, and smells around you. This can help you relax and reduce stress.

4. Progressive muscle relaxation: Tense and then relax each muscle group in your body, starting from your toes and moving up to your head. This can help you release tension and relax your body.

5. Mantra repetition: Repeat a calming phrase or mantra to yourself, such as "I am calm" or "I am relaxed." This can help you focus your mind and reduce stress.

6. Walking meditation: Take a slow, mindful walk and focus on each step you take. Notice the feeling of your feet touching the ground and the movement of your legs. This can help you reduce stress and increase mindfulness.

7. Yoga: Practice yoga poses that focus on relaxation and mindfulness, such as child's pose, downward facing dog, and savasana. This can help you reduce stress and increase mindfulness.

8. Guided meditation: Listen to a guided meditation or relaxation exercise. This can help you focus your mind and reduce stress.

9. Mindful eating: Take a few deep breaths and focus on the taste, texture, and smell of your food. This can help you reduce stress and increase mindfulness.

10. Nature walk: Take a slow, mindful walk in nature and focus on the sights, sounds, and smells around you. This can help you reduce stress and increase mindfulness.

Remember to practice these techniques regularly to reduce stress and increase mindfulness.

You can also try using apps such as Headspace, Calm, or Insight Timer to help you practice mindfulness and relaxation techniques. These apps offer guided meditations, breathing exercises, and other mindfulness practices.

Remember to be patient and consistent with your practice, and don't be afraid to try different techniques to find what works best for you.

You can also try seeking support from a therapist or counselor if you are struggling with

Training procedure

<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>

This model was trained with SFT.

Evaluation

This model has been loaded in 4-bit and evaluated with lighteval

Framework versions

• PEFT 0.17.1
• TRL: 0.23.1
• Transformers: 4.57.0
• Pytorch: 2.8.0+cu128
• Datasets: 4.2.0
• Tokenizers: 0.22.1

Citations

@misc{beck2024xlstmextendedlongshortterm,
      title={xLSTM: Extended Long Short-Term Memory}, 
      author={Maximilian Beck and Korbinian Pöppel and Markus Spanring and Andreas Auer and Oleksandra Prudnikova and Michael Kopp and Günter Klambauer and Johannes Brandstetter and Sepp Hochreiter},
      year={2024},
      eprint={2405.04517},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2405.04517}, 
}

@misc{han2024parameterefficientfinetuninglargemodels,
      title={Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey}, 
      author={Zeyu Han and Chao Gao and Jinyang Liu and Jeff Zhang and Sai Qian Zhang},
      year={2024},
      eprint={2403.14608},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2403.14608}, 
}

@misc{liu2024doraweightdecomposedlowrankadaptation,
      title={DoRA: Weight-Decomposed Low-Rank Adaptation}, 
      author={Shih-Yang Liu and Chien-Yi Wang and Hongxu Yin and Pavlo Molchanov and Yu-Chiang Frank Wang and Kwang-Ting Cheng and Min-Hung Chen},
      year={2024},
      eprint={2402.09353},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2402.09353}, 
}

@misc{vonwerra2022trl,
	title        = {{TRL: Transformer Reinforcement Learning}},
	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
	year         = 2020,
	journal      = {GitHub repository},
	publisher    = {GitHub},
	howpublished = {\url{https://github.com/huggingface/trl}}
}

@misc{lighteval,
  author = {Habib, Nathan and Fourrier, Clémentine and Kydlíček, Hynek and Wolf, Thomas and Tunstall, Lewis},
  title = {LightEval: A lightweight framework for LLM evaluation},
  year = {2023},
  version = {0.11.0},
  url = {https://github.com/huggingface/lighteval}
}