GEMMV-Llama-8b

GEMMV: An LLM-based Automated Performance-Aware Framework for GEMM Verilog Generation

<img src="image/overview.jpg" style="zoom:30%;" />

GEMMV is an innovative series of open-source, instruction-tuned Large Language Models (LLMs) specifically designed for the generation of high-quality GEMM Verilog code, addressing the challenges faced by existing models in this domain. (This repo is under development)

Models and Datasets

Quick Start

import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the model and tokenizer
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_name = "/home/python/transformers/GEMMV-DS-8b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name).to(device)
prompt = "FILL IN THE QUESTION"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
# Generate sample with sampling enabled and necessary parameters
sample = model.generate(
    input_ids,
    max_length=4096,
    temperature=0.8,
    top_p=0.9,
    do_sample=True            # Enable sampling to use temperature and top_p
)
response = tokenizer.decode(sample[0], truncate_before_pattern=[r"endmodule"]) + "endmodule"
print("Response:", response)

Citation

If you find GEMMV useful for your research, please cite:

@ARTICLE{10994474,
  author={Zhang, Gaoche and Zou, Dingyang and Sun, Kairui and Chen, Zhihuan and Wang, Meiqi and Wang, Zhongfeng},
  journal={IEEE Journal on Emerging and Selected Topics in Circuits and Systems}, 
  title={GEMMV: An LLM-Based Automated Performance-Aware Framework for GEMM Verilog Generation}, 
  year={2025},
  volume={15},
  number={2},
  pages={325-336},
  keywords={Adders;Hardware design languages;Codes;Hardware;Artificial intelligence;AI accelerators;Register transfer level;Training;Syntactics;Optimization;AI accelerators;design automation;LLMs;fine-tuning;GEMM},
  doi={10.1109/JETCAS.2025.3568712}
}