Llama-2-7b-hf

Llama 2 (7B): Optimized for AMD ROCm

Llama 2 (7B) is Meta's causal language model, served via vLLM's OpenAI-compatible HTTP API. This repository packages evaluation/inference for causal language modeling using vLLM, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.

This is based on the implementation of Llama 2 found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the Llama-2-7B-hf AMD scripts to reproduce results or export with custom configurations.

---

Task Overview

Task: Causal language modeling (text generation)

Dataset: HellaSwag, ARC-Challenge, Winogrande, MMLU, GSM8K (via lm-evaluation-harness)

Output metrics: acc / acc_norm / exact_match per task (not perplexity)

Model variants: Only the base 7B model is auto-bootstrapped.

vLLM note: This version of vLLM does not work on kraken2e devices — ROCm has no supporting kernels for gfx1153.

Gated model: Llama 2 weights cannot be downloaded until you accept Meta's license on Hugging Face and authenticate via hf auth login.

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the Llama-2-7B-hf on GitHub.

---

Model Details

Model Type: Causal language model (decoder-only transformer)

Base Model: meta-llama/Llama-2-7b-hf

---

Performance Summary

No results have been recorded in this checkout yet — run make eval-accuracy-all-slow (or eval-accuracy-fast for a quicker subset), then make metrics to populate METRICS_SUMMARY.md. Meta's published numbers below are for sanity-checking a fresh run; values far outside these ranges signal a pipeline issue, not a model issue.

Expected Accuracy Ranges (Meta reference)

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Scripts for the supported runners
• Additional model variants and datasets
• Benchmarking and reproduction instructions