ForSureTesterSim

Llama-3.1-8B-TransMLA-PreRecovery

ForSureTesterSim/Llama-3.1-8B-TransMLA-PreRecovery

Llama-3.1-8B-TransMLA-PreRecovery is a text generation model from ForSureTesterSim. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.

We simply applied the methodology from the paper titled "TransMLA: MLA Is All You Need" to convert the attention mechanism within the Llama-3 model from Grouped Query Attention (GQA) to Multi-Head Latent Attention (ML…