meta-llama.Llama-3.1-70B-Instruct_W8A8_FP8

meta-llama/Llama-3.1-70B-Instruct - W8A8_FP8 Compression

This is a compressed model using llmcompressor.

Compression Configuration

• Base Model: meta-llama/Llama-3.1-70B-Instruct
• Compression Scheme: W8A8_FP8
• Dataset: HuggingFaceH4/ultrachat_200k
• Dataset Split: train_sft
• Number of Samples: 512
• Preprocessor: chat
• Maximum Sequence Length: 131072

Sample Output

Prompt:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

<|eot_id|><|start_header_id|>user<|end_header_id|>

Who is Alan Turing?<|eot_id|>

Output:

<|begin_of_text|><|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

<|eot_id|><|start_header_id|>user<|end_header_id|>

Who is Alan Turing?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Alan Turing (1912-1954) was a British mathematician, computer scientist, logician, and philosopher. He is widely considered one of the most influential figures of the 20th century. Turing made significant contributions to various fields, including computer science, artificial intelligence, cryptography, and mathematics.

**Early Life and Education**

Turing was born in London, England, and showed a keen interest in mathematics and science from an early age. He studied mathematics at King's College, Cambridge, where he graduated in 1934. He was elected a scholar of King's College in 1933 and was awarded

Evaluation

<TODO>