llama3-satcom-70b

LLama3 SatCom 70B

LLama3 SatCom 70B is a high-capacity fine-tuned Large Language Model (LLM) developed under the ESA ARTES programme as part of the SatcomLLM / SCEVA (SatCom Expert Virtual Assistant) project.
It represents the largest and most capable SatCom-specialised LLM, purpose-built to support satellite communications (SatCom) professionals through advanced reasoning, multi-step problem solving, and expert-level Q&A.

---

Model Description

• Base model: meta-llama/Llama-3.3-70B-Instruct
• Fine-tuning type: Instruction fine-tuning (IFT)
• Training data: Curated and synthetic SatCom QA, including chain-of-thought annotated datasets
• Architecture: Decoder-only transformer, 70 billion parameters
• Languages: English
• License: LLama-3.3 Community License Agreement

This large-scale variant extends the reasoning and factual accuracy of the 8B model, providing enhanced comprehension of technical systems, higher stability in multi-step mathematical reasoning, and greater contextual understanding across complex SatCom documents.
It excels in tasks such as link budget evaluation, propagation modeling, 5G/6G NTN design, and mission architecture analysis. Thanks to its larger parameter count and extended context window, it can process longer technical passages, integrate multiple data sources, and maintain coherence across complex analytical workflows.

---

Training Datasets

---

Intended Use

Primary use cases:

• Advanced reasoning and Q&A for SatCom system design and analysis
• Automated support for link budget calculations and RF engineering tasks
• Conceptual guidance for 5G/6G NTN and inter-satellite network operations
• Mission design evaluation and anomaly diagnosis assistance
• Research, education, and technical documentation in satellite communications

Intended users:

• ESA engineers and mission planners
• SatCom and aerospace researchers
• System architects and technical operators
• Academic institutions and educational users

---

Limitations

• The model does not access live telemetry or proprietary ESA mission data.
• Generated answers should be validated by domain experts before operational use.
• It is not suitable for safety-critical or real-time decision-making.
• May produce confident but incorrect answers outside its domain

---

Technical Details

---

Evaluation

Evaluation Datasets

The model was evaluated across SatCom-specific and general-domain benchmarks, focusing on mathematical reasoning, conceptual understanding, and applied problem solving.

---

Results

<table> <thead> <tr> <th rowspan="2">Model</th> <th colspan="3">MCQA (Accuracy)</th> <th colspan="3">Satcom-QA</th> <th colspan="3">EVE-QA</th> </tr> <tr> <th>Satcom</th> <th>EVE</th> <th>TeleQnA</th> <th>Norm.</th> <th>Bin.</th> <th>WR</th> <th>Norm.</th> <th>Bin.</th> <th>WR</th> </tr> </thead> <tbody> <tr> <td>Llama-3.3-70B-Instruct</td> <td>87.73</td> <td>85.29</td> <td>74.70</td> <td>87.39</td> <td>85.20</td> <td>—</td> <td>66.44</td> <td>75.30</td> <td>—</td> </tr> <tr style="border-top: 1px dashed #999;"> <td><code>llama3-satcom-70b</code></td> <td><strong>88.08</strong></td> <td><strong>86.62</strong></td> <td><strong>75.03</strong></td> <td><strong>90.84</strong></td> <td><strong>86.80</strong></td> <td><strong>51.01</strong></td> <td><strong>69.01</strong></td> <td><strong>76.40</strong></td> <td><strong>51.22</strong></td> </tr> </tbody> </table>

Table: Evaluation results (%).

• Norm. denotes the normalized score obtained by averaging 1–5 ratings from a panel of LLM judges (Qwen3, gpt-4.1-mini, Mistral-Large-2512, and DeepSeek-V3.2) and scaling to [0,1].
• Bin. is the binary accuracy computed from correctness judgments.
• WR (Adjusted Win Rate) is defined as (wins + 0.5 × ties) / total, based on pairwise comparisons with randomized answer order.
  Multiple-choice performance is measured using standard accuracy.
  All results are averaged over 3–5 runs; standard deviation ≤ 0.25 pp for open-ended QA and ≤ 0.10 pp for MCQA.

---

Summary

LLama3 SatCom 70B combines the linguistic precision of Llama 3.3 with domain-specialised fine-tuning from ESA’s SCEVA project, achieving great reasoning and analytical performance in satellite communications.
It represents a step toward intelligent, domain-grounded AI assistants capable of supporting complex engineering and research workflows across the space communications ecosystem.

---