AMALIA-9B-1225-DPO

<div align="center"> <img width="400px" src="https://github.com/AMALIA-LLM/amalia-llm.github.io/blob/main/source/_static/logo/logo-color-black.png?raw=true">

</div>

Model Card

AMALIA is an open-source language model targeting European Portuguese.

Model Description

AMALIA is developed by a consortium of Portuguese universities and research centres, including NOVA University Lisbon, Instituto Superior Técnico, the University of Coimbra, the University of Porto, the University of Minho, and the Foundation for Science and Technology (FCT). Development also includes collaborations with the University of Beira Interior, the University of Évora, and the Lisbon School of Engineering (ISEL).

This project is funded by the Government of Portugal's Development and Innovation Programmes, with the goal of creating an effective, sovereign, and transparent LLM, tailored for European Portuguese.

AMALIA uses open-source data in its training, such as data from Arquivo.pt, along with curated data specifically in European Portuguese. Training was carried out in two phases: pre-training and post-training. Post-training applied Supervised Fine-Tuning (SFT) and Preference Tuning techniques (using Direct Preference Optimization, DPO).

Training Details

Training Data

The following data were included in AMALIA's pre-training phase:

• Data from Arquivo.pt, obtained after processing collections of public web pages and free books;
• Pre-training data from the EuroLLM model;
• Long-context data samples from Stack-v2;
• Synthetic data to improve long-context retention, filtered from public datasets (needle-llama3-16x8k, needle_32k_finetuning_dataset).

The post-training SFT component used a mixture of manually created, synthetically generated, and publicly available data. For the DPO component, a preference dataset was created, including on-policy pairs, created by leveraging responses generated in part by AMALIA-SFT itself. We release the SFT and DPO datasets. For full details on both training datasets, please refer to the technical report.

Training Process

We extend EuroLLM pretraining to better cover European Portuguese knowledge and extend the maximum sequence length to 32k tokens.

The SFT phase aims to improve AMALIA's conversational and instruction-following capabilities, with a special focus on European Portuguese, using the datasets described in Training Data. Training ran for 76 hours, using 64 NVIDIA H100 GPUs, for a total of 14k steps.

The DPO phase aims to align AMALIA's behaviour with human preferences, refining the model's responses through learning based on pairwise comparisons. In this phase, the model learns to distinguish between higher- and lower-quality responses to the same instruction, optimizing itself to generate more useful, safe, and value-aligned outputs, while simultaneously minimizing undesired behaviours such as hallucinations, toxicity, or deviations from the given instructions. Training ran for 12 hours, using 64 NVIDIA H100 GPUs. All benchmark results can be accessed in the technical report.

All training phases were carried out on the MareNostrum5 supercomputer hosted at the Barcelona Supercomputing Center and the DEUCALION supercomputer hosted at Minho Advanced Computing Center.

Intended Use

AMALIA is intended as a general-purpose, open language model for European Portuguese (pt-PT). Its primary intended uses include:

• Conversational assistance and instruction following in European Portuguese: question answering, summarization, drafting, rewriting, translation, and general text generation.
• Research and educational applications focused on the Portuguese language and culture, including Portuguese-language NLP research and the study of European Portuguese specifically.
• A base model for downstream development. Given its open release, AMALIA is intended to be fine-tuned, adapted, and built upon by other developers and researchers, for example, in public-sector and sovereign AI applications.

The model targets European Portuguese specifically. While it will handle other Portuguese variants to some degree, it is optimized and curated for pt-PT and should not be assumed equivalent across variants.

Out-of-Scope Use

The following uses fall outside what AMALIA is designed or validated for:

• High-stakes, unsupervised decision-making without qualified human oversight. AMALIA should not be used as a decision-making authority in domains where errors carry serious consequences, without a qualified human in the loop.
• Use as a sole source of factual truth. The model can hallucinate. Outputs should be verified before use in any context that demands factuality. AMALIA was trained to support retrieval-augmented generation (RAG), so grounding responses in retrieved source documents is the recommended approach for factual or knowledge-intensive tasks. Even so, outputs should be checked in any factual-critical context.
• Languages and cultural contexts other than Portuguese and English.
• Reliance on safety guarantees. Safety tuning reduces but does not eliminate harmful, toxic, or biased outputs; the model should not be deployed in settings that assume it is fully safe by default. Instead, developers should employ appropriate guardrails and safety mechanisms tailored to their application, and ensure logging and traceability.
• Generation of harmful, toxic or violated content. AMALIA must not be used to intentionally produce content that is harmful, abusive, discriminatory, or otherwise violating, including hate speech, harassment, or material that exploits or endangers individuals.
• Production deployment without further evaluation. As an open model, AMALIA should undergo task-specific and domain-specific testing before being deployed in any production system.
• Uses that violate applicable law. AMALIA must not be deployed in ways that conflict with the EU AI Act, the General Data Protection Regulation (GDPR), or other applicable legal frameworks. This includes prohibited AI practices under the AI Act, namely all high-risk applications that have not undergone the required conformity assessment, and any processing of personal data without a valid legal basis.

Ethical Considerations and Risk

Developing large language models raises a number of ethical concerns. By releasing AMALIA as an open model for European Portuguese, the consortium considered the following:

• Bias and fairness. Language models trained on large-scale, real-world text reflect the socio-cultural biases present in their training material. The model is also explicitly targeted at European Portuguese, and its handling of the pt-PT/pt-BR distinction is itself a fairness consideration that the consortium treats as a first-class evaluation dimension.
• Misinformation and misuse. Like any LLM, AMALIA can produce text that is false, misleading, or harmful. Developers building on AMALIA are encouraged to communicate these limitations to end users and to provide mechanisms for reporting misuse.
• Transparency and accountability. This card documents the model's training data, training process, and intended and out-of-scope uses so that developers and researchers can make informed decisions. Releasing AMALIA openly is intended to make European Portuguese language technology accessible to the wider research and developer community.

Risks identified and mitigations:

• Generation of harmful content. AMALIA's post-training included safety data in both the SFT and DPO phases, and the DPO phase explicitly targets reductions in toxicity and harmful output. These measures reduce but do not eliminate the risk; downstream developers should add their own content-safety safeguards appropriate to their product and use case.
• Misuse for malicious purposes. The consortium provides documentation of the model's intended use and limitations, and encourages developers who build on AMALIA to establish their own mechanisms for users to report misuse within their applications.
• Perpetuation of biases. Continuous monitoring, through evaluation metrics and human review, and the use of de-biasing techniques during training, fine-tuning, and downstream adaptation were encouraged to limit the reinforcement of existing biases.
• Adversarial inputs and prompt injection. Applications built on AMALIA may be vulnerable to prompt injection and other adversarial inputs. Developers should apply input validation and output filtering appropriate to their deployment context.

Citation

If you use AMALIA in your work, please cite:

@inproceedings{simplicio-etal-2026-amalia,
    title = "{AMALIA}: A Fully Open Large Language Model for {E}uropean {P}ortuguese",
    author = "Simpl{{\'i}}cio, Afonso and Vinagre, Gon{{\c{{c}}}}alo and Ramos, Miguel Moura and Tavares, Diogo and Ferreira, Rafael and Attanasio, Giuseppe and Alves, Duarte M. and Calvo, In{{\^e}}s and Vieira, In{{\^e}}s and Guerra, Rui and Furtado, James and Canaverde, Beatriz and Paulo, Iago and Ramos, Vasco and Gl{{\'o}}ria-Silva, Diogo and Faria, Miguel and Treviso, Marcos and Gomes, Daniel and Gomes, Pedro and Semedo, David and Martins, Andr{{\'e}} and Magalh{{\~a}}es, Jo{{\~a}}o",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {{P}}ortuguese ({{PROPOR}} 2026) - Vol. 1",
    month = apr,
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.propor-1.38/",
    pages = "380--391",
    isbn = "979-8-89176-387-6"
}