combined-literature-en-fi-preview

combined-literature-en-fi-preview

This is an English–Finnish translation model finetuned on a combination of backtranslated literature works.

This model was finetuned with paragraphs of multiple sentences. Therefore, it is recommended that the input sentences are concatenated into batches of several sentences (max ~100 tokens) in order to gain best translation quality.

Intended use

This model is primarily intended for scientific research in the field of machine translation.

Out-of-scope use

This model is not recommended for production use. The training data (see below) includes NSFW content. While the model is not supposed to produce NSFW output with SFW input, this cannot be guaranteed.

Data

Data sources

The data is based on Finnish short stories available online. The largest parts of the data come from the fan fiction website Finfanfun (~14000 short stories), the writing sections of the forum Suomi24 (~12000 short stories), and the erotic fiction website Herkku/Koodi (~11000 short stories). The exact composition of the dataset is available at request.

The stories were deduplicated and underage sexual content was removed.

Data processing

The data was split into paragraphs and each story was translated into English using the cpatonn/Qwen3-30B-A3B-Instruct-2507-AWQ-4bit model with a paragraph-aligned output. The model was asked to produce a table with paragraph-aligned translations and it was forced to output correct input paragraph column using teacher-forcing.

Afterwards, the data was tokenized and the paragraphs were shuffled. The resulting training dataset consists of 562,771 paragraphs tokenized into 40,207,052 Finnish and 39,118,779 English SentencePiece tokens. The validation set consists of 1000 paragraphs randomly sampled from the whole dataset.

Training

The base model was finetuned for 7000 iterations (over 3 epochs) on a single RTX 5090 card using MarianNMT. After iteration 7000, the valid loss stalled and training was stopped. The learning rate was 5e-5.

Evaluation

TODO