foszt2oszt

Paper

We publish an abstractive summarizer for Hungarian, an encoder-decoder model initialized with huBERT, and fine-tuned on the ELTE.DH corpus of former Hungarian news portals. The model produces fluent output in the correct topic, but it hallucinates frequently. Our quantitative evaluation on automatic and human transcripts of news (with automatic and human-made punctuation, Tündik et al. (2019), Tündik and Szaszák (2019)) shows that the model is robust with respect to errors in either automatic speech recognition or automatic punctuation restoration. In fine-tuning and inference, we followed a jupyter notebook by Patrick von Platen. Most hyper-parameters are the same as those by von Platen, but we found it advantageous to change the minimum length of the summary to 8 word- pieces (instead of 56), and the number of beams in beam search to 5 (instead of 4). Our model was fine-tuned on a server of the SZTAKI-HLT group, which kindly provided access to it.