Downloads · 30 days
81
67% of all-time downloads
ispras-crawlers/newsxlm-xlmroberta-ae
newsxlm-xlmroberta-ae is a token classification model from ispras-crawlers. Use it when you need labels on individual words, such as names. The card lists the license as mit.
[](https://ispranproceedings.elpub.ru/jour/article/view/2164/1961) [](https://huggingface.co/spaces/pbedrin/newsxlm-demo)
Downloads · 30 days
81
67% of all-time downloads
All-time downloads
121
Public
Parameters
277M
1.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.1 GB · 98%
From the Hugging Face model README
This repository contains the fine-tuned XLM-RoBERTa baseline evaluated in the NewsXLM paper for news page HTML node classification.
It labels HTML nodes as title, publication_date, text, author, tag, or none.
pip install -r requirements.txt
html = '''
<html><body>
<nav><a>Home</a><a>News</a></nav>
<main><article>
<h1>Scientists discover new planet beyond our solar system</h1>
<div class="author"><a>John Smith</a></div>
<time>November 3, 2026, 10:30 AM</time>
<div class="body">
<p>Scientists announced the discovery on Tuesday.</p>
<p>The planet is twice the size of Earth.</p>
</div>
<div>Tags: <a>Science</a><a>Space</a></div>
</article></main>
<footer>All rights reserved</footer>
</body></html>
'''
The extraction pipeline groups the results by attribute:
from transformers import pipeline
extractor = pipeline(
"extraction",
model="ispras-crawlers/newsxlm-xlmroberta-ae",
trust_remote_code=True,
)
extracted = extractor(html)
print(extracted)
Example result:
{
"title": ["Scientists discover new planet beyond our solar system"],
"publication_date": ["November 3, 2026, 10:30 AM"],
"text": ["Scientists announced the discovery on Tuesday.", ...],
"author": ["John Smith"],
"tag": ["Science", "Space"]
}
The node-classification pipeline returns a prediction for each HTML node.
from transformers import pipeline
node_classifier = pipeline(
"node-classification",
model="ispras-crawlers/newsxlm-xlmroberta-ae",
trust_remote_code=True,
)
nodes = node_classifier(html)
print(nodes)
Example result:
{
"/html/body/article/h1": {
"pred": "title",
"probs": {
"title": 0.97,
"publication_date": 0.01,
"text": 0.01,
"author": 0.00,
"tag": 0.00,
"none": 0.01
},
"text": "Scientists discover new planet beyond our solar system"
}, ...
}
chunk_batch_size when creating the pipeline or calling it.import torch
from transformers import AutoModelForTokenClassification, AutoProcessor
repo_id = "ispras-crawlers/newsxlm-xlmroberta-ae"
processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForTokenClassification.from_pretrained(repo_id)
model.eval()
inputs = processor(html, return_tensors="pt")
with torch.inference_mode():
outputs = model(**inputs)
If you use NewsXLM in your research, please cite:
@article{newsxlm,
author = {Bedrin, P. A. and Varlamov, M. I. and Yatskov, A. K.},
title = {NewsXLM: A Multilingual Dataset and Model for Information Extraction from News Web Pages},
journal = {Proceedings of ISP RAS},
year = {2026},
volume = {38},
number = {2},
pages = {149--164},
doi = {10.15514/ISPRAS-2026-38(2)-10}
}