horizon-intervention_areas-classifier

nicolauduran45/horizon-intervention_areas-classifier

horizon-intervention_areas-classifier is a text classification model from nicolauduran45. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.

This model is based on SPECTER2-base, fine-tuned for multilabel classification of scientific publications into Horizon Europe clusters.

Downloads · 30 days

5% of all-time downloads

All-time downloads

128

Public

Parameters

110M

440 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors440 MB · 100%

At a glance

Task: Text Classification
Library: transformers
License: apache-2.0
Model type: bert
Access: Public
Created: Jul 16, 2025
Updated: Jul 16, 2025
SHA: 4a152504

Try a prompt

Base models

allenai/specter2_base

Datasets

nicolauduran45/horizon_clusters_annotated

Model card

From the Hugging Face model README

SPECTER2-base Multilabel Horizon Intervention Areas Classifier

This model is based on SPECTER2-base, fine-tuned for multilabel classification of scientific publications into Horizon Europe clusters.

Model Description

Base model: allenai/specter2_base
Task: Multilabel classification (assigns one or more clusters per document)
Labels: 36 Horizon Europe intervention areas (see below)
Languages: English
Input: Title and abstract concatenated

Training Details

Training framework: Hugging Face Transformers (Trainer)
Batch size: 4
Learning rate: 2e-5
Epochs: 6
Optimizer: AdamW with weight decay 0.01
Loss: Binary Cross-Entropy with Logits
Best model selection: F1-score on validation set

Evaluation Metrics

Epoch	Training Loss	Validation Loss	F1	ROC AUC	Accuracy
1	No log	0.1823	0.280	0.582	0.093
2	0.1912	0.1605	0.493	0.676	0.151
3	0.1133	0.1390	0.593	0.732	0.236
4	0.0902	0.1316	0.644	0.762	0.281
5	0.0740	0.1221	0.697	0.791	0.334
6	0.0619	0.1216	0.722	0.809	0.374
7	0.0535	0.1204	0.741	0.820	0.382
8	0.0467	0.1195	0.750	0.826	0.414
9	0.0422	0.1188	0.759	0.830	0.430
10	0.0384	0.1184	0.765	0.834	0.435

Best epoch: 10 (highest F1/accuracy)

Final validation loss: 0.1184

Final F1: 0.7647

Final ROC AUC: 0.8344

Final Accuracy: 0.4350

Per-Category Classification Report

Label	Precision	Recall	F1-score	Support
Advanced Materials	1.00	0.71	0.83	38
Advanced computing and big data	0.85	0.74	0.79	39
Agriculture, forestry and rural areas	0.86	0.60	0.71	20
Artificial intelligence and robotics	0.85	1.00	0.92	17
Bio-based innovation systems in the bioeconomy	0.83	0.59	0.69	17
Biodiversity and natural resources	1.00	0.14	0.25	28
Buildings and industrial facilities in energy transition	0.79	0.52	0.62	29
Circular Industries	0.00	0.00	0.00	6
Circular systems	0.52	0.74	0.61	19
Clean, safe and accessible transport and mobility	0.93	0.78	0.85	18
Climate science and solutions	0.82	0.80	0.81	41
Communities and cities	0.87	0.61	0.72	44
Culture, cultural heritage and creativity	0.97	0.87	0.92	45
Cybersecurity	1.00	0.33	0.50	3
Democracy and Governance	1.00	0.67	0.80	18
Disaster-resilient societies	1.00	0.48	0.65	27
Emerging enabling technologies	0.84	0.71	0.77	82
Energy storage	0.00	0.00	0.00	4
Energy supply	0.62	0.89	0.73	37
Energy systems and grids	0.83	0.66	0.73	44
Environmental and social health determinants	1.00	0.58	0.74	12
Environmental observation	1.00	0.56	0.72	16
Food systems	0.00	0.00	0.00	11
Health throughout the life course	1.00	0.63	0.77	19
Healthcare systems	0.96	0.83	0.89	29
Industrial competitiveness in transport	1.00	0.20	0.33	5
Infectious diseases, including poverty-related and neglected diseases	0.93	0.93	0.93	15
Key digital technologies	0.94	0.66	0.78	89
Manufacturing technologies	0.98	0.94	0.96	49
Net-zero and less polluting Industries	0.80	0.25	0.38	16
Next generation internet	0.78	0.54	0.64	13
Non-communicable and rare diseases	0.96	1.00	0.98	22
Protection and security	1.00	0.57	0.73	28
Seas, oceans and inland waters	1.00	0.26	0.42	19
Smart mobility	0.80	0.67	0.73	12
Social and economic transformations	0.90	0.64	0.75	59
Space, including Earth observation	1.00	0.55	0.71	11
Tools, technologies and digital solutions for health and care, including personalised medicine	0.98	0.92	0.95	49
micro avg	0.88	0.68	0.76	1050
macro avg	0.83	0.59	0.67	1050
weighted avg	0.88	0.68	0.75	1050
samples avg	0.85	0.72	0.76	1050

License

This model is licensed under the Apache License 2.0.

Task: Text Classification
Library: transformers
Type: bert
License: apache-2.0
Created: Jul 16, 2025
Updated: Jul 16, 2025