parliament_classifier_colonial

Colonial Parliamentary Debate Classifier

Model Description

parliament_classifier_colonial is a fine-tuned roberta-large model that has been trained to identify parliamentary debates on colonial topics. We take the initialized roberta-large weights and fine tune on a custom dataset of titles of parliamentary debates drawn from the Historic Hansard.

Usage

This model can be used with HuggingFace Transformer's Pipelines API for text classification:

from transformers import pipeline

classifier = pipeline(
"text-classification",
 model='imperial-science/jstor_classifier_colonial',
 device=0,
 truncation=True,
 max_length=512
)

example = """
SPAIN—CUBA—IMPORTATION OF COOLIES FROM THE BRITISH WEST INDIES.—QUESTION.
"""

def classify(title):
    """
    Classify text(s) as either 'COLONIAL' or 'NOT'.

    Parameters:
    article (str or list of str): The text(s) to be classified. Can be a single string
                                  or a list of strings.

    Returns:
    list of dict: A list of dictionaries, each containing:
                  - 'label' (str): Either 'COLONIAL' or 'NOT'
                  - 'score' (float): The confidence score of the classification
  
    """
    if isinstance(article, str):
        article = [article]
    
    results = classifier(article)
    
    output = []
    for result in results:
        label = "COLONIAL" if result["label"] == "LABEL_1" else "NOT"
        output.append({'label': label, 'score': result["score"]})
    
    return output


print(classify(example))

Training data

The custom dataset consists of 2,328 titles of parliamentary debates drawn from the Historic Hansard corpus between 1802-1899.

Labels were generated by GPT-4o with the following prompt and a subsample of one-third was checked by a graduate student:

You are an expert in labelling for text classification, trained to analyze British parliamentary debate titles between 1802-1899.
        
The debate titles may have OCR errors.
        
Your task is to classify a given article based on its focus.
        
Specifically, you should determine if the title suggests that it is a debate on topics relating to the British Empire, British imperialism, scientific exploration, colonists and settlers, indigenous people, or other colonial topics relating to Britain (including those related to Ireland).
        
If the title is vague e.g. "QUESTIONS" or "BUSINESS OF THE HOUSE", then it is not a debate relating to our topics of interest.
        
Do not provide an explanation. Simply respond in JSON format, so that if the debate title suggests a debate on the topics from above, respond with 'response' of ‘yes’. If it is ‘not’, set 'response' to ‘no’.
        
Here is the debate title text:

We make no attempt to correct for OCR errors in any way beforehand, so that the model is exposed to these errors in training.

Our class balance for the labels was: 24.1% 'yes', and 75.9% 'no'.

The data was randomly split 60-20-20 (train-val-test), and fine tuning was done with a batch size of 160 and learning rate of 2e-5 for 40 epochs.

Evaluation

The F1 on the test set was 95.5%.