Sustainable-Finance-BERT

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Sustainable-Finance-BERT is a fine-tuned BERT model for classifying text documents into categories of sustainable finance and non-sustainable finance. It assigns labels to input text, indicating whether the content aligns with sustainable finance standards (label_0) or non-sustainable finance standards (label_1).

Model Details

1. Architecture: BERT (Bidirectional Encoder Representations from Transformers)

2. Training Approach: Fine-tuning on top of the pre-trained BERT model using a binary classification objective.

3. Pre-trained Model: The model was initialized with weights from a pre-trained BERT model: 'bert-base-uncased'.

4. Fine-tuning Data: The model was fine-tuned on a dataset of 14,000 text samples from sustainable finance standards and non-sustainable finance standards.

5. Fine-tuning Objective: Binary classification, with label_0 indicating sustainable finance and label_1 indicating non-sustainable finance.

6. Tokenization: Utilized BERT's tokenization scheme, which breaks down input text into subword tokens and converts them into numerical representations suitable for model input.

7. Optimizer: Adam optimizer with a learning rate of 2e-5.

8. Loss Function: Cross-entropy loss was employed as the optimization criterion during training.

9. Training Duration: The duration of training may vary depending on the size of the dataset, hardware resources, and convergence criteria.

10. Hyperparameters: Parameters such as batch size:16, learning rate:2e-5, and number of training epochs:4 were tuned during the fine-tuning process to optimize model performance.

Model Description

This model is capable of analyzing textual content and assigning labels indicating whether the material aligns with sustainable finance standards (label_0) or non-sustainable finance standards (label_1).

• Developed by: Pelumioluwa Abiola
• Model type: Fine-tuned BertForSequenceClassification for text classification
• Language(s) (NLP): Python, utilizing Hugging Face's Transformers library
• Finetuned from model [optional]: Pre-trained BERT model - BertForSequenceClassification

This model offers a powerful tool for automatically categorizing finance-related documents, aiding financial institutions, researchers, policymakers, and other stakeholders in identifying content relevant to sustainable finance initiatives. It can facilitate decision-making processes, risk assessment, and compliance monitoring in the finance sector.

Model Sources [optional]

Model Sources [optional]

For additional information and resources related to the model, please refer to the following links:

• Repository: Sustainable_Finance_Analyzer GitHub Repository
• Guidance: This model was guided by Chris McCormick's series on BERT, available here.

These resources above provide valuable insights into the development, usage, and fine-tuning of the Sustainable-Finance-BERT model. Additionally, the GitHub repository contains data cleaning and usage guidance for the model, facilitating its implementation and integration into various applications.

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Uses

The Sustainable-Finance-BERT is for automated classification of text documents into categories of sustainable finance and non-sustainable finance. It serves various purposes and can be directly utilized in several contexts:

Direct Use

Financial Institutions:

• Risk Assessment: Financial institutions can use the model to assess the sustainability of their investment portfolios by classifying documents related to financial products, companies, or projects.
• Compliance Monitoring: It aids in compliance monitoring with sustainable finance regulations and standards by automatically categorizing documents according to sustainability criteria.

Researchers:

• Trend Analysis: Researchers can analyze trends and developments in sustainable finance by classifying large volumes of textual data, such as news articles, research papers, and policy documents.
• Identifying Best Practices: The model helps identify best practices and emerging themes in sustainable finance initiatives by categorizing relevant literature and reports.

Policymakers:

• Policy Evaluation: Policymakers can evaluate the effectiveness of sustainable finance policies and initiatives by categorizing documents discussing their implementation and impact.
• Policy Formulation: It assists in formulating new policies and regulations related to sustainable finance by analyzing textual data on industry standards.

Environmental, Social, and Governance (ESG) Analysts:

• ESG Integration: ESG analysts can integrate the model into their workflow to quickly screen companies and investment opportunities based on their alignment with sustainable finance principles.
• Performance Evaluation: It facilitates the evaluation of companies' ESG performance by classifying sustainability reports, disclosures, and corporate communications.

Educational Institutions:

• Curriculum Development: Educational institutions can use the model to develop curriculum materials on sustainable finance topics by categorizing relevant literature and case studies.
• Student Projects: Students can utilize the model for research projects and assignments focusing on sustainable finance trends, policies, and practices.

Foreseeable Users

• Financial Analysts: Professionals involved in financial analysis, investment management, and risk assessment.
• Sustainability Specialists: Individuals working in sustainability consulting, corporate sustainability, and environmental advocacy.
• Policy Analysts: Experts involved in policy research, advocacy, and government advisory roles.
• Data Scientists and Machine Learning Engineers: Professionals working in the development and deployment of natural language processing (NLP) models.
• Academic Researchers: Scholars conducting research in finance, economics, sustainability, and related fields.

The Sustainable-Finance-BERT has broad applicability across various sectors, providing valuable insights and facilitating informed decision-making in the realm of sustainable finance.

Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

The Sustainable-Finance-BERT can be further fine-tuned for specific tasks or integrated into larger ecosystems and applications to serve diverse purposes. Below are potential downstream uses of the model:

1. Fine-tune the model to align with specific regulatory frameworks and sustainability standards relevant to different jurisdictions or industry sectors.
2. Analyze trends and patterns in sustainable finance discourse by applying the model to large-scale textual datasets, identifying emerging topics, key influencers, and evolving narratives.
3. Fine-tune the model further based on specific criteria or preferences of investors, allowing for personalized recommendations and portfolio customization.

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

While the model excels in classifying text documents into categories of sustainable finance and non-sustainable finance, there are certain uses that fall out of its scope or may not yield optimal results:

• Sentiment Analysis: The model is not specifically designed for sentiment analysis tasks and may not accurately capture sentiment nuances in text related to sustainable finance.
• Topic Modeling: While the model can identify documents relevant to sustainable finance, it may not be suitable for topic modeling tasks requiring finer granularity in identifying specific themes or topics within the domain.
• Legal Compliance: The model should not be solely relied upon for legal compliance purposes, as it may not capture all regulatory nuances or legal requirements relevant to sustainable finance.
• Highly Specialized Domains: Use of the model in highly specialized domains outside the scope of sustainable finance may yield suboptimal results, as it is specifically trained on data from this domain.

It's important to consider the model's limitations and ensure that its use aligns with its intended scope and capabilities to achieve the best outcomes.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[More Information Needed]

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]