customer-concentration-risk-llama4-scout-lora

Model Card: Customer Concentration Risk Analysis — Llama 4 Scout 17B LoRA

Model Details

Model Description

This is a LoRA adapter fine-tuned on top of Llama-4-Scout-17B-16E-Instruct to analyze customer/segment concentration risk from SEC Form 10-K risk factor disclosures. Given a company's disclosed customer or segment concentration percentage across two consecutive fiscal years, the model classifies the trend (worsening, improving, or stable -- a change of ≤1.5 percentage points is treated as stable) and determines whether the disclosure crosses the standard U.S. GAAP/SEC materiality threshold (10% of net sales or receivables) between the two periods, along with a short risk interpretation.

This targets a genuine reasoning gap: correctly classifying a concentration trend and threshold crossing requires actually computing the percentage-point change and comparing both values against a fixed threshold, not just recognizing that "customer concentration" is a risk topic -- a distinction base LLMs often blur, treating the presence of risk language as equivalent to a correctly quantified trend.

Training labels were generated by construction: given the two disclosed percentages, trend_classification and crossed_materiality_threshold follow deterministically from the stated ≤1.5pp-stable rule and the 10% threshold comparison. Every one of the 3,130 rows was independently re-verified against this rule with zero mismatches found.

Submitted to the AutoScientist Challenge (Market Analysis & News track) by Adaption Labs.

• Developed by: Grold Otieno Mboya
• Shared by: Grold Otieno Mboya
• Model type: LoRA adapter for causal language modeling (financial disclosure / concentration-risk reasoning)
• Language(s): English
• License: Llama 4 Community License Agreement (inherited from the base model)
• Finetuned from model: meta-llama/Llama-4-Scout-17B-16E-Instruct (trained via the 4-bit quantized variant togethercomputer/Llama-4-Scout-17B-16E-Instruct_bnb_4bit, per this adapter's own adapter_config.json)

Model Sources

Live Demo: https://riskscreener.adaptionlabs.app/

Uses

Direct Use

Screening a company's customer/segment concentration disclosures for trend direction and materiality-threshold crossings across two fiscal years -- useful for equity/credit analysts doing quick disclosure triage, portfolio managers monitoring concentration exposure across a watchlist, and researchers building datasets from 10-K risk factor sections.

Downstream Use

Could be extended to more than two fiscal years (multi-period trend analysis), other GAAP materiality thresholds (e.g. receivables concentration, geographic concentration), or integrated into a larger disclosure-monitoring pipeline that flags filings for analyst review.

Out-of-Scope Use

Not a substitute for reading the actual 10-K filing or consulting a financial professional before making investment decisions. Not evaluated for general capability, factuality, or safety beyond the concentration-risk task described here. The model reasons only over the two percentages it is given -- it does not verify that those figures are accurately transcribed from a real filing, so output quality depends entirely on the accuracy of the input data supplied to it.

Bias, Risks, and Limitations

The training data combines 30 rows sourced from a real company's actual disclosed filings (Allison Transmission Holdings Inc's disclosed concentration with Daimler AG and Navistar International Corporation / Traton SE, plus a small number of anonymized entries labeled by SEC CIK number) with 3,100 synthetic rows using invented company and customer names but realistic percentage ranges. Every row's trend_classification and crossed_materiality_threshold label was independently re-derived from the two raw percentages using the stated rule and confirmed to match exactly across all 3,130 rows, with zero inconsistencies found.

Known limitation -- real-company representation: only one real company (Allison Transmission Holdings Inc) is meaningfully represented across the 30 real rows; this is not a broad, diverse sample of real SEC filings, and performance on real-world filings from other companies or industries has not been separately measured.

Known limitation -- no source verification at inference time: the model reasons only over whatever percentages it is given in the prompt. It does not check those figures against a real filing, so it cannot catch a transcription error or a fabricated input -- output correctness depends on the accuracy of the numbers supplied by the user.

Known limitation -- single fixed threshold: the model is trained specifically around the 10% GAAP/SEC materiality convention and the ±1.5 percentage-point stability rule; it has not been evaluated against company- or industry-specific materiality thresholds that may differ from this default.

Recommendations

Treat outputs as a fast triage/screening aid, not a substitute for reading the underlying filing. Verify any input figures against the actual 10-K before relying on the trend classification, particularly for real-world use outside the training distribution's single real-company anchor.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
adapter_id = "Gro97/customer-concentration-risk-llama4-scout-lora"

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

system_prompt = (
    "You are a financial analyst specializing in SEC disclosure analysis. "
    "Given a company's disclosed customer or segment concentration-risk "
    "percentage across two consecutive fiscal years, and the standard U.S. "
    "GAAP materiality threshold for concentration disclosure (10% of net "
    "sales or receivables), determine whether the risk is worsening, "
    "improving, or stable, and whether it crosses the materiality threshold "
    "between the two periods. A change of 1.5 percentage points or less "
    "should be classified as stable. Respond only with strict JSON with "
    "exactly these keys: company, customer_or_segment, "
    "disclosed_materiality_threshold_pct, period_a_year, period_a_value_pct, "
    "period_b_year, period_b_value_pct, trend_classification ('worsening', "
    "'improving', or 'stable'), crossed_materiality_threshold (boolean), "
    "risk_interpretation. Base your answer only on the two reported values given."
)
user_prompt = (
    "ExampleCo's Form 10-K risk factors disclose customer concentration "
    "risk under the standard SEC/GAAP materiality threshold of 10.0% of "
    "net sales. For fiscal year 2022, ExampleCo reported that Customer X "
    "accounted for 8.2% of net sales. For fiscal year 2023, ExampleCo "
    "reported that Customer X accounted for 12.7% of net sales."
)

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=300)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

Custom-built dataset customer_concentration_risk_10k: 3,130 rows.

Average prompt length 58 words, average completion length 51 words. Every row's trend_classification and crossed_materiality_threshold label was independently recomputed from the two disclosed percentages and confirmed to match exactly across all 3,130 rows.

• Hugging Face: https://huggingface.co/Gro97/customer-concentration-risk-llama4-scout-lora
• Kaggle: https://www.kaggle.com/models/groldotieno/customer-concentration-risk-llama4-scout-lora

Training Procedure

Fine-tuned via Adaption Labs' AutoScientist platform using LoRA.

Training Hyperparameters

• Training regime: 4-bit quantized base model (bnb_4bit) with LoRA adapters trained in higher precision (QLoRA-style)
• LoRA rank / alpha / dropout / target modules / epochs: see adapter_config.json and trainer_state.json in this repository for the exact configuration used for this run -- not independently re-confirmed in this write-up.

Speeds, Sizes, Times

Not independently benchmarked by the author; training was run on Adaption Labs' hosted infrastructure. Exported adapter size: ~850MB (adapter_model.safetensors).

Evaluation

Testing Data, Factors & Metrics

Testing Data

Two evaluation sets, as reported by Adaption Labs' AutoScientist platform: (1) a held-out split of this project's own dataset, (2) a broader set of unseen tasks from Adaption's internal Market Analysis & News category benchmark (labeled "Corporate & Business" in the platform's own domain taxonomy -- the same evaluation category, reported under Adaption's internal domain tag rather than the challenge's category name).

Metrics

Pairwise preference win rate against the un-adapted base model, as reported by Adaption Labs' AutoScientist platform (judge methodology not disclosed to the end user).

Results

Summary

The adapter shows a substantial improvement over the base model both on its own training distribution (64% win rate) and, more notably, an even larger improvement on the broader category benchmark (72% win rate) -- the broader-category gain exceeding the own-dataset gain is a distinct pattern from this author's other submissions, and is worth noting rather than assuming it simply generalizes better: it may also reflect that the broader category's baseline (28%) started weaker than this dataset's own baseline (36%), giving more room for improvement rather than necessarily indicating stronger transfer.

Environmental Impact

• Hardware Type: Not disclosed by the training platform for this run.
• Hours used: Not disclosed.
• Cloud Provider: Adaption Labs' hosted infrastructure.
• Compute Region: Not disclosed.
• Carbon Emitted: Not calculated.

Technical Specifications

Model Architecture and Objective

LoRA adapter applied to meta-llama/Llama-4-Scout-17B-16E-Instruct (a Mixture-of-Experts architecture, trained here via a 4-bit quantized variant), trained via supervised fine-tuning to classify customer concentration risk trends and materiality-threshold crossings from SEC 10-K disclosure data.

Compute Infrastructure

Software

• PEFT 0.15.1
• Trained via Adaption Labs' AutoScientist platform

Citation

BibTeX:

@misc{mboya2026concentrationrisk,
  author = {Mboya, Grold Otieno},
  title = {Customer Concentration Risk Analysis: A LoRA Adapter for
           SEC 10-K Disclosure Trend Classification},
  year = {2026},
  howpublished = {AutoScientist Challenge submission, Adaption Labs},
  url = {https://huggingface.co/Gro97/customer-concentration-risk-llama4-scout-lora}
}

Model Card Contact

See author's Hugging Face profile: https://huggingface.co/Gro97

Framework versions

• PEFT 0.15.1