Credless_env_blog

FinVerse: Multi-Agent Credit Investigation Environment for OpenEnv Hackathon Problem Statement: Beyond Binary Classification Traditional credit scoring systems exclude millions of people lacking formal credit histories. While LLMs show promise in financial decision-making, they often fall into shallow next-token reasoning and exploit shortcuts in simple classification tasks.

The Real Challenge: Can we build environments that force models to act as professional investigators, maintaining consistent internal state and performing "real work" in partially observable, multi-step scenarios?

What We Built: FinVerse Environment FinVerse (evolved from CredLess-Env) is a professional credit investigation pipeline built on the OpenEnv framework, targeting Theme #3.1 (Professional Tasks) and Theme #2 (Long-Horizon Planning).

Environment Architecture

Data CSV / Synthetic Applicant ↓ Feature Cleaning + Engineering ↓ Agent 1 Training ↓ credless_model/model.pkl ↓ OpenEnv API /reset ↓ CreditAnalystEnvironment ↓ Applicant + Market Scenario ↓ Observation Payload ↓ Agent 1 Oracle Scoring ↓ Risk Score + Confidence + Top Factors ↓ Agent 2 Policy ↓ Request Info / Query Market / Flag Fraud ↓ Terminal Decision ↓ Approve / Deny / Escalate ↓ Grader + Auditor ↓ Oracle Agreement + Policy Compliance ↓ Reward + Penalties ↓ Episode Score ↓ JSONL Results + Summary Metrics

Episode Structure:

5-8 step investigation per credit applicant 20+ engineered real-world behavioral features (150K samples from real financial data) Partially observable: Features start hidden; agent must strategically reveal data Dynamic market conditions: Economic context affects risk assessment Adversarial behaviors: Fabricated fields, withheld information, fraud patterns

Action schema in project:

credless_env/models/init.py

{ "action_type": "request_info | query_market | flag_fraud | approve | deny | escalate", "params": {}, "reasoning": "" }

Meaning per action:

{ "action_type": "request_info", "params": {"field": "<missing field name>"} }

{ "action_type": "query_market", "params": {} }

{ "action_type": "flag_fraud", "params": {"reason": "..."}, "reasoning": "..." }

{ "action_type": "approve", "params": {"tier": "optional", "rate": "optional float"}, "reasoning": "..." }

{ "action_type": "deny", "params": {"tier": "optional", "rate": "optional float"}, "reasoning": "..." }

{ "action_type": "escalate", "params": {"review_note": "optional"}, "reasoning": "..." }

Observation schema in project:

credless_env/models/init.py

{ "applicant": {}, "conversation_history": [], "action_history": [], "oracle_risk": 0.0, "oracle_confidence": 0.0, "market_visible": false, "market_queried": false, "fraud_checked": false, "market_state": null, "current_policy": {}, "compliance_history": [], "step": 0, "max_steps": 8, "fraud_flags_raised": [], "step_reward": 0.0, "cumulative_reward": 0.0, "done": false, "message": "", "episode_score": 0.0, "task_name": "binary_decision" }

Important applicant sub-structure: { "applicant": { "applicant_id": "...", "profile": { "debt_burden_score": { "value": 0.65, "confidence": 0.92 } }, "missing_fields": ["income_capacity_score", "overdraft_risk"] } }

Multi-Agent Architecture (Theme #1)

Agent 1: The Oracle (Ground Truth) Type: Logistic Regression Performance: Role: Provides deterministic risk scores and confidence bounds Features: 42 engineered behavioral signals from real lending data

Key Capabilities:

predict_risk(features) → Default-risk score [0.0, 1.0] Returns positive-class probability from logistic regression.

get_confidence(features) → Model certainty Returns max class probability from logistic regression.

get_top_factors(features, top_k=5) → Feature importance explanation Uses SHAP when available. Falls back to logistic-regression coefficient contribution.

Function meaning: predict_risk Input: feature dictionary keyed by engineered feature names. Output: probability applicant defaults.

get_confidence Input: same feature dictionary. Output: highest class probability. Higher value means stronger certainty.

get_top_factors Input: same feature dictionary. Output: top contributing features like:

[ ["debt_burden_score", 0.184221], ["payment_reliability", -0.151002], ["overdraft_risk", 0.102884] ]

Positive value- pushes toward higher risk. Negative value:pushes toward lower risk.

Feature Importance / Oracle Decision Screenshot

Risk score output:

Oracle risk score: 0.76
Oracle confidence: 0.90
Decision: REJECT

Explanation output:

Agent 2: The Decision Policy (LLM)

Agent 2 acts as the Decision Agent (or Loan Officer), responsible for synthesizing complex financial data into a final terminal decision: Approve or Deny Base Models Tested:

Qwen/Qwen2.5-0.5B-Instruct (primary - optimized for <200ms inference) Llama-3.1-8B-Instruct (alternative)

Training Pipeline:

Supervised Fine-Tuning (SFT): Format priming on 4K oracle-labeled examples Group Robust Policy Optimization (GRPO): Behavioral improvement via RL

Training Flow / Learning Loop

State → Action → Reward → Feedback → Policy Update

Agent 2 Implementation:

Agent 2 supervised training results : (images/agent2.png)

Lightweight Backend: SGDClassifier trained on Agent1 outputs + SHAP values Feature Engineering: Risk score + 40 numeric fields + SHAP statistics (mean/max/min/count) Explainability: By training on these SHAP values, Agent 2 learns to identify why an applicant is risky (e.g., identifying if overdraft_count was the primary driver for a rejection) Prompt Dataset: 4K+ samples exported to JSONL for HuggingFace TRL training

Agent Extensions (Bonus Criteria)

Auditor Agent (Fleet AI - Scalable Oversight):

Monitors Agent2 decisions for bias, consistency violations, and policy drift Generates audit trails and compliance reports Scores decisions on interpretability and fairness metrics

Market Agent (Dynamic World):

Simulates economic shifts (recession, boom, sector-specific shocks) Updates risk thresholds based on macroeconomic indicators Adds temporal complexity to long-horizon planning

Reward Function: Anti-Hacking & Verification Reward Function: Anti-Hacking & Verification Dense Reward Explanation=(images/rewards-explanation.png)

Positive Rewards

Penalties

Demo Details

<img src="images/demo_details.png" alt="Agent 2 supervised training results" /> Enforce valid policy outputs Oracle mismatch -0.3 Align with ground truth

Anti-Hacking Mechanisms Multiple independent reward checkers (environment + external grader) Restricted execution sandbox (no file system access, no globals) Deterministic oracle (prevents exploitation of stochasticity) Episode diversity (150K unique applicant profiles)

Colab Notebook Structure:

CELL 1: Dependencies

!pip install unsloth[colab-new] trl peft xformers

CELL 2-4: Load repository + validate checkpoint

- Extract agent2-decision-base.zip

- Verify SFT checkpoint structure (PEFT vs merged)

- Load Agent1 oracle + environment

CELL 5: Generate rollouts

- 256-512 episodes from environment

- Mix task types: binary_decision, adaptive_inquiry

- Collect: (observation, action, reward, oracle_risk)

CELL 6-7: Build GRPO dataset

- Convert transitions to (prompt, response, reward) tuples

- Implement compute_grpo_reward() matching environment logic

CELL 8-9: Load Qwen SFT checkpoint

model, tokenizer = FastLanguageModel.from_pretrained(

"Qwen/Qwen2.5-0.5B-Instruct",  # or path to SFT checkpoint

max_seq_length=512,

load_in_4bit=True

)

CELL 10-11: GRPO Trainer

from trl import GRPOConfig, GRPOTrainer

trainer = GRPOTrainer(

model=model,

tokenizer=tokenizer,

train_dataset=grpo_dataset,

reward_function=compute_grpo_reward,

args=GRPOConfig(

    per_device_train_batch_size=2,

    gradient_accumulation_steps=8,

    learning_rate=5e-6,  # Lower than SFT

    num_train_epochs=3

)

)

CELL 12-13: Train + validate

trainer.train()

model.save_pretrained("/content/grpo_final")

Key Implementation Details:

SFT Checkpoint Loading: Handles both PEFT adapter and merged checkpoints Tokenizer Fixes: Automatically sets pad_token = eos_token for Qwen Memory Optimization: 4-bit quantization + gradient checkpointing for Colab T4 Reward Alignment: compute_grpo_reward() replicates environment reward logic exactly Error Handling: Proactive fixes for OOM, import errors, tokenizer mismatches

Reward Curve & Metrics Baseline Performance (Pre-GRPO) Inference on 500 episodes (PPO placeholder data):

{

"mean_reward": 0.5629,

"std_reward": 0.1588,

"approve_rate": 0.606,

"oracle_agreement": 0.78

}

Per-Batch Trends:

Batch 0-96: Mean reward 0.51-0.60 (exploration phase) Batch 96-224: Stabilization around 0.55 (learning terminal conditions) Batch 224-480: Improvement to 0.60+ (policy refinement) Expected GRPO Improvements Target Metrics (Post-Training):

Mean reward: 0.70+ (vs 0.56 baseline) Oracle agreement: 0.85+ (vs 0.78 baseline) Average episode length: 4.2 steps (vs 5.1 baseline) Fraud detection F1: 0.75+ (new capability)

Observable Evidence:

Reward curve plot: Batch-wise mean rewards over 3 GRPO epochs Before/after demo: Same applicant, different policy behaviors Ablation study: Oracle-only vs Agent2-only vs blended decisions

Dataset & Features (Open Innovation Bonus) Source: Real alternative credit dataset (150K samples) Features: 42 behavioral signals engineered from:

Traditional credit: Delinquency scores, debt ratios, payment history Alternative data: UPI transactions, utility payments, rent consistency Socioeconomic: Employment stability, savings, government schemes Behavioral: Transaction timing, overdrafts, income variability

Feature Engineering Examples:

Composite score calculations

debt_burden_score = (0.4 * debtratio + 0.3 * medical_debt_ratio

• 0.3*emi_payment_ratio )

payment_reliability = ( 0.5 * (1 - failed_txn_ratio) +0.3 * utility_payment_ratio

• 0.2 * rent_payment_regular )

overdraft_risk = ( overdraft_count / bank_account_age_months )

Data Quality:

Zero null values after preprocessing Balanced classes (50/50 approve/deny) 70/15/15 train/val/test split Stratified sampling by risk deciles

Theme Alignment & Bonus Objectives Primary Theme: #3.1 Professional Tasks Real-world credit investigation workflow Partially observable environment (hidden features) Tool orchestration (request_info, query_market, flag_fraud) Multi-step causal reasoning (5-8 actions per episode) Persistent world model (action history, applicant state)

Secondary Theme: #2 Long-Horizon Planning

8-step timeout penalty (forces strategic planning) Early mistake recovery (can request more data after poor initial assessment) State tracking across steps (revealed fields, fraud flags, market context)

Bonus: Fleet AI (Scalable Oversight) Implemented: Agent1/Agent2 interaction with oracle supervision Planned: Auditor agent monitoring Agent2 decisions for bias and consistency Alignment: Agent2 learns to justify decisions via SHAP explanations Bonus: Scale AI (Business Workflows) Task: Credit approval is a real business workflow in lending operations Complexity: Multi-step investigation, fraud detection, escalation paths Non-Code Use Case: Directly applicable to underwriting teams

Technical Stack Environment:

Framework: OpenEnv (latest release - validated) Server: FastAPI with session-based locking Deployment: HuggingFace Spaces (Docker SDK) Persistence: File-based session store (JSON snapshots)

Agents:

Oracle: Scikit-learn + SHAP (joblib serialization) Policy: Unsloth + TRL (Qwen base model) Training: Colab T4 GPU (4-bit quantization)

Validation:

validate-submission.sh for OpenEnv compliance smoke_test.py for environment sanity checks inference.py for offline evaluation

Environment Innovation Multi-step partially observable design (not just classification) Real-world dataset with 42 behavioral features (not synthetic) Adversarial elements (fraud, withheld data, fabricated fields) Dynamic market conditions (economic context affects risk)

Clear problem (financial exclusion) Relatable analogy (detective investigating applicant) Compelling demo (before/after policy improvement) Real-world impact (unbanked populations)

Reward Improvement Baseline: 0.56 → Target: 0.70+ (25% improvement) Observable: Reward curve plots + metrics logs Verifiable: Independent oracle scoring

Pipeline Quality Clean reward logic (environment + grader alignment) Reproducible training (Colab notebook + checkpoints) Anti-hacking safeguards (sandbox, determinism, diversity)

Repository Structure credless_env/

├── credless_model/ # Agent1 oracle

│ ├── model.pkl # Trained oracle artifact

│ ├── metadata.json # Feature names, version

│ └── train.py # Oracle training script

├── agent2-decision-base/ # Agent2 policy

│ ├── checkpoints/

│ │ ├── agent2_policy.pkl # Supervised baseline

│ │ └── supervised_dataset.jsonl # TRL training data

│ └── train.py # Agent2 supervised training

├── server/

│ ├── environment.py # OpenEnv credit investigation

│ ├── app.py # FastAPI endpoints

│ └── client.py # WebSocket client

├── rl/

│ ├── rollout_collector.py # Trajectory generation

│ ├── reward_logger.py # Metrics tracking

│ └── training_summary.json # Current performance

├── data/

│ ├── cd_updated.csv # 150K real samples

│ └── feature_docs.md # Dataset documentation

├── inference.py # Offline evaluation

├── baseline.py # Baseline comparison

├── train.py # Agent1 training entry

└── requirements.txt # Dependencies

Links

GitHub:https://github.com/anushaa-m/credless_env HuggingFace Space:https://huggingface.co/spaces/anushaa-m/credless-env Demo Video: [YouTube <2min]

Built for OpenEnv Hackathon 2025 - Targeting Theme #3.1 (Professional Tasks) + Theme #2 (Long-Horizon Planning) + Fleet AI Bonus (Scalable Oversight)