pricer-lora-ft-v3

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Description

pricer-lora-ft-v3 is a fine-tuned large language model (with the base model: MightyOctopus/pricer-merged-model-A-v1) specialized in numeric price prediction for consumer products(e.g. Amazon products etc). The model predicts approximate product prices from textual metadata such as product title, description, and category. It demonstrates that a fully open source LLM can be adapted for structured numeric regression tasks traditionally handled by classical ML models.

• Developed by: MyungHwan Hong (MightyOctopus)
• Funded by: Self-funded / independent research
• Shared by: MyungHwan Hong
• Model type: Causal Language Model (Numeric Prediction / Regression via Text-to-Number)
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: meta-llama/Llama-3.1-8B

Training Process

Model Sources

<!-- Provide the basic links for the model. -->

• Repository:: https://huggingface.co/spaces/MightyOctopus/worth-brain
• Research Note: (https://docs.google.com/document/d/1PwuOCS6wgO3MqKexnEdAqpVswXMqGilqKEuFyhUGk7M/edit?tab=t.0)
• Demo:: https://huggingface.co/spaces/MightyOctopus/worth-brain
• Colab Code for Training: https://colab.research.google.com/drive/1Vhv8u3mcvMBAto0A4vqgXyxo3W2mVB76?usp=sharing
• Colab Code for Model Testing: https://colab.research.google.com/drive/1ASyJq6JJDUqNG3X4fv__GvUH2REn6Rsh?usp=sharing
• Dataset: https://huggingface.co/datasets/MightyOctopus/amazon-pricer-dataset-v2-0
• Experiment Log: https://docs.google.com/document/d/1PwuOCS6wgO3MqKexnEdAqpVswXMqGilqKEuFyhUGk7M/edit?tab=t.0

Direct Use

• Predicting approximate Amazon product prices from text metadata

• Research on LLM-based numeric regression

• Benchmarking open-source LLMs against frontier models (e.g., GPT-4o-mini)

• Educational experiments on LoRA fine-tuning and evaluation

Downstream Use

• Price estimation pipelines (non-production)

• Feature generation for pricing analytics

• Comparative studies with classical ML regressors

Out-of-Scope Use

• Real-time or production pricing systems

• Financial decision-making

• Legal, medical, or safety-critical applications

• Use as an authoritative price source

Bias, Risks, and Limitations

• Predictions are approximate, not exact

• Performance depends on similarity to training data distribution

• The model may hallucinate prices for unfamiliar or novel products

• Prices may reflect historical or dataset-specific biases

• Not robust to rapid market price changes

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

• Treat outputs as estimates, not ground truth

• Validate predictions against real pricing data

• Avoid using the model in high-stakes or commercial systems

• Be aware of dataset and temporal bias

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

TOKENIZER_MODEL = "meta-llama/Llama-3.1-8B"
BASE_MODE_ID = "MightyOctopus/pricer-merged-model-A-v1"
FINE_TUNED_ADAPTER  = "MightyOctopus/pricer-lora-ft-v3"

tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_MODEL)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODE_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

tokenizer.pad_token = tokenizer.eos_token
base_model.generation_config.pad_token_id = tokenizer.pad_token_id

fine_tuned_model = PeftModel.from_pretrained(
    base_model,
    FINE_TUNED_ADAPTER
)

fine_tuned_model.eval()

prompt = """Product:
Title: Stainless Steel Electric Kettle 1.7L
Category: Home & Kitchen
Description: Fast boiling electric kettle with auto shut-off.

Price is $"""

inputs = tokenizer(prompt, return_tensors="pt").to(fine_tuned_model.device)

with torch.no_grad():
    outputs = fine_tuned_model.generate(
        **inputs,
        max_new_tokens=10,
        temperature=0.2
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

• Amazon product metadata dataset

• Fields include title, description, category, and ground-truth price

• Prices normalized via structured text prompts

• Dataset split into training, validation, and test sets

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Preprocessing

Preprocessing

• Text normalization

• Structured prompt formatting

• Numeric price represented as plain text output

• Loss applied only to answer tokens using response masking

Training Hyperparameters

Training regime: Supervised Fine-Tuning (SFT) with LoRA

Key Hyperparameters:

• Optimizer: AdamW

• Learning Rate: 2e-5

• Epochs: 2

• Batch Size: 16

• Gradient Accumulation: 2

• Effective Batch Size: 3

• LoRA Rank: 32

• LoRA Alpha: 64

• LoRA Dropout: 0.0

• Weight Decay: 0.0

• Precision: bfloat16

Speeds, Sizes, Times

• Base model: 8B parameters

• LoRA parameters: ~0.5% of base model

• Training time: Approx. 21 hours on single GPU

Evaluation

Testing Data

• Held-out Amazon product samples

• Products not seen during training

Factors

• Product category

• Price range distribution

• Description length

• Token length

Metrics

• Mean Absolute Error (MAE)

• Root Mean Squared Logarithmic Error (RMSLE)

• Hit Rate (prediction within ±20% of ground truth)

Results

Perforance GPT VS Pricer-Lora-FT-V3 (Fine-Tuned)

1. GPT 4o Mini

2. pricer-lora-ft--v3

Summary

Model Examination

<!-- Relevant interpretability work for the model goes here -->

The model demonstrates that fine-tuned open-source LLMs can outperform frontier zero-shot models on specialized numeric tasks when trained with domain-specific data and structured prompts.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: NVIDIA GPU (e.g., A100)
• Hours used: ~21 hours
• Cloud Provider: Google Colab / Hugging Face
• Compute Region: Unknown (Colab GPUs used)
• Carbon Emitted: Not estimated

Technical Specifications

Model Architecture and Objective

• Transformer-based causal language model

• Objective: Next-token prediction optimized for numeric accuracy

• Loss applied selectively to price tokens

Compute Infrastructure

• Single-GPU fine-tuning

• LoRA-based parameter-efficient training

Hardware

• NVIDIA GPU (A100)

Software

• Transformers

• TRL

• PEFT 0.14.0

• PyTorch

Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

@misc{hong2025pricer, author = {Hong, MyungHwan}, title = {Pricer LoRA Fine-Tuned LLaMA 3.1 8B Model}, year = {2025}, url = {https://huggingface.co/MightyOctopus/pricer-lora-ft-v3} }

APA:

MyungHwan Hong, (2025). Pricer LoRA Fine-Tuned LLaMA 3.1 8B Model. Hugging Face. https://huggingface.co/MightyOctopus/pricer-lora-ft-v3

Glossary

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

More Information

[More Information Needed]

Model Card Authors

MyungHwan Hong

Model Card Contact

Hugging Face: MightyOctopus

Framework versions

• PEFT 0.14.0