pricer-merged-model-A-v1

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

A merged LLaMA 3.1 8B checkpoint specialized for numeric price prediction from product text. This model was created by merging LLaMA 3.1 8B with a LoRA adapter (Pricer LoRA v1) and is intended to serve as a base model for further LoRA fine-tuning.

Model Details

Overall Training Process

Model Description

<!-- Provide a longer summary of what this model is. -->

pricer-merged-model-A-v1 is a transformer-based causal language model optimized for estimating approximate consumer product prices from textual metadata such as title, description, and category. This model represents a merged checkpoint (base model + LoRA), not an only-LoRA adapter.

• Developed by: MyungHwan Hong (MightyOctopus)
• Funded by: Self-funded / independent research
• Shared by: MyungHwan Hong
• Model type: Causal Language Model (Text-to-Number / Numeric Prediction)
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: meta-llama/Llama-3.1-8B

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: https://huggingface.co/MightyOctopus/pricer-merged-model-A-v1
• Colab Code for Training: https://colab.research.google.com/drive/1Vhv8u3mcvMBAto0A4vqgXyxo3W2mVB76?usp=sharing
• Colab Code for Model Testing: https://colab.research.google.com/drive/1ASyJq6JJDUqNG3X4fv__GvUH2REn6Rsh?usp=sharing
• merge_and_unload Code: https://colab.research.google.com/drive/1I6FqgnWcezlFHgWZ47y0FvLQ2pu2RGCY?usp=sharing
• Experiment Log: https://docs.google.com/document/d/1PwuOCS6wgO3MqKexnEdAqpVswXMqGilqKEuFyhUGk7M/edit?tab=t.0

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

• Base checkpoint for price-prediction inference

• Base model for further LoRA fine-tuning

• Research on LLM-based numeric regression

Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

The model is primarily intended to serve as a base checkpoint for further LoRA fine-tuning.

Possible downstream applications include:

• Domain-specific product price predictors (e.g., electronics, books, apparel)
• Research on LLM-based numeric regression
• Comparative studies against classical ML regression models
• Experiments on LoRA merging and adapter stacking strategies

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

• Financial decision-making

• Legal, medical, or safety-critical applications

• Use as an authoritative price source

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

• Predictions are approximate, not exact

• Performance depends on similarity to training data distribution

• May hallucinate prices for unfamiliar products

• Reflects historical and dataset-specific price biases

• Not robust to rapid market price changes

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

• Treat outputs as estimates, not ground truth

• Validate predictions with real pricing data

• Avoid high-stakes or commercial deployment

• Be aware of temporal and dataset bias

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "MightyOctopus/pricer-merged-model-A-v1"
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = """Product:
Title: Stainless Steel Electric Kettle 1.7L
Category: Home & Kitchen
Description: Fast boiling electric kettle with auto shut-off.

Price is $"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(inputs.input_ids, max_new_tokens=10)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

• Amazon product metadata

• Fields: title, description, category, ground-truth price

• Prices represented as structured text outputs

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

This model was created by merging a LoRA adapter (Pricer LoRA v1) into LLaMA 3.1 8B. No additional training was performed after merging.

Training Hyperparameters

• Training regime: bfloat16 mixed precision (inherited from LoRA training)

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

This merged checkpoint was not evaluated independently. Evaluation was only performed on downstream fine-tuned adapters (e.g. pricer-lora-ft-v3: https://huggingface.co/MightyOctopus/pricer-lora-ft-v3/blob/main/README.md).

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: NVIDIA GPU (e.g. T4, A100)
• Hours used: ~20 hours (LoRA v1 training)
• Cloud Provider: Google Colab / Hugging Face
• Compute Region: Unknown
• Carbon Emitted: Not estimated

Model Architecture and Objective

Transformer-based causal language model

Objective: Next-token prediction optimized for numeric output consistency

Hardware

NVIDIA GPU (T4, L4, A100-class)

Software

• Transformers

• PEFT

• PyTorch

BibTeX:

@misc{hong2025pricermerged, author = {MyungHwan Hong}, title = {Pricer Merged LLaMA 3.1 8B Model}, year = {2025}, url = {https://huggingface.co/MightyOctopus/pricer-merged-model-A-v1} }

APA:

MyungHwan Hong. (2025). Pricer Merged LLaMA 3.1 8B Model. Hugging Face.

Model Card Authors [optional]

MyungHwan Hong

Model Card Contact

Hugging Face: MightyOctopus