ecomm_shop_intent_pretrained

Commerce Intent

Model Overview

Commerce Intent is a pretrained sequential behavioral model for e-commerce session understanding. It is trained to predict the next item in a user session based on historical interaction sequences.

The model learns representations from multi-modal structured signals, including:

• Item ID
• Brand
• Category
• Event type (view, cart and purchase)
• Normalized price
• Positional order within the session

It is designed as a foundation model for downstream recommendation and behavioral modeling tasks.

---

Model Details

Model Description

Commerce Intent models user behavior within a session as an autoregressive sequence modeling problem. Given a sequence of past interactions, the model predicts the next likely item.

The architecture consists of:

• Multi-embedding token fusion (item, brand, category, event)
• Continuous price projection
• Positional encoding
• Transformer encoder with causal masking
• Linear head for next-item prediction

This model is pretrained and can be fine-tuned for recommendation, ranking, or conversion modeling tasks.

• Developed by: infinity6
• Model type: Sequential autoregressive transformer
• Language(s): Structured e-commerce interaction data (non-NLP)
• License: Apache 2.0
• Finetuned from model: None (trained from scratch)

---

Dependencies

This model depends on the external package:

• https://github.com/infinity6-ai/i6model_ecomm

The package contains the custom architecture required to correctly load and run the model. You must install it before using Commerce Intent.

Installation

Clone the repository:

git clone https://github.com/infinity6-ai/i6model_ecomm.git
cd i6model_ecomm
pip install .

---

Intended Use

Direct Use

The model can be used directly for:

• Next-item prediction
• Session-based recommendation
• Behavioral embedding extraction
• Purchase intent modeling
• Real-time ranking systems

Example:

import torch

from i6modelecomm.model import i6modelecomm

model = i6modelecomm.CommerceIntent.from_pretrained(
    "infinity6/ecomm_shop_intent_pretrained"
)

# TODO: map items and remap categories.

# TODO: freeze layers and train with your data.

model.eval()

D = 'cpu'

# batch_size | seq_len = 3
itms = torch.tensor([[12, 45, 78]], dtype=torch.long).to(D)
brds = torch.tensor([[3, 7, 2]], dtype=torch.long).to(D)
cats = torch.tensor([[8, 8, 15]], dtype=torch.long).to(D)
prcs = torch.tensor([[29.9, 35.0, 15.5]], dtype=torch.float).to(D)
evts = torch.tensor([[1, 1, 2]], dtype=torch.long).to(D)

# mask
mask = torch.tensor([[1, 1, 1]], dtype=torch.bool).to(D)

with torch.no_grad():
    outputs = model(
        itms=itms,      # items
        brds=brds,      # brands
        cats=cats,      # categories
        prcs=prcs,      # prices
        evts=evts,      # events
        attention_mask=mask,
        labels=None     # inference only -- no loss computation
    )

# logits tem shape (B, L-1, num_itm)
logits = outputs.logits

print("Logits shape:", logits.shape)

Inputs must include:

• itms
• brds
• cats
• evts
• prcs
• attention_mask

---

Downstream Use

The model can be fine-tuned for:

• Conversion prediction
• Cart abandonment modeling
• Customer lifetime value modeling
• Cross-sell / upsell recommendation
• Personalized search ranking

---

Out-of-Scope Use

This model is not suitable for:

• Natural language tasks
• Image tasks
• Generative text modeling
• Multi-user graph modeling without adaptation
• Cold-start scenarios without item mappings and category remapping

---

Bias, Risks, and Limitations

• The model reflects behavioral biases present in historical e-commerce data.
• Popularity bias may emerge due to item frequency distribution.
• Model performance depends on session length and interaction quality.
• Cold-start performance for unseen items is limited.
• It does not encode demographic or identity-aware fairness constraints.

Recommendations

• Monitor recommendation fairness and popularity skew.
• Retrain periodically to reflect new item distributions.
• Apply business constraints in production systems.
• Use A/B testing before large-scale deployment.

---

Training Details

Training Data

The model was trained on large-scale anonymized e-commerce interaction logs containing:

• Session-based user interactions
• Item identifiers
• Brand identifiers
• Category identifiers
• Event types
• Timestamped behavioral sequences
• Price values (log-normalized and standardized)

Sessions shorter than a minimum threshold were filtered.

---

Training Data

Data Sources and Preparation

The model was trained on a unified, large-scale corpus of e-commerce interaction data, aggregating and normalizing multiple public datasets to create a robust foundation for sequential behavior modeling.

The training data combines the following sources:

Data Unification and Normalization

All datasets underwent a rigorous unification and normalization process:

• Schema Alignment: Standardized field names and types across all sources (item_id, brand_id, category_id, event_type, timestamp, price)
• Event Type Normalization: Mapped varied event nomenclature to a standardized taxonomy (view, cart, purchase)
• ID Harmonization: Created consistent ID spaces for items, brands, and categories through cross-dataset mapping
• Temporal Alignment: Unified timestamp formats and established consistent session windows
• Price Normalization: Applied log-normalization (log1p) followed by standardization using global statistics
• Session Construction: Reconstructed user sessions based on temporal proximity and interaction patterns
• Quality Filtering: Removed sessions below minimum length threshold and filtered anomalous interactions

This diverse and comprehensive training corpus enables the model to learn robust representations of e-commerce behavior patterns across different platforms, markets, and interaction types, serving as a strong foundation for downstream fine-tuning tasks.

---

Preprocessing

• Missing categorical values replaced with UNK
• Price values transformed via log1p
• Standardization using global mean and standard deviation
• Session truncation to fixed-length sequences
• Right padding with attention masking

---

Training Objective

Next-item autoregressive prediction using cross-entropy loss with padding ignored.

---

Training Regime

• Precision: FP32
• Optimizer: AdamW
• Learning Rate: 1e-3 with warmup
• Gradient Clipping: 5.0
• Causal masking applied

---

Evaluation

Metrics

• Cross-Entropy Loss
• Perplexity
• Recall@20

Results

On the evaluation split, the model achieved:

• Perplexity: 24.04
• Recall@20: 0.6823

These results indicate strong next-item prediction performance in session-based e-commerce interaction modeling.

Summary

The model demonstrates:

• Low predictive uncertainty (Perplexity 24.04)
• High ranking quality for next-item recommendation (Recall@20 of 68.23%)

Performance may vary depending on dataset distribution, session length, and preprocessing configuration.

---

Environmental Impact

• Hardware: GPU NVIDIA H100 NVL (94GB PCIe 5.0)
• Precision: FP32
• Training Duration: Several hours (varies by configuration)
• Carbon Impact: ≈45 kg CO₂e (estimated based on energy consumption of 30h on H100 GPU)

---

Limitations

• No long-term user modeling beyond session scope
• Does not include user-level embeddings
• Requires predefined categorical vocabularies
• Limited generalization to unseen item IDs