personal-color-ai

🎨 Personal Color AI — Sub-Season Classifier

RandomForest classifier that predicts a person's Personal Color season and sub-season (16 classes) from LAB color features extracted from a face photo, plus lighting condition and makeup signals.

The model is shipped in two formats:

100% of the reference app runs client-side in the browser — no image or result is ever sent to a server. This repo hosts the model artefacts only.

---

📊 Model summary

16 sub-seasons

Full class order (index 0..15):

Autumn_Deep, Autumn_Muted, Autumn_Soft, Autumn_Warm,
Spring_Bright, Spring_Clear, Spring_Light, Spring_Warm,
Summer_Cool, Summer_Light, Summer_Muted, Summer_Soft,
Winter_Bright, Winter_Clear, Winter_Cool, Winter_Deep

---

📈 Training report

Model              : RandomForest (n=400, max_feat=0.4)
Total features     : 50 (base=37, eng=13)

-- Test Set ---------------------------------------------
Accuracy           : 0.6130  (61.30%)   sub-season
F1-macro           : 0.6109
F1-weighted        : 0.6110
Season accuracy    : 0.9296  (92.96%)   4 major seasons

-- 3-Fold CV (train set) --------------------------------
Accuracy  : 0.6062 +/- 0.0146
F1-macro  : 0.6048 +/- 0.0149

Why sub-season accuracy is ~61%: adjacent sub-seasons (e.g. Spring_Clear ↔ Spring_Bright) overlap by nature, so ~60–70% is expected. Distinguishing the 4 major seasons reaches ~93%. The model is deliberately designed so that skin + undertone are the dominant signal and hair darkness does not dominate — this keeps light-skinned + dark-haired people (common in East Asia) correctly classified.

Best/worst classes (F1): Winter_Deep 0.80, Autumn_Deep 0.78 (best); Spring_Clear 0.48, Winter_Clear 0.52 (hardest — overlap with neighbours).

---

🧬 Feature schema (50 features, exact order)

The model input is a single float32 vector of length 50 in this exact order.

Base features (37):

face_L, face_a, face_b, face_chroma, face_redness, face_uniformity,
body_L, body_a, body_b, body_chroma,
delta_L, delta_a, delta_b,
hair_L, hair_chroma, hair_undertone,
eye_L, eye_chroma, eye_clarity,
contrast_face_hair, contrast_face_eye, contrast_overall,
uv_index, cloud_cover, sun_angle,
ambient_light_condition, white_balance_shift, shadow_level,
highlight_level, noise_level,
has_foundation, has_lipstick, has_contour, has_blush,
ethnicity_baseline_L, ethnicity_baseline_a, ethnicity_baseline_b

Engineered features (13) — you MUST compute these from the base features before inference (the same math is applied at train time):

eps = 1e-6
ita_face      = degrees(arctan2(face_L - 50, max(face_b, eps)))
ita_body      = degrees(arctan2(body_L - 50, max(body_b, eps)))
warmth_score  = face_a*0.4 + face_b*0.4 + hair_undertone*10
depth_score   = (100 - face_L)*0.7 + (100 - hair_L)*0.3
clarity_score = contrast_overall*0.6 + eye_clarity*0.4
face_hair_dE  = sqrt((face_L - hair_L)**2 + face_a**2 + face_b**2)
chroma_ratio  = face_chroma / (body_chroma + eps)
skin_quality  = face_redness * face_uniformity
light_intensity = uv_index/11.0 * (1 - cloud_cover/100)
face_L_res    = face_L - ethnicity_baseline_L
face_a_res    = face_a - ethnicity_baseline_a
face_b_res    = face_b - ethnicity_baseline_b
hair_warmth   = hair_chroma * (hair_undertone + 1) / 2

Engineered feature order (appended after the 37 base features):

ita_face, ita_body, warmth_score, depth_score, clarity_score,
face_hair_dE, chroma_ratio, skin_quality, light_intensity,
face_L_res, face_a_res, face_b_res, hair_warmth

feature_names.json in this repo contains the canonical full 50-length order.

---

🐍 Usage — Python (model.pkl)

pip install scikit-learn numpy huggingface_hub

import json, pickle, numpy as np
from huggingface_hub import hf_hub_download

REPO = "mr4/personal-color-ai"   # <-- change to your repo id

pkl_path   = hf_hub_download(REPO, "model.pkl")
feat_path  = hf_hub_download(REPO, "feature_names.json")

with open(pkl_path, "rb") as f:
    pipeline = pickle.load(f)          # StandardScaler + RandomForest
with open(feat_path) as f:
    feature_names = json.load(f)       # 50 names, canonical order

# ---- build the 37 base features from your extraction pipeline ----
base = {
    "face_L": 68.0, "face_a": 12.0, "face_b": 18.0, "face_chroma": 21.6,
    "face_redness": 0.5, "face_uniformity": 0.8,
    "body_L": 66.0, "body_a": 11.0, "body_b": 17.0, "body_chroma": 20.3,
    "delta_L": 2.0, "delta_a": 1.0, "delta_b": 1.0,
    "hair_L": 20.0, "hair_chroma": 8.0, "hair_undertone": 0.2,
    "eye_L": 30.0, "eye_chroma": 10.0, "eye_clarity": 0.6,
    "contrast_face_hair": 48.0, "contrast_face_eye": 38.0, "contrast_overall": 0.5,
    "uv_index": 5.0, "cloud_cover": 40.0, "sun_angle": 45.0,
    "ambient_light_condition": 0.7, "white_balance_shift": 0.0, "shadow_level": 0.2,
    "highlight_level": 0.3, "noise_level": 0.1,
    "has_foundation": 0, "has_lipstick": 0, "has_contour": 0, "has_blush": 0,
    "ethnicity_baseline_L": 65.0, "ethnicity_baseline_a": 12.0, "ethnicity_baseline_b": 17.0,
}

# ---- engineered features (identical to training) ----
eps = 1e-6
f = dict(base)
f["ita_face"]        = np.degrees(np.arctan2(f["face_L"]-50, max(f["face_b"], eps)))
f["ita_body"]        = np.degrees(np.arctan2(f["body_L"]-50, max(f["body_b"], eps)))
f["warmth_score"]    = f["face_a"]*0.4 + f["face_b"]*0.4 + f["hair_undertone"]*10
f["depth_score"]     = (100-f["face_L"])*0.7 + (100-f["hair_L"])*0.3
f["clarity_score"]   = f["contrast_overall"]*0.6 + f["eye_clarity"]*0.4
f["face_hair_dE"]    = np.sqrt((f["face_L"]-f["hair_L"])**2 + f["face_a"]**2 + f["face_b"]**2)
f["chroma_ratio"]    = f["face_chroma"] / (f["body_chroma"]+eps)
f["skin_quality"]    = f["face_redness"] * f["face_uniformity"]
f["light_intensity"] = f["uv_index"]/11.0 * (1 - f["cloud_cover"]/100)
f["face_L_res"]      = f["face_L"] - f["ethnicity_baseline_L"]
f["face_a_res"]      = f["face_a"] - f["ethnicity_baseline_a"]
f["face_b_res"]      = f["face_b"] - f["ethnicity_baseline_b"]
f["hair_warmth"]     = f["hair_chroma"] * (f["hair_undertone"]+1) / 2

# ---- assemble in canonical order and predict ----
X = np.array([[f[name] for name in feature_names]], dtype=np.float32)  # shape (1, 50)

pred  = pipeline.predict(X)[0]
proba = pipeline.predict_proba(X)[0]
classes = pipeline.classes_          # or load label_encoder.pkl

print("Predicted:", pred)
print("Confidence:", float(proba.max()))

Note: model.pkl is a full pipeline — it applies StandardScaler internally, so pass raw (unscaled) features.

---

🌐 Usage — JavaScript / Browser (model.onnx)

npm install onnxruntime-web

import * as ort from "onnxruntime-web";

// model.onnx and feature_names.json served statically (or fetched from HF)
const session = await ort.InferenceSession.create("/model.onnx");

// 1) build the 50-length feature vector in canonical order
function buildFeatures(b) {
  const eps = 1e-6;
  const d2r = Math.PI / 180, deg = r => r * 180 / Math.PI;
  const e = {
    ita_face: deg(Math.atan2(b.face_L - 50, Math.max(b.face_b, eps))),
    ita_body: deg(Math.atan2(b.body_L - 50, Math.max(b.body_b, eps))),
    warmth_score: b.face_a*0.4 + b.face_b*0.4 + b.hair_undertone*10,
    depth_score: (100-b.face_L)*0.7 + (100-b.hair_L)*0.3,
    clarity_score: b.contrast_overall*0.6 + b.eye_clarity*0.4,
    face_hair_dE: Math.sqrt((b.face_L-b.hair_L)**2 + b.face_a**2 + b.face_b**2),
    chroma_ratio: b.face_chroma / (b.body_chroma + eps),
    skin_quality: b.face_redness * b.face_uniformity,
    light_intensity: b.uv_index/11.0 * (1 - b.cloud_cover/100),
    face_L_res: b.face_L - b.ethnicity_baseline_L,
    face_a_res: b.face_a - b.ethnicity_baseline_a,
    face_b_res: b.face_b - b.ethnicity_baseline_b,
    hair_warmth: b.hair_chroma * (b.hair_undertone + 1) / 2,
  };
  const all = { ...b, ...e };
  // FEATURE_NAMES = the array from feature_names.json (length 50)
  return Float32Array.from(FEATURE_NAMES.map(n => all[n]));
}

const x = buildFeatures(baseFeatures);
const input = new ort.Tensor("float32", x, [1, 50]);

// ONNX exported with zipmap=false → outputs: [label, probabilities]
const out = await session.run({ float_input: input });
const outNames = session.outputNames;              // e.g. ["label", "probabilities"]
const label = out[outNames[0]].data[0];            // predicted class index
const probs = out[outNames[1]].data;               // Float32Array length 16

const CLASSES = [ /* 16 names from model_metadata.json, same order */ ];
console.log("Predicted:", CLASSES[Number(label)]);
console.log("Confidence:", Math.max(...probs));

The ONNX graph was exported with zipmap=false, so the second output is a plain probability tensor of shape [1, 16] (not a map). Input tensor name is float_input.

---

📦 Repo contents

---

⚠️ Limitations & intended use

• Trained on synthetic data — accuracy on real photos will differ. Fine-tune with real labelled data before any serious use.
• Not a medical, dermatological, or professional styling tool; recommendations are cosmetic suggestions only.
• Feature extraction (BiSeNet face parsing + MediaPipe iris + CIE-LAB) is done outside this model in the reference web app.
• Do not use for identity, demographic profiling, or any surveillance purpose. The ethnicity_baseline_* inputs are color calibration constants, not a prediction of ethnicity.

📝 License

MIT — free to use, modify, and distribute.