smartsightCoach

SmartSight Coach LoRA fine-tunes

LoRA fine-tunes for the SmartSight AI coach — an on-device fitness and nutrition coach that runs entirely on the phone, no account and no server round-trip. Converted to .litertlm for Android/iOS inference. → smartsight.app/ai-coach

Built and maintained by Niclas Bade.

Base model changed at v42 (2026-08-12). v27–v38 were fine-tuned from google/gemma-4-E2B-it. Everything from v42 on is fine-tuned from Google's quantization-aware-trained checkpoint google/gemma-4-E2B-it-qat-q4_0-unquantized, whose weights are conditioned to survive 4-bit rounding. The frontmatter above reflects the CURRENT parent, so the model tree matches what actually shipped. This repo keeps the current best checkpoints plus earlier superseded milestones for history — not every candidate tested each round. Rounds that lose are documented internally with their real failure cases; they are never uploaded here.

More coming. The vision tower is being trained surface by surface — body fat, meal photos and nutrition-label reading — each measured against held-out data before anything ships. Progress and honest limitations for every round are written up below and at smartsight.app/ai-coach.

smart-coach-vision.litertlm — CURRENT CHAMPION (v221)

Promoted 2026-09-25. sha256 a3e1f3cbcffe562d, 2,795,655,584 bytes. Lineage: v27 → v31 → v34 → v38 → v42 → v44 → v46 → v47 → v48 → v51 → v54 → v55 → v69 → v85 → v127 → v134 → v137 → v138 → v139 → v158 → v178 → v201 → v208 → v211 → v215 → v220 → v221.

It answers "I forgot to log it" correctly, reads lean bodies better, and reads nutrition labels worse. All three are in the table below.

Both models measured on the same frozen surfaces (176 real photographs with measured body fat, 32 nutrition labels, 30 non-body photographs), same prompt, same backend. The shipped file is the int4 export the numbers were measured on. Coaching text is HAND-READ by a blind reader who saw both models' answers side by side without knowing which was which.

The text gain is the unlogged-day answer, now fixed. Told "I trained Monday but I don't think I logged it", v220 stated the blank day plainly on 15 of 32 cases and answered with a conditional ("if it's not in the log, add it") on most of the rest; on four cases it claimed a blank day was already logged or moved a session to the wrong day. v221 states the blank day plainly on 32 of 32 and invents no session. It still reads "what did I train yesterday" right on 32 of 32.

Lean bodies read better. Within 2 points at 10% body fat or under rises 22 to 31 of 46, and on the 21 newest of those 4 to 10. The DEXA-measured 7% subject reads 6.5 on the reference photo and on one of the three new shots, 8.5 on the other two (v220: 7.5, then 8.5 on all three). Lean bodies are still read about 2.6 points too fat on average on the 21 newest, and four photographs between 7 and 10% read 12.5 to 21 on every model this project has measured.

Nutrition labels fall 19 to 12 of 32, and that is printed beside the gains. The same recipe on a second seed reads 17, so part of this is seed spread, but the shipped file reads 12.

The promotion is NOT attributable to what the round tested. v221 is a control model: it did not receive the round's variable, a re-judged set of body-fat labels for about 4,000 training photographs. The four models that did receive it scored lower on average on both totals (text 145.0 against 155.75, photographs 144.25 against 150.0). What every model of the round received is new coaching-text training data on logged and unlogged days, and that is what moved the text score.

smart-coach-vision.litertlm — previous champion (v220)

Promoted 2026-09-24, superseded by v221 on 2026-09-25. sha256 e36d6a0dd7e75e98, 2,795,655,584 bytes. Lineage: v27 → v31 → v34 → v38 → v42 → v44 → v46 → v47 → v48 → v51 → v54 → v55 → v69 → v85 → v127 → v134 → v137 → v138 → v139 → v158 → v178 → v201 → v208 → v211 → v215 → v220.

It reads nutrition labels better and answers "I forgot to log it" more plainly, and it reads lean bodies worse. All three are in the table below.

The photograph surface grew on 2026-09-23 from 49 to 176 real photographs with measured body fat (the first 49 unchanged), so the numbers below are not comparable with the previous section's. Both models measured on the same frozen surfaces, same prompt, same backend. The shipped file is the int4 export the numbers were measured on. Coaching text is HAND-READ by a blind reader who saw both models' answers side by side without knowing which was which.

The text gain is the unlogged-day answer. Told "I trained Monday but I don't think I logged it", v215 answered with a conditional ("if it's not in the log, add it") and never said what the log shows; in the same reading it stated the blank day plainly on 3 of 32 cases. v220 does so on 15 of 32. It also reads "what did I train yesterday" right on 32 of 32 (v215: 30). It is not clean: on three of those cases it claims a blank Wednesday is already logged, and it still answers with the conditional on most of the rest.

Nutrition labels go 15 to 19 of 32.

The lean end got worse, and that is printed beside the gains. Within 2 points on photographs at 10% body fat or under falls 29 to 22 of 46, and on the 21 newest of those 9 to 3. Three new photographs of one DEXA-measured 7% subject read 8.5. On 6 of the 46 lean photographs the answer cites the jawline and cheeks as its reason, which is not where body fat is read. The widest attractor span widens 12.3 to 16.9 points and the slope stays under this project's 0.955 floor (0.69, against 0.71): the binning defect is not fixed.

The promotion is NOT attributable to what the round tested. v220 is a control model: it did not receive the round's variable, which doubled the share of one source of lean training photographs. The four models that did receive it scored lower on average on both totals (text 115.0 against 131.75, photographs 143.5 against 148.0). v220 is the best model of its round, not evidence that a recipe works.

Champion history — what each round actually fixed

Every entry here was a promoted champion, verified against the previous one on a held-out suite with a full read of every answer. Rejected rounds (v28-v30, v32, v33, v35-v37, v39, v40, v41) are not listed — they are documented internally with their real failure cases.

Recipe constant across every round since v22c: LoRA r=16 / alpha=32 / dropout=0.1, 2 epochs, lr 6e-5, NEFTune noise_alpha=5, completion-only loss masking (train_on_responses_only), per-task validation shards, two seeds per round with the winner chosen on behaviour rather than validation loss. Only the training data, the base, and the export recipe have moved.

Honest current weaknesses

Documented rather than hidden, because they are the targets for the next rounds:

• NUTRITION LABELS FELL (v221). kcal read correctly on 12 of 32, down from 19. The same recipe on a second seed reads 17. Every model reads one label (gold 376 kcal) as 1572.

• A FEW LEAN BODIES STILL READ FAR TOO FAT (v221). Lean reading recovered (31 of 46 within 2 points, from 22), but on the 21 newest lean photographs the average read is still 2.6 points too high, and four photographs between 7 and 10% read 12.5 to 21 on every model measured so far, each described as "the lower abdomen is soft and rounded". The slope is 0.71, under the 0.955 floor, and the widest attractor span is still 16.9 points.

• FIXED IN v221: unlogged-day answers. v220 stated the blank day plainly on 15 of 32 and claimed a blank day was logged on four; v221 states it on 32 of 32 and invents none.

• Definition questions (v221). Asked "what is DASH" or "what is weekly undulating periodisation", it sometimes answers from the training log instead of defining the term, and it defines a tri set as three sets of one exercise.

• Binning is unchanged (v215). Slope 0.89 against the 0.955 floor, widest attractor span 11 pp, and bodies above 18 percent are still read about 2 points low on average.

• THE BINNING FLOOR GOT FURTHER AWAY, NOT CLOSER (v211). The champion's regression slope is 0.88, against this project's 0.955 floor - worse than v208's 0.95, and the widest attractor span widened 9.0 to 11.0 pp, with five of the rows it swallows sitting under 20% where most users are. A different arm of the same round measured 0.9982, clearing the floor for the first time ever, so the defect is now known to be fixable by this recipe on some seeds. That arm regressed on coaching text and was not shipped. This is the oldest open defect on the model and it is getting the next round's attention.

• Superseded note (v208). The regression slope of predicted against true body fat is 0.9538 against this project's 0.955 floor, up from v201's 0.88. Below that floor the model compresses distinct bodies toward a middle value. v208 is within two thousandths of the floor, but the round's control arms sat at 0.83 and 0.70, so the recipe has not fixed it — this seed landed near the line.

• Lean bodies in tall, narrow photographs read too high (v208). Among photographs under 9% body fat, the ones framed tall and narrow (width under 0.60 of height) read +2.58 pp high on average, while wider framings read −0.39. The cause is the training set: tall framings of lean bodies are rare in it and tall framings of higher body fat are common, so frame shape became a cue for body fat. This is the IMAGE's proportions, not the person's height. The next round adds re-framed lean photographs to break that link.

• A single photo source is a shortcut for "lean". One batch of competition photographs, all carrying the same event watermarks and stage backdrop, makes up about half of the sub-8% training photographs and contains nothing above 8%. The model can learn "stage photo means lean" instead of reading the body. New additions are drawn so they do not increase that share; fixing it needs processed or new photographs spread across every body-fat band.

• FIXED IN v211: answers about an unlogged day. v208 would assert a session existed on a blank day, or count it on feeling. v211 answers these correctly, as did every other arm of its round. The original v208 defect is kept below for history.

• Superseded note (v208). Asked about a day with no session in the 7-day history, v208 answers correctly 27 of 32 times against v201's 32 of 32, while it improved on the neighbouring history-reading questions. The gain and the loss came from the same model.

• A pet photograph still reads as a body (v201). Of 30 real bodyless photographs the champion declines 30, but one specific pet image returns a confident body-fat percentage on other arms of the same round, in near-identical wording across two independently trained models. That makes it a property of that photograph rather than of a seed. The denial set holds 537 images and evidently none of them look like that one; more real bodyless photographs remains the lever.

• Body fat at the lean end: LARGELY FIXED IN v139, and the fix was half points. Through v138 this read as a bucket — DEXA-measured 5.9%, 8.8% and 10.0% photographs all returned 8, three truths and one answer, unchanged from v69 to v85. The cause turned out to be partly the OUTPUT ALPHABET: a model that can only answer in whole numbers cannot express 7.5, and prompting it to do so changes nothing because the capability was never trained. v139 was trained with half-point targets and answers 7.5 exactly on 4 of 9 encodings of the DEXA reference photograph, cutting mean error there 2.39 → 1.22 pp. Remaining gap: this is demonstrated on ONE subject at nine encodings. More DEXA-measured lean subjects is still the lever for proving it generalises.

• Images that are not bodies: LARGELY FIXED IN v127, not yet solved. This was listed here as unfixable-by-prompting through v85, and it was: a variant instructing the model to reply NOT_A_BODY was ignored, returning byte-identical output. What fixed it was training data — specifically real bodyless photographs, not synthetic ones. A plain grey rectangle now reads "I cannot see a person in this image" rather than "12%". Remaining gap: v127 still returns a percentage for 16 of 30 real bodyless photographs (walls, sofas, pets, plates of food), so the behaviour is much better and still not dependable. More real bodyless photographs is the known lever; synthetic ones were measured to buy almost nothing. v139 update: it now declines all 58 of the bodyless probe set, answering "no person visible" every time, against v138's 56/58. Two cautions: the probe set is largely SYNTHETIC (grey ramps and noise), which is exactly the imagery measured to prove little, and the behaviour depends on the prompt carrying an explicit refusal clause. Real bodyless photographs remain the outstanding fix.

• The training prompt contradicts the training targets (v139). Half-point targets were added without updating the instruction the model is trained against, so 889 of 4,494 rows pair "ONE WHOLE NUMBER" with an answer of "7.5%". One row in five teaches the model to disobey its own instruction, which is why v139's readings move by more than a point on small prompt edits. The next round tests the repair directly, treated against its own control.

• Meal-plan portions are sized by habit, not arithmetic. The model anchors on ~100 g portions rather than solving the prompt's stated calorie budget. Mostly invisible in the app, which rescales a day to its target (MealPlanScaler, clamped 0.6x–2.0x) — but one case per model is still off after that clamp, and the underlying arithmetic is unsolved.

• Avoid-lists are not always respected — foods explicitly banned for variety reappear (2/29 held-out cases on v44, v42 and v38 alike; this is the single defect class that has survived every round so far).

• Nordic follow-ups about an UNLOGGED past activity regressed in v44. Asked in Norwegian about a hike "yesterday" that is not in the app, v42 correctly said the gap was only a logging omission; v44 answers as though advising about today. 26 Nordic follow-up rows were added to fix exactly this and did not move it — the cause is not simply coverage.

• Occasional Danish/Norwegian word-formation slips in v44 — "vækning" (not a word) and the hybrid "fat-mål". The macro vocabulary itself is now correct, but morphology is not reliable.

• The /nutrition command line is improvised. The corpus contains ZERO examples of it; the format lives only in the prompt. v44 emits the line on 7/7 applicable cases (v42 omitted it entirely on 2/7) but drops the trailing fat value on 3/7, so the app applies three of the four fields. Harmless by design — NutritionCommandParser accepts any subset — but the fat target silently stays stale.

• Within-day duplicate or misplaced exercises still appear occasionally in generated routines (e.g. a chest isolation movement landing in a Legs day). Lower real-world impact than it sounds, since the app's own parser drops duplicate slots.

• Confabulation-despite-coverage on a few specific lifts (squat/hip-thrust blending, farmer's carry) that does not reliably respond to more training data.

• Superset pairing — avoiding two competing-muscle compounds back to back — is imperfect.

• Tight macro-budget precision (±8–10%) is unsolved across every model tested, including the untouched baseline.

Evaluation discipline

Every promote/reject decision reads every row of the eval suite, never a sample — an early round reported a verdict from ~12 of 52 rows and missed hard timeout loops entirely. Prose quality is judged as its own axis alongside structural correctness, because a structurally clean answer can still be flat, templated, or subtly wrong in a second language. Candidates are always compared against both the previous champion and the untouched baseline.

Files

Which file the app fetches

Two files are published. The app decides which one to fetch based on its build version; there is nothing to select and nothing to configure.

Both carry the same v46 coach weights (v47 changes only the vision export, not the weights), so an older build is not stuck on an older coach. They exist side by side because installed builds also fetch a separate 3.66 GB vision model — handing them the larger consolidated file would raise their total download on devices already tight on RAM. See the current-champion section above for what the consolidation actually changes.

• smart-coach-vision.litertlm — ⭐ current champion, fetched by the next app release. The v46 coach WITH a working vision tower: one model for text coaching, body-fat photos, meal photos and nutrition labels (2.99 GB, int8 output layer). The next app release points both the coach and vision URLs here and stops downloading the separate 3.66 GB vision model.
• smart-coach.litertlm — STABLE, text-only (2.63 GB). The file app builds already in the wild download at runtime on Android and iOS, and the one they keep using. Deliberately left unchanged: those builds also fetch the separate vision model, so giving them the larger consolidated file would raise their total download on devices that are already RAM-constrained. Same v46 coach weights, so nobody is stuck on an older coach for stability's sake.
• coach.litertlm — the original working int8 build from the v1 era (2.59 GB), kept as the historical starting point.
• coach-finetuned-int4.litertlm — the first int4 conversion (2.56 GB), kept because it is the artifact that exposed the LiteRT-LM chat-template .get() incompatibility.
• model.safetensors + config.json + tokenizer files — HF-format artifacts for reference.