Downloads · 30 days
20
19% of all-time downloads
lightonai-lateon-code-edge-f32
embedme/lightonai-lateon-code-edge-f32
lightonai-lateon-code-edge-f32 is a feature extraction model from embedme. Use it when you need embeddings to search or compare text. The card lists the license as apache-2.0.
Lossless GGUF conversion of lightonai/LateOn-Code-edge for use with litembeddings.
All-time downloads
103
Public
Repo size
69 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf69 MB · 100%
At a glance
- Task
- Feature Extraction
- License
- apache-2.0
- Access
- Public
- Created
- Mar 11, 2026
- Updated
- Mar 11, 2026
- SHA
- d8aa125e
Base models
Model card
From the Hugging Face model README
LateOn-Code-edge (GGUF f32 + Projection)
Lossless GGUF conversion of lightonai/LateOn-Code-edge for use with litembeddings.
This is the original float32 precision — no quantization loss.
Model Details
| Property | Value |
|---|---|
| Base model | lightonai/LateOn-Code-edge |
| Architecture | ModernBERT (17M params) |
| Output dimensions | 48 (after projection) |
| Context length | 8,192 tokens |
| Quantization | f32 (original precision) |
| GGUF size | 66 MB |
| Projection | 256 → 48 (composed from two PyLate Dense layers: 256→512→48) |
| Use case | Fast, CPU-friendly code search with late interaction (ColBERT-style) |
Variants
| Variant | Size | Quality |
|---|---|---|
| f32 (this repo) | 66 MB | Original precision (lossless) |
| f16 | 34 MB | Lossless — 100% top-1 agreement, 240/300 weighted |
| Q8_0 | 19 MB | 79% weighted score, 96-100% top-1 agreement, 3.5× smaller |
Files
| File | Size | Description |
|---|---|---|
lightonai-lateon-code-edge-f32.gguf | 66 MB | ModernBERT encoder in GGUF f32 format |
lightonai-lateon-code-edge-f32.projection | 49 KB | Composed projection matrix (48×256, float32) |
Usage with litembeddings
.load ./build/litembeddings
-- Load model with projection
SELECT lembed_model('lightonai-lateon-code-edge-f32.gguf',
'{"colbert_projection": "lightonai-lateon-code-edge-f32.projection"}');
-- Generate token embeddings for code
SELECT lembed_tokens('async fn get_connection(pool: &Pool) -> Result<Connection>');
-- Code search with MaxSim
SELECT
id, code,
lembed_maxsim(lembed_tokens('database connection pool'), token_emb) AS score
FROM code_embeddings
ORDER BY score DESC
LIMIT 10;
Quantization Quality Benchmark
Tested across 3 codebases (jq/C, Rails/Ruby, FastAPI/Python) with 150 questions total (15 easy + 20 medium + 15 hard per codebase). Weighted scoring: easy×1, medium×2, hard×3 = 100 points per codebase, 300 total.
Aggregate Weighted Scores
| Variant | Weighted Score | Percentage |
|---|---|---|
| f32 | 240 / 300 | 80.0% |
| f16 | 240 / 300 | 80.0% |
| Q8_0 | 237 / 300 | 79.0% |
Per-Corpus Scores
| Corpus | f32 | f16 | Q8_0 |
|---|---|---|---|
| jq (C) | 66/100 | 66/100 | 63/100 |
| Rails (Ruby) | 79/100 | 79/100 | 79/100 |
| FastAPI (Python) | 95/100 | 95/100 | 95/100 |
Quantization Quality (Top-1 Agreement vs f32)
| Corpus | f16 | Q8_0 |
|---|---|---|
| jq | 100.0% | 96.0% |
| Rails | 100.0% | 100.0% |
| FastAPI | 100.0% | 98.0% |
Key Findings
- f16 is lossless — identical weighted score (240/300) and 100% top-1 agreement across all codebases
- Q8_0 loses only 1% — 237/300 vs 240/300, drops only on hard queries in jq corpus
- Q8_0 is fastest — 2.5s avg query vs 3.4s f32 vs 13.4s f16 (CPU without FP16 hardware)
- Easy/medium questions show zero quality difference between all variants
Conversion
Converted using litembeddings' ColBERT converter with PyLate projection support:
python scripts/convert_colbert_to_gguf.py lightonai/LateOn-Code-edge ./models \
--name lightonai-lateon-code-edge-f32 --quantize f32
- Task
- Feature Extraction
- License
- apache-2.0
- Created
- Mar 11, 2026
- Updated
- Mar 11, 2026