braille256-v5

braille256-v5: Multimodal Universal Braille Model

The first language model trained on multimodal data encoded as 8-dot Braille.

Key Innovation

This model demonstrates that any data type (text, images, audio, binary) can be encoded into 8-dot Braille Unicode (U+2800-U+28FF) and processed by a single unified model.

def byte_to_braille(byte: int) -> str:
    """Direct 1:1 mapping: 256 bytes → 256 Braille patterns"""
    return chr(0x2800 + byte)

Model Details

Training Data

• Text: 218 files from Project Gutenberg (7 languages) encoded as 8-dot Braille
• Audio: Synthetic WAV files encoded as Braille
• Total: 2M tokens from multimodal corpus

Compression

Universal Encoding

The 8-dot Braille encoding enables:

1. Text → UTF-8 bytes → Braille
2. Images → Raw bytes → Braille
3. Audio → WAV bytes → Braille
4. Any file → Bytes → Braille

Modality Headers

⣿⠁ = TEXT
⣿⠃ = IMAGE
⣿⠇ = AUDIO
⣿⠏ = BINARY

Usage

import torch
import sentencepiece as spm

# Load tokenizer
sp = spm.SentencePieceProcessor()
sp.load("tokenizer.model")

# Load model
from train_multimodal_v5 import Braille256MultimodalModel, MultimodalConfig
import json

with open("config.json") as f:
    config = MultimodalConfig.from_dict(json.load(f))

model = Braille256MultimodalModel(config)
model.load_state_dict(torch.load("pytorch_model.bin", map_location="cpu"))
model.eval()

# Encode any data as Braille
def bytes_to_braille(data: bytes) -> str:
    return ''.join(chr(0x2800 + b) for b in data)

# Generate from Braille prompt
braille_text = "⠞⠓⠑⠀⠟⠥⠊⠉⠅"  # "the quick" in Braille
tokens = sp.encode(braille_text)
input_ids = torch.tensor([tokens])
output = model.generate(input_ids, max_length=50)
generated = sp.decode(output[0].tolist())
print(generated)

Model Family

Why 8-dot Braille?

• 256 patterns = exactly 1 byte
• Universal encoding: Any data → Braille
• Tactile AI: Blind users can "feel" any data
• Cross-modal learning: Single representation for all modalities

Citation

@misc{braille256v5,
  title={braille256-v5: Multimodal Universal Braille Model},
  author={Barrett, Ryan},
  year={2024},
  publisher={HuggingFace},
  url={https://huggingface.co/ryanscottbarrett/braille256-v5}
}

License

MIT