bytebox

<style> img { display: block; position: static; width: min(76%, 480px); height: auto; max-width: 100%; margin: 3rem auto 2.5rem; object-fit: contain; border: 0px solid rgba(255, 255, 255, 0.16); border-radius: 1rem; outline: none; user-select: none; -webkit-user-select: none; -moz-user-select: none; -webkit-user-drag: none; filter: none !important; transform: scale(1) !important; animation: none !important; box-shadow: none !important; position: relative; z-index: 1; transform-origin: center; } </style> <img src="https://huggingface.co/appvoid/bytebox.1/resolve/main/bytebox.png"/>

details

bytebox is a model designed to change public perspective on byte-level language models. It's a deep model pretrained on roughly 1B tokens that demonstrates instruction-following skills with only 60m parameters. Training and inference codes are included.

To the author's knowledge, this is the first-ever decoder-only instruction pretrained byte-level transformer openly available to date.

python infer_bytefast60m.py --architecture-script bytefalcon_fast60m.py \
  --model /bytefast-60m/ \
  --tokenizer /byte-tokenizer/ \
  --instruction "Fix the sentence" \
  --text "this sentence need fixing you know i love it" \
  --max-new-tokens 128 \
  --temperature 0

---

Prompt

What is Python?

Output

Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than would be possible in languages such as C++ or Java. The language provides confidence towards the use of programming languages and their debugging problems."<eos>