Qwen3.5-0.8B-Hypnosis-Sentence-Pacing

Model Card: Qwen3.5-0.8B-Hypnosis-Sentence-Pacing

This is a specialized causal language model finetuned to predict reading speeds and pause durations for natural language transcripts. Built on top of unsloth/Qwen3.5-0.8B-Base, the model processes continuous text and inserts targeted prosody tokens representing tenth-of-a-second delays and calculated reading velocity.

Model Details

• Developed by: selfhypnosis-ai
• Base Model: unsloth/Qwen3.5-0.8B-Base
• Architecture: 0.8 Billion Parameter Decoder-Only Transformer with Rank-Stabilized LoRA (rsLoRA).
• Precision: bf16
• Context Length: Extended to 52,000 tokens.
• License: Apache 2.0 (Inherited)

Intended Use

This model is designed to act as a bridge between raw text and Text-To-Speech (TTS) engines. Given an un-timed transcript, the model injects pacing annotations to facilitate natural, human-like voice synthesis.

---

<details open> <summary><h3>Training Data & Preprocessing</h3></summary>

The dataset was procedurally generated from ASR (Automatic Speech Recognition) JSON files containing word-level timestamps utilizing a proprietary preprocessing pipeline.

• Rule-Based Linguistic Weighting: A RuleDurationEstimator (adapted from Omni Voice) evaluates text complexity. Characters are assigned cognitive "weights" based on their Unicode block (e.g., CJK=3.0, Hangul=2.5, Latin=1.0, Punctuation=0.5).
• Reading Speed: Sentence speed is determined by dividing the total linguistic weight of a sentence block by its temporal duration. Outputs are normalized (offset by 4) and rendered visually as visual units (e.g., [ > > > > ]). Speeds over 20.0 or word durations under 0.02 seconds are filtered out as ASR hallucinations.
• Pause Extraction: Word-boundary gaps of 0.1 seconds or greater are logged as pauses.
• Visual Tokenization: To maximize alignment and minimize parameter confusion, speeds and delays aren't rendered as standard integers. They are rendered as progress bars:
  • Pauses: [ - - - - ] (Each hyphen represents 0.1 seconds, capped at 3.0 seconds).
  • Speeds: [ > > > ] (Each chevron represents a normalized speed unit).

</details> <details open> <summary><h3>Custom Masking Strategy</h3></summary>

A paramount feature of this training run is the selective masking algorithm.

Standard causal language modeling trains a network to predict every next token. Because this model's sole objective is to insert pauses and speeds into an existing text stream, the loss function ignores standard transcript tokens.

1. All standard tokens, punctuation, and whitespaces receive a label of -100.
2. The model scans for bracketed segments indicating prosody (containing - or >).
3. Only the inner tokens of these speed/pause tags, as well as the closing bracket, are unmasked for loss calculation.
4. The opening bracket [ is intentionally masked so the model purely focuses on the magnitude inside the tags instead of the syntactic triggering of the tag itself.

</details> <details> <summary><h3>Training Hyperparameters</h3></summary>

The model was finetuned utilizing unsloth for high-efficiency VRAM usage and a custom sequence-to-sequence data collator to handle dynamic -100 label padding.

LoRA Configuration:

• Type: Rank-Stabilized LoRA (use_rslora = True)
• Rank (r): 128
• Alpha: 256
• Target Modules: all-linear (Q, K, V, O, gate, up, down)
• Dropout / Bias: 0 / none

Optimization:

• Optimizer: adamw_8bit
• Learning Rate: 1e-5
• Scheduler: Cosine with 50 warmup steps
• Weight Decay: 0.01
• Epochs: 3

</details>

---

Production Inference Strategy

Because of the model's specialized training masking, it is not designed to take a large block of text and autonomously rewrite it with tags via a single generation pass. Instead, it acts as a stateful orchestrator that is continuously iteratively polled.

To use this model correctly in production, inference must be managed incrementally using a deterministic extraction loop:

1. Incremental Context Building

Text is parsed incrementally (e.g., word-by-word or by phrase breakpoints). Words are appended to a running "prompt context" containing all historically generated text and raw symbols.

2. Forced Decoding (Tag Polling)

Because the opening bracket [ was intentionally heavily masked during training, the model will rarely output it organically. At desired sentence boundaries, the inference engine must aggressively prompt the model by appending an opening bracket [ to the context and instructing the language model to generate tokens, with the closing bracket ] set as the strict stopping criteria.

This is performed in a two-pass sequence for each boundary:

• Speed Pass: The model evaluates the preceding sentence and generates the speed magnitude (e.g., > > > >).
• Pause Pass: Immediately following the speed tag, another exact interval is polled to extract the subsequent pause length (e.g., - - -).

3. Translation & Output

The visual symbols generated by the model are intercepted by the inference engine and translated via deterministic rules:

• Speed Decoding: The chevron count > is mathematically decoded by restoring the offset (adding 4) and comparing it against the calculated phonetic weight of the phrase via the Omni Voice duration estimator. This yields an exact sentence float duration (e.g., 4.2 seconds).
• Pause Decoding: The hyphen count - is multiplied by 0.1 to yield an exact float duration (e.g., 0.6 seconds).

Custom tags like unknown speed or extended pause are handled as fallback outliers. The converted timestamps can then be formatted cleanly for the TTS engine.

4. Duration Guardrail Splitting

To prevent context drift and ensure biological realism, the inference server monitors the calculated duration. If the decoded length of a single text block exceeds 15.0 seconds, the loop proactively splits the chunk at its nearest center punctuation point and forces the model to re-evaluate the two smaller isolated halves independently.

5. Finalizing Context

Once a boundary's prosody has been evaluated and scaled, the raw output tags (including their brackets) are permanently structurally appended to the model's KV context window. This ensures all subsequent passages inherit the precise rhythmic trajectory of the preceding paragraphs without the end-user needing to view the raw training syntax.