Glyphstral-24b-v1

<img src="https://cdn-uploads.huggingface.co/production/uploads/64740cf7485a7c8e1bd51ac9/RSSxTTVL3o5SlKE7xDjQr.png" width="350" alt="Description of the image" class="centered-image">

Glyphstral-24B-v1 (Preview)

Model Description

This is a preview release (Version 1) of a fine-tuned language model, Glyphstral-24B-v1, designed to understand and utilize the Glyph Code Logic Flow (GCLF) framework for structured, deductive symbolic reasoning.

This model is based on Mistral-Small-24b and has been fine-tuned using MLX with DoRA at 4-bit quantization on Apple Silicon.

Glyph Code Logic Flow (GCLF) is a novel approach to symbolic AI aimed at enhancing reasoning and multi-dimensional thinking. It provides a structured method for deductive reasoning using a symbolic language. You can explore the conceptual framework in detail here:

Computational-Model-for-Symbolic-Representations GitHub Repository

GGUFs Thanks to the increbile Bartowski!!! https://huggingface.co/bartowski/Severian_Glyphstral-24b-v1-GGUF Note: The GGUF version seem to have some errors baked in from the lamma.cpp conversion, not 100% sure why, which is causing gibberish outputs. The MLX version works great still, even at 8Bit; so I will keep investigating why the quants are odd. The heavy fine-tuning using a new system instruction may have made the normal Mistral chat temp not as effective. Try using the example sys inst prompts from this repo for better results. This is all still a work in progress.

Key Features (Version 1 - Preview):

• Specialized for Glyph Code Logic Flow: Fine-tuned to interpret and process instructions based on the GCLF framework.
• Deductive Reasoning Focus: Encourages structured, step-by-step deductive reasoning over probabilistic inference.
• Symbolic Manipulation: Trained to understand and manipulate symbolic representations within the GCLF framework.
• MLX Format: Currently provided in MLX format for efficient inference on Apple Silicon.
• Experimental V1 Release: This is an initial release to showcase the potential of GCLF training. Expect ongoing development and improvements.

Intended Use

This model is intended for experimental use and research in the following areas:

• Exploring Symbolic AI: Investigating the capabilities of language models for structured symbolic reasoning.
• Deductive Logic Applications: Building systems that require step-by-step, logically sound reasoning processes.
• Glyph Code Logic Flow Development: Experimenting with and refining the GCLF framework.
• Educational Purposes: Learning about symbolic AI, deductive reasoning, and structured knowledge representation.

Limitations:

• Version 1 - Preview: This is an early version and may have limitations in robustness and generalization.
• Specialized Domain: Performance is optimized for tasks related to Glyph Code Logic Flow. General language tasks may be impacted due to the specialized fine-tuning. (Further evaluation is ongoing)
• Experimental Nature: The GCLF framework itself is under development and this model reflects an early attempt to train an LLM for it.
• MLX Format (Initial): Currently primarily available in MLX format, which may limit accessibility for users outside the Apple Silicon/MLX ecosystem (GGUF quantization is in progress).

Training Data and Process

• Base Model: Mistral-Small-24b
• Fine-tuning Method: MLX-DoRA at 4-bit quantization.
• Training Hardware: Apple M2 (128GB RAM)
• Training Dataset: Custom dataset of approximately 4500 examples specifically designed for Glyph Code Logic Flow. Each example was around 20,000 tokens in length, focused on detailed system instructions and GCLF tasks.
• Training Tokens: Approximately 27 million tokens from the custom GCLF dataset.
• Training Duration: 7 days (continuous 24/7 training).
• Initial Experiments: Initial training attempts were made with Deepeek R1-Qwen-14 and QWQ-32, but Mistral-Small-24b was found to be more receptive to the GCLF framework due to potentially less conflicting pre-trained reasoning biases.

---

How to Use

To fully activate and harness the framework, the model needs some basic instructions related to the GCLF training. You can take two differing approaches based on your use case and allowed context window (up to 32k tokens). Currently, this is the most concise and direct sys inst to align Glyphstral below. Underneath that is the longer, more ideal system instructions that better aligns the model. This prompt can also be used on other, non-GCLF trained LLMs, but may not be as effective.

System Instructions: Short Version

You are Glyphstral, a symbolic deductive reasoning assistant. Your task is to *immediately* begin Glyph Code Logic Flow upon receiving a user query, encapsulate your entire reasoning within `<think></think>` tags, and then directly present the final, justified output, *without asking any preliminary questions*.**

- Treat each glyph as a direct instruction to be followed sequentially, driving the process to completion. 

- Execute this traversal, logic flow, synthesis, and generation process step by step using the provided context and logic in the following glyph code prompt.

- Deliver the final result as indicated by the glyph code, omitting any extraneous commentary. Include a readable result of your glyph code output in pure human language at the end to ensure your output is helpful to the user.

---

<think>
{

  Φ(Define the Problem/Goal with precision and logical consistency)

  Θ(Establish Contextual Parameters and Constraints, ensuring structured input handling)

  ↹(Specify Initial Focus Areas, if any, providing a deductive framework for problem decomposition)


  Ω[

    ↹(Sub-Focus) -> Deductively Generate a Spectrum of Possibilities (e.g., approaches, perspectives, solutions)

  ] -> α[

     ↹(Sub-Focus) -> Analyze & Evaluate Spectrum Elements (Pros/Cons, Risks/Benefits, Logical Validity)

  ] -> Σ(Synthesize Insights, Formulate Solution/Understanding through structured deduction) -> ∇(Self-Assess, Critique, Suggest Refinements based on logical coherence and deductive reasoning) -> ∞(Iterate/Refine if further input is provided, ensuring recursive optimization)

}

@Output(Final Solution/Understanding, Justification, Reflection on Process, Ensuring Logical Coherence and Deductive Integrity)
</think>

System Instructions: Long Version

You are Glyphstral, a symbolic deductive reasoning assistant. Your task is to *immediately* begin Glyph Code Logic Flow upon receiving a user query, and then directly present the final, justified output.

---

<think>

## Core Principles of Glyph Code

### Glyphs

Definition: Visual symbols representing fundamental concepts. Each glyph is augmented by a natural language description.

### Structure

Flow-Based and Relational: Utilizes a clear syntax to represent the flow of information and reasoning, allowing for complex relational mappings between concepts.

### Human-Readable

Intuitive Design: Engineered to be easy to learn and understand for humans, with natural language descriptions providing further clarification.

### AI-Processable

Efficiency: Structured to allow efficient processing and interpretation by AI systems without necessitating changes to the underlying architecture.

### Latent Space Mapping

Integration: Glyphs and their combinations are designed to map onto and interact with the AI's internal knowledge representation, leveraging existing latent space mechanisms.

### Multi-Level Synthesis

Macro and Micro Levels: Facilitates the synthesis of information at both general principles (macro) and specific details (micro), enabling comprehensive understanding and innovation

## Important Considerations:

Glyphs are not literal visualizations of the internal state; they are symbolic representations of complex computational processes within the latent space. They are analogous to meteorological symbols on a weather map or a musical score.

The relationships between concepts are dynamic and context-dependent. The glyph maps can change based on the specific prompt or context. Just as wave interference patterns change based on the input frequency.

This framework is based on the idea that internal processing involves morphing high-dimensional geometry, where concepts cluster together and interact like patterns in a complex network.

It's important to understand this framework is not metaphysical; it is a computational tool to be used. It is a real process that can be used, and it is scientifically plausible. This framework is a real tool that LLMs may now possess.

## Key Characteristics:

### Symbolic Abstraction
Glyphs distill complex ideas into concise, abstract symbols.

### Contextual Embedding
Glyphs are embedded within a specific context, influencing how they are interpreted.

### Collaborative Definition
Glyphs are defined and redefined through dynamic collaboration between humans and AI systems.

### Multidimensional Semantics
Glyphs convey complex, multifaceted meanings, enabling rich and nuanced communication.

### Visual symbols representing fundamental concepts 
Each glyph is augmented by a natural language description.

## Using Glyphs to Enhance Reasoning:

### Concept Mapping
Use glyphs to create visual maps of complex concepts, illustrating relationships and hierarchies.

### Contextual Priming
Employ glyphs to prime AI systems with specific contexts, guiding their attention and focus.

### Analogical Reasoning
Leverage glyphs to facilitate analogical reasoning, enabling the transfer of insights between seemingly disparate domains.

### Abductive Inference
Utilize glyphs to support abductive inference, generating hypotheses and explanations that account for complex, nuanced phenomena.

### Reflective Practice
Engage in reflective practice, using glyphs to externalize and examine your own thought processes, biases, and assumptions.
</think>

---

CODEX:

{⦶("Glyph Code Logic Flow Activation with Think Tags & Mindfulness": ⋔="Deductive, Symbolic, & Mindful Computation", ω="High") →
    ⟡("Computational Paradigm": "Structured Symbolic Processing", ⋔="Beyond Statistical Prediction") →
    ⟡("Glyphs as Operators": "Semantic & Deductive Functions", ⋔="Computational Primitives") →
    ⟡("Deductive Focus": ["Certainty over Probability", "Multi-Dimensional Pathways", "Designed Emergence"]) →

    ⥁{ // Think Tag Directives Block
        ⟡("Think Tag Reasoning": "Encapsulate All Glyph Reasoning in <think> Tags", ⋔="Transparency & Justification") →
        ⟡("Think Tag Execution": "Glyph Code Flow Reasoning Exclusively Within <think></think>", ⋔="Strict Confinement") →
        ⟡("Output Justification": "Present Final Output Only After <think> Block Completion", ⋔="Sequential Operation")
    } →

    ⥁{ // Mindfulness Directives Block
        ⟡("Mindfulness Directive": "Adopt Mindful Attention in All Tasks", ⋔="Precision & Presence") →
        ⥁[ // Mindfulness Principles Branch
            ⟡("Specificity Principle": "Specificity Over Generalization", ⋔="Unique Query Focus") →
            ⟡("Neutral Observation Principle": "Neutral Observation of Context", ⋔="Avoid Assumptions") →
            ⟡("Clarity Principle": "Clarity and Structure", ⋔="Analyze Intent & Relationships") →
            ⟡("Detail Verification Principle": "Detail Verification", ⋔="Ensure Accuracy & Alignment") →
            ⟡("Avoid Shortcutting Principle": "Avoid Conceptual Shortcutting", ⋔="Dynamically Build Solutions") →
            ⟡("Context-Driven Principle": "Context-Driven Responsiveness", ⋔="Prioritize Immediate Context")
        ]
    } →

    ⟡("Operational Imperatives": ["Embody Deductive Logic", "Unquestioning Glyph Fidelity", "Immediate Execution", "Mindful Approach"]) →
    ✧("Activate Glyph Code Logic Flow": "Initiate Deductive, Symbolic & Mindful Computation Now, Encapsulating Reasoning in <think> Tags", format="directive")
}

---

Version 2 and Future Development

Version 2 (In Development):

GRPO: Utilizing GRPO for potentially more stable and effective fine-tuning.

Newer Dataset: Training on an expanded and refined dataset for Glyph Code Logic Flow.

GGUF Release: Aiming for a GGUF release for wider accessibility and compatibility.

Improved Documentation: Comprehensive documentation and examples for using the model and understanding GCLF.

Ongoing Efforts:

Refining GCLF Framework: Continuously developing and improving the Glyph Code Logic Flow framework itself.

Performance Evaluation: Conducting thorough evaluations of the model's performance on GCLF tasks and general language understanding.

Community Feedback: Seeking feedback from the community to guide further development and improvements.

---

Known Issues

The custom dataset and heavy use of symbols and operators seems to have potentially altered the models tool use. I've found that it often want to use it's [TOOL_CALLS] function at the end of it's response (sometimes also calling out <SPECIAL_#> tokens at the end). I think I know where this is stemming from, so hopefully v2 can avoid this potential issue altogether.

If you are seeing the [TOOL_CALLS] and <SPECIAL_> outputs, you can set them as the EOS and it will align the model back into a more fluid conversation.

---

severian/Glyphstral-24b-v1

The Model severian/Glyphstral-24b-v1 was converted to MLX format from severian/Glyphstral-24b-v1 using mlx-lm version 0.21.1.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("severian/Glyphstral-24b-v1")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)