notion-agent-gguf

Notion MCP Specialized Local Model System

This directory contains the training pipeline, dataset generator, quantization scripts, and Ollama Modelfile for building and deploying notion-agent — an ultra-compact (~980MB RAM), fast local AI planner and router specialized for the Notion Model Context Protocol (MCP).

---

🎯 Architecture & Objective

The goal of notion-agent is to run locally on resource-constrained laptops (e.g. 16GB RAM / Intel CPU) to:

1. Directly Execute Notion MCP Tools: Handle routine tasks like API-post-page (task/note creation) and API-query-data-source (status checks) with 0 cloud latency and 0 Gemini API quota usage.
2. Deterministic Routing: Act as a primary router. Escalate complex multi-document synthesis or creative queries to Google Gemini Cloud (gemini-3.7-flash).

flowchart TD
    User["👤 User Prompt"] --> NotionAgent["🤖 notion-agent (Local Qwen 2.5 1.5B / 0.5B)"]
    
    NotionAgent --> Decision{"Deterministic Routing Decision"}
    
    Decision -->|"Action: CALL_NOTION_TOOL\n(Tasks / Notes CRUD)"| MCP["📦 Notion MCP Server (Local)"]
    Decision -->|"Action: ESCALATE_TO_GEMINI\n(Deep Cross-Doc Synthesis)"| Gemini["✨ Google Gemini 3.7 Flash"]
    Decision -->|"Action: DIRECT_ANSWER\n(Simple greeting)"| Direct["💬 Instant Local Response"]

    MCP --> Output["📊 Response with Execution Trace"]
    Gemini --> Output
    Direct --> Output

---

🚀 Quick Start (Local Build in 30 Seconds)

Prerequisites:

• Install Ollama for Windows.

Build and Register notion-agent:

Run the one-click build script from this directory:

model\scripts\build_model.bat

Or run via Ollama CLI:

cd model
ollama pull qwen2.5:1.5b
ollama create notion-agent -f Modelfile

Test it in terminal:

ollama run notion-agent "Add task 'Review Q3 metrics' with High priority"

---

📂 Directory Layout

model/
├── Modelfile                     <- Ollama custom model definition with Notion MCP rules
├── README.md                     <- This documentation guide
├── data/
│   └── notion_mcp_dataset.json   <- Synthetic training & fine-tuning dataset
├── scripts/
│   └── build_model.bat           <- One-click Windows builder script
└── src/
    ├── dataset_generator.py      <- Generates thousands of synthetic Notion prompt/tool pairs
    ├── train_lora.py             <- PEFT/LoRA fine-tuning script for Qwen 2.5
    ├── export_gguf.py            <- GGUF conversion & 4-bit quantization (llama.cpp)
    └── upload_to_hf.py           <- Uploads quantized GGUF directly to Hugging Face Hub

---

🧠 Training & Fine-Tuning Pipeline

1. Generate Synthetic Notion MCP Dataset

To generate fresh prompt-to-tool training pairs:

python model/src/dataset_generator.py

2. Fine-Tune with LoRA (Unsloth / TRL)

python model/src/train_lora.py

3. Quantize to GGUF (Q4_K_M)

Merge LoRA weights and quantize using llama.cpp:

python model/src/export_gguf.py

---

🤗 Publishing & Hosting on Hugging Face

Because Git repositories have a 100MB file limit, we host quantized GGUF weights on Hugging Face Hub (100% free and unlimited):

1. Create a free account at huggingface.co.
2. Go to Settings -> Access Tokens and generate a new token with write permissions.
3. Upload your quantized model using the included upload script:

   python model/src/upload_to_hf.py ./notion-qwen-1.5b.Q4_K_M.gguf <your-username>/notion-mcp-qwen-1.5b-gguf <your-hf-token>
   

4. Any user can now pull and run your model directly:

   ollama run hf.co/<your-username>/notion-mcp-qwen-1.5b-gguf