Fara-7B-GGUF

<span style="color: #7FFF7F;">Fara-7B GGUF Models</span>

<span style="color: #7F7FFF;">Model Generation Details</span>

This model was generated using llama.cpp at commit 3d07caa99.

---

<span style="color: #7FFF7F;">Quantization Beyond the IMatrix</span>

I've been experimenting with a new quantization approach that selectively elevates the precision of key layers beyond what the default IMatrix configuration provides.

In my testing, standard IMatrix quantization underperforms at lower bit depths, especially with Mixture of Experts (MoE) models. To address this, I'm using the --tensor-type option in llama.cpp to manually "bump" important layers to higher precision. You can see the implementation here:
👉 Layer bumping with llama.cpp

While this does increase model file size, it significantly improves precision for a given quantization level.

I'd love your feedback—have you tried this? How does it perform for you?

---

<a href="https://readyforquantum.com/huggingface_gguf_selection_guide.html" style="color: #7FFF7F;"> Click here to get info on choosing the right GGUF model format </a>

---

<!--Begin Original Model Card-->

Fara-7B: An Efficient Agentic Model for Computer Use

Official Microsoft Blog<br> Technical Report<br> Github<br> Microsoft Foundry<br>

Model Summary

Developer: Microsoft Research

Description:
Fara-7B is Microsoft's first agentic small language model (SLM) designed specifically for computer use. With only 7 billion parameters, Fara-7B is an ultra-compact Computer Use Agent (CUA) that achieves state-of-the-art performance within its size class and is competitive with larger, more resource-intensive agentic systems.

Model Architecture:
Multimodal decoder-only language model that takes an image (screenshot) + text context. It directly predicts thoughts and actions with grounded arguments. Current production baselines leverage Qwen 2.5-VL (7B).

Parameters: 7 Billion

Inputs: User goal (text), current screenshot(s), history of previous outputs (thoughts + actions text) from the agent.

Context Length: 128k

Outputs: Generated text in response to the input, with a chain-of-thought block followed by a tool call block to indicate the action.

GPUs: 64 H100s

Training Time: 2.5 days

Public Data Summary: N/A

Dates: Trained between 26th October 2025 to 29th October 2025

Status: Static model trained on public and private data

Release Date: November 24th, 2025

License: MIT

Model Dependencies: Qwen 2.5 VL

Additional Assets: N/A

Acceptable Use Policy: N/A

---

1. Model Overview

Fara is a 7B Computer Use Agent (CUA) model specialized for taking actions on the web to accomplish high-level user tasks. Beyond understanding webpage layout and basic action mechanics, it plans and executes high-level goals like booking restaurants, applying for jobs, planning trips, and buying shopping lists. Its training relies on a large-scale, fully synthetic dataset of action trajectories generated and verified by a multi-agent pipeline.

Fara perceives browser inputs via screenshots, while internal reasoning and state history are recorded textually. Based on recent screenshots and a full history of actions, it predicts the next action with necessary arguments (e.g., coordinates for clicks).

1.1 Alignment Approach

Fara-7B uses a robust post-training safety approach leveraging open-source and in-house synthetic datasets. It incorporates critical point recognition—situations requiring user permission or sensitive information—to safely halt actions. The model is trained to refuse harmful tasks and undergoes automated red teaming to assess risks, including grounding, jailbreaks, harmful content, and copyright violations.

1.2 Safeguards

Fara-7B is trained to refuse tasks in categories that violate usage policy:

Critical points where the agent stops include entering personal info, completing purchases, making calls, sending emails, submitting applications, and signing into accounts.

---

2. Usage

2.1 Primary Use Cases

• Automating web tasks such as shopping, booking travel, restaurant reservations, info-seeking, or account workflows.
• Performs actions step-by-step using multimodal understanding from browser screenshots.
• On-device execution provides privacy guarantees and lower latency.

2.2 Out-of-Scope Use Cases

• Model not evaluated for all downstream purposes; consider limitations of LLMs for accuracy, safety, and fairness.
• Must adhere to applicable laws and regulations.
• English-only support.

2.3 Distribution Channels

• Hugging Face
• Azure AI Foundry

2.4 Input Formats

Given the nature of the training data, always use the ChatML template with the following system prompt for inference:

---

System Prompt:

You are a web automation agent that performs actions on websites to fulfill user requests by calling various tools.

You should stop execution at Critical Points. A Critical Point occurs in tasks like:

• Checkout
• Book
• Purchase
• Call
• Email
• Order

A Critical Point requires the user's permission or personal/sensitive information (name, email, credit card, address, payment information, resume, etc.) to complete a transaction (purchase, reservation, sign-up, etc.), or to communicate as a human would (call, email, apply to a job, etc.).

Guideline: Solve the task as far as possible up until a Critical Point.

Examples:

• If the task is to "call a restaurant to make a reservation," do not actually make the call. Instead, navigate to the restaurant's page and find the phone number.
• If the task is to "order new size 12 running shoes," do not place the order. Instead, search for the right shoes that meet the criteria and add them to the cart.

Some tasks, like answering questions, may not encounter a Critical Point at all.

---

Function Signatures:

You are provided with function signatures within XML tags:

{
  "type": "function",
  "function": {
    "name": "computer_use",
    "description": "Use a mouse and keyboard to interact with a computer, and take screenshots.\n* This is an interface to a desktop GUI. You do not have access to a terminal or applications menu. You must click on desktop icons to start applications.\n* Some applications may take time to start or process actions, so you may need to wait and take successive screenshots to see the results of your actions. E.g. if you click on Firefox and a window doesn't open, try wait and taking another screenshot.\n* The screen's resolution is 1428x896.\n* Whenever you intend to move the cursor to click on an element like an icon, you should consult a screenshot to determine the coordinates of the element before moving the cursor.\n* If you tried clicking on a program or link but it failed to load, even after waiting, try adjusting your cursor position so that the tip of the cursor visually falls on the element that you want to click.\n* Make sure to click any buttons, links, icons, etc with the cursor tip in the center of the element. Don't click boxes on their edges unless asked.\n* When a separate scrollable container prominently overlays the webpage, if you want to scroll within it, you typically need to mouse_move() over it first and then scroll().\n* If a popup window appears that you want to close, if left_click() on the 'X' or close button doesn't work, try key(keys=['Escape']) to close it.\n* On some search bars, when you type(), you may need to press_enter=False and instead separately call left_click() on the search button to submit the search query. This is especially true of search bars that have auto-suggest popups for e.g. locations\n* For calendar widgets, you usually need to left_click() on arrows to move between months and left_click() on dates to select them; type() is not typically used to input dates there.",
    "parameters": {
      "properties": {
        "action": {
          "description": "The action to perform. The available actions are:\n* key: Performs key down presses on the arguments passed in order, then performs key releases in reverse order. Includes 'Enter', 'Alt', 'Shift', 'Tab', 'Control', 'Backspace', 'Delete', 'Escape', 'ArrowUp', 'ArrowDown', 'ArrowLeft', 'ArrowRight', 'PageDown', 'PageUp', 'Shift', etc.\n* type: Type a string of text on the keyboard.\n* mouse_move: Move the cursor to a specified (x, y) pixel coordinate on the screen.\n* left_click: Click the left mouse button.\n* scroll: Performs a scroll of the mouse scroll wheel.\n* visit_url: Visit a specified URL.\n* web_search: Perform a web search with a specified query.\n* history_back: Go back to the previous page in the browser history.\n* pause_and_memorize_fact: Pause and memorize a fact for future reference.\n* wait: Wait specified seconds for the change to happen.\n* terminate: Terminate the current task and report its completion status.",
          "enum": ["key", "type", "mouse_move", "left_click", "scroll", "visit_url", "web_search", "history_back", "pause_and_memorize_fact", "wait", "terminate"],
          "type": "string"
        },
        "keys": {"description": "Required only by action=key.", "type": "array"},
        "text": {"description": "Required only by action=type.", "type": "string"},
        "coordinate": {"description": "(x, y) coordinates for mouse actions. Required only by action=left_click, action=mouse_move, and action=type.", "type": "array"},
        "pixels": {"description": "Amount of scrolling. Positive = up, Negative = down. Required only by action=scroll.", "type": "number"},
        "url": {"description": "The URL to visit. Required only by action=visit_url.", "type": "string"},
        "query": {"description": "The query to search for. Required only by action=web_search.", "type": "string"},
        "fact": {"description": "The fact to remember for the future. Required only by action=pause_and_memorize_fact.", "type": "string"},
        "time": {"description": "Seconds to wait. Required only by action=wait.", "type": "number"},
        "status": {"description": "Status of the task. Required only by action=terminate.", "type": "string", "enum": ["success", "failure"]}
      },
      "required": ["action"],
      "type": "object"
    }
  }
}

For each function call, return a JSON object with the function name and arguments within XML tags:

```json
{
  "name": "<function-name>",
  "arguments": <args-json-object>
}

• Function signatures provided for all actions (key, type, mouse_move, left_click, scroll, visit_url, web_search, history_back, pause_and_memorize_fact, wait, terminate).

2.5 Technical Requirements & Integration

• Required packages: torch >=2.7.1, transformers >=4.53.3, vllm >=0.10.0
• Tested on NVIDIA A6000, A100, H100 GPUs (Ubuntu 24.04.3 LTS)
• Recommended on vLLM server with bf16 precision
• Provided implementation via Magentic-UI in Docker sandbox for safe web execution

2.6 Responsible AI Considerations

• English-only; other languages may have degraded performance
• Potential stereotype reinforcement or inappropriate content
• Verify outputs, especially in high-stakes or regulated domains
• Misuse includes fraud, spam, malware generation
• Use safety services like Azure AI Content Safety where possible
• Recommended: human-in-the-loop, sandboxing, access control, output verification

---

3. Data Overview

3.1 Training, Testing, Validation Datasets

• Multi-agent data generation pipeline produces synthetic trajectories from seed URLs and open-source tasks
• Records screenshots, thoughts, action traces, and verification via verifier agents
• Includes high-quality public datasets: image and text modalities
• Specialized data: grounding, UI understanding (VQA, captioning, OCR), safety/refusal datasets

---

4. Quality and Performance Evaluation

Table: Online Agent Evaluation Results

The table reports task completion success rates on WebVoyager, Online-Mind2Web, DeepShop, and WebTailBench for both SoM agents and native computer-use agents. Scores are averaged over 3 runs.

4.2 Safety Evaluation & Red-Teaming

• Post-training safety with critical point design
• Red-teaming on Azure: grounding, jailbreaks, harmful content, copyright

Guidelines for Safe Use

• Human-in-the-loop monitoring recommended
• Do not share sensitive data
• Run in sandboxed environments
• Limit internet access via allow-lists/block-lists
• Avoid use in commercial, high-stakes, or regulated domains

Security Considerations:

• Automates interactions across websites, apps, OS; requires strict access control, sandboxing, and monitoring

Contact for More Information: [email protected]

---

Appendix: Benchmarks

<!--End Original Model Card-->

---

<span id="testllm" style="color: #7F7FFF;">🚀 If you find these models useful</span>

Help me test my AI-Powered Quantum Network Monitor Assistant with quantum-ready security checks:

👉 Quantum Network Monitor

The full Open Source Code for the Quantum Network Monitor Service available at my github repos ( repos with NetworkMonitor in the name) : Source Code Quantum Network Monitor. You will also find the code I use to quantize the models if you want to do it yourself GGUFModelBuilder

💬 How to test:
Choose an AI assistant type:

• TurboLLM (GPT-4.1-mini)
• HugLLM (Hugginface Open-source models)
• TestLLM (Experimental CPU-only)

What I’m Testing

I’m pushing the limits of small open-source models for AI network monitoring, specifically:

• Function calling against live network services
• How small can a model go while still handling:
  • Automated Nmap security scans
  • Quantum-readiness checks
  • Network Monitoring tasks

🟡 TestLLM – Current experimental model (llama.cpp on 2 CPU threads on huggingface docker space):

• ✅ Zero-configuration setup
• ⏳ 30s load time (slow inference but no API costs) . No token limited as the cost is low.
• 🔧 Help wanted! If you’re into edge-device AI, let’s collaborate!

Other Assistants

🟢 TurboLLM – Uses gpt-4.1-mini :

• **It performs very well but unfortunatly OpenAI charges per token. For this reason tokens usage is limited.
• Create custom cmd processors to run .net code on Quantum Network Monitor Agents
• Real-time network diagnostics and monitoring
• Security Audits
• Penetration testing (Nmap/Metasploit)

🔵 HugLLM – Latest Open-source models:

• 🌐 Runs on Hugging Face Inference API. Performs pretty well using the lastest models hosted on Novita.

💡 Example commands you could test:

1. "Give me info on my websites SSL certificate"
2. "Check if my server is using quantum safe encyption for communication"
3. "Run a comprehensive security audit on my server"
4. '"Create a cmd processor to .. (what ever you want)" Note you need to install a Quantum Network Monitor Agent to run the .net code on. This is a very flexible and powerful feature. Use with caution!

Final Word

I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is open source. Feel free to use whatever you find helpful.

If you appreciate the work, please consider buying me a coffee ☕. Your support helps cover service costs and allows me to raise token limits for everyone.

I'm also open to job opportunities or sponsorship.

Thank you! 😊