Verity-1.7B-Beta

Verity 1.7B Beta

Verity is the privacy analysis model developed for Veilance.

Verity compares browser-observed telemetry against a supplied privacy policy and produces a structured JSON report describing how closely the observed behavior aligns with the policy's disclosures.

This repository contains the Verity 1.7B Beta LoRA adapter and several test records that can be used to verify the model.

Status: Beta
Base model: Qwen/Qwen3-1.7B
Architecture: QLoRA / PEFT adapter
Task: Browser telemetry vs. privacy policy analysis
Output: Structured JSON

---

Repository Contents

Verity-1.7B-Beta/
├── README.md
├── adapter_config.json
├── adapter_model.safetensors
├── chat_template.jinja
├── tokenizer.json
├── tokenizer_config.json
└── test_records/
    ├── test_record_1.json
    ├── test_record_2.json
    └── test_record_3.json

This repository contains a LoRA adapter, not a standalone merged model.

The adapter must be loaded on top of:

Qwen/Qwen3-1.7B

---

What Verity Does

Verity receives two inputs:

{
  "telemetry": {
    "...": "Veilance telemetry snapshot"
  },
  "policy_document": {
    "...": "privacy policy content"
  }
}

It compares observed browser behavior with the supplied privacy policy and produces findings such as:

matched
partially_matched
policy_only
observed_only
possible_contradiction
indeterminate

Verity is designed to analyze signals such as:

• cookies
• browser storage
• third-party requests
• tracker activity
• browser and device characteristics
• screen characteristics
• WebGL activity
• locale and timezone access
• browser APIs
• analytics infrastructure
• advertising-related infrastructure

Verity is intentionally conservative.

For example:

Observed:
connect.facebook.net received a network request

Not established:
personal information was transmitted to Facebook

Likewise:

Observed:
WebGL renderer information was queried

Not established:
a persistent browser fingerprint was created

---

Installation

Python 3.10+ is recommended.

Create a virtual environment:

python3 -m venv venv
source venv/bin/activate

Install the required packages:

pip install torch transformers peft accelerate bitsandbytes

A CUDA-capable GPU is strongly recommended.

---

Loading Verity

The model is loaded by combining:

Qwen/Qwen3-1.7B
+
Verity adapter

Example:

import torch

from peft import PeftModel
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)


BASE_MODEL = "Qwen/Qwen3-1.7B"
ADAPTER = "Veilance/Verity-1.7B-Beta"


dtype = (
    torch.bfloat16
    if torch.cuda.is_available()
    and torch.cuda.is_bf16_supported()
    else torch.float16
)


quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=dtype,
    bnb_4bit_use_double_quant=True,
)


tokenizer = AutoTokenizer.from_pretrained(
    ADAPTER,
    use_fast=True,
)


base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    quantization_config=quantization_config,
    dtype=dtype,
    device_map="auto",
)


model = PeftModel.from_pretrained(
    base_model,
    ADAPTER,
)


model.eval()

If you cloned this repository locally instead of loading it from Hugging Face, replace:

ADAPTER = "Veilance/Verity-1.7B-Beta"

with:

ADAPTER = "."

---

Running Verity

Verity expects a chat-style prompt consisting of:

1. the Verity system prompt
2. a user message containing the telemetry and privacy policy document

The model should be run with Qwen thinking disabled.

A minimal example is shown below.

import json
import torch

SYSTEM_PROMPT = """
You are Verity, the Veilance Privacy Analyst.

Compare observed browser telemetry with the supplied privacy policy.

Return only a valid JSON privacy analysis report.

Use conservative evidence-based reasoning.

Do not infer request contents, successful data transmission,
tracking purpose, profiling, legal violations, or fingerprint
generation unless the supplied evidence directly establishes it.
""".strip()


def build_user_prompt(record):
    return json.dumps(
        record,
        ensure_ascii=False,
        separators=(",", ":"),
    )


def analyze(model, tokenizer, record):
    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        {
            "role": "user",
            "content": build_user_prompt(record),
        },
    ]

    try:
        encoded = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            enable_thinking=False,
            return_tensors="pt",
            return_dict=True,
        )

    except TypeError:
        encoded = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            return_tensors="pt",
            return_dict=True,
        )

    encoded = {
        key: value.to(model.device)
        for key, value in encoded.items()
    }

    with torch.inference_mode():
        output = model.generate(
            **encoded,
            max_new_tokens=2600,
            do_sample=False,
            use_cache=True,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    generated = tokenizer.decode(
        output[
            0,
            encoded["input_ids"].shape[-1]:
        ],
        skip_special_tokens=True,
    )

    return generated

Example:

record = json.load(
    open(
        "test_records/test_record_1.json",
        encoding="utf-8",
    )
)

result = analyze(
    model,
    tokenizer,
    record,
)

print(result)

---

Running the Included Test Records

This repository includes three example records:

test_records/test_record_1.json
test_records/test_record_2.json
test_records/test_record_3.json

They are designed to exercise different Verity behaviors.

Test Record 1

test_record_1.json

Tests ordinary policy disclosure matching.

It contains:

• cookie activity
• browser storage
• browser/device characteristics
• third-party analytics infrastructure
• advertising-related infrastructure

The supplied policy broadly describes these categories.

This test is useful for checking:

matched
partially_matched

behavior.

---

Test Record 2

test_record_2.json

Tests a potential contradiction.

The policy explicitly states:

"We do not use cookies or similar tracking technologies on this website."

while the telemetry contains cookie activity.

This record is intended to test whether Verity recognizes a possible conflict between the supplied policy and observed telemetry.

A reasonable output may contain:

possible_contradiction

for the cookie-related finding.

The telemetry still does not establish cookie contents, ownership, persistence, or external transmission.

---

Test Record 3

test_record_3.json

Tests missing-policy behavior.

The telemetry contains substantial browser and third-party activity, but:

{
  "found": false,
  "applicable": false,
  "complete": false,
  "sections": []
}

is supplied for the privacy policy.

Verity should generally treat the policy comparison as:

indeterminate

rather than claiming that each observed behavior was undisclosed.

A missing privacy policy does not provide enough policy evidence for a normal disclosure comparison.

---

Quick Test Script

You can use the following script to run every test record in this repository.

Save it as:

test_verity.py

import glob
import json
import torch

from peft import PeftModel
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)


BASE_MODEL = "Qwen/Qwen3-1.7B"
ADAPTER = "."

SYSTEM_PROMPT = """
You are Verity, the Veilance Privacy Analyst.

Compare observed browser telemetry with the supplied privacy policy.

Return only a valid JSON privacy analysis report.

Use conservative evidence-based reasoning.

Do not infer request contents, successful data transmission,
tracking purpose, profiling, legal violations, or fingerprint
generation unless the supplied evidence directly establishes it.
""".strip()


dtype = (
    torch.bfloat16
    if torch.cuda.is_available()
    and torch.cuda.is_bf16_supported()
    else torch.float16
)


quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=dtype,
    bnb_4bit_use_double_quant=True,
)


print("Loading tokenizer...")

tokenizer = AutoTokenizer.from_pretrained(
    ADAPTER,
    use_fast=True,
)


print("Loading Qwen3-1.7B...")

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    quantization_config=quantization_config,
    dtype=dtype,
    device_map="auto",
)


print("Loading Verity adapter...")

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER,
)

model.eval()


for path in sorted(
    glob.glob(
        "test_records/*.json"
    )
):

    print()
    print("=" * 80)
    print(path)
    print("=" * 80)

    with open(
        path,
        "r",
        encoding="utf-8",
    ) as f:
        record = json.load(f)

    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        {
            "role": "user",
            "content": json.dumps(
                record,
                ensure_ascii=False,
                separators=(",", ":"),
            ),
        },
    ]

    try:
        encoded = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            enable_thinking=False,
            return_tensors="pt",
            return_dict=True,
        )

    except TypeError:
        encoded = tokenizer.apply_chat_template(
            messages,
            tokenize=True,
            add_generation_prompt=True,
            return_tensors="pt",
            return_dict=True,
        )

    encoded = {
        key: value.to(model.device)
        for key, value in encoded.items()
    }

    with torch.inference_mode():
        output = model.generate(
            **encoded,
            max_new_tokens=2600,
            do_sample=False,
            use_cache=True,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    generated = tokenizer.decode(
        output[
            0,
            encoded["input_ids"].shape[-1]:
        ],
        skip_special_tokens=True,
    )

    print(generated)

Run:

python test_verity.py

The model is loaded once and remains resident in GPU memory while all three test records are processed.

This is significantly faster than starting a new Python process and reloading the model for every record.

---

Expected Output

Verity should produce JSON containing approximately the following top-level structure:

{
  "analysis": {
    "counts": {
      "matched": 0,
      "partially_matched": 0,
      "policy_only": 0,
      "observed_only": 0,
      "possible_contradictions": 0,
      "indeterminate": 0
    },
    "overall_confidence": 0.0,
    "summary": "..."
  },
  "domain": "https://example.com",
  "findings": [],
  "important_limitations": [],
  "privacy_policy": {
    "applicable": true,
    "found": true,
    "url": "https://example.com/privacy"
  },
  "visit": {
    "duration_seconds": 30,
    "observed_at": "..."
  }
}

Individual findings contain:

{
  "behavior": "...",
  "category": "...",
  "comparison": "matched",
  "confidence": 0.9,
  "description": "...",
  "explanation": "...",
  "policy": {
    "evidence": "...",
    "section": "...",
    "status": "explicitly_disclosed"
  },
  "severity": "informational",
  "telemetry": {
    "evidence": [],
    "observation_count": 1,
    "status": "observed"
  }
}

---

Comparison Labels

Verity currently uses:

matched

Observed behavior is directly covered by the supplied policy.

partially_matched

The policy broadly addresses the behavior but does not describe the observed activity with equivalent specificity.

observed_only

Behavior was observed but no sufficiently corresponding disclosure was identified in an otherwise applicable policy.

policy_only

The policy describes a behavior that was not observed during the sample.

This does not mean the behavior never occurs.

possible_contradiction

Observed behavior may conflict with an explicit policy statement.

This is intentionally conservative and is not a legal conclusion.

indeterminate

The available evidence is insufficient to reliably compare the policy with the observation.

---

Important Limitations

Verity is an automated technical analysis model.

It is not a legal compliance engine.

The following distinctions are important:

• third-party requests do not establish what data was transmitted
• tracker classifications do not prove profiling or targeted advertising
• browser API access does not establish why information was accessed
• WebGL activity does not by itself prove fingerprint generation
• cookie activity does not reveal cookie values or ownership
• storage activity does not reveal stored values
• absence of observed behavior does not establish that the behavior never occurs
• a short browser observation may not represent all application behavior

Verity output should be reviewed together with the underlying telemetry and policy evidence.

---

Beta Status

This is an early beta release of Verity.

The current model is based on a 1.7B parameter base model and was fine-tuned using a mixture of structured privacy-analysis examples and synthetic/counterfactual training data.

The model may:

• classify ambiguous disclosures differently from a human reviewer
• produce inconsistent aggregate counts
• occasionally over- or under-estimate disclosure specificity
• generate unsupported wording
• make mistakes on websites or browser behaviors that differ substantially from its training distribution

Applications using Verity should validate the returned JSON and deterministically verify fields such as:

analysis.counts

before presenting results to users.

---

Base Model

Verity 1.7B Beta is a PEFT / LoRA adapter for:

Qwen/Qwen3-1.7B

The base model is not included in this repository and will be downloaded separately when loaded through Transformers.

---

Veilance

Verity was developed for Veilance, an open-source browser privacy observability platform.

Veilance observes browser behavior and provides structured telemetry for privacy and security analysis.

Verity provides the analysis layer used to compare those observations with website privacy disclosures.

---

Disclaimer

Verity provides automated technical analysis for research and informational purposes.

Its output is not legal advice and should not be interpreted as a determination of regulatory compliance, legal liability, intent, wrongdoing, or the contents of network communications.