deepseek-governed-no-amnesia

CODE: https://github.com/frank-morales2020/AST/blob/main/DEEPSEEK_PRIME_ANCHORE_LLM.ipynb

deepseek-governed-no-amnesia

An experimental repository pairing DeepSeek-R1-Distill-Qwen-7B with a "Prime-Anchored Spectral Governor" artifact. This card describes exactly what the repository contains and what the accompanying run did and did not do.

What this repository is

• The model weights are identical to the base model, deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. The governor did not modify them.
• Attached is governor_state.pt, which records the governor configuration (anchor primes, threshold, cached anchor rows, gate statistics, and a SHA-256 signature of the prime-indexed embedding rows).
• VERIFICATION.txt records the run details.

What the governor does (and doesn't)

The governor evaluates a gate on each training step and pins six prime-indexed embedding rows ([2, 3, 5, 7, 11, 13]) to their original values. In the run that produced this repository, the optimizer step was a no-op (STEP_GATED_NO_MUTATION), so the model was not trained or fine-tuned.

As a result:

• The prime-indexed rows are unchanged (anchor signature matches).
• The full model is byte-for-byte the base model.
• This run does not demonstrate training, fine-tuning, or that any form of catastrophic forgetting was prevented, because the model was not modified. The repository name reflects the project's intent, not a measured property of this checkpoint.

Intended use

Research and experimentation with the governor tooling. For general use, loading the base model directly is equivalent and avoids downloading a duplicate copy of the weights.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

REPO_ID = "frankmorales2020/deepseek-governed-no-amnesia"

tokenizer = AutoTokenizer.from_pretrained(REPO_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    REPO_ID, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model.eval()

prompt = "Explain why prime numbers are important in cryptography."
text = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=False, add_generation_prompt=True,
)
enc = tokenizer(text, return_tensors="pt").to(model.device)

out = model.generate(
    **enc,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))

This is a DeepSeek R1 distill (reasoning) model. Feed prompts through the chat template, and expect a <think> ... </think> reasoning trace before the final answer. To keep only the final answer, split the output on </think>.

Inspecting the governor artifact

from huggingface_hub import hf_hub_download
import torch

state = torch.load(
    hf_hub_download(REPO_ID, "governor_state.pt"),
    map_location="cpu", weights_only=False,
)
print(state["primes"], state["LAMBDA_12"], state["anchor_signature"], state["stats"])

License

Released under the MIT license, inherited from the base model.

Acknowledgements

Base model: DeepSeek-R1-Distill-Qwen-7B by DeepSeek-AI.