pcf-qwen3-14b-compact-30k-conf0p6-lora

PCF Qwen3-14B Compact 30k Conf0.6 LoRA

This repository contains a LoRA adapter for Qwen/Qwen3-14B trained for the Past-Creates-Future (PCF) algorithm-idea generation workflow.

This is the compact-output variant. It takes an existing algorithm description plus a mechanism-level method delta and generates a compact improved algorithm description:

existing algorithm + method delta -> improved algorithm

The output format focuses on:

novelty_claim
core_idea
algorithm_description

It is useful for quickly generating candidate algorithm ideas. For more structured downstream implementation work, the detail-to-detail PCF adapter may be easier to use.

Model Details

• Base model: Qwen/Qwen3-14B
• Adapter type: LoRA / PEFT
• Task type: causal language modeling
• LoRA rank: 16
• LoRA alpha: 16
• LoRA dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Training dtype: bf16

Intended Use

This adapter is designed for algorithm-research ideation. It can be used to:

• generate compact candidate algorithm improvements;
• apply a known method delta to a related algorithm;
• brainstorm reusable mechanism changes;
• produce initial algorithm sketches before manual validation.

The model should be treated as a hypothesis generator. Generated ideas need human review and experimental testing.

Out-of-Scope Use

Do not use this adapter to make final claims about algorithmic improvement, optimality, correctness, or empirical performance. It does not run experiments and may produce plausible but unverified claims.

Input and Output

The prompt includes an existing algorithm and a method delta. The method delta may include fields such as:

problem_shift
method_delta
component_changed
mechanism_added
mechanism_removed
abstraction_shift
why_improves
reuse_pattern
delta_keywords

The expected response is a compact JSON-like algorithm description with:

novelty_claim
core_idea
algorithm_description

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base_model = "Qwen/Qwen3-14B"
adapter = "ssfc/pcf-qwen3-14b-compact-30k-conf0p6-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto",
    quantization_config=quantization_config,
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter)
model.eval()

For the prompt format used in this project, see finetune/inference.py in the Past-Creates-Future repository.

Training Data

Training data file:

gpt_oss_method_delta_details_v4_flash_30k5562_conf0p6.jsonl

Despite the historical file prefix, this dataset is part of the PCF method-delta training pipeline. It was built from algorithm-paper method extraction and delta summarization. This version uses a 30k-character extraction context and a confidence threshold of 0.6.

Compared with the detail-to-detail dataset, this compact variant uses detailed extraction information during dataset construction but trains the model to emit compact target fields.

Training Configuration

• Epochs: 3
• Per-device batch size: 1
• Gradient accumulation steps: 16
• Learning rate: 1e-4
• Scheduler: cosine
• Max sequence length: 2048
• Optimizer: fused AdamW
• Gradient checkpointing: enabled
• Hardware used locally: single NVIDIA GPU with 24 GB VRAM

Evaluation

This adapter was compared with earlier PCF adapters using held-out generation samples and downstream MAPD/TFO idea-generation trials. In those trials, compact ideas were used as starting points for manual algorithm implementation and sandbox benchmarking.

The uploaded adapter was downloaded back from Hugging Face and verified against the local upload source by SHA256 for adapter_config.json, adapter_model.safetensors, and README.md.

Limitations

• Compact outputs are easier to read but contain fewer implementation details.
• The model may overstate improvement claims unless the user treats them as hypotheses.
• Generated algorithms require manual implementation and experimental validation.
• It is specialized for algorithmic method-delta ideation, not general instruction following.

Framework Versions

• PEFT 0.18.1