Downloads · 30 days
0
Pablo-Flores-Mollinedo/verilog-llm-rigorous-rtl-benchmark-report
verilog-llm-rigorous-rtl-benchmark-report is a machine learning model from Pablo-Flores-Mollinedo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Paper-oriented benchmark/report artifact repo for Verilog LoRA and verified-agentic RTL experiments.
Downloads · 30 days
0
Access
Public
Updated Jun 14, 2026
Repo size
3.7 MB
Likes
0
Public
Click a slice to open those files.
.pdf3.7 MB · 97%
From the Hugging Face model README
Paper-oriented benchmark/report artifact repo for Verilog LoRA and verified-agentic RTL experiments.
Best clean single adapter: v33 Qwen3.5 LoRA
VerilogEval v2 spec-to-RTL direct n=1: compile 113/156, pass 80/156
Previous practical workflow: v29 v9-primary specialist PEFT-MoE selector
VerilogEval v2 spec-to-RTL: compile 150/156, pass 84/156
Best practical workflow: v36 verified agentic pipeline
VerilogEval v2 spec-to-RTL max B=5 model calls/problem: compile 145/156, pass 95/156
Important: v29 and v36 are not standalone adapters and not zero-shot single-model pass@1. They are verified workflows using iverilog/vvp feedback/selection.
Active matrix includes:
v9_direct Qwen2.5-Coder-7B + v9 baseline LoRA
v30b_direct Qwen2.5-Coder-7B + v30b LoRA
v33_direct Qwen3.5-9B + v33 LoRA
v34_direct Qwen3.5-9B + v34 manual repair LoRA
v29_selector Qwen2.5 v9-primary specialist PEFT-MoE verifier selector
v36_agentic Qwen3.5 v33/v34 verified repair loop
Benchmarks:
VerilogEval v2 spec-to-RTL
VerilogEval v2 code-complete ICCAD 2023
paper-style 30-task functional suite # direct adapters only
RTLLM v2 verified-reference subset # direct adapters only
Default protocol:
3 repeats per system/benchmark
temperature=0, top_p=0.01
every RTL source recompiled and simulated
Current running status is in:
documentation/paper_rigorous_running_status.md
Final CSV/plots will be uploaded here when matrix completes.
documentation/paper_rigorous_evaluation_plan.md
documentation/paper_rigorous_running_status.md
scripts/run_paper_rigorous_matrix.sh
scripts/run_rtllm_direct_benchmark.py
scripts/aggregate_paper_rigorous_results.py
scripts/run_v29_runtime_primary_specialist_selector.py
scripts/run_v36_agentic_verified_pipeline.py
results/v36_agentic_verified_pipeline/*
documentation/sources/papers/*.pdf
Use separate tables for:
direct single-adapter pass@1: v9/v30b/v33/v34
verified selector workflow: v29
verified agentic workflow: v36
external RTLLM/paper-style generalization: direct adapters only