NICoLE-LLM-GGUF

NICoLE-LLM

NICoLE is a compact LLM-based controller for congestion-aware RTP/WebRTC adaptive video streaming.

It predicts:

• ECN
• Current Profile (CP)
• Next Profile (NP)

from RTP packetization and queue telemetry using compact symbolic prompting.

Optimized for:

• low-latency inference
• edge deployment
• GGUF quantization
• deterministic structured outputs

Applications:

• WebRTC adaptive streaming
• congestion-aware real-time video encoding adaptation
• in-Network QoE Optimization
• edge AI networking

---

Profiles

The dataset was generated using real-time WebRTC streaming under a 40 Mbps bottleneck shared between background traffic and adaptive RTP video streaming.

---

Prompt Format

Input order:

PS FS IFGS IFGR CQ LQ E

Output order:

E C N

Example:

I:PS FS IFGS IFGR CQ LQ E
O:E C N

U:1400,40,34,33,2,0,0

A:

Expected output:

0,1,1

---

Hugging Face Usage (Python code)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "alirezashirmarz/NICoLE-LLM"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto"
)

prompt = """I:PS FS IFGS IFGR CQ LQ E
O:E C N
U:1400,40,34,33,2,0,0
A:"""

inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=6,
    do_sample=False
)
print(tok.decode(out[0], skip_special_tokens=True))

---

GGUF / llama.cpp Usage

./llama-cli \
-no-cnv \
-t 4 \
-m nicole-q4.gguf \
-p "I:PS FS IFGS IFGR CQ LQ E
O:E C N

U:1400,40,34,33,2,0,0

A:" \
-n 6 \
--temp 0 \
--top-k 1

---

Runtime Configuration

---

CPU Core Benchmark

Best CPU deployment:

• 4 threads
• 343 ms response time
• 2.91 decisions/sec

Compact symbolic prompting significantly reduces:

• prompt tokens
• KV-cache usage
• inference latency
• deployment overhead compared to verbose natural-language prompting.

---

Quantized Models

Available quantization:

• Q4_K_M (recommended)

Runtime:

• llama.cpp

Designed for:

• edge deployment
• CPU inference
• bounded symbolic control inference
• real-time congestion-aware adaptation

---

Limitations

• Trained under a 40 Mbps bottleneck scenario
• Designed for bounded RTP/WebRTC streaming tasks
• Not intended for open-ended conversational generation

---

Citation

If you use this model, please cite the NICoLE paper and repository.

• A. Shirmarz, F. L. Verdi, G. Patra and G. Pongrácz, "NICoLE: Are In-Network LLM-Based Agents Cost-Feasible for RTP Video Streaming?," 2026 IFIP Networking Conference (IFIP Networking), Lugano, Switzerland, 2026, pp. 1-6, doi: 10.23919/IFIPNetworking70592.2026.11579010.