Downloads · 30 days
0
balkisoues/lf-delay-monitor-policies
lf-delay-monitor-policies is a reinforcement learning model from balkisoues. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Trained PPO and SAC expert and delay-distilled learner policies for the HalfCheetah-v5 locomotion task, produced as part of research on runtime monitoring of observation-delay-tolerant RL policies using Lingua Franca…
Downloads · 30 days
0
Access
Public
Updated Aug 3, 2026
Repo size
142 MB
Likes
0
Public
Click a slice to open those files.
.pt127 MB · 90%
From the Hugging Face model README
Trained PPO and SAC expert and delay-distilled learner policies for the HalfCheetah-v5 locomotion task, produced as part of research on runtime monitoring of observation-delay-tolerant RL policies using Lingua Franca (LF).
This repository accompanies the paper:
"Runtime Monitoring of Observation-Delay Tolerant RL Policies in Lingua Franca" (submitted to IEEE Embedded Systems Letters / TCRS 2026)
Code: balkisoues/lf-delay-runtime-monitor
Policies are trained on HalfCheetah-v5 under two reward conditions and distilled to tolerate varying levels of observation delay using DIDA (Delayed Imitation with Dataset Aggregation). An LF-based runtime monitor enforces a statistical delay-robustness envelope as a hard deadline over the deployed policy.
d (in environment steps).{algorithm}_{condition}_{role}[_{delay}]
| Component | Values | Meaning |
|---|---|---|
algorithm | ppo, sac | RL algorithm used to train the expert |
condition | nr, sr | nr = no reward shaping (baseline), sr = RBF-based reward shaping |
role | expert, learner | expert = undelayed teacher policy, learner = DIDA-distilled delay-tolerant policy |
delay (learners only) | d3, d5, d7, d10, d20 | Observation delay (in steps) the learner was distilled to tolerate |
Examples:
ppo_sr_expert.zip → PPO expert, reward-shaped, no delaysac_nr_learner_d10/ → SAC learner, no reward shaping, distilled for a 10-step observation delay├── ppo_nr_expert.zip # PPO expert, no shaping
├── ppo_sr_expert.zip # PPO expert, RBF reward shaping
├── sac_nr_expert.zip # SAC expert, no shaping
├── sac_sr_expert.zip # SAC expert, RBF reward shaping
├── ppo_nr_learner_d3/ # PPO learner (nr), delay = 3 steps
├── ppo_nr_learner_d5/
├── ppo_nr_learner_d7/
├── ppo_nr_learner_d10/
├── ppo_nr_learner_d20/
├── ppo_sr_learner_d3/ # PPO learner (sr), delay = 3 steps
├── ppo_sr_learner_d5/
├── ppo_sr_learner_d7/
├── ppo_sr_learner_d10/
├── ppo_sr_learner_d20/
├── sac_nr_learner_d3/
├── sac_nr_learner_d5/
├── sac_nr_learner_d7/
├── sac_nr_learner_d10/
├── sac_nr_learner_d20/
├── sac_sr_learner_d3/
├── sac_sr_learner_d5/
├── sac_sr_learner_d7/
├── sac_sr_learner_d10/
├── sac_sr_learner_d20/
└── LICENSE
Each learner_dX folder contains the distilled policy checkpoint(s) (.pt) for that delay setting.
sr condition): RBF-based shaping on normalized velocity tracking errorThese checkpoints are released for reproducibility of the accompanying paper — reproducing training curves, delay-robustness envelopes, and runtime monitor evaluation results. They are research artifacts, not production-ready controllers.
from stable_baselines3 import PPO, SAC
# Example: load a PPO expert
model = PPO.load("ppo_sr_expert.zip")
# Example: load a SAC learner distilled for 10-step delay
# (adjust to your actual checkpoint filename inside the folder)
model = SAC.load("sac_nr_learner_d10/<checkpoint_name>.pt")
Released under the MIT License.