q-FrozenLake-v1-4x4-noSlippery

Q-Learning Agent playing FrozenLake-v1

This is a trained model of a Q-Learning agent playing FrozenLake-v1 (HF Deep RL Course Unit 2).

Model Details

• Algorithm: tabular Q-Learning (from scratch, NumPy Q-table)
• Library: Gymnasium + NumPy (no Stable-Baselines3 / no GPU kernels)
• Training episodes: 10,000
• Eval result: mean_reward = 1.00 +/- 0.00
• Non-slippery 4x4 map. Reward is 1 only on reaching the goal.

Usage

model = load_from_hub(repo_id="kaleido-jean/q-FrozenLake-v1-4x4-noSlippery", filename="q-learning.pkl")

# Don't forget to check if you need to add additional attributes (is_slippery=False etc)
env = gym.make(model["env_id"])

Training Details

Training Procedure

• Training regime: fp64/CPU NumPy table updates (no mixed precision)
• Wall-clock train time: 10,000 episodes ≈ 20k eps/s (tqdm wall-clock ~0.5 s)

Training Hardware / Carbon

Tabular Q-Learning (NumPy) is CPU-only. The training node also has an NVIDIA H100 80GB GPU, but it was not used.

• Hardware Type: CPU — Intel(R) Xeon(R) Platinum 8470 (node also has NVIDIA H100 80GB HBM3, unused)
• Hours used: ~0.00014 h (~0.5 s) (wall-clock training loop only; excludes Hub upload / video encode)
• Cloud Provider: PSC Bridges-2 / university HPC cluster
• Compute Region: US (Pittsburgh Supercomputing Center)
• Carbon Emitted: negligible for this wall-clock (sub-minute CPU job)

Training Hyperparameters

See q-learning.pkl keys: learning_rate, gamma, max_epsilon, min_epsilon, decay_rate, n_training_episodes, max_steps.

Technical Specifications

Compute Infrastructure

• Training: single CPU process on Intel Xeon Platinum 8470
• GPU: not required / not used (tabular method)