uoro_time_series_forecasting

Vanilla RNNs trained with Unbiased Online Recurrent Optimization (UORO) for Time-Series Forecasting

This page documents the reference implementation used in the following paper on online training of vanilla RNNs using UORO for medical time-series forecasting.

Paper

• Title: Prediction of the position of external markers using a recurrent neural network trained with unbiased online recurrent optimization for safe lung cancer radiotherapy
• Authors: Michel Pohl, Mitsuru Uesaka, Hiroyuki Takahashi, Kazuyuki Demachi, Ritu Bhusal Chhatkuli
• arXiv: https://doi.org/10.48550/arXiv.2106.01100
• Published version: https://doi.org/10.1016/j.cmpb.2022.106908
• Hugging Face paper page: https://huggingface.co/papers/2106.01100
• Related article (Towards Data Science Editors’ Pick, mirrored on my blog) : Predicting respiratory motion using online learning of RNNs for safer lung radiotherapy

Code

• GitHub repository: https://github.com/pohl-michel/time-series-forecasting-uoro-rtrl
• Latest release (Zenodo): https://doi.org/10.5281/zenodo.13910198

Description

We provide an implementation of UORO using closed-form simplifications for quantities appearing in the loss gradient estimation for vanilla RNNs. We experimentally compare UORO with several baselines, including Real-Time Recurrent Learning (RTRL), in a respiratory motion-trace forecasting problem relevant to radiotherapy. We take into account hyperparameter tuning, inference time, prediction oscillations, and the influence of the prediction horizon h and ground-truth signal irregularity. UORO yielded lower horizon-averaged RMSE, nRMSE, and maximum error than the other algorithms, and was generally the most accurate method for medium-to-high values of h. It also exhibited satisfactory robustness to input-signal unsteadiness, limited step-to-step prediction fluctuations, and relatively low processing time.

Citation

@article{pohl2022prediction,
  title={Prediction of the position of external markers using a recurrent neural network trained with unbiased online recurrent optimization for safe lung cancer radiotherapy},
  author={Pohl, Michel and Uesaka, Mitsuru and Takahashi, Hiroyuki and Demachi, Kazuyuki and Chhatkuli, Ritu Bhusal},
  journal={Computer Methods and Programs in Biomedicine},
  volume={222},
  pages={106908},
  year={2022},
  publisher={Elsevier}
}