Mathematical modelling of control-loss detection via risk-sensitive reinforcement learning on partially observable Markov decision processes

Abstract

 This work introduces a method for identifying high-risk loss-of-control episodes in digital settings by combining risk-sensitive reinforcement learning with decision-making under partial observability. We motivate the need to reason with incomplete and noisy information — typical of real-world deployments and, in particular, of monitoring user behaviour during critical states. The agent-environment interaction is modelled within the partially observable Markov decision process formalism, which maintains a belief (probabilistic posterior) over latent states given histories of actions and observations. Behaviour is analysed at the trajectory level, and tail risk is quantified via the Conditional Value at Risk (CVaR), enabling the assessment of expected losses in worst-case regimes rather than average-case performance. To ensure transparency and foster trust, we integrate explainable AI (XAI) techniques that reveal the factors driving risk estimates and action choices. The resulting pipeline provides a principled basis for adaptive detection of critical states and for early-warning interventions in complex digital environments, supporting reliable and accountable decision support.