Understanding and predicting large time series with Markov-modulated non-homogeneous Poisson processes

Abstract

This thesis seeks to produce new methods for the analysis and prediction of counting processes through the usage of Markov-modulated non-homogeneous Poisson processes. The modelling framework was motivated by problems arising in general (non-life) insurnace but the developed techniques are applicable to a variety of different modelling contexts although specific innovations are implemented to deal with large data sources.

An important issue dealt with in this thesis is that in many real world circumstances, it is often the case that the practitioner is able to rely on some domain knowledge or past experience to inform the modelling process. However, it is also unlikely that all drivers of events of interest are able to be adequately captured by a practitioner's chosen model due to data limitations and materiality concerns.

The proposed modelling framework serves to link the modellable factors (allowing much flexibility in terms of model choice) with unmodellable/hard-to-model factors through the use of a hidden Markov structure. The separation of these two components provides advantages in terms of adaptability, interpretability and realism. Further, the model may also be utilised as a diagnostic/performance evaluation tool for the practitioner's chosen model and also for iterative model improvement.

This thesis further produces a model calibration algorithm through the use of an Expectation-Maximisation procedure. Several improvements are made to enhance the practicability of the calibration by severely reducing computational requirements while allowing the procedure to be run on standard statistical software such as R.

Additionally, both simulated and real world empirical demonstrations are provided to highlight that the out-of-sample predictions generated by this approach can produce superior forecast distributions compared to standard methods in practice.

Finally, a computationally feasible method to generate parameter error estimates is developed to to assess the uncertainty of the model. Computational considerations are particularly necessary with large data sets where latent models such as the MMNPP provide value. This approach is shown to provide more realistic and distributionally accurate out-of-sample predictions. The benefits of this assessment are demonstrated using simulated and real data case studies which provide evidence of the advantages of incorporating parameter error considerations under the MMNPP framework.