Consistent nonlinear dynamics: identifying model inadequacy

Empirical modelling often aims for the simplest model consistent with the data. A new technique is presented which quantifies the consistency of the model dynamics as a function of location in state space. As is well-known, traditional statistics of nonlinear models like root-mean-square (RMS) forecast error can prove misleading. Testing consistency is shown to overcome some of the deficiencies of RMS error, both within the perfect model scenario and when applied to data from several physical systems using previously published models. In particular, testing for consistent nonlinear dynamics provides insight towards (i) identifying when a delay reconstruction fails to be an embedding, (ii) allowing state-dependent model selection and (iii) optimising local neighbourhood size. It also provides a more relevant (state dependent) threshold for identifying false nearest neighbours.