Joint Species Distribution Modeling with species competition and non-stationary spatial random effects

Joint species distribution models (JSDM) are among the most important statistical tools in community ecology. However, existing JSDMs cannot model mutual exclusion between species. We tackle this deficiency by developing a novel hierarchical JSDM with Dirichlet-Multinomial observation process for mutually exclusive species groups. We apply non-stationary multivariate Gaussian processes to describe species niche preferences and conduct Bayesian inference using Markov chain Monte Carlo. We propose decision theoretic model comparison and validation methods to assess the goodness of the proposed model and its alternatives in a case study on modeling vegetation cover in a boreal peatland in Finland. Our results show that ignoring the interspecific interactions and competition significantly reduces models predictive performance and through that leads to biased estimates for total cover of individual species and over all species combined. Models relative predictive performance also depends on the predictive task highlighting that model comparison and assessment method should resemble the true predictive task. Our results also demonstrate that the proposed joint species distribution model can be used to simultaneously infer interspecific correlations in niche preference as well as mutual competition for space and through that provide novel insight into ecological research.