On Piecewise Affine Reachability with Bellman Operators

A piecewise affine map is one of the simplest mathematical objects exhibiting complex dynamics. The reachability problem of piecewise affine maps is given as follows: Given two vectors $\mathbf{s}, \mathbf{t} \in \mathbb{Q}^d$ and a piecewise affine map $f$, is there $n\in \mathbb{N}$ such that $f^{n}(\mathbf{s}) = \mathbf{t}$? Koiran, Cosnard, and Garzon show that the reachability problem of piecewise affine maps is undecidable even in dimension 2. Most of the recent progress has been focused on decision procedures for one-dimensional piecewise affine maps, where the reachability problem has been shown to be decidable for some subclasses. However, the general undecidability discouraged research into positive results in arbitrary dimension. In this work, we consider a rich subclass of piecewise affine maps defined by Bellman operators of Markov decision processes (MDPs). We then investigate the restriction of the piecewise affine reachability problem to that with Bellman operators and, in particular, its decidability in any dimension. As one of our primary contributions, we establish the decidability of reachability for two-dimensional Bellman operators, in contrast to the negative result known for general piecewise affine maps.