On the Identity Problem and the Group Problem in nilpotent groups

Let $\mathcal{G}$ be a finite set of matrices in a unipotent matrix group $G$ over $\mathbb{Q}$, where $G$ has nilpotency class at most ten. We exhibit a polynomial time algorithm that computes the subset of $\mathcal{G}$ which generates the group of units of the semigroup $\langle\mathcal{G}\rangle$ generated by $\mathcal{G}$. In particular, this result shows that the Identity Problem (does $\langle\mathcal{G}\rangle$ contain the identity matrix?) and the Group Problem (is $\langle\mathcal{G}\rangle$ a group?) are decidable in polynomial time for unipotent matrix group of class at most ten. This extends the earlier work by Babai, Beals, Cai, Ivanyos and Luks on commutative matrix groups to nilpotent matrix groups. An important implication of our result is the decidability of the Identity Problem and the Group Problem within finitely generated nilpotent groups of class at most ten. Our main idea is to analyze the logarithm of the matrices appearing in $\langle\mathcal{G}\rangle$. This allows us to employ Lie algebra methods to study this semigroup. In particular, we prove several new properties of the Baker-Campbell-Hausdorff formula, which help us characterize the convex cone spanned by the elements in $\log \langle\mathcal{G}\rangle$. Furthermore, we formulate a sufficient condition in order for our results to generalize to unipotent matrix groups of class $d > 10$. For every such $d$, we exhibit an effective procedure that verifies this sufficient condition in case it is true.