In Defence of Visual Analytics Systems: Replies to Critics

The last decade has witnessed many visual analytics (VA) systems that make successful applications to wide-ranging domains such as urban analytics and explainable AI. However, those systems are often designed, developed, and evaluated on an ad-hoc basis, provoking and spotlighting criticisms about the research rigor and contributions within the visualization community. We come in defence of VA systems by contributing two interview studies with VA researchers to gather critics and replies to those critics. First, we interview 24 researchers about criticisms for VA systems they have received from peers. Through an iterative coding and refinement process, we summarize the interview data into a list of 36 common criticisms. Second, we interview 17 researchers to validate our list and collect replies to those criticisms. We conclude by discussing eight important problems and future research opportunities to advance the theoretical and practical underpinnings of VA systems. We highlight that the presented knowledge is deep, extensive, but also imperfect, provocative, and controversial, and thus recommend reading with an inclusive and critical eye. We hope our work can provide solid foundations and spark discussions to promote the research field forward more rigorously and vibrantly.