The promise of multimodal models for real-world applications has inspired research in visualizing and understanding their internal mechanics with the end goal of empowering stakeholders to visualize model behavior, perform model debugging, and promote trust in machine learning models. However, modern multimodal models are typically black-box neural networks, which makes it challenging to understand their internal mechanics. How can we visualize the internal modeling of multimodal interactions in these models? Our paper aims to fill this gap by proposing MultiViz, a method for analyzing the behavior of multimodal models by scaffolding the problem of interpretability into 4 stages: (1) unimodal importance: how each modality contributes towards downstream modeling and prediction, (2) cross-modal interactions: how different modalities relate with each other, (3) multimodal representations: how unimodal and cross-modal interactions are represented in decision-level features, and (4) multimodal prediction: how decision-level features are composed to make a prediction. MultiViz is designed to operate on diverse modalities, models, tasks, and research areas. Through experiments on 8 trained models across 6 real-world tasks, we show that the complementary stages in MultiViz together enable users to (1) simulate model predictions, (2) assign interpretable concepts to features, (3) perform error analysis on model misclassifications, and (4) use insights from error analysis to debug models. MultiViz is publicly available, will be regularly updated with new interpretation tools and metrics, and welcomes inputs from the community.
翻译:现实应用的多式联运模式的希望激励了人们研究如何直观和理解其内部机理,最终目标是使利益攸关方能够直观地看待模型行为,进行模型调试,并促进对机器学习模型的信任;然而,现代多式联运模式通常是黑盒神经网络,这给理解其内部机理带来挑战;我们如何在这些模型中设想多式联运互动的内部模型?我们的文件旨在通过提出多维兹(MultiViz)来填补这一空白,该多维兹是一种分析多式联运模式行为的方法,将可解释性问题分为四个阶段:(1) 单式重要性:每种模式如何促进下游建模和预测;(2) 跨模式互动:不同模式如何相互关联;(3) 多式联运表述:决定层面的特点如何代表非模式和跨模式互动;(4) 多式联运预测:决定层面的构成如何进行预测;多维兹设计了一个分析模式,通过将可解释性工具分为不同模式、模式、任务和研究领域;通过对8种经过培训的工具进行实验,我们显示多维兹(MuldViz)共同进行互补阶段,使用户能够定期进行模拟、模拟分析、模拟分析、模拟分析、模拟分析、模拟、模拟分析、模拟分析、模拟分析、模拟、模拟分析、模拟分析、模拟、模拟分析、模拟、模拟分析、模拟、模拟、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟、模拟、模拟分析、模拟、模拟、模拟、模拟、模拟分析、模拟分析、模拟、模拟分析、模拟、模拟、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟、模拟、模拟、模拟、模拟分析、模拟、模拟、模拟、模拟、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟、模拟、模拟分析、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟分析、模拟分析、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟、模拟分析、模拟分析、模拟分析、模拟分析、模拟分析、模拟、模拟、模拟、