Feature selection of high-dimensional labeled data with limited observations is critical for making powerful predictive modeling accessible, scalable, and interpretable for domain experts. Spectroscopy data, which records the interaction between matter and electromagnetic radiation, particularly holds a lot of information in a single sample. Since acquiring such high-dimensional data is a complex task, it is crucial to exploit the best analytical tools to extract necessary information. In this paper, we investigate the most commonly used feature selection techniques and introduce applying recent explainable AI techniques to interpret the prediction outcomes of high-dimensional and limited spectral data. Interpretation of the prediction outcome is beneficial for the domain experts as it ensures the transparency and faithfulness of the ML models to the domain knowledge. Due to the instrument resolution limitations, pinpointing important regions of the spectroscopy data creates a pathway to optimize the data collection process through the miniaturization of the spectrometer device. Reducing the device size and power and therefore cost is a requirement for the real-world deployment of such a sensor-to-prediction system as a whole. We specifically design three different scenarios to ensure that the evaluation of ML models is robust for the real-time practice of the developed methodologies and to uncover the hidden effect of noise sources on the final outcome.
翻译:由于获得这种高维数据是一项复杂的任务,必须利用最佳分析工具来获取必要的信息。在本文件中,我们调查最常用的特征选择技术,并采用最新的可解释的AI技术来解释高维和有限光谱数据的预测结果。对预测结果的解释有益于域专家,因为它能确保ML模型对域知识的透明度和忠实性。由于仪器分辨率的限制,确定光谱数据的重要区域为通过光谱仪装置的微型化优化数据收集进程开辟了一条途径。降低装置大小和功率,因此成本是实际部署这种高维和有限光谱数据的传感器到定位系统的一项要求。我们特别设计了三种不同的情景,以确保ML模型对实际结果源的隐藏效果的可靠评估。