Material Classification in Acoustic NLOS Environments Using an Attention-Based U-Net and Multimodal Fusion With the ANLOS-R Dataset

Abstract

Determining the position, shape, and material properties of objects in non-line-of-sight (NLOS) scenarios has become an important research topic in recent years. Material classification in acoustic NLOS environments is a challenging problem due to high noise levels, multiple reflection effects, and weak echo signals. This study proposes a multimodal acoustic NLOS framework that simultaneously addresses echo region isolation and material classification using only reflected acoustic waves. While most existing acoustic NLOS studies primarily focus on target detection or localization, the proposed framework uniquely integrates echo region isolation and material classification within a unified learning-based system. The proposed system consists of a multi-modal fusion classifier that combines temporal and spectral acoustic features with a U-Net-based segmentation network supported by an attention mechanism for separating reflection regions. Within the scope of this study, a new publicly available dataset, ANLOS-R (Acoustic Non-Line-of-Sight with Reflection), was created, consisting of a total of 1,440 echo samples collected using different material types and three different speaker-microphone configurations. Experimental results show that models based on single features exhibit limited performance, with the best GRU-based model achieving 65% accuracy. In contrast, the proposed multi-modal fusion approach significantly improved performance, reaching 74% accuracy and demonstrating high generalization ability in both single and multiple material scenarios. The findings reveal that combining attention-based echo isolation with multi-modal feature fusion provides a robust and reliable material classification solution in challenging acoustic NLOS environments. The proposed method establishes a strong foundation for future acoustic imaging, autonomous sensing, and NLOS-based perception systems.