KnitStructNet: A Structure-Aware Dual-Scale Framework for Stitch-Level Segmentation on Physically Captured Knitted Fabrics

Abstract

Semantic segmentation of real knitted fabrics remains challenging due to extreme class imbalance, subtle inter-class similarity, and loop-level geometric variations that synthetic datasets fail to represent. Prior approaches rely heavily on rendered or hybrid imagery and therefore struggle with the photometric and structural irregularities found in physical textiles. This study introduces a fully non-synthetic segmentation framework that reconstructs 13-class stitch instruction maps directly from geometrically aligned, physically captured knitted surfaces. The model integrates multi-scale contextual reasoning with row-column structural modeling to jointly capture fine-grained loop detail and broader directional dependencies. Evaluated exclusively on real knitted samples, the system achieves 95.65% pixel accuracy, a 0.7829 mean IoU, and an 80.2% macro F1-score, demonstrating robust performance across both frequent and rare stitch types. By eliminating reliance on synthetic renderers, the proposed approach establishes a physically grounded baseline for stitch-level instruction-map reconstruction; full program-level instruction-code generation and industrial inspection pipelines are beyond the scope of this work and are left for future study.