Recent Achievement of the Team: Weakly Supervised Pseudo Labeling Texture Semantic Segmentation (PTS) Network Based on Scribble Annotation
Our team published a paper titled “A pseudo-labeling based weakly supervised segmentation method for few-shot texture images” in the international journal “Expert Systems With Applications” (IF: 8.5, Computer Science, Q1 TOP). The School of Computer Engineering and Science at Shanghai University is the first affiliation, Han Yuexing is the first author, Li Ruiqi is the second author, and Han Yuexing and Chen Qiaochuan are the corresponding authors.
Deep learning-based segmentation of material microstructure images faces challenges such as scarce samples, annotation difficulty, and model generalization. In material images with complex textures, accurately delineating boundaries between different phases is often challenging. These issues have resulted in poor performance of existing deep learning networks for segmenting material microstructure images. To address these problems, this paper proposes a weakly supervised pseudo-labeling texture segmentation (PTS) network based on line annotation. Compared to full annotation, line annotation only requires labeling a small number of pixels with clear categories, significantly reducing the domain knowledge required for annotation. However, line annotation covers a very small number of pixels, making it difficult to provide sufficient supervision for complex neural networks. Therefore, the PTS network generates pseudo-labels during the training phase to obtain more available supervision.
As shown in the figure above, the PTS network adopts a dual-branch structure consisting of a main branch and an auxiliary branch. The main branch is used for feature extraction and segmentation prediction, while the auxiliary branch generates pseudo-labels to assist the training of the main branch, thus achieving dual supervision (line annotation and pseudo-labels) of the segmentation prediction results of the main branch. During the testing phase, only the main branch is used to predict the segmentation results of the images. The PTS network achieves the training of a generalizable image segmentation model for complex textured material images based on one to two line annotations and exhibits universality. The figure below shows the segmentation results based on line annotation for different models on the titanium alloy dataset.
After applying the t-SNE method to reduce the dimensionality of the features extracted from the PTS network and the Unet++ network separately, we obtain the feature distributions shown in Figure 3. By comparing the feature distributions of the PTS network and the Unet++ network, it can be observed that the different class features in the Unet++ network are more confounded compared to the PTS network. In the figure below, the distributions of blue and orange feature points in the Unet++ network overlap with each other, which may limit the final segmentation results of the Unet++ network. In contrast, the feature distribution of the PTS network exhibits clearer boundaries between different classes, which aids in better predictive segmentation.
Our code and paper are both publicly available at: https://github.com/han-yuexing/Scribble_Segmentation
