Nuclear Image Segmentation based on improved MaskR-CNN

• 1、School of Information Engineering, Wuhan University of Technology

Abstract：The development of automatic nuclear segmentation, driven by the rise of deep learning, has significantly impacted the medical field. However, due to the problems of nuclear aggregation, adhesion, shape difference and low contrast, it is still difficult to capture effective features and blur edges. For the sake of solving the above problems, this paper proposes an enhanced MaskR-CNN framework combined with attention mechanism (CA-MaskR-CNN) for segmenting nuclear images, which uses separate ResNet to increase the dimension of network cardinality to improve the ability of network semantic extraction, and introduces coordinate attention mechanism to pay attention to spatial information to improve the performance of network in nuclear image recognNuclear Image Segmentation based on improved MaskR-CNNition. Through the improved softening non-maximal inhibition screening, the detection effect of clusters and overlapping nuclei is improved, and a more appropriate loss function is selected to enhance the efficacy of nuclear segmentation. The experimental outcomes demonstrate that the F1 score index (F1-score) and superposition Jacquard index (AJI) of CA-MaskR-CNN are 0.8454 and 0.6245 respectively, which can effectively improve the accuracy of nuclear segmentation.

Keywords： Nuclear Image Attention Mechanism Feature Fusion Non-Maximum Suppression Loss Function