Cross-Domain Self-Supervised Learning for Histopathological Images With Multi-Granularity Constraints

In pathological image analysis, the scarcity of labeled data and the difficulty of producing scores of annotated images to high-level details hinder the development of large, high-quality datasets, which in turn affects the advancement of deep learning-based diagnostic tools. The lack of annotations can be addressed to some extent by self-supervised learning methods, leveraging the production of general representations from unlabeled data. However, in such deployment, self-supervised learning methods and general machine learning models suffer from significant performance degradation when transferring to a dataset from an unseen clinic or device, due to domain shifts in data distribution between different datasets across clinics. Such domain shifts can cause high inter-class similarities and low cross-domain correlations. Previous domain generalization methods are often susceptible to under-generalization and over-generalization, especially in handling histopathological images. In this paper, we propose a novel cross-domain self-supervised learning method based on multi-granularity constraints. It combines pathology-specific imaging mechanisms to bound the generalized features of histopathological images in a stable representation space and to extend the generalization distance as much as possible under an adversarial constraint for preventing over-generalization. The effectiveness of the proposed approach against the state-of-the-art methods has been demonstrated in a variety of analysis tasks on four independent prostate histopathological imagery datasets. The proposed method achieves an improved generalization in fine-tuned classification and segmentation using limited annotated samples.