Применение методов извлечения дискриминирующих и глубоких элементов для полного анализа слайдов: обследование.

Introduction

Digital pathology technologies and solutions have significantly improved modern clinical practices. 61 Among these technologies is the digital histology slide scanner, also known as a whole slide scanner, which captures high-resolution digital images of histological slides. Histological slides are thin sections of tissue samples that are mounted on glass slides and stained for microscopic examination. Traditionally, pathologists would view these slides under a microscope to analyze and diagnose various diseases and conditions. 127 Influenced by the progress of digital image processing, recent advances in microscopic scanning led to the development of a new imaging technology named whole slide imaging (WSI) that captures tissue glass slides in a single image. These digital full-color images facilitate storing, viewing, processing, and sharing scans of tissue slides. 8 It also enables simultaneous case reviews and the international sharing of scans of tissue slides with experts and represents a rich source of clinical diagnostic information, rich color information, across multiple scales of resolution. WSI opens the door for developing various solutions for complex problems specific to the highly detailed content of these images for automating certain aspects of histopathology workflows. However, the resolution of these images is extremely high, with the sizes of individual images reaching several gigabytes. This leads to several technical challenges, currently and extensively addressed by the scientific community. 127 In this context, feature extraction methods play a crucial role in transforming raw image data into meaningful representations that capture relevant information for analysis. These methods encompass a wide range of techniques, including traditional handcrafted features, as well as more advanced approaches utilizing deep learning architectures. These techniques facilitate the extraction of morphological, textural, and contextual features from whole slide images, enabling the characterization of tissue structures, cellular properties, and pathological patterns.

Various kinds of organs and tissues show different visual characteristics depending on the organ type and disease type. These characteristics are embedded in WSIs2 and require a special kind of analysis based on the underlying clinical procedure. 15 For instance, the morphological features of nuclei are considered primary indicators of cancerous tissue. 3 For example, normal cells have a regular and ellipsoid shape, while cancer cells are often irregular and contoured. When dealing with lung cancer lethality, the diversity of shape and morphology of nuclei is one of the main characteristics. 25 , 74 The distribution of histologic primitives, including glands and nuclei, is also one of the analysis methods commonly employed to predict the aggressiveness of various cancers. Pathology associations design and develop specific analysis procedures and guidelines based on them. Specifically, nuclear pleomorphism and morphologic heterogeneity are strongly related to cellular diversity, which is one of the main indicators of cancer type and grade. Several cancer-grading schemes rely on cellular diversity, where diversity in nuclear size, shape, and appearance is a critical element of the analysis process. 124 Moreover, the texture of tissue images is one of the main features commonly used in processing these images. These features are widely used to determine regions of interest with unique clinical properties showing the spread of the abnormal group of cells in tissue samples. Texture features are also commonly used for the segmentation of different elements of tissue images. 1

Due to the large size of WSIs and high variations in color and texture, in addition to variations in scanning parameters and technologies, traditional morphology and shape features may not provide the best representation of WSIs. Recently, a new feature extraction and analysis method that relies on deep learning has emerged. It has the potential to provide an accurate and complete representation of WSI contents. Features extracted based on deep learning proved superior in terms of accuracy in many histology-related problems, including image segmentation, classification, and description. 63 , 102 , 203 WSIs analysis based on feature extraction using deep learning has diverse clinical applications. It aids in cancer diagnosis and subtyping, tumor grading, and prognosis prediction. For instance, it supports precision medicine by identifying biomarkers and enabling personalized treatment plans. It is also valuable for disease detection and quantification, such as identifying skin lesions or quantifying renal features. Overall, feature extraction from WSIs using deep learning enhances pathology practices, enabling accurate diagnosis, prognosis, treatment planning, and disease monitoring. 54 , 142

This survey paper aims to provide a comprehensive overview of the applications of feature extraction using handcrafted methods and deep learning techniques for the analysis of whole slide images. By surveying the existing literature, we aim to shed light on the advancements, challenges, and opportunities in this rapidly evolving field. Understanding the potential of these approaches is essential for both researchers and practitioners seeking to leverage the power of computational techniques for accurate diagnosis, prognosis, and decision-making in digital pathology. We aim to provide a comprehensive understanding of the state-of-the-art methodologies, techniques, and applications of feature extraction methods and deep learning for whole slide image analysis.

The survey is geared towards pathologists and lab technicians who wish to have a broad, technical overview over the state-of-the-art, categorized by application, to make informed choices when it comes to implementing or evaluating automated techniques. It is also geared towards machine learning engineers and researchers who want to conduct research in the digital pathology field.

In this survey, we review literature related to digital pathology applications, focusing on AI and deep learning solutions. Specifically, we analyze and discuss the utilization of discriminative and geometric features of visual components of WSIs (i.e., features suited to support the diagnostic process extracted by “engineered” methods as opposed to AI) in the development of computational image analysis tools. Additionally, we explore predictive modeling techniques applied to histology images using AI and deep learning methodologies. To provide a structured approach to our survey, we classify these features based on the type of descriptive information they capture, the application domains they serve, and the level of representation they provide. Furthermore, we explore the various analytical information that can be extracted from these features, categorizing them based on the extraction method employed and their geometric locality. To provide a comprehensive view, we also review notable applications that have been developed based on these features thereby facilitating the creation of analysis tools and predictive modeling applications in the field. Lastly, in the context of feature analysis and deep learning techniques, we present a brief discussion on the challenges that emerge specifically in the analysis and processing of whole slide images. By focusing on these challenges, we aim to emphasize their significance and the need for effective solutions. Additionally, we highlight potential research opportunities within this domain, providing valuable research directions for further exploration and advancement.

Review methodology

Digital pathology has seen a huge interest in recent years, as indicated by the growing volume of literature on WSI and its applications. The use of AI for WSI analysis in pathology departments around the world is rapidly increasing. Given the immense potential of digital pathology for many beneficial use cases (e.g., workload balance, remote diagnosis, teleconsultation, quality control, and image analysis), further research is devoted to handling new technical issues arising from these new applications. Since the development of whole slide images, the number of studies has grown exponentially to meet the demand for solutions for these technical issues. To structure, organize, and summarize the huge volume of research in this field, several review and survey studies have been conducted using various strategies based on scope, topic, application, and methodologies.

Niazi et al. 127 highlight the most important applications of AI in digital pathology and whole slide imaging for several general applications such as education, quality assurance, and clinical diagnosis. Yet, their review did not address the use of AI for more specific applications with a high impact on the daily clinical diagnosis tasks, which depend mainly on extracting information from WSIs. Tizhoosh and Pantanowitz 187 shed light on some of the most important challenges and opportunities for exploiting AI potentials in computational pathology. Despite the new research trends explored in their review, they did not discuss how these new problems were approached by recent studies. Bera et al. 25 provide a broad review of research directions incorporating AI and machine learning techniques. The review focuses on the impact of AI on biomarker development in clinical oncology using data-driven methodologies such as machine learning and deep learning with little focus on image analysis and information extraction methods. The impact of AI on clinical pathology is also discussed by Colling et al. 48 ; while emphasizing the impact of biomarker development in digital pathology, the scope of their review is limited by exploring tasks related to the clinical pipeline without a detailed discussion of available solutions in the literature.

Our survey provides an exploration of various methods and solutions, presenting a detailed discussion on the use and the impact of AI on digital pathology development concerning WSIs analysis. Several review studies address the impact of AI on WSIs. The earliest is the review conducted by Madabhushi and Lee 133 to explore the development of image analysis tools for predictive modeling using handcrafted and deep learning features extraction methods focusing on detection, segmentation, feature extraction, and tissue classification perspective. Yet, the review did not provide a detailed discussion of solutions for these tasks. Deng et al. 54 review some of the applications of deep learning solutions in digital pathology. They address limited applications classified based on region, gland, and cell-based analysis. The review conducted by Salvi et al. 160 provides an overview of deep learning methods that are used to either optimally prepare the input (pre-processing) or improve the results of the network output (post-processing), with a focus on WSIs. However, the scope of the review is reduced to discussing the impact of these methods on a few digital pathology applications, such as cancer detection and WSIs segmentation. A recent survey was conducted by Srinidhi et al. 175 focusing on deep neural network training methodologies with respect to a limited number of applications, including classification, regression, segmentation, and survival prediction models. Ahmedt-Aristizabal et al. 5 presents a review of graph-based deep-learning methodologies for computational histopathology. Yet, the survey provides a discussion on the training paradigms based on graph-based learning with little focus on digital pathology clinical applications. Finally, Baxi et al. 22 examined the potential and limitations of using AI-based techniques for identifying biomarkers and selecting patients, and explored how digital pathology and AI advancements should be taken into account in the field of translational medicine, together with the challenges that may arise if this technology is adopted in clinical settings.

On the other side, we focus on manual and deep learning feature extraction methods from technical and clinical perspectives. We discuss various manual and automatic feature engineering methods for WSI analysis and explain how these methods are exploited to extract useful information from these images for solving various digital pathology challenges. We classify these methods according to scope and application type. We also collect and summarize some of the most common applications of feature engineering methods in digital pathology. In this survey, we include peer-reviewed articles published between 2000 and 2022. We exclude extended abstracts and papers written in languages other than English. We retrieved papers from the Scopus and Google Scholar search engines using the search query “image feature extraction in digital pathology for whole slide images”, “manual features for whole slide images”, “deep learning features for whole slide images”, and “manual and automatic feature extraction for whole slide images”. We only considered the first hits for each query, until the title and keywords showed that the hits were no longer relevant. This yielded about 1000 papers. We then removed duplicates and screened the papers by abstract, using our exclusion criteria. After extensive forward and backward referencing to collect the most relevant publications, more than 200 papers remained. While we reference all of these remaining papers in this survey, we then identified 93 papers that significantly contribute to the understanding and advancement of manual and automatic feature extraction methods for the analysis of WSIs. In this step, we focused on scientific publications with clear improvement and impact on clinical applications of digital pathology. We discuss these 93 core papers in-depth in this survey, aiming at striking a balance between the length and depth of this survey.

Whole slide imaging

Whole slide imaging (WSI) is the digital equivalent of histological glass slides scanned using digital slide scanners. These images are normally examined by experts, stored, transmitted, or fed to automated image analysis tools. High-resolution images are generated by digital scanners within a short time. Scanners can apply multiple magnifications and focal planes at different resolution levels. WSI is considered beneficial for educational and diagnostic purposes by experts, especially in comparison to static digital images. However, there are practical challenges to WSI analysis. Some of these challenges are related to the scanning technology, which does not provide satisfactory output for 3D cell groups and thick smears. Moreover, poor staining of material and tissue folds may negatively affect the quality of the scanned slides. Therefore, the workflow of the WSI systems requires adaptive technologies and solutions to overcome challenges and increase reliability and throughput. 1

The workflow of WSI can be divided into 4 sequential steps: image acquisition, storage, processing, and visualization. Image acquisition can further be divided into 2 components: image capture and image display. The image-capturing component is composed of a trinocular digital microscope, combining a traditional binocular microscope with a third port to which a high-resolution camera can be mounted. The capturing microscope also has a robotic unit to control illumination intensity and a mechanical stage to hold and move the glass slide. The microscope also allows for automatic fine and coarse focusing. Images are divided into a set of tiles which are captured in sequential order and assembled to create the full digital image replica of the glass slide.

Preparing the glass slide for the scanning requires complying with a standardized approach in preparation and staining, especially for surgical pathology where the placement of the tissue section may reduce the quality if not centered away from the coverslip edges. 109 In addition to the X- and Y-axis, the Z-axis is also considered while scanning WSI images. This axis represents the depth focus which is changed according to the region of interest through observation. The transition of the depth of focus is captured and saved as a Z-axis to enable viewing the images at multiple depth scales. Digital scanners differ according to the methodology of Z-axis focusing, with high-quality scanners focusing on a range of depths across each tile which is a time-consuming process. 205 Unlike surgical pathology, which requires scanning at ×20 magnification, cytology slides require Z-stack scanning at ×40 magnification to ensure higher diagnostic accuracy. The scanning time and file size mainly depend on the Z-stack scanning depth with some scanners, for ×20 resolution, requiring up to 2 min, while ×40 resolution takes up to 4 min. 64 , 117

WSI features analysis

The structure and topology of human tissue are composed of sophisticated and dense biological constructs. WSI absorbs this information in a static, complex form represented by color and texture. The information embedded in these images is huge, which raises many issues concerning information extraction and manipulation. In many cases, the normal processing pipeline of these images involves dividing the image into patches to process each patch individually. Yet, this kind of processing has a high time complexity level since each WSI can be composed of hundreds of patches. Image features are normally extracted from these patches and combined to perform analysis and inference based on the overall WSI. 221

The description of WSIs can be achieved using many digital image processing techniques, each providing a different level of detail. For instance, spatial moments, color profiles, and histograms provide the lowest level of information since they are established based on plain color intensity values. Other techniques, such as model- and transform-based techniques, perform processing based on a higher hierarchy level to extract edges and detect objects within images. Therefore, image processing techniques in WSI can be subdivided into 3 main categories; pixel-, object-, and region-level. 40

Pixel-level techniques describe the lowest level of image details based on color intensity, texture information, and spatial pattern. Object-level techniques describe a higher level of information that characterizes semantic objects within images. Such objects in WSI images can be mitoses, nuclei, lumen, stroma, ducts, crypts, etc. In order to perform object-level processing, image segmentation is used to detect objects at several levels of detail depending on the anatomical elements of these objects. 92 The basic segmentation level detects low-level components such as nuclei, whereas more sophisticated methods detect complex structures such as tumors. Object-level features are then extracted by analyzing the morphology and appearance of the detected objects. On the other hand, region-level techniques describe the highest level of detail by aggregating multiple objects within images that share common characteristics. Regions are then described by a set of features that can be obtained based on topological properties and morphological appearance of aggregated elements. 10 , 144 These features are normally used as input to machine learning techniques for classification, segmentation, clustering, etc. Table 1 lists the most common classical (i.e., non-deep) machine learning techniques utilized by some of the reviewed publications in our survey. In this review, we classify feature extraction techniques into 5 main categories: color, texture, morphological, topological, and deep learning features. We provide a more detailed description and discussion of these features in the following sections.

Color features

Visual examination of WSIs reveals differences in color and appearance between various kinds of tissues. For instance, variation in color between cancerous and benign tissue increases as the level of malignancy increases. The reason is that epithelial nuclei stained in blue spread within red-stained stroma tissue, exhibiting different color intensity values and spatial moments for both cancerous and benign tissues. Therefore, color features based on a range of color spaces, such as RGB, CMYK, HSV, CIEL*a*b, gray-level, etc., are widely used for WSI analysis. 34 , 55 , 124 , 138 , 145 , 182 Table 2 presents color features with corresponding papers.

Pezoa et al. 146 use some of these features for segmentation of HER2 protein over-expression in immunohistochemically stained breast cancer images. Lu et al. 124 use spatial moments to measure similarity between neighboring cells to construct cell clusters based on a similarity distance measure. These features, in addition to other features extracted based on shape analysis methods and cell graph clusters, are used to predict patient survival. Gray-level representation of WSI tiles is used by Peikari et al. 145 to detect regions of interest for breast cancer. Running final detection on the trained models shows that most of the relevant tissue regions are identified. Likewise, Mercan et al. 138 perform region of interest localization based on color features. They use both deconvoluted hematoxylin and eosin color channels as bag of words features to represent tiled patches and sub-patches.

Color variations can also be spotted using color histograms based on tissue, region, and image level. Many studies use color histograms for cancer diagnosis. 97 , 122 , 141 , 167 , 216 Tabesh et al. 182 employ color histograms for prostate cancer diagnosis and to determine the Gleason 69 score that is typically used to estimate prostate cancer grade. The study concludes that color histograms of tumor and non-tumor image samples clearly show different patterns and the performance in terms of accuracy increases when color histograms are generated after removing the background from images in the YCbCr color space. Tissue-level description and quantification can be provided using color histograms in many applications such as image retrieval, 34 tissue micro-environment analysis, 101 and region of interest description. 23 Color histograms are used by Keller et al. 101 to provide a tissue-level description of biological tissue characteristics such as collagenous stroma, duct-lobular unit density, and the presence of elastosis. The authors also compare quantitative representations of these characteristics for visual assessment by pathologists and found that color histograms are effective for the discrimination between tissue elements in digitized histology specimens. Additionally, region-level descriptions can be provided by using color histograms 23 : in this case, clustering is performed according to spatial features to detect meaningful tissue structures such as lobules and ducts. Finally, color histograms can be used for image retrieval: Qi et al. 147 propose a content-based image retrieval (CBIR) framework based on a hierarchical searching algorithm. To this end, they consider a hierarchical annular histogram (HAH) in which the intensity color histograms of consecutive concentric rectangular rings are concatenated together to form a searching feature vector.

Texture and edge features

Image characteristics, such as smoothness, regularity, depth, etc., can be observed in the image texture and color intensity values. These features provide a description of variations between neighboring pixels in terms of color, contrast, and brightness, which emphasize sharp edges. 188 Texture features can be categorized into statistical, structural, model-based, and transform-based features. Statistical texture analysis is widely used to describe images based on higher-order moments and histograms. Besides, the gray-level co-occurrence matrix (GLCM) is a statistical method that uses second-order statistics of the gray-level histograms for texture and color intensity representation. Other statistical methods include the run-length matrix, the singular value decomposition, etc. 26 On the other hand, structural texture analysis is mainly used to process regular textures of WSIs, providing a description of well-defined regularly spaced elements such as parallel lines. 70 In addition to statistical and structural texture, model-based texture analysis techniques are also widely used for the empirical modeling of pixel intensities based on localization. Autoregressive and fractal models (see Section 4.2.3) are 2 examples of model-based descriptors where descriptive features are extracted from the estimated model parameters. Transform-based techniques, on the other hand, convert images into a new form based on their spatial component. Several transforms can be used to describe image texture, such as fast Fourier transform (FFT), Gabor filters, 65 and wavelet transforms. 26 Table 2 presents various texture and edge feature extraction methods with corresponding papers. Cross and Cotton 49 first suggested that fractal models may provide useful morphometric features in histopathology: this intuition was successfully applied for improving menangioma classification together with a combined Gaussian Markov random field, 9 and together with Probabilistic Pairwise Markov models for the detection of prostate cancer. 210

Statistical texture descriptors

Statistical texture features are used by Rathore et al. 149 to represent the distribution of chromatin within nuclei. They use gray-level run-length (GLRLM), co-occurrence matrices, and other morphological and clinical features to identify low- and high-grade gliomas and predict cancer grade. They report GLCM features to be effective for brain cancer diagnosis. Durgamahanthi et al. 58 apply color correction on images before extracting the GLCM features, thereby improving the accuracy performance significantly, whereas Mohan 141 extracts the features by considering all the regions regardless of interest. The study concludes that these features offer good tissue-level discrimination for cancer detection. Region-based description using GLCM is also evaluated by Bejnordi et al. 23 for breast cancer diagnosis, where region-based clustering is established to detect ductal carcinoma in situ. Moreover, combining GLCM with deep learning features extracted using CNN significantly improves the accuracy performance for classification and segmentation of breast cancer as suggested by He et al. 80

GLCM is widely used for nuclei- and tissue-level description and quantification. Keller et al. 101 use it for tissue-level quantification, where representations of tissue elements are compared and correlated with biological visual assessment estimated by experts. GLCM features are also used by Aziz et al. 19 to quantify and analyze tissue-level objects such as cytoplasm, nuclei, lymphocytes, and blood cells. The authors study the impact of color correction on image features for tissue-level object discrimination. They are also used by Chaddad and Tanougast 38 to describe identified abnormal cells for predicting the continuum of colorectal cancer. Dodington et al. 57 use GLCM to measure and analyze pathological response to neoadjuvant chemotherapy used to treat high-risk breast cancer patients. The study extracts features from nuclei to quantitatively study the impact of this treatment on tumor growth and micro-environment. The extracted features include Haralick texture features and statistical metrics based on GLCM. These features are also used by Sharma and Mehra 167 to describe the stochastic texture resulting from the random distribution of cells in WSIs. They are also used by Pezoa et al. 146 in breast cancer for segmentation of HER2 protein overexpression in immunohistochemically stained WSIs.

Structural texture descriptors

Structural texture approaches derive geometrical representations of texture that can be thought of as a spatial organization of texture components. The main components of the resulting models are the structure and spatial organization of texture elements. Texture elements are prominent local constructs that reflect the spatial organization of spatially varying image signals. Image edges, shapes, and Voronoi polygons are examples of texture components. Visual descriptors of texture can be obtained using the Local Binary Pattern (LBP) method. LBP features are extracted based on pixel intensity values where neighboring pixels at a fixed radius are given new binary values. Pixels with intensity greater than or equal to the center pixel are set to 1, whereas the rest are set to 0. The generated binary representation can be normalized and binned to describe patterns within images. Features can then be used for several tasks such as image segmentation, classification, and registration. Variants of LBP are commonly used in the literature for processing WSIs. 62 , 80 , 138 , 141 , 170 , 201 Farooq et al. 62 use different variants of LBP to obtain features from the texture of raw images. They use uniform rotation-invariants for prostate cancer grading. Uniform LBP is also exploited by Bejnordi et al. 23 for region-based WSI description where regions are matched to perform clustering-based detection of ductal carcinoma in situ. A multi-radial LBP is employed by Simon et al. 170 to detect glomerular and distinguish between disease and control glomeruli. A Fourier LBP is used by Mohan 141 for brain cancer grading, whereas variants of LBP descriptors are utilized by Xu et al. 201 for Gleason grading in prostate cancer.

Model-based descriptors

Model-based texture features are obtained by fitting an empirical model based on color intensity values within a range of the pixel neighborhood. These features can improve the accuracy performance of classification models significantly. Fractal coding is an example of model-based descriptors where features are extracted based on image similarity. Images are divided into small blocks mapped using a discrete counteractive affine transformation. Self-similarity is captured at multiple scales where larger blocks at higher resolution are mapped into a transformed version of smaller blocks at lower resolution. 162 . This feature is used by Tabesh et al. 182 to extract mathematical representations of WSIs which are then used to detect prostate cancer. Results show that fractal code features can be used to identify cancerous tissues accurately. Grading of prostate cancer is conducted by Xu et al. 201 using 4 groups of fractal code features where different grid sizes are used for fractal analysis.

Techniques based on the autocorrelation function describe dependencies between pixel neighborhoods based on gray-level intensity. Autoregressive models are typically used to model this linear dependency where a set of parameters are estimated on the 2D spatial domain represented as texture. Durgamahanthi et al. 59 propose a multiresolution autoregressive model to study the complex random texture of WSIs based on nonlinear spatial interaction in the wavelet domain. The wavelet autoregressive model attains higher accuracy performance compared to that achieved by simple statistical autoregressive models. The experiments are conducted to predict malignant brain cancer tissue images. Vaishali et al. 190 , 191 use autoregressive features for brain cancer diagnosis where multiple autoregressive models of different orders are used in the evaluation experiments. Results show that the estimated parameters provide a high discrimination function for malignancy detection.

Transformation-based descriptors

Transformation-based texture analysis of WSIs provides information about the objects’ structural appearance in images. Wavelet decomposition, for instance, transforms images into another domain in which frequency and spatial information are preserved. This kind of transformation provides an in-depth, detailed multi-resolution representation of images, providing more explanatory information found useful for object detection and analysis. 120 The filter bank is also one of the texture descriptors that can be utilized to describe the content of a WSI. An array of bandpass filters can be used to decompose image texture components into multiple components to reveal patterns and emphasize edges. This feature extraction method is exploited by Peikari et al. 145 for automatically triaging WSIs to identify relevant tissue region candidates for tissue microarray analysis. Features are then extracted from the normalized luminance after discarding the hue and saturation channels. To be able to produce feature descriptors invariant to rotation and scaling, 380 root filters are used to describe image tiles. Statistical aggregation functions are used to compress feature vectors into 48 statistical measures. Feature vectors of image tiles are then combined using a bag of visual words approach, which enables modeling images with complex contexts. Results show that these features have the highest impact on classification accuracy performance. A similar conclusion is made by Mohan 141 and Bejnordi et al. 23 for edge detection and texture encoding using the Gabor filter, which is found very effective for processing histopathology images. Farooq et al. 62 use Gabor wavelets to represent spatial details of WSIs and to detect edges. Feature vectors are obtained by rotation and dilation of the mother function to generate Gabor filter variants. Features are then used to do cancer grading. Cancer grading is also conducted by Xu et al., 201 where 12 Gabor filter-related features are extracted from image patches and aggregated using the mean, standard deviation, skewness, and kurtosis of each feature. Gabor wavelet filters are also used by Ma et al. 131 for image retrieval in breast cancer, where a multi-level feature extraction is applied to generate hash codes for similarity estimation. A set of morphological and Gabor features are used as basic-level features, whereas a latent Dirichlet allocation 28 model is utilized for higher-level semantic mining features. Results show that images abundant in texture have been accurately retrieved by Gabor features. Wavelet transforms tend to perform better than Fourier transform since they are able to maintain the localization of frequency and space components in discrete data. Wavelet coefficients represent different resolutions to model texture features uniquely at each level. Therefore, wavelet texture analysis is commonly used to extract meaningful representative features from images. 26 It is used in Tabesh et al. 182 and Almuntashri et al. 13 for prostate cancer detection and grading. In this case, wavelet features are combined with other morphometric features to perform the detection. Multi-scale texture features can be extracted using 3D wavelet decomposition. 37 Each data sample consists of 16 multi-spectral images with a size of 512 × 512 pixels, forming a volume of 512 × 512 × 16 voxels. Wavelet coefficients are then summarized through multiple scales using variance, entropy, and energy functions. The extracted features are then used to classify colorectal cancer tissues.

When it comes to image registration and matching, the scale-invariant feature transform (SIFT) 123 is one of the most popular techniques. Extracting a set of features representing key points in the image provides an adequate measure of similarity between images and between objects within images. Mehta et al. 136 use SIFT to perform image indexing and sub-image retrieval, where a query sub-image is used as a reference to find similar patches within WSIs. Edge features can be used to provide a content-based description of WSI. They are exploited by Caicedo et al. 34 for image retrieval, where histograms based on Sobel 173 edges and Tamura 184 features are part of the set of low-level features extracted from images.

Morphological features

A broad spectrum of digital pathology tasks depends mainly on the analysis of the morphological structure of the tissue. Tissue appearance and nuclei shape are the main characteristics whereby pathologists attempt to make prognosis decisions. 149 Generic shape properties can be accurately described using a variety of generic descriptors, such as moment invariants and Fourier descriptors. Single characteristic shape descriptors can also be used to discriminate objects via different measures such as rectilinearity, circularity, curvature, etc. Evaluation measures of each of these descriptors use different methodologies taking into account processing task, computation time, and application. 223

Region-based descriptors

The description of the interior characterizes the object’s body within its closed boundary. Region-based descriptors can be used to provide a representation of the object’s interior. These descriptors commonly use a wide range of primitives (such as deformable templates, skeleton, and moment invariants) to describe regional shape properties. Cancer cells in WSIs tend to show abnormal shape characteristics because they have unstable growth resulting in a non-uniform shape. Table 3 presents a list of region-based features classified by level of representation. Hu 86 moments are statistical descriptors that discriminate normal from abnormal cells based on geometric shape and morphology. These features are used by Sharma and Mehra 167 and Rebouças Filho et al. 150 for cancer detection and diagnosis. Rathore et al. 149 use a combination of different features, including morphological features extracted from WSIs, to develop a cancer grading model. The set of morphological features includes Euler number, convex area, etc. They use K-means clustering to create connected components and average the extracted feature values for each component. Averages are then used as input to train a classification model. Results show that accuracy is significantly improved by combining morphological and texture features with clinical features. Tabesh et al. 182 use the MAGIC system 186 to segment WSIs into smaller components. Identified objects generated by the segmentation process include nuclei, stroma, lumens, etc. The study uses the MAGIC system to extract 48 descriptive features from the identified objects to detect prostate cancer.

Morphological features are also used to measure the diversity and homogeneity of nuclei in WSI images. Lu et al. 124 use shape, area, solidity, circularity, etc. to measure the morphological proximity of nuclei cells and construct cluster graphs. Features are then extracted from graphs and used as input to the prediction model. Nuclei region-based features are also utilized in Chankong et al. 39 for cervical cell classification, where 9 features are input to 5 machine-learning models to classify cell images. Aziz et al. 19 use region-based features to quantify and compare tissue-level objects such as cytoplasm, nuclei, lymphocytes, and blood cells. They are also used by Dodington et al. 57 to study the impact of neoadjuvant chemotherapy treatment on breast cancer patients. They are used to compare responses of different patients based on nuclei- and tissue-level features.

Geometric region-based shape descriptors can also be extracted from tumor regions in WSIs. Wang et al. 197 develop a CNN architecture to detect tumor regions and extract region- and boundary-based descriptors from tumor regions to measure the survival outcome of patients. They report these features to be highly correlated with patients’ survival outcomes. Lopez et al. 122 develop a computational pipeline to predict prostate carcinoma biological behavior. The study used a set of geometrical region-based features from the lumen, nuclei, and gland units. These features are then combined with architectural features to predict Gleason patterns.

Spatial characteristics in WSIs are commonly employed to quantify and analyze tumoral heterogeneity. Mi et al. 139 develop a multi-module workflow to quantify spatial heterogeneity within the tumor micro-environment using quantitative correlation analysis of 5 immune markers in different tissue regions. The study uses image processing to perform segmentation to extract and identify immune marker cells. Point pattern analysis is then conducted to measure the density of cell markers within each region and generate cell cluster hierarchies. The study characterizes 3 different regions, including normal tissue, central tumor, and invasive front. Inter- and intra-tumoral heterogeneities are compared by examining the correlation of immune markers for each region. Regional shape descriptors, such as convexity, circularity, and eccentricity, are extracted from cell clusters to measure immune contexture heterogeneity. The study concludes that invasive front tissue has higher immune cell density than tumor and normal tissues. Heterogeneity between different kinds of ducts in ductal carcinoma in situ (DCIS) breast cancer is also estimated in Hayward et al. 78 by using a set of region-based features including major and minor axis and cross-sectional area.

Contour-based descriptors

Object visual characteristics can be observed in the boundary’s structural and geometrical appearance. The surrounding boundary curves can be used to discriminate objects and identify unusual patterns. Curves can be described using different models based on the analysis method and extracted information. This information can be utilized to describe shape variability. Table 4 presents some contour-based features classified by objects in WSI. Regular sharp nuclei are commonly used to characterize and indicate low-grade gliomas, while high-grade gliomas tend to show higher irregularity and heterogeneity in the boundary. Rathore et al. 149 use a set of boundary geometric descriptors to model boundary smoothness and regularity to evaluate nuclei and tumor heterogeneity for cancer grading based on visual appearance. Boundary regularity and shape circularity is found to be helpful in learning and predicting low- and high-grade gliomas.

In segmentation and nuclei detection problems, identifying the exact nuclei contour is a challenge since variations in size and shape between different types of nuclei increase the complexity of extracting the exact boundary. To address this challenge, Nagase et al. 125 use radial distance to estimate the size and boundary of nuclei in well-differentiated hepatocellular carcinoma. The nuclei density estimation is used for cancer diagnosis and WSI image visualization.

Object detection is one of the common problems in computer vision and image processing. Finding particular objects with a given probability in addition to the coordinates of the bounding box is the output of the detection process. In many cases, coordinates of additional supporting points are also generated to pinpoint the main common shape characteristics of the object. These points represent object landmarks that are used for many applications, such as image registration, reconstruction, classification, etc. WSIs tend to have a complex structural and contextual appearance with relatively high similarities among different images. Uniquely identifying and describing different objects within WSIs require sophisticated shape analysis approaches. Landmark-based image analysis is one of the shape descriptors commonly used to analyze WSIs. Zhang et al. 212 develop a Bayesian-based method for tumor region landmarks detection. The study proposes to perform lung cancer survival prediction using landmark-based features categorized as distance- and model-based features. These features are mainly used by the study to describe the roughness of the tumor contour, which is one of the important indicators for cancer staging. The landmark-based analysis is also used to reconstruct 3D WSIs by detecting corresponding landmarks in multiple consecutive slice images. Kugler et al. 108 use template matching to detect landmarks in different slice images for mapping based on the smoothed trajectories. The objective is to match corresponding similar landmarks to align images vertically. They use trajectory smoothing to alleviate the problem of damaged and folded portions by rejecting unreliable landmarks based on a confidence factor.

Irregularly shaped objects in WSIs can be described quantitatively using fractal geometry. The fractal dimension is different from the geometric dimension in that it measures the space-filling capacity of an object. The fractal dimension of a straight line is one, which is similar to its topological dimension. 182 Edge information in WSIs describes finer image details and can be used to detect similar patterns to characterize different tissues. Fractal dimension features measure similarities between patterns in histological images. They also measure the complexity and geometrical description of the space allocated for each object or a set of points. 13 . Tabesh et al. 182 use the features extracted through analysis of their fractal dimension to develop a prostate cancer diagnosis framework. The results show that fractal dimension features alone can be used to produce up to 92% accuracy performance. Almuntashri et al. 13 derived fractal dimensions from wavelet coefficients extracted from images. Wavelet-based fractal features are combined with fractal dimension features extracted from raw images and Haar wavelet energy coefficients to form the feature vectors. Curvature-based shape analysis is established based on differential geometry describing closed contours of objects in WSIs, such as nuclei and tumors. Geometric modeling of curves can be used for object discrimination based on contour representation. Al-Thelaya et al. 10 and Agus et al. 4 propose a shape processing framework for visual exploration of WSIs based on curvature analysis of the closed contours of nuclei. The framework uses elliptic Fourier analysis to form a feature vector invariant to rigid shape transformation and translation. The resulting features are then harnessed to discriminate, classify, and visualize nuclei based on contour analysis.

Topological features

The topological structure of tissue images reflects the biological changes resulting from various diseases. These changes explain the underlying functional state and condition of a tissue sample. Modeling these changes in a topological relational form reveals constructive patterns that can be used for biological analysis to predict different diseases. 52 Cancer aggressiveness identification depends on many topological characteristics, such as the interplay, distribution, and arrangement of nuclei. Tissue types have different nuclear topological structures that exhibit variable graph densities, connections, and topology. Graph-based approaches are commonly used to accurately represent these characteristics based on the distribution of nuclei in histopathology images. The interest in these approaches is rapidly growing to characterize nuclei features and arrangements. Table 5 presents several topological feature extraction methods based on different digital pathology applications. Lu et al. 124 use a computational graph-based approach to describe the distribution and structure of nuclei in histopathology images. The set of nuclear morphological features and spatial position are used to construct a nuclei cell cluster graph. Quantification measurements are extracted for cluster graphs to describe the diversity of WSIs in terms of cellular morphology, appearance, and architecture. These features are mainly used to measure morphological heterogeneity within nuclei in WSIs. In addition to histomorphometric features, these features are used to develop a machine learning classifier for lung cancer short- and long-term survival prediction. They are also used to predict the human papillomavirus (HPV) status of oropharyngeal squamous cell carcinoma (OP-SCCs). Graph-based features are also leveraged by Yener 207 for modeling the structure–function relationship. The study proposes a cell-graph modeling framework for WSI image analysis. After cell segmentation, graphs are constructed based on Euclidean distance thresholding which reveals interesting patterns in images. A set of local- and global-level features are then extracted from the constructed graph. Local-level features such as cell size, convexity, physical contact, shape, etc. are extracted based on attributes of individual cells. In contrast, scope and global-level features are extracted by computing the distribution of cell- and graph-level features, such as the size of connected components and the spectrum of the graph. Global graph features are also used by Sharma et al. 165 to provide a visual description of gastric cancer. Several graphs are constructed based on nuclei attributes. Features are then extracted from each graph separately. Global graph features include not only moment-based vertex and edge attributes but also the number of edges and vertices, graph density and irregularity, number of connected components and triangles, and cyclomatic number.

The spatial distribution of the tissue architectural components, such as nuclei, lumen, and gland units, can be modeled using the Delaunay 51 triangulation. Graph-based features extracted from the Delaunay triangulation can be used to describe different structural characteristics of histological images. 44 , 111 A triangulation is established by joining a set of points using non-intersecting straight lines to form a set of triangles. Each triangle is formed by 3 vertices preserving the empty circumcircle property, which implies that each triangle vertices can be joined using a circle such that no vertex point falls within the interior of the circle. 128 Lopez et al. 122 extract a set of graph-based features using the Delaunay triangulation to perform cancer grading. They calculated the side lengths and area of the triangles from the Delaunay triangulation to classify WSIs into 4 classes based on the Gleason pattern. Lee et al. 111 evaluate the use of graph features for predicting disease recurrence in prostate cancer. A set of graph-based, Voronoi diagrams 17 and Delaunay triangulation features are extracted from a group of tissue microarray cores scanned using a digital whole slide scanner. Statistical and classification analysis show that features extracted from the benign tissue field surrounding the tumor region field can be successfully used for recurrence prediction. Results show that shape descriptors and nuclear topology features are the predominant predictors of disease recurrence. Delaunay triangulation is also exploited by Cheng et al. 44 to generate a topological representation to represent relationships between different types of nuclei based on appearance and distance. The extracted features are then used for survival prediction for patients with kidney cancer.

Deep learning features

The application of deep learning methods in digital pathology is rapidly increasing. The output of these applications proved to be generally superior to the use of “engineered” features and had a major impact on digital pathology progression. Deep learning methods are used to segment and detect nuclei and regions of interest within WSIs. They are also used for cancer diagnosis, grading, and nuclei classification. 144 Table 6 , Table 7 present a classification of features extracted using several deep learning architectures based on digital pathology applications.

Convolutional neural networks (CNNs)

Features learned by convolutional neural networks (CNNs) effectively describe whole slide images (WSIs). These features can be used as input to other deep learning models or fully connected layers for classification, segmentation, and detection tasks. Fig. 1 illustrates a typical WSI analysis workflow using a simple CNN architecture. Many well-known deep architectures, such as Inception, DenseNet, and ResNet, are based on CNNs. Due to their high accuracy, many solutions have been developed based on these architectures to process WSIs using models pre-trained on large data sets, such as ImageNet. 107 Some studies 67 , 84 , 195 , 197 , 213 propose a custom deep learning CNN architecture that better fits their learning task. In Zhang et al. 213 and Gao et al., 67 features are used to describe WSIs and form a superpixel representation, 2 which is then used to construct a graph embedding representation. The graph embedding features, in addition to the CNN’s features, are used as input to a graph convolutional network (GCN) for cancer diagnosis and cancer region localization. On the other hand, 2 different CNN architectures are evaluated in Höfener et al. 84 for nuclei detection based on the map of points methodology, in which points pinpoint the center of each cell in the WSI image. CNN models are also used for the survival prediction of cancer patients: Wang et al. 197 develop a deep CNN to detect tumor regions within WSIs. A set of shape and boundary descriptors are extracted from the detected tumor regions to predict patient survival outcomes. CNN models are also used in Wang et al. 195 to perform segmentation of WSI images to extract nuclei and other objects. In the following, we discuss how features extracted based on well-known deep CNN architectures are used to provide efficient solutions for WSI analysis.

Inception networks 181

The Inception family of CNNs has been successfully applied in many histopathology applications, such as cancer detection and nuclei classification. For example, Wang et al. 196 used Inception-V3 for colorectal cancer detection, collecting data from hospitals and medical research centers in 3 countries. The authors validated the data set through a labeling review process involving domain experts. The data set is available online for researchers to further advance the field. The developed model demonstrated higher performance than human domain experts. Similarly, Arvaniti et al. 16 evaluated Inception-V3 for grading prostate cancer using manually annotated images of 886 patients. Results showed that deep CNN architectures could provide assistance to pathologists in real-life scenarios.

In addition, Iizuka et al. 90 used Inception-V3 for the classification of image tiles into adenocarcinoma, adenoma, and non-neoplastic. A long short-term memory (LSTM) was employed to aggregate features generated by Inception-V3 by separating whole slide images into a collection of tiles. The authors annotated a set of WSI images diagnosed with colon and stomach cancer, which were used to train and validate both Inception-V3 and LSTM models.

DenseNet networks 89

Huang et al. 88 proposed DenseNet, a deep convolutional neural network architecture, in 2016. It introduces the innovative notion of “dense connections,” in which each layer takes input from all preceding layers and delivers its feature maps to all following layers. This method promotes maximum information flow and reuse throughout the network, resulting in greater accuracy with fewer parameters. DenseNet networks have demonstrated cutting-edge performance in a variety of computer vision applications such as picture classification, object identification, and segmentation. Because of its ability to collect complex descriptive characteristics and generate meaningful representations from medical images, DenseNet architectures are widely employed in biomedical image analysis. Deep learning features extracted using DenseNet are used in Kalra et al. 97 to describe image patch samples for image retrieval and matching. The study uses DenseNet to generate the feature vectors of size 1, 024 by applying the Global Average Pooling (GAP) over the feature maps from the last convolution layer. DenseNet networks capture more complex descriptive features compared to other networks, such as VGG19 and Inception. DenseNet features are also used in Shi et al. 169 for cell clustering. Clusters are learned using K-means based on the visual similarity of nuclei. The generated clusters are then used for graph embedding using GCN.

GoogleNet networks 180

GoogleNet is developed based on the Inception architecture, where Inception modules are combined based on multiple filters with different sizes. Blocks are stacked with occasional max-pooling layers. GoogleNet is utilized in del Toro et al. 189 for the classification of high-grade Gleason in prostate cancer. Randomly sampled patches are extracted from WSIs to reduce the computation time. Different parameters, such as patch size, number of random samples, and color channels, are evaluated to determine the best configuration.

AlexNet networks 107

AlexNet consists of 8 layers, including convolutions, max-pooling, and fully connected layers. All layers, except for the output layer, use rectified linear units (ReLU) instead of the hyperbolic tangent (tanh) activation, and model overfitting is avoided by using dropout layers. 82 It was proposed in 2012 by Krizhevsky et al. 107 and won the ImageNet large-scale visual recognition challenge. Many studies develop WSI analysis solutions based on the AlexNet architecture. It is used by del Toro et al. 189 for the classification of high-grade Gleason in prostate cancer. The study compares the speed and accuracy performance of AlexNet with GoogleNet and found that AlexNet was faster while producing comparable accuracy. Ren et al. 152 use AlexNet to learn features of image patches for survival prediction of prostate cancer patients. These features along with genomic sequences are used as input to an LSTM model to learn the spatial relationship of adjacent image patches.

ResNet networks 79

Chen et al. 41 trained a ResNet-50 model to describe WSIs divided into patches. These features are then used to form a graph embedding for the whole slide image, which is then used as input to a GCN model to perform survival prediction of cancer patients. Furthermore, Arvaniti et al. 16 compare ResNet-50 with other deep architectures for Gleason grading in prostate cancer; the deep learning architectures produce comparable accuracy performance compared to human experts. ResNet-50 is used by Cheng et al. 45 for cervical cancer diagnosis. Features are extracted from low-resolution images to identify the most suspicious regions within WSIs, which are then verified using higher-resolution images by another ResNet-50 model, resulting in better accuracy.

ResNet-152 and ResNet-101 are used by Dodington et al. 57 for tumor region identification and nuclei segmentation, where a set of texture and shape features are extracted from nuclei to analyze the impact of neoadjuvant chemotherapy treatment on breast cancer patients. In another study by Campanella et al., 35 ResNet-34 is utilized for cancer grading based on the multiple instance learning approach, where top-ranking image tiles features are aggregated using an RNN model.

U-Nets 155

U-Net is one of the most popular deep convolutional neural network architectures for segmenting and locating anatomical structures in WSIs. Jha et al. 94 propose a detect-then-segment framework for glomerulus detection based on the U-Net architecture. Objects within images are first detected during the initial training phase, then segmentations of detected glomeruli are further refined using a second training phase. The proposed framework is evaluated against previously detected glomeruli image data, showing improved performance. A U-Net architecture is also used by Dodington et al. 57 for tumor region identification and nuclei segmentation in breast cancer.

VGG networks 171

VGG network is a widely used deep convolutional neural network architecture for segmenting and recognizing the different scales of context and semantic patterns in WSIs. Chen et al. 43 use VGG16 to extract multi-level feature maps that are superimposed by superpixel contours to recognize the different scales of context and semantic patterns to segment WSIs. In comparison, Xu et al. 201 utilize VGG16 as a baseline model and compare it with an SVM model trained using LBP features for Gleason grading in prostate cancer.

For region-based tissue classification and visualization of WSI images, Faust et al. 63 use VGG19 to conduct probability scores and leverage t-SNE dimension reduction to visualize different types of tissue classes identified by the model. 132 VGG19 is also used by Dodington et al. 57 for tumor region localization, followed by tiled images segmentation using a U-Net architecture to do benign vs. malignant tissue-level classification. Finally, tissue-level quantification is conducted based on nuclei shape, intensity, and texture features to study the impact of neoadjuvant chemotherapy treatment on breast cancer patients.

MobileNet networks 85

MobileNet is a simple architecture initially developed to perform the inference step on small devices with low computation power. It shows high accuracy performance in many application domains, especially the computer vision domain, for tasks such as object detection, face recognition, and fine-grained classification. 209 Due to its lightweight, efficient, and portable architecture, it has become one of the most commonly adapted neural networks. Using depth- and point-wise separable convolution layers significantly reduces the number of parameters and keeps the accuracy performance at a level comparable to heavyweight architectures. MobileNet is used in many histopathology applications. Arvaniti et al. 16 develop a deep learning processing framework based on the MobileNet architecture. Despite its lightweight architecture, MobileNet showed higher performance than other architectures, such as VGG-16, Inception-V3, ResNet-50, and DenseNet-121, as evaluated in the experiments by the aforementioned authors. The authors analyze the performance of the 5 architectures for Gleason grading in prostate cancer, where a set of manually annotated WSIs are generated by 2 pathologists and released publicly. The accuracy of the pathologists is also analyzed and compared with that of the deep learning architectures. Deep learning models achieve expert-level annotations and grading of prostate cancer, demonstrating the potential of using deep learning technology to assist pathologists.

Recurrent neural networks (RNNs)

RNNs are artificial neural networks that receive sequential or time series data as input. 157 These neural networks are often employed for ordinal or temporal research problems like natural language processing (NLP), speech recognition, and image captioning. Several prominent applications like Apple’s Siri, voice search, and Google Translate are developed based on these neural networks. RNNs distinguish themselves from CNNs by their “memory,” which allows them to modify their current state and output using information from previous inputs. While typical deep neural networks assume that individual input/output pairs are independent of each other, the output of recurrent neural networks is dependent on the sequence’s prior inputs. While original simple RNNs are mainly used to solve unsophisticated problems using their shallow learning capability, more sophisticated RNN variants such as long short-term memory (LSTM) networks 83 and gated recurrent unit (GRU) networks 47 can be used to solve more complex problems that require more training capacity and higher memory. RNNs are typically used in the context of WSI to aggregate features generated by dividing WSIs into a sequence of adjacent tiles. 32 , 35 , 45 , 90 , 151 , 152 Campanella et al. 35 and Kanavati et al. 98 utilize RNNs for cancer grading using a slide-level aggregation of top-ranking tiles based on features generated by a CNN model. To predict the final slide-level classification, the most suspicious tiles in each slide are consecutively passed to the RNN model. Ren et al. 152 use LSTMs to model the spatial relationship between adjacent image tiles based on the image features generated by an AlexNet model together with genomic sequencing data. Features learned by the LSTM are used as input to a multilayer perceptron (MLP) model to identify computational biomarkers for survival prediction of prostate cancer patients. LSTMs are also utilized by Iizuka et al. 90 to aggregate features generated using a CNN model by dividing WSIs into a set of tiles. An arbitrary number of tiles are extracted from the tissue areas for each slide and fed all of the tiles into the RNN model. To avoid reliance on tile input order, the order of the features of the tiles is randomized at each step during training.

Mukherjee et al. 143 develop a new variant of RNNs based on a sequence of CNNs. The new RNN model is called recurrent CNN (RCNN). The model is developed to solve the problem of super-resolution in WSI by utilizing multiple instances of each WSI image, each at a different resolution. Each image instance is used as input to a single CNN to generate the next higher-resolution instance of the same image. Features generated by this step are used as input to the next CNN, whose input is the next resolution level of the same image generating the third resolution level. The process continues until the highest-resolution image instance is reconstructed. The name “RCNN” reflects the fact that features of each CNN in the sequence are used as input to the adjacent sequential CNN. Multiple resolution WSIs are also used by Cheng et al. 45 to predict lesion cells using a progressive lesion cell recognition method that combines low- and high-resolution WSIs, as well as a recurrent neural network-based WSI classification model that evaluates the lesion degree of WSIs. A CNN model first analyzes WSIs at low resolution to identify positive regions, which are then validated at high resolution by another CNN model. Finally, the system identifies the 10 most suspicious lesion regions in each slide for cytopathologists to analyze further. An RNN model is used to provide a lesion degree likelihood based on the features of the top 10 suspicious images extracted using a CNN.

Deep AutoEncoder networks

The output layer of an autoencoder neural network has the same dimensionality as the input layer. The number of output units equals the number of input units. Therefore, an autoencoder is an unsupervised neural network that replicates data from input to output and is thus sometimes referred to as a replicator neural network. The challenge is to replicate the input data based on a reduced smaller representation generated by the middle layers of the autoencoder. 20 Different types of deep autoencoders are used for extracting features from WSI, including sparse autoencoders, variational autoencoder, 178 and convolutional autoencoder. 156

Due to their potential for representation learning and transfer learning for deep architectures, autoencoders are used in a wide range of applications. The use of features learned by autoencoders for classification and regression is widespread. Lomacenkova and Arandjelovic 121 conduct an experimental evaluation based on deep autoencoder to study the effect of the latent representation and the loss function on the patch-based classification of WSIs for breast cancer detection. The results demonstrated that task-specific loss function adjustments that take into account the content of individual patches in a more sophisticated manner result in a large drop in the false-negative rate. Çelik and Karabatak 36 conduct experiments to evaluate the impact of using different input image sizes for the compression of WSIs. The study shows that autoencoders can be used to reconstruct high-dimensional WSIs using low-dimensional latent representations.

Stacked sparse autoencoders are used by Xu et al. 202 for nuclei detection in breast cancer WSIs by learning high-level features based on various regions of interest containing nuclei. To do this, a sliding window is moved across the entire image to identify candidate image regions for a subsequent classification stage. A trained Softmax classifier is then used for a binary prediction of the presence or absence of a nucleus in each patch. The sparse autoencoder is also used by Cheng et al. 44 for survival prediction of patients with renal tumors. Image patches of different types of nuclei are used as input to the sparse autoencoder model for a low-dimensionality representation of nuclei. Image features are then clustered using k-means to group nuclei with similar appearance and form a topological representation of the whole slide image using Delaunay triangulation.

The convolutional autoencoder architecture is developed based on CNN, where the encoder layers are convolution layers, and the decoder layers are deconvolution layers. This architecture is used by Roy et al. 156 based on a combination of supervised and unsupervised approaches for learning class conditional data-driven feature distributions. The study presents a convolutional autoencoder (CAE)-based method for learning structural image features for segmenting and classifying liver cancer WSI. The proposed network comprises 2 modules: an autoencoder for learning image features and a classifier for supervised classification. Registration of WSI images is commonly achieved using features learned by autoencoders. Awan and Rajpoot 18 conducted a study to evaluate the use of convolutional autoencoders for WSI registration by maximizing feature similarity between the fixed and moving images.

Variational autoencoders use a variational approach for latent representation. The stochastic gradient variational Bayes estimator is used as an additional loss component and a specific estimator for the training algorithm. Variational autoencoders are used by Sun et al. 178 for survival prediction of individuals with colorectal cancer by analyzing the relationship between the features learned via the variational autoencoder and the prognosis of colorectal cancer patients. Following that, the generated features are used to predict survival after adjuvant chemotherapy to improve risk classification for colorectal cancer patients. The signature-based features could assist clinicians in determining the duration of chemotherapy in patients.

Graph deep embedding

In many applications, data is represented as a graph to model the relationships between entities. Various graph structures can be chosen based on the underlying problem domain and the data structure. The simplest graph representation is composed of n nodes connected with undirected edges with no parallel edges and no loops where the degree of every vertex is at most ( n–1 ). However, several applications require more complicated graph structures, such as trees, cyclic graphs, acyclic graphs, and bipartite graphs. Different deep graph embeddings have been proposed in the literature, such as structural deep network embeddings, deep neural networks for learning graph representations, and graph convolutional networks. 71 Node embeddings aim at reducing the data dimensionality of the graph representation and the node’s locality and neighborhood information. These node embeddings can then be used to train machine learning and deep learning models to perform link prediction or node classification. 76 In graph learning, nodes are associated with attributes that are usually used as input features. In many situations, the aim is to predict the type of node based on a set of pre-labeled nodes in the graph. Connections between nodes are represented as weighted edges summarized by an adjacency matrix. This graph learning methodology aims to consider the neighborhood of the predicted node by feeding attributes of the connected nodes in addition to the attributes of the predicted node to the machine- or deep learning model.

Graph convolutional networks (GCNs)

GCNs were developed based on the concept of graph convolutions, in which features of the relevant neighborhood of each node are convolutionally aggregated through multiple layers. Weights of aggregation filters are trained to predict the node’s type. Input data can be represented as a feature matrix with the size of the number of nodes times their attributes. Edges are also represented as an adjacency matrix. Feature matrices are then processed through network layers, generating an updated feature matrix in each layer by training a set of aggregation functions that consider the local node’s neighborhood as input. Feature sets become increasingly more abstract as they pass through network layers. 103 GCNs are effective deep learning networks that capture complex spatial relationships and distribution of images, especially large WSI images. However, challenges in training deep GCNs have only recently been successfully addressed by Li et al. 115 Spatial features of different nuclei and tissue may not be sufficiently representative for deep learning modeling. Other information, such as the relation between different objects within images, could provide useful input to improve the deep learning model accuracy. Integrating morphology in addition to topological information captures cellular morphology and global distribution of nuclei. 67 Fig. 2 shows a typical workflow of WSI graph deep embedding using GCN.

Graphs are constructed using the spatial attributes and structure of components of WSIs to establish the relationship between identified components. This process involves high pre-processing overhead, including segmentation and spatial feature extraction. The final step is to train the GCN based on the constructed graph structure and node spatial attributes. 41 , 166 , 179 , 219 To avoid segmentation of WSIs, some studies extract random patches scattered across the whole image, which are used as nodes in the graph. Yet, spatial features and relationship establishment are required to form the final graph. In most cases, CNNs are employed to perform spatial feature extraction. 215 Gao et al. 67 develop a patch-level CNN–GCN framework for the diagnosis of breast cancer. A CNN is used to describe patches and generate spatial features, and the relationship between nodes is defined using the K-nearest neighbor algorithm. The constructed graph is then used as input to the clique GCN model, where the final output features are fed into the graph pooling layer, followed by fully connected layers and a softmax function. A patch-level methodology is also used by Chen et al., 41 in which features are extracted from patches using the ResNet-50 CNN model. These features are then used as attributes to describe nodes in the graph. On the other hand, graph edges are established based on the Euclidean distance where messages are passed between nodes in the local Euclidean neighborhood. The graph is used as input to the GCN model.

Graph edges can also be established through cell clustering. Shi et al. 169 use features learned using DenseNet-121 for cell clustering based on image similarity. Relations between nodes are then established based on the generated clusters. The graph representation is then input to the GCN model to learn features integrated with CNN features by liner projection. Cell-level graph representation is also used by Zhou et al. 219 for grading colorectal cancer using GCN. A set of shape and appearance features are used to describe the segmented nuclei represented as nodes in the graph. Relations between nodes are estimated using the Euclidean distance, where the nodes’ degree is limited based on a k-nearest neighbor predefined parameter. A similar approach is used by Wang et al. 195 for Gleason grading in prostate cancer, where a CNN model is developed to perform nuclei segmentation. Morphology descriptors and texture features are used to describe each nucleus, and the GCN model is used to perform Gleason scoring based on weakly supervised methodology. The features learned by segmentation models can also be used to generate graph embedding to train GCN. Ye et al. 206 exploits the semantic features generated by the U-Net segmentation model. Graph nodes and edges are formed by capturing the dependency between lesion areas based on these features.

Attention-based networks

The transformer neural network is a unique encoder–decoder architecture that aims to tackle sequence-to-sequence problems preserving deep dependencies between data sequences. It was first proposed for natural language processing in the paper “Attention Is All You Need”. 192 Yet, it is currently a cutting-edge approach for many application domains, including medical image analysis. 81

In digital pathology, transformer architectures are used to model the relationship between image patches that belong to a single WSI. Some studies adopt this technique to handle the problem of irrelevance between image patches in the multiple instance learning (MIL) approaches. It is also used to capture local and global dependencies between multiple WSIs of each patient or multiple patients. Liang et al. 118 propose an attention-based context-aware GCN network for the localization and classification of lymph node metastasis in breast cancer. A transformer-based MIL is proposed by Shao et al. 164 to model the dependency between input instances and explore the spatial and morphological relationship between them. To change the attention region in WSIs adaptively based on input clinical records, Takagi et al. 183 propose a transformer-based framework to model the relationships between medical imaging data and clinical data of each individual patient. Stegmüller et al. 176 use a transformer-based model to solve the problem of discarding the inter-patches interactions while uniformly processing WSIs patches. The attention-based network allocates discriminative regions within images by exploiting a differentiable recommendation to dedicate computational resources according to a region-based attention mechanism. Yin et al. 208 propose a lightweight pyramidal architecture to minimize model parameters by introducing a model’s feature extractor with a token-to-token vision transformer. The model’s performance is improved by integrating local and global features based on picture pyramids of various receptive fields. Chen et al. 42 propose a transformer-based framework to learn co-attention mapping between genomic features and WSIs. The co-attention mapping reduces the space complexity of bags of words and provides a visualization of multimodal interactions. A hierarchical mapping between genomics and WSIs is also suggested by Li et al. 114 for survival prediction using a hierarchical-based multimodal transformer.

Deep generative adversarial networks

Generative modeling is an unsupervised machine learning task that involves automatically discovering and learning the regularities or patterns in input data. In this way, the obtained models can be used to generate or output new examples that could have been drawn from the original data set. Generative adversarial networks (GANs) is a generative modeling approach that employs deep learning methods such as convolutional neural networks. GANs are a way of training 2 mutually interconnected models, where one of them is a generative model, and the other model is a discriminator. The generator model is mainly used to generate new examples, and the discriminator model classifies examples as either real (from the domain) or fake (generated). The 2 models are trained in an adversarial zero-sum game until the discriminator model is fooled about half the time, indicating that the generator model generates plausible examples. 159

GANs are shown to be successful in many image-generation applications, such as image-to-image translation, text-to-image translation, image reconstruction, image synthesis, image segmentation, etc. These applications triggered new research in digital pathology for processing WSIs using generative models, including WSI enhancement, reconstruction, segmentation, augmentation, stain normalization, etc. 96 GAN-based architectures are widely used for WSI-style transfer. 142 BenTaieb and Hamarneh 24 and Cho et al. 46 develop GAN models for stain normalization across different data sets. The models are evaluated for the segmentation and classification of WSIs. They propose a special loss function to minimize the difference between the target and input images’ latent features, thus preserving the classifier’s performance. Zanjani et al. 211 conduct experiments on 3 deep unsupervised learning architectures, including GAN, variational autoencoder, and deep convolutional Gaussian mixture. The later architecture shows higher performance in terms of color constancy of the normalized images.

CycleGAN 220 is a GAN-based architecture for automatically training image-to-image translation models without paired samples. The models are trained using a set of unpaired images from the source and target domains by learning a mapping between the 2 domains such that the distributions of the 2 domains are indistinguishable. CycleGAN is used by Zhou et al. 218 for color normalization of H&E stained images. A template-free cycle consistency GAN method is proposed to automatically learn a stable stain color domain based on the domain of the training data set. Shaban et al. 163 use the CycleGAN model to stain normalize input images before feeding them into another model for classification. Cai et al. 33 develop a new unpaired GAN-based architecture called transitive adversarial network. The new architecture is simpler and achieves better results compare to other GAN-based architectures. Segmentation of WSIs and nuclei detection has been widely achieved using variants of GAN-based architectures. Koyun and Yildirim 106 approach the nuclei segmentation problem as an image-to-image translation. The input nuclei domain is represented by WSIs, whereas the target domain is represented by randomly generated nuclei shapes. Li et al. 116 present a new conditional generative adversarial network to identify nuclei to explore the spatial distribution of the probability map. They also introduce Residual Attention U-Net, a generator created by fusing the residual attention module with U-Net for producing local spatial relation information from WSIs. The conditional GAN (cGAN) 140 is also used for nuclei segmentation and classification. Mahmood et al. 134 train a cGAN network for multi-organ nuclei segmentation based on spectral normalization and gradient penalty. An adversarial term is used to impose higher-order spatial consistency during the training. The task is formulated as a regression problem rather than a classification problem.

Applications of feature engineering in digital pathology

The development of WSI opens the door for several applications in digital and computational pathology. It allows storing tissue slides in a digital format, enabling the automation of several tasks in the histopathology workflow. The highly diverse and complex content of WSIs results in a set of complex problems in different application domains. Various solutions are proposed based on AI solutions. Table 8 presents some applications of feature extraction methods in digital pathology.

WSI image segmentation

Identifying and quantifying different nuclei, glandular structures, and anatomical regions with different visual characteristics are essential for the clinical analysis of WSI images. Pathologists often need to annotate regions that provide indications of biological evidence or clinical symptoms. Segmentation of molecular objects with different visual properties within WSI images facilitates the visual characterization and the analytic description of these objects. Moreover, WSI segmentation is a fundamental requirement for further analysis of WSI images on a granular level. Nuclei classification, tumor localization, tissue type identification, etc. require pre-segmented images to process each object separately. 94 , 102 Fig. 3 illustrates a typical workflow of WSI segmentation.

Some histological image analyses involve a cumbersome and complicated identification and scoring process. Scoring of kidney glomeruli samples is one of the most complicated operations that require high-throughput and consistent handling. Expert pathologists may need statistical analysis to identify glomerular features in order to perform sample scoring. Therefore, highly accurate automation is one of the preferred solutions to reduce the complexity of histological analysis. 168 Automatic glomerular instance segmentation is an evolving research area to provide pixel-wise annotation for each glomerulus to facilitate quantitative measurements of glomerular features. Jha et al. 94 propose a framework based on U-Net’s deep network architecture to perform instance-level segmentation of glomeruli. Ye et al. 206 also propose an encoder–decoder architecture based on U-Net to perform segmentation of breast cancer region-of-interest (ROI) images, whereas a GCN is used to perform pixel-level labeling of image regions into normal, benign, in-situ carcinoma, and invasive carcinoma. Wang et al. 197 address the segmentation problem by developing a deep CNN to divide WSI into 3 regions (tumor, normal, and background). They divide images into patches and transfer the gray-scale images into ternary images. A set of shape and boundary descriptors are then extracted from the detected tumor regions and used to predict patient survival outcomes.

To enable efficient supervised deep learning for segmentation tasks, a sufficient number of segmented WSI images should be provided. The availability of this data is a real challenge since it requires tremendous effort. WSI images include millions of nuclei and other types of tissue components. Pathologists have to look into these images individually, identifying components and accurately defining their boundaries. The complexity of this analysis results from several parameters related to tissue types such as tissue structure, morphology attributes, color variations, staining materials, and scanning technologies. These parameters complicate the generation of sufficient data to train deep learning models. To overcome this problem, weakly supervised methods and models are used to perform WSI segmentation based on limited data sets with low quality and quantity. Rough annotations such as bounding boxes, scribbles, or points are commonly used to identify image-level labels rather than object-level labels. Though imperfect, these approximate annotations enable the model to identify unknown pixel-level regions by learning strong contextual correlations of input samples. Typically, the effort spent to generate these data sets is minimal compared to fully pixel-level segmentation, yet weakly supervised training could be used to extract key features from these data sets and produce a comparable performance to fully supervised models. Zhang et al. 213 propose a weakly supervised approach to perform WSI image segmentation to detect cancer regions using CNN and GCN networks. Graph nodes are formed by super-pixels generated from the CNN output features. The graph features and CNN features are then used as input to the GCN. Chen et al. 43 propose a weakly supervised framework using sparse point annotation to learn and classify hierarchical pixel and super-pixel features. To recognize the different context scales and semantic patterns, fine-tuned CNNs are employed to extract multi-level feature maps superimposed by super-pixel contours.

Nuclei detection and classification

Various histopathology assessment tasks rely on the quantification of cell nuclei of tissue samples for determining various biomarkers, identifying and localizing T-cells, quantifying tumor immune infiltrates, and other types of pathological analysis that are essential for cancer prognosis. 84 Visual estimation is labor-intensive, time-consuming, and prone to high inter- and intra-observer variability. Therefore, automatic detection and quantification of nuclei in WSIs increase assessment reliability and productivity. However, the similarity between different types of nuclei and variations of staining and tissue preparation conditions increase the complexity of nuclei automatic detection and classification. 169 Therefore, morphological features and automatic features are commonly used in the literature for nuclei detection and classification. 19 , 39 , 84

Morphological features of nuclei are used by Chankong et al. 39 to describe nuclei and cytoplasm for cervical cell classification. Five different classifiers are used to classify cells into normal, squamous cell carcinoma, as well as low- and high-grade squamous intraepithelial lesions. Color pre-possessing of WSI is found to improve the accuracy of nuclei detection. Aziz et al. 19 use a set of texture and morphological features for nuclei detection where color correction is applied to improve the accuracy.

The features extracted by the deep learning models provide a higher level of nuclei detection accuracy compared to traditional manual feature extraction methods. Detection of nuclei using deep learning features is conducted by Höfener et al. 84 by generating a map of points where each point corresponds to a single nucleus center. Two different CNN architectures are evaluated to learn image classification and regression. Post-processing of target maps of points is found to improve the accuracy significantly.

Classification of nuclei is essential for cancer detection and classification. Shi et al. 169 propose a relation-based feature extraction framework based on DenseNet and GCN features for cervical cell classification. DenseNet features are used to perform clustering of cell images which is used to create graph embedding features. Graph representation is used to train a GCN model to learn a set of relation features combined with DenseNet features.

Cancer detection and diagnosis

Medical radiology imaging materials such as MRI and CT scans are valuable for tumor region detection and patient medical condition evaluation. These technologies are also used for many other purposes, including cancer surveillance, screening, and planning of treatment. Physicians depend on these scans to identify tumor region shape and stage. Many studies are conducted to automatically detect tumor regions from CT and MRI scans. 27 , 119 WSIs, however, are becoming more popular among the medical community. They are increasingly used for cancer detection. Automated detection solutions based on WSIs are in demand, and research in this area is growing. Cancer diagnosis involves tissue slide inspection by pathologists, which is a complex task and requires highly qualified pathologists. Pathologists perform cancer diagnosis by examining the glandular structure of the specimen. In normal tissue, gland units are composed of circles of epithelial cells around the lumen. A fibromuscular tissue called stroma holds gland units together. In tumor tissue, epithelial cells duplicate in an unexpected non-systematic way causing irregular gland unit architecture. In the worst case, stroma virtually vanishes, and lumens are filled with epithelial cells. 182

The demand for automatic detection of tumor cells is rapidly increasing to reduce time, increase productivity, and improve the quality of the detection process by reducing false-negative rates. 198 Prostate cancer detection is conducted in Tabesh et al. 182 and Almuntashri et al. 13 by extracting several features from images, including fractal dimensions, color histograms, and wavelet decomposition. Other types of morphological features of objects in WSIs are also extracted by Tabesh et al. 182 using the MAGIC system. 186 Object-level features generated by morphological analysis produced higher accuracy. The detection of colorectal cancer is investigated by Chaddad et al. 37 The study uses a set of texture features extracted from 3D WSI volume using 3D wavelet transforms. It analyzes and compares different pathological tissues of colorectal cancer. Texture features are also used by Durgamahanthi et al. 58 and Vaishali et al. 190 , 191 for brain cancer diagnosis.

Extracting features from WSI images using deep learning models produces superior results in cancer diagnosis. Gao et al. 67 develop a CNN-GCN framework for breast cancer diagnosis. The proposed framework attains higher accuracy performance when compared with other approaches. Labeling of segmented regions of interest for cancer detection is generated using GCN by Ye et al. 206 where morphological features are generated from the segmented regions using a U-Net model. Region detection is also conducted by Faust et al. 63 to classify WSI images into 13 tissue and lesion classes. The patch-level approach is used to learn a VGG19-CNN model for the classification and detection of gliosarcoma and hemangioblastoma tumors. Probability scores are then used for region-based visualization of WSI. Wang et al. 197 address the problem by developing a deep CNN to divide the WSI image into 3 regions (tumor, normal, and background). They divide images into patches and transfer the gray-scale images into ternary images. A set of shape and boundary descriptors are then extracted from the detected tumor regions and used to predict patient survival outcomes. The study concluded that many features are associated with the survival outcome.

Cancer grading and survival prediction

Cancer treatment goes through different stages, during which several kinds of therapies can be given to patients. The detection of cancer stage and grade determines the most appropriate treatment: it also helps with identifying the severity of the disease and patient survival. 182 The cancer stage refers to the size and the spread of the tumor, whereas the grade is related to the appearance of the tumor cells. Histological screening and tests of cell appearance should be conducted to determine cancer grade. This process is manual, subjective, and depends on personal opinion and experience, resulting in wide variability in scoring. Accurate clinical results in these cases are required, and cancer diagnosis must consider heterogeneity between different tissues. To overcome these issues, solutions using image processing techniques and AI methods can be incorporated into the diagnosis pipeline. Several solutions have been developed to contribute to the automation of this process. Rathore et al. 149 develop a predictive modeling pipeline to perform cancer grading. A set of features describing phenotypic information within WSI images are used to reflect different histological characteristics such as mitotic activity, microvascular proliferation level, and abnormal appearance of cell nuclei. Cancer grading is achieved by predicting low- and high-grade gliomas (LGG and HGG). Gleason pattern grading is also conducted by Farooq et al. 62 by extracting texture features from images using the Gabor filter and LBP method. The study concludes that uniform rotation LBP invariant achieved the highest accuracy results.

The morphological architecture of the tumor region is also found useful in determining cancer grade. Morphological features are used by Lu et al. 124 to form cell clusters to construct local graphs. Features are then extracted from graphs and associated with other descriptive features to describe tumor region heterogeneity and determine cancer grade. Heterogeneity estimation is also employed by Hayward et al. 78 for ductal carcinoma in situ (DCIS) breast cancer detection and grading. The study considers processing ducts separately and in groups. Morphological features are extracted from nuclei within ducts and used to detect normal and DCIS ducts and score them.

Graph-based features and morphological features extracted from objects in segmented WSI images are used by Lopez et al. 122 to predict the biological behavior of prostate carcinoma and detect Gleason pattern grades. Grading of colorectal cancer and Gleason score is conducted by Zhou et al. 219 and Wang et al. 195 using GCN, where nodes represent cells and edges represent Euclidean distance. A set of shape and texture features are employed to describe nodes, where K-nearest neighbor is utilized to connect nodes in the graph. On the other hand, nodes are described by Chen et al. 41 using a ResNet-50 model, where WSI images are represented as a graph by dividing images into patches. Patch-level graph representation is fed into a GCN model for survival prediction of cancer patients. Deep learning features extracted using LeNet, AlexNet, and GoogleNet are utilized by del Toro et al. 189 for Gleason pattern grading. Features are extracted based on both gray-level intensity values and RGB channels.

Tumor region reconstruction

Tumor size and shape are 2 critical elements in cancer prognosis and treatment. These 2 elements can be estimated using automatic tumor region detection. In radiography images, malignant tumors in breast cancer can be identified by their rough boundary, which is also associated with lung cancer’s worst prognosis. 194 However, WSIs are privileged with higher spatial context and resolution, allowing for higher tumor region detection accuracy. Boundary roughness can as well be measured from WSIs. 61 The development of accurate tumor shape and boundary detection and identification using WSIs is a growing research field in digital pathology. WSIs are a rich source of information that can be used to characterize tumor region shape and boundary. Zhang et al. 212 develop a model-based Bayesian landmark detection approach to determine a set of landmarks setting on the boundary of the tumor region to characterize tumor shape and measure heterogeneity. Region of interest localization is conducted by Mercan et al. 138 using a set of color and texture features. The authors use the visual bag of words model to localize diagnostically important regions in breast cancer. Results show that basic image features such as color and texture could be enough to characterize tumor regions.

Tumor micro-environment quantification

Modulating tumor progression can be significantly influenced by the tumor micro-environment. For instance, the influence of the tumor on the resistance of the immune checkpoint molecules and its evasion ability can be clearly observed in the tumor micro-environment. Analyzing the tumor micro-environment promoted the discovery of immunotherapy which stimulates anti-tumor immunity. This discovery led to many treatments that have been found to be successful for cancer. These immunotherapies resulted in the overall survival of patients with advanced programmed death-ligand 1 (PD-L1) positive TNBC. Melanoma cancer treatment is also promoted by immune checkpoint blockade. Therefore, understanding the tumor micro-environment is a fundamental step to characterizing the complex relationship between pro-and anti-tumor immunity.

To examine the impact of the tumor micro-environment on the triple-negative breast cancer (TNBC) progression, Mi et al. 139 apply spatial analysis to quantify the spatial distribution of immune markers to measure tumor micro-environment heterogeneity. They found that intra- and inter-tumoral heterogeneities vary across different specimens, and the density of immune cells is higher in the invasive tumor front. The same conclusion is validated by Hayward et al., 78 who studied the heterogeneity in DCIS breast cancer. The study used a set of morphological features to describe nuclei in a duct-by-duct manner. They perform image segmentation to identify and label different ducts classified as normal and DCIS ducts, which are then classified into grade 1, grade 2, and grade 3 DCIS ducts. The Simpson’s diversity index 172 is used to estimate heterogeneity within each duct. They find that the higher the DCIS ducts grade, the higher the diversity index, where the lowest is 0.1 for grade 1 ducts and the highest is 0.6 for grade 3 ducts.

Dodington et al. 57 study the role of nuclei- and tissue-level texture and morphological features extracted from tiled WSIs on quantifying breast tumor response to neoadjuvant chemotherapy treatment. The study concludes that nuclear intensity and GLCM are strongly associated with response to neoadjuvant chemotherapy. Another study, conducted by Pezoa et al., 146 performs segmentation of Human Epidermal growth factor Receptor 2 (HER2) in immunohistochemistry-stained WSIs. HER2 is a typical biomarker observed on the cell membrane surface of various organs. In tumor tissue, this biomarker can be observed in high abnormal amounts leading to abnormal cellular growth. Analysis and quantification of HER2 overexpression are used to evaluate the progression of various cancers. The study concludes that color and texture features extracted from tiled images can be used for HER2 segmentation.

Glomerulosclerosis identification

WSI can best be used for the detection and characterization of glomeruli, which is a key procedure in many nephropathology studies. 30 Glomeruli are capillary clusters that are responsible for the disposal of waste and surplus fluids that are not needed by the human body. Several symptoms, etiology, immunopathology, and morphological changes can be exploited for the classification of Glomerular diseases. The morphological alterations can be used to describe glomerulosclerosis by presenting the glomerulus with different degrees of sclerosis. The quantification of the total glomeruli in renal biopsy provides an indication of the clinical state of the kidney. This process is tedious and cumbersome. Therefore, manual feature extraction methods 31 , 168 , 170 , 214 and deep learning techniques 30 are widely used for the quantification of glomeruli based on WSI analysis.

WSI volume reconstruction

For some clinical analyses, such as phenotyping the percentage of glomerulosclerosis, 2D images might not be enough to produce accurate quantitative assessment. 53 In tubular glomeruli, for example, 3D visualization of all WSI nephron sections is required to confirm pathological changes and quantification assessment. Manual and semi-automated approaches to perform 3D reconstruction and quantification are technically costly with low throughput and prone to errors. The need to develop automatic 3D reconstruction for WSI images is increasing to generate more precise and reproducible assessments. 53 These images are commonly used for multi-scale analysis of microscopic images and visualization of the different anatomical structures. 108

To obtain a 3D volume of organ tissue, a set of images scanned from stained slides of sliced sectioned chemically fixed tissue are stacked up and aligned. Aligning section slides are prone to errors due to inevitable tissue folding and tearing inconsistency resulting from the imperfect mounting of the tissue sections onto the glass slides. 77

Images can be aligned using intensity-based methods where intensity similarity is used to match corresponding pixels. Geometric-based methods can also be used to find matching points on the 2 images. To this end, Kugler et al. 108 propose a landmark-based method to map corresponding landmark locations between consecutive images using evaluation criteria based on the location and smoothness of landmark trajectories. They use template matching to detect landmarks. Trajectories of landmarks at folded or blurred portions are terminated to generate smoothed trajectories.

WSI images registration

In many cases, pathologists might need to view and align multiple scans of tissue simultaneously for evaluation and comparison purposes. They need to manually find and match similar regions, which is a time-consuming task, especially when comparing scans of different samples. 199 Image registration can be used to align multiple WSIs based on morphological similarity. 29 Yet, it is not a straightforward process due to the complexity of the WSIs emerging from large image sizes and different scanning parameters, such as staining, in addition to the artifacts generated by the scanning process. 174 Image alignment can be achieved using feature- and intensity-based techniques. The registration process aims to find a transformation operation to match the source or moving image with a fixed target single or multiple images. Different spatial transformation operations are conducted to map the source image to the target image. In feature-based matching, a set of shape features, such as points, lines, and contours, are extracted from both source and target images for alignment. Intensity-based registration approaches use correlation and spatial moments metrics based on entire images or sub-image. After features matching, geometrical transformation is then applied on the source image to the target image. 222

Both intensity- and feature-based methods can also be developed to perform the registration. Déniz et al. 55 develop an intensity-based multi-resolution registration approach using a similarity metric based on mutual information. The study uses a bilinear transformation and an evolutionary process as an optimizer. Registration of WSI is conducted by Awan and Rajpoot 18 using deep autoencoders by maximizing the mutual information between the source and target images. The use of the autoencoder reduces the dimensionality of image features by a factor of 16, enabling a faster and more tractable representation of images.

WSI image retrieval

The availability of many WSI repositories results in many challenges related to managing these types of images because of their large resolution and staining variations. These challenges make searching within image repositories for similar images for matching a complicated task because it requires more effective descriptive ways than those used for simple natural images with manageable resolution and contents. Matching and comparing images can be attained based on content-based image retrieval (CBIR) algorithms 34 that describe visual properties of images via feature vectors generated using different kinds of image analysis methods. Feature vectors can be matched using similarity matching algorithms such as nearest-neighbor matching.

Contents of images can be encoded using various methods, such as image texture, color histogram, Fourier and wavelet coefficients, etc., to generate feature vectors that can be employed to perform similarity matching. Fig. 4 depicts a typical workflow of image retrieval in WSI. Caicedo et al. 34 exploit a set of features obtained using color histogram, Tamura texture histogram, Sobel histogram, and LBP to provide a low-level representation of images. Histograms are then aggregated using a higher level of semantic features based on statistical functions such as the mean, deviation, skewness, kurtosis, etc.

Zheng et al. 216 use texture features, color histograms, Fourier transformation coefficients, and wavelet coefficients to generate a signature for each WSI image. Weighted cosine similarity is then used to measure the similarity between the query image and images in the database. The study also performs clustering based on image signatures to reduce the search space. The study found that images with similar visual properties form distinct groups, 2 of them representing cancer clusters.

Texture and morphological features are also used by Ma et al. 131 for image retrieval in breast cancer. Features are obtained based on 2 hierarchy levels where basic features are extracted using Gabor wavelet filters and nuclei region-based descriptors. The high-level semantic description is obtained using the latent Dirichlet allocation model through probabilistic representations of different hierarchical levels of data abstraction. Locality-sensitive hashing is then leveraged to generate hash coding for WSI matching. Signatures can also be used to query sub-images of WSI as by Mehta et al., 136 who utilize SIFT features to perform image indexing and sub-image query.

Deep learning features are expected to generate better image encoding and higher indexing accuracy. Kalra et al. 97 propose an intelligent indexing algorithm to provide easy and fast image retrieval based on similarity indexes. The study uses histograms from image mosaics to cluster similar patches based on color properties. Driven by clustered groups, patches are randomly sampled to represent the whole slide image, and barcodes are built using the DenseNet network. Barcodes are then converted using the MinMax algorithm 135 into lightweight vectors, which are then used to perform image retrieval and matching.

WSI visualization

Clinical analysis and diagnosis of tissue diseases involve careful examination and quantification of different types of nuclei, looking for unusual changes and abnormal patterns within biopsy, resection, exfoliation, or fluid tissues. Extracting this information from WSIs is useful for manual examination and clinical analysis. However, manual examination of WSIs remains a challenging task given the effort spent by pathologists to examine and estimate a quantitative approximation of the number of nuclei or the area of invasive inflammation or any other quantitative and qualitative measurements. To enable quick manual examination of WSIs, many WSI visualization solutions are proposed in the literature. 60 , 105 , 112 , 125 Visualization solutions mainly facilitate the estimation of the nuclear density of ROI in WSI images. These systems can identify different regions within WSI based on visual attributes and contextual analysis. Clustering techniques and similarity approximation solutions are generally used to group nuclei providing a higher-level representation of cancer or non-cancer regions.

Visualization of WSIs captures the underlying contextual shape and structure relationship between the cellular, regional, and hierarchical architecture of WSI image contents. These relationship descriptions involve a complex processing pipeline starting with image segmentation, cellular description, regional aggregation, representation, and visual analysis. Representation of relationships between elements of WSIs can be described using topological analysis, where images are divided into smaller components and connected based on visual attributes and contextual analysis. Fig. 5 shows a typical pipeline of WSI visualization. Levy et al. 112 propose a topological description of WSIs based on learning patch-level CNN features to construct a graph embedding representation which is then used to train a GNN model to generate patch-level embedding. A topological data analysis tool called Mapper is employed to build a higher-order graph embedding to represent the region of interest based on the similarity distance between patch-level embedding. The Mapper graph is established based on the relationship between different regions identified using patch-level similarity overlapping. Visualization based on dimensionality reduction is used by Faust et al. 63 to depict histomorphological information embedded in WSI images. CNN features embedding is learned by discretizing the relationships between different classes. The probability scores learned by CNN are used as input to t-distributed stochastic neighbor embedding (t-SNE) to learn new undefined classes in the data.

WSI stain normalization

Color variations in digital slides impact the accuracy of the deep learning algorithms negatively. The tissue slide preparation process involves biochemical staining, which produces different physical color variations based on differences in staining protocols and variability in the staining amount and time. 204 Other imaging and digitization parameters can also result in color variations. 91 Therefore, color normalization of digital slides is an important step in digital pathology applications for pre-processing WSIs. Pre-processing histopathological WSIs is critical for improving image quality before applying AI learning algorithms. Öztürk and Akdemir 129 demonstrated the importance of pre-processing WSIs by evaluating the impact of various pre-processing techniques on the classification output of a CNN architecture.

GAN-based techniques and approaches are state-of-the-art for color normalization of WSIs based on a style transfer method. The target image is generated by modifying the style of the input image using another input-style image while preserving the details of the input image. 142 GAN-based architecture is used by BenTaieb and Hamarneh 24 , Cho et al. 46 , and Zanjani et al. 211 to stain normalize WSI from different visual domains. CycleGAN model is also adopted by Zhou et al. 218 and Shaban et al. 163 for pre-processing WSIs before feeding them as input to another model for the classification of breast cancer.

Other applications

As new AI technologies emerge, the application of these technologies to WSIs analysis and feature extraction is rapidly increasing. Recent applications include the prediction of tissue type and gene mutation from pathology images. 95 , 148 Genomic analysis, such as high-throughput molecular profiling, identifies key drivers of genetic alterations connected to targeted therapy of different types of cancers. The prediction of gene expression is also conducted by Schmauch et al. 161 using a deep learning framework improving the prediction of a specific molecular phenotype, such as microsatellite instability.

The grading of interstitial fibrosis and tubular atrophy based on WSIs processed from human renal biopsies is widely conducted using deep learning techniques. 217 It is an important prognostic predictor primarily relying on visual assessment and semi-quantitative grading, which may not capture finer details or heterogeneity throughout the entire slide. 75 Therefore, many studies propose leveraging advances in digital pathology and developing modern data analytic technologies, such as deep learning, for comprehensive image analysis of kidney pathology. 68

The detection of mitotic events is critical in many biomedical activities, including biological research and medical diagnostics. Cell division is a key stage in the cell’s life cycle, and mitosis detection can reveal valuable information about cell activities. The proliferation of different cancers is one of the most important prognostic markers that is typically evaluated by counting mitotic figures in WSIs. However, mitotic counting is time-consuming, subjective, and prone to high inter-observer variations, which enables a new research domain for automatic analysis using computer vision technologies. 126 , 185 , 193

Discussion

The rapid development and success of computational pathology is, among other factors, fueled by feature engineering and deep learning technologies. These form the technical foundation for the development of various solutions seeking to automate whole slide image analysis. Recent advances in the development of deep learning techniques and solutions in particular opened numerous new research opportunities and challenges. In the following, we discuss the challenges we deem most important.

Manual annotation and validation: The grand challenges of our data-driven age are no stranger to computational histopathology. While AI offers unique opportunities to turn data into insights and, thus, to provide efficient decision-support systems to specialists, AI requires extensive human effort for data cleaning, annotation, and labeling. In a sense, supervised AI is still parasitic, feeding on human labor. This, in turn, limits the amount of data suitable for training and prohibits supervised AI from unlocking its full potential. While the first frameworks for supervised-to-unsupervised 200 have demonstrated some success, they still rely on large quantities of labeled data. We believe that there is a dire need for tools to support this transition from supervised to semi-supervised, weakly supervised, or unsupervised learning. In our survey, we presented multiple solutions developed based on these approaches to tackle this issue. 43 , 56 , 103 , 130 , 195 , 213 Whereas the annotation process can be crowd-sourced for many data sets, this is not the case for histopathology. We, therefore, see an acute demand for visual tools supporting histopathology workflows from annotation to computation. Such tools can come in the form of better tools for annotation, improving on existing solutions such as NuClick 104 , 92 with advanced user interfaces and modern input devices such as digital pens. In the ideal case, such tools integrate elements of online learning, in which a model suggests labels and positions to histopathologists whose proofreading then triggers a continual training process of the network. Recent efforts 72 go towards the direction of developing human–AI collaborative tools mimicking the examination process carried out by pathologists, in order to improve AI’s integration into routine examinations. In this context, collaborative digital histopathology platforms 93 could also gain importance, as they provide the means for effective parallelization of the workload across multiple domain experts. We believe that the development of such tools will significantly boost the amount of data available for modeling in the public domain.

Image super-resolution: Challenges more specific to histopathology include compensating for ill-sliced nuclei in the tissue glass slide preparation process, better alignment/registration algorithms for scans of deformed tissue glass slides, and, potentially, further increases in the image resolution. As an example, transmission electron microscopy (TEM) provides a much higher resolution than optical microscopy, lowering the learning curve in annotating nuclei for non-experts and potentially opening the door to crowd-sourced annotations. However, the preparation process for TEM is much more involved, including, among others, expert selection of individual nuclei, and is, thus, unlikely to produce data at the same rate. Due to the difference in WSI preparation, it is not possible at the time of writing to image the same sample using optical and electron microscopes. As a result, improvements in image resolution must either come from super-resolution techniques or from the optical domain. This article also shows that most data sets and applications focus on cancer research. While an important and high-impact topic, data sets suitable for training models for other applications, e.g., pediatrics (in which inflammatory cells are of high importance), are scarce, and adequate tools to generate data annotations at scale could alleviate this problem. Deep learning architectures have shown superior image super-resolution in several fields and they represent a good solution for this problem. The earliest work to explore this research direction is conducted by Mukherjee et al. 143 who developed a new variant of RNNs based on a sequence of CNNs to address this issue. Li et al. 113 developed a deep learning-based approach for WSIs super-resolution using CNN and self-supervised color normalization. An unsupervised deep learning approach is also proposed by Ma et al. 130 using hyperspectral images. Further research is needed to address this challenge.

Continual learning and integration: Another challenge relates to continual learning and integration, which refers to the ability of a model to incorporate multiple sources of information and make use of prior knowledge. Histopathology images often contain multiple types of tissue and multiple regions of interest, and incorporating this information in a deep learning model can be difficult. Additionally, histopathology images are often accompanied by other types of data, such as patient demographics and clinical information, which can be used to improve diagnostic accuracy. Incorporating these additional sources of information into deep learning models is expected to boost classification and detection accuracy, but it requires additional engineering and development efforts. The research in this field is in the early stages. 21 , 99 , 100

Uncertainty quantification: Another research direction that is gaining more traction in the machine-learning community is uncertainty quantification. Although a model’s output is typically interpreted as “hard” labels, models usually generate fuzzy classifications and segmentations (i.e., probability distributions). The information lost in the process of converting these probabilities into a hard decision is valuable to experts, and methods such as conformal predictions 14 have already been applied to cancer detection. 110 The resulting labels, be they point annotations or segmentations, further require novel visualization techniques as the resulting data is nominal and no longer continuous. First advances in this direction have been made, 11 , 12 but more research into the effective visualization of nominal annotation at scale is needed.

Size of WSIs: WSIs are huge and complex. Developing effective solutions for extracting features meaningfully supporting the diagnostic process from these images is one of the most challenging tasks in digital pathology. The development of deep learning models that accurately represent information embedded in these images is evolving rapidly. The demand to provide better solutions is increasing, leading to potential research challenges and opportunities. The size of WSIs is one of the challenges that most image analysis techniques in AI have to overcome in order to extract useful information from these images. Therefore, a sliding patch strategy is widely used to deal with the enormous size of the WSIs. It is usual in the pathology area to have 2 classification steps: one for fine-grained patch-level classification and another, coarse-grained, for slide-level classification, using the patch-based classification as an input parameter. 153 Moreover, fine-grained manual annotations are exceedingly expensive and time-consuming due to their large size. Thus, patch-wise target labels are usually not available, and only the labels of each WSI are known, making traditional supervised learning inaccessible. To tackle this problem, some studies adopt multiple-instance learning approaches based on unsupervised and weakly supervised learning. Each WSI is regarded as a bag containing several patches (instances). If a WSI (bag) is designated positive, it contains at least 1 positive instance. If a WSI is negative, all patches in the bag are negative. 215 Several approaches in the literature are proposed to reduce the complexity of the multiple-instance learning approach using intelligent sampling and grouping as proposed by Su et al. 177 However, research aimed to address this issue is needed to reduce the processing time and labeling effort of WSIs.

Data quality: The clinical analysis of WSI images requires identifying and quantifying various nuclei, glandular structures, and anatomical regions with varying visual characteristics. The diversity of human tissue structure and topology is reflected by wide variations of WSIs in color and texture. Generating effective solutions for WSIs analysis requires handling this diversity by taking into consideration a broad spectrum of input data. Moreover, the accurate annotation and labeling of these images is another challenge since the clinical prognosis of each case varies depending on the annotator’s personal experience and perspective. Therefore, the availability of sufficient input data used to train deep learning models and ensuring the quality and consistency of this data is another problem in computational pathology. Recently, several data sets have been released to fulfill these requirements, 7 , 66 , 87 , 137 , 158 yet the enormous amount of available unlabeled data and the huge number of diseases and diagnosis procedures increase the need for clean labeled input data for developing efficient and accurate algorithms.

Transfer learning and model generalization: Transfer learning is an exciting field of study in computational pathology because it allows models trained on one type of tissue or disease to be adapted for use on other types of tissues or diseases. This will reduce the time and effort required to train deep learning models as it will allow for reusing previous knowledge stored in the form of network weights. However, transferring knowledge in the form of trained deep learning models requires further improvement and testing. Much work remains to be done in developing models that can effectively generalize across various data sources and domains. In our survey, we conducted a thorough review of various studies that have employed transfer learning techniques for WSI analysis (c.f. Section 4.5). Through this analysis, we observed that transfer learning has shown promising results in leveraging pre-trained models to extract data that support the diagnostic process from WSIs. However, it is worth noting that the field of deep learning is dynamic, and there have been recent advances in model architectures and learning strategies that offer enhanced feature learning and analysis capabilities for WSIs. For instance, models like adversarial-, graph-, and transformer-based networks have demonstrated state-of-the-art performance in various computer vision tasks, including image classification and segmentation. To fully benefit from these advances, it is crucial to investigate how these novel models can be adapted and fine-tuned for WSI analysis. This involves exploring strategies to transfer the knowledge and expertise acquired by these models on diverse image data sets to the specific domain of digital pathology. By leveraging the pre-trained weights and architectures of these advanced models, research can potentially improve the accuracy, efficiency, and generalization capabilities of WSI analysis tasks.

Integration with clinical workflows: Finally, to ensure that computational pathology tools and models are used successfully in practice, they must be integrated with clinical workflows. This involves creating user-friendly interfaces, connecting to electronic medical records, and addressing privacy and data security concerns. Despite the unprecedented advances in the development of AI medical solutions achieved by clinical organizations, there is still a gap between these automated systems and physicians. One of the primary causes of this disparity is the lack of trust among medical experts in this technology. This lack of trust may result from the level of uncertainty introduced by this technology, especially for challenging tasks. Although some sophisticated AI technologies, such as deep learning, provide a high level of accuracy, it lacks the ability to provide an explanation or justification to warrant decisions made by this technology, representing another source of doubt and uncertainty. Providing explicable accurate solutions that can be easily integrated with clinical workflows is another challenge in research and practice.

Overall, the field of computational pathology based on whole slide images and deep learning is quickly evolving, with numerous intriguing research opportunities and challenges. By addressing these issues, researchers and clinicians will be able to create more accurate and effective tools for diagnosing and treating a wide variety of diseases.

Pathologists’ collaboration and expertise in computational pathology present a vibrant opportunity for further research. Their insights into challenges and potential solutions contribute to the development of AI-based models that have a deep impact on related disciplines, such as automated classification, segmentation, and visualization. Expert involvement ensures that the research aligns with the requirements of pathological practice and benefits various applications beyond cancer research, such as pediatrics and other specialized areas. However, the role of AI is that of a diagnostic tool supporting pathology work flows, and does not provide means to replace pathologists. The role of pathologists in the development of AI and deep learning solutions may thus include the following:

Data cleaning, annotation, and labeling: The development of AI applications mostly relays on the availability of sufficient and clean input data. Pathologists contribute their expertise in data cleaning and annotation, ensuring the quality and accuracy of the data sets used for AI analysis. They provide valuable insights into identifying and quantifying various nuclei, glandular structures, and anatomical regions with varying visual characteristics. Pathologists’ experience and knowledge help in standardizing and making annotations consistent, addressing the challenges of diverse tissue structures and topology.

Transition to semi-supervised and unsupervised learning: The transition from supervised to semi-supervised, weakly supervised, or unsupervised learning offers a huge potential to reduce the amount of target labels for learning. However, this transition process requires extensive evaluation and proofreading. Pathologists contribute to this stage of the deep learning pipeline by evaluating and validating the output of these modeling techniques. In many cases, pathologists might need to apply manual corrections for annotations and output generated by these techniques. Additionally, pathologists’ insights can facilitate the development of tools that integrate elements of online learning, allowing AI models to incorporate knowledge inherited from different sources. This expertise is critical for proofreading and triggering continual training processes.

Algorithm development and training: Pathologists collaborate with AI researchers and data scientists to develop algorithms for feature extraction and engineering. They offer insights into the intricate nuances of pathological findings. For instance, in breast cancer diagnosis, pathologists assist in designing algorithms that can detect subtle patterns or classify different subtypes of tumors based on tissue histological characteristics. Pathologists may also identify relevant morphological features, such as cellular patterns or architectural abnormalities, which are incorporated into AI models to aid in accurate diagnosis and prognosis.

Validation and evaluation: Validating the output of deep learning and AI applications is one of the most critical phases in the development life cycle of these applications. The results of this phase might involve the approval of using these applications in real-life scenarios. Pathologists play a crucial role in validating and evaluating the performance of AI models. They compare AI-generated results with their own interpretations or consensus diagnoses to assess accuracy and reliability. In the field of dermatopathology, for example, pathologists may evaluate AI algorithms for skin lesion classification by comparing their predictions with histopathological analysis of biopsy specimens.

Clinical integration and workflow: Pathologists work closely with AI developers to integrate AI tools into their clinical workflow effectively. For instance, they provide feedback on user interfaces, workflow integration, and interpretability of AI-generated outputs. Pathologists may collaborate with AI engineers to design user-friendly interfaces that allow seamless integration of AI algorithms into existing pathology systems. Furthermore, pathologists may take the lead in conducting clinical validation studies for AI models.

By actively engaging pathologists in these roles, AI applications can be tailored to meet the specific needs and challenges of pathology practice. The expertise and input of pathologists ensure that AI models are accurate, reliable, and ethically implemented, thereby enhancing diagnostic accuracy, improving patient outcomes, and streamlining clinical decision-making processes.

Conclusion

In this survey, we have reviewed state-of-the-art machine learning approaches to automate parts of pathology workflows, for use by histopathologists, lab technicians, and practitioners and researchers in machine learning and AI.

WSI has improved clinical practices by facilitating the management and analysis of scans of tissue glass slides. Researchers have proposed numerous AI solutions to automate image analysis and extract diagnosis information from WSIs. However, challenges include the need for human effort in data cleaning and annotation, the high resolution and size of WSI images, integration of clinical data into AI models, reducing complexity in multiple-instance learning, and ensuring data quality and availability. Transfer learning techniques have shown promise in leveraging pre-trained models for WSI analysis. More research is needed to explore the impact of several deep learning approaches, such as semi-supervised learning, weakly supervised learning, continual learning, etc, on the analysis of WSIs.

We therefore believe that teams of histopathologists and AI researchers are needed to drive the development in AI in order to ensure accurate and reliable models that indeed become clinical tools enhancing diagnostic accuracy. While the field of computational pathology offers plenty of research opportunities to improve automated classification, segmentation, and visualization, the role of pathologists, both as early adopters as well as in the validation phase (e.g., leading clinical studies) is of utmost importance. We hope that this survey will spark many such collaborations between pathologists and engineers, ultimately leading to tools for better disease diagnosis and treatment.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

1. Abels E., Pantanowitz L., Aeffner F., et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the digital pathology association. J Pathol. 2019;249:286–294. doi: 10.1002/path.5331.

2. Achanta R., Shaji A., Smith K., Lucchi A., Fua P., Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell. 2012;34:2274–2282. doi: 10.1109/TPAMI.2012.120.

3. Adiga U., Malladi R., Fernandez-Gonzalez R., de Solorzano C. High-throughput analysis of multispectral images of breast cancer tissue. IEEE Trans Image Process. 2006;15:2259–2268. doi: 10.1109/TIP.2006.875205.

4. Agus M., Al-Thelaya K., Calí C., et al. Eurographics Workshop on Visual Computing for Biology and Medicine. The Eurographics Association; 2020. InShaDe: invariant shape descriptors for visual analysis of histology 2D cellular and nuclear shapes; pp. 61–70.

5. Ahmedt-Aristizabal D., Armin M.A., Denman S., Fookes C., Petersson L. A survey on graph-based deep learning for computational histopathology. Comput Med Imaging Graphics. 2021:102027. doi: 10.1016/j.compmedimag.2021.102027. https://www.sciencedirect.com/science/article/pii/S0895611121001762

6. Ahn S.W., Ferland B., Jonas O.H., et al. An interactive pipeline for quantitative histopathological analysis of spatially defined drug effects in tumors. J Pathol Inform. 2021;12:34. doi: 10.4103/jpi.jpi_17_21.

7. Aksac A., Demetrick D.J., Ozyer T., Alhajj R. Brecahad: a dataset for breast cancer histopathological annotation and diagnosis. BMC Res Notes. 2019;12:1–3. doi: 10.1186/s13104-019-4121-7.

8. Al-Janabi S., Huisman A., van Diest P.J. Digital pathology: current status and future perspectives. Histopathology. 2012;61:1–9. doi: 10.1111/j.1365-2559.2011.03814.x.

9. Al-Kadi O.S. Texture measures combination for improved meningioma classification of histopathological images. Pattern Recogn. 2010;43:2043–2053. doi: 10.1016/j.patcog.2010.01.005.

10. Al-Thelaya K., Agus M., Gilal N.U., et al. InShaDe: invariant shape descriptors for visual 2D and 3D cellular and nuclear shape analysis and classification. Comput Graph. 2021;98:105–125. doi: 10.1016/j.cag.2021.04.037.

11. Al-Thelaya K., Agus M., Schneider J. The mixture graph-a data structure for compressing, rendering, and querying segmentation histograms. IEEE Trans Visual Comput Graphics (TVCG) 2021;27:645–655. doi: 10.1109/TVCG.2020.3030451.

12. Al-Thelaya K., Joad F., Gilal N., et al. Eurographics Workshop on Visual Computing for Biology and Medicine (VCBM) 2022. Histocontours: a framework for visual annotation of histopathology whole slide images; pp. 99–109.

13. Almuntashri A., Agaian S., Thompson I., Rabah D., Al-Abdin O.Z., Nicolas M. IEEE International Conference on Systems, Man, and Cybernetics. IEEE; 2011. Gleason grade-based automatic classification of prostate cancer pathological images; pp. 2696–2701.

14. Angelopoulos A., Bates S. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv:2107.07511. 2021 doi: 10.48550/arXiv.2107.07511.

15. Arevalo J., Cruz-Roa A., González O., F.A Histopathology image representation for automatic analysis: a state-of-the-art review. Rev Med. 2014;22:79–91. http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0121-52562014000200009

16. Arvaniti E., Fricker K.S., Moret M., et al. Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci Rep. 2018;8:1–11. doi: 10.1038/s41598-018-30535-1.

17. Aurenhammer F. Voronoi diagrams – a survey of a fundamental geometric data structure. ACM Comput Surv. 1991;23:345–405. doi: 10.1145/116873.116880.

18. Awan R., Rajpoot N. Annual Conference on Medical Image Understanding and Analysis. Springer; 2018. Deep autoencoder features for registration of histology images; pp. 371–378.

19. Aziz M.A., Nakamura T., Yamaguchi M., et al. Effectiveness of color correction on the quantitative analysis of histopathological images acquired by different whole-slide scanners. Artif Life Robot. 2019;24:28–37. doi: 10.1007/s10015-018-0451-0.

20. Baldi P. ICML Workshop on Unsupervised and Transfer Learning, JMLR Workshop and Conference Proceedings. 2012. Autoencoders, unsupervised learning, and deep architectures; pp. 37–49. https://proceedings.mlr.press/v27/baldi12a.html

21. Bándi P., Balkenhol M., van Dijk M., et al. Continual learning strategies for cancer-independent detection of lymph node metastases. Med Image Anal. 2023;85 doi: 10.1016/j.media.2023.102755.

22. Baxi V., Edwards R., Montalto M., Saha S. Digital pathology and artificial intelligence in translational medicine and clinical practice. Modern Pathol. 2022;35:23–32. doi: 10.1038/s41379-021-00919-2. https://www.nature.com/articles/s41379-021-00919-2

23. Bejnordi B.E., Balkenhol M., Litjens G., et al. Automated detection of DCIS in whole-slide H&E stained breast histopathology images. IEEE Trans Med Imaging. 2016;35:2141–2150. doi: 10.1109/tmi.2016.2550620.

24. BenTaieb A., Hamarneh G. Adversarial stain transfer for histopathology image analysis. IEEE Trans Med Imaging. 2017;37:792–802. doi: 10.1109/TMI.2017.2781228.

25. Bera K., Schalper K.A., Rimm D.L., Velcheti V., Madabhushi A. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nat Rev Clin Oncol. 2019;16:703–715. doi: 10.1038/s41571-019-0252-y.

26. Bharati M.H., Liu J., MacGregor J.F. Image texture analysis: methods and comparisons. Chemom Intell Lab Syst. 2004;72:57–71. doi: 10.1016/j.chemolab.2004.02.005.

27. Bi W.L., Hosny A., Schabath M.B., et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin. 2019;69:127–157. doi: 10.3322/caac.21552.

28. Blei D.M., Ng A.Y., Jordan M.I. Latent Dirichlet allocation. J Mach Learn Res. 2003;3:993–1022. https://jmlr.csail.mit.edu/papers/v3/blei03a.html

29. Borovec J. BIRL: benchmark on image registration methods with landmark validation. arXiv:1912.13452v2. 2019 doi: 10.48550/arXiv.1912.13452.

30. Bueno G., Fernandez-Carrobles M.M., Gonzalez-Lopez L., Deniz O. Glomerulosclerosis identification in whole slide images using semantic segmentation. Comput Methods Prog Biomed. 2020;184 doi: 10.1016/j.cmpb.2019.105273. https://www.sciencedirect.com/science/article/pii/S0169260719311381

31. Bukowy J.D., Dayton A., Cloutier D., et al. Region-based convolutional neural nets for localization of glomeruli in trichrome-stained whole kidney sections. J Am Soc Nephrol. 2018;29:2081–2088. doi: 10.1681/asn.2017111210.

32. Bychkov D., Linder N., Turkki R., et al. Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci Rep. 2018;8:#3395. doi: 10.1038/s41598-018-21758-3.

33. Cai S., Xue Y., Gao Q., et al. International Workshop on Machine Learning for Medical Image Reconstruction. Springer; 2019. Stain style transfer using transitive adversarial networks; pp. 163–172.

34. Caicedo J.C., Gonzalez F.A., Romero E. Asia Information Retrieval Symposium. Springer; 2008. A semantic content-based retrieval method for histopathology images; pp. 51–60.

35. Campanella G., Hanna M.G., Geneslaw L., et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. 2019;25:1301–1309. doi: 10.1038/s41591-019-0508-1.

36. Çelik Y., Karabatak M. Extracting low dimensional representations from large size whole slide images using deep convolutional autoencoders. Expert Syst. 2021;#e12819 doi: 10.1111/exsy.12819.

37. Chaddad A., Daniel P., Niazi T. Radiomics evaluation of histological heterogeneity using multiscale textures derived from 3D wavelet transformation of multispectral images. Front Oncol. 2018;8:96. doi: 10.3389/fonc.2018.00096.

38. Chaddad A., Tanougast C. Texture analysis of abnormal cell images for predicting the continuum of colorectal cancer. Anal Cell Pathol. 2017;2017 doi: 10.1155/2017/8428102.

39. Chankong T., Theera-Umpon N., Auephanwiriyakul S. Automatic cervical cell segmentation and classification in Pap smears. Comput Methods Prog Biomed. 2014;113:539–556. doi: 10.1016/j.cmpb.2013.12.012.

40. Chen H., Wang K., Zhu Y., et al. Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VIII 24. Springer; 2021. From pixel to whole slide: Automatic detection of microvascular invasion in hepatocellular carcinoma on histopathological image via cascaded networks; pp. 196–205.

41. Chen R.J., Lu M.Y., Shaban M., et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2021. Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolutional networks; pp. 339–349.

42. Chen R.J., Lu M.Y., Weng W.H., et al. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2021. Multimodal co-attention transformer for survival prediction in gigapixel whole slide images; pp. 4015–4025.

43. Chen Z., Chen Z., Liu J., et al. Weakly supervised histopathology image segmentation with sparse point annotations. IEEE J Biomed Health Inform. 2020;25:1673–1685. doi: 10.1109/JBHI.2020.3024262.

44. Cheng J., Mo X., Wang X., Parwani A., Feng Q., Huang K. Identification of topological features in renal tumor microenvironment associated with patient survival. Bioinformatics. 2018;34:1024–1030. doi: 10.1093/bioinformatics/btx723.

45. Cheng S., Liu S., Yu J., et al. Robust whole slide image analysis for cervical cancer screening using deep learning. Nat Commun. 2021;12:1–10. doi: 10.1038/s41467-021-25296-x.

46. Cho H., Lim S., Choi G., Min H. Asian Conference on Machine Learning (ACML) Workshop on MLAIP. 2017. Neural stain-style transfer learning using gan for histopathological images. https://arxiv.org/abs/1710.08543

47. Cho K., van Merrienboer B., Bahdanau D., Bengio Y. On the properties of neural machine translation: encoder-decoder approaches. arXiv:1409.1259v2. 2014 doi: 10.48550/arXiv.1409.1259.

48. Colling R., Pitman H., Oien K., et al. Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice. J Pathol. 2019;249:143–150. doi: 10.1002/path.5310.

49. Cross S.S., Cotton D.W. The fractal dimension may be a useful morphometric discriminant in histopathology. J Pathol. 1992;166:409–411. doi: 10.1002/path.1711660414.

50. Cruz-Roa A., Basavanhally A., González F., et al. Medical Imaging 2014: Digital Pathology. SPIE; 2014. Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks; p. #904103.

51. Delaunay B. Sur la sphère vide. A la mémoire de Georges Voronoï. Bulletin de l’Académie des Sciences de l’IRSS, Classe des Sciences Matheématiques et Naturelles. 1934;6:793–800. https://www.mathnet.ru/php/archive.phtml?wshow=paper&jrnid=im&paperid=4937&option_lang=eng

52. Demir C., Gultekin S.H., Yener B. Augmented cell-graphs for automated cancer diagnosis. Bioinformatics. 2005;21:ii7–ii12. doi: 10.1093/bioinformatics/bti1100.

53. Deng R., Yang H., Jha A., et al. Map3D: registration based multi-object tracking on 3D serial whole slide images. IEEE Trans Med Imaging. 2021;40:1924–1933. doi: 10.1109/TMI.2021.3069154.

54. Deng S., Zhang X., Yan W., et al. Deep learning in digital pathology image analysis: a survey. Front Med. 2020;14:470–487. doi: 10.1007/s11684-020-0782-9.

55. Déniz O., Toomey D., Conway C., Bueno G. In: Medical Imaging 2015: Digital Pathology. Gurcan M.N., Madabhushi A., editors. SPIE; 2015. Multi-stained whole slide image alignment in digital pathology; pp. 240–245.

56. Dimitriou N., Arandjelovic O., Caie P.D. Deep learning for whole slide image Dimitriou analysis: an overview. Front Med. 2019;6:264. doi: 10.3389/fmed.2019.00264.

57. Dodington D.W., Lagree A., Tabbarah S., et al. Analysis of tumor nuclear features using artificial intelligence to predict response to neoadjuvant chemotherapy in high-risk breast cancer patients. Breast Cancer Res Treat. 2021;186:379–389. doi: 10.1007/s10549-020-06093-4.

58. Durgamahanthi V., Anita Christaline J., Shirly Edward A. International Conference on Intelligent Computing and Applications. Springer; 2021. GLCM and GLRLM based texture analysis: application to brain cancer diagnosis using histopathology images; pp. 691–706.

59. Durgamahanthi V., Rangaswami R., Gomathy C., Victor A.C.J. Texture analysis using wavelet-based multiresolution autoregressive model: application to brain cancer histopathology. J Med Imaging Health Inform. 2017;7:1188–1195. doi: 10.1166/jmihi.2017.2255.

60. Falk M., Ynnerman A., Treanor D., Lundström C. Interactive visualization of 3D histopathology in native resolution. IEEE Trans Vis Comput Graph. 2018;25:1008–1017. doi: 10.1109/tvcg.2018.2864816.

61. Farahani N., Parwani A.V., Pantanowitz L. Whole slide imaging in pathology: advantages, limitations, and emerging perspectives. Pathol Lab Med Int. 2015;7:23–33. doi: 10.2147/PLMI.S59826.

62. Farooq M.T., Shaukat A., Akram U., Waqas O., Ahmad M. International Conference on Telecommunications and Signal Processing. IEEE; 2017. Automatic Gleason grading of prostate cancer using Gabor filter and local binary patterns; pp. 642–645.

63. Faust K., Xie Q., Han D., et al. Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction. BMC Bioinform. 2018;19:1–15. doi: 10.1186/s12859-018-2184-4.

64. Fraggetta F., Garozzo S., Zannoni G.F., Pantanowitz L., Rossi E.D. Routine digital pathology workflow: the Catania experience. J Pathol Inform. 2017;8 doi: 10.4103/jpi.jpi_58_17.

65. Gabor D. Theory of communication. Part 1: the analysis of information. J Inst Elect Eng Part III Radio Commun Eng. 1946;93:429–441. doi: 10.1049/ji-3-2.1946.0074.

66. Gamper J., Alemi Koohbanani N., Benet K., Khuram A., Rajpoot N. Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, Proceedings 15. Springer; 2019. Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification; pp. 11–19.

67. Gao Z., Lu Z., Wang J., Ying S., Shi J. A convolutional neural network and graph convolutional network based framework for classification of breast histopathological images. IEEE J Biomed Health Inform. 2022;26:3163–3173. doi: 10.1109/jbhi.2022.3153671.

68. Ginley B., Jen K.Y., Han S.S., et al. Automated computational detection of interstitial fibrosis, tubular atrophy, and glomerulosclerosis. J Am Soc Nephrol. 2021;32:837. doi: 10.1681/ASN.2020050652.

69. Gleason D.F. In: Urologic Pathology: The Prostate. Tannenbaum M., editor. Lea and Febiger; Philadelphia: 1977. The veteran’s administration cooperative urologic research group: Histologic grading and clinical staging of prostatic carcinoma; pp. 171–198. ISBN 0-8121-0546-X.

70. Gonzalez R.C., Woods R.E. 4th ed. Pearson; 2017. Digital Image Processing. https://www.pearson.com/en-us/subject-catalog/p/digital-image-processing/P200000003224/9780137848560

71. Goyal P., Ferrara E. Graph embedding techniques, applications, and performance: a survey. Knowl-Based Syst. 2018;151:78–94. doi: 10.1016/j.knosys.2018.03.022.

72. Gu H., Liang Y., Xu Y., et al. Improving workflow integration with XPath: design and evaluation of a human-AI diagnosis system in pathology. ACM Trans Comput Hum Interact. 2022 doi: 10.1145/3577011.

73. Gupta L., Klinkhammer B.M., Boor P., Merhof D., Gadermayr M. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. Gan-based image enrichment in digital pathology boosts segmentation accuracy; pp. 631–639.

74. Gurcan M.N., Boucheron L.E., Can A., Madabhushi A., Rajpoot N.M., Yener B. Histopathological image analysis: a review. IEEE Rev Biomed Eng. 2009;2:147–171. doi: 10.1109/RBME.2009.2034865.

75. Hadi A.M., Mouchaers K.T., Schalij I., et al. Rapid quantification of myocardial fibrosis: a new macro-based automated analysis. Cell Oncol. 2011;34:343–354. doi: 10.1007/s13402-011-0035-7.

76. Hamilton W., Ying Z., Leskovec J. Advances in Neural Information Processing Systems. 2017. Inductive representation learning on large graphs; pp. 1025–1035. https://papers.nips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html

77. Hashimoto N., Bautista P.A., Haneishi H., Snuderl M., Yagi Y. Development of a 2D image reconstruction and viewing system for histological images from multiple tissue blocks: towards high-resolution whole-organ 3D histological images. Pathobiology. 2016;83:127–139. doi: 10.1159/000443278.

78. Hayward M.K., Jones J.L., Hall A., et al. Derivation of a nuclear heterogeneity image index to grade DCIS. Comput Struct Biotechnol J. 2020;18:4063–4070. doi: 10.1016/j.csbj.2020.11.040.

79. He K., Zhang X., Ren S., Sun J. IEEE Conference on Computer Vision and Pattern Recognition. 2016. Deep residual learning for image recognition; pp. 770–778.

80. He S., Ruan J., Long Y., et al. International Symposium on Computational Intelligence and Design. 2018. Combining deep learning with traditional features for classification and segmentation of pathological images of breast cancer; pp. 3–6.

81. Henry E.U., Emebob O., Omonhinmin C.A. Vision transformers in medical imaging: a review. arXiv preprint arXiv:2211.10043. 2022 doi: 10.48550/arXiv.2211.10043.

82. Hinton G.E., Srivastava N., Krizhevsky A., Salakhutdinov R.R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580v1. 2012 doi: 10.48550/arXiv.1207.0580.

83. Hochreiter S., Schmidhuber J. Long short-term memory. Neural Comput. 1987;9:1735–1780. doi: 10.1162/neco.1997.9.8.1735.

84. Höfener H., Homeyer A., Weiss N., Molin J., Lundström C.F., Hahn H.K. Deep learning nuclei detection: a simple approach can deliver state-of-the-art results. Comput Med Imaging Graph. 2018;70:43–52. doi: 10.1016/j.compmedimag.2018.08.010.

85. Howard A.G., Zhu M., Chen B., et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861v1. 2017 doi: 10.48550/arXiv.1704.04861.

86. Hu M.K. Visual pattern recognition by moment invariants. IRE Trans Inform Theory. 1962;8:179–187. doi: 10.1109/TIT.1962.1057692.

87. Hu W., Li C., Li X., et al. Gashissdb: a new gastric histopathology image dataset for computer aided diagnosis of gastric cancer. Comput Biol Med. 2022;142 doi: 10.1016/j.compbiomed.2021.105207. https://www.sciencedirect.com/science/article/pii/S0010482521010015

88. Huang G., Liu Z., van der Maaten L., Weinberger K.Q. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017. Densely connected convolutional networks.

89. Iandola F., Moskewicz M., Karayev S., Girshick R., Darrell T., Keutzer K. DenseNet: implementing efficient convnet descriptor pyramids. arXiv:1404.1869v1. 2014 doi: 10.48550/arXiv.1404.1869.

90. Iizuka O., Kanavati F., Kato K., Rambeau M., Arihiro K., Tsuneki M. Deep learning models for histopathological classification of gastric and colonic epithelial tumours. Sci Rep. 2020;10:1–11. doi: 10.1038/s41598-020-58467-9.

91. Inoue T., Yagi Y. Color standardization and optimization in whole slide imaging. Clin Diagnos Pathol. 2020;4 doi: 10.1186/1746-1596-6-S1-S15.

92. Jahanifar M., Tajeddin N.Z., Koohbanani N.A., Rajpoot N.M. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops. 2021. Robust interactive semantic segmentation of pathology images with minimal user input; pp. 674–683.

93. Jeong W.K., Schneider J., Hansen A., et al. A collaborative digital pathology system for multi-touch mobile and desktop computing platforms. Comput Graphics Forum. 2013;32:227–242. doi: 10.1111/cgf.12137.

94. Jha A., Yang H., Deng R., Kapp M.E., Fogo A.B., Huo Y. Instance segmentation for whole slide imaging: end-to-end or detect-then-segment. J Med Imag. 2021;8:#014001. doi: 10.1117/1.JMI.8.1.014001.

95. Jiang Y., Chan C.K., Chan R.C., et al. Identification of tissue types and gene mutations from histopathology images for advancing colorectal cancer biology. IEEE Open J Eng Med Biol. 2022;3:115–123. doi: 10.1109/OJEMB.2022.3192103.

96. Jose L., Liu S., Russo C., Nadort A., Di Ieva A. Generative adversarial networks in digital pathology and histopathological image processing: A review. J Pathol Inform. 2021;12:43. doi: 10.4103/jpi.jpi_103_20. https://www.sciencedirect.com/science/article/pii/S2153353922001651

97. Kalra S., Tizhoosh H., Choi C., et al. Yottixel — an image search engine for large archives of histopathology whole slide images. Med Image Anal. 2020;65:#101757. doi: 10.1016/j.media.2020.101757.

98. Kanavati F., Ichihara S., Tsuneki M. A deep learning model for breast ductal carcinoma in situ classification in whole slide images. Virchows Arch. 2022;480:1009–1022. doi: 10.1007/s00428-021-03241-z.

99. Kaustaban V., et al. In: Medical Optical Imaging and Virtual Microscopy Image Analysis. MOVI 2022. Lecture Notes in Computer Science, vol 13578. Huo Y., Millis B.A., Zhou Y., Wang X., Harrison A.P., Xu Z., editors. Springer; Cham: 2022. Characterizing Continual Learning Scenarios for Tumor Classification in Histopathology Images.

100. Kaustaban V., Ba Q., Bhattacharya I., et al. Continual learning for tumor classification in histopathology images. arXiv preprint arXiv:2208.03609. 2022

101. Keller B.M., Gastounioti A., Batiste R.C., Kontos D., Feldman M.D. Medical Imaging 2016: Digital Pathology. SPIE; 2016. Preliminary evaluation of a fully automated quantitative framework for characterizing general breast tissue histology via color histogram and color texture analysis; pp. 69–74.

102. Khened M., Kori A., Rajkumar H., Krishnamurthi G., Srinivasan B. A generalized deep learning framework for whole-slide image segmentation and analysis. Sci Rep. 2021;11:1–14. doi: 10.1038/s41598-021-90444-8.

103. Kipf T.N., Welling M. 2016. Semi-supervised classification with graph convolutional networks.

104. Koohbanani N.A., Jahanifar M., Tajadin N.Z., Rajpoot N. NuClick: a deep learning framework for interactive segmentation of microscopy images. arXiv:2005.14511v2. 2020 doi: 10.48550/arXiv.2005.14511.

105. Kothari S., Phan J.H., Stokes T.H., Wang M.D. Pathology imaging informatics for quantitative analysis of whole-slide images. J Am Med Inform Assoc. 2013;20:1099–1108. doi: 10.1136/amiajnl-2012-001540.

106. Koyun O.C., Yildirim T. IEEE International Symposium on Innovations in Intelligent Systems and Applications. IEEE; 2019. Adversarial nuclei segmentation on h&e stained histopathology images; pp. 1–5.

107. Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst. 2012;25 https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html

108. Kugler M., Goto Y., Kawamura N., et al. International Workshop on Computational Pathology. Springer; 2018. Accurate 3D reconstruction of a whole pancreatic cancer tumor from pathology images with different stains; pp. 35–43.

109. Kumar N., Gupta R., Gupta S. Whole slide imaging (wsi) in pathology: current perspectives and future directions. J Digit Imaging. 2020;33:1034–1040. doi: 10.1007/s10278-020-00351-z.

110. Lambrou A., Papadopoulos H., Gammerman A. Int'l Conf. on Information Technology and Applications in Biomedicine. 2009. Evolutionary conformal prediction for breast cancer diagnosis; pp. 1–4.

111. Lee G., Veltri R.W., Zhu G., Ali S., Epstein J.I., Madabhushi A. Nuclear shape and architecture in benign fields predict biochemical recurrence in prostate cancer patients following radical prostatectomy: preliminary findings. Eur Urol Focus. 2017;3:457–466. doi: 10.1016/j.euf.2016.05.009.

112. Levy J., Haudenschild C., Barwick C., Christensen B., Vaickus L. Pacific Symposium on Biocomputing. World Scientific; 2020. Topological feature extraction and visualization of whole slide images using graph neural networks; pp. 285–296. https://pubmed.ncbi.nlm.nih.gov/33691025/

113. Li B., Keikhosravi A., Loeffler A.G., Eliceiri K.W. Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization. Med Image Anal. 2021;68 doi: 10.1016/j.media.2020.101938.

114. Li C., Zhu X., Yao J., Huang J. 2022 26th International Conference on Pattern Recognition (ICPR) 2022. Hierarchical transformer for survival prediction using multimodality whole slide images and genomics; pp. 4256–4262.

115. Li G., Müller M., Thabet A., Ghanem B. IEEE/CVF International Conference on Computer Vision (ICCV) 2019. DeepGCNs: can GCNs go as deep as CNNs? pp. 9266–9275.

116. Li J., Shao W., Li Z., Li W., Zhang D. International Workshop on Machine Learning in Medical Imaging. Springer; 2019. Residual attention generative adversarial networks for nuclei detection on routine colon cancer histology images; pp. 142–150.

117. Li N., Lv T., Sun Y., Liu X., Zeng S., Lv X. High throughput slanted scanning whole slide imaging system for digital pathology. J Biophotonics. 2021;14 doi: 10.1002/jbio.202000499. #e202000499.

118. Liang M., Chen Q., Li B., et al. Interpretable classification of pathology whole-slide images using attention based context-aware graph convolutional neural network. Comput Methods Prog Biomed. 2023;229 doi: 10.1016/j.cmpb.2022.107268.

119. Lin L., Dou Q., Jin Y.M., et al. Deep learning for automated contouring of primary tumor volumes by MRI for nasopharyngeal carcinoma. Radiology. 2019;291:677–686. doi: 10.1148/radiol.2019182012.

120. Livens S., Scheunders P., van de Wouwer G., van Dyck D. International Conference on Image Processing and Its Applications. IET; 1997. Wavelets for texture analysis, an overview; pp. 581–585.

121. Lomacenkova A., Arandjelovic O. IEEE EMBS International Conference on Biomedical and Health Informatics. IEEE; 2021. Whole slide pathology image patch based deep classification: an investigation of the effects of the latent autoencoder representation and the loss function form; pp. 1–4.

122. Lopez C.M., Agaian S., Sanchez I., et al. IEEE International Conference on Systems, Man, and Cybernetics. IEEE; 2012. Exploration of efficacy of gland morphology and architectural features in prostate cancer Gleason grading; pp. 2849–2854.

123. Lowe D.G. IEEE International Conference on Computer Vision (ICCV) IEEE; 1999. Object recognition from local scale-invariant features; pp. 1150–1157.

124. Lu C., Koyuncu C., Corredor G., et al. Feature-driven local cell graph (FLocK): new computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers. Med Image Anal. 2021;68:#101903. doi: 10.1016/j.media.2020.101903.

125. Nagase A., Takahashi M., Nakano M. Automatic calculation and visualization of nuclear density in whole slide images of hepatic histological sections. Biomed Mater Eng. 2015;26:S1335–S1344. doi: 10.3233/bme-151431.

126. Nateghi R., Danyali H., Helfroush M.S. A deep learning approach for mitosis detection: Application in tumor proliferation prediction from whole slide images. Artif Intell Med. 2021;114 doi: 10.1016/j.artmed.2021.102048. https://www.sciencedirect.com/science/article/pii/S0933365721000415

127. Niazi M.K.K., Parwani A.V., Gurcan M.N. Digital pathology and artificial intelligence. Lancet Oncol. 2019;20:e253–e261. doi: 10.1016/S1470-2045(19)30154-8.

128. Okabe A., Boots B., Sugihara K., Chiu S.N. 2nd ed. Vol. 501. John Wiley & Sons; 2000. Spatial Tessellations: Concepts and Applications of Voronoi Diagrams. https://www.wiley.com/en-gb/Spatial+Tessellations%3A+Concepts+and+Applications+of+Voronoi+Diagrams%2C+2nd+Edition-p-9780470317853.j.media.2020.101903

129. Öztürk S., Akdemir B. 2018. Effects of Histopathological Image.

130. Ma L., Rathgeb A., Mubarak H., Tran M., Fei B. Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging. J Biomed Opt. 2022;27:056502. doi: 10.1117/1.JBO.27.5.056502.

131. Ma Y., Jiang Z., Zhang H., et al. Breast histopathological image retrieval based on latent Dirichlet allocation. IEEE J Biomed Health Inform. 2016;21:1114–1123. doi: 10.1109/JBHI.2016.2611615.

132. van der Maaten L., Hinton G.E. Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579–2605. https://www.jmlr.org/papers/v9/vandermaaten08a.html

133. Madabhushi A., Lee G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal. 2016;33:170–175. doi: 10.1016/j.media.2016.06.037.

134. Mahmood F., Borders D., Chen R.J., et al. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE Trans Med Imaging. 2019;39:3257–3267. doi: 10.1109/TMI.2019.2927182.

135. Maschler M., Solan E., Zamir S. Cambridge University Press; 2013. Game Theory. ISBN 978-110700548-8.

136. Mehta N., Alomari R.S., Chaudhary V. International Conference of the IEEE Engineering in Medicine and Biology Society. 2009. Content based sub-image retrieval system for high resolution pathology images using salient interest points; pp. 3719–3722.

137. Meng Z., Zhao Z., Li B., Su F., Guo L. A cervical histopathology dataset for computer aided diagnosis of precancerous lesions. IEEE Trans Med Imaging. 2021;40:1531–1541. doi: 10.1109/TMI.2021.3059699.

138. Mercan E., Aksoy S., Shapiro L.G., Weaver D.L., Brunyé T.T., Elmore J.G. Localization of diagnostically relevant regions of interest in whole slide images: a comparative study. J Digit Imaging. 2016;29:496–506. doi: 10.1007/s10278-016-9873-1.

139. Mi H., Gong C., Sulam J., et al. Digital pathology analysis quantifies spatial heterogeneity of CD3, CD4, CD8, CD20, and FOXP3 immune markers in triple-negative breast cancer. Front Physiol. 2020;11:#583333. doi: 10.3389/fphys.2020.583333.

140. Mirza M., Osindero S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. 2014 doi: 10.48550/arXiv.1411.1784.

141. Mohan G. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. 2022. Intelligent framework for brain tumor grading using advanced feature analysis; pp. 1–19.

142. Morrison D., Harris-Birtill D., Caie P.D. Generative deep learning in digital pathology workflows. Am J Pathol. 2021;191:1717–1723. doi: 10.1016/j.ajpath.2021.02.024.

143. Mukherjee L., Bui H.D., Keikhosravi A., Loeffler A., Eliceiri K.W. Super-resolution recurrent convolutional neural networks for learning with multi-resolution whole slide images. J Biomed Optics Preprocess Convol Neural Netw Proc Comput Sci. 2019;132:396–403. doi: 10.1016/j.procs.2018.05.166.

144. Pantanowitz L., Sharma A., Carter A.B., Kurc T., Sussman A., Saltz J. Twenty years of digital pathology: an overview of the road travelled, what is on the horizon, and the emergence of vendor-neutral archives. J Pathol Inform. 2018;9 doi: 10.4103/jpi.jpi_69_18.

145. Peikari M., Gangeh M.J., Zubovits J., Clarke G., Martel A.L. Triaging diagnostically relevant regions from pathology whole slides of breast cancer: a texture based approach. IEEE Trans Med Imaging. 2015;35:307–315. doi: 10.1109/tmi.2015.2470529.

146. Pezoa R., Salinas L., Torres C., Härtel S., Maureira-Fredes C., Arce P. Journal of Physics: Conference Series. IOP Publishing; 2016. Segmentation of HER2 protein overexpression in immunohistochemically stained breast cancer images using support vector machines; p. #012050.

147. Qi X., Wang D., Rodero I., et al. Content-based histopathology image retrieval using cometcloud. BMC Bioinformatics. 2014;15:1–17. doi: 10.1186/1471-2105-15-287.

148. Qu H., Zhou M., Yan Z., et al. Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precision Oncol. 2021;5:1–11. doi: 10.1038/s41698-021-00225-9.

149. Rathore S., Niazi T., Iftikhar M.A., Chaddad A. Glioma grading via analysis of digital pathology images using machine learning. Cancers. 2020;12:#578. doi: 10.3390/cancers12030578.

150. Rebouças Filho P.P., Peixoto S.A., da Nóbrega R.V.M., et al. Automatic histologically-closer classification of skin lesions. Comput Med Imaging Graph. 2018;68:40–54. doi: 10.1016/j.compmedimag.2018.05.004.

151. Ren J., Karagoz K., Gatza M., Foran D.J., Qi X. Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. SPIE; 2018. Differentiation among prostate cancer patients with Gleason score of 7 using histopathology whole-slide image and genomic data; p. #1057904.

152. Ren J., Karagoz K., Gatza M.L., et al. Recurrence analysis on prostate cancer patients with Gleason score 7 using integrated histopathology whole-slide images and genomic data through deep neural networks. J Med Imag. 2018;5:#047501. doi: 10.1117/1.jmi.5.4.047501.

153. Rodriguez J.P.M., Rodriguez R., Silva V.W.K., et al. Artificial intelligence as a tool for diagnosis in digital pathology whole slide images: a systematic review. J Pathol Inform. 2022:100138. doi: 10.1016/j.jpi.2022.100138. https://www.sciencedirect.com/science/article/pii/S2153353922007325

154. Romo D., Romero E., González F. Learning regions of interest from low level maps in virtual microscopy. Diagn Pathol. 2011;6:1–8. doi: 10.1186/1746-1596-6-S1-S22.

155. Ronneberger O., Fischer P., Brox T. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2015. U-Net: Convolutional networks for biomedical image segmentation; pp. 234–241.

156. Roy M., Kong J., Kashyap S., et al. Convolutional autoencoder based model HistoCAE for segmentation of viable tumor regions in liver whole-slide images. Sci Rep. 2021;11:1–10. doi: 10.1038/s41598-020-80610-9.

157. Rumelhart D.E., Hinton G.E., Williams R.J. Learning representations by back-propagating errors. Nature. 1986;323:533–536. doi: 10.1038/323533a0.

158. Ryu J., Puche A.V., Shin J., et al. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023. Ocelot: overlapped cell on tissue dataset for histopathology; pp. 23902–23912.

159. Salakhutdinov R. Learning deep generative models. Annu Rev Stat Appl. 2015;2:361–385. doi: 10.1146/annurev-statistics-010814-020120.

160. Salvi M., Acharya U.R., Molinari F., Meiburger K.M. The impact of pre-and post-image processing techniques on deep learning frameworks: a comprehensive review for digital pathology image analysis. Comput Biol Med. 2021;128 doi: 10.1016/j.compbiomed.2020.104129.

161. Schmauch B., Romagnoni A., Pronier E., et al. A deep learning model to predict rna-seq expression of tumours from whole slide images. Nat Commun. 2020;11:1–15. doi: 10.1038/s41467-020-17678-4.

162. Schouten B.A., de Zeeuw P.M. International Conference on Advances in Visual Information Systems. Springer; 1999. Feature extraction using fractal codes; pp. 483–493.

163. Shaban M.T., Baur C., Navab N., Albarqouni S. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) 2019. Staingan: stain style transfer for digital histological images; pp. 953–956.

164. Shao Z., Bian H., Chen Y., et al. Transmil: transformer based correlated multiple instance learning for whole slide image classification. Adv Neural Inf Proces Syst. 2021;34:2136–2147. https://proceedings.neurips.cc/paper/2021/file/10c272d06794d3e5785d5e7c5356e9ff-Paper.pdf

165. Sharma H., Zerbe N., Böger C., Wienert S., Hellwich O., Hufnagl P. IEEE International Symposium on Computer-Based Medical Systems. IEEE; 2017. A comparative study of cell nuclei attributed relational graphs for knowledge description and categorization in histopathological gastric cancer whole slide images; pp. 61–66.

166. Sharma H., Zerbe N., Lohmann S., Kayser K., Hellwich O., Hufnagl P. A review of graph-based methods for image analysis in digital histopathology. Diagn Pathol. 2015;1:61. doi: 10.17629/www.diagnosticpathology.eu-2015-1.

167. Sharma S., Mehra R. Conventional machine learning and deep learning approach for multi-classification of breast cancer histopathology images—a comparative insight. J Digit Imaging. 2020;33:632–654. doi: 10.1007/s10278-019-00307-y.

168. Sheehan S.M., Korstanje R. Automatic glomerular identification and quantification of histological phenotypes using image analysis and machine learning. Am J Physiol Renal Physiol. 2018;315:F1644–F1651. doi: 10.1152/ajprenal.00629.2017.

169. Shi J., Wang R., Zheng Y., Jiang Z., Zhang H., Yu L. Cervical cell classification with graph convolutional network. Comput Methods Prog Biomed. 2021;198:#105807. doi: 10.1016/j.cmpb.2020.105807.

170. Simon O., Yacoub R., Jain S., Tomaszewski J.E., Sarder P. Multi-radial LBP features as a tool for rapid glomerular detection and assessment in whole slide histopathology images. Sci Rep. 2018;8:1–11. doi: 10.1038/s41598-018-20453-7.

171. Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556v6. 2014 doi: 10.48550/arXiv.1409.1556.

172. Simpson E.H. Measurement of diversity. Nature. 1949;163:#688. doi: 10.1038/163688a0.

173. Sobel I., Feldman G. Talk at the Stanford Artificial Intelligence Project (SAIL) 1968. A 3 × 3 isotropic gradient operator for image processing. https://www.researchgate.net/publication/239398674_An_Isotropic_3x3_Image_Gradient_Operator

174. Solorzano L., Almeida G.M., Mesquita B., Martins D., Oliveira C., Wählby C. International Workshop on Computational Pathology. Springer; 2018. Whole slide image registration for the study of tumor heterogeneity; pp. 95–102.

175. Srinidhi C.L., Ciga O., Martel A.L. Deep neural network models for computational histopathology: a survey. Med Image Anal. 2021;67 doi: 10.1016/j.media.2020.101813. https://www.sciencedirect.com/science/article/pii/S1361841520301778

176. Stegmüller T., Bozorgtabar B., Spahr A., Thiran J.P. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023. Scorenet: learning non-uniform attention and augmentation for transformer-based histopathological image classification; pp. 6170–6179.

177. Su Z., Tavolara T.E., Carreno-Galeano G., Lee S.J., Gurcan M.N., Niazi M. Attention2majority: weak multiple instance learning for regenerative kidney grading on whole slide images. Med Image Anal. 2022;79 doi: 10.1016/j.media.2022.102462. https://www.sciencedirect.com/science/article/pii/S1361841522001098

178. Sun C., Li B., Wei G., et al. Deep learning with whole slide images can improve the prognostic risk stratification with stage iii colorectal cancer. Comput Methods Prog Biomed. 2022;221:#106914. doi: 10.1016/j.cmpb.2022.106914.

179. Sureka M., Patil A., Anand D., Sethi A. International Conference on Bioinformatics and Bioengineering. 2020. Visualization for histopathology images using graph convolutional neural networks; pp. 331–335.

180. Szegedy C., Liu W., Jia Y., et al. IEEE Conference on Computer Vision and Pattern Recognition. 2015. Going deeper with convolutions; pp. 1–9.

181. Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. IEEE Conference on Computer Vision and Pattern Recognition. 2016. Rethinking the inception architecture for computer vision; pp. 2818–2826.

182. Tabesh A., Teverovskiy M., Pang H.Y., et al. Multifeature prostate cancer diagnosis and Gleason grading of histological images. IEEE Trans Med Imaging. 2007;26:1366–1378. doi: 10.1109/tmi.2007.898536.

183. Takagi Y., Hashimoto N., Masuda H., et al. Transformer-based personalized attention mechanism for medical images with clinical records. J Pathol Inform. 2023;100185 doi: 10.1016/j.jpi.2022.100185.

184. Tamura H., Mori S., Yamawaki T. Textural features corresponding to visual perception. IEEE Trans Syst Man Cybern. 1978;8:460–473. doi: 10.1109/TSMC.1978.4309999.

185. Tellez D., Balkenhol M., Otte-Höller I., et al. Whole-slide mitosis detection in h&e breast histology using phh3 as a reference to train distilled stain-invariant convolutional networks. IEEE Trans Med Imaging. 2018;37:2126–2136. doi: 10.1109/TMI.2018.2820199.

186. Teverovskiy M., Kumar V., Ma J., et al. IEEE International Symposium on Biomedical Imaging: Nano to Macro. IEEE; 2004. Improved prediction of prostate cancer recurrence based on an automated tissue image analysis system; pp. 257–260.

187. Tizhoosh H.R., Pantanowitz L. Artificial intelligence and digital pathology: challenges and opportunities. J Pathol Inform. 2018;9:38. doi: 10.4103/jpi.jpi_53_18. https://www.sciencedirect.com/science/article/pii/S2153353922003510

188. Tomita F., Tsuji S. Vol. 102. Springer Science & Business Media; 2013. Computer Analysis of Visual Textures.

189. del Toro O.J., Atzori M., Otálora S., et al. Medical Imaging 2017: Digital Pathology. SPIE; 2017. Convolutional neural networks for an automatic classification of prostate tissue slides with high-grade Gleason score; pp. 165–173.

190. Vaishali D., Ramesh R., Christaline J.A. Histopathology image analysis and classification for cancer detection using 2D autoregressive model. Int Rev Comput Softw. 2015;10:182–188. doi: 10.15866/irecos.v10i2.5113.

191. Vaishali D., Ramesh R., Christaline J.A. Autoregressive modelling: application to mitosis detection in brain cancer histopathology. Int J Biomed Eng Technol. 2016;20:179–194. doi: 10.1504/IJBET.2016.074202.

192. Vaswani A., Shazeer N., Parmar N., et al. Attention is all you need. Adv Neural Inf Proces Syst. 2017;30 https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

193. Veta M., Van Diest P.J., Willems S.M., et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal. 2015;20:237–248. doi: 10.1016/j.media.2014.11.010. https://www.sciencedirect.com/science/article/pii/S1361841514001807

194. Vogl T.J., Worst T.S., Naguib N.N., Ackermann H., Gruber-Rouh T., Nour-Eldin N.E.A. Factors influencing local tumor control in patients with neoplastic pulmonary nodules treated with microwave ablation: a risk-factor analysis. Am J Roentgenol. 2013;200:665–672. doi: 10.2214/AJR.12.8721.

195. Wang J., Chen R.J., Lu M.Y., Baras A., Mahmood F. International Symposium on Biomedical Imaging. IEEE; 2020. Weakly supervised prostate TMA classification via graph convolutional networks; pp. 239–243.

196. Wang K.S., Yu G., Xu C., et al. Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence. BMC Med. 2021;19:1–12. doi: 10.1186/s12916-021-01942-5.

197. Wang S., Chen A., Yang L., et al. Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome. Sci Rep. 2018;8:1–9. doi: 10.1038/s41598-018-27707-4.

198. Wang S., Yang D.M., Rong R., et al. Artificial intelligence in lung cancer pathology image analysis. Cancers. 2019;11:#1673. doi: 10.3390/cancers11111673.

199. West J., Fitzpatrick J.M., Wang M.Y., et al. Comparison and evaluation of retrospective intermodality brain image registration techniques. J Comput Assist Tomogr. 1997;21:554–568. doi: 10.1097/00004728-199707000-00007.

200. Xie Q., Luong M.T., Hovy E., Le Q.V. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) 2020. Self-training with noisy student improves imagenet classification; pp. 10687–10698. https://openaccess.thecvf.com/content_CVPR_2020/html/Xie_Self-Training_With_Noisy_Student_Improves_ImageNet_Classification_CVPR_2020_paper.html

201. Xu H., Park S., Hwang T.H. Computerized classification of prostate cancer Gleason scores from whole slide images. IEEE/ACM Trans Comput Biol Bioinform. 2019;17:1871–1882. doi: 10.1109/tcbb.2019.2941195.

202. Xu J., Xiang L., Liu Q., et al. Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging. 2015;35:119–130. doi: 10.1109/tmi.2015.2458702.

203. Xu Y., Jia Z., Wang L.B., et al. Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinform. 2017;18:1–17. doi: 10.1186/s12859-017-1685-x.

204. Yagi Y. Diagnostic Pathology. Springer; 2011. Color standardization and optimization in whole slide imaging; pp. 1–12.

205. Yamashiro K., Taira K., Matsubayashi S., et al. Comparison between a traditional single still image and a multiframe video image along the z-axis of the same microscopic field of interest in cytology: which does contribute to telecytology? Diagn Cytopathol. 2009;37:727–731. doi: 10.1002/dc.21078.

206. Ye H., Wang D.H., Li J., Zhu S., Zhu C. International Conference on Computing and Pattern Recognition. 2019. Improving histopathological image segmentation and classification using graph convolution network; pp. 192–198.

207. Yener B. Cell-graphs: image-driven modeling of structure-function relationship. Commun ACM. 2016;60:74–84. doi: 10.1145/2960404.

208. Yin P., Yu B., Jiang C., Chen H. 2022 Eleventh International Conference on Image Processing Theory, Tools and Applications (IPTA) IEEE; 2022. Pyramid tokens-to-token vision transformer for thyroid pathology image classification; pp. 1–6.

209. You M., Han X., Xu Y., Li L. Systematic evaluation of deep face recognition methods. Neurocomputing. 2020;388:144–156. doi: 10.1016/j.neucom.2020.01.023.

210. Yu E., Monaco J.P., Tomaszewski J., Shih N., Feldman M., Madabhushi A. 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE; 2011. Detection of prostate cancer on histopathology using color fractals and probabilistic pairwise markov models; pp. 3427–3430.

211. Zanjani F.G., Zinger S., Bejnordi B.E., van der Laak J.A., de With P.H. Medical Imaging with Deep Learning. 2018. Histopathology stain-color normalization using deep generative models. https://openreview.net/forum?id=SkjdxkhoG

212. Zhang C., Xiao G., Moon C., Chen M., Li Q. Bayesian landmark-based shape analysis of tumor pathology images. arXiv:2012.01149. 2020 doi: 10.48550/arXiv.2012.01149.

213. Zhang J., Hua Z., Yan K., et al. Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images. Med Image Anal. 2021;73:#102183. doi: 10.1016/j.media.2021.102183.

214. Zhao Y., Black E.F., Marini L., et al. 2016 IEEE 12th International Conference on e-Science (e-Science) IEEE; 2016. Automatic glomerulus extraction in whole slide images towards computer aided diagnosis; pp. 165–174.

215. Zhao Y., Yang F., Fang Y., et al. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution; pp. 4837–4846. https://openaccess.thecvf.com/content_CVPR_2020/html/Zhao_Predicting_Lymph_Node_Metastasis_Using_Histopathological_Images_Based_on_Multiple_CVPR_2020_paper.html

216. Zheng L., Wetzel A.W., Gilbertson J., Becich M.J. Design and analysis of a content-based pathology image retrieval system. IEEE Trans Inf Technol Biomed. 2003;7:249–255. doi: 10.1109/TITB.2003.822952.

217. Zheng Y., Cassol C.A., Jung S., et al. Deep-learning–driven quantification of interstitial fibrosis in digitized kidney biopsies. Am J Pathol. 2021;191:1442–1453. doi: 10.1016/j.ajpath.2021.05.005. https://www.sciencedirect.com/science/article/pii/S000294402100208X

218. Zhou N., Cai D., Han X., Yao J. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019. Enhanced cycle-consistent generative adversarial network for color normalization of H&E stained images; pp. 694–702.

219. Zhou Y., Graham S., Alemi Koohbanani N., Shaban M., Heng P.A., Rajpoot N. IEEE/CVF International Conference on Computer Vision Workshops. 2019. CGC-Net: cell graph convolutional network for grading of colorectal cancer histology images; pp. 388–398.

220. Zhu J.Y., Park T., Isola P., Efros A.A. Computer Vision (ICCV), 2017 IEEE International Conference. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks; pp. 2223–2232.

221. Zhu Y., Tong L., Deshpande S.R., Wang M.D. IEEE EMBS International Conference on Biomedical & Health Informatics. IEEE; 2019. Improved prediction on heart transplant rejection using convolutional autoencoder and multiple instance learning on whole-slide imaging; pp. 1–4.

222. Zitova B., Flusser J. Image registration methods: a survey. Image Vis Comput. 2003;21:977–1000. doi: 10.1016/S0262-8856(03)00137-9.

223. Zunic J. Shape descriptors for image analysis. Zbornik Radova MI-SANU. 2012;15:5–38. https://eudml.org/doc/256799