Introduction

Hepatocellular carcinoma (HCC) is the fourth leading cause of cancer-related mortality and is currently the main cause of liver-related death, leading to more than one million deaths annually worldwide1,2,3. Over several decades, substantial progress had been made in the understanding of HCC risk factors, epidemiology, and molecular pathogenesis. The early detection of HCC increases the chance of curative therapies in high overall survival. Unfortunately, most HCC patients are diagnosed at the intermediate to late-stage, which significantly decreases the overall survival4. Various predominant clinical risk factors for the development of HCC have been defined, including alcohol abuse, cirrhosis, metabolic syndrome, and hepatitis B and/or C virus infection5,6,7,8. However, multiple genetic alternation and signaling cascades also have a great influence on tumor progression and overall survival9.

The understanding of HCC molecular pathogenesis has been significantly improved over the past decade10. The development of genomic analysis has identified the major drivers that are responsible for cancer development and progression. HCC has been reported to have around 40 genomic aberrations, some of which are deemed as drivers. Several frequent HCC genomic alternations have been identified, including mutations in the CTNNB1 (β-catenin WNT pathway activation), TP53, telomere reverse transcriptase (telomere maintenance), AT-rich interaction domain 1A (ARID1A; chromatin remodeling), mammalian target of rapamycin signaling, RAS signaling, oxidative stress pathway activation, and aberrations in DNA methylation11. Previous studies have reported that the heterogeneity of HCC at both molecular and histological levels are correlated with gene mutations and oncogenic pathways12. The mutually exclusive CTNNB1 (40%) and TP53 (21%) mutations have been identified as two major groups of HCC according to its distinct phenotype. CTNNB1 mutated HCC is generally well-differentiated and large, with pseudoglandular and microtrabecular patterns, and lacks inflammatory infiltrates; whereas TP53 mutated HCC is generally poor-differentiated, with compact patterns, frequent vascular invasion, and pleomorphic, multinucleated cells13. The deeper understandings of the HCC phenotypes are essential for improving targeted therapies and clinical translation.

Pathologists could provide limited information regarding cancer reorganization from normal liver tissue and assess its histopathological grade via visual inspection, but it still lacks the underlying biological differences in HCC gene mutations associated with overall survival. The recent advances in artificial intelligence (AI) provided a novel way to assist clinicians to classify medical information and images14,15,4). Finally, the liver cancer tiles dataset consisted of four subsets, including the training, testing, internal validation, and external validation sets. The data in the training and internal validation cohorts from the Genomic Data Commons portal (https://portal.gdc.cancer.gov/) were publicly available without restriction, authentication or authorization. The independent external validation cohort we used consisted of slide images without identifiable information and all participants had provided written informed consent. Our study was approved by the SRRSH of Medicine Institutional Review Board (KY20181209-5).

Fig. 4: Strategy of preparing tiles dataset.
figure 4

First, each WSI of liver tissue was selected from GDC-portal or SRRSH. Then, they were cropped into lots of tiles. Finally, the tiles less than 80% area of surface with tissue were removed, and the remaining tiles were used for further analysis.

Full size image

Technical detail on frozen slides in the external validation cohort

The obtained specimens (e.g., liver tissues) were macroscopically examined, measured, sectioned through their longest axis, and then midsections were examined. The material was frozen at −28 °C, cut into 5–10 µm thick sections, Hematoxylin-Eosin (H&E) stained, and then analysed by pathologists with the light microscope. There were 67 out of 70 patients diagnosed as HCC and the related frozen slide were collected. Notably, normal liver tissues cannot be available in half of the obtained specimens, because normal liver tissues should be at least 2 cm away from tumors. Therefore, there were only 34 WSIs of normal liver tissues. In order to obtain digital pathology images, each slide was scanned at a magnification of 20× by using digital pathology scanner VS120 (Olympus).

Deep-learning with convolution neural networks

Typical convolutional neural networks contain several levels of convolution filters, pooling layers, and fully connected layers. In our study, we primarily used inception V3 architecture, which makes use of inception modules which are made from a spread of convolutions having different kernel sizes and a max-pooling layer. The initial five convolution nodes are combined with two max-pooling operations and followed by 11 stacks of inception modules. A fully connected layer to the end of the inception modules was then added to permit us to utilize the pre-trained model and finetune the parameters for our own task. Finally, a softmax layer was added as a classifier outputting a probability for every class, and the one with the highest probability was chosen as the predicted class.

We used the pre-trained model offered by TensorFlow and finetuned it using histopathological images. It was pre-trained on the ImageNet dataset and available at the TensorFlow-Slim image classification library (http://tensorflow.org). We initialized the parameters from the pre-trained model because pre-training can speed up the convergence of the network. Most importantly, it was difficult to train a deep network with a small number of images due to the massive number of network parameters.

Comparison with pathologists

One hundred and one WSIs of liver tissues without a label from the external validation cohort were used to test pathologist’s performance and compared with our model performance. All pathologists should report whether there is HCC, and if there is HCC, they should report histopathological grade via digital pathology images. The outcomes reported by six pathologists with 2-years, 5-years, and 10-years experience (two pathologists in each category) and our model were collected and analyzed by the R 3.6.0 (https://www.r-project.org). Cohen’s Kappa analysis was performed to assess inter-observer agreement. Good inter-operator agreements were observed in pathologists with 2-year experience (Kappa = 0.894; 95% CI, 0.837–0.944), pathologists with 5-year experience (Kappa = 0.933; 95% CI, 0.888–0.975), and pathologists with 5-year experience (Kappa = 0.967; 95% CI, 0.930–0.992).

Identification of significantly mutated genes

The gene mutation data for the matched patient sample were downloaded from the cancer genome atlas (TCGA). The gene mutated at least 10% of the available liver cancer samples were selected from the 283 cancer-related genes (Supplementary Fig. 2). The least absolute shrinkage and selection operator (LASSO) regression with a 10-fold cross-validation method was then performed to identify significant prognosis-related gene mutations by using R software packages (http://www.r-project.org). Finally, the ten most significant prognosis-related gene mutations, including ARID1A, ASH1L, CSMD1, CTNNB1, EYS, FMN2, MDM4, RB1, TP53, and ZFX4 were identified (Fig. 5).

Fig. 5: Prognosis-related mutated genes selection using the least absolute shrinkage and selection operator (LASSO) Cox regression model.
figure 5

a Selection of the super parameter λ in the LASSO model via 10-fold cross-validation based on the minimum standard. The optimal λ value of 0.122. b Shown here is a coefficient section view plotted against the log(λ) magnitude. The optimal λ corresponding to ten non-zero coefficients were obtained where the vertical line was drawn.

Full size image

Training deep-learning network

Pathological diagnosis was the primary endpoint of interest for the classifier that recognizes tumors from normal liver tissue and the assessment of the histopathological grade. The status of gene mutation (mutation or wild type), based on the next-generation sequencing results, was the primary prerequisite in the classifier of mutation prediction. The model’s training strategy was based on an easy-to-use platform called EASY DL (https://ai.baidu.com/easydl/) that uses PaddlePaddle deep learning framework V3.0 created by Baidu Brain AI technology, inception V3 network developed by Google, and packaging code form Coudray20 and co-workers. The training set was used for training, and the testing set was used to evaluate the performances, finetune those parameters, and improve the models. A final model was selected according to the results of the testing set, where the F1-scores as a stop** rule. Notably, the subsets were grouped based on HCC patients rather than the WSIs. This method could maximize the size of the training set and avoid training and testing on tiles originating from the same human subjects. Thereby preventing the classifier from relying on intra-subject correlations between samples and resulting in inflated estimates of accuracy. In order to reduce selection bias, the performance of our model was then validated in the internal and external validation sets.

Statistical analysis

The ten most common and prognostic mutated genes were identified using the LASSO Cox regression model, and any differences of overall survival were evaluated by the Kaplan–Meier method with a log-rank test. The performance of those models was evaluated with F1-scores, MCC, and AUC. The F1-scores, ranging from 1 (perfect) to 0 (bad), is the harmonic average of the precision and recall21. MCC ranges from 1 (perfect) to −1 (bad). In addition, the probability of gene mutation was estimated and compared using the two-tailed Mann–Whitney U-tests. A P value of less than 0.05, was considered as statistical significance.