Over the past few decades, medical imaging techniques, such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), mammography, ultrasound, and X-ray, have been used for the early detection, diagnosis, and treatment of diseases. In the clinic, medical image interpretation has been performed mostly by human experts such as radiologists and physicians.
However, given wide variations in pathology and the potential fatigue of human experts, researchers and doctors have begun to benefit from the machine learning methods. The process of applying machine learning methods in medical image analysis is called medical image computation. We will introduce our work in medical image synthesis, classification, and segmentation.
Medical image synthesis:
Complementary imaging modalities are always acquired simultaneously to indicate the disease areas, present the various tissue properties, and help to make an accurate and early diagnosis. However, some imaging modalities are unavailable or lacking due to different reasons such as cost, radiation or other limitations. In such cases, medical imaging synthesis is a novel and effective solution.
Although the classic synthesis algorithms has achieved remarkable results, they are confronted with the same fundamental limitation: it is difficult to generate plausible images with significantly diverse structures, because the generator learns to largely ignore the latent vectors (i.e. the noise vectors input) without any prior knowledge in the training process of GANs.
Especially for the generation of brain images that have diverse structural details (e.g. gyri and sulci) between different subjects. To deal with this challenge, our team proposed a novel end-to-end network, called Bidirectional GAN [1], where image contexts and latent vector were effectively used and jointly optimized for the brain MR to PET synthesis. The framework of the proposed Bidirectional GAN is shown in Fig 1.
To be more specific, a bidirectional mapping mechanism between the latent vector and the output image was introduced, while an advanced generator architecture was adopted to optimally extract and generate the intrinsic features of PET images.
Finally, this work devised a composite loss function containing an additional pixel-wise loss and perceptual loss to encourage less blurring and yield visually more realistic results. As an attempt to bridge the gap between network generative capability and real medical images, the proposed method not only focused on synthesizing perceptually realistic images, but also concentrated on reflecting the diverse brain attributes of different subjects.
Medical image segmentation
Medical image segmentation plays an important role in computer-aided diagnosis (CAD) for the detection and diagnosis of diseases. However, traditional segmentation needed to process manually by pathologists and is thus subjective and time-consuming. Therefore, automatic methods for segmentation are in urgent demand to get measurements in the clinical practice.
Fully supervised training requires a large number of manually labeled masks, which is hard to obtain and only experts can provide reliable annotations. To address this issue, we proposed a novel method named Consistent Perception GAN for semi-supervised segmentation task. Firstly, we joined the similarity connection module into the segmentation network to address the challenges of encoder-decoder architectures mentioned above. This module combined skip connection with local and non-local operations, collected multi-scale feature map to capture long-range spatial information.
Moreover, the proposed Assistant network was verified to improve the performance of discriminator using meaningful feature representations. A consistent transformation strategy was developed in the adversarial training which encouraged a consistent prediction of the segmentation network. Semi-supervised loss was designed according to the discriminators judgment, which limited segmentation network to making approximate prediction between labeled and unlabeled images. The proposed model was employed for skin lesion segmentation [4] and stroke lesion segmentation (Fig 3).
Medical image classification
In medical imaging, the accurate diagnosis or assessment of a disease depends on both image acquisition and image interpretation. Medical image classification can be seen as the core of image interpretation. Generative adversarial network has attracted much attention for medical image classification as it is capable of generating samples without explicitly modeling the probability density function.
It is intelligent for the discriminator to incorporate unlabeled data into the training process by utilizing the adversarial loss. Our team proposed a novel Tensorizing GAN with High-order pooling for medical image classification. Fig. 4 shows the framework of the proposed Tensorizing GAN with High-order pooling. More specifically, the proposed model utilized the compatible learning objects of the three-player cooperative game. Instead of vectorizing each layer as conventional GAN, the tensor-train decomposition was applied to all layers in classifier and discriminator, including fully-connected layers and convolutional layers. Besides, in such a tensor-train format, our model could benefit from the structural information of the object. The proposed model was employed to detect Alzheimers disease [2].
Diabetic retinopathy is one of the major causes of blindness. It is of great significance to apply deep-learning techniques for DR recognition. However, deep-learning algorithms often depend on large amounts of labeled data, which is expensive and time-consuming to obtain in the medical imaging area. To address this issue, we proposed a multichannel-based generative adversarial network (MGAN) with semi-supervision to grade DR [3]. By minimizing the dependence on labeled data, the proposed semi-supervised MGAN could identify the inconspicuous lesion features by using high-resolution fundus images without compression.
Future works:
Finally, we will continue to overcome the challenges of medical image computation so as to:
First, most works still adopt traditional computer vision metrics such as Mean Absolute Error (MAE), Peak-Signal-to-Noise Ratio (PSNR), or Structural Similarity Index Measure (SSIM) for evaluating the quality of synthetic images. The validity of these metrics for medical images remains to be explored. And we will explore some other metrics that are relevant to diagnosis.
Second, deep learning methods have often been described as black boxes. We will focus on the researches about the interpretability of medical image computation.
References:
[1] Hu Shengye, Wang Shuqiang et al. Brain MR to PET Synthesis via Bidirectional Generative Adversarial Network. MICCAI 2020
[2] Lei Baiying, Wang Shuqiang et al. Deep and joint learning of longitudinal data for Alzheimers disease prediction.Pattern Recognition102 (2020): 107247.
[3] Wang Shuqiang, Xiangyu Wang et al., Diabetic Retinopathy Diagnosis using Multi-channel Generative Adversarial Network with Semi-supervision, IEEE Transactions on Automation Science and Engineering, DOI: 10.1109/TASE.2020.2981637, 2020
[4] Lei Baiying, Wang Shuqiang et al. Skin Lesion Segmentation via Generative Adversarial Networks with Dual Discriminators.Medical Image Analysis(2020): 101716.
About Prof. Shuqiang Wang
Shuqiang Wang is currently an Associate Professor with Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Science. He received the Ph.D. degree from the City University of Hong Kong in 2012. He was a Research Scientist with Huawei Technologies Noahs Ark Lab. Before joining the SIAT, he was a Post-Doctoral Fellow with The University of Hong Kong. He has published more than 50 papers on Pattern Recognition, Medical Image Analysis, IEEE Trans on SMC, IEEE Trans on ASE, MICCAI et al. He has applied more than 40 patents of which 15 patents are authorized. His current research interests include machine learning, medical image computing, and optimization theory. As for the medical image computing, He mainly focuses on medical image synthesis, medical segmentation and medical classification. As for the machine learning, he mainly focuses on the GAN theory and its application.
Views expressed in this article do not represent the opinion of Synced Review or its editors.
Synced Report |A Survey of Chinas Artificial Intelligence Solutions in Response to the COVID-19 Pandemic 87 Case Studies from 700+ AI Vendors
This report offers a look at how the Chinese government and business owners have leveraged artificial intelligence technologies in the battle against COVID-19. It is also available onAmazon Kindle.
Clickhereto find more reports from us.
We know you dont want to miss any story.Subscribe to our popularSynced Global AI Weeklyto get weekly AI updates.
Thinking of contributing to Synced Review?Synceds new columnShare My Researchwelcomes scholars to share their own research breakthroughs with global AI enthusiasts.
Like Loading...
Link:
Medical Image Computation and the Application - Synced
