A Review of Recent Deep Learning Models in COVID-19 Diagnosis

— COVID-19 has emerged as the latest worrisome pandemic, which is reported to have its outbreak in Wuhan, China. The infection spreads by means of human contact, as a result, it has caused massive infections across 200 countries around the world. Artificial intelligence has likewise contributed to managing the COVID-19 pandemic in various aspects within a short span of time. Deep Neural Networks that are explored in this paper have contributed to the detection of COVID-19 from imaging sources. The datasets, pre-processing, segmentation, feature extraction, classification and test results which can be useful for discovering future directions in the domain of automatic diagnosis of the disease, utilizing artificial intelligence-based frameworks, have been investigated in this paper.

I. INTRODUCTION 1 During late 2019, the novel Coronavirus Disease (COVID -19) has been proven to be highly contagious through human interaction and gradually became a pandemic in the year 2020. The COVID-19 pandemic has challenged the scientific community to promote futuristic research for prognosis and prevention of the disease by exploring different research modalities. The identification methodologies like Computed Tomography (CT), Chest X-Ray (CXR) and Pulmonary Ultrasound (PUS) are crucial in identifying the actual stage of the disease. The CT-image along with the reference of RT-PCR helps in effective COVID-19 diagnosis. The benchmark for COVID-19 diagnosis still remains the Reverse Transcription Polymerase Chain Reaction (RT-PCR).
Evidently, various research groups became interested towards the development of automated COVID-19 diagnosis based on image-based data sources of CT scanning and CXR. Many articles have proposed different methods to analyze CT scans in order to distinguish COVID-19 from other types of pneumonia, like Community Acquired Pneumonia (CAP) & Interstitial Lung Disease (ILD) [1] based on the abnormal findings of COVID-19 lesion segmentation, and of accurate lobe segmentation in the lungs.
This paper encompasses ongoing research to compile existing resources and to support the researchers in zeroing on, in solutions for the diagnosis of COVID-19. The remaining paper is organized as follows: Section II presents the classification of accessible datasets as per CT  sources. Segment III gives the overview of the COVID-19 diagnosis while presenting the steps involved in diagnosis workflow. Section IV and V present contributions made by several research methodologies. Next, we examine various difficulties confronting this research in Section VI. At last, Section VII discussed the future research avenues and concluded the paper.

II. DATASETS AND IMAGE SOURCES
Several datasets have been made available and classified into CT and CXR image sources for the purpose of experimentation. In recent months, an influx of academic publications has resulted in a deluge of textual data. The extraction of data from clinical studies is already underway by using language modelling frameworks. There are datasets for bibliometric analysis that can be conveniently gathered from pre-prints such as arXiv, medRxiv, and biorXiv. For CT images based works, Chen et al. [2] gained 89% accuracy and Qiu et al. [3] gained 83.62% accuracy using the COVID-19 CT segmentation dataset. Two other methods used the Lung Image Database Consortium (LIDC) dataset [4], [5] and obtained an accuracy above 90%. Several researchers used different pre-processing techniques to increase the dataset size. The prominent datasets are given in Table I.

III. TECHNIQUES FOR DIAGNOSIS
To interpret COVID-19 after data acquisition, preprocessing, feature extraction, segmentation and techniques for classification are the typical steps followed in the pipeline. Among the few imaging methods that may require the preprocessing step are Magnetic Resonance Imaging (MRI Scan), Computed Tomography (CT Scan), Positron Emission Tomography (PET Scan), Optical & Digital Microscopic Imaging Techniques and Pathology and X-Ray. A typical block diagram of the steps of Covid-19 detection is shown in Fig. 1.

A. Pre-processing Techniques
By introducing data augmentation, the pre-processing stage is used to minimize the impact of over-fitting. The most used methods of data augmentation are resizing, scaling, cropping, flipping, rotation. In [7], the proposed method used a General Adversarial Network (GAN) to increase the sample size in the dataset.

B. Segmentation
Segmentation enhances the efficacy of COVID-19 detection by finding the region of interest (ROI) such as the lung field. Based on Convolutional Neural Network (CNN), the U-Net architecture was designed to achieve high segmentation efficacy [8]. The major benefit of the U-Net is that the information of the down-sampling path and the upsampling path are combined to predict a better segmentation map in order to get information containing localization and context [8]. U-Net based strategies were utilized in [2], [5], [10]- [20], [35] for lung segmentation to extract the lung region as ROI. In order to keep contextual information between the slices in CT images, researchers [11], [21] applied 3D versions of U-Net for lung segmentation. Due to the poor contrast in CT images of the infected areas and due to wide variations of body shape and location over different patients, it became difficult to find the infection regions from the chest CT scans. Considering this issue, Fei Shan et al. [22] developed VB-Net which is a modified 3-D CNN based on V-Net [23]. Chen et al. [2] utilized the UNet++ architecture [24] for their research. The AI-model has been trained on 289 scans and tested on 600 scans and high accuracy was achieved in identifying the infected regions for COVID-19 cases. For 2D slice categorization and fine-grained localization of involved areas in lungs, UNet along with ResNet have also been applied [8].

C. Feature Extraction
For feature extraction, CNNs are extensively used for feature extraction of CT-image datasets in order to detect COVID19. The research affirms that CNN-based networks can precisely identify COVID-19 disease & distinguish it from CAP & other pulmonary conditions.

D. Importance of Convolutional Neural Networks (CNN)
The preponderance of CNN, till date, in investigating the visual imagery forms the basic building block of the major COVID-19-diagnostic model. There are a few layers in a CNN architecture portrayed as follows: a. Convolution layer (CL) is the primary foundation of a CNN. The parameters of this layer are made up of a cluster of trainable kernels or filters.
b. Nonlinear layer (NL) is the layer where the change of the output is not proportional to the change of the input. This layer uses activation functions to convey non-linearity to data by adding after each convolution layer. The activation functions used can be Rectified Linear Unit (ReLu), Tanh   ResNet-CNN architectures in the past had a deterioration of performance due to addition of many additional layers, but ResNet [25] showed appreciable performance even with a large number of layers due to the addition of skip connections. ResNet has a number of variants such as ResNet-18, ResNet-50, ResNet-152 etc. Various CNN-based models (i.e., ResNet-50, Inception and InceptionResNet) are utilised to detect COVID-19 in Chest X-rays of pneumonia patients in [43]. The results show 98% accuracy for the pre-trained ResNet-50 model [44]. Farooq et al. [26] proposed another architecture (called COVID-ResNet) based on a fine-tuned ResNet-50 for detecting COVID-19 with improved accuracy.

Densely Connected Convolutional Network (DenseNet)
is formed by densely connected CNN layers, each of which has its output connected with all descendant layers. DenseNet [27] made improvements in the performance by proposing these distinctive network designs with the feature reuse step. Xin et al. and Zhang et al. [28] considered DenseNet-like architecture and optimised it for the classification task to detect COVID-19 infection.

A. COVID-Net
Researchers [29] proposed COVID-Net based on CXR images for COVID-19 diagnosis. The dataset collection comprises 13,800 CXR images from three open access data sources from 13,725 patient events. COVID-Net has two stages each of which has the backbone of ResNet-50. It takes as input a series of CT slices and generates the classification prediction of the given CT image. The CNN features extracted from each CT slice are amalgamated by a maxpooling layer. The resultant feature map is channeled to an FC-layer to make classification based on probability scores for individual classes of COVID-19, CAP, or healthy patients.

C. Combined CNNs
The dataset of Cohen et al. [31], the dataset of Kaggle Footnote and the dataset of common bacterial-pneumonia Xrays [32] were combined by Apostolopoulos et al [33], [34] to differentiate between COVID-19 and CAP. Almost five CNNs with com-mon hyper-parameters, namely Inception, VGG-19, Xception, MobileNetV2 and InceptionResNetV2 were used. The findings indicate that MobileNetv2 and VGG-19 are superior in terms of precision, specificity, and sensitivity.
MobileNetV2 was used for classifying six most common pulmonary diseases. By using transfer learning and hybrid feature extraction, MobileNetV2 collects features from CXR images. To decrease the overfitting effect, a layer of Global Average Pooling (GAP) has been introduced over MobileNetV2. Recent CXR from COVID-19 cases relating to various pulmonary disorders are included in the data collection. The images of COVID-19 (nearly of count, 455) are from Cohen et al. [31].
The authors in [35]

D. COVID-CAPS
COVID-CAPS is a capsule-network model developed for the detection of COVID-19 based on CXRs, which was proposed in [36]. Each of its layers has several capsules which in turn utilize several artificial neurons. The capsule layer signifies a particular image instance at a definite position and the length of a capsule determines the probability of the presence of the given related instance. The model was pre-trained for various thoracic diseases with transfer learning on the public NIH data collection of CXRs. COVID-CAPS produces either a COVID-19 positive or negative as output. The COVID-CAPS model has appreciably performed with an accuracy of 95.7%, sensitivity of 90% and precision of 95.8%.

E. COVID-RENet
The authors [37] designed fine-tuned CNNs based on transfer learning clubbed with SVM classification and dynamic feature space concatenation. The findings revealed that in the identification of COVID-19 cases, COVID-RENet and customized VGG-16 followed by the SVM demonstrate improved efficiency of 98.3% accuracy.

F. EfficientNet B4 and DenseNet121-FPN
In order to differentiate COVID-19 & viral pneumonia from CT images, Tan et al. [38] introduced the EfficientNet B4 [39]. The diagnosis of the corresponding patients by six radiologists was used to assess the effectiveness of the findings collected by the given model. For lung segmentation, DenseNet121-FPN [40] was applied and for classification purposes, COVID-19-Net was introduced. Two validation sets considered with high AUC scores were recorded by the authors.

G. DRE-Net
Ying et al. [41] have built a network called DRE-Net based on improvements to pre-trained ResNet-50. CT scans were conducted on 88 COVID-19 positive cases, 101 patients affected by bacterial pneumonia and 86 healthy individuals. The pre-trained models, DenseNet, ResNet and VGG16, were compared to the proposed network. The obtained results demonstrated that the proposed model outperformed other networks.

H. FCONet
FCONet, proposed by Ko et al. [42], is a fast-track COVID-19 classification network. It uses InceptionV3, ResNet-50, VGG16 and Xception as the foundation framework. For inputs, images of COVID-19, CAP or other lung conditions are used. The authors used 1194 COVID-19 cases, 264 low-quality COVID-19 cases (for testing purposes only), 2239 pneumonia cases & healthy CT scans in the experimentation. In order to increase the count of training samples, they employed techniques like rotation and zooming based data augmentation procedures. It was concluded that FCONet outperformed other pre-trained models with 96.97% accuracy.

I. COVNet
COVNet uses ResNet50 as a cornerstone to distinguish between COVID-19 & other pulmonary conditions, suggested by Li et al. [43]. 4352 CT thorax scans from 3322 patients were collected for the research. For the detection of COVID-19 class, this model has achieved a sensitivity of 90%, specificity of 96%, and ROC AUC scores of 0.96.

VI. RESEARCH CHALLENGES
Effectively applying deep learning models for helping radiologists in COVID-19 detection has many on-going challenges.

A. Open Source Datasets
The integration of AI & open source datasets provides a realistic solution that can be employed in hospitals worldwide for convenient and accurate COVID-19 diagnosis. However, accessibility of large-scale datasets and likewise, preparing such test data is quite tedious and requires proficient clinical staff.

B. Necessity of early Diagnosis
Timely diagnosis is crucial for the management of rising infections like COVID-19. Although deep learning has been used to represent and learn certain relationships in various sorts of information, it holds promise for applications in exactness medication.

C. Lack of Centralized Cloud-based CT Repository
On the premise of the potency of the deep models for COVID-19 respiratory disorder, a cloud-based open-access computing platform can be made conveniently to help for investigation of COVID-19 respiratory disorder worldwide. CT scan pictures may well be uploaded freely by each clinician and researcher as an assisting tool, particularly in alternative provinces or nations unacquainted with.

D. Similarity between CAP and COVID-19 Pneumonia
COVID-19 pneumonia may have similarity with pneumonia caused by other viral conditions. The instances to have covered non-COVID bacterial pneumonia, viral infection (e.g., viral influenza) and organizing pneumonia due to other causes. It might be appropriate to require a glance at the general performance of deep models in identifying COVID-19 from other cases of pneumonia that may have RT-PCR affirmation of the viral agent during a future study.

E. Feature Pyramid Network Enhancing Object Detection
Three separate deep convolution networks in [44] were trained to classify the CT scan images into COVID-19 or normal. The improved version of ResNet50V2 with a pyramid network feature was the one that achieved the best overall accuracy of all. The system was tested on more than 230 patients and 7796 images. Resnet50V2 with FPN & Xception models performed with 94.96% & 98.02% sensitivity for COVID-19 class, respectively.

VII. CONCLUSION
In this study, many deep learning models in detection of COVID-19 have been explored. Contrasted with that of SARS-CoV and MERS-CoV in 2003 and 2012 respectively, AI advancements have been considerably applied in almost every means of detection of COVID-19. The use of AI in this COVID-19 investigation can be summed up in terms of clinical image examination, drug design and pandemic prediction in the battle against contagious coronavirus. A chest X-ray (CXR) has been one of the major diagnostic imaging techniques for detecting lung tumours for decades. Computed tomography (CT) has garnered significant acclaim in the recent decade for its capacity to create threedimensional (3-D) pictures of the chest, allowing for better delineation of lesions and tumors pathology. CT scan imaging can be used to test patients for COVID-19. It provides a detailed view of a specific region, allowing us to discover interior abnormalities, lesions, part dimensions, cancers, and so on. A CT scan is a more reliable approach than the existing RT-PCR method. It is a useful approach for classifying COVID-19 sufferers' images. The findings are delivered in a detailed and reliable manner. Patients may be exposed to radiation as a result of numerous CT scans, which is one of the adverse effects of CT scan screening. The sooner the ailment is treated, the better prognosis it would show. However, mild symptoms can also result in inaccurate diagnosis of a severe or minor disease. The high falsepositive rate of CT-based screening, which can be as much as 50% depending on the research, contributes to unnecessary subsequent invasive procedures and increased patient concern. Though early diagnosis is clearly advantageous to patients, the relative benefits of CXR versus CT as the best appropriate method of lung cancer screening remains elusive. The study revealed that in a significant number of patients with suspected COVID-19 pneumonia, CT should be investigated following CXR, potentially causing impairment in the absence of pre-defined diagnostic work-up criteria. COVID-19 instances are increasing on a daily basis. The present medical system is collapsing. The automated methods can aid in the early detection of COVID-19 instances and the prevention of the virus transmission. Instead of employing transfer learning techniques, the DNN architectures are built from the ground up. The use of a restricted dataset is a disadvantage of this study. To validate deep learning methods, more COVID-19 and other pulmonary diseases samples are required. This paper has attempted to make a review of deep architectures from literature pertinent to COVID-19 diagnosis. With the availability of more labelled datasets and powerful, efficient deep models, early COVID-19 detection will be profoundly useful in this seemingly long battle.