The problem of poor alignment and noise in non-lesion area of CXR images which hinders the performance of network is overcome by building three branch attention guided CNN which is discussed in [20]. It helps to identify thorax diseases. Here, AGCNN is explored which works in the same manner as radiologist wherein ResNet50 is the backbone of AGCNN. Radiologist first browse the complete image and then gradually narrows down the focus on small lesion specific region. AGCNN mainly focus on small local region which is disease specific such as in case of Nodule. AGCNN has three branches local branch, global branch, and fusion branch. If lesion region is distributed throughout the image, then the pathologies which were missed by local branch in terms of loss of information such as in case of pneumonia were captured by global branch. Global and local branches are then fuse together to fine tune the CNN before drawing final conclusion. The training of AGCNN is done in different training orders. G_LF (Global branch then Local and Fusion together), GL_F (Global and Local together followed by Fusion), GLF all together, and then G_L_F (Global, Local and Fusion separately) one after another.
Lack of availability of annotated images majorly hinders the performance of deep learning model designed for localization or segmentation [53]. To deal with this issue, a novel loss function is proposed and the conditional random field layer is included in the backbone model of ResNet50 [22] whose last two layers are excluded and weights initialized on ImageNet have been used. In order to make CNN shift invariant, a low pass antialiasing filter as proposed by [73] is inserted prior to down sampling of network. This supports in achieving better accuracy across many models. NIH ChestX-ray14 has been used by the author which have very limited annotated images. Only 984 images with bounding boxes are used for detecting 8 chest pathologies and 11,240 images are having only labels associated with them. Furthermore, chest x-ray dataset is investigated which has many images with uncertain labels. To dispense this issue, a label smoothing regularization [44, 66] is adopted in the ensemble models proposed in [47] which performs averaging of output generated by the pre-trained models, i.e., DenseNet-121, DenseNet-169, DenseNet-201 [25], Inception-ResNet-v2 [64], Xception [12], and NASNetLarge [74]. Instead of ReLU, sigmoid function is utilized as an activation. In addition, label smoothing is applied on uncertain sample images which helped in improving AUC score.
A multiple instance learning (MEL) assures good performance of localization and multi-classification albeit in case of availability of less number of annotated images is discussed in [37]. Latest version of residual network pre-act-ResNet [22] has been employed to correctly locate site of disease. Initially, model is allowed to learn information of all images, namely, class and location. Later, input annotated image is divided into four patches and model is allowed to train for each patch. The learning task becomes a completely supervised problem for an image with bounding box annotation, since the disease mark for each patch can be calculated by the overlap between the patch and the bounding box. The task is formulated as a multiple-instance learning (MIL) problem where at least one patch in the image belongs to that disease. All patches have to be disease-free if there is no illness in the picture.
Considering orientation, rotation and tilting problems of images, hybrid deep learning framework, i.e., VDSNet by combining VGG, data augmentation, and spatial transformer network (STN) with CNN for detection of lung diseases such as asthma, TB, and pneumonia from NIH CXR dataset is presented in [7]. The comparison is performed with CapsNet, vanilla RGB, vanilla gray, and hybrid CNN VGG and result shows that the VDSNet achieved better accuracy of 73% than other models but is time consuming. In [67], a technique of using predefined deep CNN, namely, AlexNet, VGG16, ResNet18, Inception-v3, DenseNet121 with weights either initialized from ImageNet dataset or initialized with random values from scratch is adopted for classification of chest radiographs into normal and abnormal class. Pretrained weights of ImageNet performed better than random initialized weights from scratch. Deeper CNN works better for detection or segmentation kind of task rather than binary classification. ResNet outperformed training from scratch for moderate sized dataset (example, 8,500 rather than 18,000).
A customized U-NET–based CNN model is developed in [8] for the detection and localization of cardiomegaly which is one of the 14 pathologies of thorax region. To perform the experimentation ChestX-ray8 database was used which consist of 1,010 images of cardiomegaly. Modified (Low Contrast) Adaptive Histogram Equalization (LC-AHE) was applied to enhance the feature of image or sharpen the image. Brightness of low intensity pixel of small selected region is amplified from the intensities of all neighbouring pixels which sharpens the low intensity regions of given image. Considering the medical fact that the Cardiomegaly can be easily located just by observing significant thickening of cardiac ventricular walls, authors developed their own customized mask to locate it and separated out that infected region as image. This helped in achieving an accuracy of 93% which is better than VGG16, VGG19, and ResNet models.
Thoracic pathology detection not only restricted from CXR images but can also be done from video data of lung sonography. Deep learning approach for detection of COVID-19–related pathologies from Lung Ultrasonography is developed in [51]. By applying the facts that the augmented Lung Ultrasound (LUS) images improve the performance of network [62] in detecting healthy and ill patient and keeping consistencies in perturbed and original images, hence robust and more generalized network can be constructed [52, 55]. To do so, Regularized Spatial Transformer Network (Reg-STN) is developed. Later, CNN and spatial transformer network (STN) are jointly trained using ADAMS optimizer. Network lung sonography videos of 35 patients from various clinical centers from Italy were captured and then divided into 58,924 frames. The localization of COVID-19 pathologies were detected through STN which is based on the concept that the pathologies are located in a very small portion of image therefore no need to consider complete image.
A three-layer Fusion High Resolution Network (FHRNet) has been applied for feature extraction and fusion CNN is adopted for classifying pathologies in CXR is presented in [26]. FHRNet helped in reducing noise and highlighting lung region. Moreover, FHRN has three branches: local feature extraction, global feature extraction, and feature fusion module wherein local and global feature extraction network finds probabilities of one of the 14 classes. Input given to local feature extractor is a small lung region obtained by applying mask generated from global feature extractor. Two HRNets are adjusted to obtain prominent feature from lung region and whole image. HRNet is connected to global feature extraction layer through feature fusion layer having SoftMax classifier at the end which helps in classifying input image into one of the 14 pathologies. Another deep CNN consisting of 121 layer is developed to detect 5 different chest pathologies: Consolidation, Mass, Pneumonia, Nodule, and Atelectasis [43] entropy, as a loss function is utilized and achieved better AUC-ROC values for Pneumonia, Nodule, and Atelactasis than the model by Wang et al. [70].
Recently, due to cataclysmic outbreak of COVID-19, it is found by researchers that, as time passes, the lesions due to infection of virus spread more and more. The biggest challenge at such situations, however, is that it takes a lot of valuable time and the presence of medical specialists in the field to analyse each X-ray picture and extract important findings. Software assistance is therefore necessary for medical practitioners to help identify COVID-19 cases with X-ray images. Therefore, researchers have tried their expertise to design deep learning models on the data shared word wide to identify different perspective of spread of this virus in the chest. Many authors have created augmented data due to unavailability of enough CXRs images and applied deep learning models for detecting pneumonia caused due to COVID virus and other virus induced pneumonia. Author designed a new two-stage based deep learning model to detect COVID-induced pneumonia [31]. At the first stage, clinical CXR are given as input to ResNet50 deep network architecture for classification into virus induced pneumonia, bacterial pneumonia and normal cases. In addition, as COVID-19–induced pneumonia is due to virus, all identified cases of viral pneumonia are therefore differentiated with ResNet101 deep network architecture in the second stage, thus classifying the input image into COVID-19–induced pneumonia and other viral pneumonia. This two-stage strategy is intended to provide a fast, rational, and consistent computer-aided solution. A binary classifier model is proposed for classifying CXR image into COVID and non-COVID category along with multiclass model for COVID, non-COVID, and pneumonia classes in [45]. Authors adopted DarkNet model
