DenseNet-CNN Architectural Model for Detection of Abnormality of Acute Pulmonary Edema

-Acute pulmonary edema (EPA) is a condition of emergency respiratory distress that results from the sudden and rapid build-up of fluid into the lungs. Rapid screening of EPA patients is necessary so that radiologists can make the prognosis as early as possible. In addition, reliance on the expert’s knowledge of reasoning also hinders the diagnostic process. This research was conducted by developing an architectural model for machine learning systems with a deep learning approach. With the concept of representative learning, the denseNet-CNN algorithm connects each layer to another by means of a feed-forward. The data used is Image CXR-14 specifically labeled pulmonary edema pathology. The size of each CXR-14 image is 1024 × 1024 with a value of 8 bits grayscale. The size of each CXR-14 image is 1024 × 1024 with a value of 8 bits grayscale. The architectural model development stages consist of the preparation stage, data resampling, data training and data testing. Optimizer parameters used are Adam’s optimizer, learning rate of 0.0001 and weight decay = 1e-5 and the loss used is binary cross entropy. The resulting mean AUROC analysis showed the sensitivity value of the 10% dataset was 71.493% and the specificity value of 10.011% was obtained at the second hold of the k-fold cross validation method after holdout validation, so that the resulting model was valid. The detection system developed from the denseNet-CNN model is expected to help radiologists identify abnormalities in CXR images quickly, precisely, and consistently. The denseNet-CNN model is also developed in the form of a heatmap visualization by localizing the features you are looking for. With localization in the form of a heat map, detection of pathological abnormalities of PEA is easier to do and to be recognized.


Introduction
Acute pulmonary edema (EPA) is a condition of urgency characterized by a rapid respiratory emergency caused by displacement and accumulation of fluid in the lungs. According to the source of the cause, EPA is divided into two types, namely cardiogenic EPA and noncardiogenic EPA. The high prevalence rate of EPA can be seen from a study involving approximately 600 hospitals in Europe, Latin America, and Australia. Data show EPA is present in 37% of patients with acute heart failure [1] [2].
Fast and precise screening for patients with EPA cases is needed so that doctors can make the prognosis as early as possible. One of the initial screenings performed on EPA patients is by doing a chest X-ray examination, known as ChestX-Ray (CXR) screening. Generally, the reading and diagnosis of CXR images is performed by a radiologist by comparing the CXR images of EPA patients with normal CXR images to detect abnormalities [3 [4].
Dependence on the knowledge of an expert radiologist regarding the principles of anatomy, physiology, and pathology is a factor that can hinder making a diagnosis as early as possible [5]. Another difficulty in the process of detecting CXR images is the difficulty of expert radiologists in developing consistent reasoning techniques in reading CXR images while considering all common chest diseases that require a long time to diagnose a CXR image [6].
One solution to solve this problem is to develop a machine learning system. Amit Kumar Jaiswal's research, entitled Identifying Pneumonia in Chest X-Rays: A Deep Learning Approach uses the Mask-RCNN method, used a deep neural network that combines global and local features for pixel-based segmentation. The identification model proposed in the study achieved reasonably good performance after evaluating a dataset of chest radiographs depicting potential causes of pneumonia [7].
The approach recommended in this study is to use deep learning (DL) convolutional neural network models. The DL approach adopts the way the human brain works in managing data as representative learning which is then classified into layers. Research by Huang, Liu, Weinberger, & van der Maaten conducted in 2017 shows the importance of layer depth, better accuracy, and efficient training will be achieved if CNN has a closer connection between the layers that are close to the input and the layers that are close to it. with output [8]. DenseNet has an architecture that connects each layer to another in a feed-forward manner. Densenet itself has several advantages, namely: reducing the vanishing-gradient problem, strengthening feature propagation, reusing features, and reducing the number of parameters [9][10] [11].
The computer-assisted detection system is expected to help radiologists identify abnormalities in CXR images quickly, precisely, and consistently. Computers can be taught to read and process a very large number of CXR image scans in a short amount of time. to confirm the results found by the radiologist and potentially identify other findings that may have been found. The resulting model is an artificial intelligence mechanism that can direct radiologists to make better diagnostic decisions for patients.

a. Data collection
The CXR images used in the CNN architectural model developed in the study were obtained from the public dataset of NIH Clinical Center, a clinical research hospital for the National Institutes of Health based in Maryland -United States, called ChestX-ray14 . This public dataset is the largest CXR dataset available to the public, containing 112,120 anonymous CXR front view images derived from 30,805 patients including patients with advanced lung disease. Each dataset summary page also contains the license terms and citation requirements which can be accessed on TCIA datasets from Cloud Storage, BigQuery, or by using the NIH Clinical Center's Cloud Healthcare API [12] [13]. One of the CXR-14 images used is shown in Figure 1.  The number of positive labels for pulmonary edema was 2,303 with the assumption that there would be redundancy of patient data between existing pathology labels.

b. DenseNet CNN Procedure
The proposed denseNet-CNN model development procedure is as follows: Step 1: The preparation stage In this preparation stage, analysis is carried out first to adjust the Deep Learning Framework to be applied with the availability of hardware in conducting training, validation and testing of the CNN model to be developed, as for the specifications of the hardware to be used are as follows: - Step 2: Re-Sampling Data Modifications to the algorithm applied to the CXR-14 PEA dataset resulted in an underfit model due to imbalance data, therefore re-sampling the CXR-14 dataset, in order to provide valid results. The process of re-sampling this dataset begins with training on less data, which in this study was determined as much as 10% of the total dataset, which includes: all data with a positive PE label found on CXR-14, and the rest randomly chooses data labeled PE negative. [14] Re-sampling the dataset obtained is separated systematically using Python coding into training data, validation data and testing data. Resampling Dataset for Hold-out validation is illustrated in Figur 2 & 3.  Step 3: Training Stage Furthermore, experiments and validations were carried out on the CheXNet scripting sources found on the github repository portal, this study determines the implementation of the PyTorch DL Framework made by Andrey G. (zoogzog) on the link https://github.com/ zoogzog/chexnet . Referring to the selected scripting, some environmental elements for the DL framework need to be prepared first, namely using: [12] -Operating System: Ubuntu 17.10 -Compilers: C (GCC) 7.2.0 -CUDA compilation tools: NVIDIA CUDA, V8.0.61 Experiments were carried out by replicating Python coding in the PyTorch environment that implements training and validation functions so that the model is in the form of a file that will be used in the testing stage. The optimizer used is Adam's algorithm as a standard parameter that can produce output quickly. The learning rate used is 0.0001, the weight decay is 1e-5, and the loss used is the cross entropy loss.
Step 4: Testing Phase Furthermore, the testing process is carried out to produce AUROC performance against the PEA pathology contained in CXR-14. The distribution of the amount of training, validation, and testing data used in scripping for the DL framework is shown in table 2. Step 5: The resulting output The AUROC performance obtained at this preparation stage is then compared with the results obtained by Rajpurkar et.al., in their journals as a reference in proceeding to the research implementation stage, as shown in table 3 [15].

CNN's DenseNet Architecture Model Modifications
The architectural model developed using the DenseNet CNN algorithm which has advantages in implementing the depth of a layer, better accuracy, and efficient training will be achieved if CNN has a closer connection between the layer close to the input and the layer close to the output. Densenet CNN has an architecture that connects each layer to another by means of feed-forward [16][17] [18].
The modification of CNN's DenseNet architectural model in this study can be seen in Figure 4. Generally, the CNN architecture consists of the input layer, hidden layer and output layer stages. CNN's DenseNet architecture above by skipping processes in hidden layers. The effect of the skip hidden layer is that the loss value from the previous layer can be carried to the next network so that the loss after the skip process is a combination of loss with the loss brought before by the skip connection. In this study, the skip connection added to the CNN architecture with the DenseNet algorithm can minimize the vanishing-gradient problem so that feature propagation can be strengthened, repetitive features on features, and reduces the use of parameters in the architectural model being developed.
Adam Optimizer was chosen because it is the most popular algorithm and it can produce output faster and better than other methods. The optimization done by Adam can be used as a substitute for the classic stochastic gradient descent procedure so that it can update the network weight repetitively on features based on training data. The stochastic gradient descent function maintains one learning rate for all weight updates used, where the learning rate will not change during the training process. [19] [20] The learning rate used is 0.0001. The learning rate is maintained for each network weight (parameter) and is adapted separately as learning unfolds. With Adam's algorithm it is also possible to calculate individual adaptive learning rates for parameters different from the predicted first and second moments of the gradient. [21] In the classification layer, a full connected softmax and a global average pool will be generated where the diagnoses of how many abnormalities are detected in the CXR-14 heatmap image.

Result a. Holdout Validation at 10% Chest X-ray 14 Dataset
The distribution of the dataset for this method is carried out in accordance with the value of the distribution of training, validation and testing data shown in table 4, with the method scheme illustrated in Figure 5 with a value of x = 10. The training is carried out using the same Python programming as the 100% dataset training. The best results of modeling the three holdout validation methods at 10% of the ChestX-ray14 dataset are shown in table 4. The best mean AUROC value achieved is 0.9114 with Binary Cross Entropy (BCELoss) between the target and the output is 0.3238 which is obtained from the 90-division. 10. , determining the amount of training data, validation and schematic testing is shown in Figure 5. Graphic images of BCELoss on each batch of epoch validation are shown in Figure 6.
The results of model accuracy with three hold-out validation methods at 10% of the CXR-14 dataset are shown in table 5.  The number of positive PE data labels and negative PE data labels on the 90-10 holdout validation method at 10% of the ChestX-ray14 dataset is mentioned in table 6.

b. Application of the K-fold cross validation over holdout validation method on 10% of the ChestX-ray 14
Furthermore, the K-fold cross validation over holdout validation method is applied, with a value of k = 10 (10 fold) based on the best 10% ChestX-ray14 dataset that has been obtained from the holdout validation method previously carried out, namely in the 90-10 division, determining the amount of data. training, validation and schematic testing are shown in Figure 7.
The results obtained from the modeling method of 10 fold cross validation over holdout validation on 10% of the ChestX-ray14 dataset are shown in Table 7. The best mean AUROC produced was 0.9164, with a BCELoss value of 0.3167, which was achieved by the 2nd fold in the 42nd batch epoch validation of the 50 defined epoch batches. Graphic of BCELoss achievement in each epoch batch is shown in Figure 8.

c. AUROC analysis of 10% ChestX-ray 14 dataset
The area under receiver operating characteristics (AUROC) is a performance metric that you can use to evaluate a classification model. The analysis was to determine the sensitivity of the best model produced, namely the second fold of the k-fold cross validation method after holdout validation. The calculation of the mean AUROC is shown in Figure 9 and a slice of the output dataset is shown in Table 8.  From the calculation results, it is known that the sensitivity value has increased significantly so that it is above the cut-off value, with compensation there is an increase in the specificity value, which means that the resulting model has increased the likelihood of getting a true positive value when detecting a CXR-14 image with a PE label. positive is better, although there is a decrease in the confidence value of the negative PEA label training results.
With the results of the analysis obtained, it is concluded that the resulting model is valid, with a better sensitivity value obtained from the 10% dataset of 71.493% and a specificity value of 10.011%. The comparison of the model test results is shown in table 9. The research continues to the next stage by bringing the best model file obtained, from modeling 80-20 holdout validation on 10% of the CXR-14 dataset as a model for testing CXR input in detecting PEA pathology by displaying a heat map to visualize areas on CXR-14, where disease is present, by using the Class Activation Mapping. Implementing this application can assist medical personnel in localizing the features they looking for.

Conclusion
The valid model was obtained in the 6th modification of the second k-fold with a mean AUROC value of 0.9164, a better sensitivity value was obtained from the 10% dataset of 71.493% and a specificity value of 10.011%. Optimizer parameters used are Adam's optimizer, learning rate of 0.0001 and weight decay = 1e-5 and the loss used is binary cross entropy.
The denseNet-CNN model is also developed in the form of a heatmap visualization by localizing the features to be searched. The architecture is able to recognize predictive information with localization in the form of a heat map. This makes easier to detect and recognize PEA pathologic abnormalities.