Abstract

INTRODUCTION

Diabetic kidney disease (DKD) is one of the most common complications of diabetes. The prevalence of DKD in China is about 20% to 40%, and it has become the second cause of endstage renal disease in China, which seriously affects the quality of life in patients with diabetes [1]. At the same time, the treatment of DKD brings a heavy economic burden to patients, their families and the whole society [2]. Clinically, the early diagnosis of DKD is usually based on the level of microalbuminuria. However, once the diagnosis of DKD is made, the kidney has been irreversibly damaged. About one-third of the patients’ condition will progress and deteriorate even if active intervention is carried out [3]. If preventive intervention was carried out for all diabetes patients, it would consume huge human resource and medical expenses. Considering that the risk of DKD in some diabetes patients is not high, to save medical expenses, it is very important to screen out the diabetes patients with high risk of DKD for future preventive intervention, therefore, a smart screen tool is needed.

Scholars used statistical methods such as meta-analysis and regressions to construct risk predictive models of DKDs [4,5]. However, the scoring systems of these models were roughly developed by researchers, and furthermore, they predict the risk of DKD based on the baseline data, without considering the variability through years, resulting in lower accuracy. We used data from Taiwan externally validated a predictive formula published on diabetes care [4], and found that it’s accuracy, precision, and recall were all low (60.9%, 43.2%%, and 53.0%, respectively) [6], indicating that the performance of the formula was not ideal.

Conventional machine learning methods, such as random forest, decision tree, and support vector machine (SVM), were used to develop DKD predictive models for type 2 diabetes mellitus (T2DM) patients, found that the performance of the models based on machine learning technology is obviously superior to the statistical regression based formulas [7,8]. However, these shallow artificial neural networks have limited speed and room for data training. It has to manually select and adjust the features, and are prone to over-fitting phenomenon [9], which show a high level of accuracy/precision in the process of model training, but the accuracy/precision is much lower during the testing process.

In recent years, the development of deep learning technology brings new ideas for accurate prediction of DKD. It can explore the deeper features of data, which has attracted the interest of scholars in research and clinical fields [10]. In the medical field, deep learning techniques can be used to predict the risk of disease, including breast cancer risk prediction [11-13], cardiovascular disease risk [14-16], and risk of acute respiratory disease events and death in smoking patients [17]. In imaging diagnosis, it has been used in the diagnosis and grading of diabetes retinopathy [18-21] and the diagnosis of pulmonary nodules [22]. A large number of studies have confirmed that the prediction and diagnosis tools based on deep learning technology has high accuracy, fast speed, and greatly saves labor cost, showing a strong application prospect [23]. As a type of deep learning technique, long short term memory (LSTM) networks with memory and timing, has unique advantages in fully considering the impact of previous data on subsequent outcomes [24-26].

At present, there is only one study from Japan that uses deep learning technology to build a DKD predictive model for T2DM. They extracted raw features from the previous 6 months as the reference period and selected 24 factors to find time series patterns relating to 6-month DKD aggravation, with predictive accuracy of 71% [27]. A 6-month was rather too short, and to improve the performance of predictive model, data with longer duration of follow-up is needed.

Previous studies have confirmed that the variability of glycosylated hemoglobin (HbA1c), systolic blood pressure (SBP) and pulse pressure (PP) is the key influencing factor of the development and development of DKD [28-30]. However, most current studies did not consider the influence of variability parameters on the outcome when predicting the risk of DKD.

This study intends to develop a deep learning based model to predict risks of DKD in patients with diabetes, and evaluate it’s accuracy, precision and area under the curve (AUC) in predicting the DKD risk. We would also analyze the impact of the variability of HbA1c, SBP and PP on the overall performance of the model.

METHODS

LSTM neural networks building

Pytorch language was used to build the networks. After the LSTM neural networks was set, the data of training set was inputted to the networks, training the LSTM networks, enabling the networks learning the features of the data, and comparing the output results with marked result, as previously described (0 represents did not developed DKD and 1 represents developed DKD). The difference values were propagated back to achieve continuous corrections and adjustments of the neural network parameters, and finally the network model achieves the optimum status. After the model is trained to the optimal state by using the training set data, the data in testing set were used to test the model’s performance.

Dropout was used to prevent the over-fitting phenomenon (the model appears to have a high accuracy in the training process, but have a much lower predictive accuracy in the test process when use the testing data set). We used dropout to solve the problem, and 20% of the neural nerves were randomly removed. The construction process of the structural design of the LSTM neural network prediction model constructed in this study is illustrated in Table 1.

Table 1.

Process of the LSTM prediction model

The construction process is based on the structural design of the LSTM neural network prediction model.
class lstm (nn.Module): #define the network structure
def_init_ (self,input_size,hidden_size,output_size,num_layer):
super (lstm,self)._init_()
self.layer1=nn.LSTM (input_size,hidden_size,num_layer)
self.layer2=nn.Linear (hidden_size,output_size)
self.dropout=nn.Dropout (p=0)
def forward (self,x): #define network output information
x,_=self.layer1(x)
s,b,h=x.size()
x=x.view(s*b,h) #convert lstm’s 3D output to 2D sequences
x=self.layer2(x)
x=self.dropout(x)
x=x.view(s,b,-1)
return F.sigmoid(x)
model=lstm(38,16,1,1) #define of network parameter structure
criterion=nn.MSELoss() #define of loss function
optimizer=torch.optim.Adam(model.parameters(),lr=1e-3) #define optimizer

Open in a separate window

LSTM, long short term memory.

For disease risk prediction models, the smaller the difference between the predicted risk value and the actual disease risk, the more the more prediction accuracy. The loss function is used to measure the gap between the predicted value and the actual value. The most commonly used loss function, mean squared error (MSE), was in this study to calculate the absolute error between the real value and the estimated value. The smaller the value of MSE, the better the prediction value of the model. The functional expression is as follows (1):

$$\mathrm{MSE}=\frac{1}{n}\sum _{i=1}^n{w}_i{\left(y_i-{\hat{y}}_i\right)}^2$$

(1)

RESULTS

Performance comparison between LSTM and SVM models

Based on the 7-year consecutive data of Lee’s United Clinic in Taiwan, the model was trained to use the first 2 years data to predict the DKD risk in next 5 years. According to the inclusion and exclusion criteria, a total of 6,040 T2DM patients were included in the study, and 30% (1,812 cases) of which were divided into the test set. Diabetes patients (n=1,268) did not develop DKD and 544 patients developed DKD.

The prediction performance of LSTM model were: 411 TP, 120 FP, 133 FN, and 1,148 TN, while for the SVM model, 283 TP, 163 FP, 261 FN, and 1,105 TN. Therefore, in this study, the precision of the LSTM model was 411/(411+120)×100%=77%, while the precision of the prediction model constructed based on SVM algorithm was 283/(283+163)×100%=63%, and the precision of the LSTM model was higher (F=128.521, P<0.001). The accuracy of LSTM model was (411+1,148)/(411+1,148+120+133)×100%=86%, much higher than that of SVM model: (283+1,105)/(283+1,105+163+261)×100%=76% (F=71.785, P<0.001). The recall for LSTM model was 411/(411+133)×100%=76%, much better than that of SVM model: 283/(283+261)×100%=52% (F=226.598, P<0.001). Thus, the overall performance of LSTM model superior to that of SVM model (Fig. 1).

Open in a separate window

Fig. 1.

Comparison of precision, accuracy and recall of models constructed by long short term memory (LSTM) and support vector machine (SVM).

In this study, the optimal cutoff point and AUC for LSTM model were 0.55 and 0.83 and which of the SVM model were 0.75 and 0.73 indicating that the performance of the LSTM model was superior to that of SVM algorithm. The ROC curves of the two are shown in Fig. 2.

Open in a separate window

Fig. 2.

Comparison of receiver operating characteristic curves for the long short term memory (LSTM) and support vector machine (SVM) models. AUC, area under the curve.

Comparison of accuracy and AUC area of DKD models

The results show that the best cutoff point of the optimal LSTM prediction model with all variability parameters is 0.55, and the AUC value is 0.83. When we remove the risk factor of HbA1c, SBP, or PP variability one by one, the accuracy of each model were 78% (P<0.001), 79% (P<0.001) and 81% (P<0.001), and the best cutoff point for the DKD risk predictive models were 0.60, 0.55, and 0.60, respectively, with AUC values of 0.72, 0.75, and 0.77, respectively (Fig. 3). The comparison between the model included all HbA1c, SBP, or PP variabilities with other models, which missed one of three variabilities, that the AUC of LSTM model with all variability parameters was significantly better than that of the other three prediction models, all P< 0.05, indicating that the inclusion of variability parameters such as HbA1c, SBP, and PP can significantly increase the overall performance of the model.

Open in a separate window

Fig. 3.

Comparison of area under the curve areas among four diabetic kidney disease prediction models based on long short term memory (LSTM) neural network. PP, pulse pressure; SBP, systolic blood pressure; HbA1c, glycosylated hemoglobin.

DISCUSSION

In this study, we used LSTM neural network developed DKD risk prediction model for diabetes patients, with high accuracy and AUC. At the same time, HbA1c variability, SBP variability and PP variability were added as new risk factors of DKD to develop the DKD risk prediction model and improved the model’s performance.

At present, only one study from Japan successfully constructed the DKD risk prediction model using deep learning technology, with the accuracy of 71% and AUC of 0.743 [27]. The accuracy and AUC value of our study were higher. The following reasons may explain the difference. First, the different deep learning technology made the difference. The Japanese study used convolutional neural network while our study used LSTM neural network, and the latter has unique advantages over convolutional neural network in DKD risk model building, because LSTM has memory and timing concept, fully considering the impact of previous metabolic control on the occurrence of DKD subsequent years [33]. Second, before developing the model, we not only identified the risk factors through systematical reviews physician specialists focus group, but also included HbA1c, SBP, and PP variability as important characteristic parameters in the model for training. Compared with our study, the factors included in this Japanese study were only routine indexes of serum and urine tests of patients in the previous 6 months, such as blood glucose, HbA1c, and blood lipid, and did not fully consider the important influence of fluctuation of HbA1c and blood pressure on the development of DKD. Therefore, the feature parameters included in our study are more comprehensive and have more prediction value, which can improve the overall performance of the prediction model to a certain extent. Finally, our study used 7 years consecutive data while the Japanese study use 6 months data. Since the process of developing DKD is relatively long, involving years, so 6-month of data is not long enough to let DKD happen.

It was also demonstrated that in our study, when HbA1c, SBP, and PP variability were added as risk factors to the characteristic parameters to develop the model, the overall performance was significantly improved. Some studied demonstrated that HbA1c, SBP, and PP variabilities impacted the development of DKD in patients with T2DM [34], but the mechanism is still unclear, which may closely related to oxidative stress, endothelial dysfunction and increased inflammatory factors caused by the fluctuation of HbA1c, SBP, and PP. Oxidative stress, endothelial dysfunction, and increased inflammatory factors are key pathogenic factors leading to kidney tissue damage and impaired renal function leading to the development of DKD [35-39]. Further prospective studies are needed to elucidate the mechanism on the fluctuation of HbA1c, SBP, and PP impacting the development of DKD.

This study also has several limitations. First, the study subjects in this study were from Taiwan, and the factors impact DKD development may be partially influenced by the fact that there is a degree of variation in different regions. Second, the data used to train the model and test the performance of the model were from the same source. They were all from Taiwan Lee’s United Clinic, and this may affect the generalization of the model. So in the future, external validation is needed. Third, the overall performance of the model still has room for improvement; therefore, continues adjustment is warranted.

In conclusion, our study used LSTM neural networks to develop the DKD prediction model with higher performance, as indicated by precision, accuracy, recall, and AUC value. When variability of HbA1c, SBP, and PP was added to the model as featured characteristics, the performance of the model was greatly improved, which suggests that minimizing the variability of HbA1c, SBP, and PP would help to prevent DKD in patients with diabetes. The LSTM based DKD prediction model brings a new idea to screen diabetes patients with high risk of DKD, enabling precise early preventative interventions, so as to prevent DKD economically. The results not only urge patients and health care professionals to consider the important impact of variability of metabolic parameters on DKD development, but also provide new ideas and methods for developing predicting models for other chronic diseases such as diabetic retinopathy.

Acknowledgments

Footnotes

REFERENCES