2.1. Noise Reduction

The PCG signal collected is generally composed of the heart sound and noise. The noise is generally divided into ambient noise and the noise of the instrument itself. Ambient noise is the noise in the environment in which the heart sounds are collected, and also contains respiratory sound. The noise generated when the instrument rubs against the skin is also an environmental noise. The instrument itself can also generate noise when collecting heart sounds due to the characteristics of its internal electronics. Ambient noise can be avoided artificially, instrument noise can also be reduced by improved devices, but it is still impossible to remove 100%, so a certain denoising step is required to make the useful components more prominent. PCG denoising essentially involves looking for a suitable filter to separate the wanted signal from the noise to provide a denoising effect. At present, the commonly used heart sound denoising methods include the empirical mode decomposition (EMD) denoising method, Hilbert yellow transform denoising method, and the wavelet denoising method [1,2,3].

2.2. Segmentation

The beat cycle segmentation is a very important part of the PCG automatic diagnosis research based on the feature extraction machine learning model. There are four main types of methods for heart sound segmentation [4]. (1) Envelope-based method—usually uses various techniques to construct an envelope of a heart sound signal, thereby performing heart sound segmentation. Typical methods for constructing envelopes include Shannon energy [5,6], Hilbert transform [7], extraction of heart sound signature waveforms [8], and square energy [9]. (2) Feature-based approach—this method requires the calculation of heart sound features and then segmentation decisions. Typical features include amplitude–frequency features [10,11], phase features [12], periodic component features [13], complexity-based features [14], multilevel wavelet coefficient features [15], and higher-order statistics such as kurtosis [16]. (3) The machine learning-based approach uses neural networks or other nonlinear classifiers that utilize extracted features, for example, a multilayer perceptron neural network [17], K-means clustering [18], dynamic clustering [19], and unsupervised learning methods [20]. (4) Hidden Markov Model (HMM) were originally proposed by Gamero and Watrous [21] in the context of heart sounds. Ricke et al. [22] subsequently developed this method. Then, Gill et al. incorporated timing duration and homomorphic filtering into the HMM method to improve the segmentation accuracy [23,24], and Schmidt et al. extended HMM to a hidden semi-Markov model (HSMM) by modeling the expected duration of heart sounds [25]. Springer et al. [26] further developed the HSMM method by using additional input features, logistic regression, and an improved Viterbi algorithm, which solved the boundary conditions and improved the overall segmentation of short-term heart sound recording.

2.3. Feature Extraction

The features that the PCG can extract include time-domain features such as mean, variance, area, and amplitude, and various frequency domain features, such as the early-time short-time Fourier transform (STFT) [27,28]. The works use STFT to construct a time-varying transfer function and a time-varying coherence function that relate the heart and chest heart sounds; Ref. [28] used STFT to identify congenital heart disease such as atrial septal defects, ventricular septal defects, and arteriovenous catheterization. STFT reduces the signal stability requirements, is easy to implement, and is easy to calculate. However, its time–frequency window is fixed in the entire time–frequency plane, which cannot meet the requirements of different window functions for different components. Wavelet transform can realize the time-varying spectrum analysis of signals [29], and can analyze the details of the signal in more detail, which is the most widely used heart sound analysis method. In addition, empirical mode decomposition, Mel frequency cepstral coefficients (MFCC), linear predictive cepstral coefficients, etc., can also be used for PCG feature extraction [30,31,32].

3.1. PCG Noise Reduction and Removal of Spikes

There is also some noise in the PCG signal, which is inevitable during the acquisition process. Like the ECG signal, the PCG noise can be removed to some extent by the Butterworth filter. Since the frequency of the PCG signal in the data set is 0–2000 Hz, most single-sample PCG signals are also longer in length, so downsampling is first performed to reduce the frequency to below 1000 Hz. The Butterworth band-stop filter is then used to remove the low frequencies in the signal, i.e., to remove signals below 25 Hz and signals above 400 Hz, which ensures that important information is substantially preserved while effectively removing most of the noise.

The PCG signal may have some signal spikes that interfere with the model classification. The algorithm for removing signal spikes mentioned by Schmidt in [25] is mainly for PCG signals. The peak signal in the data set used in this experiment appears less, but since the time complexity of this algorithm for removing spikes is small, the actual experiment does not take up significant time, and it is also reserved.

3.2. PCG Signal Segmentation

There are many mature PCG signal segmentation methods. Chen et al. [38] mainly use K-means cluster analysis for PCG segmentation [39] and use neural networks to segment Fundamental Heart Sounds (FHS) between S1 and S2. Because of the PCG dataset used in this chapter, some samples contain ECG data collected at the same time, and the segmentation effect of ECG signals is very good and the accuracy is extremely high. Therefore, the semihierarchical Markov chain segmentation method proposed in [26] is adopted. After the ECG signal is segmented, the corresponding point is migrated to the corresponding PCG signal as a training set. Figure 2 is a diagram of the heart sound signal and the ECG signal corresponding to the same heart beat information.

Open in a separate window

Figure 2

Waveform of PCG.

The specific steps are as follows:

• (1)

  The signal is divided into 500 ms per window.

• (2)

  Find the maximum absolute amplitude (MAA) in each window.

• (3)

  If at least one MAA exceeds three times the median of all window MAAs, perform the following steps. If not, go to step (4).

  1. Select the window with the largest MAA value.

  2. In the selected window, the position of the MAA point is defined as the top of the noise spike.

  3. The beginning of the noise spike is defined as the last zero crossing point before the MAA point.

  4. The end of the peak is defined as the first zero crossing point after the maximum point.

  5. The defined noise spike is set to zero.

  6. Go back to step (2) and continue.

• (4)

  The program is complete.

In the data set used in the experiment in this paper, the occurrence of spike signals is less frequent, and because the time complexity of the spike removal algorithm is still small, it is also reserved.

3.3. Filter Feature Extraction

The deep learning algorithm itself has its own feature extraction capability, which can directly feed the continuous state samples that are related before and after. After the ECG signal is preprocessed, some continuous transformations can be performed to convert the signal into other states that can be interpreted, such as spectral transform. Through these feature transformations, it is possible to capture the feature information implied by the original information into other spaces or at other levels, while maintaining the continuity of the features themselves.

Through the deformation of the heart sound signal, this paper mainly extracts three types of continuous feature models for deep learning. The methods adopted are homomorphic filtering, Hilbert transform, and power spectral density (PSD) estimation.

• a.

  Homomorphic filtering is often used for image processing and is a kind of frequency domain filtering. However, homomorphic filtering has its own advantages. Frequency domain filtering can flexibly solve the additive noise problem, but it cannot reduce multiplicative or convolution noise. In order to separate the additive combination signal, the linear filtering method is often used. The nonadditive signal combination commonly uses the homomorphic filtering technique to convert the nonlinear problem into a linear problem processing, that is, first to nonlinear (multiplicative or convolution). The mixed signal is a mathematical operation that is transformed into additive. Then, it is processed by the linear filtering method, and finally the inverse transform operation is performed to restore the processed image.

In image processing, it is often found that the dynamic range is large but the details of the dark area are unclear. We hope to enhance the dark area details without losing the details of the bright area. In general, we can image f(x, y) modeled as the product of the illumination intensity i(x, y) and the reflection intensity r(x, y):

In general, the illumination of natural pictures is uniform, so i should be a low-frequency component, and the reflection of light by different objects has abrupt changes, so r is a high-frequency component. Now take the logarithm of the two sides and perform the Fourier transform to get the frequency domain of the linear combination:

After the operation is completed, the inverse transformation is performed, and the processed image is obtained by exponentiation. Using this for heart sounds can reduce the low-frequency detail and pull up the high frequency.

• b.

  Hilbert transform is a common method in signal processing. It has a real-valued function x(t) whose Hilbert transform is denoted as $\widehat{x}\left(t\right)$ or H[x(t)] as follows:

  $$x\left(t\right)=H\left[x\left(t\right)\right]=-\frac{1}{\pi }\underset{-\infty }{\overset{\infty }{{\displaystyle \int }}}\frac{x\left(\tau \right)}{t-\tau }d\tau$$

  (4)

In contrast to the concept of convolution, it can be found that the expression of the Hilbert transform given above is actually the result of convolving the original signal with a signal. Therefore, the Hilbert transform can be thought of as passing the original signal through a filter, or a system whose impulse response is H(t). Performing a Fourier transform on H(jω), one can get the following:

• c.

  Power spectral density estimation:

  $$P_X\left(e^{j\omega }\right)={\displaystyle \sum }_{m=-\infty }^{+\infty }{r}_X\left(m\right)e^{-j\omega m}$$

  (6)

  where r_x (m) is the autocorrelation function of the random signal. The power spectrum reflects that the power of the signal is distributed in the frequency domain with frequency ω, so P_x (e^jω) is also called the power density:

  $${\widehat{P}}_{AC}\left(\omega \right)={\displaystyle \sum }_{m=-M}^M\widehat{r}\left(m\right)e^{-j\omega m}, \left|M\right|\le N-1$$

  (7)

Because the power spectrum obtained by this method is obtained indirectly through the autocorrelation function, it is called an indirect method, also known as an Auto-Correlation (AC) method. When M is small, this formula is not very large, so this method was a commonly used spectral estimation method before the FFT was introduced (that is, before the periodogram method was widely used).

3.4. Transfer Learning LKNet

This paper proposes a transfer learning architecture with a large kernel network (LKNet) as shown in Figure 3. LKNet is a large-size convolution network based on a residual network, which is called transfer learning LKNet. Because ECG data are more common and has cardiac activity related data with PCG, this algorithm uses a transfer learning method to migrate knowledge from the ECG to the PCG network.

Open in a separate window

Figure 3

The transfer learning of large kernel network LKNet processes from ECG to PCG.

Transfer learning is to move the knowledge of one domain (i.e., the source domain) to another domain (i.e., the target domain), so that the target domain can achieve better learning results. Usually, the amount of data in the source domain is sufficient, and the amount of data in the target domain is less. This scenario is very suitable for migration learning. For example, we have to classify a task, but the data in this task are not sufficient (target domain), although there is a large amount of relevant training data (source domain); these training data are different from the test data of feature distribution in the classification task that needs to be performed. In this case, if the appropriate transfer learning method can be adopted, the sample is not sufficiently improved.

Transfer learning is widely used in the field of image classification. Usually, a super-large neural network that has been trained is regarded as a source domain, and a model network specific to small sample image data is used as a target domain. At this time, transfer learning is more effective; for example, ResNet101 trained in the ImageNet dataset is used to classify the small sample cat and dog image data collected by itself. Generally speaking, the more source domain samples there are, the fewer target domains there are relative to the source domain. The more similar the samples are, the more obvious the improvement in accuracy is compared to the migration of the target field samples.

The PCG is stored by conversion into audio, and the ECG is stored by conversion into an electrical signal. These two signals, however, are strictly different signals in different fields, which mean that the similarity is not high enough to be reminiscent of migration learning. Nevertheless, under the existing conditions, through the analysis of the database, the ECG data are relatively clean, the classification effect is very good, and the ECG data sample size is several times that of the PCG, and then the combination of their collection source and the target are the characteristics of the heartbeat. In order to explore the commonality between ECG and PCG signals in deep learning, and the generalization of the improved model LKNet, the migration learning experiment was conducted.

As shown in Figure 3, the specific process follows:

• (1)

  From the Normal, Atrial Fibrillation (AF) Abnormal, Noise, and Other types of data samples in the ECG database, Noise and Others are removed, and the retained Normal and AF are exactly the same as the sample categories in the PCG database.

• (2)

  The ECG data are first input to Full Connected (FC), than it is trained by LKNet, and eight blocks are used to obtain a source network with an accuracy of 90% or more.

• (3)

  The last four of the eight blocks in the source network are removed, then the first four are fixed, and eight blocks are added to form a new network for training PCG data.

• (4)

  Figure 3 shows the process for the PCG migration from ECG data.

4.2. Comparison of Different Segmentation

Based on the similarity of data structures between PCG and ECG, and the similarity of the purpose of disease diagnosis based on them, the deep learning model classification part of this work directly uses LKNet. The operation of the sliding window segmentation method previously proposed for the ECG signal does not bring about a significant change in accuracy. The Table 1 shows the results of the experimental classification.

When the stride and the window are equal, the sliding segmentation becomes a fixed segmentation. The table below shows the results of the window segmentation, fixed segmentation, RR interval segmentation and nonsegmentation. The final results of the three methods of segmentation and the nonsegmentation are almost the same, and the effect of sliding window segmentation is obviously better. Since the length of most PCG signal samples does not exceed 1 min, the window is set to 10 s. The best Mean Accuracy of Percentage (MAcc%) is 79.25 for sliding segmentation with stride 1 and window(s) 2.

4.3. Comparison of Single Classification Method

In this subsection, the classification methods with single features compared in this experimental method are Random Forest (RF), Support Vector Machine (SVM), and Gradient Boosting Decision Tree (GBDT), which are the current mainstream machine learning methods. At present, the mainstream deep learning algorithm is the Convolution Neural Network (CNN). Our proposed Large Kernel Network (LKNet) has several versions of improvement, such as LKNet with Homo, Hilbert, Power Spectral Density (PSD), LKNet with ensemble SVM, and LKNet with transfer learning. Among them, transfer learning gives the best performance. The detailed experimental results are shown in Table 2.

It can be seen from Table 2 that, after the PCG signal undergoes homomorphic filtering, Hilbert transform, and PSD estimation, the extracted feature is added to LKNet and the MAcc value is 80.16, which is 1% higher than the original signal input. The addition of transfer learning has increased the model classification results by more than two percentage points. Both PCG signal transformation and transfer learning can bring about an improvement in the model classification accuracy. The traditional machine learning method linear SVM used in this paper has relatively low precision, although many features have been extracted through feature engineering. If we only consider the single module, our transfer learning LKNet has achieved relatively good results.