Unsupported sleeper detection

From the machine model development for detecting unsupported sleepers, the accuracy of each model is shown in Table 4.

From the table, it can be seen that each model performs well. The accuracy of each model is higher than 90% when the data processing is appropriate. CNN performs the best based on its accuracies. When CNN is applied with FFT and padding, the accuracies are the first and second highest compared to other models. For RNN and ResNet, the accuracies are higher than 90% when specific data processing is used. However, the accuracies become 80% approximately when another data processing technique is used. For FCN, data processing is not needed. The FCN model can achieve an accuracy of 95%. From the table, the models with the highest accuracy are CNN, RNN, FCN, and ResNet respectively. The complicated architecture of ResNet does not guarantee the highest accuracy. Moreover, the training time of ResNet (46s/epoch) is the longest followed by RNN (6s/epoch), FCN (2s/epoch), and CNN (1s/epoch) respectively. It can be concluded that the CNN model is the best model to detect supported sleepers in this study because it provides the highest accuracy or 100% while the training time is the lowest. At the same time, easy data processing likes padding is good enough to provide a good result. It is better than FFT in the CNN model which requires longer data processing. The accuracy of testing data of each model is shown in Fig.8.

Accuracies of testing data on unsupported sleeper detection.

The tuned hyperparameters of the CNN model with padding data are shown in Table 5.

Compared to the previous study, Sysyn et al.1 applied statistical methods and KNN which provided the best detection accuracy of 65%. The accuracy of the CNN model developed in this study is significantly higher. It can be assumed that the machine learning techniques used in this study are more powerful than the ones used in the previous study. Moreover, CNN is proven that it is suitable for pattern recognition.

For the unsupported sleeper severity classification, the performance of each model is shown in Table 6.

From the table, it can be seen that the CNN model still performs the best with an accuracy of 92.89% and provides good results with both data processing. However, the accuracies of RNN and ResNet significantly drop when unsuitable data processing is conducted. For example, the accuracy of the RNN model with padding drops to 33.89%. The best performance that RNN can achieve is 71.56% which is the lowest compared to other models. This is because of the limitation of RNN that vanishing gradient occurs when time-series data is too long. In this study, the number of data points for padding data is 1181 which can result in the issue. Therefore, RNN does not perform well. ResNet performs well with an accuracy of 92.42% close to CNN while the accuracy of FCN is fairly well. For the training time, CNN is the fastest model with the training time of 1s/epoch followed by FCN (2s/epoch), RNN (5s/epoch), and ResNet (32s/epoch) respectively. From these, it can be concluded that the CNN model is the best model for unsupported sleeper severity classification in this study. Moreover, it can be concluded that CNN and ResNet are suitable with padding data while RNN is suitable with FFT data. The accuracy of testing data of each model is shown in Fig.9.

Accuracies of testing data on unsupported sleeper severity classification.

The confusion matrix of the CNN model is shown in Table 7.

To clearly demonstrate the performance of each model, precision and recall are shown in Table 8.

From the table, the precisions and recalls of CNN and ResNet are fairly good with values higher than 80% while RNN is the worst. Some precisions of RNN are lower than 60% which cannot be used in realistic situations. CNN seems to be the better model than ResNet because all precisions are higher than 90%. Although some precisions of ResNet are higher than CNN, the precision of class 2 is about 80%. Therefore, the use of the CNN model is better.

For hyperparameter tuning, the tuned hyperparameters of CNN are shown in Table 9.

The rest is here:
Prognostics of unsupported railway sleepers and their severity diagnostics using machine learning | Scientific Reports - Nature.com