The experimental arrangement in MMD is a prime factor that defines the integrity of the dataset. The dataset is obtained in lab environment with a PI sensitive coil made up of muti-stranded wire with coil diameter of 170mm. It is mounted on a transparent acrylic sheet with a miniaturized Tx/Rx (also mounted) at a distance of 100mm. The electromagnetic field (EMF) simulation of search-head in close proximity of mine is shown in Fig.7. The received signal is digitized, and synchronized data is obtained for both the transmitted positive and negative pulses. The dataset is then populated with this synchronized pulse data. The pulse repetition frequency, including both pulses, is 880Hz.The number of pulses M (refer to Eq.(1)) obtained per class is 1330, representing concatenated positive and negative pulses. It is done to simplify the model, with a total number of concatenated samples being N=244, consisting of 122 samples from each received pulse, respectively. It is approximately 3s of pulsed data per class.

Shows Electromagnetic field simulation of search head in (a) and search head in proximity of mine in (b).

The samples/targets used to represent the nine classes (previously discussed) include minrl/brick (mineralized soil), sand (non-mineralized soil), APM (standard 0.2 gm) and vertical paper pins (0.2 gm). Mineralization is an indication of magnetic permeability (or susceptibility) of the surface soils that have been exposed to high temperatures and heavy rainfall or water for extended periods of time, often exhibit high mineralization due to the presence of residual iron components. For an in-depth exploration of the magnetic susceptibility across a wide range of soil types, you can find comprehensive information in reference18. The choice of using brick, a clay-based material, as a representative sample for mineralized soil is grounded in its unique composition. It contains minerals like iron oxide, such as magnetite or hematite, and exhibits relatively low electrical conductivity19. These distinctive characteristics significantly enhance its detectable response when subjected to a MMD. In fact, this response is typically more robust than that of conventional mineralized soil (from which it originates) or even APM. For the sake of simplicity and consistency, we will refer to this material as "minrl" throughout this paper.

All of the targets mentioned pose their own challenges, but they are placed in close proximity to the MMD, within a distance of no more than 20mm parallel to the surface of the coil. The targets are positioned at the center of the coil. The received signals from different target samples of a positive and a negative transmitted pulses can be observed in Figs. 8 and 9 respectively. The figures display a magnified section of the received signal, focusing on the initial samples that are more strongly influenced by the secondary magnetic field compared to later samples. It can also be seen that signals vary in opposite directions as per polarity of the transmitted pulses.

Received signals of a positive transmitted pulse picked up at the sensor coil from the secondary magnetic field produced by the eddy currents induced within the targets. The x-axis shows few numbers of samples (initial part of the signal) per pulse and y-axis shows amplitude of the signal in volts. Signals from nine targets air, APM, pins, minrl, minrl+APM, minrl+pins, sand, sand+APM and sand+pins have been shown.

Received signals of a negative transmitted pulse picked up at the sensor coil from the secondary magnetic field produced by the eddy currents induced within the targets. The x-axis shows few numbers of samples (initial part of the signal) per pulse and y-axis shows amplitude of the signal in volts. Signals from nine targets air, APM, pins, minrl, minrl+APM, minrl+pins, sand, sand+APM and sand+pins have been shown.

The overall dataset comprises a total of 11,970 pulses, representing nine different classes. The dataset is sufficiently diverse, as illustrated in Fig.10 by examining inter-class distances. For this analysis, two distances are employed: Euclidean distance, which measures point-to-point distance, and Bhattacharyya distance, a metric indicating dissimilarity between two probability distributions. Two cases will be briefly discussed here: one involving the Euclidean distance between air and pins, where the maximum distance is observed as depicted in Fig.10, which is also evident in the received signal shown in Figs. 8 and 9. The second case pertains to the Bhattacharyya distance between air and sand, illustrating minimal dissimilarity. The impact of this dissimilarity will become evident in the overall results. To prepare this dataset for modelling, these pulses are randomly shuffled and subsequently split into two separate sets: a training dataset containing 10,773 pulses and a validation dataset comprising 1197 pulses.

Shows inter-class similarity through Euclidean and Bhattacharyya distances.

During the model training phase, input data is structured as a matrix with dimensions [10,773244], and the output, following a supervised learning approach, is provided as a one-hot encoded labeled matrix with dimensions [10,7739]. The accuracy of the trained model on the provided data is tracked across multiple epochs, including both training and validation accuracy. In the context of this training process, one epoch signifies a complete iteration over the entire training dataset of size [10,773244], with all training samples processed by the model. Figure11 depicts the trend, showing that as the training process repeats over multiple epochs, the model steadily enhances its performance and optimizes its parameters. After 4000 epochs, the trained accuracy reaches approximately 98%, while the validation accuracy hovers above 93%. It also shows that the DL-MMD model has more or less converged at 4000epochs, by achieving the optimum training performance. Likewise, its evident that the models error loss diminishes with the progression of epochs, as illustrated in Fig.12.

Shows the accuracy and validation accuracy of novel DL-MMD model versus epochs. For comparison, the validation accuracy of KNN and SVM classifier are also shown for k=8 and C=100 respectively.

Shows the loss and validation loss of novel DL-MMD model versus epochs.

Figure11, also shows that the presented model performs substantially better compared to support vector machine (SVM) and K-Nearest Neighbors (KNN) classifiers. The main working principle of SVM is to separate several classes in the training set with a surface that maximizes the margin (decision boundary) between them. It uses Structural Risk Minimization principle (SRM) that allows the minimization of a bound on the generalization error20. SVM model used in this research achieved a training accuracy of 93.6% and a validation accuracy of 86.5%, which is far lower than the performance achieved by the presented model. The parameter for kernel function used is the most popular i.e. radial basis function (RBF) and the value of regularization parameter c optimally selected is 100. The regularization parameter controls the trade-off between classifying the training data correctly and the smoothness of the decision boundary. Figure13 shows the influence of the regularization parameter c, on the performance of the classifier. The gamma is automatically calculated based on the inverse of the number of features, which ensures that each feature contributes equally to the decision boundary. The hyperparameter optimization is achieved through a manual grid search method. The code iterates through a predefined list of C values [0.1, 1, 10, 100, 1000, 10000], and for each value of C, it trains a Support Vector Machine (SVM) classifier with a radial basis function (RBF) kernel and evaluates its performance on the training and test sets. The accuracy and C values are then plotted to visually check the best performance. It can be seen that the generalization error increases when the value of C is greater than 100, the SVM starts to overfit the training data and thus resulting in decrease in validation accuracy.

Shows the accuracy of SVM classifier versus regularization parameter C.

While K-Nearest Neighbors (KNN) model with 8 neighbors (k) achieved a training accuracy of 92.6% and a validation accuracy of 90.7% (see Fig.11), which is lower than the performance achieved by the presented model. To enable comparative analysis, it is essential to showcase the performance of this non-parametric machine learning algorithm. In this context, the algorithm predicts the value of a new data point by considering the majority vote or average of its k nearest neighbors within the feature space21. Figure14 illustrates the influence of the hyperparameter k, the number of neighbors, on the performance of the algorithm. The graph demonstrates that the validation accuracy reaches a maximum of 90.7% when 8 neighbors are considered.

Shows the accuracy of KNN classifier versus number of neighbors k.

To further analyze the DL-MMD model versus the experimental data, one more graph has been plotted shown in Fig.15. This graph illustrates the comparative performance of the presented model using a different data split ratio (7030), with 70% for training and 30% for validation. The graph shows a slightly degraded performance when compared to the split ratio (9010) of 90% for training and 10% for validation. However, it still shows validation accuracy of above 88% at 4000 epochs. This degradation is attributed to epistemic uncertainty (model uncertainty) due to slightly less effective learning on a reduced training data and as the training data increases, this uncertainty also reduces.

Shows the accuracy and validation accuracy of novel DL-MMD model versus epochs at two different data split ratios i.e. of 9010 and 7030.

The performance of the model can also be inferred from the confusion matrix shown in Fig.16. It provides a tabular representation of the predicted and actual class labels, giving a very important analysis of the models in terms of true positives, true negatives, false positives, and false negatives. For an application perspective of an MMD, safety of the user is of utmost importance for which false negative matters a lot since mine as target must not be missed.. The overall prediction accuracy is above 93.5%, however, for cases of air and sand it is approximately 85 and 86.5% respectively, inferred from the confusion matrix. These two classification cases of relatively less prediction accuracy can be neglected since sand being wrongly classified as air only and vice-versa. These two classes (air & sand) do not trigger any detection alarm by an MMD, thus misclassification of them will not impact efficiency of DL-MMD classifier. It also highlights the fact that sand (of river) has minimal mineralized content and is generally designated as non-mineralised soil. It is therefore difficult to separate the boundary between these two classes in presence of noise and interference.

Confusion matrix of the proposed DL-MMD classification on 9 classes.

In addition to this, two further cases need to be examined: one involves mineralized soil (minrl) being wrongly classified as APM, and the other involves APM in sand (sand+APM) being wrongly classified as minrl. The first case is of false positive, it will generate a false alarm and will waste time of the user by requiring unnecessary further investigation. The second case is of more importance i.e. of false negative where an APM is detected but wrongly classified by a DL-MMD and will be discussed in next section. Apart from them, there are minor cases e.g. an APM misclassified as APM in sand (sand+APM), it will not have any impact since target of concern (APM) will remain the same but now being shown buried in sand. The occurrence of all these misclassification cases (apart from the air/sand case & vice-versa) is less than 5% approximately.

These results have been obtained by a substantial dataset based on actual data acquired in two sets of 665 (pulses per class) each obtained at two different times through the experimental setup explained previously and then combined together. Comprehensive simulations have been carried out in the Tensor Flow environment for evaluation of the proposed method. In addition to this, the algorithm has been extensively tested with an increased number of layers and channels, resulting in overfitting. Furthermore, the proposed model has been tested with different optimizers, such as Adagrad, Adamax, and Adam. The comparative analysis of Adam and Adamax can be seen in Fig.17. Both show equivalent performance after 2000epochs.

Shows the accuracy and validation accuracy of novel DL-MMD model versus epochs using two different optimizers Adamax and Adam.

In addition to the aforementioned analysis, the dataset underwent evaluation using other prevalent classification algorithms22, which utilize the principle of ensemble learning. However, upon comparison, the proposed deep learning architecture exhibited superior performance, achieving an accuracy exceeding 90%. The confusion matrices of these classification algorithms, AdaBoost and Bagged tree, are depicted in Figs. 18, 19, and 20, with the dataset partitioned into an 80/20 ratio, resulting in accuracies of 75.4%, 80%, and 83.3%, respectively. AdaBoost was employed without PCA, utilizing the maximum number of splits and learners set to 30, with a learning rate of 0.1. For Bagged tree, only Model 2 underwent preprocessing with PCA with a variance of 95%. They both utilized the same number of learners as AdaBoost and a maximum split of 11,969.

Confusion matrix model 1 AdaBoost.

Confusion matrix model 2 Bagged Tree.

Confusion matrix model 3 Bagged Tree.

It is pertinent to mention that there is always redundant information within the received signal that creates background bias, especially in sensitive areas with low metal content. Information regarding the detection of APM mines buried at different depths is available (in the parameter decay rate), but it is not utilized. Therefore, for an APM buried at a different depth (relative to the search head) to the one it is trained on, there is a chance that it can be misclassified. The information exists, but it needs to be pre-processed before feeding the signal to the model. One approach could be to use focused AI models, similar to those shown in Ref23, that inject synthetic bias into the signal to generalize the model in our case at different depths. Another approach can be to localize the area with different decay rates, similar to the one shown in Ref24 for 2D image application. One of the future work will be to utilize this information and integrate it into the DL_MMD architecture.

Read the original:

Deep learning-based classification of anti-personnel mines and sub-gram metal content in mineralized soil (DL-MMD ... - Nature.com