Finally, we calculate the confidence score (C) for determining the extent to which the given active garment is a garment of interest using Equation (3.4).
The confidence score (C) is calculated for each pair-wise distinct set of detected person’s wrists and active garment where the highest score is retained for that active garment. If this confidence score exceeds a specified confidence threshold (δ), then the active garment in consideration is considered to be a garment of interest to the given customer as his/her wrist landmarks are in close proximity with the garment. The proposed approach also notes the time duration for which such interaction takes place by keeping track of the number of frames for which the active garment is a garment of interest for the given customer.
3.4 Experimental Results
In this section, we discuss the results of our proposed approach on the garment dataset. The experiments were conducted on a device with IntelⓇ Core i7-8750H 2.20 GHz CPU, NVIDIA RTX 2070 with 8 GB VRAM, and 16 GB DDR4 RAM. Additionally, some programs were computed on Google Colab [21] with IntelⓇ XeonⓇ 2.00 GHz CPU, NVIDIA Tesla T4 GPU, 16 GB GDDR6 VRAM, and 13 GB RAM. The programs were written in Python 3.6 and utilized OpenCV 4.2.0 and OpenPose 1.6.0.
3.4.1 Dataset Used
The dataset used to evaluate the proposed approach is a collection of video surveillance footage of an Indian garment store. The videos capture the interaction between the salesperson and the customers while selecting a garment for purchase. The videos are obtained from a CCTV camera in the store’s infrastructure, thus enabling us to work on real-life surveillance videos that have practical illumination conditions and background noise. The raw videos in the dataset have a resolution of 944 × 576 pixels. These videos were pre-processed to have a resolution of 1,920 × 1,080 pixels and a consistent frame rate of 30 FPS before computation. There a total of 33 videos in the dataset with an average duration of 1 minute. Among these videos, 22 videos contain a single customer, 8 videos contain two customers, and 3 videos contain three or more customers. Thus, the chosen dataset represents practical conditions and is adequately diverse. Figure 3.4 illustrates a few samples from the dataset.
3.4.2 Experimental Results and Statistics
The proposed approach was evaluated by measuring the effectiveness of the detection of active garments and the identification of garments of interest.
Figure 3.4 Samples from the dataset.
Table 3.2 Precision and recall of active garment detection.
| Precision (%) | Recall (%) |
| 87.36 | 85.92 |
The effectiveness of active garment detection was evaluated by measuring its precision and recall, which are shown in Table 3.2. These metrics have been calculated as follows:
(3.5)
(3.6)
where TP, TN, FP, and FN denote the true positive, true negative, false positive, and false negative observations, respectively.
The garments of interest were verified manually with estimations (as the actual ground truth for the degree to which an active garment is a garment of interest to the customers was not available due to the subjective nature of the task). Visual illustrations of the results obtained in different scenarios have been displayed in Figures 3.5 to 3.7. In these figures, a green box indicates a garment of interest, while a red box indicates an active garment which is not a garment of interest.
Figure 3.5 Identification of garment of interest in the presence of a single customer and a single active garment.
Figure 3.6 Identification of garments of interest in the presence of a single customer and multiple active garments.
Figure 3.7 Identification of garments of interest in the presence of multiple customers.
The variation in the average confidence scores taken at varying confidence thresholds (δ) for a garment of interest of a given color is illustrated in Figure 3.8. From this figure, it can be observed that the variation takes the form of an S-curve, which signifies that the average confidence score mimics a logistic curve. Using this knowledge, we can easily observe that the confidence threshold (δ) of 0.6 is an optimal value (i.e., when the curve starts to flatten immediately after the point of inflection) to classify whether an active garment is a garment of interest to a given customer. The average confidence score at this confidence threshold (δ) exceeds 0.8 for all the colors under observation. This ensures that the wrists of the interacting person are sufficiently close to the active garment.
Figure 3.8 Variation of average confidence score with respect to changes in confidence threshold for garments of interest of a given color.
The
