Q4E Case Study 29 – A Validation Study to Assess the Accuracy and Reliability of Quintic Automatic Tracking
|
Proposed subject
|
||
|
Sports Science / Mathematics / Physics (A/AS level / Degree Yr 1/2) |
Introduction
Data collection can introduce errors, which can be mistakenly construed as movement variability (Bartlett, Bussey and Flyger, 2006). These errors can be associated with measurement techniques such as skin movement, digitisation, electrical interference and motion artefacts (Robertson, Caldwell, Hamill, Kamen & Whittlesey, 2004). Physical activity can cause markers attached to soft tissue skin/muscles to move differently to the underlying bone (Benoit et al. 2005). Other errors associated with biomechanical data collection that can lead to inaccuracies include; inability to consistently locate the centre of a tracking marker and errors associated with the smoothing process of the data (Nielsen and Daugaard2008).
These errors sustained during data collection are unwanted noises that are different from the signal, therefore data needs to be smoothed in order to eliminate noise that has been produced (Robertson et al., 2004). When raw data is differentiated errors are amplified therefore a low-pass filter is needed to remove random noise (Kristianslund, Krosshaug & Van den Bogert, 2012).
Historically biomechanists have used one value to smooth all variables in all axis, however the Quintic Performance Analysis software will choose the optimum value for each variable for each different axis (X&Y). Butterworth filters can be used to remove high frequency noise present in data collected using high-speed cameras (Winter, 1990). A low-pass Butterworth filter allows wave frequencies lower than a specified cut-off to pass through the filter unaffected, while waves with frequencies greater than the cut-off are severely attenuated and eliminated from the output signal (Manal & Rose, 2007).
Selection of cut-off frequencies is extremely important however this is a subjective determination based upon knowledge of the noise and signal through visual inspection (Robertson et al. 2004). Kinetic calculations are influenced by the filtering of the segmental data (Bisseling & Hof, 2006). Residual analysis is an algorithm method that selects an optimal cut-off frequency for raw data based on the analysis of the residuals.
The Quintic Filtering window will illustrate the optimal Butterworth Filter values that have been placed on the data to smooth any data anomalies that may have occurred. Therefore the data presents an averaged difference between the filtered and unfiltered signal for cut-off frequencies.
When smoothing the Butterworth filter values for both the X and Y coordinates of each marker that have been tracked shall be considered in this data filtering window, however these Butterworth filter values can also be changed. The diagram below shows what happens when data is over and under-smoothed. Under-smoothing the data may not give a true representation of the signal and may overestimate the movement by retaining too much of the raw signal. Over-smoothing data distorts the original displacement curves and thus would be an underestimated representation of the movement (Bisseling & Hof, 2006).
Aims and Hypothesis;
-
The aim of the study was to validate the accuracy and reliability of Quintic automatic tracking in Quintic Biomechanics v26 Software, when data is collected in a stationary position.
-
The main hypothesis was that the accuracy and reliability should decrease as the distance increases
Methodology:
Procedures
Eight markers were placed on a dark background at the same height perpendicular to the camera. The markers used were 4mm, 6.4 mm, 9.5 mm, 12.7 mm, 14 mm, 15.9 mm, 19 mm and 25.4 mm (see figure 3). In application these markers are used for tracking small joints, e.g. the foot, to being used in Equine. The different sizes were used to validate if different marker sizes move different distances during auto tracking, when capturing data from a stationary position. Data was collected for all 8 markers at 14 different distances, including, 1.5 m, 2 m, 2.5 m, 3 m, 3.5 m, 4 m, 4.5 m, 5 m, 5.5 m, 6 m, 6.5 m, 7 m, 7.5 m and 8 m.
A USB 2 high-speed camera, with a 6-13 mm C mount lens, was used in the study, and data was collected at 100 Hz. Data was collected as a default compression of 75 and at a compression of 100 when recording data via the single camera capture in Biomechanics v26 software.
Data Analysis
Ten seconds (1000 frames) were randomly selected to automatically digitise. An additional camera file of a known distance was used to calibrate the data. The data files could then be automatically digitised by creating a frame template and identifying all of the eight markers. Once the data had been digitised, using the linear analysis tool the raw data was exported into excel to identify the distance that the marker moved without any smoothing to the data. Using optimal Butterworth filters for both coordinates (X&Y) the data was smoothed to remove any errors from data capture.
Results
Distance the marker moved (mm)
Figure 4 demonstrates the set-up of the markers that was used to digitise the markers. If the markers are too small or too large, the Quintic Automatic Tracking software cannot recognise and track the markers. For example at a distance of 1.5 m the 25.4mm marker was too large for the Quintic Automatic Tracking software to recognise, therefore the marker could not be tracke
Table 1. The distance the marker moved (mm) for each marker at 14 distances for the smoothed and raw data when compressed at 75%.
Table 2. The distance the marker moved (mm) for each marker at 14 distances for the smoothed and raw data when compressed at 100%.
The results in table 1 and 2 show the accumulative distance moved for each marker. This was calculated from the first frame to the last frame and converted into mm for each distance and each marker. The raw data was compared to the smoothed data, compressed at 100% and 75% (see table 1 and 2). The results indicate that when the data is smoothed, the distance the marker moves is reduced. The table shows that there are some raw data values where there is a lot of marker movement, this may be due to the software not been able to catch onto the marker during the digitisation process. The Quintic Biomechanics Software looks for a circle and tries to ‘catch on’ to the centre of the marker. Figure 5 demonstrates an example of when the Quintic software does not focus on the centre of the circle during the digitisation process. This can therefore lead to inaccurate data.
The results detailed throughout this study focuses specifically on the 6.4 mm marker at a distance of 3 metres, unless otherwise stated. This was randomly selected and all other results can be found in the appendices.
The results in table 3 indicate that there is no difference in marker movement between the smoothed data when compressed at 75% or 100%. When the raw data is compressed to 100% the marker moved a smaller distance compared to 75% suggesting that compressing the data to 100% creates less error. The results also quantify that smoothing the data produces less error when compared to the raw data.
Data Smoothing
Any anomalies in data (spikes and optimal 1:00 filter values) were manually changed to 10% of the 100 Hz. Using the linear analysis tool the smoothed data was also exported into excel, in order to compare the distance that the marker moved.
Table 4. The distance the marker moved (mm) with different filter values. (Optimal X=18.20 Y=13.40).
The results in table 4 indicate that when changing the filter value the distance that the marker moves is dramatically influenced. The results support the assumption that over and under-smoothing your data can over/ under-estimate the distance that the marker has moved. Figure 6 demonstrates that increasing the filter value (Hz) increased the distance that the marker moved. When using a filter of 27 Hz it is apparent in table 4 and figure 6, that it is not a true representation of the signal and over estimates the distance the marker has moved by retaining too much of the raw signal. A cut-off frequency of 20 Hz, 22.5 Hz and 25 Hz, still showed to retain too much of the noise in the data thus distracting from the original signal. If high-frequency components are present in the motion then filtering the data at a too low cut-off frequency distorts the original displacement of the curve, thus resulting in velocities and accelerations being underestimated (Bisseling & Hof, 2006). In addition to this when using a filter of 2.5 Hz, 5 Hz or 7.5 Hz showed to underestimate the distance the marker moved, resulting in over-smoothing the data (figure 7).
The results in table 5 indicate that when the data is smoothed to the optimum value in Quintic Biomechanics v26 Software the distance the marker moved is reduced when compared to the raw data. There is no difference in the distance the marker moved when compared to the data been compressed to 75% and 100%. Please see appendix A for the full table of X and Y coordinates for all markers.
Table 5. The smoothed (optimal filters) X and Y coordinates for compression 75 and 100, and the distance the marker moved,
compared to the raw value.
Compressing the data
When saving a video within the Quintic single high-speed camera software you can set the Jpeg quality compression (1 being the lowest quality and 100 being the highest quality). Quintic recommend a setting of 75 as going higher than this value may result in dropped frames during data capture, however when compressing to 100 this created less error and zero frames were dropped during data capture. When capturing your data you want to ensure that you have the quality of the image in comparison to a large image size. This will ensure you have a high quality compressed Jpeg video, and a high frame rate by cropping the image.
The differences between the data being compressed to 100% and the default value of 75% were assessed to determine if compressing to 100% creates less error. For the 6.4 mm marker the results demonstrated in table 5 indicate that this does not influence the distance the marker moved. However smoothing the data significantly effects how much the marker moved when comparing to the raw data. The compressed 100% file does in fact produce less maker movement compared to compressed 75%. (See Appendix A).
Conclusion:
To conclude the results of the study when looking at 10 seconds of data indicate that smoothing the data does reduce the amount of noise that is associated with movement error. By smoothing the data this resulted in the distance that the stationary marker moved reduced. The results also indicated that some raw data values moved the marker a great distance (some values up to 10cm), however this could be due to the software not being able to catch onto the marker during the digitisation process due to the marker not been clear. When compressing the data to 100% this does produce less error when compared to compressing the file to the default value of 75% for both the smoothed and raw data.
The results indicate that over and under-smoothing is detrimental to the data in terms of marker movement, and that the optimal Butterworth filter that Quintic Biomechanics v26 produces will reduce subjective data and help to eliminate error. A cut-off frequency of 20 Hz, 22.5, 25 and 27 Hz showed that it is not a true representation of the signal and overestimates the distance the marker has moved by. In addition to this using a filter of 2.5 Hz, 5 Hz and 7.5 Hz similarly showed to underestimate the distance the marker moved, resulting in over-smoothing the data. Therefore it is important to find a filter that allows a true representation of the data. The results revealed that when the data is smoothed to the optimum value in Quintic Biomechanics v26 software the distance the marker moved is reduced when compared to the raw data.
When collecting data you can reduce the amount of noise by wearing tight clothing to reduce marker movement. Marker placement is essential in determining the exact location of the joint. The quality of the marker is vital when collecting data in order to digitise your data. The quality of the marker is determined by your camera set-up which will influence the quality of the image. By obtaining the correct zoom, focus, aperture and shutter speed will influence the quality of your data collection.
References
1. Barlett, R., Bussey, M., & Flyger, N. (2006). Movement variability cannot be determined from no-marker conditions. Journal of Biomechanics, 39 (16), 3076-3079.
2. Bisseling, R. W & Hof, L. A. (2006). Handling of impact forces in inverse dynamics. Journal of Biomechanics, 39, 2438-2444.
3. Kristianslund, E., Krosshaug, T., & Van den Bogert, A. J. (2012). Effect of low pass filtering on joint moments from inverse dynamics: implications for injury prevention. Journal of Biomechanics, 45, 666-671.
4. Manal, K., & Rose, W. (2007). A general solution for time delay introduced by a low-pass Butterworth digital filter: An application to musculoskeletal modelling. Journal of Biomechanics, 40(3), 678-681.
5. Nielsen, D. B., Daugaard, M. (2008). Comparison of angular measurements by 2D and 3D gait analysis. Jönköping. Thesis, 15 HP
6. Robertson, G. E., Caldwell, G. E., Hamill, J., Kamen, G., & Whittlesey, S. N. (2004). Research methods in biomechanics. United States, Champaign IL: Human Kinetics.
7. Winter, D. A. (1990). Biomechanics and Motor Control of Human Movement (Second Edition). Wiley, New York.
Downloads:
|
Appendix A
|
The accuracy of different reflective marker Sizes | Quintic Sports