Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Bioinformatics, Computational and Systems Biology

Bioinformatics, Computational and Systems Biology - Poster Session E

Poster L11 - Capturing visual search strategy through a novel application of time series clustering to raw eye-tracking data

Saturday, October 26, 2024
10:00 AM - 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Situational awareness – the ability to comprehend the surrounding environment and anticipate future events – is necessary for safely avoiding hazards while performing a task (e.g., driving a car without crashing). Good situational awareness relies on sufficient visual search of the environment, known as visual search strategy. Insufficient visual search can result in a failure to detect hazards, such as a pedestrian entering the roadway from an obstructed crosswalk. To date, the study of visual search strategy remains limited to small cohorts and short experimental time periods due to the conventional labor-intensive approach of capturing visual search strategy: video coding. To address this limitation, this study presents an efficient, scalable, unsupervised machine learning approach to capture visual search strategy: time series clustering. Unlike unsupervised clustering of discrete metrics, which requires pre-processing of the raw data into metrics that may be task-dependent, time series clustering requires little to no pre-processing and, therefore, is task-independent, avoiding any potential bias introduced by pre-defined metrics. To our knowledge, time series clustering has yet to be utilized to capture visual search strategy. Therefore, the aim of this study was to demonstrate the feasibility and utility of time series clustering to capture visual search strategies.

Materials and

Methods:  Time series clustering was applied to eye-tracking data collected during a curved roadway in a driving simulator. A curved roadway has previously been shown to differentiate drivers based on their visual search behavior. Eye-tracking data were collected from a cohort of young (16-24 yrs) drivers (n=36) at 10 Hz using a Tobii Eye Tracker 5. Raw time-series eye-tracking data were compared using dynamic localized coordinate aligned warping (DCLAW), an extension of dynamic time warping. Visual search strategy clusters were identified using k-medoids unsupervised clustering. The optimal number of clusters (k) was determined using a combination of identifying inflection points via the “elbow method” in the within cluster sum-of-squared errors (WCSS) plot and identifying the highest mean Silhouette Score. Characteristics of the visual search strategy clusters were defined by review of the cluster medoids. To ensure that no visual search strategies were missed by time series clustering, scene videos with eye-tracking overlayed were manually video coded. The visual scene was divided into five areas of interest (AOIs) for video coding: (1) far-extent, (2) near-extent, (3) speedometer, (4) mirrors, (5) surrounding scenery. Similar visual search strategies were grouped based on observed glance characteristics between AOIs. To demonstrate the added information that time series clustering provides over conventional eye-tracking metrics, spread of search – a commonly used metric to assess eye-tracking during driving that has been shown to be associated with safe driving outcomes – was computed and compared between clusters using a Wilcoxon rank sum test.

Results, Conclusions, and Discussions: The aim of this study was to demonstrate the feasibility of time series clustering to capture meaningful visual search strategies from raw time-series eye-tracking data, without the need for pre-processing. The optimal number of clusters was 5 (see Figure 2), as indicated by the corresponding peak and elbow at k=5 in the mean silhouette score and WCSS plots (see Figure 1), respectively. Cluster 1 (n=25) exhibited gaze focused exclusively on the roadway immediately in front of the vehicle, therefore was named “Near-Extent”. Cluster 2 (n=8) exhibited gaze focused toward this far-extent of the curve; therefore was named “Far-Extent”. Cluster 3 (n=1) and Cluster 4 (n=1) both exhibited glances toward the side and rearview mirrors. Given their similarity in mirror glances, these two outlier clusters were combined into a single cluster (n=2), named “Mirror Checking”. Cluster 5 (n=1) exhibited gaze focused on irrelevant scenery, therefore was named “Unfocused Scanning”. We consider these visual search strategies meaningful because they correspond to previous research exploring visual search and its relationship to driving. Visual search strategy clusters and cluster membership identified through video coding were congruent with those identified by time series clustering. Horizontal and vertical spread of search for each visual search strategy are shown in Figure 3. Notably, the far-extent and near-extent visual search strategies were indistinguishable when compared using horizontal (p=0.30) and vertical (p=0.25) spread of search. The inability of this conventional metric to differentiate visual search strategies further highlights the importance of using more comprehensive methods to analyze eye-tracking data, such as the time series clustering procedure presented herein, to holistically capture participants’ visual search. In summary, time series clustering successfully identified four meaningful, distinct visual search strategy clusters from raw time-series eye-tracking data during a common driving task. This methodology can minimize the need for manual pre-processing of raw eye-tracking data, allowing for analysis of larger, more generalizable datasets to study visual search strategy.

Acknowledgements (Optional):  The authors would like to thank the volunteers who participated in the simulated driving study. The authors would like to acknowledge Chelsea Ward-McIntosh for her assistance with collecting the eye-tracking data. The authors would like to acknowledge Drs. Kristy B. Arbogast, Timothy Roberts, and Daniel Romer for their valued guidance and scientific input.