Poster Z9 - Modeling temporal EEG responses to cyclic binary visual stimulus patterns using an artificial neural network

Introduction: The human visual system serves as the basis for many common computer vision and machine learning approaches. These commonly utilized techniques are inspired by the characteristics of individual neuron behavior as well as small network behavior. Detailed biophysical models of these systems exist but are unwieldy when scaled up and lack connectivity to measurable physiological signals. This presents the need to develop an end-to-end visual system model from stimulus to the signals generated at the visual cortex, such as those recorded by scalp electroencephalogram (EEG). The creation of such a model would not only provide a better understanding of visual processing pathways and inform new machine learning approaches but would also facilitate the design and evaluation of more robust visual stimuli for brain-computer interfaces (BCIs).

Materials and

Methods: A novel experiment was conducted where 15 participants observed 6 different stereotyped visual stimuli, similar to those used for generating visual evoked potentials (VEPs), while their EEG was recorded simultaneously. The resulting EEG responses were characterized across participants. Then a Residual Connection Feed Forward system identification Neural Network (ReCon FFNN) was implemented as a preliminary end-to-end model of the visual system that uses the temporal characteristics of the visual stimulus as the model input and the corresponding EEG time series as the model output.

Results, Conclusions, and Discussions: The spatially filtered EEG responses from the participants retained the harmonic characteristics of those seen in VEP experiments. As stimulus-to-EEG modeling has not been previously explored to our knowledge, analysis techniques were borrowed from those used to validate time series forecasting artificial neural networks. Classification of the recorded EEG and model outputs from individual participants to the stimulus used to generate them was performed using a k-nearest neighbor algorithm for both time and frequency domain content using correlation as the distance metric. The time domain classifiers used amplitude of each point in the time series as features for classification while the frequency domain classifiers used magnitude coefficients. Results from both time and frequency domain classifiers suggest that the model was able to accurately reproduce many characteristics of the EEG recorded. The time domain classifier using the recorded EEG as reference data achieved 85% and 75% average accuracy on the model training and testing outputs respectively. The time domain classifier using model outputs as reference data obtained an average accuracy of 66% with the accuracy for individual participants ranging from 42% to 90%. The frequency domain classifier with recorded EEG as reference data achieved 91% and 80% average accuracy on the model training and testing outputs. The frequency domain classifier using model outputs as reference data had an average accuracy of 95% when classifying the recorded EEG. The similarity between the model outputs and recorded EEG was also compared using the correlation between autocorrelation (ACC) of the two signals. This procedure found that the model outputs and recorded EEG shared much of their harmonic characteristics. The average ACC was .77 and .62 when correlating the training and testing model outputs to the recorded EEG. This was significantly higher than the mean ACC between the model outputs and pre-trial baselines, which were both approximately .19. The preliminary visual system model was able to reproduce temporal and spectral characteristics of the EEG and serves as a proof of concept and inspiration for future artificial neural network or biophysical models that incorporate spatio-temporal information to model the visual system from input to visual cortex.

Acknowledgements (Optional):