Affordable Access

Access to the full text

The Automatic Detection of Cognition Using EEG and Facial Expressions

Authors
  • El Kerdawy, Mohamed1
  • El Halaby, Mohamed
  • Hassan, Afnan1
  • Maher, Mohamed1
  • Fayed, Hatem2
  • Shawky, Doaa2
  • Badawi, Ashraf1
  • 1 (A.B.)
  • 2 Engineering Mathematics Department, Faculty of Engineering, Cairo University, Giza 12613, Egypt
Type
Published Article
Journal
Sensors
Publisher
MDPI AG
Publication Date
Jun 21, 2020
Volume
20
Issue
12
Identifiers
DOI: 10.3390/s20123516
PMID: 32575909
PMCID: PMC7349015
Source
PubMed Central
Keywords
License
Green

Abstract

Detecting cognitive profiles is critical to efficient adaptive learning systems that automatically adjust the content delivered depending on the learner’s cognitive states and skills. This study explores electroencephalography (EEG) and facial expressions as physiological monitoring tools to build models that detect two cognitive states, namely, engagement and instantaneous attention, and three cognitive skills, namely, focused attention, planning, and shifting. First, while wearing a 14-channel EEG Headset and being videotaped, data has been collected from 127 subjects taking two scientifically validated cognitive assessments. Second, labeling was performed based on the scores obtained from the used tools. Third, different shallow and deep models were experimented in the two modalities of EEG and facial expressions. Finally, the best performing models for the analyzed states are determined. According to the used performance measure, which is the f-beta score with beta = 2, the best obtained results for engagement, instantaneous attention, and focused attention are EEG-based models with 0.86, 0.82, and 0.63 scores, respectively. As for planning and shifting, the best performing models are facial expressions-based models with 0.78 and 0.81, respectively. The obtained results show that EEG and facial expressions contain important and different cues and features about the analyzed cognitive states, and hence, can be used to automatically and non-intrusively detect them.

Report this publication

Statistics

Seen <100 times