Date of Award
Summer 2021
Document Type
Thesis
Degree Name
Master of Science (MS)
Department
Electrical and Computer Engineering
First Advisor
Medeiros, Henry
Second Advisor
Povinelli, Richard
Third Advisor
Odone, Francesca
Abstract
Assisted living environments must be able to efficiently and unobtrusively gather information on a person's well-being. Human gaze direction provides some of the strongest indicators of how a person behaves and interacts with their environment. To that end, this thesis proposes a gaze tracking method that uses a neural network regressor to estimate gaze direction from facial keypoints and integrates them over time using various temporal methods, specifically through moving averages and a Kalman filter. Our gaze regression model uses confidence gated units to handle cases of keypoint occlusion and is able to estimate its own prediction uncertainty. This approach makes it possible to understand gaze direction patterns over time, which then can be used in the assessment of the well-being of individuals in assisted living environments. Experimental results on a dataset collected in an assisted living facility demonstrate that our gaze regression network performs on par with a complex, dataset-specific baseline, while its uncertainty predictions are highly correlated with the actual angular error of corresponding estimations. Furthermore, evaluations of our temporal integration methods on the assisted living facility dataset and on a publicly available gaze estimation dataset show promising results for more accurate and stable gaze predictions.