Date of Award

Summer 2021

Document Type


Degree Name

Master of Science (MS)


Electrical and Computer Engineering

First Advisor

Medeiros, Henry

Second Advisor

Povinelli, Richard

Third Advisor

Odone, Francesca


Assisted living environments must be able to efficiently and unobtrusively gather information on a person's well-being. Human gaze direction provides some of the strongest indicators of how a person behaves and interacts with their environment. To that end, this thesis proposes a gaze tracking method that uses a neural network regressor to estimate gaze direction from facial keypoints and integrates them over time using various temporal methods, specifically through moving averages and a Kalman filter. Our gaze regression model uses confidence gated units to handle cases of keypoint occlusion and is able to estimate its own prediction uncertainty. This approach makes it possible to understand gaze direction patterns over time, which then can be used in the assessment of the well-being of individuals in assisted living environments. Experimental results on a dataset collected in an assisted living facility demonstrate that our gaze regression network performs on par with a complex, dataset-specific baseline, while its uncertainty predictions are highly correlated with the actual angular error of corresponding estimations. Furthermore, evaluations of our temporal integration methods on the assisted living facility dataset and on a publicly available gaze estimation dataset show promising results for more accurate and stable gaze predictions.

Included in

Engineering Commons