Applying Affective Feedback to Reinforcement Learning in ZOEI, a Comic Humanoid Robot

Document Type

Conference Proceeding



Format of Original

6 p.

Publication Date



Institute of Electrical and Electronics Engineers (IEEE)

Source Publication

2014 RO-MAN: The 23rd IEEE International Symposium on Robot and Human Interactive Communication

Source ISSN


Original Item ID

doi: 10.1109/ROMAN.2014.6926289


As robotic technologies of varying shapes and forms continue to make their way into our everyday lives, the significance of a humanoid robot's ability to make a human interaction feel natural, engaging and entertaining becomes an area of keen interest in sociable robotics. In this paper, we present our findings on how affective feedback can be used to drive reinforcement learning in human-robot interactions (HRI) and other dialogue systems. We implemented a system where a humanoid robot, named ZOEI, acts as a standup comedian by entertaining a human audience in a bid to generate humor and positively influence the emotional state of the humans. The mood rating of the audience is recorded prior to the interaction session. Using a survey, the eventual emotional state of the human participant is captured after the HRI session. For each audience member, we capture feedback regarding how funny each joke was. We present the implementation of the content selection framework. We share our findings to substantiate the idea that by using expressive behaviors of the humanoid to influence the delivery of content (in this case, jokes) as well as employing reinforcement learning techniques for driving targeted content selection, the robot was able to improve the human mood score progressively across the 16 people who engaged in the study.


Published as part of the proceedings of the conference, 2014 RO-MAN: The 23rd IEEE International Symposium on Robot and Human Interactive Communication, 2014: 423-428. DOI.