Liang He
< >

VRSurus is a smart device designed to recognize the puppeteer’s gestures and render tactile feedback to enhance the interactivity of physical puppets in virtual reality (VR). It is wireless, self-contained, and small enough to be mounted upon any physical puppets. Using machine-learning techniques, VRSurus is able to recognize three gestures: swiping, shaking and thrusting. Actuators (e.g., solenoids, servos and vibration motors) assist with the puppetry visible to the audience and provide tactile feedback on the puppeteer’s forearm. As a proof of concept, we implemented a tangible serious VR game using VRSurus that aimed at inspiring children to protect the environment and demonstrated it at the ACM UIST 2015 Student Innovation Contest. Our 3D models, circuitry and the source code are publicly available at

Full Citations
Ruofei Du, and Liang He. 2016. VRSurus: Enhancing Interactivity and Tangibility of Puppets in Virtual Reality. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 2454-2461. DOI:
 Download Paper  ACM DL