|Human Motion Capture|
Dr. Pierre Payeur
SITE, University of Ottawa
Dr. Gilles Comeau
Dept. of Music, University of Ottawa
Piano Pedagogy Research Laboratory
University of Ottawa
Natural Sciences and Engineering Research Council of Canada
University of Ottawa - Interdisciplinary Initiatives Development Fund
Yamaha Canada Music
This project develops new computer vision technologies for monitoring the complex movements executed
by piano players. Piano teachers still widely use subjective and incomplete visual observation of the
pianist's posture and gestures to help their students improve their play. This does not provide a
quantitative method for evaluating and comparing pianists habits and performance and to detect problematic
situations. It has been demonstrated that bad habits can lead to severe piano-playing related health
problems (PRHP). This reveals a need for technologies that would contribute in the prevention of such
injuries and to the improvement of pedagogical methods, without being too complex to operate.
Existing motion capture technologies used in medicine and movie production remain invasive and cumbersome as they involve massive cabling and/or reflective targets to be installed on the body of the performer. Other approaches rely on contrasting backgrounds or on assumptions about the motion and complexity of the scene. These impositions yield an environment that is foreign to a performer, leading him to behave differently than he would in a more comfortable environment. The limitations of such techniques may also obfuscate key performance markers.
In order to alleviate these constraints, advanced computer vision and image processing approaches are investigated as sensing techniques in the context of piano playing. A multi-camera video monitoring network has been designed along with a complete calibration procedure. An innovative image segmentation technique able to adapt to unconstrained environments has also been developed. The system can perform live motion capture of a piano player without using any marker nor imposing a specific dress code to the pianist. The objective of the first phase of research is to achieve markerless motion capture of a performer that can operate without constraints on the surrounding environment. Early experimental results demonstrated the feasibility of the approach, resulting in accurate colored 3D reconstructions of a subject performing some movement.
Future phases are going to investigate the recognition and classification of patterns of movement from the quantitative information extracted from the motion capture system. This will open the door to an exploration of biomechanical impacts of the repetition of some gestures and to the prevention of PRHPs. It will also provide the necessary background to support the development of computerized piano teaching resources that will automatically analyze the movement of the performer's body as well as the depression of keys on the keyboard.