Integrating optical finger motion tracking with surface touch events

Capacitive Sensing for Improving Contact Rendering With Tangible Objects in VR

We combine tracking information from a tangible object instrumented with capacitive sensors and an optical tracking system, to improve contact rendering when interacting with tangibles in VR. A human-subject study shows that combining capacitive sensing with optical tracking significantly improves the visuohaptic synchronization and immersion of the VR experience.

Effects of Trunk Motion, Touch, and Articulation on Upper-Limb Velocities and on Joint Contribution to Endpoint Velocities During the Production of Loud Piano Tones

Piano performance involves several levels of motor abundancy. Identification of kinematic strategies that enhance performance and reduce risks of practice-related musculoskeletal disorders (PRMD) represents an important research topic since more than half of professional pianists might suffer from PRMD during their career. Studies in biomechanics have highlighted the benefits of using proximal upper-limb joints to reduce the load on distal segments by effectively creating velocity and force at the finger–key interaction. If scientific research has documented postural and expressive features of pianists’ trunk motion, there is currently a lack of scientific evidence assessing the role of trunk motion in sound production and in injury prevention. We address this gap by integrating motion of the pelvis and thorax in the analysis of both upper-limb linear velocities and joint angular contribution to endpoint velocities. Specifically, this study aims to assess kinematic features of different types of touch and articulation and the impact of trunk motion on these features. Twelve pianists performed repetitive loud and slow-paced keystrokes. They were asked to vary (i) body implication (use of trunk and upper-limb motion or use of only upper-limb motion), (ii) touch (struck touch, initiating the attack with the fingertip at a certain distance from the key surface, or pressed touch, initiating the attack with the fingertip in contact with the key surface), and (iii) articulation (staccato, short finger–key contact time, or tenuto, sustained finger–key contact time). Data were collected using a 3D motion capture system and a sound recording device. Results show that body implication, touch, and articulation modified kinematic features of loud keystrokes, which exhibited not only downward but also important forward segmental velocities (particularly in pressed touch and staccato articulation). Pelvic anterior rotation had a prominent role in the production of loud tones as it effectively contributed to creating forward linear velocities at the upper limb. The reported findings have implications for the performance, teaching, and research domains since they provide evidence of how pianists’ trunk motion can actively contribute to the sound production and might not only be associated with either postural or expressive features.

The Improvisational State of Mind: A Multidisciplinary Study of an Improvisatory Approach to Classical Music Repertoire Performance

The recent re-introduction of improvisation as a professional practice within classical music, however cautious and still rare, allows direct and detailed contemporary comparison between improvised and "standard" approaches to performances of the same composition, comparisons which hitherto could only be inferred from impressionistic historical accounts. This study takes an interdisciplinary multi-method approach to discovering the contrasting nature and effects of prepared and improvised approaches during live chamber-music concert performances of a movement from Franz Schubert's "Shepherd on the Rock," given by a professional trio consisting of voice, flute, and piano, in the presence of an invited audience of 22 adults with varying levels of musical experience and training. The improvised performances were found to differ systematically from prepared performances in their timing, dynamic, and timbral features as well as in the degree of risk-taking and "mind reading" between performers, which included moments of spontaneously exchanging extemporized notes. Post-performance critical reflection by the performers characterized distinct mental states underlying the two modes of performance. The amount of overall body movements was reduced in the improvised performances, which showed less unco-ordinated movements between performers when compared to the prepared performance. Audience members, who were told only that the two performances would be different, but not how, rated the improvised version as more emotionally compelling and musically convincing than the prepared version. The size of this effect was not affected by whether or not the audience could see the performers, or by levels of musical training. EEG measurements from 19 scalp locations showed higher levels of Lempel-Ziv complexity (associated with awareness and alertness) in the improvised version in both performers and audience. Results are discussed in terms of their potential support for an "improvisatory state of mind" which may have aspects of flow (as characterized by Csikszentmihalyi, 1997) and primary states (as characterized by the Entropic Brain Hypothesis of Carhart-Harris et al., 2014). In a group setting, such as a live concert, our evidence suggests that this state of mind is communicable between performers and audience thus contributing to a heightened quality of shared experience.