Investigating pianists' individuality in the performance of five timbral nuances through patterns of articulation, touch, dynamics, and pedaling

Coordinating upper limbs for octave playing on the piano via neuro-musculoskeletal modeling

Understanding the coordination of multiple biomechanical degrees of freedom in biological organisms is crucial for unraveling the neurophysiological control of sophisticated motor tasks. This study focuses on the cooperative behavior of upper-limb motor movements in the context of octave playing on the piano. While the vertebrate locomotor system has been extensively investigated, the coherence and precision timing of rhythmic movements in the upper-limb system remain incompletely understood. Inspired by the spinal cord neuronal circuits (central pattern generator, CPG), a computational neuro-musculoskeletal model is proposed to explore the coordination of upper-limb motor movements during octave playing across varying tempos and volumes. The proposed model incorporates a CPG-based nervous system, a physiologically-informed mechanical body, and a piano environment to mimic human joint coordination and expressiveness. The model integrates neural rhythm generation, spinal reflex circuits, and biomechanical muscle dynamics while considering piano playing quality and energy expenditure. Based on real-world human subject experiments, the model has been refined to study tempo transitions and volume control during piano playing. This computational approach offers insights into the neurophysiological basis of upper-limb motor coordination in piano playing and its relation to expressive features.

Neural Phenomenon in Musicality: The Interpretation of Dual-Processing Modes in Melodic Perception

The confluence of creativity in music performance finds itself in performance practices and cultural motifs, the communication of the human body along with the instrument it interacts with, and individual performers' perceptual, motor, and cognitive abilities that contribute to varied musical interpretations of the same piece or melodic line. The musical and artistic execution of a player, as well as the product of this phenomena can become determinant causes in a creative mental state. With advances in neurocognitive measures, the state of one's artistic intuition and execution has been a growing interest in understanding the creative thought process of human behavior, particularly in improvising artists. This article discusses the implementation on the concurrence of spontaneous (Type-1) and controlled (Type-2) processing modes that may be apparent in the perception of non-improvising artists on how melodic lines are perceived in music performance. Elucidating the cortical-subcortical activity in the dual-process model may extend to non-improvising musicians explored in the paradigm of neural correlates. These interactions may open new possibilities for expanding the repertoire of executive functions, creativity, and the coordinated activity of cortical-subcortical regions that regulate the free flow of artistic ideas and expressive spontaneity in future neuromusical research.

The Use of Sonification for the Analysis and Teaching of Interpretive Auditory Nuances

The integrative powers of human auditory perception in masterful performing artists can create nuances of sound from the interpretation of notes on a written page that often seem beyond one’s grasp as a listener. It is important to consider what type of feedback can provide a clearer understanding of nuances in sound to guide motor learning for the acquisition of new skills for expression. Master pianists’ performance has been used as a model for imitation. However, to ensure the accuracy of imitation and clear understanding, sonification was examined for its effectiveness in providing a more immediate understanding of individual interpretations in terms of inter-onset timing and velocity of the notes. Three master concert pianists volunteered to record a performance of the Chopin Nocturne Opus 15 No. 1 on a Yamaha Disklavier Pro MIDI (music instrument digital interface) grand piano. Logic software was used to analyze and compare MIDI data from each performance from the perspective of phrase-by-phrase, note onset timings, and corresponding data from the other pianists. The study objectives were to examine commonalities and differences in timing and dynamics among the performances using MIDI measurements; to probe whether listening to comparative performance data assisted with feedback from sonification would enhance music students’ understanding of the interpretive nuances through imitation; and to determine whether auditory-assisted sonification feedback could be used as a tool to expand students’ interpretive choices and enhance performance. Participants imitated selected phrases of each of the master pianists, first with only music listening, and then with feedback from the sonified, comparative performance data. Results showed limited success in attempts to imitate the model with auditory feedback alone. Auditory-assisted sonification feedback significantly enhanced the participants’ abilities to imitate the model faster and with greater accuracy in the final imitation experiments. The data gathered by the study provide insights on this kind of sonification as an effective feedback tool for heightening participants’ auditory understanding of nuances in sound, as well as providing an effective teaching tool for imitation exercises.

The Communication of Timbral Intentions Between Pianists and Listeners and Its Dependence on Auditory-Visual Conditions

The perceptual experiment reported in this article explored whether the communication of five pairs of timbral intentions (bright/dark, heavy/light, round/sharp, tense/relaxed, and dry/velvety) between pianists and listeners is reliable and the extent to which performers' gestures provide visual cues that influence the perceived timbre. Three pianists played three musical excerpts with 10 different timbral intentions (3 × 10 = 30 music stimuli) and 21 piano students were asked to rate perceived timbral qualities on both unipolar Likert scales and non-verbal sensory scales (shape, size, and brightness) under three modes (vision-alone, audio-alone, and audio-visual). The results revealed that nine of the timbral intentions were reliably communicated between the pianists and the listeners, except for the dark timbre. The communication of tense and relaxed timbres was improved by the visual conditions regardless of who is performing; for the rest, we found the individuality in each pianist's preference for using visual cues. The results also revealed a strong cross-modal association between timbre and shape. This study implies that the communication of piano timbre is not based on acoustic cues alone but relates to a shared understanding of sensorimotor experiences between the performers and the listeners.

Quantitative analysis of piano performance proficiency focusing on difference between hands

Quantitative evaluation of piano performance is of interests in many fields, including music education and computational performance rendering. Previous studies utilized features extracted from audio or musical instrument digital interface (MIDI) files but did not address the difference between hands (DBH), which might be an important aspect of high-quality performance. Therefore, we investigated DBH as an important factor determining performance proficiency. To this end, 34 experts and 34 amateurs were recruited to play two excerpts on a Yamaha Disklavier. Each performance was recorded in MIDI, and handcrafted features were extracted separately for the right hand (RH) and left hand (LH). These were conventional MIDI features representing temporal and dynamic attributes of each note and computed as absolute values (e. g., MIDI velocity) or ratios between performance and corresponding scores (e. g., ratio of duration or inter-onset interval (IOI)). These note-based features were rearranged into additional features representing DBH by simple subtraction between features of both hands. Statistical analyses showed that DBH was more significant in experts than in amateurs across features. Regarding temporal features, experts pressed keys longer and faster with the RH than did amateurs. Regarding dynamic features, RH exhibited both greater values and a smoother change along melodic intonations in experts that in amateurs. Further experiments using principal component analysis (PCA) and support vector machine (SVM) verified that hand-difference features can successfully differentiate experts from amateurs according to performance proficiency. Moreover, existing note-based raw feature values (Basic features) and DBH features were tested repeatedly via 10-fold cross-validation, suggesting that adding DBH features to Basic features improved F1 scores to 93.6% (by 3.5%) over Basic features. Our results suggest that differently controlling both hands simultaneously is an important skill for pianists; therefore, DBH features should be considered in the quantitative evaluation of piano performance.

Effects of Trunk Motion, Touch, and Articulation on Upper-Limb Velocities and on Joint Contribution to Endpoint Velocities During the Production of Loud Piano Tones

Piano performance involves several levels of motor abundancy. Identification of kinematic strategies that enhance performance and reduce risks of practice-related musculoskeletal disorders (PRMD) represents an important research topic since more than half of professional pianists might suffer from PRMD during their career. Studies in biomechanics have highlighted the benefits of using proximal upper-limb joints to reduce the load on distal segments by effectively creating velocity and force at the finger–key interaction. If scientific research has documented postural and expressive features of pianists’ trunk motion, there is currently a lack of scientific evidence assessing the role of trunk motion in sound production and in injury prevention. We address this gap by integrating motion of the pelvis and thorax in the analysis of both upper-limb linear velocities and joint angular contribution to endpoint velocities. Specifically, this study aims to assess kinematic features of different types of touch and articulation and the impact of trunk motion on these features. Twelve pianists performed repetitive loud and slow-paced keystrokes. They were asked to vary (i) body implication (use of trunk and upper-limb motion or use of only upper-limb motion), (ii) touch (struck touch, initiating the attack with the fingertip at a certain distance from the key surface, or pressed touch, initiating the attack with the fingertip in contact with the key surface), and (iii) articulation (staccato, short finger–key contact time, or tenuto, sustained finger–key contact time). Data were collected using a 3D motion capture system and a sound recording device. Results show that body implication, touch, and articulation modified kinematic features of loud keystrokes, which exhibited not only downward but also important forward segmental velocities (particularly in pressed touch and staccato articulation). Pelvic anterior rotation had a prominent role in the production of loud tones as it effectively contributed to creating forward linear velocities at the upper limb. The reported findings have implications for the performance, teaching, and research domains since they provide evidence of how pianists’ trunk motion can actively contribute to the sound production and might not only be associated with either postural or expressive features.

Pianism: Performance Communication and the Playing Technique

A pianist’s movements are fundamental to music-making by producing the musical sounds and the expressive movements of the trunk and arms which communicate the music’s structural and emotional information making it valuable for this review to examine upper-body movement in the performance process in combination with the factors important in skill acquisition. The underpinning playing technique must be efficient with economic muscle use by using body segments according to their design and movement potential with the arm segments mechanically linked to produce coordinated and fluent movement. Two physiologically and pianistically important actions proposed by early music scientists to deliver the keystroke involve dropping the hand from the shoulders toward the keys via a wave action with the joints activated sequentially, and forearm rotation to position the fingers for the keystroke, an action followed by the elbow/upper-arm rotating in the opposite direction. Both actions spare the forearm muscles by generating the energy needed in the larger shoulder muscles. The hand in the playing position has a curved palm through action of the metacarpal (knuckle) joints and curved fingers. Palm/finger posture controls sound quality from loud, high tempo sounds to a more mellow legato articulation, and to perform effectively the forearms should slope down toward the keyboard. The technique must be automatic through systematic practice which develops the motor skills for proficient playing, with practice duration tempered to reduce the risk of causing injury through overuse of the forearm muscles. Efficient movement patterns and strategic muscle relaxation which results in faster movement are realized only through extensive training. The constant movements of the head and trunk, and flowing arm movement with frequent hand lifts and rotational elbow movements, although generated in producing the playing technique, resonate with audience members who perceive them as expressive and thereby creating in them an empathic engagement with the music. It was proposed that music students be trained in the mechanical aspects of upper-body use in the playing technique, and practice strategies, with specialist pedagogy for children to develop motor skills for efficient playing, and training methods fostering an appreciation of the communicative aspects of music performance.

Relations between affective music and speech: evidence from dynamics of affective piano performance and speech production

This study compares affective piano performance with speech production from the perspective of dynamics: unlike previous research, this study uses finger force and articulatory effort as indexes reflecting the dynamics of affective piano performance and speech production respectively. Moreover, for the first time physical constraints such as piano fingerings and speech articulatory constraints are included due to their potential contribution to different patterns of dynamics. A piano performance experiment and speech production experiment were conducted in four emotions: anger, fear, happiness and sadness. The results show that in both piano performance and speech production, anger and happiness generally have high dynamics while sadness has the lowest dynamics. Fingerings interact with fear in the piano experiment and articulatory constraints interact with anger in the speech experiment, i.e., large physical constraints produce significantly higher dynamics than small physical constraints in piano performance under the condition of fear and in speech production under the condition of anger. Using production experiments, this study firstly supports previous perception studies on relations between affective music and speech. Moreover, this is the first study to show quantitative evidence for the importance of considering motor aspects such as dynamics in comparing music performance and speech production in which motor mechanisms play a crucial role.

Integrating optical finger motion tracking with surface touch events

This paper presents a method of integrating two contrasting sensor systems for studying human interaction with a mechanical system, using piano performance as the case study. Piano technique requires both precise small-scale motion of fingers on the key surfaces and planned large-scale movement of the hands and arms. Where studies of performance often focus on one of these scales in isolation, this paper investigates the relationship between them. Two sensor systems were installed on an acoustic grand piano: a monocular high-speed camera tracking the position of painted markers on the hands, and capacitive touch sensors attach to the key surfaces which measure the location of finger-key contacts. This paper highlights a method of fusing the data from these systems, including temporal and spatial alignment, segmentation into notes and automatic fingering annotation. Three case studies demonstrate the utility of the multi-sensor data: analysis of finger flexion or extension based on touch and camera marker location, timing analysis of finger-key contact preceding and following key presses, and characterization of individual finger movements in the transitions between successive key presses. Piano performance is the focus of this paper, but the sensor method could equally apply to other fine motor control scenarios, with applications to human-computer interaction.