Toward a dynamical theory of body movement in musical performance

Exploring the dynamics of intentional sensorimotor desynchronization using phasing performance in music

Humans tend to synchronize spontaneously to rhythmic stimuli or with other humans, but they can also desynchronize intentionally in certain situations. In this study, we investigate the dynamics of intentional sensorimotor desynchronization using phasing performance in music as an experimental paradigm. Phasing is a compositional technique in modern music that requires musicians to desynchronize from each other in a controlled manner. A previous case study found systematic nonlinear trajectories in the phasing performance between two expert musicians, which were explained by coordination dynamics arising from the interaction between the intrinsic tendency of synchronization and the intention of desynchronization. A recent exploratory study further examined the dynamics of phasing performance using a simplified task of phasing against a metronome. Here we present a further analysis and modeling of the data from the exploratory study, focusing on the various types of phasing behavior found in non-expert participants. Participants were instructed to perform one phasing lap, and individual trials were classified as successful (1 lap), unsuccessful (> 1 laps), or incomplete (0 lap) based on the number of laps made. It was found that successful phasing required a gradual increment of relative phase and that different types of failure (unsuccessful vs. incomplete) were prevalent at slow vs. fast metronome tempi. The results are explained from a dynamical systems perspective, and a dynamical model of phasing performance is proposed which captures the interaction of intrinsic dynamics and intentional control in an adaptive-frequency oscillator coupled to a periodic external stimulus. It is shown that the model can replicate the multiple types of phasing behavior as well as the effect of tempo observed in the human experiment. This study provides further evidence that phasing performance is governed by the nonlinear dynamics of rhythmic coordination. It also demonstrates that the musical technique of phasing provides a unique experimental paradigm for investigating human rhythmic behavior.

Interpersonal synchronization of spontaneously generated body movements

Interpersonal movement synchrony (IMS) is central to social behavior in several species. In humans, IMS is typically studied using structured tasks requiring participants to produce specific body movements. Instead, spontaneously generated (i.e., not instructed) movements have received less attention. To test whether spontaneous movements synchronize interpersonally, we recorded full-body kinematics from dyads of participants who were only asked to sit face-to-face and to look at each other. We manipulated interpersonal (i) visual contact and (ii) spatial proximity. We found that spontaneous movements synchronized across participants only when they could see each other and regardless of interpersonal spatial proximity. This synchronization emerged very rapidly and did not selectively entail homologous body parts (as in mimicry); rather, the synchrony generalized to nearly all possible combinations of body parts. Hence, spontaneous behavior alone can lead to IMS. More generally, our results highlight that IMS can be studied under natural and unconstrained conditions.

Interpersonal sensorimotor communication shapes intrapersonal coordination in a musical ensemble

Social behaviors rely on the coordination of multiple effectors within one's own body as well as between the interacting bodies. However, little is known about how coupling at the interpersonal level impacts coordination among body parts at the intrapersonal level, especially in ecological, complex, situations. Here, we perturbed interpersonal sensorimotor communication in violin players of an orchestra and investigated how this impacted musicians' intrapersonal movements coordination. More precisely, first section violinists were asked to turn their back to the conductor and to face the second section of violinists, who still faced the conductor. Motion capture of head and bow kinematics showed that altering the usual interpersonal coupling scheme increased intrapersonal coordination. Our perturbation also induced smaller yet more complex head movements, which spanned multiple, faster timescales that closely matched the metrical levels of the musical score. Importantly, perturbation differentially increased intrapersonal coordination across these timescales. We interpret this behavioral shift as a sensorimotor strategy that exploits periodical movements to effectively tune sensory processing in time and allows coping with the disruption in the interpersonal coupling scheme. As such, head movements, which are usually deemed to fulfill communicative functions, may possibly be adapted to help regulate own performance in time.

The dynamics of musical participation

In this paper we argue that our comprehension of musical participation-the complex network of interactive dynamics involved in collaborative musical experience-can benefit from an analysis inspired by the existing frameworks of dynamical systems theory and coordination dynamics. These approaches can offer novel theoretical tools to help music researchers describe a number of central aspects of joint musical experience in greater detail, such as prediction, adaptivity, social cohesion, reciprocity, and reward. While most musicians involved in collective forms of musicking already have some familiarity with these terms and their associated experiences, we currently lack an analytical vocabulary to approach them in a more targeted way. To fill this gap, we adopt insights from these frameworks to suggest that musical participation may be advantageously characterized as an open, non-equilibrium, dynamical system. In particular, we suggest that research informed by dynamical systems theory might stimulate new interdisciplinary scholarship at the crossroads of musicology, psychology, philosophy, and cognitive (neuro)science, pointing toward new understandings of the core features of musical participation.

The influence of a conductor and co-performer on auditory-motor synchronisation, temporal prediction, and ancillary entrainment in a musical drumming task

Interpersonal coordination is exemplified in ensemble musicians, who coordinate their actions deliberately in order to achieve temporal synchronisation in their performances. However, musicians also move parts of their bodies unintentionally or spontaneously, sometimes in ways that do not directly produce sound from their instruments. Musicians' movements-intentional or otherwise-provide visual signals to co-performers, which might facilitate temporal synchronisation. In large ensembles, a conductor also provides a visual cue, which has been shown to enhance synchronisation. In the present study, we tested how visual cues from a co-performer and a conductor affect processes of temporal anticipation, synchronisation, and ancillary movements in a sample of primarily non-musicians. We used a dyadic synchronisation drumming task, in which paired participants drummed to the beat of tempo-changing music. We manipulated visual access between partners and a virtual conductor. Results showed that the conductor improved synchronisation with the music, but synchrony with the music did not improve when partners could see each other. Temporal prediction was improved when partners saw the conductor, but not each other. Ancillary movements of the head were more synchronised between partners when they could see each other, and greater ancillary synchrony at beat-related frequencies of movement was associated with greater drumming synchrony. These results suggest that compatible audio-visual cues can improve intentional synchronisation, that ancillary movements are affected by seeing a partner, and that attended vs. incidental visual cues thus have partially dissociable effects on temporal coordination during joint action.

Origins of 1/f noise in human music performance from short-range autocorrelations related to rhythmic structures

1/f fluctuations have been described in numerous physical and biological processes. This noise structure describes an inverse relationship between the intensity and frequency of events in a time series (for example reflected in power spectra), and is believed to indicate long-range dependence, whereby events at one time point influence events many observations later. 1/f has been identified in rhythmic behaviors, such as music, and is typically attributed to long-range correlations. However short-range dependence in musical performance is a well-established finding and past research has suggested that 1/f can arise from multiple continuing short-range processes. We tested this possibility using simulations and time-series modeling, complemented by traditional analyses using power spectra and detrended fluctuation analysis (as often adopted more recently). Our results show that 1/f-type fluctuations in musical contexts may be explained by short-range models involving multiple time lags, and the temporal ranges in which rhythmic hierarchies are expressed are apt to create these fluctuations through such short-range autocorrelations. We also analyzed gait, heartbeat, and resting-state EEG data, demonstrating the coexistence of multiple short-range processes and 1/f fluctuation in a variety of phenomena. This suggests that 1/f fluctuation might not indicate long-range correlations, and points to its likely origins in musical rhythm and related structures.

Pianism: Performance Communication and the Playing Technique

A pianist’s movements are fundamental to music-making by producing the musical sounds and the expressive movements of the trunk and arms which communicate the music’s structural and emotional information making it valuable for this review to examine upper-body movement in the performance process in combination with the factors important in skill acquisition. The underpinning playing technique must be efficient with economic muscle use by using body segments according to their design and movement potential with the arm segments mechanically linked to produce coordinated and fluent movement. Two physiologically and pianistically important actions proposed by early music scientists to deliver the keystroke involve dropping the hand from the shoulders toward the keys via a wave action with the joints activated sequentially, and forearm rotation to position the fingers for the keystroke, an action followed by the elbow/upper-arm rotating in the opposite direction. Both actions spare the forearm muscles by generating the energy needed in the larger shoulder muscles. The hand in the playing position has a curved palm through action of the metacarpal (knuckle) joints and curved fingers. Palm/finger posture controls sound quality from loud, high tempo sounds to a more mellow legato articulation, and to perform effectively the forearms should slope down toward the keyboard. The technique must be automatic through systematic practice which develops the motor skills for proficient playing, with practice duration tempered to reduce the risk of causing injury through overuse of the forearm muscles. Efficient movement patterns and strategic muscle relaxation which results in faster movement are realized only through extensive training. The constant movements of the head and trunk, and flowing arm movement with frequent hand lifts and rotational elbow movements, although generated in producing the playing technique, resonate with audience members who perceive them as expressive and thereby creating in them an empathic engagement with the music. It was proposed that music students be trained in the mechanical aspects of upper-body use in the playing technique, and practice strategies, with specialist pedagogy for children to develop motor skills for efficient playing, and training methods fostering an appreciation of the communicative aspects of music performance.

Creating Time: Social Collaboration in Music Improvisation

Musical collaboration emerges from the complex interaction of environmental and informational constraints, including those of the instruments and the performance context. Music improvisation in particular is more like everyday interaction in that dynamics emerge spontaneously without a rehearsed score or script. We examined how the structure of the musical context affords and shapes interactions between improvising musicians. Six pairs of professional piano players improvised with two different backing tracks while we recorded both the music produced and the movements of their heads, left arms, and right arms. The backing tracks varied in rhythmic and harmonic information, from a chord progression to a continuous drone. Differences in movement coordination and playing behavior were evaluated using the mathematical tools of complex dynamical systems, with the aim of uncovering the multiscale dynamics that characterize musical collaboration. Collectively, the findings indicated that each backing track afforded the emergence of different patterns of coordination with respect to how the musicians played together, how they moved together, as well as their experience collaborating with each other. Additionally, listeners' experiences of the music when rating audio recordings of the improvised performances were related to the way the musicians coordinated both their playing behavior and their bodily movements. Accordingly, the study revealed how complex dynamical systems methods (namely recurrence analysis) can capture the turn-taking dynamics that characterized both the social exchange of the music improvisation and the sounds of collaboration more generally. The study also demonstrated how musical improvisation provides a way of understanding how social interaction emerges from the structure of the behavioral task context.

Sensorimotor Grounding of Musical Embodiment and the Role of Prediction: A Review

In a previous article, we reviewed empirical evidence demonstrating action-based effects on music perception to substantiate the musical embodiment thesis (Maes et al., 2014). Evidence was largely based on studies demonstrating that music perception automatically engages motor processes, or that body states/movements influence music perception. Here, we argue that more rigorous evidence is needed before any decisive conclusion in favor of a “radical” musical embodiment thesis can be posited. In the current article, we provide a focused review of recent research to collect further evidence for the “radical” embodiment thesis that music perception is a dynamic process firmly rooted in the natural disposition of sounds and the human auditory and motor system. Though, we emphasize that, on top of these natural dispositions, long-term processes operate, rooted in repeated sensorimotor experiences and leading to learning, prediction, and error minimization. This approach sheds new light on the development of musical repertoires, and may refine our understanding of action-based effects on music perception as discussed in our previous article (Maes et al., 2014). Additionally, we discuss two of our recent empirical studies demonstrating that music performance relies on similar principles of sensorimotor dynamics and predictive processing.

Improvisation and the self-organization of multiple musical bodies

Understanding everyday behavior relies heavily upon understanding our ability to improvise, how we are able to continuously anticipate and adapt in order to coordinate with our environment and others. Here we consider the ability of musicians to improvise, where they must spontaneously coordinate their actions with co-performers in order to produce novel musical expressions. Investigations of this behavior have traditionally focused on describing the organization of cognitive structures. The focus, here, however, is on the ability of the time-evolving patterns of inter-musician movement coordination as revealed by the mathematical tools of complex dynamical systems to provide a new understanding of what potentiates the novelty of spontaneous musical action. We demonstrate this approach through the application of cross wavelet spectral analysis, which isolates the strength and patterning of the behavioral coordination that occurs between improvising musicians across a range of nested time-scales. Revealing the sophistication of the previously unexplored dynamics of movement coordination between improvising musicians is an important step toward understanding how creative musical expressions emerge from the spontaneous coordination of multiple musical bodies.

Doing Duo - a case study of entrainment in William Forsythe's choreography "Duo"

Entrainment theory focuses on processes in which interacting (i.e., coupled) rhythmic systems stabilize, producing synchronization in the ideal sense, and forms of phase related rhythmic coordination in complex cases. In human action, entrainment involves spatiotemporal and social aspects, characterizing the meaningful activities of music, dance, and communication. How can the phenomenon of human entrainment be meaningfully studied in complex situations such as dance? We present an in-progress case study of entrainment in William Forsythe's choreography Duo, a duet in which coordinated rhythmic activity is achieved without an external musical beat and without touch-based interaction. Using concepts of entrainment from different disciplines as well as insight from Duo performer Riley Watts, we question definitions of entrainment in the context of dance. The functions of chorusing, turn-taking, complementary action, cues, and alignments are discussed and linked to supporting annotated video material. While Duo challenges the definition of entrainment in dance as coordinated response to an external musical or rhythmic signal, it supports the definition of entrainment as coordinated interplay of motion and sound production by active agents (i.e., dancers) in the field. Agreeing that human entrainment should be studied on multiple levels, we suggest that entrainment between the dancers in Duo is elastic in time and propose how to test this hypothesis empirically. We do not claim that our proposed model of elasticity is applicable to all forms of human entrainment nor to all examples of entrainment in dance. Rather, we suggest studying higher order phase correction (the stabilizing tendency of entrainment) as a potential aspect to be incorporated into other models.