Daniel Bedoya presented at the International Symposium of Performance Science (ISPS) 2021. He described how CosmoNote is being used to create a vocabulary and representations to capture the prosodic elements of music.
ISPS is a biennial meeting where performers and scientists share their research and have lively discussions circling around all aspects of performance science. Thematic sessions this year included topics such as: The arts and health, Capturing performance and The learning performer. ISPS2021 was originally planned to happen in Montréal but was instead held entirely online because of the changing health restrictions.
Watch the full presentation (available for a limited time) here.
From left-right top-bottom: Christine Guptill (chair), Gabriel Jones and Anders Friberg, Daniel Bedoya, Ariana Pedrosa
Towards a Set of Conventions for Representing and Annotating Musical Prosody
Performers introduce acoustic variations in their playing for expressive effect or to communicate musical structures, drawing on their technical abilities and experience. When such manipulations occur in speech, they are called prosody. The term musical prosody highlights the shared properties of expressivity in music and speech. Successful conventions like ToBI (Tones and Breaks Indices) have been created for annotating speech prosody. However, such a unified ontology of prosody does not yet exist for performed music. To fill this gap, we have started building a vocabulary and representations to capture the prosodic elements of music, with focus on the piano. The terminology draws inspiration from ToBI, with consideration for musicological concepts.
Our goal is to create a universal representation of musical expressivity by developing a set of conventions for representing and annotating prosodic elements in music. We begin by capturing the elements relating to segmentation – pace/rate, pause/breath to indicate phrases/subphrases, figural groupings – and to prominence – accent/stress, tipping points, tension. Each can be associated with changes in musical features such as tempo, loudness, and harmony. An objective is to develop a library of specimens to serve as exemplars of the prosodic categories. We embed the annotation conventions in a widely accessible web-based tool so that experts and non-experts can participate in the identification and annotation of the prosodic elements. Leveraging citizen scientists’ volunteer thinking, the conventions can be iteratively refined to become a standard in the music performance research community.
The main contributions are: 1) initiating a vocabulary for musical prosody with preliminary conventions for marking up prosodic elements in representations of performance; 2) introducing CosmoNote, our web-based citizen science platform for annotating musical prosody; and, 3) starting an audio-visual library of specimens of the proposed prosodic elements as training and tutorial material for the citizen science initiative. CosmoNote’s annotation tools include four structural boundary categories adapted from ToBI’s break indices, superimposable time regions, and customizable note groups. The visualization layers include piano roll and audio wave representations, tempo, loudness, and harmonic tension. The audio-visual library contains illustrative specimens of each archetypical music prosodic category. All examples have been annotated using CosmoNote and made available on the web.
Having a set of conventions and a vocabulary around musical prosody will promote understanding of performance to a broad audience. Citizen scientists contributing their annotations to CosmoNote will learn to recognize and question performance nuances, thus advancing knowledge about music expressivity and transforming the way people view and think about music performance. The instructional videos and annotated examples will provide additional scaffolding for abstracting principles about performance variations. The annotations collected will enable large-scale, ecological studies on variations in the perception of structures shaped in performance, and will form a crucial resource for the building of explanatory models of music expressivity.
Bedoya, D, L Fyfe, E Chew (2021). Towards a Set of Conventions for Representing and Annotating Musical Prosody. In Proceedings of the International Symposium on Performance Science, Oct 2021, Montreal/online.
