Playing with Affect: Music Performance with awareness of Score and Audience

Abstract

An exquisite music performance can move an audience to tears of joy, or those of sorrow. The job of a good performer then is to convey not just the music's notated structure, but also its emotional metadata. In time, the performer will also learn to respond musically to the state of a live audience; matching their emotional ebb and flow to maintain interest for the concert's entirety. This work will discuss an Affective Performance framework in which compositions are marked up with emotional intent, or narrative. This mark-up directs the emotive adaptation of the symbolic score's reproduction, enhancing the computer music's realism. Examining the cognitive model of emotions appraisal theory, we highlight some key evidence underlying the principle of music expectancy and its central role in maintaining a listener's musical interest. This knowledge will play a significant role in developing an adaptive music engine in which the traditionally static score is manipulated in real time. We will also show how new Affective Computing technologies for reading the emotional state of users can be employed to further adapt the performance to account for audience emotional state. We end with an illustration of the framework, with examples in computer gaming engines and enhanced live performances for a distributed audience.