Measurement and time series analysis of emotion in music

Abstract

This thesis examines the relations among emotions and musical features and their changes with time, based on the assertion that there exist underlying, culturally specific, quantifiable rules which govern these relations. I designed, programmed and tested a computer controlled Two-Dimensional Emotion Space (2DES) which administered and controlled all aspects of the experimental work. The 2DES instrument consisted of two bipolar emotional response (ER) dimensions: valence (happiness-sadness) and arousal (activeness-sleepiness). The instrument had a test-retest reliability exceeding 0.83 (p > 0.01, N = 28) when words and pictures of facial expressions were used as the test stimuli. Construct validity was quantified (r < 0.84, p > 0.01). The 2DES was developed to collect continuous responses to recordings of four movements of music (N = 67) chosen to elicit responses in all quadrants of the 2DES: "Morning" from Peer Gynt, Adagio from Rodrigo’s Concierto de Aranjuez (Aranjuez), Dvorak’s Slavonic Dance Op 42, No. 1 and Pizzicato Polka by Strauss. Test-retest reliability was 0.74 (p > 0.001, N = 14). Five salient and objectively quantifiable features of the musical signal (MFs) were scaled and used for time series analysis of the stimuli: melodic pitch, tempo, loudness, frequency spectrum centroid (timbral sharpness) and texture (number of different instruments playing). A quantitative analysis consisted of: (1) first order differencing to remove trends, (2) determination of suitable, lagged MFs to keep as regressors via stepwise regression, and (3) regression of each ER onto selected MFs with first order autoregressive adjustment for serial correlation. Regression coefficients indicated that first order differenced (∆) loudness and ∆tempo had the largest correlations with ∆arousal across all pieces, and ∆melodic pitch correlated with ∆valence for Aranjuez (p > 0.01 for all coefficients). The models were able to explain up to 73% of mean response variance. Additional variation was explained qualitatively as being due to interruptions, interactions and collinearity: The minor key and dissonances in a tonal context moved valence toward the negative direction; Short duration and perfect cadences moved valence in the positive direction. The 2DES measure and serial correlation adjusted regression models were, together, shown to be powerful tools for understanding relations among musical features and emotional response.