Prosody and Speech Expressiveness (Sandra Madureira – Integrated Acoustic Analysis and Cognition Laboratory- LIAAC- PUCSP)
The objective of this lecture is to consider theoretical and experimental issues related to the research on prosody and speech expressiveness and relevant concepts to the exploration of speech expressiveness such as sound metaphor (Fonagy, 1983, 2000), sound symbolism (Hinton et al 1994; Ohala, 1997), the covariance and configuration models signaling vocal affect and the push and pull effects (Scherer, 2003).
The fact that speech impresses and not only informs (Bolinger, 1986) has to do with the potentiality of speech sounds to express meanings and that of meaning effects being expressed by speech sounds (Albano, 1988) and these correlations between sound and meaning concern speech expressivity.
The direct links between sound and meaning have been discussed from different theoretical frameworks for centuries. Plato, in his dialogue "Kratylo" discusses the correspondences between names and objects and explores issues related to the arbitrary and non-arbitrary aspects of language, taking into account correlations between names and objects in terms of sound symbolism
A large amount of work has also been carried out on the correlations between speech acoustic characteristics and its impressive effects on the listener, mainly on the way attitudes and emotions are expressed by prosodic elements such as intonation patterns. Other prosodic elements such as voice quality, in spite of being relevant, have not been extensively researched. Although there has been an extensive search for the acoustic correlates of attitudes and emotions from the seventies of the last century up to recent years, our current understanding of the processes involved in speech expressivity is rather limited. One of the reasons for the state of affairs just pointed out, being the need of experimental work on the role of sound symbolism in languages (Chuenwattanapranithi et al, 2008).
Research on speech expressivity is necessarily pluridisciplinary, involving linguistic, semiotic, psychological and social aspects (Pittam, 1994). Investigating speech expressivity requires taking into account a typology of speech styles, the nature of speakers' roles, the symbolic associations between sound and physical characteristics, the acoustic correlates of the impressive effects of speech characteristics on the listener, the dynamics of speech production and meaning production.
To investigate speech expressivity, a research methodology comprising text meaning analysis, prosodic perceptual analysis, prosodic acoustic analysis and affective states evaluation tests is proposed. For the perceptual evaluation of prosodic aspects (voice quality settings and voice dynamics), the VPAS, the scheme developed by Laver et al (2000) and adapted by (Camargo & Madureira, 2008) has been used. For the acoustic analysis, based on PRAAT, manual and automatic measures (SG detector and SG Expressive Evaluator developed by Barbosa to analyze speech expressivity (Barbosa, 2009) have been taken into account. Statistically measures concerning f0 in Hz, VV duration in ms, Long Term Average Spectra, relative intensity and spectral tilt have been considered. For the perceptual evaluation of the expressive uses of prosodic aspects, perceptual tests, such as the semantic differential scale questionnaires have to be applied. The perceptual identification tests are meant to evaluate emotion primitives (activation: calm/activated; valence: pleasant/unpleasant; dominance; under control or out of control), emotions (i.e., joy, sadness, anger and fear), attitudes (i.e., indignation, admonition, satisfaction, annoyance and nervousness) and speech acts (i.e., advice, admonition, order and plea).
The results of research works which take into account the application of the proposed research methodological procedures to the analysis of poem reciting and character building in narratives are considered.
Albano, E. (1988) Fazendo sentido do som. Ilha do Desterro 18, 11-26.
Bolinger, D. (1986) Intonation and its parts. Stanford University Press, 1986.
Barbosa, P. A. (2009) Detecting changes in speech expressiveness in participants of a
Radioprogram. In: Proceedings. of Interspeech.v.1, 2155-2158. Brighton, Reino Unido.
Chuenwattanapranithi, S.; Xu, Y.; Thipakorn, B. and Maneewonggvatana, S. (2008)
Encoding emotions in speech with the size code: a perceptual investigation. Phonetica 65, 210-230.
Fonagy., I. (1983) La Vive Voix: Essais de psycho-phonétique, Payot.
________ (2000) Languages within languages: an evolutive approach. John Benjamins Publishing Company.
Hinton, L. Nichols, J. and Ohala, J. J. (eds.). (1994) .Sound Symbolism. Cambridge:
Cambridge University Press.
Laver, J. (2000) Phonetic evaluation of voice quality. In: Kent, R. D.; BallL,. M. J. Voice quality measurement. San Diego, Singular Publishing Group Inc.,37-48.
Camargo, Z., Madureira, S. (2008) Voice quality analysis from a phonetic perspective: Voice Profile Analysis Scheme Profile for Brazilian Portuguese (BP-VPAS) In: Proceedings of the Fourth Conference on Speech Prosody, Campinas, v.1. p.57–60.
Ohala, J. J. (1997). Sound symbolism. 4th Seoul International Conference on Linguistics [SICOL]v.1, 98-103. Seoul, Coreia.
Pittam, J. (1994) Voice in social interaction; an interdisciplinary approach. London, SAGE Publications.
Scherer, K. (2003). Vocal communication of emotion: a review of research paradigms. Speech Communication, 40, 227-256.