Speech Expressiveness (Sandra Madureira – Integrated Acoustic Analysis and Cognition Laboratory- LIAAC- PUCSP)
The objective of this lecture is to consider
theoretical and experimental issues related to the research on prosody and
speech expressiveness and relevant concepts to the exploration of speech
expressiveness such as sound metaphor (Fonagy, 1983, 2000), sound symbolism
(Hinton et al 1994; Ohala, 1997), the
covariance and configuration models signaling vocal affect and the push and
pull effects (Scherer, 2003).
The fact that speech impresses and
not only informs (Bolinger, 1986) has to do with the potentiality of speech
sounds to express meanings and that of meaning effects being expressed by
speech sounds (Albano, 1988) and these correlations between sound and meaning
concern speech expressivity.
The direct links between sound and meaning
have been discussed from different theoretical frameworks for centuries. Plato,
in his dialogue "Kratylo" discusses the correspondences between names
and objects and explores issues related to the arbitrary and non-arbitrary
aspects of language, taking into account correlations between names and objects
in terms of sound symbolism
A large amount of work has also been
carried out on the correlations between speech acoustic characteristics and its
impressive effects on the listener, mainly on the way attitudes and emotions
are expressed by prosodic elements such as intonation patterns. Other prosodic
elements such as voice quality, in spite of being relevant, have not been
extensively researched. Although there has been an extensive search for the
acoustic correlates of attitudes and emotions from the seventies of the last
century up to recent years, our
current understanding of the processes involved in speech expressivity is
rather limited. One of the reasons for the state of affairs just pointed out,
being the need of experimental work on the role of sound symbolism in languages
(Chuenwattanapranithi et al, 2008).
Research on speech expressivity is
necessarily pluridisciplinary, involving linguistic, semiotic, psychological and
social aspects (Pittam, 1994). Investigating speech expressivity requires
taking into account a typology of speech styles, the nature of speakers' roles,
the symbolic associations between sound and physical characteristics, the
acoustic correlates of the impressive effects of speech characteristics on the
listener, the dynamics of speech production and meaning production.
To investigate speech
expressivity, a research methodology comprising text meaning analysis, prosodic
perceptual analysis, prosodic acoustic analysis and affective states evaluation
tests is proposed. For the perceptual evaluation of prosodic aspects (voice
quality settings and voice dynamics), the VPAS, the scheme developed by Laver
et al (2000) and adapted by (Camargo & Madureira, 2008) has been used. For
the acoustic analysis, based on PRAAT, manual and automatic measures (SG
detector and SG Expressive Evaluator developed by Barbosa to analyze speech
expressivity (Barbosa, 2009) have been taken into account. Statistically
measures concerning f0 in Hz, VV duration in ms, Long Term Average Spectra,
relative intensity and spectral tilt have been considered. For the perceptual
evaluation of the expressive uses of prosodic aspects, perceptual tests, such
as the semantic differential scale questionnaires have to be applied. The
perceptual identification tests are meant to evaluate emotion primitives (activation:
calm/activated; valence: pleasant/unpleasant; dominance; under control or out
of control), emotions (i.e., joy, sadness, anger and fear), attitudes (i.e.,
indignation, admonition, satisfaction, annoyance and nervousness) and speech
acts (i.e., advice, admonition, order and plea).
The results of research works which take into account
the application of the proposed research methodological procedures to the
analysis of poem reciting and character building in narratives are considered.
E. (1988) Fazendo sentido do som.
Ilha do Desterro 18, 11-26.
(1986) Intonation and its parts. Stanford
University Press, 1986.
Barbosa, P. A. (2009) Detecting changes in speech expressiveness in
participants of a
Radioprogram. In: Proceedings. of
Interspeech.v.1, 2155-2158. Brighton,
S.; Xu, Y.; Thipakorn, B. and Maneewonggvatana, S. (2008)
in speech with the size code: a perceptual investigation. Phonetica 65, 210-230.
Fonagy., I. (1983) La
Vive Voix: Essais de psycho-phonétique, Payot.
________ (2000) Languages within languages: an evolutive
approach. John Benjamins Publishing Company.
Nichols, J. and Ohala, J. J. (eds.). (1994) .Sound Symbolism. Cambridge:
Cambridge University Press.
Laver, J. (2000) Phonetic evaluation of voice quality. In: Kent, R. D.; BallL,. M. J. Voice quality measurement. San Diego,
Singular Publishing Group Inc.,37-48.
Madureira, S. (2008) Voice quality analysis from a phonetic perspective: Voice
Profile Analysis Scheme Profile for Brazilian Portuguese (BP-VPAS) In: Proceedings of the Fourth Conference on
Speech Prosody, Campinas,
Ohala, J. J.
symbolism. 4th Seoul International Conference on Linguistics [SICOL]v.1, 98-103. Seoul, Coreia.
Pittam, J. (1994) Voice
in social interaction; an interdisciplinary approach. London, SAGE Publications.
Scherer, K. (2003). Vocal communication of
emotion: a review of research paradigms.
Speech Communication, 40, 227-256.