Abstract
Social behaviors are guided in part by motivational and emotional responses to affective facial expressions. In daily life, facial expressions communicate varying degrees social reward signals (happiness), social threat signals (anger), or social reward-threat conflict signals (co-occurring happiness and anger). Thus, motivational and emotional responses must be sensitive to variations in social signal intensity to effectively guide social behavior. We recently developed a novel social approach-avoidance paradigm (SAAP), which uses morphed facial expressions to assess sensitivity to linear increases in social reward and/or social threat intensity. Prior to large-scale studies validating the test quality of the SAAP, however, it is necessary to first establish the psychometric properties and generalizability of these sensitivity metrics. In Study 1, we independently replicated SAAP task effects and demonstrated that motivational and emotional sensitivity measures exhibit strong psychometric properties and robust individual variability. In Study 2, we demonstrated that more complex social judgements (e.g., trustworthiness) are also sensitive to linear increases in social signal intensity, which differs across judgements. Although future research in larger samples will be needed to establish the test quality of the SAAP, these preliminary findings suggest that the SAAP exhibits adequate psychometric properties to justify this type of large-scale individual differences research.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11031-024-10076-z/MediaObjects/11031_2024_10076_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11031-024-10076-z/MediaObjects/11031_2024_10076_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11031-024-10076-z/MediaObjects/11031_2024_10076_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11031-024-10076-z/MediaObjects/11031_2024_10076_Fig4_HTML.png)
Notes
Overall, we observed that a linear slope term provided a strong fit to participant response patterns in the SAAP (Adjusted Linear R2 = 0.47), whereas including a non-linear slope term did not substantively increase the variance explained for participant response patterns (Adjusted Quadratic R2 = 0.50). Together, these results suggest that 1) Participant response patterns in the SAAP are predominantly linear in trajectory and 2). Responses to intermediate stimulus values (e.g., 50%Happy) do not substantively deviate from these largely linear response patterns.
References
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005). Deciphering the Enigmatic Face: The Importance of Facial Dynamics in Interpreting Subtle Facial Expressions. Psychological Science, 16(5), 403–410. https://doi.org/10.1111/j.0956-7976.2005.01548.x
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The “Reading the Mind in the Eyes” Test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. The Journal of Child Psychology and Psychiatry and Allied Disciplines, 42(2), 241–251.
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Beevers, C. G., Wells, T. T., Ellis, A. J., & Fischer, K. (2009). Identification of emotionally ambiguous interpersonal stimuli among dysphoric and nondysphoric individuals. Cognitive Therapy and Research, 33(3), 283–290. https://doi.org/10.1007/s10608-008-9198-6
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H., & White, J. S. (2009). Generalized linear mixed models: a practical guide for ecology and evolution. Trends in Ecology & Evolution, 24(3), 127–135. https://doi.org/10.1016/j.tree.2008.10.008
Calvo, M. G., Fernández-Martín, A., Recio, G., & Lundqvist, D. (2018). Human observers and automated assessment of dynamic emotional facial expressions: KDEF-dyn database validation. Frontiers in Psychology, 9, 397727.
Calvo, M. G., Gutiérrez-García, A., Fernández-Martín, A., & Nummenmaa, L. (2014). Recognition of facial expressions of emotion is related to their frequency in everyday life. Journal of Nonverbal Behavior, 38, 549–567.
Carroll, J. M., & Russell, J. A. (1997). Facial expressions in Hollywood’s protrayal of emotion. Journal of Personality and Social Psychology, 72(1), 164.
Chen, C., & Jack, R. E. (2017). 2017/10/01/). Discovering cultural differences (and similarities) in facial expressions of emotion. Current Opinion in Psychology, 17, 61–66. https://doi.org/10.1016/j.copsyc.2017.06.010
Cho, E., & Kim, S. (2015). Cronbach’s Coefficient Alpha: Well Known but Poorly Understood. Organizational Research Methods, 18(2), 207–230. https://doi.org/10.1177/1094428114555994
Cooper, S. R., Gonthier, C., Barch, D. M., & Braver, T. S. (2017). The role of psychometrics in individual differences research in cognition: A case study of the AX-CPT. Frontiers in psychology, 8, 290877
Cosker, D., Krumhuber, E., & Hilton, A. (2010, July). Perception of linear and nonlinear motion properties using a FACS validated 3D facial model. In Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization (pp. 101–108).
DeBruine, L. M. (2005). Trustworthy but not lust-worthy: context-specific effects of facial resemblance. Proceedings of the Biological Sciences, 272(1566), 919–922. https://doi.org/10.1098/rspb.2004.3003
Dingemanse, N. J., & Dochtermann, N. A. (2013). Quantifying individual variation in behaviour: mixed-effect modelling approaches [https://doi.org/10.1111/1365-2656.12013]. Journal of Animal Ecology, 82(1), 39–54. https://doi.org/10.1111/1365-2656.12013.
Dobs, K., Bülthoff, I., Breidt, M., Vuong, Q. C., Curio, C., & Schultz, J. (2014). Quantifying human sensitivity to spatio-temporal information in dynamic faces. Vision Research, 100, 78–87.
Du, S., & Martinez, A. M. (2015). Compound facial expressions of emotion: From basic research to clinical applications. Dialogues in Clinical Neuroscience, 17(4), 443–455.
Du, S., Tao, Y., & Martinez, A. M. (2014). Compound facial expressions of emotion. Proceedings of the National Academy of Sciences, 111(15), E1454–E1462.
Duchaine, B., & Nakayama, K. (2006). The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants. Neuropsychologia, 44(4), 576–585. https://doi.org/10.1016/j.neuropsychologia.2005.07.001
Evans, T. C., & Britton, J. C. (2018). Sep). Improving the psychometric properties of dot-probe attention measures using response-based computation. Journal of Behavior Therapy and Experimental Psychiatry, 60, 95–103. https://doi.org/10.1016/j.jbtep.2018.01.009
Evans, T. C., & Britton, J. C. (2020). Social avoidance behaviour modulates automatic avoidance actions to social reward-threat conflict. Cognition and Emotion, 34(8), 1711–1720. https://doi.org/10.1080/02699931.2020.1787353
Evans, T. C., Esterman, M., & Britton, J. C. (2022). Social avoidance behavior modulates motivational responses to social reward-threat conflict signals: A preliminary fMRI study. Cognitive, Affective, & Behavioral Neuroscience. https://doi.org/10.3758/s13415-022-01031-x
Evans, T. C., Walukevich, K. A., Seager, I., & Britton, J. C. (2018). A psychometric comparison of anxiety-relevant attention measures. Anxiety Stress Co**, 31(5), 539–554. https://doi.org/10.1080/10615806.2018.1489536
Frith, C. (2009). Role of facial expressions in social interactions. Philos Trans R Soc Lond B Biol Sci, 364(1535), 3453–3458. https://doi.org/10.1098/rstb.2009.0142
Fröhner, J. H., Teckentrup, V., Smolka, M. N., & Kroemer, N. B. (2019). Jul 15). Addressing the reliability fallacy in fMRI: Similar group effects may arise from unreliable individual effects. NeuroImage, 195, 174–189. https://doi.org/10.1016/j.neuroimage.2019.03.053
Golle, J., Mast, F. W., & Lobmaier, J. S. (2014). Something to smile about: The interrelationship between attractiveness and emotional expression. Cognition and Emotion, 28(2), 298–310. https://doi.org/10.1080/02699931.2013.817383
Goodhew, S. C., & Edwards, M. (2019). 2019/03/01/). Translating experimental paradigms into individual-differences research: Contributions, challenges, and practical recommendations. Consciousness and Cognition, 69, 14–25. https://doi.org/10.1016/j.concog.2019.01.008
Goodwin, L. D., & Leech, N. L. (2006). Understanding Correlation: Factors That Affect the Size of r. The Journal of Experimental Education, 74(3), 249–266. https://doi.org/10.3200/JEXE.74.3.249-266
Gutiérrez-García, A., & Calvo, M. G. (2014). Social anxiety and interpretation of ambiguous smiles. Anxiety Stress Co**, 27(1), 74–89. https://doi.org/10.1080/10615806.2013.794941
Gutiérrez-García, A., & Calvo, M. G. (2016). Oct 30). Social anxiety and perception of (un)trustworthiness in smiling faces. Psychiatry Research, 244, 28–36. https://doi.org/10.1016/j.psychres.2016.07.004
Hayden, E. P. (2022). A call for renewed attention to construct validity and measurement in psychopathology research. Psychological Medicine, 52(14), 2930–2936. https://doi.org/10.1017/S0033291722003221
Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. https://doi.org/10.3758/s13428-017-0935-1
Infantolino, Z. P., Luking, K. R., Sauder, C. L., Curtin, J. J., & Hajcak, G. (2018). Jun). Robust is not necessarily reliable: From within-subjects fMRI contrasts to between-subjects comparisons. NeuroImage, 173, 146–152. https://doi.org/10.1016/j.neuroimage.2018.02.024
Jenkins, R., White, D., Van Montfort, X., & Mike Burton, A. (2011). Variability in photos of the same face. Cognition, 121(3), 313–323. https://doi.org/10.1016/j.cognition.2011.08.001
Kliegl, R., Wei, P., Dambacher, M., Yan, M., & Zhou, X. (2011). Experimental effects and individual differences in linear mixed models: Estimating the relationship between spatial, object, and attraction effects in visual attention. Frontiers in psychology, 1, 2124.
Knutson, B. (1996). Facial expressions of emotion influence interpersonal trait inferences. Journal of Nonverbal Behavior, 20(3), 165–182.
Korolkova, O. A. (2018). The role of temporal inversion in the perception of realistic and morphed dynamic transitions between facial expressions. Vision Research, 143, 42–51.
Krumhuber, E. G., Skora, L. I., Hill, H. C., & Lander, K. (2023). The role of facial movements in emotion recognition. Nature Reviews Psychology, 2(5), 283–296.
Kuckertz, J. M., Strege, M. V., & Amir, N. (2017). Intolerance for approach of ambiguity in social anxiety disorder. Cognition and Emotion, 31(4), 747–754. https://doi.org/10.1080/02699931.2016.1145105
LaPlante, D., & Ambady, N. (2000). Multiple messages: Facial recognition advantage for compound expressions. Journal of Nonverbal Behavior, 24, 211–224.
Lüdecke, D. (2018). ggeffects: Tidy data frames of marginal effects from regression models. Journal of Open Source Software, 3(26), 772.
Luke, S. G. (2017). Evaluating significance in linear mixed-effects models in R. Behavior Research Methods, 49(4), 1494–1502. https://doi.org/10.3758/s13428-016-0809-y
Magezi, D. A. (2015). Linear mixed-effects models for within-participant psychology experiments: An introductory tutorial and free, graphical user interface (LMMgui). Frontiers in Psychology, 6, 2. https://doi.org/10.3389/fpsyg.2015.00002
Marsh, A. A., Ambady, N., & Kleck, R. E. (2005). The effects of fear and anger facial expressions on approach- and avoidance-related behaviors. Emotion, 5(1), 119–124. https://doi.org/10.1037/1528-3542.5.1.119
Martinez, J. E., Funk, F., & Todorov, A. (2020). Quantifying idiosyncratic and shared contributions to judgment. Behavior Research Methods, 52(4), 1428–1444. https://doi.org/10.3758/s13428-019-01323-0
Matsumoto, D., & Hwang, H. C. (2014). Judgments of subtle facial expressions of emotion. Emotion, 14(2), 349.
McCrae, R. R., Kurtz, J. E., Yamagata, S., & Terracciano, A. (2011). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15(1), 28–50. https://doi.org/10.1177/1088868310366253
Miller, J., & Ulrich, R. (2013). Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes. Psychonomic Bulletin & Review, 20(5), 819–858. https://doi.org/10.3758/s13423-013-0404-5
Montepare, J. M., & Dobish, H. (2003). The contribution of emotion perceptions and their overgeneralizations to trait impressions. Journal of Nonverbal behavior, 27, 237–254.
Morrison, E. R., Morris, P. H., & Bard, K. A. (2013). The stability of facial attractiveness: Is it what you’ve got or what you do with it? Journal of Nonverbal Behavior, 37, 59–67.
Nikitin, J., & Freund, A. M. (2019). The motivational power of the happy face. Brain Sciences, 9(1), 6.
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Struhl, M. K., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, Robustness, and Reproducibility in Psychological Science. Annual Review of Psychology, 73(1), 719–748. https://doi.org/10.1146/annurev-psych-020821-114157
Nunnally, J. C. (1967). Psychometric Theory. New York: McGraw-Hill.
Parsons, S., Kruijt, A.-W., & Fox, E. (2019). Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements. Advances in Methods and Practices in Psychological Science, 2(4), 378–395. https://doi.org/10.1177/2515245919879695
Paulus, A., & Wentura, D. (2014). Threatening joy: Approach and avoidance reactions to emotions are influenced by the group membership of the expresser. Cognition and Emotion, 28(4), 656–677. https://doi.org/10.1080/02699931.2013.849659
Polit, D. F. (2014). Getting serious about test–retest reliability: a critique of retest research and some recommendations. Quality of Life Research, 23(6), 1713–1720. https://doi.org/10.1007/s11136-014-0632-9
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/
Radke, S., Kalt, T., Wagels, L., & Derntl, B. (2018). Implicit and explicit motivational tendencies to faces varying in trustworthiness and dominance in men. Frontiers in Behavioral Neuroscience, 12, 8.
Renard, S. B., de Jong, P. J., & Pijnenborg, G. H. M. (2017). Approach-avoidance of facial affect is moderated by the presence of an observer-irrelevant trigger. Motivation and Emotion, 41(2), 265–272. https://doi.org/10.1007/s11031-016-9595-1
Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
Rilling, J. K., & Sanfey, A. G. (2011). The neuroscience of social decision-making. Annual Review of Psychology, 62, 23–48. https://doi.org/10.1146/annurev.psych.121208.131647
Roelofs, K., Hagenaars, M. A., & Stins, J. (2010). Facing freeze: Social threat induces bodily freeze in humans. Psychological Science, 21(11), 1575–1581. https://doi.org/10.1177/0956797610384746
Rönkkö, M., & Cho, E. (2022). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6–14.
Ross, E. D., Gupta, S. S., Adnan, A. M., Holden, T. L., Havlicek, J., & Radhakrishnan, S. (2016). Neurophysiology of spontaneous facial expressions: I. Motor control of the upper and lower face is behaviorally independent in adults. Cortex, 76, 28–42.
Scherer, K. R., & Ceschi, G. (2000). Criteria for emotion recognition from verbal and nonverbal expression: Studying baggage loss in the airport. Personality and Social Psychology Bulletin, 26(3), 327–339.
Scherer, K. R., & Ellgring, H. (2007). Are facial expressions of emotion produced by categorical affect programs or dynamically driven by appraisal? Emotion, 7(1), 113.
Seidel, E.-M., Habel, U., Kirschner, M., Gur, R. C., & Derntl, B. (2010). The impact of facial emotional expressions on behavioral tendencies in women and men. Journal of Experimental Psychology: Human Perception and Performance, 36(2), 500–507. https://doi.org/10.1037/a0018169
Shah, P., Gaule, A., Sowden, S., Bird, G., & Cook, R. (2015). The 20-item prosopagnosia index (PI20): a self-report instrument for identifying developmental prosopagnosia. Royal Society Open Science, 2(6), 140343. https://doi.org/10.1098/rsos.140343
Staugaard, S. R. (2010). Threatening faces and social anxiety: a literature review. Clinical Psychology Review, 30(6), 669–690. https://doi.org/10.1016/j.cpr.2010.05.001
Stins, J. F., Roelofs, K., Villan, J., Kooijman, K., Hagenaars, M. A., & Beek, P. J. (2011). Walk to me when I smile, step back when I’m angry: emotional faces modulate whole-body approach–avoidance behaviors. Experimental Brain Research, 212(4), 603–611. https://doi.org/10.1007/s00221-011-2767-z
Strack, F., & Deutsch, R. (2004). Reflective and Impulsive Determinants of Social Behavior. Personality and Social Psychology Review, 8(3), 220–247. https://doi.org/10.1207/s15327957pspr0803_1
Tamir, D. I., & Hughes, B. L. (2018). Social rewards: From basic social building blocks to complex social behavior. Perspectives on Psychological Science, 13(6), 700–717.
Todorov, A. (2008). Evaluating faces on trustworthiness: An extension of systems for recognition of emotions signaling approach/avoidance behaviors. Annals of the New York Academy of Sciences, 1124(1), 208–224.
Todorov, A. (2011). Evaluating faces on social dimensions. Social neuroscience: Toward understanding the underpinnings of the social mind, 1, 54–76.
Todorov, A., Olivola, C. Y., Dotsch, R., & Mende-Siedlecki, P. (2015). Social attributions from faces: Determinants, consequences, accuracy, and functional significance. Annual Review of Psychology, 66, 519–545.
Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., Marcus, D. J., Westerlund, A., Casey, B. J., & Nelson, C. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168(3), 242–249.
Vrana, S. R., & Gross, D. (2004). Reactions to facial expressions: effects of social context and speech anxiety on responses to neutral, anger, and joy expressions. Biological Psychology, 66(1), 63–78. https://doi.org/10.1016/j.biopsycho.2003.07.004
Zebrowitz, L. (2018). Reading faces: Window to the soul?. Routledge.
Funding
This research was supported by a VA RR&D SPiRE grant (1I21RX003727-01A1) to ME, an R01 grant from the National Eye Institute to JD (R01EY032510-02), and an Office of Academic Affiliations fellowship to TCE.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Evans, T.C., Carlson, J., Zuberer, A. et al. A preliminary characterization of the psychometric properties and generalizability of a novel social approach-avoidance paradigm. Motiv Emot (2024). https://doi.org/10.1007/s11031-024-10076-z
Accepted:
Published:
DOI: https://doi.org/10.1007/s11031-024-10076-z