Introduction

The ability to make judgments about the human face is integral to human social interaction. Through the study of face perception and its underlying neural correlates, multiple face-related processing indicators have been uncovered, which have, in turn, deepened our understanding of face processing (e.g., Poltoratski et al., 2021). One of the most important theoretical constructs for face perception is holistic processing—that is, the integrated processing of multiple facial parts (Farah et al., 1998; Hayward et al., 2013; Maurer et al., 2002; Rossion, 2013). Evidence for this claim comes from numerous studies demonstrating that processing of one facial part is often influenced by the other facial parts (Amishav & Kimchi, 2010; Hayward et al., 2016; Meinhardt-Injac et al., 2014; Richler et al., 2015; Tanaka & Sengco, 1997). However, theoretical clarity for the construct of “holistic processing” is lacking, as noted by Rossion (2008), who proposes a definition of holistic processing as “the simultaneous integration of the multiple features of a face into a single perceptual representation,” yet notes that most empirical evidence “essentially shows that faces features are interdependent” (p. 275). Perhaps not surprisingly, given this lack of theoretical clarity, evidence from studies purporting to investigate holistic processing have shown a variety of results. In this paper, we seek to propose a new framework for understanding the nature of interdependent processing of facial features, with the goal to gain a better understanding of what is meant by the construct of “holistic processing” in order to allow for the development of a more comprehensive and testable theory.

Holistic face processing is most commonly measured by two popular paradigms: the composite task and the part–whole task. In the composite taskFootnote 1 (Hole, 1994; Richler & Gauthier, 2014; Rossion, 2013; Young et al., 1987), participants are shown two composite faces (Fig. 1B), which are created by aligning the top and bottom facial halves from different individuals, and are instructed to determine if particular halves of the two faces are identical while ignoring the other halves. Results suggest, for example, that the same top half looks different when it is aligned with different bottom halves; however, this effect is reduced when the two halves are misaligned. The task stands in contrast to the part–whole task (Tanaka & Farah, 1993; Fig. 1A): participants first learn a study face and then are given two isolated features (e.g., eyes), one of which is in the original face, or two whole faces which are identical except for one feature. Participants are instructed to choose which of the two isolated features or two whole faces is the learned stimulus. Identification performance is typically better for whole faces than for parts, even though the additional information in the whole faces is identical (Tanaka & Farah, 1993; Tanaka & Simonyi, 2016). These paradigms provide important evidence that processing of one face part is influenced by the other parts, supporting the holistic face processing hypothesis.

Fig. 1
figure 1

Experimental designs of different tasks used to assess holistic face processing. A The part–whole (PW) task. After studying a face, test stimuli are either two versions of the study face varying in one feature only (top row) or two versions of a single feature (bottom row). B Example stimuli used in the standard composite face task (SCF) and complete composite face task (CCF). Both rows display two face composites, where the top facial halves are the same and the bottom halves are different. Composites may be aligned (top row) or misaligned (bottom row). C Schematic composite pairs in the SCF and CCF, showing Congruency when top halves are the target stimulus, and letters denote identity of the original faces. Left and right columns are trials where the target (i.e., top) halves are the same or different. The first and second rows are trials where the relationships between target and irrelevant (i.e., bottom) halves in study and test faces are congruent and incongruent. (1) The CCF consists of all the four trial types, where the composite effect is typically characterized by the interaction between Congruency (congruent vs. incongruent) and Alignment (aligned vs. misaligned). (2) Different from the CCF, the SCF only includes the same-incongruent and different-congruent trials, as shown in the two dashed line rectangles. The composite effect, by contrast, is indexed by the differences in “same(-incongruent)” trials between the aligned and misaligned conditions. (3) The trials in the PW task can be characterized as being equivalent to the same-congruent and different-incongruent trial types, as outlined with the solid rectangles (for more information, see the General Discussion)

Full size image

As noted above, these paradigms are widely referred to in the literature as measuring a construct that is called “holistic face processing,” but this term is so vague that the extent to which these tasks measure the same construct is difficult to determine. Several studies have failed to find significant correlations between the part–whole and the standard composite tasks (e.g., Boutet et al., 2021; Rezlescu et al., 2017; Wang et al., 2012); further, it appears that distinct neural mechanisms may underlie these tasks (Li et al., 2017, 2019). DeGutis and colleagues (DeGutis et al., 2013) observed significant correlations between the part–whole task and the complete composite task when regressions, but not subtractions, were employed to calculate the holistic effects.Footnote 2 This finding suggests that the correlation between the part–whole and complete composite effect is not reliably observed. Few studies have explored the relationships between the standard and complete composite tasks, probably because researchers usually only choose one of them within a single study. One key exception is Richler and Gauthier (2014), who performed a meta-analysis to explore their potential relationships and did not observe any significant correlations between the effect sizes in standard and complete composite tasks, even though the former were a subset of the latter. Overall, the relations among the effects measured by these paradigms are mixed but the evidence seems to suggest that they do not measure the same aspect of holistic face processing.

To make progress in this field, we examined the aspects of holistic processing that are measured by these different paradigms. Here, we re-visit them in terms of the nature of the interdependence between the target and irrelevant facial parts in each task. In the part–whole task, the “redundant” facial parts facilitate the judgment of the target feature (Tanaka & Simonyi, 2016). In the standard composite task, the bottom halves are always different between study and test faces and interfere with recognition of the aligned same top halves (Hole, 1994; Young et al., 1987). In the complete composite task, both the top and bottom facial halves may be identical or different on each trial. When participants make identity judgments about the target halves of two composites, the irrelevant halves facilitate or interfere with target performance based on whether the identity relationship between the irrelevant halves is identical to (congruent) or different from (incongruent) that of the target halves. In summary, different tasks that purport to measure holistic face processing can be differentiated by whether irrelevant parts lead to facilitation (as in the part–whole task and congruent trials of the complete composite task) or interference (as in the standard composite task and incongruent trials of the complete composite task).

It is not immediately clear whether these two behavioral phenomena—facilitation and interference—originate from the same aspect of holistic processing. If they result from the same aspect, we should observe similar results on both facilitation and interference from the same manipulations (i.e., the effects vary symmetrically; conditions leading to greater facilitation in one task also lead to greater interference). Alternatively, if facilitation and interference stem from different aspects of holistic processing, an asymmetry between the two effects should be observed. Asymmetries in these effects would help explain why different tasks showed poor correlations in previous studies. To investigate this issue, we inspected facilitation and interference in a single holistic face processing paradigm to exclude the influence of potential confounds when using multiple paradigms, such as task formats (e.g., two-alternative-force-choice in part–whole task vs. sequential matching in composite tasks) and response bias (Richler et al., 2011; Rossion, 2013; Rossion & Retter, 2015). Specifically, we investigated facilitation and interference observed in congruent and incongruent trials in the complete composite task. Experiment 1 examined both facilitation and interference effects, and their dependency on the location of the target halves (top vs. bottom). In Experiment 2, the probability of cueing particular target halves was manipulated (e.g., the top half was cued 75% of trials in one session compared to 25% in a second session) to examine the impact of strategic processes on facilitation and interference.

With the use of the complete composite task, we followed its main proponents, Richler and Gauthier (2014), and adopt their definition of holistic processing reflecting “obligatory encoding of all object parts because a strategy of attending to all parts cannot be ‘turned off’.” More specifically in the complete composite task, holistic processing is measured by “the failure of selective attention” (i.e., the extent to which participants could not ignore the influence of the irrelevant parts; Richler & Gauthier, 2014; see also Farah et al., 1998). If faces are processed holistically, we should observe greater influence of irrelevant parts on target parts for aligned faces—that is, the larger congruency effect (better performance for congruent compared to incongruent conditions) for aligned relative to misaligned faces (i.e., the interaction between Congruency and Alignment; see more below). Moreover, regarding our specific observation of interest, if there is facilitation, we should observe better performance for aligned congruent faces compared to misaligned congruent faces. If there is interference, we should observe worse performance for aligned incongruent faces relative to misaligned incongruent faces. Any perceptual or cognitive effect that stems from nonholistic processing or is not specific to aligned faces should not affect these observations of facilitation or interference since such effects would be expected to influence aligned and misaligned stimuli identically.

Experiment 1

Methods

Participants

We employed G*Power (Faul et al., 2009) to plan the sample size. It suggests that at least 31 participants would suffice for the statistical power of 95% with the alpha of 0.05 and the partial eta square of 0.32, which was the average effect size for the composite effect estimated by Richler and Gauthier (2014).

Thirty-two Chinese students (15 females and 17 males, mean age = 24.18 years) from the University of Hong Kong participated and were compensated with one course credit or 60 HKD. Participants gave written informed consent prior to the experiment and reported that they were right-handed and had normal or corrected-to-normal vision. The study protocol was conducted in accordance with the Declaration of Helsinki and approved by the local ethics committee.

Stimuli

Photos of 40 Chinese (20 females and 20 males) faces with neutral expression were converted to greyscale and trimmed into an oval shape with external features (e.g., hair and ears) excluded. The luminance and contrast were controlled using the SHINE toolbox (Willenbockel et al., 2010). Each face was divided into one top and one bottom half from the middle of the face (Fig. 2). A three-pixel white line separated the two face halves; thus, the top and bottom facial halves were unambiguous to participants. To render composite faces more biologically plausible and retain the randomization of creating composites from facial parts, we divided the stimuli into ten sets, with each containing four different individuals of the same gender. The top and bottom facial halves in the same set were randomly combined to form composite faces. The study and test composites on each trial were from the same stimulus set. We used the complete design with all trial types of composites (Fig. 1C), including misaligned conditions. The study and test faces were presented at the center of the screen with a homogenous gray background. The top and bottom halves of the study faces were always aligned. The test faces in half of the trials were aligned and in the other half of the trials were misaligned, where the irrelevant facial half shifted half of the face width to the right. Aligned and misaligned composite faces subtended 5.59° × 7.24° and 8.39° × 7.24° of visual angle, respectively.

Fig. 2
figure 2

Trial sequence of the composite task in Experiments 1 and 2. Example female and male composites are shown in the top and bottom panels, respectively. On each trial, participants judged if the target halves (cued with white bracket on the test faces) were identical between the study and test faces

Full size image

Apparatus and procedure

Participants sat approximately 50 cm in front of a 17-inch flat screen with a refresh rate of 60 Hz. The task was administrated with Psychtoolbox (Kleiner et al., 2007). On each trial (Fig. 2), a fixation cross appeared at the center of the screen for 500 ms, followed by a blank for 200 ms, a study face for 200 ms, a mask for 500 ms, then a test face and a white bracket were presented simultaneously for 200 ms. The white bracket was presented randomly either above or below the face, indicating either the top or bottom half of the face was the target. Participants were instructed to judge whether the cued target parts were identical between the study and test faces by pressing one of two keys, with the response map** counterbalanced across participants. Both accuracy and speed were emphasized. The next trial started 1,000–1,200 ms after a key press. The experiment lasted around 50 min.

This experiment involved four within-subject factors: Target (top vs. bottom), Congruency (congruent vs. incongruent), Alignment (aligned vs. misaligned), and Correct Response (same vs. different). There was a total of 640 trials, with 40 trials in each condition. Before the actual experiment, participants performed 32 trials with line-drawing stimuli with the same proportion of trials in each condition.

Open science statements

The analysis codes and datasets generated and/or analyzed for this study are available online (https://osf.io/yhm3s/). The experiments were not preregistered.

Data analysis

All analyses were conducted in R (Version 4.0.5) and RStudio (Version 1.4) on a local computer, except for model fitting, which was carried out in R (Version 3.6) on the High-Performance Computing platform at New York University Abu Dhabi. We tidied up the data with the “tidyverse” package (Wickham et al., 2019). All trials were included in the analyses except when no response was recorded (one trial each from three participants). To avoid the limitations of analyses of variance (ANOVAs; Aarts et al., 2014; Boisgontier & Cheval, 2016; Kristensen & Hansen, 2004; Quené & van den Bergh, 2004), such as the additional sphericity assumption, we employed generalized linear mixed-effects models (GLMM) to analyze behavior choices and correct response times (RT)Footnote 3 with “lme4” package (Bates, Mächler et al., 2015b) where successive difference contrast coding was applied. Follow-up comparisons were performed using the “emmeans” package (Lenth, 2023) with “asymptotic” methods estimating statistical results.

Although GLMM with the maximal random-effects structure (i.e., the maximal model) is preferred for confirmatory hypothesis testing (Barr et al., 2013), such models usually cannot converge. Thus, we built the random effects in GLMM by following Bates, Kliegl, and colleagues (Bates, Kliegl et al., 2021; Rezlescu et al., 2017; Wang et al., 2012) are unsurprising. One limitation here was that we tested facilitation and interference in only one paradigm to minimize the influence of potential confounds, but did not test them in other paradigms. Therefore, we must be cautious in generalizing the facilitation and interference in the complete composite task to the effects observed with the standard composite task and the part–whole task directly. Future studies can discern facilitation and interference tested in the part–whole and other paradigms, and may inspect relationships among the different components via individual differences. The understanding of facilitation and interference may also benefit from examining the contributions of other cognitive or decision-making components underlying face processing. These efforts have the potential to disentangle mixed findings from different popular paradigms measuring holistic processing in one unified framework. All in all, we characterize facilitation and interference as reflecting two asymmetric effects of holistic face processing; a clear account of these different effects will be necessary to explain the nature of holistic face processing.