Introduction

Compared to numbers in tables, visualization of data allow us to harness the computational power of the visual system and extract relevant information at low effort in little time (e.g., Schnotz and Bannert 2003). Making high stakes decisions under time pressure is supported by data graphs at the stock exchange (Duclos 2015) or by decision trees when allocating participants to the intensive care unit (Green and Mehr 1997; Raab and Gigerenzer 2015, see Padilla et al. 2018, for an overview). Scheduling teams of caregivers in a manner that minimizes the potential for spreading a disease is one example of a task where the interrelatedness of elements (caregivers) from different categories (shifts/teams) is most relevant. While there is work on how to display such information in network graphs (e.g., Henry et al. 2007; Holten 2006), less is known about whether such formats allow to extract relevant information under time pressure.

Network graphs are used in many fields and can inform individuals about aspects of connectivity. A popular way to visualize a network are node-link diagrams. In social network analysis, a node usually represents an individual and a link their connection (McPherson et al. 2001). A cluster refers to a set of nodes with a shared characteristic (e.g., team, school class, sex) and is often indicated by the shape or color of a node. Therefore, node-link diagrams can also depict intergroup relations. If links between nodes of the same clusters are more likely than links between nodes of different clusters, it is referred to as homophily (McPherson et al. 2001). Homophily measures are relevant in many areas including econometrics (Golub and Jackson 2012), computer science (Cristani et al. 2016) and bibliometrics (Rubin and O’Connor 2018).

Homophily and its causes and effects have been intensively studied in social networks. McPherson et al. mentioned that the tendency of people to connect with similar others have been observed in many fields including ethnicity, sex, age, religion, education, social class, and behavior. Possible causes are seen in processes of selection (similarity already before forming a connection) and influence or adjustment (similarity after forming a link). Social homophily has been used in models for understanding how a critical mass can be formed for collective action, a condition with the potential to promote social change (Centola 2013). Homophily in node-link diagrams can be detected by comparing the number of links between nodes of the same cluster with the number of links between nodes of different clusters (Meulemans and Schulz 2015).

Meulemans and Schulz showed that observers were able to visually assess the degree of homophily in node-link diagrams with two clusters. Furthermore, they found that the layout algorithm used to draw the network had an influence on accuracy and response time of the participants (bipartite outperformed polarized). The bipartite layout (see Fig. 1, left side) has an extreme form of cluster separation and contains two opposed rows of nodes. Cross-cluster links are straight lines between the clusters and same-cluster links are curved and on the outsides. The polarized layout (Fig. 1, right side) has a moderate degree of cluster separation and contains only straight lines. Multiple authors (e.g., Kobourov et al. 2015) have discussed that perception of graphs like node-link diagrams rely on the Gestalt laws of perceptual organization. Two laws seem to particularly enable judgments of homophily: Nodes are perceived together when they are similar in shape or color (law of similarity) or when are close to each other (law of proximity, cf. Kubovy and van den Berg 2008). Interestingly, original work by Wertheimer (1923) already included examples with opposing rows of dots and circles that are very similar to nodes in node-link diagrams.

Fig. 1
figure 1

Example graphs. The left column shows graphs with a bipartite layout and the right column shows graphs with a polarized layout. The upper row shows graphs with a homophily level of 75% and the lower line shows graphs with a homophily level of 25%. Nodes with a square shape are blue and nodes with a circle shape are orange

Full size image

The current study extends the work of Meulemans and Schulz by varying time constraints and using a diverse sample. The majority of the participants in the study by Meulemans and Schulz were mathematicians and computer scientists. It is, therefore, possible that the conclusions of their study are restricted to people with specific characteristics, i.e., to those with a technical affinity. Expertise is a well-known moderator for design principles in learning environments (cf. Rey and Buchwald 2011). Hence, we do not know how well people in everyday life, laypeople, can detect homophily in node-link diagrams. Yet, given the social relevance of the homophily concept and the intuitive design of node-link diagrams, it would be a desirable aim. As noted above, for many practical purposes, it is relevant how accurately people can derive information from node-link diagrams within a short time. For example, using network graphs, decision makers in schools could obtain a quick overview of the connections existing between students of different classes. This can be relevant for the prevention of disease spreading.

The current study, therefore, tested the extent to which homophily perception depends upon the time granted to the observers: Is most of what can be gained from looking at the diagram gained within the first seconds? Or does extra time pay off, leading to substantially better judgments? Shortening the time observers invest also restricts the possible strategies they can use (e.g., estimation rather than counting (cf. Gaschler et al. 2015; Haider et al. 2005; Hansen et al. 2015). Rather than manipulating the time granted, Meulemans and Schulz used accuracy and response time as dependent variables. The average response time for assessing homophily was approximately 13 s per diagram. The resulting performance in accuracy judgment can be an outcome of different patterns within information processing. For example, it is possible that the first few seconds already account for the maximum performance an individual can achieve. In fact, several studies showed that quick, snap judgments can be accurate in a variety of fields. For instance, studies on perception of correlation in scatterplots indicate that people can make accurate judgments within few hundred milliseconds (Rensink 2014). Lee et al. (2006) mentioned that people sometimes use a quick overview strategy to perceive network graphs. Based on theories of automatic and controlled information processing (e.g., Schneider and Shiffrin 1977), it is possible that people immediately form a relatively accurate impression of a graph and then try to increase accuracy by using controlled processing. Some studies in the context of person perception (e.g., Rule et al. 2009) showed that additional time can even lead to lower accuracy in judgments.

On the other side, literature on the speed–accuracy–tradeoff (Pachella 1974) indicates that more time should clearly result in more accurate judgments. Similarly, Huang et al. (2009) pointed out that enabling the viewer to perform a task in a graph with both a short response time and a high accuracy is a desired claim but often unrealistic. Thus, perception of homophily in a graph could follow a more steady increase of accurate information processing (until a certain point is reached).

In order to get a deeper understanding of how people perceive homophily in network graphs, it could be useful to compare the performance in different time conditions. One possible approach that we applied in the present study is using time as an independent variable between subjects. In doing so, we varied the presentation time of the stimuli. Based on the difference in performance between different time conditions, we could assess to what extent longer presentation time comes with additional information extraction. Furthermore, the current study allows to estimate to what extent people without a specific technical background are able to visually assess the degree of homophily in node-link diagrams and whether the advantage of bipartite layouts over polarized layouts found in the expert sample by Meulemans and Schulz (2015) would replicate in a more heterogeneous sample.

Method

Participants

592 people participated in the experiment run as an online study. We deleted all cases that stated that they did not participate seriously, did not give informed consent, or did not finish the study completely. This led to 531 participants (279 women, 248 men, 1 diverse, 3 people did not report sex, age M = 40.71 years, SD = 12.38). 182 (34.27%) out of the 531 people were psychology students. Recruitment efforts included using diverse strategies such as e-mails to acquaintances, social media etc. We used one control question to check whether our sample was actually mainly laypeople. Participants were asked to state whether they were already familiar with node-link diagrams before the study on a four point Likert scale. 62% replied fully disagree, 21% rather disagree, 12% rather agree, and 4% fully agree. 1% did not respond. Random assignment to time granted led to 187 participants in the 5 s condition, 182 participants in the 10 s condition and 162 participants in the 15 s condition.

Material and procedure

The experiment was conducted via an online questionnaire with integrated node-links diagrams in the tool Unipark. The stimuli were taken from the study of Meulemans and Schulz (2015) and are available online: https://tutte.fernuni-hagen.de/web/userstudy/homophily/studyresults.html

In order to avoid excessive demands, we only included balanced (same-cluster size) bipartite and polarized graphs. Each person saw 40 different networks for either 5, 10, or 15 s. It was a mixed design with the within-subjects factors layout (2), size (4), homophily level (5) and a between subject factor presentation time (3): There were

  • two different layout algorithms: bipartite and polarized. As shown in Fig. 1, the layouts varied in degree of cluster separation and shape of links (linear and curvy vs only linear).

  • four different sizes: 20 nodes and 40 links, 20 nodes and 50 links, 28 nodes and 60 links, and 40 nodes and 70 links.

  • five different levels of homophily: 25%, 37.5%, 50%, 62.5%, 75%. A degree of 50% indicates an equal number of same- and cross-cluster links, 0% that only cross-cluster links exist, and 100% that only same-cluster links exist (For further details, please see Meulemans and Schulz)

  • three different time conditions: 5 s, 10 s, 15 s.

In order to construct a network, the authors developed a simple random generator with number of nodes for each cluster, number of links, and desired homophily as input. The desired homophily provided the number of cross-cluster links, and the remaining links were divided between the two clusters based on the relative cluster size. The actual links added to the network were taken randomly from all possible links (without replacement). Further technical details of the construction process of the layout are described by Meulemans and Schulz.

At the beginning of the study, participants were introduced to the concept of homophily in node-link graphs in a simple way. This was done by providing a short definition (tendency of people to connect to similar others), a common saying (the German equivalent of “birds of a feather flock together”) and example graphs (two graphs for both layout types). Figure 1 shows the examples. Participants were instructed to assess the degree of each graph on an analog scale between 0% (only cross-cluster links) and 100% (only same-cluster links). After the presentation time of each graph, the graph disappeared automatically and the scale appeared. Thus, participants could not rate before the end of the presentation time. The order of the 20 graphs for each layout type, as well as the order of both layout blocks was random.

Results

Figure 2 shows homophily ratings of the participants for the bipartite layout. In general, the ratings (bars) were relatively close to the true homophily values (black line). Furthermore, a small overestimation occurred for lower homophily degrees and a small underestimation for higher degrees of homophily. There was a tendency to approximate the true homophile value with more time in most degrees.

Fig. 2
figure 2

Homophily ratings of the bipartite layout for the different degrees of homophily and the different presentation times. Error bars represent standard errors of the mean

Full size image

Figure 3 shows homophily ratings of the participants for the polarized layout. In general, the ratings followed the true homophily value as well. However, compared to the bipartite layouts, there was a stronger overestimation for lower degrees of true homophily. There was a small tendency to approximate the true homophile value with more time as well. The patterns in Figs. 2 and 3 indicate that participants tended to underestimate the variance in homophily.

Fig. 3
figure 3

Homophily ratings of the polarized layout for the different degrees of homophily and the different presentation times. Error bars represent standard errors

Full size image

In order to build an index for accuracy of a person, we calculated the mean of the absolute difference between estimated and true homophily values. We will use the label deviations (from the actual homophily value) in the following sections. Furthermore, we inspected the correlations between estimated and actual homophily values of a person as a proxy for differentiation accuracy. This provides additional information since it is a possibility that a person can distinguish well between diagrams with lower and higher degrees of homophily (high value in correlation) without approximating the true homophily value (high value in deviations).

Deviations

A 2 × 3 ANOVA with the within-subject factor layout (polarized, bipartite) and between subject factor time (5 s, 10 s, 15 s) showed a significant main effect for layout, F(1, 528) = 169.80, p < 0.001, ηp2 = 0.24, and a significant main effect for time, F(2, 528) =  13.28, p < 0.001, ηp2 = 0.05. There was no interaction effect, F(2, 528) = 0.51, p = 0.60, ηp2 < 0.01. For the effect of time, the pairwise comparison (Bonferroni-corrected) was significant between 5 and 10 s (p = 0.003), and between 5 and 15 s (p < 0.001), but not between 10 and 15 s (p = 0.185). Figure 4 shows that in both layouts, the highest deviations of estimations from the true homophily levels resulted from the shortest presentation time. An analysis on the individual level revealed that 79% of the sample had higher deviations for the layout type polarized than for the layout type bipartite.

Fig. 4
figure 4

Influence of layout type and presentation time on deviations of the estimations from the true homophily value. Error bars represent 95% CI for the depicted mean values of deviations

Full size image

Figure 5 illustrates the distribution of deviations that led to the different mean values in Fig. 4. The distributions suggest that the mean differences between shorter and longer presentation times result from a change in mixture: The short presentation time seems to lead to some trials with estimates with large deviation. Presumably, reduced time does not lead to larger biases per se, but to a larger proportion of trials in which people respond without sufficiently taking the stimulus into account.

Fig. 5
figure 5

Relative distribution of deviations for layout bipartite (left) and layout polarized (right) for each time condition. There were no cases above a deviation of 40

Full size image

Correlations between estimated and actual homophily

For each person, we calculated the correlation between the estimated and actual homophily values across the twenty pictures for each layout type. A 2 × 3 ANOVA with the within-subject factor layout (polarized, bipartite) and between subject factor time (5 s, 10 s, 15 s) showed a significant main effect for layout, F(1, 528) = 141.17, p < 0.001, ηp2 = 0.21, and a significant main effect for time, F(2, 528) = 11.79, p < 0.001, ηp2 = 0.04. There was no interaction effect, F(2, 528) = 0.10, p = 0.91, ηp2 < 0.01. For the effect of time, the pairwise comparison (Bonferroni-corrected) was significant between 5 and 10 s (p = 0.002), and between 5 and 15 s (p < 0.001), but not between 10 and 15 s (p = 0.564). Figure 6 shows the results.

Fig. 6
figure 6

Influence of layout type and presentation time on correlations between estimations and the true homophily values. Error bars represent 95% CI

Full size image

Both layouts had the same pattern: The lowest correlations resulted for the shortest presentation time and the highest correlations for the longest presentation time.

An analysis on the individual level revealed that 77% of the sample had higher correlations for the layout type bipartite than for the layout type polarized.

It is possible that the percentage of people who were not able or willing to judge the degree of homophily and therefore produced random answers varied between the three time conditions. In order to check whether mean level group differences were driven by an influence of time granted on the proportion of people not differentiating between levels of homophily, we calculated the proportion of participants within each time condition that had “reasonable” correlation levels. To avoid determining an arbitrary cut-off value for the participants, we plotted the frequencies for a range of 0.10 to 0.90.

Figure 7 shows the proportion of participants within each time condition that had a correlation above different correlation coefficients.

Fig. 7
figure 7

Proportion of participants above the indicated correlation level on the x-axis for each time condition. Left for layout bipartite and right for layout polarized

Full size image

The proportion of participants with a correlation that indicates an active processing of the networks was high. For example, in the layout bipartite, 79.04% of the participants in the 5-s condition, 86.81% in the 10-s condition, and 90.12% in the 15-s condition had a value higher than 0.30, which indicates a medium effect size according to Cohen (1988). For the layout polarized, 60.96% of the participants in the 5-s condition, 78.02% in the 10-s condition, and 75.93% in the 15-s condition, had a value higher than 0.30. The analysis is inconsistent with the idea that higher time constraints led to a larger proportion not providing meaningful answers (e.g., giving up on trying to derive homophily from the stimuli). Substantial correlations between shown and rated homophily were present even with strong time constraints for the large majority of participants.

Further Analysis

The correlation between both performance measures (deviations and correlation) was high for each layout type: r = − 0.90, p < 0.01 for the bipartite graphs, and r =  − 0.79, p < 0.01 for polarized graphs. Thus, individuals who were able to give more accurate judgments were also the people who could discriminate different degrees of homophily better.

Discussion

In this study, we tested whether homophily could be estimated from node-link diagrams robustly even under time pressure and even by laypeople. Given that node-link diagrams can be relevant in high-stake decision-making contexts such as disease-spreading prevention involving laypeople, it is good news that these tests delivered positive results, both for the bipartite and the polarized layout. Across measures, the bipartite layout consistently leads to better estimates than the polarized layout. Our analyses suggest that most of the information is extracted from the node-link diagram within the first five seconds (see later on the discussion of time). There was no indication of motivational costs of time constraints. While time constraints led to far-off estimates on a proportion of trials, there was no indication that higher time constraints led to more people fully giving up on the task of homophily estimation (cf. Gaschler et al. 2015; Gneezy and Rustichini 2000; Haider and Frensch 1999; Haider et al. 2005). We observed a reduction of the correlation between displayed and estimated homophily rather than a breakdown in a subset of participants, whereas, for instance, Gneezy and Rustichini (2000) have documented that motivational effects can influence the proportion of participants giving up on the task. Taken together this suggests that node-link diagrams (and specifically the bipartite layout) can be used to convey much information in little time.

One goal of the present study was to investigate whether laypeople are able to visually assess the degree of homophily in node-link diagrams. We used a sample of 531 participants with mainly no prior knowledge on node-link diagrams. Past work (Meulemans and Schulz 2015) had focused on male mathematicians and computer scientists. We can conclude that, at least with a brief explanation of the concept and example stimuli, people are able to perceive network homophily. Given the wide spread of networks (cf. Holten et al. 2011), and in particular the concept of social homophily (McPherson et al. 2001), this is a useful insight. Our results imply that researchers on homophily could use node-link diagrams to communicate their results to the public.

The main goal of the study was to analyze how presentation time of the graphs affects the performance in homophily judgments. Longer presentation time was associated with higher accuracy = lower deviation of the ratings from the true homophily values. Similarly, longer presentation time was associated with higher correlations between the ratings and true homophily values. The direction of this tendency is in line with literature on speed accuracy tradeoff (e.g., Pachella 1974). However, the increase in performance with more time was rather low. For polarized layouts, the difference in deviations between the 5 and 15 s condition (17.57–15.53 = 2.05) represents an improvement of 13.2%. For bipartite layouts, the difference in deviations between the 5 and 15 s condition (14.23–11.46 = 2.77) represents an improvement of 24.2%. Given the tripling of the presentation time, these increases are relatively small. Presumably, automatic processes (e.g., Schneider and Shiffrin 1977) lead to the bulk of information extraction within little time. Leaving less room for more controlled strategies such as counting does not seem to strongly harm performance.

Future studies might test whether having participants start with unspeeded trials before transferring them to a condition with time constraints can further boost performance under time constraints. This has been documented for cases where people extract and (under time pressure) apply shortcut options from the structure of the material (Haider and Frensch 1999; Haider et al. 2005). Additionally, future studies could incorporate a higher variance in the degree of the true homophily. The theoretical range was from 25 to 75%. It is possible that our values for the correlation measures that we used as proxy for differentiation accuracy are underestimated due to a statistical restriction of range. However, our calculated values were still high. In practical application, discriminating can be more important than guessing an objective true value in a graph. For example, it could be the case that people just want to decide whether there is more homophily between two communities in one or the other city (comparing two node-link diagrams).

Concerning the layout of the networks, we could replicate the effect by Meulemans and Schulz (2015) that people are better in perceiving homophily in graphs with the bipartite layout than in graphs with the polarized layout. We think that the gestalt laws of similarity and proximity (Wertheimer 1923) favor accurate judgments in bipartite layouts. The consistent arrangement of nodes in the bipartite layout made it easier to compare same cluster with cross-cluster links. Additionally, shape of links for different clusters (linear vs curvy) was different only in the bipartite layout. Future studies could incorporate other methods like eye tracking to get a deeper understanding of the underlying mechanisms. A further issue for future work concerns the number of clusters in a network. Two clusters reflecting two social groups is a common scenario in many fields. For example, in social psychology, work on intergroup contact has often focused on such relations (e.g., Allport 1954). A further case reflects the connections between students of two different school classes as described in the Introduction. Other arrangements with two groups are work teams, work shifts, and sport teams. However, in some contexts, viewers of a network may be confronted with more than two clusters. Therefore, future work could focus on how individuals perceive structures of homophily in networks with more than two clusters.

A final suggestion for future research concerns how time constraints and type of layout can affect other relevant tasks in the perception of networks such as rating the degree of general social connectedness or node-specific judgments.

One practical consequence is that researchers and graphic designers should use the bipartite layout when they want to communicate the degree of (social) homophily in a network to both experts and laypeople.